sofastack / sofa-hessian Goto Github PK

Avoiding load hessian from com.caucho.hessian to cause classload conflict. For example: the dubbo changed hessian to hessian-lite

Could you please fix that?

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.caucho.hessian.io.HessianInput (jar:file:/.../com/caucho/hessian/main/hessian-4.0.63.jar!/) to field java.lang.Throwable.detailMessage
WARNING: Please consider reporting this to the maintainers of com.caucho.hessian.io.HessianInput
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Hessian 是一种二进制序列化协议。

由于 Hessian 自身实现关系，通过构造特定的序列化流，经过反序列化后可能会造成任意代码执行，存在安全风险，建议用户配置黑名单或白名单的方式来解决该问题。

本项目内置的黑名单来自内部实践和外部贡献，仅供参考，不做主动更新。如需要更严格的校验，请使用白名单功能。

Hessian is a binary serialization protocol.

Because of the implement of Hessian, by constructing a specific serialization stream, it may cause arbitrary code execution when doing deserialization. It is recommended that users configure blacklist or whitelist to solve the problem.

The blacklist built into the project comes from internal practices and external contributions, and is for reference only and is not actively updated. For more rigorous verification, please use the whitelist feature.

环境
jdk 8
sofa-rpc-all 5.5.0
sofa-hessian 3.3.8

错误提示：

这段代码报错

为3.3.8新加的NameBlackListFilter内LoggerSpaceManager.getLoggerBySpace("HessianSerializeLog", "com.alipay.sofa.middleware.config")。该怎么解决

There are lots of guys using hessian 4.x in their project, and they want to use it in sofa-rpc, and it will incompatible with sofa-hessian (base on hessian v3.1.3, and support generic invoke and security enhancement).

I will start v4.0.0 milestone, and make the sofa-hessian based on lasted stable version hessian-4.0.51, and also support generic invoke and security enhancement.

Describe the bug

All details is described here: apache/dubbo#2031

问题：sofastack/sofa-jarslink#121

原因：在 Hessian 4.x 版本中，ContextSerializerFactory 会尝试加载 META-INF/hessian/serializers 和 META-INF/hessian/deserializers 反序列化器，触发入口是:

1  public static ContextSerializerFactory create(ClassLoader loader)
2 {
3    synchronized (_contextRefMap) {
4     SoftReference<ContextSerializerFactory> factoryRef
5        = _contextRefMap.get(loader);
6
7     ContextSerializerFactory factory = null;
8
9      if (factoryRef != null)
10       factory = factoryRef.get();
11
12      if (factory == null) {
13       ContextSerializerFactory parent = null;
14
15       if (loader != null)
16          parent = create(loader.getParent());
17
18        factory = new ContextSerializerFactory(parent, loader);
19        factoryRef = new SoftReference<ContextSerializerFactory>(factory);
20
21        _contextRefMap.put(loader, factoryRef);
22      }
23
24     return factory;
25    }

针对每个 ClassLoader 会全局缓存一份 ContextSerializerFactory , 如果没有则会尝试使用当前 ClassLoader 以及其父 ClassLoader 依次创建。在 SOFAArk 环境中，因为子 ClassLoader 并不会委托给 父ClassLoader 加载，因此会导致一些 ClassCastException 报错。解决的方法是，在执行上述 create 方法时，只创建作为参数的传入的 ClassLoader 对应的 ContextSerializerFactory, 不考虑其父 ClassLoader，即删除 12~17 行。在 18 行构建 ContextSerializerFactory 传入的父ContextSerializerFactory 没有被使用，可以直接传入 null，理论上没有问题。

https://vuldb.com/?id.131250 ,这个问题有修复嘛，反序列化黑名单是否能再更新一波啊

I suggest you change your current package name.
In the current form of package organization, it is easy to cause class loading conflicts.
See dubbo and hessian-lite for example.

In what area(s)?

@OrezzerO

/area runtime
/area operator
/area placement
/area docs
/area test-and-release

Ask your question here

Excuse me, have you recently releases a version to fix the junit issue?

怎么增加自定义类型的序列化扩展？

like file order, so users can override by rewrite it.

在这个版本里，我们允许用户自己指定hessian黑名单，用于自身业务的一些特殊诉求。使用方法

1. 升级到hessian 3.3.6
2. 新建一个文件，放到项目的src/main/resources 目录下。可以有子目录。比如

security/blacklist.txt

3.指定启动参数
-Dserialize.blacklist.file=security/blacklist.txt 即可。hessian会从相对resources的跟路径中进行读取。

1.StackOverflowError, when object do not equals the replace
2. Serialize twice, when object equals the replace, one is WriteReplaceSerializer, and the other is JavaSerializer

如题，请问里面部分修复或者优化的问题在hessian4是否已经解决了，或者相比hessian4有什么好的特点吗，比方说使用hessian4的用户还有必要尝试你这个版本吗，谢谢。

Describe the bug

code in com/alipay/sofa/middleware/config/log/log4j/log-conf.xml

com.alibaba.common.logging.spi.log4j.DailyRollingFileAppender is internal class and will throw ClassNotFoundException when use sofa-rpc 5.8.x and log4j 1.x.

java.lang.ClassNotFoundException: com.alibaba.common.logging.spi.log4j.DailyRollingFileAppender
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)
at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:247)
at org.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:176)
at org.apache.log4j.xml.DOMConfigurator.findAppenderByReference(DOMConfigurator.java:191)
at org.apache.log4j.xml.DOMConfigurator.parseChildrenOfLoggerElement(DOMConfigurator.java:523)
at org.apache.log4j.xml.DOMConfigurator.parseCategory(DOMConfigurator.java:436)
at org.apache.log4j.xml.DOMConfigurator.parse(DOMConfigurator.java:1004)
at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:872)
at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:778)
at com.alipay.sofa.common.log.factory.LoggerSpaceFactory4Log4jBuilder.doBuild(LoggerSpaceFactory4Log4jBuilder.java:69)
at com.alipay.sofa.common.log.factory.AbstractLoggerSpaceFactoryBuilder.build(AbstractLoggerSpaceFactoryBuilder.java:67)
at com.alipay.sofa.common.log.MultiAppLoggerSpaceManager.createILoggerFactory(MultiAppLoggerSpaceManager.java:319)
at com.alipay.sofa.common.log.MultiAppLoggerSpaceManager.doInit(MultiAppLoggerSpaceManager.java:115)
at com.alipay.sofa.common.log.MultiAppLoggerSpaceManager.init(MultiAppLoggerSpaceManager.java:90)
at com.alipay.sofa.common.log.MultiAppLoggerSpaceManager.getILoggerFactoryBySpaceName(MultiAppLoggerSpaceManager.java:191)
at com.alipay.sofa.common.log.MultiAppLoggerSpaceManager.getLoggerBySpace(MultiAppLoggerSpaceManager.java:177)
at com.alipay.sofa.common.log.MultiAppLoggerSpaceManager.getLoggerBySpace(MultiAppLoggerSpaceManager.java:132)
at com.alipay.sofa.common.log.LoggerSpaceManager.getLoggerBySpace(LoggerSpaceManager.java:44)
at com.alipay.hessian.NameBlackListFilter.judgeLogger(NameBlackListFilter.java:68)

Expected behavior

A clear and concise description of what you expected to happen.

Screenshots

If applicable, add screenshots to help explain your problem.

Environment:

• Project Version [e.g. SOFA-RPC 1.0.0] : sofa-rpc 5.8.x sofa-hessian 3.3.13

Additional context

Add any other context about the problem here.

There is a new API from hessian-4.0.51 and improved in hessian-4.0.60.

public class ClassFactory
{
    private static ArrayList<Allow> _staticAllowList;

    private ClassLoader             _loader;
    private boolean                 _isWhitelist;

    private ArrayList<Allow>        _allowList;

We can change to native API for better compatibility.

BigDecimalDeserializer not in v3.3.3

https://github.com/mbechler/marshalsec/blob/master/marshalsec.pdf

from the paper, we can know there are some vulnerabilities when Unmarshall hessian

https://github.com/mbechler/marshalsec

we have provided some filters in

com.caucho.hessian.io.Hessian2Input#readObjectDefinition

com.alipay.hessian.internal.InternalNameBlackListFilter is a filter which contains classes which have vulnerabilities to hacking.

what we need to do is add more classes which have vulnerabilities to the list that provided from our security team.

won't open until fix. to inform me.

在泛化序列化的例子里面， 如果将 Hessian2Output 和 Hessian2Input 换成 HessianOutput 和 HessianInput，程序会执行失败：

Exception in thread "main" com.caucho.hessian.io.HessianProtocolException: unknown code:�
	at com.caucho.hessian.io.HessianInput.error(HessianInput.java:1717)
	at com.caucho.hessian.io.HessianInput.readObject(HessianInput.java:1203)
	at com.caucho.hessian.io.JavaDeserializer.readMap(JavaDeserializer.java:221)
	at com.caucho.hessian.io.JavaDeserializer.readMap(JavaDeserializer.java:161)
	at com.caucho.hessian.io.SerializerFactory.readMap(SerializerFactory.java:380)
	at com.caucho.hessian.io.HessianInput.readObject(HessianInput.java:1186)
       ... ... 

是否限制了只能使用 Hessian2 呢？

com.alipay.sofa:hessian 版本 = 3.3.7

https://github.com/alipay/sofa-hessian/issues/7 that sofa-hessian prevent deserialization by maintaining blacklists.
However, the blacklist does not add com.caucho.naming.QName and com.sun.org.apache.xpath.internal.objects.XString, and there are cases where it continues to be utilized. Take the following poc as an example:
https://mp.weixin.qq.com/s/vW6IgaA_Imc7-_Bac9XNQg

对于复杂类型，如：
{
String a,
B b,
}
当调用GenericUtils.convertToGenericObject(JSON.parseObject(""a":"123","b":{...}"))进行sofa rpc调用时报无法转换为B对象

just like LocalDateTime,Year etc