PythonObjectSerializer raise 'utf-8' codec can't decode byte about hazelcast-python-client HOT 2 OPEN

Hi @MASARIwot

You can use HazelcastJsonValue to store JSON values in Hazelcast IMap. No need for a custom serializer for this. Check out this example: https://github.com/hazelcast/hazelcast-python-client/blob/master/examples/pandas/pandas_example.py.

If the fields of your object is known and stable, then it's better if you use Compact serialization which is our newest serialization mechanism.

I believe following docs can help:

https://hazelcast.com/blog/introduction-to-compact-serialization/

https://github.com/hazelcast/hazelcast-python-client/blob/master/examples/serialization/compact_serialization_example.py

from hazelcast-python-client.

The issue here is the default serialiser can't handle special characters e.g. ä (\u00e4) and raises PythonObjectSerializer. The workaround is to use json.dumps serialisation that does handle it, but this is merely a workaround, which is the same as examples (df.to_json()).

I get it, that using pickle.dumps means you can pickle any Python object (that may not be JSON serialisable), if that is the intention please add support for non English characters.

In general using pickle for serialisation is a bad idea. For example see data protocols (here) so python versions need to match, and there are no guarantees of backwards compatibility. Moreover, as pickle doc says at the top The pickle module is not secure. Only unpickle data you trust. clearly clients can can deserialise data stored by other clients in hazelcast, which makes it a path for malicious code execution.

from hazelcast-python-client.