Python 3.x support about hazelcast-python-client HOT 12 CLOSED

Hi @waseem18 @asimarslan
I spend some time on the existing client code and here are my findings about this compatibility issue:

• Using six library would make writing compatible code easier, so we should use that.
• In Python 2, keys() and values() methods of dictionaries returns a copy of list of keys and values. In Python 3, these methods returns view objects. You can iterate over these view objects too but when a value or key in the dictionary is changed you would get a runtime error in Python3. So, to overcome this, all occurrences of these methods should be enclosed with list() to make a real copy in Python 3 too.
• Iteritems, iterkeys and itervalues methods of dictionaries are removed from Python 3 since items, keys and values methods now returns efficient view objects. We could use six.iteritems, six.iterkeys and six.itervalues instead of them.
• Syntax of exception reraising in Python 3 is changed. There is a method in six library called six.reraise() that corrects the difference between Python 2 and Python 3 reraise mechanism. We could change all occurrences of the raise exceptions in the form “raise exception, None, traceback” with six.reraise(None, exception, traceback).
• By default, / operator in Python 2 makes floor division when both of the operands are integers. In Python 3 same operand makes true division when both of the operands are integers. So, we should change all / operators in the existing code with // operator to make sure that Python 3 also does floor division. (All the divisions in the existing code are done between integers so we can safely make this change)
• In the hazelcast/connection.py, buffer() class is used. However, this is deprecated in Python 2.6 and removed in the early version Python 3. In Python 3 and Python2.6+ there is a class called memoryview() that does the same thing as buffer. If we are not going to support Python 2.5 and below we could safely change buffer with memoryview.
• In Python 2 bytes and string objects are represented in the same way as str class. This class have both encode and decode methods. However, in Python 3 these classes are separated and we have a bytes class that has decode method and str class that has encode method. This is the major problem for this compatibility issue because all over the code there are methods that sometimes called with bytes data and sometimes called with str data in Python 3. This causes the code to try to run encode method on bytes and decode method in str objects, try to append str to bytearrays .... and it results in an immediate error. (For instance you could check the behavior of write_utf() method in hazelcast/serialization/output.py to see this effect in Python 3). We could solve this with bunch of if statements but there would be a heavy burden on the performance of the client.
• There are also some trivial things we should do like changing all print statements with six.print_(), importing range function from six.moves module and importing renamed modules like Queue, cPickle… with their six equivalents.
• In Python 3 cmp parameter of the sorted() method is removed and replaced with key parameter that is somewhat different. However, there is an efficient method called cmp_to_key() that automatically changes cmp mechanism to key mechanism. We should use that.
• In hazelcast/util.py types.TypeType which is an alias for built-in type class is used. TypeType is removed from Python 3 since it is the same thing as type class. So, we should use built-in type class instead.
• In some of the files relative import is used. (from api import * of the hazelcast/serialization/base.py for instance). This causes some problems in Python 3. So in the import statements we should use the full path of the file.
• Basestring and long types are removed from the Python 3 and there are some codes that uses these keywords. We could change all basestring keywords with six.string_types. In the case of long, there are some conversion made to long and also some numbers with the trailing “l” so this requires some extra care.
• has_key() method of dictionaries is removed from the Python 3 so we could use “key in dict” instead of has_key in both of the Python versions.
• Python 2 assigns a default hash function which returns id(self)//16 as hash. Python 3 doesn’t do that. We should define the hash function ClassDefinition class of the hazelcast/serialization/portable/classdef.py because objects of this class is added to a set in somewhere and it requires a hash method.
• By default, dumps() method of the pickle module uses a different protocol than Python 2 version. We should use dumps method with protocol parameter set to 0 to have the same behavior in the both of the Python versions. Also note that this method returns a str object in Python 2 and bytes object in Python 3
• unichr() method is removed from Python 3 since all strings are unicode now. We should use six.unichr() to have unichr in Python 2 and chr in Python 3.

These are the things I found up to now. I think when we solve the bytes/str problem other changes could be done easily. Let me hear your thoughts about these problems and maybe we could work together to solve them.

from hazelcast-python-client.

@waseem18 Great community support for this, big thanks to you. Eager to see what you come up with.

from hazelcast-python-client.

Started working on it today! Will put a Work in progress PR when I'm done with some reasonable part. :)

from hazelcast-python-client.

Hi @vwiencek , Hazelcast Python Client only support 2.7 at the moment. We are planning to support python 3.x but not for the first release. I'll keep this issue open so that we can track python 3.x support development.

from hazelcast-python-client.

Ok thanks ... looking forward to it

from hazelcast-python-client.

Hi @vwiencek, Any more updates on this?

from hazelcast-python-client.

Bump:)

Any timeline on possible python3 support. I am just starting to investigate connecting my existing java hazecast client and first python clients together, don't really want to use python 2 if can help it.

from hazelcast-python-client.

Hi @asimarslan

I'm interested in working on this issue.

Do you want me to strictly port it to Python 3 or should it support both Python 2 & 3?

Thank You.

from hazelcast-python-client.

Hi @waseem18,

We cannot drop python 2 support as many users still use it on production so it should support both 2 & 3.

Please note that you should sign the contributor agreement before we can merge your PR

Thanks for your support.

from hazelcast-python-client.

Thanks for the reply @asimarslan . I'll start working and report the progress here 👍

from hazelcast-python-client.

Update on this issue : I've been working on making hazelcast-python-client compatible with both Python 2 and 3 and made the required changes.

I found that this python client uses hazelcast-remote-controller (hzrc which uses thrift 0.10.0) which got some non Python 2 & 3 compatible code. I'm looking into this part now.

I've got three straight holidays now, so will be fully working on this stuff and then will put a PR to both hzrc and this python client.

:)

from hazelcast-python-client.

any news?

from hazelcast-python-client.