Python client about redis HOT 16 CLOSED

Addes sets, and a small bugfix.

Ludo, sounds great! I'm not sure I understand the unicode stuff very well. The 
Redis
client should just handle binary stuff but I guess that Python supports unicode 
in a
native way, and the information that the string is unicode is stored inside the
object. If you send it to Redis you don't know anymore when you do "GETs" if the
original string was unicode or not. I guess there is no clear solution to this
problem, maybe to just abort? I wonder what memcached Python clients are doing.

Well, in most Python libraries (the ones I know at least), one of two things 
happen:

* you take care in your code of all conversions to/from unicode (which is a 
pain), or

* the library takes care of it for you, using a default encoding or one 
specified by
you at initialization time

If you plan on using redis from Django for example, encoding/decoding 
everything adds
unnecessary clutter to your code, when you could just encode/decode in a couple 
of
places in the redis client.

The problem arises from the fact that there's no datatype in redis, i.e. 
everything
is a string, so if the client tries to decode something to return it as 
unicode, he
might well break it (think of a pickled object).

What I did was the minimum sufficient to make the client work:

* if you pass an encoded string, all is well

* if you pass a unicode object, it gets encoded using Python's default encoding
(usually 'ascii'), and any encoding errors get trapped and reraised as one of 
the
client's exceptions

* nothing is done on strings coming from redis, they are returned to the user 
as they are

If there's someone with a better plan, modifying the code is trivial.

It's safe to assume that basically every string can be encoded in UTF-8. The 
correct behavior for any library 
should be to only work with unicode strings (in fact this is how Python 3.0 
works). This means that you library 
should actually enforce callers to pass only (and I mean if not raise an 
exception) unicode strings that then 
you can encode to UTF-8 and store to redis. Redis can't do anything with 
unicode because sockets only 
support byte strings and unicode is not a byte string but a memory 
representation (that in python happen to 
use UCS-2 or UCS-4).

So... My suggestion is, if you don't want to raise an exception when 
non-unicode is passed (although this is 
the only good idea to avoid an encoding mess when something changes outside of 
your library and you are 
unable to decode anything from redis), to encode unicode strings to UTF-8 and 
then store them.

Vale, it won't work: if you accept only Unicode, you have to return Unicode. 
But you
cannot, as you cannot be sure that a redis string is a string. And what if the 
string
to store in redis is a pickled or marshaled object, do we still encode it?

What I did is wrap the encoding exception we get if the client gets a unicode 
string
that cannot be decoded. It might be useful to add an optional charset arg to 
init, so
that encoding can be done with it instead of the ascii default.

[deleted comment]

Sorry for the off-topic, Redis SVN has some protocol differences, clients should
reflect this if possible, more info in the home page of the project.

Could you give me write access to the svn for the python client? If it's ok for 
you,
I'd prefer to develop against it as it will make things easier for everybody.

Ludo: sure! Just I need to find how to do it but should be hopefully simple, 
let me try.

I think I added you, please ACK as I'm not sure :)

It works perfectly, thanks!

I'll sync the client and hopefully update it with the latest svn changes by 
tonight.
Have a great weekend! :)

Thanks! A good we to you too. And I will try to don't break the protocol again 
in the
future ;)

On current SVN:

SyntaxError: Non-ASCII character '\xc3' in file redis.py on line 12, but no 
encoding
declared; see http://www.python.org/peps/pep-0263.html for details

You should add a coding-line on top of the file.

It's pretty strange, as on line 12 there's only the SVN LastChangedDate 
keyword...

Great!