Missing character when updating from pybo 0.4.0 to pybo 0.6.0, BoTokenizer to WordTokenizer about botok HOT 4 CLOSED

Thanks for the issue: I just saw that there are a few chars missing from my new implementation of the unicode table.
I will include them as soon as possible.

I also need to update the Readme and the documentation after the big refactoring that ended with the 0.6.1 release (yesterday). For example, you should not write anymore t.type != "non-bo", but t.chunk_type != "NON_WORD". Now, all the available variables are conveniently hard-coded in this file.

If you're ok with it, you could send me pybo-related code so I can update it with the new syntax, before our documentation is fully updated. If you don't feel posting it here, send it to me at [email protected]

from botok.

Please check that the missing char now parses as expected, with the 0.6.3 release.
Given the changes in my previous msg, you shouldn't have any problem anymore.

from botok.

solved:

drupchen@drupchen-Inspiron-5558:~$ pybo string "࿖ བཀྲ་ཤིས་བདེ་ལེགས།།"
Loading Trie... (2s.)
࿖_ བཀྲ་ཤིས་ བདེ་ལེགས །།

from botok.

Thank you!

from botok.