Comments (7)
Thanks for pointing out. The extended ITRANS standard we defined does not probably have a mapping for this character. I will check this over the weekend.
from indic_nlp_library.
I wonder how this transliteration compares to open-tamil package.
Anoop would you be publishing this package on python pkg repository? Where are your unittests for this project, I can't seem to find it.
from indic_nlp_library.
The open-tamil package too has some problems handling the unicodes. You will have to explicitly type out in Tamil to get the best results.Discrepancy I faced is like so -
unicode("தொ","utf-8")
#OUTPUT : u'\u0ba4\u0bc6\u0bbe'
tamil_letter = utf8.get_letters("தொ")
utf_tamil = ''.join(tamil_letter).decode("utf-8")
#OUTPUT : u'\u0ba4\u0bca'
I have used open-tamil package.In both scenarios source of the letters were different i.e. different texts.
from indic_nlp_library.
@vrindaprabhu - please create a suitable issue and we can address it.
Also http://libindic.org/ has interesting code bits.
from indic_nlp_library.
@vrindaprabhu - I checked on Python3 and Open-Tamil version 0.51, I'm not seeing this issue you report. get_letters() returns just 1 letter as element of list.
from indic_nlp_library.
Strange. Probably like I mentioned it depends on how "தொ" is written. Even I did not face the issue all the time but only with few particular sentences in the corpus.
from indic_nlp_library.
@vrindaprabhu - there are unicode normalization issues and these are fixed in version 0.65.
from indic_nlp_library.
Related Issues (20)
- Text Normalisation HOT 4
- Script conversion of danda and double danda HOT 1
- Placement of Anuswara HOT 2
- AttributeError: 'NoneType' object has no attribute 'iloc' HOT 3
- sentence_split missing all_script_phonetic_data.csv
- installation of latest version not working correctly HOT 3
- Is translate function available? HOT 1
- Schwa deletion in romanization for Hindi
- Make a kaggle dataset to use this library in the inferece of a kaggle competetion HOT 1
- Inappropriate Hindi English Transliteration HOT 1
- Text Normalisation using Indic NLP library not working HOT 2
- ImportError: No module named indicnlp.common
- BrahmiNet is down HOT 2
- Unable to do transliteration using BrahmiNet REST API HOT 1
- AttributeError: 'NoneType' object has no attribute 'iloc' HOT 1
- Broken "Getting Started" links HOT 2
- Making indic-nlp-library available via Conda Forge HOT 4
- Bad sentence splitting performance on flores 200 hindi language HOT 1
- Add more NLP models list
- Sentence tokenizer creating issue while splitting for end of the sentence.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from indic_nlp_library.