Translations are not proper when source contain the different format of numbers. about indictrans2 HOT 3 OPEN

The inference pipeline is designed to be broad-spectrum, handling texts from a wide array of domains. However, it is not foolproof.

This regex-placeholder method is applied post-hoc as we found it effective in most cases through empirical testing.

Note that the models weren't specifically trained to retain these placeholders and you can go ahead and fine-tune the models to do so.

Sentence splitting is performed using the best open-source libraries available.

The regex pattern was developed by analyzing encountered cases and covers most general-purpose use cases.

If you have any recommendations for other libraries or improved regex patterns, please let us know.

Additionally, you can choose to bypass the inference pipeline when sentence splitting is not necessary (if you are confident about the sequence length).

Below are the results when using the Fairseq model without the inference pipeline.

मौजूदा 1,87,500 रुपये की देनदारी से 20% की कमी।
कंपनी का राजस्व 2019-20 में 12,34,567 रुपये से बढ़कर 2020-21 में 56,78,910 रुपये हो गया।
जब सरकार में सबसे अधिक मूल वेतन केवल 30,000 रुपये प्रति माह था।

from indictrans2.

Thank you so much for your detailed response and for explaining the current approach and its limitations.

Could you kindly guide me on how these results were generated? Were they produced using the model.batch_translate() function?

Thank you once again for your support and assistance.

from indictrans2.

These are using joint_translate.sh, but batch_translate can also be modified do disable the regex based preprocessing and sentence splitting.

from indictrans2.