Giter VIP home page Giter VIP logo

python-i18n's Introduction

python-i18n Build Status Coverage Status Code Climate

This library provides i18n functionality for Python 3 out of the box. The usage is mostly based on Rails i18n library.

Installation

Just run

pip install python-i18n

If you want to use YAML to store your translations, use

pip install python-i18n[YAML]

Usage

Basic usage

The simplest, though not very useful usage would be

import i18n
i18n.add_translation('foo', 'bar')
i18n.t('foo') # bar

Using translation files

YAML and JSON formats are supported to store translations. With the default configuration, if you have the following foo.en.yml file

en:
  hi: Hello world !

in /path/to/translations folder, you simply need to add the folder to the translations path.

import i18n
i18n.load_path.append('/path/to/translations')
i18n.t('foo.hi') # Hello world !

Please note that YAML format is used as default file format if you have yaml module installed. If both yaml and json modules available and you want to use JSON to store translations, explicitly specify that: i18n.set('file_format', 'json')

Memoization

Setting the configuration value enable_memoization in the settings dir will load the files from disk the first time they are loaded and then store their content in memory. On the next use the file content will be provided from memory and not loaded from disk, preventing disk access. While this can be useful in some contexts, keep in mind there is no current way of issuing a command to the reloader to re-read the files from disk, so if you are updating your translation file without restarting the interpreter do not use this option.

Namespaces

File namespaces

In the above example, the translation key is foo.hi and not just hi. This is because the translation filename format is by default {namespace}.{locale}.{format}, so the {namespace} part of the file is used as translation.

To remove {namespace} from filename format please change the filename_format configuration.

i18n.set('filename_format', '{locale}.{format}')

Directory namespaces

If your files are in subfolders, the foldernames are also used as namespaces, so for example if your translation root path is /path/to/translations and you have the file /path/to/translations/my/app/name/foo.en.yml, the translation namespace for the file will be my.app.name and the file keys will therefore be accessible from my.app.name.foo.my_key.

Functionalities

Placeholder

You can of course use placeholders in your translations. With the default configuration, the placeholders are used by inserting %{placeholder_name} in the ntranslation string. Here is a sample usage.

i18n.add_translation('hi', 'Hello %{name} !')
i18n.t('hi', name='Bob') # Hello Bob !

Pluralization

Pluralization is based on Rail i18n module. By passing a count variable to your translation, it will be pluralized. The translation value should be a dictionnary with at least the keys one and many. You can add a zero or few key when needed, if it is not present many will be used instead. Here is a sample usage.

i18n.add_translation('mail_number', {
    'zero': 'You do not have any mail.',
    'one': 'You have a new mail.',
    'few': 'You only have %{count} mails.',
    'many': 'You have %{count} new mails.'
})
i18n.t('mail_number', count=0) # You do not have any mail.
i18n.t('mail_number', count=1) # You have a new mail.
i18n.t('mail_number', count=3) # You only have 3 new mails.
i18n.t('mail_number', count=12) # You have 12 new mails.

Fallback

You can set a fallback which will be used when the key is not found in the default locale.

i18n.set('locale', 'jp')
i18n.set('fallback', 'en')
i18n.add_translation('foo', 'bar', locale='en')
i18n.t('foo') # bar

Skip locale from root

Sometimes i18n structure file came from another project or not contains root element with locale eg. en name.

{
    "foo": "FooBar"
}

However we would like to use this i18n .json file in our Python sub-project or micro service as base file for translations. python-i18n has special configuration tha is skipping locale eg. en root data element from the file.

i18n.set('skip_locale_root_data', True)

python-i18n's People

Contributors

altendky avatar danhper avatar fierylynx73 avatar frewsxcv avatar girgitt avatar jayvdb avatar osallou avatar rafalkasa avatar snay avatar uovobw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

python-i18n's Issues

How to exclude files from search path?

I use

i18n.set('filename_format', 'translations.{format}')

to load translations.yml and that works fine.

However, I would like to exclude synonyms.yml from the load path because I use that file for a different purpose.

=> How to do so?

I tried to use negative look ahead regular expression

'^(?!synonyms)translations.{format}'

but that did not work.

Makes reference to another variable in the same translation file

Example:

File : lang.en.yml

  • en:
    • hi: "Hello world!!!"
    • common:
      • password: "Password"
      • username: "Username"
      • create: "Create"
      • add: "Add"
      • remove: "Remove"
      • delete: "Clear"
      • cancel: "Cancel"
      • save: "Save"
      • email: "E-mail"
      • search: "Search"
      • login: "Login"
      • register: "Register"
      • version: "Version"
      • logged: "Logged in as reference to common.username "

python-i18n do this? referencing logged to common.username.

Translation cannot start with %

Hi, if translation starts with '%' character,
translation fails on error:

Example - if translation is:

perc_of_clients: % of clients

then error is thrown:

I18nFileLoadError: Invalid YAML: while scanning for the next token found character '%', that cannot start any token

What can I do, to allow % character in translations?

YAML arrays not supported

Hello!
Either I didn't understand something, or it looks like that package doesn't support translations arrays.
I tried to put an array in my yaml file, which is correctly parsed and all, but it looks like trying to get it with i18n.t() raises a nice TypeError, when trying to format a string-which-is-actually-a-list.

Here's the YAML content I put :

fr:
  time:
    months:
      - janvier
      - février
      - mars
      - avril
      - mai
      - juin
      - juillet
      - août
      - septembre
      - octobre
      - novembre
      - décembre

And the error output:

Traceback (most recent call last):
  File "/Users/.../languages.py", line 32, in tr
    return i18n.t(key, locale=lang)
  File "/usr/local/lib/python3.8/site-packages/i18n/translator.py", line 28, in t
    return translate(key, locale=locale, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/i18n/translator.py", line 44, in translate
    return TranslationFormatter(translation).format(**kwargs)
  File "/usr/local/lib/python3.8/site-packages/i18n/translator.py", line 18, in format
    return self.safe_substitute(**kwargs)
  File "/usr/local/Cellar/[email protected]/3.8.6/Frameworks/Python.framework/Versions/3.8/lib/python3.8/string.py", line 147, in safe_substitute
    return self.pattern.sub(convert, self.template)
TypeError: expected string or bytes-like object

I guess detecting the variable type before trying to apply a substitution would be enough to fix it, but to be honest I didn't really look at the internal code, so I let you handle that part.
EDIT: Just created a PR for you, let me know if I did something wrong!

Python version: Python 3.8.6
Package version: 0.3.9

Translation tests failing

These tests are failing for me on Python 3.8 & 3.10. They used to pass iirc, so that only cause could be Python version or using latest PyYAML.

...
i18n/tests/translation_tests.py FFF..F.FFFF.....                                                                                                                                                                                      [100%]

================================================================================================================= FAILURES ==================================================================================================================
_______________________________________________________________________________________________ TestTranslationFormat.test_bad_pluralization ________________________________________________________________________________________________

self = <i18n.tests.translation_tests.TestTranslationFormat testMethod=test_bad_pluralization>

    def test_bad_pluralization(self):
        config.set('error_on_missing_plural', False)
        self.assertEqual(t('foo.normal_key', count=5), 'normal_value')
        config.set('error_on_missing_plural', True)
>       with self.assertRaises(KeyError):
E       AssertionError: KeyError not raised

i18n/tests/translation_tests.py:102: AssertionError
_______________________________________________________________________________________________ TestTranslationFormat.test_basic_placeholder ________________________________________________________________________________________________

self = <i18n.tests.translation_tests.TestTranslationFormat testMethod=test_basic_placeholder>

    def test_basic_placeholder(self):
>       self.assertEqual(t('foo.hi', name='Bob'), 'Hello Bob !')
E       AssertionError: 'foo.hi' != 'Hello Bob !'
E       - foo.hi
E       + Hello Bob !

i18n/tests/translation_tests.py:77: AssertionError
______________________________________________________________________________________________ TestTranslationFormat.test_basic_pluralization _______________________________________________________________________________________________

self = <i18n.tests.translation_tests.TestTranslationFormat testMethod=test_basic_pluralization>

    def test_basic_pluralization(self):
>       self.assertEqual(t('foo.basic_plural', count=0), '0 elems')
E       AssertionError: 'foo.basic_plural' != '0 elems'
E       - foo.basic_plural
E       + 0 elems

i18n/tests/translation_tests.py:88: AssertionError
____________________________________________________________________________________________________ TestTranslationFormat.test_fallback ____________________________________________________________________________________________________

self = <i18n.tests.translation_tests.TestTranslationFormat testMethod=test_fallback>

    def test_fallback(self):
        config.set('fallback', 'fr')
>       self.assertEqual(t('foo.hello', name='Bob'), 'Salut Bob !')
E       AssertionError: 'foo.hello' != 'Salut Bob !'
E       - foo.hello
E       + Salut Bob !

i18n/tests/translation_tests.py:70: AssertionError
_______________________________________________________________________________________________ TestTranslationFormat.test_full_pluralization _______________________________________________________________________________________________

self = <i18n.tests.translation_tests.TestTranslationFormat testMethod=test_full_pluralization>

    def test_full_pluralization(self):
>       self.assertEqual(t('foo.plural', count=0), 'no mail')
E       AssertionError: 'foo.plural' != 'no mail'
E       - foo.plural
E       + no mail

i18n/tests/translation_tests.py:93: AssertionError
_________________________________________________________________________________________________ TestTranslationFormat.test_locale_change __________________________________________________________________________________________________

self = <i18n.tests.translation_tests.TestTranslationFormat testMethod=test_locale_change>

    def test_locale_change(self):
        config.set('locale', 'fr')
>       self.assertEqual(t('foo.hello', name='Bob'), 'Salut Bob !')
E       AssertionError: 'foo.hello' != 'Salut Bob !'
E       - foo.hello
E       + Salut Bob !

i18n/tests/translation_tests.py:66: AssertionError
_______________________________________________________________________________________________ TestTranslationFormat.test_missing_placehoder _______________________________________________________________________________________________

self = <i18n.tests.translation_tests.TestTranslationFormat testMethod=test_missing_placehoder>

    def test_missing_placehoder(self):
>       self.assertEqual(t('foo.hi'), 'Hello %{name} !')
E       AssertionError: 'foo.hi' != 'Hello %{name} !'
E       - foo.hi
E       + Hello %{name} !

i18n/tests/translation_tests.py:80: AssertionError
___________________________________________________________________________________________ TestTranslationFormat.test_missing_placeholder_error ____________________________________________________________________________________________

self = <i18n.tests.translation_tests.TestTranslationFormat testMethod=test_missing_placeholder_error>

    def test_missing_placeholder_error(self):
        config.set('error_on_missing_placeholder', True)
>       with self.assertRaises(KeyError):
E       AssertionError: KeyError not raised

module 'i18n' has no attribute 'load_path'

Hi, I don't know if that library is still maintained, hope yes. I just downloaded using pip and tried to use examples from README.md and every time gives me AtributteError

  File "./run.py", line 50, in i18n_init
    i18n.load_path.append('locales')
AttributeError: module 'i18n' has no attribute 'load_path'
    i18n.add_translation('foo', 'bar')
AttributeError: module 'i18n' has no attribute 'add_translation'

Code I used:

def i18n_init() -> None:
    import i18n
    i18n.load_path.append('locales')

How to format yaml file for pluralized strings?

The docs doesn't reference how to save pluralized strings in the yaml file, just how to create them from code.

How do you add pluralized strings in the yaml file?
I tried this:

en:
  somevar:
    zero: 'zero'
    one: 'test'
    many: 'test %{count}'

but this doesn't work.

Pluralized strings seem broken

According to #8, you can set pluralized strings in yaml files easily, without having to add them with a function like in the doc.

So I tested with the following en.yml file:

en:
  hi: Hello world !
  test:
    zero: "No follower"
    one: "1 follower"
    other: "%{count} followers"

And then the following Python code:

import i18n

# configuring i18n [...]

print(i18n.t('hi'))
print(i18n.t('test', count=0))

The "hello world !" is correctly displayed, but the "No follower" isn't translated, thus only printing "test".
Having a print(i18n.t('test.zero')) however fixes the issue, but it's not really the expected way to do it lol.

Package version: 0.3.7
Python version: 3.8.5

Unable to load the translations

Hello,

I am unable to load the translation from a file

image

The code loading the file :

File translation.py :

import i18n

def i18n_loading(translationPath : str, locale : str) -> None:
    """
        Cette fonction charge les fichiers de traduction

        :param translationPath: Le dossier des fichiers de traduction
        :type translationPath: str
        :param locale: La locale a utiliser
        :type locale: str
    """    
    i18n.load_path.append(translationPath)
    i18n.set('locale', locale)
    i18n.set('fallback', 'en')

File fenetre_principale.py

class FenetrePrincipale(QMainWindow):

    def MINIMUM_SIZE() -> QSize:
        """
            Fonction permettant de simuler une variable statique pour la taille minimale

            :return: La valeur statique
            :rtype: int
        """
        return QSize(800, 800)

    def __init__(self, parent : QWidget = None, app_name : str = "Comptes", 
        x : int = 50, y : int = 50, width: int = 1000, height: int = 1000) -> None:
        """
            Constructor

            :param parent: Optional; Default : None; Le parent de cette fenetre
            :type parent: QWidget
            :param app_name: Optional; Default : "Comptes"; Le nom de l'application
            :type app_name: str
            :param x: Optional; Default : 50; Position en X du coin en haut a gauche de la fenetre sur l'ecran (en px)
            :type x: int
            :param y: Optional; Default : 50; Position en Y du coin en haut a gauche de la fenetre sur l'ecran (en px)
            :type y: int
            :param width: Optional; Default : 1000; Largeur de la fenetre (en px)
            :type width: int 
            :param height: Optional; Default : 1000; Hauteur de la fenetre (en px)
            :type height: int
        """
        super().__init__(parent)

        self.real_x = x
        self.real_y = y
        self.real_width = width
        self.real_height = height

        self.setMinimumSize(FenetrePrincipale.MINIMUM_SIZE())
        
        self.setWindowTitle(app_name)
        self.setGeometry(x, y, width, height)

        self.show()

    def resizeEvent(self, a0: QResizeEvent) -> None:
        """
            Surcharge de la fonction resizeEvent du parent pour pouvoir redimensionner nos elements et ajouter des controles sur la taille min et max

            :param a0: L'evenement de redimensionnement
            :type a0: QResizeEvent
            :override:
        """
        super().resizeEvent(a0)
        self.real_width = a0.size().width()
        self.real_height = a0.size().height()

File main.py :

from translation import i18n_loading
from fenetre_principale import FenetrePrincipale

# Si on est dans la boucle principale
if __name__=="__main__":

    # Chargement des traductions
    i18n_loading("./resources/translation", 'en')

    # Demarrage de l'application
    app=QApplication(sys.argv)

    # Creation de la fenetre principale
    ex=FenetrePrincipale(app_name=i18n.t("translate.app_name"))

    # Execution de l'application
    sys.exit(app.exec())

File ./resources/translation/translate.en.yml :

en:
  init: "Init..."
  app_name: "Accounts"
  accounts: "Accounts"
  budgets: "Budgets"
  delete: "Delete"
  menubar:
    file:
      name: "&File"
      open_file: "&Open File"
  budget:
    add: "Add"

There is a lot more files, especially for the tabs and their logic but I think this is enough to see the problem.

For information I am able to do it every time on one of my computers but I am unable to reproduce it on my other computer, yet the code is exactly the same (git versioned).

Versions on the working computer :

Version of python : 3.9.10
Version of python-i18n : 0.3.9

Versions on the not working computer :

Version of python : 3.9.10
Version of python-i18n : 0.3.9

Support loading translation files with importlib.resources files() API

Right now the library lets me pass in a string path to a directory containing translation files & internally the files are loaded using standard python file system (os.path) and file loading (open()) code.

I'd also like to be able to pass in a python style package and then the library would go away and use importlib.resources to load all the translation files in that package.

The main reason for my use case is that importlib.resources should work more reliably with various freezing programs such as PyOxidiser & PyInstaller in the future, which is useful for distributing games. Right now when including data files in my GUI library package (in this case the translations for default text on GUI elements) I have to fall back on using the __file__ hack to get a path without importlib.resources . There may be other additional benefits to importlib.resources as it is a fairly recent addition to the python standard library.

Desired user API:

i18n.load_package_path.append("data.translations")

Example of some potential importlib.resources usage internally:

from importlib.resources import files

def load_translation_file_from_package(file_name, package, locale=config.get('locale')):
    skip_locale_root_data = config.get('skip_locale_root_data')
    root_data = None if skip_locale_root_data else locale
    translations_dic = load_resource(package.joinpath(file_name), root_data)

def load_package(package_name, locale=config.get('locale')):
    package = files(package_name)
    for file in package.iterdir():
        if file.is_file() and file.name.endswith(config.get('file_format')):
            if '{locale}' in config.get('filename_format') and not locale in file.name:
                continue
            load_translation_file_from_package(file.name, package, locale)

I've skipped a bunch of the required code there, but from a swift glance over it - it should be possible to adapt what's here already to load using importlib.resources methods like .read_text() which creates a StringIO that can be parsed by the various .json and .yaml parsers.

There are also some shenanigans with configuration needed to support this as:

a) the new files() API to importlib.resources was only added in Python 3.9 replacing an older API.
b) the older API itself only existed in the standard library since Python 3.7
c) However, there is a backport called importlib_resources for pre 3.9 versions of Python - but that has the downside of introducing an additional dependency if you are leery of those (though at least this one is in the standard library past 3.7/3.9).

I'm hoping this is the last time the standard resource loading APIs for python are going to change.

Extra note in case you try testing this: data packages, just like regular python packages, need an __init__.py file in their directory. However since the 3.9 files() API you don't need an __init__.py file in parent directories of your data directory any more so you can do:

main-package
|___code_file.py
|___another_code_file.py
|___data
    |___translations
        |___ __init__.py
        |___ main-package.en.json
        |___ main-pacakge.fr.json
    

Here is also a helpful video on why importlib.resources exists if you want more information than I have provided here:
https://pyvideo.org/pycon-us-2018/get-your-resources-faster-with-importlibresources.html

Useful details on the config required here:
https://discuss.python.org/t/deprecating-importlib-resources-legacy-api/11386

Docs for importlib.resources from the backport:
https://importlib-resources.readthedocs.io/en/latest/index.html

Official documentation from Python 3.9:
https://docs.python.org/3.9/library/importlib.html#module-importlib.resources

P.S. I'm enjoying using the library, making light work of a task I've been wary of, so thanks for creating it!

I18nFileLoadError for locales ‘am’ and ‘ml’ due to Misidentification in load_directory

Body

Hello,

When the user's locale is set to 'am' or 'ml', the code in the load_directory function of resource_loader.py incorrectly assumes that a corresponding localization file exists. This leads to an i18n.loaders.loader.I18nFileLoadError. The error message displayed is:

I18nFileLoadError: error getting data from resources/translations/de.yaml: ml not defined

This issue arises due to improper file existence checking. The locales 'ml' and 'am' are mistakenly identified as existing because they are substrings of the ".yaml" file extension. This results in the erroneous assumption that the relevant localization files are present.

Suggested Fix

A more robust method of checking for the existence of localization files should be implemented to prevent substring matches within file extensions.

Strings are getting escaped

It appears strings are getting escaped. My unit tests are failing while switching out to strings generated by i18n.t(). Is there a way to prevent this? I'd like the terminal to handle the newline vs getting an escaped newline.

---
en:
  command:
    failed: |
      Idempotence test failed because of the following tasks:\n%{failed_tasks}
E         - ('Idempotence test failed because of the following tasks:\\n',)
E         ?                                                           -
E         + ('Idempotence test failed because of the following tasks:\n',)

doesn't work with AWS lambdas

my configuration


import i18n
from i18n import t

FILE_PATH = './translations/'

i18n.set('locale', 'ru')
i18n.set('fallback', 'en')
i18n.set('skip_locale_root_data', True)
i18n.load_path.append(FILE_PATH)

print('i18 configuration loaded')
print(i18n.load_path)


the output of the following stuff running in process

from os import listdir
from os.path import isfile, join
files = [f for f in listdir(FILE_PATH) if isfile(join(FILE_PATH, f))]

print(files))


['bot.en.yml', 'bot.ru.yml']


bot:
hi: Well, hello

When running within a test with pytest - it works just fine. In an AWS lambda it does't

Can't retrieve translation

Hi!

I'm trying to implement i18n in one of my python script. I tried to figure out how this lib works but I'm facing an issue. I didn't find how to retrieve the desired value in my yml file:

python script:

    i18n.load_path.append('template')
    i18n.set('file_format', 'yaml')
    i18n.set('locale', 'en')
    print(i18n.t('template.subject'))

yaml file (template.en.yml):

en:
  subject: Test %{filename}

But my script only prints "template.subject" and not the string.

What am I missing here ?

Thanks!

Tags

The latest tag at the moment is 0.3.0, while PyPI has 0.3.9.

How to get reverse translation?

Lets say I have a translation file containing a section

de_DE:
  hello: Hallo

How can I get the reverse translation Hallo => hello?

If there is now build-in function for reverse translation, how can I access the already read in translations to create a reverse dictionary from its entries?

placeholder_delimiter config is ignored

Hi @danhper and @ZRunner,

Thank you both (and other contributors) for this great piece of software.

I am facing an issue when attempting to change the placeholder delimiter (because I have data containing the default placeholder delimiter %)

Python 3.7.3 (default, Jan 22 2021, 20:04:44) 
[GCC 8.3.0] on linux
>>> import i18n
>>> i18n.set('error_on_missing_placeholder', True)
>>> i18n.set('placeholder_delimiter', '#')
>>> i18n.add_translation('hash', 'Hash: 10 # !')
>>> i18n.t('hash') # should raise the exception because it is using an invalid placeholder definition
'Hash: 10 # !'
>>> i18n.add_translation('perct', 'Percent: 10 % !')
>>> i18n.t('perct') # this one should not since I have changed the default delimiter
'Percent: 10 % !'
>>> i18n.t('perct')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/collectd-reporting/lib/python3.7/site-packages/i18n/translator.py", line 24, in t
    return translate(key, locale=locale, **kwargs)
  File "/opt/collectd-reporting/lib/python3.7/site-packages/i18n/translator.py", line 44, in translate
    return TranslationFormatter(translation).format(**kwargs)
  File "/opt/collectd-reporting/lib/python3.7/site-packages/i18n/translator.py", line 16, in format
    return self.substitute(**kwargs)
  File "/usr/lib/python3.7/string.py", line 132, in substitute
    return self.pattern.sub(convert, self.template)
  File "/usr/lib/python3.7/string.py", line 129, in convert
    self._invalid(mo)
  File "/usr/lib/python3.7/string.py", line 105, in _invalid
    (lineno, colno))
ValueError: Invalid placeholder in string: line 1, col 13

First I looked at the units tests, but none where testing that config feature 😅

So I looked at the source code, and see that it uses the Template feature of the string module.
And the documentation state :

Note further that you cannot change the delimiter after class creation (i.e. a different delimiter must be set in the subclass’s class namespace).

Which is confirmed by the use of __init_subclass__(cls) in the Template source code

To my comprehension, because your module defines the class itself, despite it contains the following code, it will always get the value of the default delimiter as the moment of its definition (never get the chance to call i18n.set() before definition) :

class TranslationFormatter(Template):
    delimiter = config.get('placeholder_delimiter')

So I said, OK, I am just going to extend it with another class setting up my own delimiter (like this Template example), but it is impossible since you use that class hardcoded.

In @danhper code for i18n/translator.py:

def translate(key, **kwargs):
    locale = kwargs.pop('locale', config.get('locale'))
    translation = translations.get(key, locale=locale)
    if 'count' in kwargs:
        translation = pluralize(key, translation, kwargs['count'])
    return TranslationFormatter(translation).format(**kwargs)

In @ZRunner code for i18n/translator.py:

def translate(key: str, **kwargs) -> Union[List[Union[str, Any]], str]:
    """Actually translate something and apply plurals/kwargs if needed
    
    If the translation is a list of strings, each string will be formatted accordingly and the whole list will be returned"""
    locale = kwargs.pop('locale', config.get('locale'))
    translation = translations.get(key, locale=locale)
    if isinstance(translation, list):
        # if we can apply plural/formats to the items, let's try
        if all(isinstance(data, (str, dict)) for data in translation):
            # if needed, we should format every item in the list
            if 'count' in kwargs:
                translation = [pluralize(key, data, kwargs['count']) for data in translation]
            # items may be non-plural dictionnaries, which we cannot format
            return [TranslationFormatter(data).format(**kwargs) if isinstance(data, str) else data for data in translation]
        return translation
    if 'count' in kwargs:
        translation = pluralize(key, translation, kwargs['count'])
    return TranslationFormatter(translation).format(**kwargs)

So I recommend, as a solution, to allow in the config to specify a class, to be used by the translate() function.

In i18n/config.py

settings = {
  ...
  'class': TranslationFormatter,
  ...

In @danhper code for i18n/translator.py:

def translate(key, **kwargs):
    locale = kwargs.pop('locale', config.get('locale'))
    translation = translations.get(key, locale=locale)
    class_ref = config.get('class')
    if 'count' in kwargs:
        translation = pluralize(key, translation, kwargs['count'])
    return class_ref(translation).format(**kwargs)

And in In @ZRunner code for i18n/translator.py:

def translate(key: str, **kwargs) -> Union[List[Union[str, Any]], str]:
    """Actually translate something and apply plurals/kwargs if needed
    
    If the translation is a list of strings, each string will be formatted accordingly and the whole list will be returned"""
    locale = kwargs.pop('locale', config.get('locale'))
    translation = translations.get(key, locale=locale)
    class_ref = config.get('class')
    if isinstance(translation, list):
        # if we can apply plural/formats to the items, let's try
        if all(isinstance(data, (str, dict)) for data in translation):
            # if needed, we should format every item in the list
            if 'count' in kwargs:
                translation = [pluralize(key, data, kwargs['count']) for data in translation]
            # items may be non-plural dictionnaries, which we cannot format
            return [class_ref(data).format(**kwargs) if isinstance(data, str) else data for data in translation]
        return translation
    if 'count' in kwargs:
        translation = pluralize(key, translation, kwargs['count'])
    return class_ref(translation).format(**kwargs)

this way I would be able to use it like this :

>>> import i18n
>>> class MyDelimiter(i18n.translator.TranslationFormatter):
>>>     delimiter = '#'
>>> i18n.set('error_on_missing_placeholder', True)
>>> i18n.set('class', MyDelimiter)
>>> i18n.add_translation('hash', 'Hash: 10 #{value} !')
>>> i18n.t('hash', value='€')
'Hash: 10 € !'

Maybe that would be a little less efficient in terms of perfomance, but that would be way more flexible.

What do you think ?

Best regards.

Support UTF encode

Hi,

We have problems with encoding, if appears some especial character in json (like in French), the error will be,
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 182: invalid continuation byte

The "conflictive" part of the json is:
"nomF": "Fichiers supervisés enregistrés"

Any idea?

Thks in advance

Make pyyaml a requirement

Ruby i18n lib supports YAML by default. For this lib pyyaml is an extra requirement. I suggest to make it a requirement. With this, issues like #7, #9, #19 and #26 could be avoided.

In addition, I will suggest to load .yaml and .yml by default, instead of using i18n.set('filename_format'...), since Ruby lib also does that by default.

[BUG] Requirement package pyyaml won't be installed.

I installed it via pip install python-i18n, and it shows installed successfully. But I can't use yaml file and I soon found that the requirement package pyyaml wasn't installed automatically. I tested it on macOS 10.15.6 and Windows 10 2004. They both showed that problem.

installation issue by poetry

Might not an issue for thin repo but would like to ask here as well.

I have try to install by poetry add python-i18n[YAML] and see it did have extras information with yaml
However, when I realize this doesn't really include i18n.loaders.yaml_loader.

Please suggest if in any case someone knows a better solution for poetry! Apprecaite!

PyYAML warning on loaded (will be deprecated)

Using yaml as load, a warning is raised:

/venv/lib64/python3.7/site-packages/i18n/loaders/yaml_loader.py:12: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

return yaml.load(file_content)

This can be simply fixed with something like:

yaml.load(ymlfile, Loader=yaml.FullLoader)

Subfolders don't work

My folder structure is as following:

| i18n
   | info
      | en.yml
      | de.yml
      | hr.yml
   | config
      | en.yml
      | de.yml
      | hr.yml

And if I load the files as following:
i18n.load_path.append(f"{os.getcwd()}{path_trimmer}i18n") (path trimmer is / or \\ depending on the os) or just i18n.load_path.append("i18n")

And I get translation config.key or info.key I don't get the translation but only the input. However in the README it was stated:

If your files are in subfolders, the foldernames are also used as namespaces, so for example if your translation root path is /path/to/translations and you have the file /path/to/translations/my/app/name/foo.en.yml, the translation namespace for the file will be my.app.name and the file keys will therefore be accessible from my.app.name.foo.my_key.

Can't use 'pip install python-i18n[YAML]'

I try to enter

'''
pip install "python-i18n[YAML]"
'''

But it outputs some message

i18n-0.3.3-py3-none-any.whl
  Ignoring pyyaml: markers 'extra == "YAML"' don't match your environment
Installing collected packages: python-i18n
Successfully installed python-i18n-0.3.3

I want to Install python-i18n[YAML].....

but it Install python-i18n ...

why?

I'm stuck on I18nFileLoadError: no loader available for extension yml

Hi, i'm novice in python and stuck on problem with i18n
It works on development and production server
pass test with ward on my local machine, but in github action fails.

log:

Run ./script/run_tests.sh
Run all tests

$ pwd:
/home/runner/work/pretrained-app/pretrained-app

$ls
#####################5
total 68K
drwxr-xr-x  [7](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:8) runner docker 4.0K Nov 23 17:3[8](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:9) .
drwxr-xr-x 11 runner docker 4.0K Nov 23 17:38 ..
-rw-r--r--  1 runner docker  20K Nov 23 17:38 .pylintrc
-rw-r--r--  1 runner docker  500 Nov 23 17:38 Dockerfile
drwxr-xr-x  7 runner docker 4.0K Nov 23 17:38 app
drwxr-xr-x  4 runner docker 4.0K Nov 23 17:38 config
-rw-r--r--  1 runner docker    0 Nov 23 17:38 conftest.py
-rw-r--r--  1 runner docker  166 Nov 23 17:38 development_server.py
drwxr-xr-x  2 runner docker 4.0K Nov 23 17:38 log
drwxr-xr-x  2 runner docker 4.0K Nov 23 17:38 monkeypatches
-rw-r--r--  1 runner docker  188 Nov 23 17:38 production_server.py
-rw-r--r--  1 runner docker   50 Nov 23 17:38 pytest.ini
-rw-r--r--  1 runner docker  264 Nov 23 17:38 requirements.txt
drwxr-xr-x  8 runner docker 4.0K Nov 23 17:38 tests

#####################7
$ ls ../.

total 64K
drwxr-xr-x 11 runner docker 4.0K Nov 23 17:38 .
drwxr-xr-x  3 runner docker 4.0K Nov 23 17:38 ..
-rw-r--r--  1 runner docker    [9](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:10) Nov 23 17:38 .envrc
drwxr-xr-x  8 runner docker 4.0K Nov 23 17:38 .git
drwxr-xr-x  3 runner docker 4.0K Nov 23 17:38 .github
-rw-r--r--  1 runner docker   88 Nov 23 17:38 .gitignore
-rw-r--r--  1 runner docker   29 Nov 23 17:38 .tool-versions
-rw-r--r--  1 runner docker 1.3K Nov 23 17:38 README.md
drwxr-xr-x  7 runner docker 4.0K Nov 23 17:38 api
drwxr-xr-x  2 runner docker 4.0K Nov 23 17:38 deploy
drwxr-xr-x  3 runner docker 4.0K Nov 23 17:38 docker
-rw-r--r--  1 runner docker  744 Nov 23 17:38 docker-compose.yml
drwxr-xr-x  3 runner docker 4.0K Nov 23 17:38 docs
drwxr-xr-x  4 runner docker 4.0K Nov 23 17:38 frontend
drwxr-xr-x  2 runner docker 4.0K Nov 23 17:38 script
drwxr-xr-x  4 runner docker 4.0K Nov 23 17:38 stages

#####################9
$ ls ../../.
total 12K
drwxr-xr-x  3 runner docker 4.0K Nov 23 17:38 .
drwxr-xr-x  6 runner root   4.0K Nov 23 17:38 ..
drwxr-xr-x [11](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:12) runner docker 4.0K Nov 23 17:38 pretrained-app
#####################11
total 24K


drwxr-xr-x  6 runner root   4.0K Nov 23 17:38 .
drwxr-xr-x [15](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:16) runner docker 4.0K Nov 23 [17](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:18):38 ..
drwxr-xr-x  3 runner docker 4.0K Nov [23](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:24) 17:[38](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:39) _PipelineMapping
drwxr-xr-x  3 runner docker 4.0K Nov 23 17:38 _actions
drwxr-xr-x  4 runner docker 4.0K Nov 23 17:[39](https://github.com/bsa7/pretrained-app/actions/runs/3534337993/jobs/5931088769#step:5:40) _temp
drwxr-xr-x  3 runner docker 4.0K Nov 23 17:38 pretrained-app
/home/runner/work/pretrained-app/pretrained-app/api

test.log trimmed
...

│ /resource_loader.py:115 in recursive_search_dir                          │  
  │                                                                          │  
  │   112 │   seeked_file = config.get('filename_format').format(namespace=s │  
  │   113 │   dir_content = os.listdir(os.path.join(root_dir, directory))    │  
  │   114 │   if seeked_file in dir_content:                                 │  
  │ ❱ 115 │   │   load_translation_file(os.path.join(directory, seeked_file) │  
  │   116 │   elif splitted_namespace[0] in dir_content:                     │  
  │   117 │   │   recursive_search_dir(splitted_namespace[1:], os.path.join( │  
  │   118                                                                    │  
  │                                                                          │  
  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
  │ │        dir_content = ['messages.en.yml']                             │ │  
  │ │          directory = ''                                              │ │  
  │ │             locale = 'en'                                            │ │  
  │ │           root_dir = '/home/runner/work/pretrained-app/pretrained-a… │ │  
  │ │        seeked_file = 'messages.en.yml'                               │ │  
  │ │ splitted_namespace = ['messages']                                    │ │  
  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
  │                                                                          │  
  │ /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/i18n │  
  │ /resource_loader.py:72 in load_translation_file                          │  
  │                                                                          │  
  │    69 def load_translation_file(filename, base_directory, locale=config. │  
  │    70 │   skip_locale_root_data = config.get('skip_locale_root_data')    │  
  │    71 │   root_data = None if skip_locale_root_data else locale          │  
  │ ❱  72 │   translations_dic = load_resource(os.path.join(base_directory,  │  
  │    73 │   namespace = get_namespace_from_filepath(filename)              │  
  │    74 │   load_translation_dic(translations_dic, namespace, locale)      │  
  │    75                                                                    │  
  │                                                                          │  
  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
  │ │        base_directory = '/home/runner/work/pretrained-app/pretraine… │ │  
  │ │              filename = 'messages.en.yml'                            │ │  
  │ │                locale = 'en'                                         │ │  
  │ │             root_data = 'en'                                         │ │  
  │ │ skip_locale_root_data = False                                        │ │  
  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
  │                                                                          │  
  │ /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/i18n │  
  │ /resource_loader.py:23 in load_resource                                  │  
  │                                                                          │  
  │    20 def load_resource(filename, root_data):                            │  
  │    21 │   extension = os.path.splitext(filename)[1][1:]                  │  
  │    22 │   if extension not in loaders:                                   │  
  │ ❱  23 │   │   raise I18nFileLoadError("no loader available for extension │  
  │    24 │   return getattr(loaders[extension], "load_resource")(filename,  │  
  │    25                                                                    │  
  │    26                                                                    │  
  │                                                                          │  
  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
  │ │ extension = 'yml'                                                    │ │  
  │ │  filename = '/home/runner/work/pretrained-app/pretrained-app/api/co… │ │  
  │ │ root_data = 'en'                                                     │ │  
  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
  ╰──────────────────────────────────────────────────────────────────────────╯  
  I18nFileLoadError: no loader available for extension yml                      
                                                                                
  Captured stdout                                                               
                                                                                
    Initialize application log for environment "environment_name='test'"        
    self.fix_pathname(os.path.abspath(__file__ + "/../../"))='/home/runner/work/pretrained-app/pretrained-app/api'           
    self.fix_pathname(os.path.abspath(__file__ + "/../../"))='/home/runner/work/pretrained-app/pretrained-app/api'           
    self.fix_pathname(os.path.abspath(__file__ + "/../../"))='/home/runner/work/pretrained-app/pretrained-app/api'           

    I18n initialization. Locale folder:                                         
    /home/runner/work/pretrained-app/pretrained-app/api/config/locales,         
    root_path: /home/runner/work/pretrained-app/pretrained-app/api              

    i18n.config.settings={'filename_format': '{namespace}.{locale}.{format}',   
    'file_format': 'yml', 'available_locales': ['en'], 'load_path':             
    ['/home/runner/work/pretrained-app/pretrained-app/api/config/locales'],     
    'locale': 'en', 'fallback': 'en', 'placeholder_delimiter': '%',             
    'error_on_missing_translation': True, 'error_on_missing_placeholder': False,
    'error_on_missing_plural': False, 'encoding': 'utf-8',                      
    'namespace_delimiter': '.', 'plural_few': 5, 'skip_locale_root_data': False,
    'enable_memoization': False}                                                
                                                                                

────────────────────────────────────────────────────────────────────────────────
╭──────────── Results ────────────╮
│  9  Tests Encountered           │
│  6  Passes             (66.7%)  │
│  3  Failures           (33.3%)  │
╰─────────────────────────────────╯
──────────────────────────── FAILED in 3.97 seconds ────────────────────────────
Error: Process completed with exit code 1.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.