Import error inside import hook. about idna HOT 7 CLOSED

check pull request here: #34

from idna.

By applying this change, it means over a megabyte of data will need to be imported for each invocation for functionality that defaults to off and is likely not commonly used. The intent of having the import in the function is to only import the UTS 46 data if you are doing a UTS 46 conversion, and if you are only doing IDNA conversion you are not penalized. Given a number of patches I've received in the past are to do with reducing memory usage, I am hesitant to add a patch like this without further analysis.

Some quick numbers:

python -c 'import idna; idna.decode("xn--p1ai")'
Maximum resident set size (kbytes): 7608

python -c 'import idna; idna.decode("xn--p1ai", uts46=True)'
Maximum resident set size (kbytes): 9060

I am not sure I fully understand the use case around why this is a problem to begin with, but is there a patch that would address your issue and not require uts46data to be loaded unless you are using UTS46 functionality? (i.e. only import that data if uts46=True)

from idna.

Also, can you provide a reduction of the problem so it can be added as a unit test? There is no 'request' method in this library so presumably that is from somewhere else.

from idna.

It looks like he's calling requests and requests is calling idna. To be honest it sounds like his environment is broken if standard 'import's aren't working. He could presumably work around the problem by calling import idna.uts46data at the start of his application so it's pre-loaded. A compromise solution might be to make that 'official' by adding a documented idna.load_data() function or something?

from idna.

The issue was copied from requests library which uses idna (I edited the issue). I found this bug with the latest version of requests.
When using import hooks (PEP302 with python2) you can't import inside a method, that will result in an import error.
I moved the import outside the method and everything is working fine.

doing a import idna.uts46data will work fine at the beginning of the file, however when the method is called uts46_remap the import error will be raised.

I will try to do an unit test to reproduce this issue

from idna.

This really sounds like a bug in your import hook rather than a bug in idna.

from idna.

@kjd since moving the import means over megabytes of data will need. I will close this one since I found a way to avoid using the uts46=True.

from idna.