Unable to numerize 225755. i tried to numerize it using:
numerize('two hundred twenty fife thousand seven hundred and fifty-five')
but the output is:
225007 hundred 55

numerize('2B/2B')

Output:
'2B1/2B'

Is it possible to handle other languages as well with spacy?

Found some cases in which the library fails to unify numbers.
Some cases identified:

• thirty two and forty one
  • Expected: 32 and 41
  • Gets: 32 and 40 1
• thirty two and forty one thousand
  • Expected: 32 and 41000
  • Gets: 32 and 40 1000

Reason found:
there's a method called andition that I could understand have the goal of unifying numbers when separated by whitespace and magnitude of them are such that should be following the way of reading (from left to right).

A fix was implemented in #21 . The tests are running without any failure.
New tests where added to control the issue.

Hey @jaidevd ,

Just tried to create an experimental wrapper of your package in R. It basically uses and still executes on your package.

https://github.com/amrrs/numerizer

Please give a try when you get time!

A

Problem:

We have been using spaCy along with the numerize extension successfully to extract money amounts in string format and convert them into integers. However, after upgrading from spaCy version 3.5.0 to 3.6.1, we are experiencing an issue to reproduce previous result for a specific pattern.

import spacy

nlp = spacy.load("en_core_web_trf")
amount = nlp("55  thousand")._.numerize() #  two spaces between 55 and thousand
print(amount)

Expected result in spaCy 3.5.0 - {55 thousand: '55000'}
Different result in spaCy 3.6.1 - {55 : '55 '}

Above code was able to correctly extract the money amount as an integer (i.e., 55000) in the old spaCy version. However, after upgrading spaCy and en_core_web_trf version to 3.6.1, it fails to pick up the amount if there are two spaces in between the digits.

Environment:

spaCy Version: 3.6.1
docker image: python:3.11.4-slim-bullseye

Please let me know if this is the right place to raise this issue. I can move this to spaCy Github repo if that's more appropriate.

There are three different spaCy models.

• en_core_web_sm
• en_core_web_md
• en_core_web_lg

Having one of them should be sufficient for the extension to work.
This part should be generalized:

>>> from numerizer import numerize
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/USR/.local/lib/python3.10/site-packages/numerizer/__init__.py", line 1, in <module>
    from .numerizer import numerize, spacy_numerize  # NOQA: F401
  File "/home/USR/.local/lib/python3.10/site-packages/numerizer/numerizer.py", line 5, in <module>
    nlp = spacy.load('en_core_web_sm')
  File "/home/USR/.local/lib/python3.10/site-packages/spacy/__init__.py", line 51, in load
    return util.load_model(
  File "/home/USR/.local/lib/python3.10/site-packages/spacy/util.py", line 427, in load_model
    raise IOError(Errors.E050.format(name=name))
OSError: [E050] Can't find model 'en_core_web_sm'. It doesn't seem to be a Python package or a valid path to a data directory.

Workaround: Install the small model:

• python3 -m spacy download en_core_web_sm

Hello,

I detected an issue with some inputs involving parenthesis and slash characters.
For example the text '2*(45+21)/6' is converted into '2*(45+211/6'
I fixed the error by replacing line 266 with:
s = re.sub(r'(?:^|[a-zA-Z])\/(\d+)', r'1/\1', s)

Thanks

Hello,
I am using numerizer directly from pip install. I guess one of the features is convertion every mention of 'a' in my text to 1 even if its not denoting 1.

Also for some one I get the same issue

Is there any workaround or fix for this?

If the input to numerizer is a spacy doc, token or span, use it's .text attribute to process the input.

Hi,

I wanted to ask if there is an intention to release a new patch version with the new fix.

I'm using the library and the issue found is affecting the correct functioning of it.

Thank you!

Great library, thank you for your work.

The issue is that floats are not really matched against. In the example below, the regex will match only the last digit 2 from the 1.2. Then it concatenates, resulting in a wrong value.

from numerizer import numerize

In [0]: numerize("1.2 million")
Out[0]: '1.2000000'

Suppose my spacy doc is as follows:

Narendra Modi is leading in Varanasi with one hundred and fifty three votes. He is contesting from two constituencies.

Then, we have two numerical entities here.

1. one hundred and fifty three is a spacy span.
2. two is a spacy token.

For both of them, we can add extensions as follows:

>>> token
two
>>> span
one hundred and fifty three
>>> token._.numerize
'2'
>>> span._.numerize
'153'