hironsan / asari Goto Github PK
View Code? Open in Web Editor NEWJapanese sentiment analyzer implemented in Python.
License: MIT License
Japanese sentiment analyzer implemented in Python.
License: MIT License
pip install git+https://github.com/Hironsan/asari
最新コードですと、下記部分でエラーになるようです。
from asari.api import Sonar
sonar = Sonar()
/usr/local/lib/python3.8/site-packages/sklearn/utils/deprecation.py:143: FutureWarning: The sklearn.svm.classes module is deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.svm. Anything that cannot be imported from sklearn.svm is now part of the private API.
warnings.warn(message, FutureWarning)
/usr/local/lib/python3.8/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator LinearSVC from version 0.20.2 when using version 0.23.1. This might lead to breaking code or invalid results. Use at your own risk.
warnings.warn(
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.8/site-packages/asari/api.py", line 21, in init
self.estimator = joblib.load(model_file)
File "/usr/local/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 585, in load
obj = _unpickle(fobj, filename, mmap_mode)
File "/usr/local/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 504, in _unpickle
obj = unpickler.load()
File "/usr/local/lib/python3.8/pickle.py", line 1210, in load
dispatchkey[0]
File "/usr/local/lib/python3.8/pickle.py", line 1526, in load_global
klass = self.find_class(module, name)
File "/usr/local/lib/python3.8/pickle.py", line 1577, in find_class
import(module, level=0)
ModuleNotFoundError: No module named 'sklearn.externals.joblib'
importing joblib from sklearn.externals is deprecated
Error: While importing "app", an ImportError was raised: Traceback (most recent call last): File "/Users/.virnev/lib/python3.7/site-packages/flask/cli.py", line 240, in locate_app __import__(module_name) File "/Users/app/app.py", line 4, in from asari.api import Sonar File "/Users/.virnev/lib/python3.7/site-packages/asari/api.py", line 12, in from sklearn.externals import joblib ImportError: cannot import name 'joblib' from 'sklearn.externals'
Python37\lib\site-packages\sklearn\externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
should be fixed by directly importing joblib
$ python sentiment_evaluate.py
C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\utils\deprecation.py:
143: FutureWarning: The sklearn.svm.classes module is deprecated in version 0.2
2 and will be removed in version 0.24. The corresponding classes / functions sho
uld instead be imported from sklearn.svm. Anything that cannot be imported from
sklearn.svm is now part of the private API.
warnings.warn(message, FutureWarning)
C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\base.py:329: UserWarn
ing: Trying to unpickle estimator LinearSVC from version 0.20.2 when using versi
on 0.23.2. This might lead to breaking code or invalid results. Use at your own
risk.
warnings.warn(
C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\utils\deprecation.py:
143: FutureWarning: The sklearn.preprocessing.label module is deprecated in ver
sion 0.22 and will be removed in version 0.24. The corresponding classes / funct
ions should instead be imported from sklearn.preprocessing. Anything that cannot
be imported from sklearn.preprocessing is now part of the private API.
warnings.warn(message, FutureWarning)
C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\base.py:329: UserWarn
ing: Trying to unpickle estimator LabelEncoder from version 0.20.2 when using ve
rsion 0.23.2. This might lead to breaking code or invalid results. Use at your o
wn risk.
warnings.warn(
C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\base.py:329: UserWarn
ing: Trying to unpickle estimator _SigmoidCalibration from version 0.20.2 when u
sing version 0.23.2. This might lead to breaking code or invalid results. Use at
your own risk.
warnings.warn(
C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\base.py:329: UserWarn
ing: Trying to unpickle estimator CalibratedClassifierCV from version 0.20.2 whe
n using version 0.23.2. This might lead to breaking code or invalid results. Use
at your own risk.
warnings.warn(
C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\base.py:329: UserWarn
ing: Trying to unpickle estimator TfidfTransformer from version 0.20.2 when usin
g version 0.23.2. This might lead to breaking code or invalid results. Use at yo
ur own risk.
warnings.warn(
C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\base.py:329: UserWarn
ing: Trying to unpickle estimator TfidfVectorizer from version 0.20.2 when using
version 0.23.2. This might lead to breaking code or invalid results. Use at you
r own risk.
warnings.warn(
2020-08-27 14:38:47 [INFO] Sentiment: Started.
Traceback (most recent call last):
File "sentiment_evaluate.py", line 40, in <module>
Sentiment().evaluate()
File "sentiment_evaluate.py", line 36, in evaluate
print(self._asari_sa(example))
File "sentiment_evaluate.py", line 31, in _asari_sa
res = self.asari.ping(text=sent)['top_class']
File "C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\asari\api.py", line 2
7, in ping
vector = self.preprocessor.transform([text])
File "C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\feature_extra
ction\text.py", line 1880, in transform
X = super().transform(raw_documents)
File "C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\feature_extra
ction\text.py", line 1250, in transform
_, X = self._count_vocab(raw_documents, fixed_vocab=True)
File "C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\feature_extra
ction\text.py", line 1110, in _count_vocab
for feature in analyze(doc):
File "C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\feature_extra
ction\text.py", line 111, in _analyze
doc = ngrams(doc)
File "C:\Users\Merli\anaconda3\envs\tm\lib\site-packages\sklearn\feature_extra
ction\text.py", line 242, in _word_ngrams
n_original_tokens = len(original_tokens)
TypeError: object of type 'generator' has no len()
What does the model here use as training data? Didn't see an explanation anywhere...
Thanks for the great project!
I am currently trying to use asari, but when I resolve one dependency, it breaks another package dependency or causes other errors.
I have tried running various python versions, joblib
, Janome
, scikit-learn
and asari
, but so far it is still not resolved.
If you could please tell me the python version and the requirements.txt
output by pip freeze > requirements.txt
that the code in the README works with.
Thank you in advance.
When I tried to run it, I got the following error.
>>> sonar.ping(text="広告多すぎる")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/site-packages/asari/api.py", line 27, in ping
vector = self.preprocessor.transform([text])
File "/usr/local/lib/python3.8/site-packages/sklearn/feature_extraction/text.py", line 1880, in transform
X = super().transform(raw_documents)
File "/usr/local/lib/python3.8/site-packages/sklearn/feature_extraction/text.py", line 1250, in transform
_, X = self._count_vocab(raw_documents, fixed_vocab=True)
File "/usr/local/lib/python3.8/site-packages/sklearn/feature_extraction/text.py", line 1110, in _count_vocab
for feature in analyze(doc):
File "/usr/local/lib/python3.8/site-packages/sklearn/feature_extraction/text.py", line 111, in _analyze
doc = ngrams(doc)
File "/usr/local/lib/python3.8/site-packages/sklearn/feature_extraction/text.py", line 242, in _word_ngrams
n_original_tokens = len(original_tokens)
TypeError: object of type 'generator' has no len()
It seems depends on old scikit-learn, so I can't use it with 0.24.2.
~/anaconda3/lib/python3.7/pickle.py in load_global(self)
1374 module = self.readline()[:-1].decode("utf-8")
1375 name = self.readline()[:-1].decode("utf-8")
-> 1376 klass = self.find_class(module, name)
1377 self.append(klass)
1378 dispatch[GLOBAL[0]] = load_global
~/anaconda3/lib/python3.7/pickle.py in find_class(self, module, name)
1424 elif module in _compat_pickle.IMPORT_MAPPING:
1425 module = _compat_pickle.IMPORT_MAPPING[module]
-> 1426 __import__(module, level=0)
1427 if self.proto >= 4:
1428 return _getattribute(sys.modules[module], name)[0]
ModuleNotFoundError: No module named 'sklearn.svm.classes'
Do you have plan to update the model?
I think it is relevant with #6 .
I am using python 3.10.9 and pip 22.3.1 on raspberry pi zero.
I tried to install asari but got the following error and could not install it.
pi@raspberrypi:~/asari $ python3 -m pip install . --log ./pip_log.txt
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/asari
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting scikit-learn<2.0.0,>=1.1.1
Using cached scikit-learn-1.2.0.tar.gz (7.2 MB)
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Installing build dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
I have tried the following.
Is there any possibility that asari is not compatible with python3.10?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.