nsorros / rasa-converter Goto Github PK
View Code? Open in Web Editor NEWA tool to convert Rasa data to other formats more friendly to tools like spacy, sklearn, transformers
License: MIT License
A tool to convert Rasa data to other formats more friendly to tools like spacy, sklearn, transformers
License: MIT License
Following instructions from the README, pytest
fails.
In a clean venv created following the README
zsh: no matches found: .[spacy,sklearn]
(.venv) ✘ AWS: mantis ~/Documents/mantis/rasa-converter feature/update-dependencies ± pip install ".[spacy,sklearn]" <aws:mantis>
Processing /home/matthew/Documents/mantis/rasa-converter
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting typer
Using cached typer-0.7.0-py3-none-any.whl (38 kB)
Collecting pyyaml
Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Collecting scikit-learn; extra == "sklearn"
Using cached scikit_learn-1.2.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.8 MB)
Collecting spacy; extra == "spacy"
Using cached spacy-3.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.7 MB)
Collecting click<9.0.0,>=7.1.1
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting scipy>=1.3.2
Downloading scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
|████████████████████████████████| 34.5 MB 6.3 MB/s
Collecting joblib>=1.1.1
Using cached joblib-1.2.0-py3-none-any.whl (297 kB)
Collecting numpy>=1.17.3
Using cached numpy-1.24.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting requests<3.0.0,>=2.13.0
Using cached requests-2.28.2-py3-none-any.whl (62 kB)
Collecting cymem<2.1.0,>=2.0.2
Using cached cymem-2.0.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (36 kB)
Requirement already satisfied: setuptools in ./.venv/lib/python3.8/site-packages (from spacy; extra == "spacy"->rasa-converter==0.0.1) (44.0.0)
Collecting tqdm<5.0.0,>=4.38.0
Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting jinja2
Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting wasabi<1.2.0,>=0.9.1
Using cached wasabi-1.1.1-py3-none-any.whl (27 kB)
Collecting thinc<8.2.0,>=8.1.0
Using cached thinc-8.1.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (828 kB)
Collecting spacy-loggers<2.0.0,>=1.0.0
Using cached spacy_loggers-1.0.4-py3-none-any.whl (11 kB)
Collecting spacy-legacy<3.1.0,>=3.0.11
Using cached spacy_legacy-3.0.12-py2.py3-none-any.whl (29 kB)
Collecting catalogue<2.1.0,>=2.0.6
Using cached catalogue-2.0.8-py3-none-any.whl (17 kB)
Collecting murmurhash<1.1.0,>=0.28.0
Using cached murmurhash-1.0.9-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (21 kB)
Collecting pydantic!=1.8,!=1.8.1,<1.11.0,>=1.7.4
Using cached pydantic-1.10.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
Collecting srsly<3.0.0,>=2.4.3
Using cached srsly-2.4.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (492 kB)
Collecting langcodes<4.0.0,>=3.2.0
Using cached langcodes-3.3.0-py3-none-any.whl (181 kB)
Collecting packaging>=20.0
Using cached packaging-23.0-py3-none-any.whl (42 kB)
Collecting preshed<3.1.0,>=3.0.2
Using cached preshed-3.0.8-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
Collecting smart-open<7.0.0,>=5.2.1
Using cached smart_open-6.3.0-py3-none-any.whl (56 kB)
Collecting pathy>=0.10.0
Using cached pathy-0.10.1-py3-none-any.whl (48 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.14-py2.py3-none-any.whl (140 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.12.7-py3-none-any.whl (155 kB)
Collecting charset-normalizer<4,>=2
Using cached charset_normalizer-3.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (195 kB)
Collecting idna<4,>=2.5
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting MarkupSafe>=2.0
Using cached MarkupSafe-2.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting confection<1.0.0,>=0.0.1
Using cached confection-0.0.4-py3-none-any.whl (32 kB)
Collecting blis<0.8.0,>=0.7.8
Using cached blis-0.7.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.2 MB)
Collecting typing-extensions>=4.2.0
Using cached typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Building wheels for collected packages: rasa-converter
Building wheel for rasa-converter (PEP 517) ... done
Created wheel for rasa-converter: filename=rasa_converter-0.0.1-py3-none-any.whl size=6864 sha256=bbe9d5c790fad396f13921bc224f7ad93cb868db82ec67b210fc982fcd9e8a9f
Stored in directory: /home/matthew/.cache/pip/wheels/da/b5/f0/9395a4bceb409fa5a2c665190d914caa55e2fb439030df22c9
Successfully built rasa-converter
Installing collected packages: click, typer, pyyaml, numpy, scipy, joblib, threadpoolctl, scikit-learn, urllib3, certifi, charset-normalizer, idna, requests, cymem, tqdm, MarkupSafe, jinja2, wasabi, catalogue, murmurhash, srsly, packaging, preshed, typing-extensions, pydantic, confection, blis, thinc, spacy-loggers, spacy-legacy, langcodes, smart-open, pathy, spacy, rasa-converter
Successfully installed MarkupSafe-2.1.2 blis-0.7.9 catalogue-2.0.8 certifi-2022.12.7 charset-normalizer-3.0.1 click-8.1.3 confection-0.0.4 cymem-2.0.7 idna-3.4 jinja2-3.1.2 joblib-1.2.0 langcodes-3.3.0 murmurhash-1.0.9 numpy-1.24.2 packaging-23.0 pathy-0.10.1 preshed-3.0.8 pydantic-1.10.5 pyyaml-6.0 rasa-converter-0.0.1 requests-2.28.2 scikit-learn-1.2.1 scipy-1.10.1 smart-open-6.3.0 spacy-3.5.0 spacy-legacy-3.0.12 spacy-loggers-1.0.4 srsly-2.4.5 thinc-8.1.7 threadpoolctl-3.1.0 tqdm-4.64.1 typer-0.7.0 typing-extensions-4.5.0 urllib3-1.26.14 wasabi-1.1.1
(.venv) AWS: mantis ~/Documents/mantis/rasa-converter feature/update-dependencies ± pytest <aws:mantis>
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/matthew/Documents/mantis/rasa-converter
plugins: hydra-core-1.2.0
collected 0 items / 3 errors
===================================================================================================== ERRORS ======================================================================================================
_____________________________________________________________________________________ ERROR collecting tests/test_convert.py ______________________________________________________________________________________
ImportError while importing test module '/home/matthew/Documents/mantis/rasa-converter/tests/test_convert.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/test_convert.py:8: in <module>
from rasa_converter.convert import convert
E ModuleNotFoundError: No module named 'rasa_converter'
------------------------------------------------------------------------------------------------- Captured stderr -------------------------------------------------------------------------------------------------
2023-02-21 11:22:02.297307: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-02-21 11:22:02.297335: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2023-02-21 11:22:03.446654: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2023-02-21 11:22:03.446684: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2023-02-21 11:22:03.446696: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (matthew-XPS-13-9310): /proc/driver/nvidia/version does not exist
________________________________________________________________________________________ ERROR collecting tests/test_io.py ________________________________________________________________________________________
ImportError while importing test module '/home/matthew/Documents/mantis/rasa-converter/tests/test_io.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/test_io.py:5: in <module>
from rasa_converter.io import read_data, write_data
E ModuleNotFoundError: No module named 'rasa_converter'
__________________________________________________________________________________ ERROR collecting tests/test_train_sklearn.py ___________________________________________________________________________________
ImportError while importing test module '/home/matthew/Documents/mantis/rasa-converter/tests/test_train_sklearn.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/test_train_sklearn.py:3: in <module>
from rasa_converter.train_sklearn import train
E ModuleNotFoundError: No module named 'rasa_converter'
================================================================================================ warnings summary =================================================================================================
../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19
/home/matthew/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
../../../.local/lib/python3.8/site-packages/botocore/httpsession.py:41
/home/matthew/.local/lib/python3.8/site-packages/botocore/httpsession.py:41: DeprecationWarning: 'urllib3.contrib.pyopenssl' module is deprecated and will be removed in a future release of urllib3 2.x. Read more in this issue: https://github.com/urllib3/urllib3/issues/2680
from urllib3.contrib.pyopenssl import orig_util_SSLContext as SSLContext
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:23
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:24
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:25
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:28
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
if hasattr(pil_image, 'HAMMING'):
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:29
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:30
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
if hasattr(pil_image, 'BOX'):
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:31
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:33
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
if hasattr(pil_image, 'LANCZOS'):
../../../.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:34
/home/matthew/.local/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================================= short test summary info =============================================================================================
ERROR tests/test_convert.py
ERROR tests/test_io.py
ERROR tests/test_train_sklearn.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 3 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================================================= 11 warnings, 3 errors in 2.12s ==========================================================================================
(.venv) ✘ AWS: mantis ~/Documents/mantis/rasa-converter feature/update-dependencies ± python -c "import rasa_converter" <aws:mantis>
(.venv) AWS: mantis ~/Documents/mantis/rasa-converter feature/update-dependencies ± python -c "import rasa_converter;print(rasa_converter.)" <aws:mantis>
(.venv) ✘ AWS: mantis ~/Documents/mantis/rasa-converter feature/update-dependencies ± <aws:mantis>
(.venv) ✘ AWS: mantis ~/Documents/mantis/rasa-converter feature/update-dependencies ± tree rasa_converter <aws:mantis>
(.venv) AWS: mantis ~/Documents/mantis/rasa-converter feature/update-dependencies ± python -c "from rasa_converter.convert import extract_entities;extract_entities()" <aws:mantis>
Traceback (most recent call last):
File "<string>", line 1, in <module>
python -m venv venv
does not work out of the box for linux users that do not have ensure pip installed. Most probably those users are still creating venv with virtualenv
. We need to amend README accordingly.
At the moment rasa-convert path_to_one_yaml data.json
does not work while it should
It would be good to expose a function in the process that gives text
and intents
which is then used to write the yaml file for someone using this as a library
So it would not recognise something like Do you [feel happy](mood)?
The main usage of the tool is for converting data not training
Currently we support only [<entity-text>](<entity-name>)
but the full syntax is
[<entity-text>]{"entity": "<entity name>", "role": "<role name>", "group": "<group name>", "value": "<entity synonym>"}
More info: https://rasa.com/docs/rasa/training-data-format/#entities
This is an known limitation but necessary to address
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.