Comments (8)
ej0cl6
commented on July 16, 2024
I am wondering if you set up the environment according to requirements.txt. It seems like those errors come from different versions of packages.
from synpg.
K-WeiMing
commented on July 16, 2024
Thanks for the quick reply, the associated packages for the conda environment used is shown below:
Conda environment shown below:
# packages in environment at /opt/conda/envs/synpg:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 1_gnu conda-forge
backcall 0.2.0 pypi_0 pypi
ca-certificates 2021.10.8 ha878542_0 conda-forge
cached-property 1.5.2 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
h5py 3.6.0 pypi_0 pypi
ipdb 0.13.9 pypi_0 pypi
ipython 7.32.0 pypi_0 pypi
jedi 0.18.1 pypi_0 pypi
ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 11.2.0 h1d223b6_15 conda-forge
libgomp 11.2.0 h1d223b6_15 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libstdcxx-ng 11.2.0 he4da1e4_15 conda-forge
libzlib 1.2.11 h166bdaf_1014 conda-forge
matplotlib-inline 0.1.3 pypi_0 pypi
ncurses 6.3 h27087fc_1 conda-forge
nltk 3.4.5 pypi_0 pypi
numpy 1.18.1 pypi_0 pypi
openssl 3.0.2 h166bdaf_1 conda-forge
parso 0.8.3 pypi_0 pypi
pexpect 4.8.0 pypi_0 pypi
pickleshare 0.7.5 pypi_0 pypi
pip 22.0.4 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.29 pypi_0 pypi
ptyprocess 0.7.0 pypi_0 pypi
pygments 2.11.2 pypi_0 pypi
python 3.7.10 hf930737_104_cpython conda-forge
python_abi 3.7 2_cp37m conda-forge
readline 8.1 h46c0cb4_0 conda-forge
setuptools 62.1.0 py37h89c1867_0 conda-forge
six 1.16.0 pypi_0 pypi
sqlite 3.38.2 h4ff8645_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
toml 0.10.2 pypi_0 pypi
torch 1.2.0 pypi_0 pypi
tqdm 4.64.0 pypi_0 pypi
traitlets 5.1.1 pypi_0 pypi
wcwidth 0.2.5 pypi_0 pypi
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
xz 5.2.5 h516909a_1 conda-forge
zlib 1.2.11 h166bdaf_1014 conda-forge
Appreciate if there is some insight into this. Thanks!
from synpg.
ej0cl6
commented on July 16, 2024
Hi unfortunately, I did not encounter these issues before. For the first one, you may use debugger like ipdb or pdb to check the value of orig and see what's wrong there. For the second one, you can try to convert byte string to normal string before parsing it. Please refer to https://stackoverflow.com/questions/606191/convert-bytes-to-a-string
from synpg.
K-WeiMing
commented on July 16, 2024
For the first issue, printing out the result gives orig[:-1]: b'', orig[-1]: 96 I'll check out issue on the second one. Was wondering if there are any package differences in the environment and perhaps I can update them to see if it resolves any of the errors. Thanks!
from synpg.
han032206
commented on July 16, 2024
For the first issue, printing out the result gives
orig[:-1]: b'', orig[-1]: 96I'll check out issue on the second one. Was wondering if there are any package differences in the environment and perhaps I can update them to see if it resolves any of the errors. Thanks!
I've encountered the same error when running fine-tuning, and I have the same advice for you to provide a full list of packages for reference. Thanks!
from synpg.
ej0cl6
commented on July 16, 2024
Since the code for BPE is partially copied from https://github.com/rsennrich/subword-nmt
Currently, I have no idea why this error happens.
I attach my environment for reference as well.
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 0_gnu conda-forge
_pytorch_select 0.2 gpu_0
argh 0.26.2 pypi_0 pypi
attrs 19.3.0 py_0
backcall 0.1.0 py_0 conda-forge
blas 1.0 mkl
bleach 3.1.0 py36_0
blis 0.4.1 pypi_0 pypi
ca-certificates 2020.1.1 0
catalogue 1.0.0 pypi_0 pypi
certifi 2019.11.28 py36_0
cffi 1.13.2 py36h2e261b9_0
chardet 3.0.4 pypi_0 pypi
comet-git-pure 0.19.14 pypi_0 pypi
comet-ml 3.0.2 pypi_0 pypi
configobj 5.0.6 pypi_0 pypi
cudatoolkit 10.0.130 0
cudnn 7.6.5 cuda10.0_0
cycler 0.10.0 pypi_0 pypi
cymem 2.0.3 pypi_0 pypi
decorator 4.4.1 py_0 conda-forge
defusedxml 0.6.0 py_0
entrypoints 0.3 py36_0
everett 1.0.2 pypi_0 pypi
gmp 6.1.2 h6c8ec71_1
h5py 2.10.0 py36h7918eee_0
hdf5 1.10.4 hb1b8bf9_0
idna 2.7 pypi_0 pypi
importlib-metadata 1.4.0 pypi_0 pypi
importlib_metadata 1.3.0 py36_0
intel-openmp 2019.4 243
ipdb 0.12.3 py_0 conda-forge
ipykernel 5.1.3 py36h39e3cac_1
ipython 7.11.1 py36h5ca1d4c_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
jedi 0.15.2 py36_0 conda-forge
jinja2 2.10.3 py_0
json5 0.8.5 py_0
jsonschema 3.0.2 pypi_0 pypi
jupyter_client 5.3.4 py36_0
jupyter_core 4.6.1 py36_0
jupyterlab 1.2.5 pyhf63ae98_0
jupyterlab_server 1.0.6 py_0
kiwisolver 1.1.0 pypi_0 pypi
libffi 3.2.1 he1b5a44_1006 conda-forge
libgcc-ng 9.2.0 h24d8f2e_2 conda-forge
libgfortran-ng 7.3.0 hdf63c60_0
libgomp 9.2.0 h24d8f2e_2 conda-forge
libsodium 1.0.16 h1bed415_0
libstdcxx-ng 9.2.0 hdf63c60_2 conda-forge
markupsafe 1.1.1 py36h7b6447c_0
matplotlib 3.0.0 pypi_0 pypi
mistune 0.8.4 py36h7b6447c_0
mkl 2019.4 243
mkl-service 2.3.0 py36he904b0f_0
mkl_fft 1.0.15 py36ha843d7b_0
mkl_random 1.1.0 py36hd6b4f25_0
more-itertools 8.1.0 pypi_0 pypi
murmurhash 1.0.2 pypi_0 pypi
nbconvert 5.6.1 py36_0
nbformat 4.4.0 py36_0
ncurses 6.1 hf484d3e_1002 conda-forge
netifaces 0.10.9 pypi_0 pypi
ninja 1.9.0 py36hfd86e86_0
nltk 3.4.5 py36_0
notebook 6.0.2 py36_0
numpy 1.18.1 pypi_0 pypi
numpy-base 1.18.1 py36hde5b4d6_1
nvidia-ml-py3 7.352.0 pypi_0 pypi
openssl 1.1.1d h7b6447c_4
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py36_1
parso 0.5.2 py_0 conda-forge
pathtools 0.1.2 pypi_0 pypi
pexpect 4.7.0 py36_0 conda-forge
pickleshare 0.7.5 py36_1000 conda-forge
pip 19.3.1 py36_0 conda-forge
plac 1.1.3 pypi_0 pypi
preshed 3.0.2 pypi_0 pypi
prometheus_client 0.7.1 py_0
prompt_toolkit 3.0.2 py_0 conda-forge
ptyprocess 0.6.0 py_1001 conda-forge
pycparser 2.19 py36_0
pygments 2.5.2 py_0 conda-forge
pyparsing 2.4.6 pypi_0 pypi
pyrsistent 0.15.7 pypi_0 pypi
python 3.6.7 h357f687_1006 conda-forge
python-dateutil 2.8.1 py_0
pytorch 1.2.0 cuda100py36h938c94c_0
pyyaml 5.3 pypi_0 pypi
pyzmq 18.1.0 py36he6710b0_0
readline 8.0 hf8c457e_0 conda-forge
requests 2.19.1 pypi_0 pypi
sacrebleu 1.2.11 pypi_0 pypi
send2trash 1.5.0 py36_0
setuptools 45.0.0 py36_1 conda-forge
six 1.14.0 py36_0 conda-forge
spacy 2.2.3 pypi_0 pypi
sqlite 3.30.1 hcee41ef_0 conda-forge
srsly 1.0.1 pypi_0 pypi
subword-nmt 0.3.5 pypi_0 pypi
terminado 0.8.3 py36_0
testpath 0.4.4 py_0
thinc 7.3.1 pypi_0 pypi
tk 8.6.10 hed695b0_0 conda-forge
torch 1.1.0 pypi_0 pypi
tornado 6.0.3 py36h7b6447c_0
tqdm 4.26.0 pypi_0 pypi
traitlets 4.3.3 py36_0 conda-forge
typing 3.7.4.1 pypi_0 pypi
urllib3 1.23 pypi_0 pypi
wasabi 0.6.0 pypi_0 pypi
watchdog 0.9.0 pypi_0 pypi
wcwidth 0.1.8 py_0 conda-forge
webencodings 0.5.1 py36_1
websocket-client 0.57.0 pypi_0 pypi
wheel 0.33.6 py36_0 conda-forge
wurlitzer 2.0.0 pypi_0 pypi
xz 5.2.4 h14c3975_1001 conda-forge
zeromq 4.3.1 he6710b0_3
zipp 1.0.0 pypi_0 pypi
zlib 1.2.11 h516909a_1006 conda-forge
Please let me know if you figure the reason, thanks!
from synpg.
han032206
commented on July 16, 2024
Since the code for BPE is partially copied from https://github.com/rsennrich/subword-nmt Currently, I have no idea why this error happens.
I attach my environment for reference as well.
# Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 0_gnu conda-forge _pytorch_select 0.2 gpu_0 argh 0.26.2 pypi_0 pypi attrs 19.3.0 py_0 backcall 0.1.0 py_0 conda-forge blas 1.0 mkl bleach 3.1.0 py36_0 blis 0.4.1 pypi_0 pypi ca-certificates 2020.1.1 0 catalogue 1.0.0 pypi_0 pypi certifi 2019.11.28 py36_0 cffi 1.13.2 py36h2e261b9_0 chardet 3.0.4 pypi_0 pypi comet-git-pure 0.19.14 pypi_0 pypi comet-ml 3.0.2 pypi_0 pypi configobj 5.0.6 pypi_0 pypi cudatoolkit 10.0.130 0 cudnn 7.6.5 cuda10.0_0 cycler 0.10.0 pypi_0 pypi cymem 2.0.3 pypi_0 pypi decorator 4.4.1 py_0 conda-forge defusedxml 0.6.0 py_0 entrypoints 0.3 py36_0 everett 1.0.2 pypi_0 pypi gmp 6.1.2 h6c8ec71_1 h5py 2.10.0 py36h7918eee_0 hdf5 1.10.4 hb1b8bf9_0 idna 2.7 pypi_0 pypi importlib-metadata 1.4.0 pypi_0 pypi importlib_metadata 1.3.0 py36_0 intel-openmp 2019.4 243 ipdb 0.12.3 py_0 conda-forge ipykernel 5.1.3 py36h39e3cac_1 ipython 7.11.1 py36h5ca1d4c_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge jedi 0.15.2 py36_0 conda-forge jinja2 2.10.3 py_0 json5 0.8.5 py_0 jsonschema 3.0.2 pypi_0 pypi jupyter_client 5.3.4 py36_0 jupyter_core 4.6.1 py36_0 jupyterlab 1.2.5 pyhf63ae98_0 jupyterlab_server 1.0.6 py_0 kiwisolver 1.1.0 pypi_0 pypi libffi 3.2.1 he1b5a44_1006 conda-forge libgcc-ng 9.2.0 h24d8f2e_2 conda-forge libgfortran-ng 7.3.0 hdf63c60_0 libgomp 9.2.0 h24d8f2e_2 conda-forge libsodium 1.0.16 h1bed415_0 libstdcxx-ng 9.2.0 hdf63c60_2 conda-forge markupsafe 1.1.1 py36h7b6447c_0 matplotlib 3.0.0 pypi_0 pypi mistune 0.8.4 py36h7b6447c_0 mkl 2019.4 243 mkl-service 2.3.0 py36he904b0f_0 mkl_fft 1.0.15 py36ha843d7b_0 mkl_random 1.1.0 py36hd6b4f25_0 more-itertools 8.1.0 pypi_0 pypi murmurhash 1.0.2 pypi_0 pypi nbconvert 5.6.1 py36_0 nbformat 4.4.0 py36_0 ncurses 6.1 hf484d3e_1002 conda-forge netifaces 0.10.9 pypi_0 pypi ninja 1.9.0 py36hfd86e86_0 nltk 3.4.5 py36_0 notebook 6.0.2 py36_0 numpy 1.18.1 pypi_0 pypi numpy-base 1.18.1 py36hde5b4d6_1 nvidia-ml-py3 7.352.0 pypi_0 pypi openssl 1.1.1d h7b6447c_4 pandoc 2.2.3.2 0 pandocfilters 1.4.2 py36_1 parso 0.5.2 py_0 conda-forge pathtools 0.1.2 pypi_0 pypi pexpect 4.7.0 py36_0 conda-forge pickleshare 0.7.5 py36_1000 conda-forge pip 19.3.1 py36_0 conda-forge plac 1.1.3 pypi_0 pypi preshed 3.0.2 pypi_0 pypi prometheus_client 0.7.1 py_0 prompt_toolkit 3.0.2 py_0 conda-forge ptyprocess 0.6.0 py_1001 conda-forge pycparser 2.19 py36_0 pygments 2.5.2 py_0 conda-forge pyparsing 2.4.6 pypi_0 pypi pyrsistent 0.15.7 pypi_0 pypi python 3.6.7 h357f687_1006 conda-forge python-dateutil 2.8.1 py_0 pytorch 1.2.0 cuda100py36h938c94c_0 pyyaml 5.3 pypi_0 pypi pyzmq 18.1.0 py36he6710b0_0 readline 8.0 hf8c457e_0 conda-forge requests 2.19.1 pypi_0 pypi sacrebleu 1.2.11 pypi_0 pypi send2trash 1.5.0 py36_0 setuptools 45.0.0 py36_1 conda-forge six 1.14.0 py36_0 conda-forge spacy 2.2.3 pypi_0 pypi sqlite 3.30.1 hcee41ef_0 conda-forge srsly 1.0.1 pypi_0 pypi subword-nmt 0.3.5 pypi_0 pypi terminado 0.8.3 py36_0 testpath 0.4.4 py_0 thinc 7.3.1 pypi_0 pypi tk 8.6.10 hed695b0_0 conda-forge torch 1.1.0 pypi_0 pypi tornado 6.0.3 py36h7b6447c_0 tqdm 4.26.0 pypi_0 pypi traitlets 4.3.3 py36_0 conda-forge typing 3.7.4.1 pypi_0 pypi urllib3 1.23 pypi_0 pypi wasabi 0.6.0 pypi_0 pypi watchdog 0.9.0 pypi_0 pypi wcwidth 0.1.8 py_0 conda-forge webencodings 0.5.1 py36_1 websocket-client 0.57.0 pypi_0 pypi wheel 0.33.6 py36_0 conda-forge wurlitzer 2.0.0 pypi_0 pypi xz 5.2.4 h14c3975_1001 conda-forge zeromq 4.3.1 he6710b0_3 zipp 1.0.0 pypi_0 pypi zlib 1.2.11 h516909a_1006 conda-forgePlease let me know if you figure the reason, thanks!
Problem solved! It turns that the version of tqdm/ipdb/h5py is the issue. Maybe you could provide them in the requirement file.
from synpg.
ej0cl6
commented on July 16, 2024
Problem solved! It turns that the version of tqdm/ipdb/h5py is the issue. Maybe you could provide them in the requirement file.
Updated! Thanks for your help on this.
from synpg.
Related Issues (8)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from synpg.