PANIC: unprotected error in call to Lua API (not enough memory) about opennmt HOT 13 CLOSED

@mdasadul you can do this:

cd ~/torch
./clean.sh
TORCH_LUA_VERSION=LUA52 ./install.sh

And then everything shoudl work

from opennmt.

it is probably the same than #20. can you check?

from opennmt.

Yes it's looks like they are same although he was getting it during saving the model but I am getting it middle of preprocessing

from opennmt.

Fixed by #29.

from opennmt.

Sorry to reopen the issue once again. I am trying to preprocess 20Millions sentences and i ended up getting the same error
PANIC: unprotected error in call to Lua API (not enough memory)

from opennmt.

Just to check though, did you get this error with the latest code? We switched to use tds instead of lua tables to avoid this problem.

from opennmt.

@Rush yes I checked out the code this morning. I was having the issue previously with 10 Million sentences and after the tds fix it was resolved but when I increased the training data to 20 million I can see that error again.

@soumith Thanks for pointing out but right now I am middle of training another Seq2seq-attn model and If I reinstall torch it will interrupt current training

from opennmt.

(you can easily install another instance of torch in another directory, there won't be any interference with your current runs - keep us updated!)

from opennmt.

I'm interested to know when this error occurs. Can you share the full preprocessing logs and the command line you ran?

Switching to Lua 5.2 is indeed a definitive fix for this issue but we would like to push the memory limit as far as possible for other Lua versions. #29 already fixes the issue for most use cases though.

from opennmt.

I am using following command line
th preprocess.lua -train_src data/src-train.txt.tok -train_tgt data/targ-train.txt.tok -valid_src data/src-test.txt.tok -valid_tgt data/targ-test.txt.tok -save_data model/28dec

Also I am getting the error when it trying to process line 18349687. So your preprocess script can handle 18349686 sentences

from opennmt.

closing the issue for now. use LUA 5.2 is a fix.

from opennmt.

Seeing same issue with 8 GB, 32 GB & 256 GB RAM. must be something other than memory issue.

from opennmt.

This is a LuaJIT memory issue. The total memory of Lua objects (tables, strings) is limited to about 2GB.

See https://stackoverflow.com/questions/35155444/why-is-luajits-memory-limited-to-1-2-gb-on-64-bit-platforms

from opennmt.