Comments (7)
I also have this same issue on an instance with 45G memory, so I do not think it is a memory issue. I am using the Dockerfile provided by the repo, so there should be no dependency problems.
I am pretty convinced that this is an error with the way the corenlp server is supposed to shut down. I was also able to "force" this error to happen when I keyboard interrupt the program early, when the "train" section is being loaded into a registry
.
train section: 6%|โโโโโโโโโโโ
[pretrained_embeddings.py] tokenize() method called ...
[pretrained_embeddings.py] tokenize() method called ...
[pretrained_embeddings.py] tokenize() method called ...
^CTraceback (most recent call last):
File "run.py", line 109, in <module>
main()
File "run.py", line 73, in main
preprocess.main(preprocess_config)
File "/app/ratsql/commands/preprocess.py", line 56, in main
preprocessor.preprocess()
File "/app/ratsql/commands/preprocess.py", line 35, in preprocess
self.model_preproc.add_item(item, section, validation_info)
File "/app/ratsql/models/enc_dec.py", line 44, in add_item
self.enc_preproc.add_item(item, section, enc_info)
File "/app/ratsql/models/spider/spider_enc.py", line 168, in add_item
preprocessed = self.preprocess_item(item, validation_info)
File "/app/ratsql/models/spider/spider_enc.py", line 203, in preprocess_item
cv_link = compute_cell_value_linking(question, item.schema)
File "/app/ratsql/models/spider/spider_match_utils.py", line 123, in compute_cell_value_linking
ret = db_word_match(word, column.orig_name, column.table.orig_name, schema.connection)
File "/app/ratsql/models/spider/spider_match_utils.py", line 91, in db_word_match
cursor.execute(p_str)
KeyboardInterrupt
train section: 6%|โโโโโโโโโโโ | 533/8659 [01:31<23:21, 5.80it/s]
Exception ignored in: <function CoreNLP.__del__ at 0x7f62693dbef0>
Traceback (most recent call last):
File "/app/ratsql/resources/corenlp.py", line 24, in __del__
File "/root/.local/lib/python3.7/site-packages/corenlp/client.py", line 83, in stop
File "/opt/conda/lib/python3.7/subprocess.py", line 1790, in kill
AttributeError: 'NoneType' object has no attribute 'SIGKILL'
Still don't know the fix, and I'm experimenting with some things. If I get the answer I'll post it here, but otherwise this information might be useful to you all :)
Edit- just following up. I did a full run of the preprocessing with L#24 of corenlp.py commented out (just replaced with with some kind of print statement for debugging). The code will reach its end, with no error, and you will get the preprocessing files that you need (check your data/ directory!). In conclusion, this corenlp error should not affect the data preprocessing at all. The error is just from the <class 'corenlp.client.CoreNLPClient'>
object terminating incorrectly :) Hope that helps!
from rat-sql.
Thank you for your comment @hclent.
I change the 23-25 lines to:
def del(self):
# self.client.stop()
pass
and it works too.
from rat-sql.
yeah, i have the same problem ๐
from rat-sql.
yeah, i have the same problem ๐
I have tried to increase docker memory to 32gb. But, got this:
train section: 100%|โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ| 8659/8659 [1:04:40<00:00, 2.23it/s]
DB connections: 100%|โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ| 166/166 [00:00<00:00, 267.34it/s]
val section: 100%|โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ| 1034/1034 [10:14<00:00, 1.68it/s]
1580 words in vocab
Exception ignored in: <function CoreNLP.del at 0x7fc9f7051290>
Traceback (most recent call last):
File "/app/ratsql/resources/corenlp.py", line 24, in del
File "/root/.local/lib/python3.7/site-packages/corenlp/client.py", line 83, in stop
File "/opt/conda/lib/python3.7/subprocess.py", line 1790, in kill
AttributeError: 'NoneType' object has no attribute 'SIGKILL'
So, i think it is not the memory problem. Have you solve the problem? Please give some suggestion,thanks.
from rat-sql.
Hi @zsLin177
I have not solved this.
from rat-sql.
Even I'm facing the same issue
from rat-sql.
Thank you @hclent for your comment
from rat-sql.
Related Issues (20)
- How did you generate the heatmap plots for the attentions? HOT 2
- Issue during preprocessing: 'CoreNLP' object has no attribute 'client' HOT 6
- Do you have notebook version for RATSQL ? HOT 5
- Can`t train the model with GPU on a server with RTX3090 HOT 1
- Execution accuracy
- An error in preprocess HOT 3
- preporcessing issue HOT 2
- 0% accuracy during evaluation from trained model. - GLOVE HOT 1
- Decoder vocabulary file not created/found HOT 1
- How to find the 'terminal' value in the questions HOT 1
- The process is killed every 280 steps HOT 2
- Schema Modeling HOT 3
- How to execute my own queries in BERT
- Failed to generate val.jsonl after the preprocess of BERT-version rat-sql
- Trained model
- System Requirements
- ่ฟไธชๆจกๅไปฃ็ ๏ผๅฏไปฅๅบ็จไธไธญๆ็ๆฐๆฎไน
- Colab Implementation of RatSQL and SPIDER dataset HOT 1
- How to customize for other dataset?
- ๅฆไฝๅฐๆฐๆฎๅบๅๅไธ็ปๅฎ็queryๅฏน้ฝ
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rat-sql.