fenchri / edge-oriented-graph Goto Github PK
View Code? Open in Web Editor NEWSource code for the EMNLP 2019 paper: "Connecting the Dots: Document-level Relation Extraction with Edge-oriented Graphs"
License: Other
Source code for the EMNLP 2019 paper: "Connecting the Dots: Document-level Relation Extraction with Edge-oriented Graphs"
License: Other
It seems that actual test dataset is not used in the experiment.
In configs/parameters_cdr.yaml, dev_filter.data is named as test file and dev_filter.data is used both in training and testing procedure. In other words, you test the model in dev dataset.
Most of all, the performence is not good enough as you report in paper.
In dev dataset:
Loading model & parameters ... TEST | LOSS = 0.46038, ACC = 0.8370 , MICRO P/R/F1 = 0.5810 0.6482 0.6128 | TP/ACTUAL/PRED = 656 /1012 /1129 , TOTAL 5087 | 0h 00m 11s
And in actual test dateset:
TEST | LOSS = 0.47821, ACC = 0.8263 , MICRO P/R/F1 = 0.5732 0.5947 0.5838 | TP/ACTUAL/PRED = 634 /1066 /1106 , TOTAL 5204 | 0h 00m 11s
Is there anthing wrong?
It seems that hypernym filtering has compact to the final results.
When i run the script sh process_cdr.sh in Datasets & Pre-processing module, i have failed.
I have found this line 'os.system('./geniass temp_file.txt temp_file.split.txt > /dev/null 2>&1')' report an error. The error screenshot is shown below. Does the code of data processing need ruby?? My linux system don't install ruby, and it is troubled to install it .
Dear Fenia,
I meet an error when run the function 'load_embeds' in 'loader.py'.
FileNotFoundError: [Errno 2] No such file or directory: '../embeds/PubMed-CDR.txt'
I didn't find the missing file in the folder embeds/ and in 'bio_nlp_vec.tar.gz'. Where does 'PubMed-CDR.txt' come from?
why the gold data you generated in CDR dataset contains 1:NR:2
? I think it should only consider about 1:CID:2
type.
For example, you generated follow lines as gold data
8701013|D015738|D003693|NON-CROSS|1:CID:2
8701013|D015738|D014456|NON-CROSS|1:NR:2
439781|D007213|D007022|NON-CROSS|1:CID:2
439781|D012964|D007022|NON-CROSS|1:NR:2
439781|D011453|D007022|CROSS|1:NR:2
439781|D000809|D007022|CROSS|1:NR:2
why not
8701013|D015738|D003693|NON-CROSS|1:CID:2
439781|D007213|D007022|NON-CROSS|1:CID:2
sh process_cdr.sh
11923it [00:00, 189285.60it/s]
Processing Doc_ID 227508: 0%| | 0/500 [00:00<?, ?it/s]Traceback (most recent call last):
File "process.py", line 123, in
main()
File "process.py", line 62, in main
split_sents = sentence_split_genia(orig_sentences)
File "/home/deep/dp/edge-oriented-graph/data_processing/tools.py", line 248, in sentence_split_genia
with open('temp_file.split.txt', 'r') as ifile:
FileNotFoundError: [Errno 2] No such file or directory: 'temp_file.split.txt'
when I run sh process_cdr.sh
The following problems occurred:
czr@DESKTOP-QR7Q65P MINGW64 /d/edge-oriented-graph-reproduceEMNLP/data_processing
$ sh process_cdr.sh
mv: cannot stat '../data/CDR/processed/Training.data': No such file or directory
mv: cannot stat '../data/CDR/processed/Development.data': No such file or directory
mv: cannot stat '../data/CDR/processed/Test.data': No such file or directory
mv: cannot stat '../data/CDR/processed/Training_filter.data': No such file or directory
mv: cannot stat '../data/CDR/processed/Development_filter.data': No such file or directory
mv: cannot stat '../data/CDR/processed/Test_filter.data': No such file or directory
cat: ../data/CDR/processed/train_filter.data: No such file or directory
cat: ../data/CDR/processed/dev_filter.data: No such file or directory
Hello, fenchri.when Irun this comman:
python3 evaluate.py --pred path_to_predictions_file --gold ../data/CDR/processed/test.gold --label 1:CDR:2
I found that I couldn't find the file "path_to_predictions_file" ,but when I created the discovery in the ** evaluation file, the results were all 0. I hope you can help me
sorry to disturb ! I was stuck at the step
and <process_gda.sh>,
it reminds me that
<FileNotFoundError: [Errno 2] No such file or directory: 'temp_file.split.txt'>
I have no idea what to do with this 'temp_file.split.txt'
PIDtokenized_sentences_separated_by_|relationdirectioncross/non-crossclosest_arg1_start-closest_arg1_endclosest_arg2_start-closest_arg2_endarg1_KB_idmentions_for_arg1_separated_by_|arg1_typestart_token_ids_arg1_separated_by_:end_token_ids_arg1_separated_by_:sentence_ids_separated_by_:arg2_KB_idmentions_for_arg2_separated_by_|arg2_typestart_token_ids_arg2_separated_by_:end_token_ids_arg2_separated_by_:sentence_ids_separated_by_:
I saw this is the processed data format....
i want to ask what is direction mean??
why
if p[1] == "L2R":
h_id, t_id = p[5], p[11]
h_start, t_start = p[8], p[14]
h_end, t_end = p[9], p[15]
else:
t_id, h_id = p[5], p[11]
t_start, h_start = p[8], p[14]
t_end, h_end = p[9], p[15]
what is L2R and R2L why have to change the position of head entity and tail entity??
Hello, sorry to bother you. I would like to ask if the CDR and GDA datasets processed by your data_processing program can be used on other document-level relation extraction models like the publicly available dataset Docred. After processing, I noticed that the files are in the .data format, while Docred dataset is in the .json format. The formats seem to be different on both sides. Any help would be greatly appreciated.
@fenchri
hi fenchri:
i am curious about the graph in this paper,is it an ordinary graph just as a data structure.and diffirent from gnn/gcn?because the words embedding is learning from glove or pretrained but not node2vec?
hope you can see what i say.
thanks!
Hello!
Thanks for your excellent work~
I have a question about filter hyponys.
filter_hypernyms.py
#102 ne[0] = 'not_include' # just don't include the negative pairs, but keep the entities
Why should we do this step~
Looking forward to your reply!
In data processing, after runing sh process_cdr.sh
, I got this error: 'FileNotFoundError: [Errno 2] No such file or directory: 'temp_file.split.txt''
The full error is:
11923it [00:00, 45980.01it/s] Processing Doc_ID 227508: 0%| | 0/500 [00:00<?, ?it/s] Traceback (most recent call last): File "process.py", line 123, in <module> main() File "process.py", line 62, in main split_sents = sentence_split_genia(orig_sentences) File "/disk/apouranb/RAMS/RAMS/classification_code/OT/EoG/edge-oriented-graph/data_processing/tools.py", line 248, in sentence_split_genia with open('temp_file.split.txt', 'r') as ifile: FileNotFoundError: [Errno 2] No such file or directory: 'temp_file.split.txt' Loading examples from ../data/CDR/processed/Training.data Mesh entities: 28470 Positive Docs: 0 Negative Docs: 0 Positive Count: 0 Initial Negative Count: 0 Final Negative Count: 0 Hyponyms: 0 0 12103it [00:00, 45581.38it/s] Processing Doc_ID 6794356: 0%| | 0/500 [00:00<?, ?it/s] Traceback (most recent call last): File "process.py", line 123, in <module> main() File "process.py", line 62, in main split_sents = sentence_split_genia(orig_sentences) File "/disk/apouranb/RAMS/RAMS/classification_code/OT/EoG/edge-oriented-graph/data_processing/tools.py", line 248, in sentence_split_genia with open('temp_file.split.txt', 'r') as ifile: FileNotFoundError: [Errno 2] No such file or directory: 'temp_file.split.txt' Loading examples from ../data/CDR/processed/Development.data Mesh entities: 28470 Positive Docs: 0 Negative Docs: 0 Positive Count: 0 Initial Negative Count: 0 Final Negative Count: 0 Hyponyms: 0 0
dear fenia, I am looking forward to your code :)
When I was compiling genia-tagger-py, I received '503 Service Temporarily Unavailable'. Please tell me how to solve it, thanks.
I use your new version code, the final result on CDR dataset F1=0.75+. Am I wrong, what is the final result in latest version?
Hello, I used your model to run on the document, but I don't know how to get the results of intra-sentence classification or inter-sentence classification. Could you tell me how to do that? Do you use intra-sentences to train your model and only test on the intra-sentence?
@fenchri
There still has a quesion with me when generate processed trained.data form processed traning.pubtator
Cay you share me an exapmle about what processed trained.data looks like.just show me a pid of trained.data.
thanks !!!
Hello,
Thank you for your valuable work!
I have recently tried to follow the procedure to process both CDR and GDA datasets. However, I found a blocking issue, where the http://www.nactem.ac.uk/y-matsu/geniass/geniass-1.00.tar.gz returns a 404 Not Found. Would it be possible to have access to this package?
Best,
Andrei
In your paper, the number of negative pairs of train, dev and test are 4202, 4075 and 4138, however, the statistics.py
gives me 1572(intra)+2822(inter)=4394
, 1709+2540=4249
, 1656+2683=4339
, could you please tell me where may the difference come from?
hi @fenchri
whethere its convenient for you to share the paper's(Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network) source code?
if you can ,this is my mail address:[email protected]
if not,that's ok.
hope you have a nice day!
Thanks for your great work.
I have been trying your model in a new task where there are 9 entity types and 97 relation types. I have done the following modifications:
Is there anything else I need to modify? I am encountering a "nan" issue in the graph layer but not sure whether it is because I forgot something important.
Thanks very much!
In modules.py
at line 56
, you used:
self.embedding.from_pretrained(pret_embeds, freeze=self.freeze)
but the from_pretrained
method is a class method, which does not change the self.embedding
, so only randomly initialized embedding is used and I think it should be
self.embedding = nn.Embedding.from_pretrained(pret_embeds, freeze=self.freeze)
but when I fixed this bug, the PubMed result (about 0.61) is not good as random (about 0.63, which is your result of PubMed) in CDR dataset.
I am a bit confused about the representation of processed_data.
PID<tab>tokenized_sentences_separated_by_|<tab>relation<tab>direction<tab>cross/non-cross<tab>closest_arg1_start-closest_arg1_end<tab>closest_arg2_start-closest_arg2_end<tab>arg1_KB_id<tab>mentions_for_arg1_separated_by_|<tab>arg1_type<tab>start_token_ids_arg1_separated_by_:<tab>end_token_ids_arg1_separated_by_:<tab>sentence_ids_separated_by_:<tab>arg2_KB_id<tab>mentions_for_arg2_separated_by_|<tab>arg2_type<tab>start_token_ids_arg2_separated_by_:<tab>end_token_ids_arg2_separated_by_:<tab>sentence_ids_separated_by_:
For example, the following text is the third segment of the first line in train_filter.data:
1:CID:2 R2L NON-CROSS 58-61 51-52 D008750 alpha - methyldopa|alpha - methyldopa Chemical 58:203 61:206 2:6 D007022 hypotensive Disease 51 52 2
which i think means there is a CID relation, in the concept level, between the entities D008750 and D007022. D008750 has two mentions at (58, 61) and (203,206), while D007022 has only 1 mention at (51,52). Is this correct?
And what does the non-cross mean here, cause it does have a relation across sentences (e.g. mention 2 of D008750 is in sentence 6 and while D007022 is in sentence 2) ? Is it specifically used for the relation between the closest mention pair?
I also got an error while use the script process_cdr,sh..... it is
I think it can't read the temp file.... but I never change the code ....
Originally posted by @miaodatiancai in #24 (comment)
Thanks for your excellent work!
I want to test the effectiveness of the model when consider all possible relations between entities in one document. So I add all the non-related entity pairs to your model, I modified the following:
I would like to ask if there is anything else that needs to be modified?
This happens when I run process.py to process CDRdata:
Traceback (most recent call last):
File "process.py", line 17, in <module>
from tools import sentence_split_genia, tokenize_genia
File "E:\TOOL\pythonProgram\edge-oriented-graph-master\data_processing\tools.py", line 25, in <module>
genia_tagger = GENIATagger(os.path.join("./common", "genia-tagger-py", "geniatagger-3.0.2", "geniatagger"))
File "./common/genia-tagger-py\geniatagger.py", line 34, in __init__
cwd = directory
File "E:\TOOL\Annaconda\envs\pycharm\lib\subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "E:\TOOL\Annaconda\envs\pycharm\lib\subprocess.py", line 1207, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] 系统找不到指定的文件。
when I am trying to run the following commands one by one in colab:
!python3 eog.py --config ../configs/parameters_cdr.yaml --train --gpu -1 --early_stop
!python3 eog.py --config ../configs/parameters_cdr.yaml --train --gpu -1 --nepoch 15 --early_stop
I got the following error:
/content/edge-oriented-graph/src/converter.py:51: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
arrays = numpy.asarray(arrays)
Traceback (most recent call last):
File "eog.py", line 87, in <module>
main()
File "eog.py", line 80, in main
train(parameters)
File "eog.py", line 50, in train
trainer.run()
File "/content/edge-oriented-graph/src/nnet/trainer.py", line 140, in run
self.train_epoch(epoch)
File "/content/edge-oriented-graph/src/nnet/trainer.py", line 178, in train_epoch
loss, stats, predictions, select = self.model(batch)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/content/edge-oriented-graph/src/nnet/network.py", line 305, in forward
batch['section'], batch['distances'])
File "/content/edge-oriented-graph/src/nnet/network.py", line 90, in graph_layer
m_cntx = self.attention(mentions, encoded_seq[info[:, 4]], info, word_sec)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 88, in forward
raise NotImplementedError
NotImplementedError.
I couldn't able to find out what kind of error is this? How do I solve this issue?
Thank you.
Firstly, you can really code, thank you for openning source. The question is that in your implementation, when connecting nodes of different types, the edge's representation depends on the direction, because you just concat first node's representation and second node's representation and ignore their types, for example, x_ms=[n_m;n_s] != x_sm=[n_s;n_m]. It seems to be more like arc, not edge, though I think it's not really matter.
Hi,
Thanks for the great work. I have a couple of questions regarding the output of the BC5CDR processed training data
document 6287825
as follows:
1:CID:2 R2L
, right? The only relation that is CID and cross is 1:CID:2 R2L CROSS 129-130 114-118 D007538 Isoniazid Chemical 129 130 7 D010523 Diseases of peripheral nerves|peripheral nerve disease|Sensori - motor neuropathy|motor neuropathy|Peripheral neuropathy|sensori - motor neuropathy Disease 0:17:51:87:90:114 4:20:55:89:92:118 0:1:3:4:5:5
, which means that this relation is inter-sentence relation between Isoniazid in sentence 7
and sensori- motor neuropathy sentence 5
based on the given node's offset. So technically the annotator inferred this based on the context of the 2 sentences, am I right? I believe you name this as positive relation in your code, true?1:NR:2 L2R CROSS
and intra-sentence relations that aren't part of the gold standard are 1:NR:2 R2L NON-CROSS
, right? I believe you name this as negative relation
in your code, true? inter-sentence relations
? In other words to define the edge construction whether it is mention-mention or mention-sentence or mention-entity or sentence-sentence? For example, I am not sure if 1:NR:2 L2R CROSS 97-98 152-154 D013831 thiamine Chemical 97 98 5 D003389 cranial neuropathy Disease 152 154 8
is sentence-sentence
? Can you point me to the part where you implemented the different edge construction approaches?C1 has a relation with C2 and C2 has a relation with C3, so we can infer that C1 has inter-sentence relation with C3
. I understand that in BC5CDR, you focus on chemical-induced-disease relations, however, for example, if I am interested in chemical-chemical relations, would it be possible to add this to the processed data?Best,
Ghadeer
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.