Comments (5)
Under the device of GeForce GTX TITAN X, the training process usually takes nearly an hour.
Thank you for your attention for TensorFlow-NRE, and I think your suggestions are really useful. We will implement the GPU optimization in the future work.
from opennre.
I find where the problem is. In my task, most entity pairs just have a few corresponding sentences (1 or 2). As a result, that for-loop will make my code extremely slow for GPU. Now, I do some over-sampling to pad the training data so that each entity pair has the same amount of sentences. And then, I can vectorize that for-loop. It speeds up my code significantly.
However, for your dataset, it seems that each entity pair has quite a bit sentences, am I right? And I think that's why it's not so slow for your task.
Do you accept a pull request? Maybe I can vectorize that for-loop and bring some speed improvements. Or you can implement this idea yourselves, it's quite intuitive I think.
from opennre.
Yes, in the NYT dataset, a lot of entity pairs have a large number of sentences. And in the initial.py you can see that I use the getsmall() function to split them to avoid out of memory during training, which happens a lot during training if not splitting them. You can also see in the training codes that I check the total number of sentences before feeding them into the feed_dict.
As for the for-loop, because the number of the sentences the entity pairs contain is uncertain, it's really a tough thing for me to do the feed_dict. So I use the for-loop inside the training/testing process which is obviously not the best choice, but I cannot come up with something better than that. If you have a better way, please feel free to inform me or directly make the pull request. In fact, TensorFlow Fold maybe is helpful to the problem.
I think you can do the pull request directly without informing me. We will check the difference and merge them if it actually makes sense.
Thank you for your support and suggestions for TensorFlow-NRE!
from opennre.
In NER tasks, is there any specific advantage of 1) grouping training sentences by entity pair and treating each individual group as a batch as opposed to 2) flattening all the sentences and using evenly sliced batches?
from opennre.
I think maybe you should read the reference papers listed in ReadMe.md firstly. The reason to group sentences by entity pairs is the sentence-level attention. Different sentences for the same entity pairs contain different amount of information for the relation. And we want to pay attention to the sentences contain more information. If you flatten all the sentences and use evenly sliced batches, each sentence has the average level of attention.
You can see more details in the paper [Lin et al., 2016] Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. Neural Relation Extraction with Selective Attention over Instances. In Proceedings of ACL.
from opennre.
Related Issues (20)
- 相对路径下的opennre无法使用,但报错显示的文件并不存在。 HOT 1
- 这个软件包是不兼容win平台吗 HOT 3
- Chinese pretrained models links broken? HOT 2
- Missing code for get_model, encoder, etc.? HOT 1
- model- bag_attention.py-98行 HOT 2
- Is downloading/using pretained models of opennre wiki80_bert_softmax links broken? I'm not able to use the opennre models, it was working fine until yesterday. HOT 1
- TypeError: Couldn't build proto file into descriptor pool: duplicate file name (sentencepiece_model.proto) HOT 2
- TypeError: expected str, bytes or os.PathLike object, not NoneType HOT 3
- OSS License compatibility question HOT 1
- import openNRE HOT 3
- Extract Chinese Relationships HOT 1
- 更换数据集 HOT 2
- Difficulties to obtain results on example/ scripts (train_supervised_bert.py)
- New datasets
- Error(s) in loading state_dict for SoftmaxNN HOT 2
- 可否提供一下训练非中英语种的详细步骤? HOT 1
- pretrain.py for window version HOT 1
- 模型是否可以用于中文关系提取
- metric is not taken into account in example/train_supervised_bert.py HOT 1
- Is downloading/using pretained models of opennre wiki80_cnn_softmax links broken? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opennre.