thu-bpm / selfore Goto Github PK

View Code? Open in Web Editor NEW

47.0 47.0 10.0 492 KB

The source code of paper "SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction"

Python 100.00%

selfore's People

Contributors

Stargazers

Watchers

Forkers

stronglily vhientran vtrn988 farahhuifanyang qcwthu seanswyi bobtuan nahed-abdelgaber wang-ck123 sefeoglu

selfore's Issues

CUDA out of memory

The model always exceeds gpu memory even i set batch_size to 1. So I wonder what's the size of your gpu? Thank you

How to pick 10 types of relations as training set

Simon's article mentions 15 types of relationships, but there is no experimental code, so there is no way to know how to select 10 types of relationships in the TRE_x dataset. Can you give some hints or help?

论文中的Figure1

请问您论文中的Figure1是用什么工具绘制的呢？

Any news for refactoring?

Hi,

we are trying to use your old code, but cannot replicate your results so far - we have trouble in particular with the DEC clustering phase (the algorithm converges with one cluster when KL is minimized).

In particular, could you tell which (if any) among the two lines is the correct one (and for which dataset):

SelfORE/adaptive_clustering.py

Lines 218 to 219 in 6fc1df1

 optimizer = torch.optim.SGD(params=model.module.parameters(), lr=0.1, momentum=0.9) 

 # optimizer = torch.optim.Adam(params=model.module.parameters(), lr=1e-2, weight_decay=1e-5)

And is the number of training epochs (default 200) correct?

BTW, when are you planning to release the refactored code?

Thanks
Benjamin

Data

How can i get train, dev, test data from raw text data? I want run the code but there was no dataset. Thank you.

关于代码的一些疑问

你好，我想请教您关于您代码当中的一些实现细节
在最新的代码当中
1.encoder的实现是: 128*768->relu->400->relu->10，在论文当中是2*768->500->500，这两者之间有什么区别吗？
2.似乎没有重构的预训练过程？
3.bert的特征似乎都是128*768（整个句子的拼接），并不是entity marker对应的2*768的特征？
谢谢！

How to get sentence_train.json, sentence_train_label.json and sentence_index.json?

Hi, can you show how to get these json files from the data-sample.txt?

Issue in Classifier.py

Hi,
I tried to run the codes, and there is problem at the line 136 in classifier.py.
The problem is :

    total_train_loss += loss.item()
AttributeError: 'str' object has no attribute 'item'

Is it related to version of the transformer?

Thanks.

Is there a reason why you put the model on CPU in the classifier?

I've been trying to run the code but it kept crashing my server. I noticed that the line that crashes it is this:

SelfORE/classifier.py

Line 86 in 6499713

outputs = self.model(self.input_ids, self.attention_masks)

Putting the data and model on GPU doesn't crash the server, and I'm just curious what the reason for the decision may be? Is there a particular reason why you didn't use the GPU For this operation?

Thanks.

Refactored version released

Hi,
I'm very interested in your paper and want to replicate it before testing it on other kinds of content. However, I can't do so since it is missing some of the code while it awaits factorization. Would it be possible to check in the old code as a different branch so that I could start to work with that? Alternatively, do you have an estimated date for the refactoring to be done and the revised code available?

Traceback (most recent call last):
  File "run.py", line 12, in <module>
    main()
  File "run.py", line 8, in main
    selfore.start()
  File "/home/ec2-user/SageMaker/ap140821-model-development/Notebooks/SelfORE/selfore.py", line 42, in start
    labels, embs = self.loop()
  File "/home/ec2-user/SageMaker/ap140821-model-development/Notebooks/SelfORE/selfore.py", line 30, in loop
    bert_embs = self.classifier.get_hidden_state()
  File "/home/ec2-user/SageMaker/ap140821-model-development/Notebooks/SelfORE/classifier.py", line 92, in get_hidden_state
    outputs = self.model(self.input_ids, self.attention_masks)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 1344, in forward
    return_dict=return_dict,
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 841, in forward
    return_dict=return_dict,
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 482, in forward
    output_attentions,
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 402, in forward
    output_attentions=output_attentions,
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 339, in forward
    output_attentions,
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 251, in forward
    mixed_value_layer = self.value(hidden_states)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 91, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/functional.py", line 1676, in linear
    output = input.matmul(weight.t())
RuntimeError: [enforce fail at CPUAllocator.cpp:64] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 12567183360 bytes. Error code 12 (Cannot allocate memory)

	optimizer = torch.optim.SGD(params=model.module.parameters(), lr=0.1, momentum=0.9)
	# optimizer = torch.optim.Adam(params=model.module.parameters(), lr=1e-2, weight_decay=1e-5)