ne7ermore / torch-light Goto Github PK

Hi,I'm new to pytorch,when I run the train.py, I got error below, I try such ways to slove, bur all not work. How this error means,?Thanks.
`D:\workspace\Python\NLP\torch_light-master\LSTM-CNNs-CRF\model.py:158: UserWarning: nn.init.xavier_uniform is now deprecated in favor of nn.init.xavier_uniform_.
init.xavier_uniform(self.char_ebd.weight)
D:\workspace\Python\NLP\torch_light-master\LSTM-CNNs-CRF\model.py:31: UserWarning: nn.init.xavier_uniform is now deprecated in favor of nn.init.xavier_uniform_.
init.xavier_uniform(self.transitions)
D:\Program\Anaconda3\lib\site-packages\torch\cuda_init_.py:116: UserWarning:
Found GPU0 Quadro K600 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
Train Processing: 0it [00:00, ?it/s]------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "D:/workspace/Python/NLP/torch_light-master/LSTM-CNNs-CRF/train.py", line 148, in
loss = train()
File "D:/workspace/Python/NLP/torch_light-master/LSTM-CNNs-CRF/train.py", line 130, in train
loss, _ = model(word, char, label)
File "D:\Program\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "D:\workspace\Python\NLP\torch_light-master\LSTM-CNNs-CRF\model.py", line 187, in forward
char_feats = self.cnn(chars)
File "D:\Program\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "D:\workspace\Python\NLP\torch_light-master\LSTM-CNNs-CRF\model.py", line 163, in forward
encode = self.char_ebd(encode).unsqueeze(1)
File "D:\Program\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "D:\Program\Anaconda3\lib\site-packages\torch\nn\modules\sparse.py", line 108, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "D:\Program\Anaconda3\lib\site-packages\torch\nn\functional.py", line 1076, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got CUDAIntTensor instead (while checking arguments for embedding)`

Game - 104 | data length - 21
kl - nan | lr_multiplier - 2.25 | loss - nan

Traceback (most recent call last):
File "train.py", line 125, in
t.run()
File "train.py", line 62, in run
print(f"Game - {step} | data length - {self.sample(game.play())}")
File "D:\theory\courses\Machine_Learning\torch-light\alpha-zero\game.py", line 152, in play
pi, next_node = mc.search(self.board, node, temperature=1)
TypeError: 'NoneType' object is not iterable

https://github.com/ne7ermore/torch_light/blob/6ec234c1e4db26404c8ef32c51c0050628c9bf64/vae-nlg/optim.py#L24

Hi, I'm reading this code for study and it helps me a lot.
I'm confused by this line:

from the source paper of BERT, I've not found any description that BERT use a conv1d layer in transformer instead of linear transformation.

And from http://nlp.seas.harvard.edu/2018/04/03/attention.html#position-wise-feed-forward-networks, this is implement by a mlp.

Can anyone kindly help me with this problem?

Hi,

@ne7ermore : So I get the following error when I try to load the actor model to the GPU (i.e. in train.py). Any help would be much appreciated.
Traceback (most recent call last): File "train.py", line 114, in <module> actor = actor.cuda() File "/home/user/anaconda2/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py", line 216, in cuda return self._apply(lambda t: t.cuda(device)) File "/home/user/anaconda2/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py", line 146, in _apply module._apply(fn) File "/home/user/anaconda2/envs/py35/lib/python3.5/site-packages/torch/nn/modules/rnn.py", line 123, in _apply self.flatten_parameters() File "/home/user/anaconda2/envs/py35/lib/python3.5/site-packages/torch/nn/modules/rnn.py", line 111, in flatten_parameters params = rnn.get_parameters(fn, handle, fn.weight_buf) File "/home/user/anaconda2/envs/py35/lib/python3.5/site-packages/torch/backends/cudnn/rnn.py", line 165, in get_parameters assert filter_dim_a.prod() == filter_dim_a[0] AssertionError

I checked to make sure that the library versions match with the requirements.

Hi ne7ermore,
Thank you a lot for your code sharing. I was running your retrieval chatbot code train.py and encountered the following error:

Traceback (most recent call last):
File "train.py", line 141, in
loss, corrects, acc, size = evaluate()
File "train.py", line 104, in evaluate
eval_loss += loss.data[0]
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

I searched for the reason and solution, and found that it was a Pytorch version issue, in version > 0.5 you have to change loss.data[0] to loss.item() as indicated in the error message. I just post this issue in order to remind others.
P.S. It seemed that loss.data.item() also worked.

Hi, where can I download the images and verification in the following paths:

/Users/nevermore/code/fuel/SCUT-FBP5500_v2/Images
 
/Users/nevermore/code/fuel/SCUT-FBP5500_v2/train_test_files/5_folders_cross_validations_files/cross_validation_1

facial-beauty-prediction/data/images
facial-beauty-prediction/data/validation

wait- your deep-srl model is for NER task? and I didnt see the viterbi hard constraint part?

您好，LSTM-CNNS-CRF中，运行train.py，报错RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.IntTensor instead (while checking arguments for embedding)。显示是model（word，char，label）的问题

Hi, I was going through your Deep SRL code which is the implementation of the paper Deep Semantic Role Labeling: What Works and What’s Next and i couldn't locate the places where you have implemented the viterbi decoding or the BIO and SRL constraints which are mentioned in the paper. Could you please help me in case i am missing something in the code or have these not been implemented in the code ?
Also, can this implementation handle multiple predicates in the same sentence?

There was a data lacking problem after I ran train.py directly, then I ran corpus.py before training to solve that.

bert中的corpus语料数据找不到了，想知道数据在哪下载

https://github.com/ne7ermore/torch-light/tree/master/Image-Cap
Hi @ne7ermore,
I found this code very educational. Thank you. But, I do not have a GPU. When I run train.py on a CPU, the estimated time of training is about 3-4 weeks. So, could you please share the trained model. It would be of great help. My email id - [email protected].
Thank You

It seems that in biLSTM-CRF model only biLSTM is trained without a CRF layer added to the output layer of biLSTM. CRF is only called after the probability of each tag for each word is predicted by loading a trained biLSTM model (in predict.py). So, the CRF only works in predicting stage rather than in training stage?

When I run train.py, show type error:

Traceback (most recent call last):
File "D:/github/torch_light/Image-Cap/train.py", line 235, in
loss = pre_train_actor()
File "D:/github/torch_light/Image-Cap/train.py", line 120, in pre_train_actor
target, _ = actor(hidden, labels)
File "D:\Anaconda3\envs\python36\lib\site-packages\torch\nn\modules\module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "D:\github\torch_light\Image-Cap\model.py", line 91, in forward
emb_enc = self.lookup_table(labels[:, i])
File "D:\Anaconda3\envs\python36\lib\site-packages\torch\nn\modules\module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda3\envs\python36\lib\site-packages\torch\nn\modules\sparse.py", line 103, in forward
self.scale_grad_by_freq, self.sparse
File "D:\Anaconda3\envs\python36\lib\site-packages\torch\nn_functions\thnn\sparse.py", line 57, in forward
output = torch.index_select(weight, 0, indices)
TypeError: torch.index_select received an invalid combination of arguments - got (torch.cuda.FloatTensor, int, !torch.cuda.IntTensor!), but expected (torch.cuda.FloatTensor source, int dim, torch.cuda.LongTensor index)

def encode(self, imgs):
        if self.training:
            enc = self.enc(imgs)[0]
        else:
            enc = self.enc(imgs)
        enc = self.enc_out(enc)
        return enc

https://github.com/ne7ermore/torch-light/blob/master/Image-Cap/model.py

Trying to train the retrieval-based-chatbots.
There has an error.

Expect for your reply.

▶ python3 train.py
==============================arguments==============================
logdir: logdir
batch_size: 64
lr: 0.001
dropout: 0.5
emb_dim: 200
first_rnn_hsz: 200
fillters: 8
kernel_size: (3, 3)
match_vec_dim: 50
second_rnn_hsz: 50
use_cuda: False
max_cont_len: 10
max_utte_len: 50
dict_size: 156260
============================================================
Traceback (most recent call last):
  File "train.py", line 88, in <module>
    model = Model(args)
  File "/Users/viosey/Workspace/Dev/ML/PyTorch/torch_light/retrieval-based-chatbots/model.py", line 32, in __init__
    self._reset_parameters()
  File "/Users/viosey/Workspace/Dev/ML/PyTorch/torch_light/retrieval-based-chatbots/model.py", line 40, in _reset_parameters
    self.transform_A.bias.data.fill_(0)
AttributeError: 'NoneType' object has no attribute 'data'

https://github.com/ne7ermore/torch-light/blob/master/vae-nlg/train.py#L20

the data parameter seems is the directory of a model, but there is only two corpus and a w2v

Thx