求助 about bert-embedding-frequently-asked-question HOT 7 CLOSED

Traceback (most recent call last):
File "write_vecs2bin.py", line 74, in
write_vec2bin.write_bert_vecs4sheets()
File "write_vecs2bin.py", line 69, in write_bert_vecs4sheets
self.write_bert_vecs(owner_name=sheet_name, num=i)
File "write_vecs2bin.py", line 50, in write_bert_vecs
orgin_query_vec = np.array(self.bc.encode(orgin_query_list))
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/sentence_transformers/SentenceTransformer.py", line 158, in encode
out_features = self.forward(features)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/sentence_transformers/models/DistilBERT.py", line 34, in forward
output_states = self.bert(**features)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/transformers/modeling_distilbert.py", line 498, in forward
inputs_embeds = self.embeddings(input_ids) # (bs, seq_length, dim)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/transformers/modeling_distilbert.py", line 108, in forward
seq_length = input_ids.size(1)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

UKPLab/sentence-transformers#406
原因如上，是因为transformers的版本太高导致的，降级即可，这个问题前段时间部署新环境的时候我也遇到过。但是当时没有及时写到工作笔记里。我现在使用的可用版本是transformers==3.0.2

from bert-embedding-frequently-asked-question.

Traceback (most recent call last):
File "write_vecs2bin.py", line 74, in
write_vec2bin.write_bert_vecs4sheets()
File "write_vecs2bin.py", line 69, in write_bert_vecs4sheets
self.write_bert_vecs(owner_name=sheet_name, num=i)
File "write_vecs2bin.py", line 50, in write_bert_vecs
orgin_query_vec = np.array(self.bc.encode(orgin_query_list))
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/sentence_transformers/SentenceTransformer.py", line 158, in encode
out_features = self.forward(features)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/sentence_transformers/models/DistilBERT.py", line 34, in forward
output_states = self.bert(**features)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/transformers/modeling_distilbert.py", line 498, in forward
inputs_embeds = self.embeddings(input_ids) # (bs, seq_length, dim)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/transformers/modeling_distilbert.py", line 108, in forward
seq_length = input_ids.size(1)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

from bert-embedding-frequently-asked-question.

执行python write_vecs2bin.py的时候报错。

from bert-embedding-frequently-asked-question.

感谢大佬。

from bert-embedding-frequently-asked-question.

感谢大佬。

客气啦，都是同行，相互交流帮助。

from bert-embedding-frequently-asked-question.

Traceback (most recent call last):
File "write_vecs2bin.py", line 74, in
write_vec2bin.write_bert_vecs4sheets()
File "write_vecs2bin.py", line 69, in write_bert_vecs4sheets
self.write_bert_vecs(owner_name=sheet_name, num=i)
File "write_vecs2bin.py", line 50, in write_bert_vecs
orgin_query_vec = np.array(self.bc.encode(orgin_query_list))
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/sentence_transformers/SentenceTransformer.py", line 158, in encode
out_features = self.forward(features)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/sentence_transformers/models/DistilBERT.py", line 34, in forward
output_states = self.bert(**features)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/transformers/modeling_distilbert.py", line 498, in forward
inputs_embeds = self.embeddings(input_ids) # (bs, seq_length, dim)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/transformers/modeling_distilbert.py", line 108, in forward
seq_length = input_ids.size(1)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

UKPLab/sentence-transformers#406
原因如上，是因为transformers的版本太高导致的，降级即可，这个问题前段时间部署新环境的时候我也遇到过。但是当时没有及时写到工作笔记里。我现在使用的可用版本是transformers==3.0.2

赞，建议写到readm里面，自己debug半天也没改对

from bert-embedding-frequently-asked-question.

Traceback (most recent call last):
File "write_vecs2bin.py", line 74, in
write_vec2bin.write_bert_vecs4sheets()
File "write_vecs2bin.py", line 69, in write_bert_vecs4sheets
self.write_bert_vecs(owner_name=sheet_name, num=i)
File "write_vecs2bin.py", line 50, in write_bert_vecs
orgin_query_vec = np.array(self.bc.encode(orgin_query_list))
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/sentence_transformers/SentenceTransformer.py", line 158, in encode
out_features = self.forward(features)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/sentence_transformers/models/DistilBERT.py", line 34, in forward
output_states = self.bert(**features)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/transformers/modeling_distilbert.py", line 498, in forward
inputs_embeds = self.embeddings(input_ids) # (bs, seq_length, dim)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/ananconda3/envs/befaq/lib/python3.6/site-packages/transformers/modeling_distilbert.py", line 108, in forward
seq_length = input_ids.size(1)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

UKPLab/sentence-transformers#406
原因如上，是因为transformers的版本太高导致的，降级即可，这个问题前段时间部署新环境的时候我也遇到过。但是当时没有及时写到工作笔记里。我现在使用的可用版本是transformers==3.0.2

赞，建议写到readm里面，自己debug半天也没改对

好的，已经更新到requirements.txt里了。

from bert-embedding-frequently-asked-question.