tbs17 / mathbert Goto Github PK

Some weights of the model checkpoint at tbs17/MathBERT were not used when initializing BertModel: ['cls.predictions.bias', 'cls.predictions.decoder.bias', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Traceback (most recent call last):
  File "D:\...\lib\site-packages\transformers\tokenization_utils_base.py", line 250, in __getattr__
    return self.data[item]
KeyError: 'size'
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "...\test.py", line 33, in <module>
    output = model(encoded_input)
  File "...\test.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "...\test.py", line 944, in forward
    input_shape = input_ids.size()
  File "...\test.py", line 252, in __getattr__
    raise AttributeError
AttributeError

Our data not only contains text but also math symbols and equations.