How to run spanbert with huggingface library ? about spanbert HOT 3 CLOSED

Using transformers version 2.1.1 (most recent as of today), I had to:

!cd /sb
!wget https://dl.fbaipublicfiles.com/fairseq/models/spanbert_hf_base.tar.gz
!tar -xzvf spanbert_hf_base.tar.gz

from transformers import BertForMaskedLM

bert = BertForMaskedLM.from_pretrained('/sb') # have to pass the directory!

For the tokenizer, just use the one from transformers:

from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained('bert-base-cased')  # same tokenizer in bert/spanbert

from spanbert.

You should be able to just register a model as:

https://github.com/facebookresearch/SpanBERT/blob/master/code/pytorch_pretrained_bert/modeling.py#L40-L47

And then you are ready to go.

We haven't tested it with the latest HuggingFace version, but our fine-tuning code is based on the previous HuggingFace's pytorch_pretrained_bert repository:
https://github.com/facebookresearch/SpanBERT/tree/master/code/pytorch_pretrained_bert

from spanbert.

You might also find this helpful, I converted the run_tacred.py to a transformers compatible version: https://github.com/phosseini/SpanBERT/blob/main/code/run_crest.py

from spanbert.