prajjwal1 / generalize_lm_nli Goto Github PK

Issue
run_glue.py fails at Line 233 "if trainer.is_world_master():"

Solution
Replace "is_world_master" with "is_world_process_zero"
Reference: dmis-lab/biobert-pytorch#7

Self-explanatory!

Google's release of the 24 smaller BERT models at https://github.com/google-research/bert do not have the .meta file. For example, the bert-base model (https://storage.googleapis.com/bert_models/2020_02_20/uncased_L-12_H-768_A-12.zip) provided in the list does not the .meta file, while the original release of bert-base by Google at https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip contains the .meta file. Is the .meta file needed if we load the TF checkpoints in Pytorch using huggingface?

Hi @prajjwal1,

Thank you very much for this great repo and work.
I would be interested in working with your bert-tiny model.

For this work, it would be of importance to know what data exactly this model was obtained from. I could not find a concrete pointer.
Would you be able to guide me to a specification of this information?

Thank you a lot in advance and kind regards!

Hi,

quick question- do all the bert-(mini / tiny / medium / small) models have the same tokenizer?

Hi

I have tried to use the bert-small-mnli model hosted on HF, and seems I have a couple of problems:

1. The tokenizer doesn't seem to have a mxlen provided, so no padding happens AFAICT. Is that correct?
2. There's a difference in the results I get from the HF API and when I locally run the inference code. Which or all of these should I have in the batch? input_ids, token_type_ids and attention_mask? If I just use input_ids, the results seem to match with the HF API (there's still a small difference, but I am tokenizing a bit differently), but otherwise, there's a huge difference.
3. In any case, the HF API (and my local inference code) results are surprisingly low: for a sample of 100 validation_matched instances, I get an acc score of 21%. Do you have any insight on this? I am using hf datasets to download mnli data, so don't know if that is responsible for something. I saw something to that effect in the README, but quite not sure how should I change the labels in the dataset, if needed.