Uploading trained model to HF/saving in HF format locally about cramming HOT 8 CLOSED

Ah sorry for that, the code is a bit dumb. You see the error messages because the converter expects checkpoints to contain identifiers for compiled model weights, but the checkpoint is saved without these.

If you run the load_local_model.py function with impl.compile_torch=False, it should work.

from cramming.

No problem - and I think you had already included those instructions somewhere, I just overlooked them. So that is on me.

I can close this issue for now, but one quick question. I am presuming with the modified architecture it will not currently work with the AutoClass from the HF library? I mainly ask as I saw the modelcard for your model: https://huggingface.co/JonasGeiping/crammed-bert implies you can.

Guessing part of the model card was autogenerated?

For now, to reload a model using the crammed-bert arch we need to use the codebase provided here?

Thanks again for the help and great repo

from cramming.

You can (if everything works correctly), if you import the cramming package first, as shown in the documentation. It will register the model as an additional AutoModelForMaskedLM.

from cramming.

You are absolutely correct - and apologies for not noticing that and probably wasting your time :). Thanks again

from cramming.

No problem

from cramming.

Sorry to come back again - but have now ran into one (I presume final) issue. When you want to use the AutoModelForSequenceClassification as defined by the crammed library, but loading a model that has been pre-trained using the MLM objective - it does not seem to allow adjusting the num_labels via the normal arguments passing.

e.g.

from transformers import AutoModelForSequenceClassification, AutoTokenizer, AutoModelForMaskedLM, AutoConfig
classifier_model = AutoModelForSequenceClassification.from_pretrained("JonasGeiping/crammed-bert", num_labels = 2)

it ends up passing None to torch.linear as it is actually looking for num_labels in the config file rather than arguments.

TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:

Any thoughts? My main desire is to use in a more straight forward fashion.

##########UPDATE#########

For now my crude fix is to replace the num_labels derivation inside crammed_bert.py from

self.num_labels = self.cfg.num_labels

Which uses the config created by the Omegconf class.

to:

self.num_labels = self.config.num_labels

Which uses the config provided from the AutoModel class.

It works - but doesn't seem ideal

from cramming.

Hm that seems like a reasonable fix for now. Really though, the whole translation between the hydra config that the model was originally trained with, and the config that huggingface expects is not so ideal in the long run.

from cramming.

Sure - its no problem really, my use case is quite specific and need to move away from the hydra config is all. It's great work and generally it meshes fine with huggingface.

from cramming.