Olmo 0724 `-hf` checkpoints don't load the proper config when instantiating with OLMoForCausalLM about olmo HOT 2 OPEN

The OLMoForCausalLM in hf_olmo and the OlmoForCausalLM in transformers are different models. The latter have -hf suffix for their repos. You are trying to load a checkpoint of the latter type into the former model, and hence seeing failures.

AutoModelForCausalLM should load both types of checkpoints properly without warnings (you just need to also import hf_olmo for the former type). Can you share the exact warning message you get when you run AutoModelForCausalLM.from_pretrained("allenai/OLMo-7B-0424-hf")?

More context: https://github.com/allenai/OLMo/blob/main/docs/Checkpoints.md

from olmo.

Oh I see, thanks for the clarification! I was confused about the casing difference, too, but this explains it. I don't get any warning when I use AutoModelForCausalLM; was using that as an example of something that's working fine.

I'm currently subclassing hf_olmo's OLMoForCausalLM with some custom inference-time hooks, but I want to load the intermediate training checkpoints, which are only available for OLMo-7B-0424-hf and not OLMo-7B-0424 from what I can see on the HF landing page. Is there any plan to add the checkpoints to the other model version?

Also, maybe we should link this markdown doc from the official HF landing page for the model checkpoints. Might be helpful to others too.

from olmo.