ImportError: cannot import name 'AutoModelWithHeads' from 'transformers' about adapters HOT 17 CLOSED

There's no standard way to integrate adapters into custom models. If our implementations for bert-based models don't work for you, please have a look at our implementation to get an idea how it might be adapted to your needs. Here are the important parts of the implementation:

• adapter_modeling.py implements the single adapter modules
• adapter_bert.py integrates the adapter modules into the BERT architecture. Stacking and Fusion of adapters are implemented here. All the integration into the Huggingface model is done via Python mixins. Note how every module of BERT in modeling_bert.py has a corresponding mixin here. When supporting a custom model, you probably want to take this file and make changes to fit your model.
• adapter_model_mixin.py implements useful methods for saving and loading adapters. The classes here are explained in https://docs.adapterhub.ml/extending.html
• in the actual model class (e.g. modeling_bert.py), we only implement the mixins. No other changes are done here compared to Huggingface.

Those are all the main classes where our changes are implemented.

Unfortunately, we can't give detailed support for implementing custom models. If your model is part of the Huggingface repo, you can open a feature request so we can potentially support it officially. Otherwise, we can try to help out if you have any further specific questions to our implementation.

from adapters.

The pip adapter-transformers 1.0.1 is built on transformers 2.11.0 link
However, the master branch of adapter-transformers is already updated to transformers 3.5.1
You can run pip install git+https://github.com/Adapter-Hub/adapter-transformers.git to get that version.

from adapters.

You can't have transformers and adapter-transformers in your env simultaneously.
Your code is trying to call AutoModelWithHeads which is available in adapter-transformers, but not in transformers.
Try creating a new env, and your bug should be fixed.

from adapters.

from adapters.

from adapters.

from adapters.

from adapters.

from adapters.

then my first reply is what you need to do.
again, you cannot have transformers and adapter-transformers in your env simultaneously.
adapter-transformers is built on transformers 3.5.1

from adapters.

from adapters.

you can find the models which are covered in our documentation.
we are continuously working on supporting more models.

from adapters.

from adapters.

from adapters.

Hi @rabeehkarimimahabadi,

I just ran the notebook you linked and didn't get the issue you mentioned. As @JoPfeiff pointed out, if you run

pip install git+https://github.com/Adapter-Hub/adapter-transformers.git

you get the latest version based on transformers 3.5.1. It will show 1.0.1 in the list of packages because that's our version number, not Huggingface's version number. Basically, it is transformers 3.5.1 although our version number is different.

You can merge the latest version of the master branch from Huggingface if you need an even newer version.

For the adapter implementations, please refer to this section in the documentation: https://docs.adapterhub.ml/extending.html or the relevant files in code: https://github.com/Adapter-Hub/adapter-transformers/blob/master/src/transformers/adapter_modeling.py, https://github.com/Adapter-Hub/adapter-transformers/blob/master/src/transformers/adapter_bert.py.

Hope this helps!

from adapters.

from adapters.

Hi @calpt and @JoPfeiff thanks for all info. I tried to read the pointers, I still have difficulty understanding the code base and essential part of adapter layers, I was wondering if any of you have time for a short chat of 15 minutes, I would be greatly thankful for your help. I need to understand what is the minimal amount of code needed to add adapters, to understand how to implement them for my model. thanks

from adapters.

As the discussion has shifted to #92, closing this thread in favor of #92 to keep discussion in one place.

from adapters.