how to merge adapter to original weights about llama-adapter HOT 8 OPEN

@yhyu13 you can refer to this re-implementation based on HF https://github.com/ChrisLiu6/transformers/tree/llama-adapter

from llama-adapter.

same, want create the merge weith version and share on HF

from llama-adapter.

@LiuPearl1 @yhyu13 do you want to merge the adapter weights to the original llama models (similar to lora) or extract the adapter weights?

from llama-adapter.

@LiuPearl1 @yhyu13 do you want to merge the adapter weights to the original llama models (similar to lora) or extract the adapter weights?

Both two ways is ok to me.

from llama-adapter.

@LiuPearl1 for v1, you can use https://github.com/ZrrSkywalker/LLaMA-Adapter/blob/main/alpaca_finetuning_v1/extract_adapter_from_checkpoint.py to extract the adapter, the adapter weight cannot merge into the llama backbone.

from llama-adapter.

@aojunzz I want to merge the adapter weights to the original llama models, does it produce hf transformer format of weights?

By doing so, I can then create a GPTQ 4bit quant using GPTQ-for-LLAMA, and deploy to my textgen-webui instance for testing on a home rig

from llama-adapter.

@yhyu13 The weights cannot be merged into the original weights since we are injecting "soft prompts" into the input, so we are not performing a reparameterization like LoRA. This article explains more in detail! If you want a method that can be merged, I would recommend LoRA.

from llama-adapter.

@BabyChouSr Thanks for clarifying the difference between llama adapater and lora. But my purpose of merging weights is to apply quantitation methods (like GPTQ and more) on a frozen weight that is ready for inference. I am not sure I understand how to perform model compression if I cannot settle on a frozen model. What are the recommended methods for model compression on llama adapted models?

from llama-adapter.