Finetuning an autocompressor model about autocompressors HOT 4 CLOSED

You only need to finetune the model -- the tokenizer remains unchanged, e.g., the tokenizer princeton-nlp/AutoCompressor-Llama-2-7b-6k is the same as the standard Llama-2 tokenizer.

from autocompressors.

This should be straightforward!

First, have a look at the train.sh and train_llama.sh scripts in the run/ folder. To fine-tune from an existing AutoCommpressor instead of an LM base model, simply change the model_url variable to a AutoCompressor on huggingface-hub or from a local checkpoint, e.g., changing this line to model_url="princeton-nlp/AutoCompressor-Llama-2-7b-6k" will fine-tune from this model.

from autocompressors.

Thanks!
Also, I couldn't find references to training/finetuning a tokenizer in the paper. But in the example usage, you refer to a custom tokenizer (I think).
tokenizer = AutoTokenizer.from_pretrained("princeton-nlp/AutoCompressor-Llama-2-7b-6k").
Do we need to finetune the tokenizer also and not just the model?

from autocompressors.

@imbalu007 Hi, have you finetuned the AutoCompressor model on downstream task? If so, what about the performance? Thanks.

from autocompressors.