Why use 512 as the max sequence length for fine tuning alpaca? about llama-adapter HOT 2 OPEN

From various sources:

1. 512 covers 95% of the alpaca data
2. reduce vram training cost
3. allows higher batching value due to 2, reduced vram training cost

From what I understand, 512 was chosen as a optimal value that balance training output, cost and speed. Obviously you can change that o 1024 or 2048 if you have larger gpu and want a better training output.

from llama-adapter.

@diegomontoya Thanks your perfect explanation.

For alpaca instruction tuning, we choose 512 as the max sequence length.
For dialog instruction tuning, we choose 2048 as the max sequence length.
For image-text alignment in LLaMa-Adapter V2, we choose 96 as the max sequence length.
For multimodal instruction tuning, we choose 512 as the max sequence length.

from llama-adapter.