aiot-mlsys-lab / efficient-llms-survey Goto Github PK

Hey Team,

Thanks for the amazing efforts for putting all those amazing literature together and benefit the research and open-source community. We recently release the codes and updated our QuantEase paper which is a new post-training quantization method:

Paper: QuantEase: Optimization-based Quantization for Language Models - An Efficient and Intuitive Algorithm
Code: https://github.com/linkedin/QuantEase

would you mind adding it in the list and the paper if you think it's a good fit? We're also actively development and adding new content in this work. Thank you in advance!

Best regards,
QQ

Thank you for conducting such an insightful survey. I wonder if it's possible to incorporate a recent ICML'23 work from UIUC. It centered on the one-shot compression technique of Pre-trained Langugae Models. This paper investigates the neural tangent kernel (NTK) of the multilayer perceptrons (MLP) modules in a PLM and propose to coin a lightweight PLM through NTK-approximating MLP fusion.

Thanks for the great survey! Could you please include a discussion of this work from Microsoft and UIUC? It proposes a general modular activation mechanism, SMA, that unifies previous works on MoE, adaptive computation, dynamic routing and sparse attention, and further applies SMA to develop a novel architecture, SeqBoat, to achieve SoTA quality-efficiency trade-off on Long Range Arena.

Thanks for the very nice survey!!!

Quantitation-aware Training should be Quantization-aware Training

I find some error in the first quantization equation, I cannot find the variable $H^{H}$

Page 27, third to last line: the training corpse -> corpus

Hi team, thanks for your awesome work.

I discovered that OpenLLM has been integrated with HuggingFace PEFT and supports LLM fine-tuning layers. You can see the documentation here: https://github.com/bentoml/OpenLLM#%EF%B8%8F-serving-fine-tuning-layers

Thanks for the great survey! I have a kind suggestion of including a discussion of this state-of-the-art work Medusa in the efficient LLM inference part.

Code repo: https://github.com/FasterDecoding/Medusa
Blog website: https://sites.google.com/view/medusa-llm
"Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads"