This is a curated list of resources dedicated to Knowledge Distillation, Recommendation System, especially Natural Language Processing (NLP).
The goal of this repository is not only storing the references personally but also sharing with people outside.
- Introducing MASS – A pre-training method that outperforms BERT and GPT in sequence to sequence language generation tasks
- A new model and dataset for long-range memory
- Visual Paper Summary: ALBERT (A Lite BERT)
- Learning and Reasoning on Graph for Recommendation
- Natural Language Recommendations: A novel research paper search engine developed entirely with embedding and transformer models
- Distilling Transformers into Simple Neural Networks with Unlabeled Transfer Data
- Attentive Student Meets Multi-Task Teacher: Improved Knowledge Distillation for Pretrained Models
- Robust Language Representation Learning via Multi-task Knowledge Distillation
- Understanding Knowledge Distillation in Neural Sequence Generation
I am waiting for people who wants to contribute to this document. If you know good papers, tutorial, whatsoever, Please pull request! :)