This is a paper list for pretrained recommend System (recommendation) models. It also contains some related research areas such as large language model.

Keyword: Recommend System, pretrained models, large language model

Welcome to open an issue or make a pull request!

• Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect, arXiv, 2020, [paper]
• Self-Supervised Learning for Recommender Systems: A Survey ,arxiv 2022, [paper]
• Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems, arxiv 2022, [paper]
• How Can Recommender Systems Benefit from Large Language Models: A Survey, arxiv 2023, [paper]

• Yelp[link]
• Petdata[link]
• M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining, CVPR 2022 [paper]
• Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems, NeurIPS 2022 [paper]

• Where to Go Next for Recommender Systems? ID-vs. Modality-based recommender models revisited, SIGIR 2023, [paper]
• Generative Recommendation: Towards Next-generation Recommender Paradigm, arxiv 2023, [paper]
• Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights,arxiv 2023, [paper] [code]

• BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer, CIKM 2019 , [paper][code]
• S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization , CIKM-2020 , [paper][code]
• Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation, Recsys 2021 , [paper][code]
• Towards Universal Sequence Representation Learning for Recommender Systems , KDD 2022 , [paper][code]
• Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders, WWW 2023, [paper] [code]

• Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation, SIGIR 2020 , [paper], [code]
• UPRec: User-Aware Pre-training for Recommender Systems ，submitted TKDE in 2021 , [paper]
• U-BERT: Pre-training user representations for improved recommendation, AAAI 2021, [paper]
• UserBERT: Self-supervised User Representation Learning , arxiv 2021 , [paper]
• One4all User Representation for Recommender Systems in E-commerce , arxiv 2021 , [paper]
• One Person, One Model, One World: Learning Continual User Representation without Forgetting, SIGIR 2021 , [paper]
• Scaling Law for Recommendation Models: Towards General-purpose User Representations , AAAI 2023 , [paper]

• Self-supervised Learning for Large-scale Item Recommendations , CIKM 2021 , [paper]
• TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedback , arxiv 2022 , [paper]
• IntTower: the Next Generation of Two-Tower Model for Pre-Ranking System, CIKM 2022 , [paper][code]

• Language Models as Recommender Systems: Evaluations and Limitations , NeurIPS 2021 Workshop ICBINB , [paper]
• CTR-BERT: Cost-effective knowledge distillation for billion-parameter teacher models, [paper]
• Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) , Recsys 2022 , [paper])
• M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems ,arxiv 2022 , [paper]
• PTab: Using the Pre-trained Language Model for Modeling Tabular Data, arxiv 2022, [paper]
• Prompt Learning for News Recommendation, SIGIR 2023, [paper]

• Is ChatGPT a Good Recommender A Preliminary Study, arxiv 2023, [paper]
• Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent, arxiv 2023, [paper]
• Uncovering ChatGPT’s Capabilities in Recommender Systems, arxiv 2023, [paper][code]
• Sparks of Artificial General Recommender (AGR): Early Experiments with ChatGPT, arxiv 2023, [paper]
• Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation, arxiv 2023,[paper] [code]
• TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation, arxiv 2023, [paper]
• PALR: Personalization Aware LLMs for Recommendation, arxiv 2023, [paper]
• Large Language Models are Zero-Shot Rankers for Recommender Systems, arxiv 2023, [paper]
• Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach, arxiv 2023, [paper]
• Leveraging Large Language Models in Conversational Recommender Systems, arxiv 2023, [paper]
• Privacy-Preserving Recommender Systems with Synthetic Query Generation using Differentially Private Large Language Models, arxiv 2023, [paper]
• Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights, arxiv 2023, [paper]
• CTRL: Connect Tabular and Language Model for CTR Prediction, arxiv 2023,[paper]

• Curriculum Pre-Training Heterogeneous Subgraph Transformer for Top-N Recommendation , arxiv 2021 ,[paper]
• Contrastive Pre-Training of GNNs on Heterogeneous Graphs , CIKM 2021 , [paper]
• Self-supervised Graph Learning for Recommendation , SIGIR 2021 , [paper]
• Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation , AAAI 2021 , [paper]

By Xiangyang Li ([email protected]) from Peking University.