This GitHub repository summarizes papers and resources related to the text-to-image generation task.

If you have any suggestions about this repository, please feel free to start a new issue or pull requests.

• [Mar. 7th] All available CVPR, ICLR, and AAAI 2024 papers and references are updated. Papers highlighted with ⚠️ will be updated as soon as their metadata is avaiable.
• [Mar. 1st] Websites of the off-the-shelf text-to-image generation products and toolkits are summarized.

• Products
• To-Do Lists
• Papers
  • Survey Papers
  • Text-to-Image Generation
    • Year 2024
    • Year 2023
    • Year 2022
    • Year 2021
    • Year 2020
  • Conditional Text-to-Image Generation
    • Year 2024
    • Year 2023
    • Year 2022
  • Personalized Text-to-Image Generation
    • Year 2024
    • Year 2023
  • Text-Guided Image Editing
    • Year 2024
    • Year 2023
    • Year 2022
  • Text Image Generation
    • Year 2024
• Datasets
• Toolkits
• Q&A
• References
• Star History

• Published Papers on Conferences
  • Update CVPR 2024 Papers
  • Update AAAI 2024 Papers
    • Update ⚠️ Papers and References
    • Update arXiv References into CVPR and AAAI Versions
  • Update ICLR 2024 Papers
  • Update NeurIPS 2024 Papers
• Create A List with only Diffusion Model-based Papers
• Regular Maintenance of Preprint arXiv Papers and Missed Papers

<🎯Back to Top>

<🎯Back to Top>

• Text-to-Image Generation
  • Year 2024
    • ACM Computing Surveys
      • Diffusion Models: A Comprehensive Survey of Methods and Applications [Paper]
  • Year 2023
    • TPAMI
      • Diffusion Models in Vision: A Survey [Paper] [Code]
    • arXiv
      • Text-to-image Diffusion Models in Generative AI: A Survey [Paper]
      • State of the Art on Diffusion Models for Visual Computing [Paper]
  • Year 2022
    • arXiv
      • Efficient Diffusion Models for Vision: A Survey [Paper]
• Conditional Text-to-Image Generation
  • Year 2024
    • arXiv
      • Controllable Generation with Text-to-Image Diffusion Models: A Survey [Paper]
• Text-Guided Image Editing
  • Year 2024
    • arXiv
      • Diffusion Model-Based Image Editing: A Survey [Paper] [Code]

<🎯Back to Top>

• Year 2024
  • CVPR
    • DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models [Paper] [Code]
    • InstanceDiffusion: Instance-level Control for Image Generation [Paper] [Code] [Project]
    • ECLIPSE: A Resource-Efficient Text-to-Image Prior for Image Generations [Paper] [Code] [Project] [Demo]
    • Instruct-Imagen: Image Generation with Multi-modal Instruction [Paper]
    • Learning Continuous 3D Words for Text-to-Image Generation [Paper] [Code]
    • HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances [Paper]
    • Rich Human Feedback for Text-to-Image Generation [Paper]
    • MarkovGen: Structured Prediction for Efficient Text-to-Image Generation [Paper]
    • Customization Assistant for Text-to-image Generation [Paper]
    • ADI: Learning Disentangled Identifiers for Action-Customized Text-to-Image Generation [Paper] [Project]
    • UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs [Paper]
    • Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation [Paper]
    • Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting [Paper] [Code]
    • CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation [Paper] [Code] [Project] [Demo]
    • ⚠️ Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder [Paper]
    • ⚠️ On the Scalability of Diffusion-based Text-to-Image Generation [Paper]
    • ⚠️ MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation [Paper]
    • ⚠️ Discriminative Probing and Tuning for Text-to-Image Generation [Paper]
    • ⚠️ Learning Multi-dimensional Human Preference for Text-to-Image Generation [Paper]
    • ⚠️ Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation [Paper]
    • ⚠️ Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning [Paper]
    • ⚠️ Adversarial Text to Continuous Image Generation [Paper]
    • ⚠️ Dynamic Prompt Optimizing for Text-to-Image Generation [Paper]
  • ICLR
    • Patched Denoising Diffusion Models For High-Resolution Image Synthesis [Paper] [Code]
    • Relay Diffusion: Unifying diffusion process across resolutions for image synthesis [Paper] [Code]
    • SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis [Paper] [Code]
    • Compose and Conquer: Diffusion-Based 3D Depth Aware Composable Image Synthesis [Paper] [Code]
    • PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis [Paper] [Code] [Project] [Demo]
  • AAAI
    • Semantic-aware Data Augmentation for Text-to-image Synthesis [Paper]
    • ⚠️ Text-to-Image Generation for Abstract Concepts [Paper]
  • arXiv
    • Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation [Paper]
    • RPG: Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs [Paper] [Code]
    • Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation [Paper] [Code]
    • ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models [Paper] [Code] [Project]
    • DiS: Scalable Diffusion Models with State Space Backbone [Paper] [Code]
    • InstantID: Zero-shot Identity-Preserving Generation in Seconds [Paper] [Code] [Project] [Demo]
    • PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models [Paper] [Code]
    • PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation [Paper] [Code] [Project]
    • CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion [Paper] [Code]
    • ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment [Paper] [Code] [Project]
    • Text2Street: Controllable Text-to-image Generation for Street Views [Paper]
    • LayerDiffuse: Transparent Image Layer Diffusion using Latent Transparency [Paper] [Code]
  • Others
    • Stable Cascade [Blog] [Code]

<🎯Back to Top>

• Year 2023
  • CVPR
    • GigaGAN: Scaling Up GANs for Text-to-Image Synthesis [Paper] [Reproduced Code] [Project] [Video]
    • ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model With Knowledge-Enhanced Mixture-of-Denoising-Experts [Paper]
    • Shifted Diffusion for Text-to-image Generation [Paper] [Code]
    • GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis [Paper] [Code]
    • Specialist Diffusion: Plug-and-Play Sample-Efficient Fine-Tuning of Text-to-Image Diffusion Models to Learn Any Unseen Style [Paper] [Code]
    • Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation [Paper]
    • RIATIG: Reliable and Imperceptible Adversarial Text-to-Image Generation with Natural Prompts [Paper] [Code]
  • NeurIPS
    • ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation [Paper] [Code]
    • RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths [Paper] [Project]
  • ICLR
    • Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis [Paper] [Code]
  • ICML
    • StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis [Paper] [Code] [Project] [Video]
    • Muse: Text-To-Image Generation via Masked Generative Transformers [Paper] [Reproduced Code] [Project]
  • ACM MM
    • SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models [Paper] [Code]
    • ControlStyle: Text-Driven Stylized Image Generation Using Diffusion Priors [Paper]
  • SIGGRAPH
    • Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models [Paper] [Code] [Project] [Demo]
  • arXiv
    • P+: Extended Textual Conditioning in Text-to-Image Generation [Paper]
    • SDXL-Turbo: Adversarial Diffusion Distillation [Paper] [Code]
    • Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models [Paper] [Code]
    • StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation [Paper] [Project]
  • Others
    • DALL-E 3: Improving Image Generation with Better Captions [Paper]

<🎯Back to Top>

• Year 2022
  • CVPR
    • 🔥 Stable Diffusion: High-Resolution Image Synthesis With Latent Diffusion Models [Paper] [Code] [Project]
    • Vector Quantized Diffusion Model for Text-to-Image Synthesis [Paper] [Code]
    • DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis [Paper] [Code]
    • LAFITE: Towards Language-Free Training for Text-to-Image Generation [Paper] [Code]
    • Text-to-Image Synthesis based on Object-Guided Joint-Decoding Transformer [Paper]
    • StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis [Paper] [Code]
  • ECCV
    • Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors [Paper] [Code] [Demo]
    • Trace Controlled Text to Image Generation [Paper]
    • Improved Masked Image Generation with Token-Critic [Paper]
    • VQGAN-CLIP: Open Domain Image Generation and Manipulation Using Natural Language [Paper] [Code]
    • TISE: Bag of Metrics for Text-to-Image Synthesis Evaluation [Paper] [Code]
    • StoryDALL-E: Adapting Pretrained Text-to-image Transformers for Story Continuation [Paper] [Code] [Demo]
  • NeurIPS
    • CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers [Paper] [Code]
    • Imagen: Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding [Paper] [Reproduced Code] [Project] [Imagen 2]
  • ACM MM
    • Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation [Paper] [Code]
    • Background Layout Generation and Object Knowledge Transfer for Text-to-Image Generation [Paper]
    • DSE-GAN: Dynamic Semantic Evolution Generative Adversarial Network for Text-to-Image Generation [Paper]
    • AtHom: Two Divergent Attentions Stimulated By Homomorphic Training in Text-to-Image Synthesis [Paper]
  • arXiv
    • DALLE-2: Hierarchical Text-Conditional Image Generation with CLIP Latents [Paper]
    • PITI: Pretraining is All You Need for Image-to-Image Translation [Paper] [Code]

<🎯Back to Top>

• Year 2021
  • ICCV
    • DAE-GAN: Dynamic Aspect-aware GAN for Text-to-Image Synthesis [Paper] [Code]
  • NeurIPS
    • CogView: Mastering Text-to-Image Generation via Transformers [Paper] [Code] [Demo]
    • UFC-BERT: Unifying Multi-Modal Controls for Conditional Image Synthesis [Paper]
  • ICML
    • DALLE-1: Zero-Shot Text-to-Image Generation [Paper] [Reproduced Code]
  • ACM MM
    • Cycle-Consistent Inverse GAN for Text-to-Image Synthesis [Paper]
    • R-GAN: Exploring Human-like Way for Reasonable Text-to-Image Synthesis via Generative Adversarial Networks [Paper]

<🎯Back to Top>

• Year 2020
  • ACM MM
    • Text-to-Image Synthesis via Aesthetic Layout [Paper]

<🎯Back to Top>

• Year 2024
  • CVPR
    • PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis [Paper]
    • One-Shot Structure-Aware Stylized Image Synthesis [Paper]
    • Grounded Text-to-Image Synthesis with Attention Refocusing [Paper] [[Project]]
    • Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis [Paper] [Code]
    • ⚠️ Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis [Paper]
  • ICLR
    • Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models [Paper] [Code]
  • WACV
    • Training-Free Layout Control with Cross-Attention Guidance [Paper] [Code] [Project] [Demo]
  • AAAI
    • SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-image Generation [Paper]
    • Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models [Paper] [Code]
  • arXiv
    • • DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations [Paper]

<🎯Back to Top>

• Year 2023
  • CVPR
    • GLIGEN: Open-Set Grounded Text-to-Image Generation [Paper] [Code] [Project] [Demo] [Video]
    • Autoregressive Image Generation using Residual Quantization [Paper] [Code]
    • SpaText: Spatio-Textual Representation for Controllable Image Generation [Paper] [Project] [Video]
    • Text to Image Generation with Semantic-Spatial Aware GAN [Paper]
    • ReCo: Region-Controlled Text-to-Image Generation [Paper] [Code]
    • LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation [Paper] [Code]
  • ICCV
    • ControlNet: Adding Conditional Control to Text-to-Image Diffusion Models [Paper] [Code]
    • SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis [Paper] [Code]
  • ICML
    • Composer: Creative and Controllable Image Synthesis with Composable Conditions [Paper] [Code] [Project]
    • MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation [Paper] [Code] [Video] [Project] [Demo]
  • SIGGRAPH
    • Sketch-Guided Text-to-Image Diffusion Models [Paper] [Reproduced Code] [Project]
  • NeurIPS
    • Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models [Paper] [Code] [Project]
    • Prompt Diffusion: In-Context Learning Unlocked for Diffusion Models [Paper] [Code] [Project]
  • WACV
    • More Control for Free! Image Synthesis with Semantic Diffusion Guidance [Paper]
  • ACM MM
  • LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation [Paper]
  • arXiv
    • T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models [Paper] [Code] [Demo]
    • BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing [Paper] [Code]

<🎯Back to Top>

• Year 2024
  • CVPR
    • Cross Initialization for Personalized Text-to-Image Generation [Paper]
    • When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation [Paper] [Code] [Project]
    • Style Aligned Image Generation via Shared Attention [Paper] [Code] [Project]
    • InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning [Paper] [Project]
    • High Fidelity Person-centric Subject-to-Image Synthesis [Paper]
    • RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization [Paper] [Project]
    • ⚠️ FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Composition [Paper]
    • ⚠️ JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation [Paper]
    • ⚠️ Countering Personalized Text-to-Image Generation with Influence Watermarks [Paper]
    • ⚠️ Personalized Residuals for Concept-Driven Text-to-Image Generation [Paper]
    • ⚠️ Improving Subject-Driven Image Synthesis with Context-Agnostic Guidance [Paper]
  • AAAI
    • Decoupled Textual Embeddings for Customized Image Generation [Paper]
• Year 2023
  • CVPR
    • Custom Diffusion: Multi-Concept Customization of Text-to-Image Diffusion [Paper] [Code] [Project]
    • DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation [Paper] [Code] [Project]
  • ICCV
    • ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation [Paper] [Code]
  • ICLR
    • Textual Inversion: An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion [Paper] [Code] [Project]
  • SIGGRAPH
    • Break-A-Scene: Extracting Multiple Concepts from a Single Image [Paper] [Code]
    • Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models [Paper] [Project]
    • LayerDiffusion: Layered Controlled Image Editing with Diffusion Models [Paper]
  • arXiv
    • DreamTuner: Single Image is Enough for Subject-Driven Generation [Paper] [Project]
    • PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding [Paper] [Code]

<🎯Back to Top>

• Year 2024
  • CVPR
    • InfEdit: Inversion-Free Image Editing with Natural Language [Paper] [Code] [Project]
    • Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editing [Paper]
    • Doubly Abductive Counterfactual Inference for Text-based Image Editing [Paper] [Code]
    • Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation [Paper] [Code]
    • Contrastive Denoising Score for Text-guided Latent Diffusion Image Editing [Paper]
    • DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing [Paper] [Code]
    • DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editing [Paper]
    • FreeDrag: Feature Dragging for Reliable Point-based Image Editing [Paper] [Code]
    • Text-Driven Image Editing via Learnable Regions [Paper] [Code] [Project] [Video]
    • LEDITS++: Limitless Image Editing using Text-to-Image Models [Paper] [Code] [Project] [Demo]
    • SmartEdit: Exploring Complex Instruction-based Image Editing with Large Language Models [Paper] [Code] [Project]
    • Edit One for All: Interactive Batch Image Editing [Paper] [Code] [Project]
    • ⚠️ TiNO-Edit: Timestep and Noise Optimization for Robust Diffusion-Based Image Editing [Paper]
    • ⚠️ Person in Place: Generating Associative Skeleton-Guidance Maps for Human-Object Interaction Image Editing [Paper]
    • ⚠️ Referring Image Editing: Object-level Image Editing via Referring Expressions [Paper]
    • ⚠️ The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing [Paper]
    • ⚠️ Prompt Augmentation for Self-supervised Text-guided Image Manipulation [Paper]
  • ICLR
    • Guiding Instruction-based Image Editing via Multimodal Large Language Models [Paper] [Code] [Project]
    • The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing [Paper] [Code] [Project]
    • Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimators [Paper] [Code] [Project]
    • Object-Aware Inversion and Reassembly for Image Editing [Paper] [Code] [Project]
    • Noise Map Guidance: Inversion with Spatial Context for Real Image Editing [Paper]
  • AAAI
    • Tuning-Free Inversion-Enhanced Control for Consistent Image Editing [Paper]
    • BARET: Balanced Attention based Real image Editing driven by Target-text Inversion [Paper]
    • Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference [Paper]
    • High-Fidelity Diffusion-based Image Editing [Paper]
    • AdapEdit: Spatio-Temporal Guided Adaptive Editing Algorithm for Text-Based Continuity-Sensitive Image Editing [Paper]
    • ⚠️ TexFit: Text-Driven Fashion Image Editing with Diffusion Models [Paper]
  • arXiv
    • An Item is Worth a Prompt: Versatile Image Editing with Disentangled Control [Paper] [Code]
    • StableDrag: Stable Dragging for Point-based Image Editing [Paper]
    • One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications [Paper] [Code] [Project]
• Year 2023
  • CVPR
    • Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models [Paper] [Code]
    • SINE: SINgle Image Editing with Text-to-Image Diffusion Models [Paper] [Code]
    • Imagic: Text-Based Real Image Editing with Diffusion Models [Paper]
    • InstructPix2Pix: Learning to Follow Image Editing Instructions [Paper] [Code] [Dataset] [Project] [Demo]
    • Null-text Inversion for Editing Real Images using Guided Diffusion Models [Paper] [Code]
  • ICCV
    • MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing [Paper] [Code] [Project] [Demo]
    • Localizing Object-level Shape Variations with Text-to-Image Diffusion Models [Paper] [Code] [Project] [Demo]
  • ICLR
    • SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations [Paper] [Code] [Project]
• Year 2022
  • CVPR
    • DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation [Paper] [Code]

<🎯Back to Top>

• Year 2024
  • arXiv
    • AnyText: Multilingual Visual Text Generation And Editing [Paper] [Code] [Project]
  • CVPR
    • ⚠️ SceneTextGen: Layout-Agnostic Scene Text Image Synthesis with Integrated Character-Level Diffusion and Contextual Consistency [Paper]

<🎯Back to Top>

• Microsoft COCO: Common Objects in Context [Paper] [Dataset]
• Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning [Paper] [Dataset]
• LAION-5B: An Open Large-Scale Dataset for Training Next Generation Image-Text Models [Paper] [Dataset]

<🎯Back to Top>

<🎯Back to Top>

• Q: The conference sequence of this paper list?
  • This paper list is organized according to the following sequence:
    • CVPR
    • ICCV
    • ECCV
    • WACV
    • NeurIPS
    • ICLR
    • ICML
    • ACM MM
    • SIGGRAPH
    • AAAI
    • arXiv
    • Others
• Q: What does Others refers to?
  • Some of the following studies (e.g., Stable Casacade) does not publish their technical report on arXiv. Instead, they tend to write a blog in their official websites. The Others category refers to such kind of studies.

<🎯Back to Top>

The reference.bib file summarizes bibtex references of up-to-date image inpainting papers, widely used datasets, and toolkits. Based on the original references, I have made the following modifications to make their results look nice in the LaTeX manuscripts:

• Refereces are normally constructed in the form of author-etal-year-nickname. Particularly, references of datasets and toolkits are directly constructed as nickname, e.g., imagenet.
• In each reference, all names of conferences/journals are converted into abbreviations, e.g., Computer Vision and Pattern Recognition -> CVPR.
• The url, doi, publisher, organization, editor, series in all references are removed.
• The pages of all references are added if they are missing.
• All paper names are in title case. Besides, I have added an additional {} to make sure that the title case would also work well in some particular templates.

If you have other demands of reference formats, you may refer to the original references of papers by searching their names in DBLP or Google Scholar.

<🎯Back to Top>

<🎯Back to Top>