xrsrke / stable-diffusion-from-scratch Goto Github PK

View Code? Open in Web Editor NEW

Implementation of Stable Diffusion from scratch [WORK IN PROGRESS]

License: MIT License

Python 0.28% Jupyter Notebook 99.71% CSS 0.02%

stable-diffusion-from-scratch's Introduction

Implementation of Stable Diffusion

This repository contains Stable Diffusion’s implementation in PyTorch.

Updates

This isn't actually an implementation of stable diffusion from scratch. I cancelled this project and started working on RLHF

Install

pip install -r requirements.txt
pip install -e .

Usage

from PIL import Image
from foundation.stable_diffusion import StableDiffusion

prompt = "holy young female battle robot flying award winning, portrait bust symmetry faded tetrachromacycolors arctic background tim hildebrandt wayne barlowe bruce pennington donato giancola larry elmore masterpiece trending on artstation cinematic composition beautiful lighting hyper detailed!!! 8 k oil on canva"

model = StableDiffusion(n_inference_steps=30)
images = model.generate(prompt)

pil_images = [Image.fromarray(image) for image in images]
pil_images[0]

Paper Reviews

CLIP - Multimodal Neurons

VAE

Variational Autoencoder (VAE) model with KL loss from the paper Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Wellin

UNET

Diffusion Process

Contributing

This repository is still a work in progress.
Currently, no downloads and no executables are provided.

I welcome many contributors who can help.

License

Licensed under the MIT license.

stable-diffusion-from-scratch's People

Stargazers

Watchers

stable-diffusion-from-scratch's Issues

Progress Updates

Update: got the pipeline working (check out github). This week, the goal is to reimplement CLIP from scratch. I'm finding the Clip tokenizer to be a bit challenging