Giter VIP home page Giter VIP logo

jdiffusion's Introduction

JDiffusion

Introduction

JDiffusion is a diffusion model library for generating images or videos based on Jittor, Jtorch, diffusers.

Installation

0. Clone JDiffusion & Prepare Env

git clone https://github.com/JittorRepos/JDiffusion.git
#We recommend using conda to configure the Python environment.
conda create -n jdiffusion python=3.9
conda activate jdiffusion

1. Install Requirements

Our code is based on JTorch, a high-performance dynamically compiled deep learning framework fully compatible with the PyTorch interface, please install our version of library.

pip install git+https://github.com/JittorRepos/jittor
pip install git+https://github.com/JittorRepos/jtorch
pip install git+https://github.com/JittorRepos/diffusers_jittor
pip install git+https://github.com/JittorRepos/transformers_jittor

or just

pip install -r requirement.txt

2. Install JDiffusion

cd JDiffusion
pip install -e .

We also provide a docker image (md5:62c305028dae6e62d3dff885d5bc9294) about our environment.

3.Optional Requirements

If you encounter No module named 'cupy':

# Install CuPy from source
pip install cupy
# Install CuPy for cuda11.2 (Recommended, change cuda version you use)
pip install cupy-cuda112

Usage

Use just like diffusers, our scripts are in ./examples/test_*.py, you can try on it.

Some example outputs on these scripts are in ./examples/output.

Stable Diffusion

"a photo of a cat"

image

Latent Consistency Model(LCM)

"Self-portrait oil painting, a beautiful cyborg with golden hair, 8k"

image

Control-Net

input image :

image

"the mona lisa"

image

SDXL

  • text to image

    "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"

    image

  • refiner

    "A majestic lion jumping from a big stone at night"

    image

Instruct-Pix2Pix

input image :

image

"make the mountains snowy":

image

AnimateDiff

"masterpiece, bestquality, highlydetailed, ultradetailed, sunset, moon on the sky"
"orange sky, warm lighting, fishing boats, ocean waves seagulls, "
"rippling water, wharf, silhouette, serene atmosphere, dusk, evening glow, "
"golden hour, coastal landscape, seaside scenery"

gif

Level of Support

We now support inference.

Some acceleration tools and memory-saving tools are currently unavailable, like offload, vae_slicing, accelerators..., but you can use jittor's shared memory system.

We welcome more individuals to contribute additional code and models to this repository, in support of the Jittor community.

jdiffusion's People

Contributors

ldyang694 avatar zjp-shadow avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

jdiffusion's Issues

Where is the docker?

I'm a jittor beginner,I noticed the docker image in README, but I can't find the docker link. Is it my problem?

Problem with import torch.cuda

在运行 run_all.py 时遇到问题

Traceback (most recent call last):
File "/root/JDiffusion/examples/dreambooth/run_all.py", line 3, in
from JDiffusion.pipelines import StableDiffusionPipeline
File "/root/JDiffusion/python/JDiffusion/init.py", line 2, in
from .models import (
File "/root/JDiffusion/python/JDiffusion/models/init.py", line 1, in
from .autoencoder_kl_jittor import AutoencoderKL
File "/root/JDiffusion/python/JDiffusion/models/autoencoder_kl_jittor.py", line 19, in
from diffusers.loaders import FromOriginalVAEMixin
File "", line 1055, in _handle_fromlist
File "/root/anaconda3/envs/jdiffusion/lib/python3.9/site-packages/diffusers/utils/import_utils.py", line 692, in getattr
module = self._get_module(self._class_to_module[name])
File "/root/anaconda3/envs/jdiffusion/lib/python3.9/site-packages/diffusers/utils/import_utils.py", line 704, in _get_module
raise RuntimeError(
RuntimeError: Failed to import diffusers.loaders.single_file because of the following error (look up to see its traceback):
No module named 'torch.cuda.amp'; 'torch.cuda' is not a package

已经确认安装了 torch

KeyError when load_lora_weights in run_all.py

I added "conv" "conv1" and "conv2" to the target_modules .Now target_modules=["to_k", "to_q", "to_v", "to_out.0", "add_k_proj", "add_v_proj","conv1","conv2","conv"] .It trained well,but when pipe.load_lora_weights in run_all.py a key error was encountered.The following picture describes the error message
640c23941449e05191e6ec15742274c
jittor=1.3.9.8
peft= 0.10.0

to_out.0 in lora doesn't work

when Lora model passed to peft(0.10.0) ,peft dosen't solve to_out.0 this layer, so we can't train this layer. The details are, to_out is a Sequential, the code in peft try to set attribute 0 of the sequential, but actually, it only set an new attribute for this Sequential, but real "0"layer is in the layers attribute of sequential is not replaced by lora.

To solve this problem, I can only modify the peft to adapt the model passed by jdiffusion.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.