genforce / ghfeat Goto Github PK

View Code? Open in Web Editor NEW

156.0 156.0 20.0 5.06 MB

[CVPR 2021] Generative Hierarchical Features from Synthesizing Images

Home Page: https://genforce.github.io/ghfeat/

Python 99.93% Shell 0.07%

ghfeat's Introduction

GenForce Lib for Generative Modeling

An efficient PyTorch library for deep generative modeling. May the Generative Force (GenForce) be with You.

Updates

Encoder Training: We support training encoders on top of pre-trained GANs for GAN inversion.
Model Converters: You can easily migrate your already started projects to this repository. Please check here for more details.

Highlights

Distributed training framework.
Fast training speed.
Modular design for prototyping new models.
Model zoo containing a rich set of pretrained GAN models, with Colab live demo to play.

Installation

Create a virtual environment via conda.

conda create -n genforce python=3.7
conda activate genforce

Install cuda and cudnn. (We use CUDA 10.0 in case you would like to use TensorFlow 1.15 for model conversion.)
```
conda install cudatoolkit=10.0 cudnn=7.6.5
```
Install torch and torchvision.
```
pip install torch==1.7 torchvision==0.8
```
Install requirements
```
pip install -r requirements.txt
```

Quick Demo

We provide a quick training demo, scripts/stylegan_training_demo.py, which allows to train StyleGAN on a toy dataset (500 animeface images with 64 x 64 resolution). Try it via

./scripts/stylegan_training_demo.sh

We also provide an inference demo, synthesize.py, which allows to synthesize images with pre-trained models. Generated images can be found at work_dirs/synthesis_results/. Try it via

python synthesize.py stylegan_ffhq1024

You can also play the demo at Colab.

Play with GANs

Test

Pre-trained models can be found at model zoo.

On local machine:

GPUS=8
CONFIG=configs/stylegan_ffhq256_val.py
WORK_DIR=work_dirs/stylegan_ffhq256_val
CHECKPOINT=checkpoints/stylegan_ffhq256.pth
./scripts/dist_test.sh ${GPUS} ${CONFIG} ${WORK_DIR} ${CHECKPOINT}

Using slurm:

CONFIG=configs/stylegan_ffhq256_val.py
WORK_DIR=work_dirs/stylegan_ffhq256_val
CHECKPOINT=checkpoints/stylegan_ffhq256.pth
GPUS=8 ./scripts/slurm_test.sh ${PARTITION} ${JOB_NAME} \
    ${CONFIG} ${WORK_DIR} ${CHECKPOINT}

Train

All log files in the training process, such as log message, checkpoints, synthesis snapshots, etc, will be saved to the work directory.

On local machine:

GPUS=8
CONFIG=configs/stylegan_ffhq256.py
WORK_DIR=work_dirs/stylegan_ffhq256_train
./scripts/dist_train.sh ${GPUS} ${CONFIG} ${WORK_DIR} \
    [--options additional_arguments]

Using slurm:

CONFIG=configs/stylegan_ffhq256.py
WORK_DIR=work_dirs/stylegan_ffhq256_train
GPUS=8 ./scripts/slurm_train.sh ${PARTITION} ${JOB_NAME} \
    ${CONFIG} ${WORK_DIR} \
    [--options additional_arguments]

Play with Encoders for GAN Inversion

Train

On local machine:

GPUS=8
CONFIG=configs/stylegan_ffhq256_encoder_y.py
WORK_DIR=work_dirs/stylegan_ffhq256_encoder_y
./scripts/dist_train.sh ${GPUS} ${CONFIG} ${WORK_DIR} \
    [--options additional_arguments]

Using slurm:

CONFIG=configs/stylegan_ffhq256_encoder_y.py
WORK_DIR=work_dirs/stylegan_ffhq256_encoder_y
GPUS=8 ./scripts/slurm_train.sh ${PARTITION} ${JOB_NAME} \
    ${CONFIG} ${WORK_DIR} \
    [--options additional_arguments]

Contributors

Member	Module
Yujun Shen	models and running controllers
Yinghao Xu	runner and loss functions
Ceyuan Yang	data loader
Jiapeng Zhu	evaluation metrics
Bolei Zhou	cheerleader

NOTE: The above form only lists the person in charge for each module. We help each other a lot and develop as a TEAM.

We welcome external contributors to join us for improving this library.

License

The project is under the MIT License.

Acknowledgement

We thank PGGAN, StyleGAN, StyleGAN2, StyleGAN2-ADA for their work on high-quality image synthesis. We thank IDInvert and GHFeat for their contribution to GAN inversion. We also thank MMCV for the inspiration on the design of controllers.

BibTex

We open source this library to the community to facilitate the research of generative modeling. If you do like our work and use the codebase or models for your research, please cite our work as follows.

@misc{genforce2020,
  title =        {GenForce},
  author =       {Shen, Yujun and Xu, Yinghao and Yang, Ceyuan and Zhu, Jiapeng and Zhou, Bolei},
  howpublished = {\url{https://github.com/genforce/genforce}},
  year =         {2020}
}

ghfeat's People

Contributors

Stargazers

Watchers

Forkers

peternara peterouzh batermj anthonyyuan hyungkwonko polalbacar lotayou xchlu xujinglin kkodoo killsking hevincent jh-001 cemberk jackzhousz whuhxb emma409 iq-scm zhangjux soumyadeep710

ghfeat's Issues

Pretrained StyleGAN on Imagenet

Brilliant work!
Would you mind providing your pretrained styleGAN model on ImageNet? Thanks!

Upgrading to TF 2.0

Hi!

I am trying to upgrade the GHFeat model to Tensorflow 2.2 from Tensorflow 1.2 for my project. To this end, I used the tf_upgrade_v2 provided in TF2.0. The function made several changes however, It omitted making changes to files in the "dnnlib/" directory. So I am having to do those manually.

There's one statement in "dnnlib/tflib/network.py" which returns an error when interpreted with TF2.0, which I not being able to fix. The troublesome statement is Line 291, given as below:
exec(self._build_module_src, module.__dict__) # pylint: disable=exec-used.
The error trace-back for the same is:
exec(self._build_module_src, module.__dict__) # pylint: disable=exec-used File "<string>", line 11, in <module> ModuleNotFoundError: No module named 'tensorflow.contrib'
I understand that I need to go to the file where module has been defined and change "tf.contrib." to "tf.compat.v1.contrib." / "tf_slim.". However since the trace-back mentions the filename as "String", I am not being able to figure out the file to which I need to make changes. I request the community and repository maintainers to kindly help me out in this regard.

So far, the following attempts of mine to fix this issue have failed:

upgrading the Pickle PROTOCOL to 5
following the steps given here
trying out python's compile() function to figure out the path to the erroneous file

Looking forward to effective insights and discussions in this regard.

Thanks.

Does the whole project include the code of local editing?

Hi, friend, your work is very exciting and wonderful. But I want to know that if the code for different task especially local editing is included in the code you reaveal? Or your code is just used to generate the hierarchical feature needed in multi-tasks you have mentioned? I am a pytorch user, your code is about tensorflow. If it is the latter condition I mentioned above, I think I can use the feature generated by your code more concisely, beccause I just need to take the feature without modifying your code. By the way，what does the “image list” mean? I suppose it is a folder containing some images, but the error that program pulled out indicates that it's not the right understanding. I will be appreciated if I can receive your reply as soon as possible. Thank you!

The pretrained file path has been changed in the readme.

The FFHQ & Tower Pretrained Encoder Dropbox path has been changed.

Encoder reconstruction yields very bad results

Hello,
I have been trying to reproduce the harmonization task, and still haven't managed to get the results shown as examples.

I use the provided face encoder "ghfeat-encoder-ffhq-256.pkl" and the "image.list" file provided in the examples folder.
When performing the feature extraction, the script takes about 20 minutes to finish, and the resulting reconstructed images look like this:

I have performed several tests, using the bedroom encoder too, and the results are always the same:

:

When installing the environment, I changed the decorator library from "decorator==5.0.0" to "decorator" (decorator=5.0.0 yields an error when trying to install).

My distro is Ubuntu 20.04.3 LTS.

how to reproduce the tasks?

Is there a tutorial on how to reproduce the tasks like face detection, harmonization?

model on imagenet

Do u have any plan to release the model trained on ImageNet? lol

the pytorch version and all 9 tasks

Will you release PyTorch version codes? Will you release the related codes for all 9 tasks in your paper?

The pytorch version.

Hello, thank you for your contribution and share your codes. Will you release PyTorch version codes?

how to translate the feature map to the GH-Feature?

Hi, thanks for your work. I want to do some experiments based on your work. Howerer, in your paper, the detail of encoder structure didn't show how to translate the feature map(eg, 102444) to the GH-Feat(512*1). Can you provide it? Thank you very much!

Is it possible to train the encoder and generator together?

Hello, dear authors, thank you for your great work. I have read several of your papers and know that your team has very good insight on generative models. So I would like to ask several questions:

I think this work and Pixel2style2pixel are somewhat similar in proposing a hierarchical stylegan encoder, however, it's hard for both method to get precisely reconstructed images. Though results looks similar to original images, they are very different pixel-wise, and the SSIM and PSNR are relatively low (compared to other reconstuction tasks like SR, Deblur, De-rain, etc). What do you think is the reason? Is there any method to impore the reconstruction quality?
VAE-GAN combines VAE and GAN, is there any chance to train a stylegan and stylegan encoder in this way, or some similar framework?

genforce / ghfeat Goto Github PK

ghfeat's Introduction

GenForce Lib for Generative Modeling

Updates

Highlights

Installation

Quick Demo

Play with GANs

Test

Train

Play with Encoders for GAN Inversion

Train

Contributors

License

Acknowledgement

BibTex

ghfeat's People

Contributors

Stargazers

Watchers

Forkers

ghfeat's Issues

Recommend Projects

Recommend Topics

Recommend Org