Giter VIP home page Giter VIP logo

arcanegan's Introduction

ArcaneGAN by Alex Spirin

Photos Colab

Videos Colab for videos

visitors

If you like what I'm doing you can:

Thank you for being awesome!

Changelog

ArcaneGAN v0.4

The main differences are:

  • lighter styling (closer to original input)
  • sharper result
  • happier faces
  • reduced childish eyes effect
  • reduced stubble on feminine faces
  • increased temporal stability on videos
  • reduced mouth\teeth artifacts

Image samples

v0.3 vs v0.4

v3-4

Video samples

ryan.mp4
obama.mp4

ArcaneGAN v0.3

Videos processed by the huggingface video inference colab.

obama2.mp4
ryan2.mp4

Image samples

arcaneganv03

Faces were enhanced via GPEN before applying the ArcaneGAN v0.3 filter.

ArcaneGAN v0.2

The release is here image photo_2021-12-04_08-05-34 photo_2021-12-04_07-23-17 weewq

Implementation Details

The model is a pytroch *.jit of a fastai v1 flavored u-net trained on a paired dataset, generated via a blended stylegan2.

arcanegan's People

Contributors

amrzv avatar sxela avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

arcanegan's Issues

support arbitrary image size?

Great work!

The unet prediction result will be cropped to be the same size as the training input, e.g. 256 or 512.
For arbitrary image size (e.g. 1280*720), how to config or set the model to output the same size of the input image as your colab did?
Thank you.

RuntimeError: CUDA out of memory

Добрый вечер.Извините,это опять я.Снова эта ошибка появляется.Можно ли,самому эту ошибку решать?Или исправлять можете только вы?Обьясните пожалуйста подробно.

How to prevent eyes occur in nose?

Hello, I try your model and it's amazing, but I find in some pictures if the nose is too big, there will be eyes in the nose. I try to lower the 'target_face' and it can work. But the details like the light of the eyes and background will also lose when I lower the 'target_face'. So I wonder is there a way to prevent the eyes occurs in the nose and keep the details in the meantime?
image

Can it run local on my PC

Hi,
i'm really interested in repo and i want to run in my computer instead of hugging face. How can i do it?

How do you change the style of the whole image

Nice work!
My only confusion is how you change the style of the whole image instead of just the face. Usually, StyleGAN generates aligned face images by fine-tuning the FFHQ checkpoint. How does the pix2pix model trained with these face image pairs work with the full image or frame.

Any news for training code?

Interesting topic... I wonder how you trained the model, especially the augmentation part.
Fixed crop limitation is a well-known problem and would like to know how you handle it. :)

How to make the style stronger?

The following are input image, my training output from pair label supervision, and the output from your test model。
I trained my model (Super-Resolution model) on the images from your model outputs, I find it difficult to change the facial features。
Like the eyes and face texture are changed, how to do it ? I use L1Loss (weight is 1) + PerceptualLoss (weight is 1)+ GANLoss (weight is 0.1),

6W2HG4GXC2

tuple issue

Was trying the ArcaneGan video colab but I am having a tuple issue can you please help, i am really excited to try the Arcane video can you please help out

stylegan data

Hi, how many pictures did you use to train stylegan model?

How to convert the FastAI model to Pytorch JIT

Hi,

I trained a model with unet_learner but I can't convert it to jit.

I run the following code:
torch.jit.save(torch.jit.script(learn.model), 'jit.pt')

Here is the error:

UnsupportedNodeError: GeneratorExp aren't supported: File "/usr/local/lib/python3.7/dist-packages/fastai/callbacks/hooks.py", line 21 "Applies hook_functomodule, input, output." if self.detach: input = (o.detach() for o in input ) if is_listy(input ) else input.detach() ~ <--- HERE output = (o.detach() for o in output) if is_listy(output) else output.detach() self.stored = self.hook_func(module, input, output)

May I know how you convert it to a jit model?
Thanks

about the paired datasets generated by stylegan

how do you make sure the background and expression similarity between the generated input(face) and target(style face) ? I find that the style is too weak when less finetune and the similarity is too weak when more finetune, how do you solve it ? Would you like to share the paired datasets generated code with me ? thanks a lot ~

What GPU is used for training?

Hi,

I want to train the Fastai u-net model. However, when I try to train the critic (learn_critic.fit_one_cycle(6, 1e-3)), I get the following error:

CUDA out of memory. Tried to allocate 4.00 GiB (GPU 0; 14.76 GiB total capacity; 9.78 GiB already allocated; 891.75 MiB free; 12.57 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

The GPU is a Tesla T4 with 16 GB of VRAM. My batch size is 4 and the training images size is 512*512. I also tried with lower numbers, but I'm still getting the same error.

Ошибка

Добрый вечер.В ArcaneGAN на colab for videos,выдаёт ошибку:

RuntimeError: CUDA out of memory. Tried to allocate 2.80 GiB (GPU 0; 11.17 GiB total capacity; 5.74 GiB already allocated; 2.21 GiB free; 8.44 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Помогите пожалуйста!

Low resolution?

Is there any way to increase the output resolution on collab, as the current output is quite low.

Thank you!

more Kawaii plz

It's so cool but I'd like to try to generate the outputs with Japanese anime style.
What can I do that?

Architecture for video

Hi, what does the architecture look like?
Is it similar to Pix2Pix?
And for processing of the video, are you doing anything extra to make sure the frames are consistent?

face alignment

Hi, do you make face alignment before you give it to your Unet model?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.