hideunderbush / ui2i_via_stylegan2 Goto Github PK

hi, Appreciate for sharing the implementation! There is a question that the style code of the specified reference is not used in gen_ref.py. Noise is used to generate ref and identity. Could you please to fix it ?
Thanks!

Traceback (most recent call last):
File "convert_weight.py", line 235, in
with open(args.path, "rb") as f:
PermissionError: [Errno 13] Permission denied: 'D:/converted'

I set up environment correctly in conda, python 3.6 and tf=1.14 torch with cpp extension and after defeating all the errors i got this one. Any idea what this might be? :) thanks ia

How do you specify a picture for testing?

Hi @HideUnderBush, thanks for your great work, and when I try to use it, I encounter some problems.
When I use python closed_from _factorization.py, I got a RuntimeError: no default TensorFlow session found. please call dnnlib.init_tf(), then I do like it says, call dnnlin.init_tf() before tf.get_default_session(), but got no module error as above.
Hope you can help, thanks~

Thanks for your magnificent research!
And I wonder if I can get your Anime dataset since all of my dataset can not give me the reasonable result.

BUG:
Line 104 and Line 107 should add parameter input_is_latent=True
or the content and the reference cannot be properly used.

ISSUE:
Also truncation=0.5 can sometimes be too strict, making the output's content and style not matched with the input content image and the reference image.

Hi!

According to your paper, it takes about a second to invert an image to its latent representation: "...and another 0.8 − 1 s for the inversion process". However, in your current implementation it is an iterative optimization process, which takes more than a minute with the default settings. Could you, please, clarify how do you successfully accomplish the inversion task within 1 second?

I’ve been digging through GitHub for help on g_ema tweaking from generator

I have this ticket - danielroich/PTI#26

The maths is a bit beyond me / but I suspect I need to update g_ema like you’ve done here.
I need to play around with this repo to investigate further.

@HideUnderBush, I got this error when run closed_form_factorization.py

Setting up TensorFlow plugin "fused_bias_act.cu": Preprocessing... Loading... Done.
Setting up TensorFlow plugin "upfirdn_2d.cu": Preprocessing... Loading... Done.
Traceback (most recent call last):
File "closed_form_factorization.py", line 14, in
ckpt = torch.load(args.ckpt, map_location='cuda:0')
File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 595, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 766, in _legacy_load
raise RuntimeError("Invalid magic number; corrupt file?")
RuntimeError: Invalid magic number; corrupt file?

Thanks for your excellent work, but it seems that when finetuning the model on a new domain data, the mapping net(8mlp) was not frozen, which conflicts with your papers, though requires_grad==False was set in L422-425 of train.py. The gradient is activated in L229 again and the G_optimizer optimize all parameters of G. When I print the parameters of 8MLP on the original model and finetuned, they indeed different.

Hi, @HideUnderBush! Thanks for you amazing works!
I try to reimplement the face2anime experiments on Danbooru Datasets. However, I face some confusions, could you give me some advice?
Step 1: According to your scripts, I use the 512 px stylegan2 ckpt pretrained on ffhq datasets as base, and finetune on Danbooru Datasets. (I didn't change any other params, is that right?)
Step 2: I use closed_form_factorization.py to decompose the model has trained 35000 iterations (35000.pt) to get factor.out file.
Step 3: I try to achieve image inversion (size is 512), however, when the optimization program finished, I got an almost black result. The MSE loss is very large. (The loss is about 1.4-1.7).
Are there any key points I forgot? I wish you can point out some mistakes about my steps. Thanks for your jobs!

Thank you for your amazing work. I am a little confused about the layer swap part in your implementation. It seems that you first pass the latent code into the base model and then extract the intermediate results for the target model as the following.

img1, swap_res = g_ema1([input_latent], input_is_latent=True, save_for_swap=True, swap_layer=args.swap_layer)

for i in range(args.stylenum):
    sample_z_style = torch.randn(1, 512, device=args.device)
    img_style, _ = g_ema2([input_latent], truncation=0.5, truncation_latent=mean_latent, swap=True, swap_layer=args.swap_layer,  swap_tensor=swap_res, multi_style=True, multi_style_latent=[sample_z_style])
    print(i)
    img_style_name = args.output + "_style_" + str(i) + ".png"
    img_style = make_image(img_style)
    out_style = Image.fromarray(img_style[0])
    out_style.save(img_style_name)```

Is it true that you are trying to keep the low level information such as shape and pose from original model and put the lightening and texture from the target model? 

Hi just stumbled across this and it looks great, particularly the anime generation images. Looks like you're essentially using the method I described in some of my blog posts around transfer learning, using one latent code from one model in another and layer swapping. (https://.www.justinpinkney.com) and I'm glad to see you cite Doron and I in for our Toonify work!

We actually have a paper on arxiv that descirbes this approach, particularly focussing on the idea of layer swapping you're using
It would be really great if you could cite our actual paper: Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domains

Perhaps as prior work where you are describing the "layer swapping" you perform?

Hi @HideUnderBush, after downloading the 550000.pt and try to convert the image that you provide, I got a result like below

Hi, can you please add a google colab?