bryandlee / freezeg Goto Github PK

I am trying to run generate.py on your pretrained model 55000.pt and on my finetuned model which is done on your pretrained ffhq model. But I am getting an error.

command: python generate.py --ckpt 550000.pt --size 256

error:
Traceback (most recent call last): File "generate.py", line 47, in <module> g_ema.load_state_dict(checkpoint['g_ema']) File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1045, in load_state_dict self.__class__.__name__, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for Generator: Missing key(s) in state_dict: "noises.noise_0", "noises.noise_1", "noises.noise_2", "noises.noise_3", "noises.noise_4", "noises.noise_5", "noises.noise_6", "noises.noi se_7", "noises.noise_8", "noises.noise_9", "noises.noise_10", "noises.noise_11", "noises.noise_12".

Thank you for your nice work, I try to generate pair data by pretrained model(ffhq model and ffhq2met model are download from #3) by follow steps:

1. get latent vector from picture by project.py:
   python projector.py --ckpt checkpoint/550000.pt ffhq_sample/sample/000000.png

2. generate picture by generate.py and latent vecotor by step 1:
   python generate_pair_data.py --size 256 --ckpt checkpoint/face2met_10k.pt --project_latent_file 000000.pt

generate.py is modified as follows:

import argparse
import os

import torch
from torchvision import utils
from model import Generator
from tqdm import tqdm
import glob
import numpy as np
from PIL import Image

def make_image(tensor):
    return (
        tensor.detach()
            .clamp_(min=-1, max=1)
            .add(1)
            .div_(2)
            .mul(255)
            .type(torch.uint8)
            .permute(0, 2, 3, 1)
            .to("cpu")
            .numpy()
    )

def generate(args, g_ema, device, sample_z_s, mean_latent, sample_noise_s):

    with torch.no_grad():
        g_ema.eval()
        for k,v in tqdm(sample_z_s.items()):
            #sample_z = torch.randn(args.sample, args.latent, device=device)
            

            noise=sample_noise_s[k]
            sample, _ = g_ema(
                [v], truncation=args.truncation, truncation_latent=mean_latent, input_is_latent=True, noise=noise
            )

           
            utils.save_image(
                sample,
                f"metface_dir/{os.path.basename(k)}.png",
                nrow=1,
                normalize=True,
                range=(-1, 1),
            )




if __name__ == "__main__":
    device = "cuda"

    parser = argparse.ArgumentParser(description="Generate samples from the generator")

    parser.add_argument(
        "--size", type=int, default=1024, help="output image size of the generator"
    )
    parser.add_argument(
        "--sample",
        type=int,
        default=1,
        help="number of samples to be generated for each image",
    )
    parser.add_argument(
        "--pics", type=int, default=20, help="number of images to be generated"
    )
    parser.add_argument("--truncation", type=float, default=1, help="truncation ratio")
    parser.add_argument(
        "--truncation_mean",
        type=int,
        default=4096,
        help="number of vectors to calculate mean for the truncation",
    )
    parser.add_argument(
        "--ckpt",
        type=str,
        default="stylegan2-ffhq-config-f.pt",
        help="path to the model checkpoint",
    )
    parser.add_argument(
        "--channel_multiplier",
        type=int,
        default=2,
        help="channel multiplier of the generator. config-f = 2, else = 1",
    )
    parser.add_argument(
        "--project_latent_file",
        type=str,
        default="000000.pt",
        help="path to the latent file",)
    args = parser.parse_args()

    args.latent = 512
    args.n_mlp = 8

    #g_ema = Generator(
    #    args.size, args.latent, args.n_mlp, channel_multiplier=args.channel_multiplier
    #).to(device)
    g_ema = Generator(
        args.size, args.latent, args.n_mlp).to(device)
    checkpoint = torch.load(args.ckpt)

    g_ema.load_state_dict(checkpoint["g_ema"], strict=False)

    if args.truncation < 1:
        with torch.no_grad():
            mean_latent = g_ema.mean_latent(args.truncation_mean)
    else:
        mean_latent = None

    sample_z_s = {}
    sample_noise_s = {}
    result_file = torch.load(args.project_latent_file)
    for k,v in result_file.items():
        sample_z_s[k] = torch.unsqueeze(v["latent"], 0)
        sample_noise_s[k] = v["noise"]


    generate(args, g_ema, device, sample_z_s, mean_latent, sample_noise_s)

but the result i get is diffrent from your face2art samples:

what is my problem? thank you.

Could you share the exact hyperparameters for Face2Simpson model?

Hello and great work!

I have a few questions regarding a gif you posted https://github.com/bryandlee/FreezeG/blob/master/imgs/face2art/2.gif and I hope you can help me with it.

I would like to do the same transfer of real face to art on my own images. Am I right to assume that you are using 2 stylegan2 models trained on FFHQ and Metfaces. Then you generated latent vectors using ffhq model and then generate the art image using this latent vectors and the model trained on Metfaces?

Also, at the end of the gif there is a constant image (of a boy) and the style changes. May I know which eigenvector (vectors) are you using for this?

Thanks!

I have converted stylegan2-ffhq-config-f.pkl from official repo to stylegan2-ffhq-config-f.pt using convert_weight.py, then converted my dataset with prepare_data.py.

After that I am running this command: python train.py --finetune_loc 2 --ckpt stylegan2-ffhq-config-f.pt ./data_processed/ to finetune on my dataset but getting error:

load model: stylegan2-ffhq-config-f.pt Traceback (most recent call last): File "train.py", line 439, in <module> generator.load_state_dict(ckpt["g"], strict=False) KeyError: 'g'

The results look very ambitious! When do you plan to publish a paper or open share code, presenting them?
Thanks!

Thanks for the nice experiments. I was wondering if you plan to release the dataset for cropped simpsons/malnyun faces. And how many images did you use for the simpsons experiment?

Hello!

Could you share pretrained model for Face2Simpsons?

Thank you!

Thanks for your work, simple but effective.
Freezing D&G both work, I wonder which one is a better way, or can we freeze simultaneously both G&D ?
Another question is about manipulating the latent, is that mean we only pass the edited latent to high level layers of StyleGAN and keep the original latent in low level layers?

Thanks a lot for your work, it is fantastic!

Could you please share the pre-trained models?

Many Thanks!!!

Hi! Really nice work, the examples are great. I was wondering if you had listed the training configuration somewhere, specifically which resolution layers your froze? I'm also wondering if you did any experiments with freezing different subsets of the modules within each conv?

For context I worked on the "post training" version of this method, i.e. swapping the trained layers form one model into another, see: https://arxiv.org/abs/2010.05334 When I was doing this work I thought that doing something like your freezeG approach might be a more direct way of getting the same effect. I tried a couple of runs myself at the time, but could never get good results (the network failed to learn to generate good images). Curious if you have any insight of things you tried which didn't work?

Thank you for your great work.
After I read description in ReamMe and codes, I got confuse.

updae: Sorry, I get mistake in the pervious post. The 32 in convs.15 's weight ([1, 32, 32, 3, 3]) is channel.
So you train large feature map resolution of generator, right?

==========
Did you freeze generator layers in 1616, 3232 resolution, etc.?
At first I thought you freeze large feature map resolution weights. However, from the code requires_grad(generator, True, target_layer=f'convs.{generator.num_layers-2-2*loc}'), your turn on grade in convs.15.
convs.15 's weight is [1, 32, 32, 3, 3]. It means you train low feature map resolution weight in generator?
(You didn't train in large feature map resolution, such as, 256, 512).
Is it correct?

When 'finetune_loc' > 0, mapping layer (mlp) is also fixed, right?