Official implementation of the paper "Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis" in ICLR 2021
License: GNU General Public License v3.0
When I run the eval.py, there is something wrong with exit code 1:KeyError: 'init.init.0.weight_orig'
In the eval.py code, why you annotated the net.ig.eval() ? So you didn't use net_ig.eval(). But when we do the test, we would load the model and then do the model.eval().
what is 0.8 in def train_d? How did you choose the number?
Line 39 in 902a473
Hello!
Thanks for your beautiful work.
I noticed that in your network, there are BatchNorms in the whole network. So I confused why you do not use the InstanceNorm since InstanceNorm is better than BN in the image synthesis task?
The code works on a single gpu, but I have two gpus. When I set "multi_gpu = True" in train.py, it will report an error like this:
File "train.py", line 205, in
train(args)
File "train.py", line 149, in train
err_dr, rec_img_all, rec_img_small, rec_img_part = train_d(netD, real_image, label="real")
File "train.py", line 55, in train_d
pred, [rec_all, rec_small, rec_part], part = net(data, label)
File "/home/cskuan/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/cskuan/anaconda3/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 156, in forward
return self.gather(outputs, self.output_device)
File "/home/cskuan/anaconda3/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 168, in gather
return gather(outputs, output_device, dim=self.dim)
File "/home/cskuan/anaconda3/lib/python3.8/site-packages/torch/nn/parallel/scatter_gather.py", line 68, in gather
res = gather_map(outputs)
File "/home/cskuan/anaconda3/lib/python3.8/site-packages/torch/nn/parallel/scatter_gather.py", line 63, in gather_map
return type(out)(map(gather_map, zip(*outputs)))
File "/home/cskuan/anaconda3/lib/python3.8/site-packages/torch/nn/parallel/scatter_gather.py", line 63, in gather_map
return type(out)(map(gather_map, zip(*outputs)))
TypeError: 'int' object is not iterable
I am trying to use the code in this repo and the first part work:
python train.py --path ../imgs/ --name fast --iter 500 --batch_size 2 --im_size 1024
Setting up Perceptual loss...
Loading model from: /data/FastGAN-pytorch/lpips/weights/v0.1/vgg.pth
...[net-lin [vgg]] initialized
...Done
Namespace(batch_size=2, ckpt='None', cuda=0, im_size=1024, iter=500, name='fast', path='../imgs/', start_iter=0)
GAN: loss d: 2.85532 loss g: -1.84026
100%| .... | 501/501 [03:16<00:00, 2.55it/s]
This generates the following:
ls train_results/fast/
args.txt diffaug.py eval.py images models models.py operation.py __pycache__ train.py
I am having trouble when trying to use eval.py
$ python eval.py --n_sample 10
Traceback (most recent call last):
File "eval.py", line 66, in <module>
checkpoint = torch.load(ckpt, map_location=lambda a,b: a)
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py", line 525, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py", line 212, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py", line 193, in __init__
super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: './models/60000.pth'
Is there a particular reason it is looking for 60000.pth?
How could FastGAN generate pickle .pkl files instead of .pth files.
Wouldn't there be more compatibility with stylegan2-ada? Or am I missing something? Could you convert this. or change in manually in the code?
It is stated in the repo that on some datasets this GAN does not converge. Could you please provide some information about what those datasets are?
How can I compute the Lpips with the code in LPIPS folder？THX.
Thanks for your work!
In the paper, you stated that reconstruction loss is just MSE, but later in the paper and in the code, you use LPIPS. So was the model trained with LPIPS instead of MSE?
Newer versions of skiimage cannot be run because the functions for calculating ssim have been changed.
ImportError: cannot import name 'compare_ssim' from 'skimage.measure' (/usr/local/var/pyenv/versions/image_generator/lib/python3.8/site-packages/skimage/measure/__init__.py)
Hi. Thanks for your excellent research work.
I have a question why the discriminator in the code uses skip-layer channel-wise excitation, but not in the paper?
feat_16 = self.se_2_16(feat_2, feat_16)
feat_32 = self.se_4_32(feat_4, feat_32)
feat_last = self.se_8_64(feat_8, feat_last)
Whenever I'm trying to run this, I'm getting the error
Traceback (most recent call last):
File "C:\Users\SAMSUNG\FastGAN-pytorch\train.py", line 18, in <module>
percept = lpips.PerceptualLoss(model='net-lin', net='vgg', use_gpu=True)
File "C:\Users\SAMSUNG\FastGAN-pytorch\lpips\__init__.py", line 22, in __init__
self.model.initialize(model=model, net=net, use_gpu=use_gpu, colorspace=colorspace, spatial=self.spatial, gpu_ids=gpu_ids)
File "C:\Users\SAMSUNG\FastGAN-pytorch\lpips\dist_model.py", line 99, in initialize
self.net.to(gpu_ids[0])
File "C:\Users\SAMSUNG\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 673, in to
return self._apply(convert)
File "C:\Users\SAMSUNG\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 387, in _apply
module._apply(fn)
File "C:\Users\SAMSUNG\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 430, in _apply
self._buffers[key] = fn(buf)
File "C:\Users\SAMSUNG\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 671, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
File "C:\Users\SAMSUNG\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\cuda\__init__.py", line 170, in _lazy_init
torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
which, I think, is because I don't have a NVIDIA graphic card.
So it does not support CPU only runs?
I mean this seems to be reasonably lighter than other styleGAN related stuffs.
1> If the dog face dataset is used the program works and trains. However if the anime face dataset or other larger datasets are used the error below occurs
ml1@ml1:~/home/ml2FastGAN-pytorch$ python train.py --name test1 --path /home/ml1/datasets/fast-gan-datasets/anime-face/
/home/ml1/anaconda3/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.2
warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
Setting up Perceptual loss...
/home/ml1/anaconda3/lib/python3.9/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.
warnings.warn(
/home/ml1/anaconda3/lib/python3.9/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Loading model from: /home/ml1/home/ml2FastGAN-pytorch/lpips/weights/v0.1/vgg.pth
...[net-lin [vgg]] initialized
...Done
Namespace(path='/home/ml1/datasets/fast-gan-datasets/anime-face/', cuda=0, name='test1', iter=50000, start_iter=0, batch_size=8, im_size=1024, ckpt='None')
Traceback (most recent call last):
File "/home/ml1/home/ml2FastGAN-pytorch/train.py", line 202, in <module>
train(args)
File "/home/ml1/home/ml2FastGAN-pytorch/train.py", line 92, in train
dataloader = iter(DataLoader(dataset, batch_size=batch_size, shuffle=False,
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 444, in __iter__
return self._get_iterator()
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 390, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1115, in __init__
self._reset(loader, first_iter=True)
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1148, in _reset
self._try_put_index()
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1382, in _try_put_index
index = self._next_index()
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 671, in _next_index
return next(self._sampler_iter) # may raise StopIteration
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/utils/data/sampler.py", line 247, in __iter__
for idx in self.sampler:
File "/home/ml1/home/ml2FastGAN-pytorch/operation.py", line 16, in InfiniteSampler
yield order[i]
IndexError: index -1 is out of bounds for axis 0 with size 0
Also
2> Needs upgrade for
/home/ml1/anaconda3/lib/python3.9/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.
/home/ml1/anaconda3/lib/python3.9/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
What's the difference of optimizerG.state_dict() and netG.state_dict(), thank you！
Thank you for sharing your code; I really appreciate the clarity and simplicity in your implementation.
The "Nature Photograph" dataset which you use in the paper is not in provided few-shot-dataset.zip. Are there any copyright issues with the data, or could you also share it? If not, could you provide instructions on how to get the data? This would be great for reproducibility/comparison.
Thanks!
I found a slight problem with DiffAug and LPIPS.
PerceptualLoss expects images between [-1, +1] but by using DiffAug with the default color policy we get values below -1 and /or above +1. This affects only the second argument of percept(x, y) because the first argument comes from the neural network with tanh.
This can be verified by inserting after line 40 in train.py some print statements:
print("rec_all", rec_all.min(), rec_all.max()) # -0.9973, 0.9963
print("rec_small", rec_small.min(), rec_small.max()) # -0.9997, 0.9999
print("rec_part", rec_part.min(), rec_part.max()) # -1.0000, 0.9998
print("data", data.min(), data.max()) # -1.3135, 0.6860 -----> WRONG
Only a few of the datasets presented in the paper have pre-trained models in google drive! Any plan to release the other ones or at least release the hyperparameters?
in the calc_inception.py, there is something wrong with from beninception import InceptionV3,:
Traceback (most recent call last):
File "D:/FastGAN-pytorch-main/benchmarking/calc_inception.py", line 14, in
from beninception import InceptionV3
ModuleNotFoundError: No module named 'beninception'
Process finished with exit code 1
Hi,
Thank you for making your code public.
python eval.py --n_sample 100 --dist pretrained_models/
Traceback (most recent call last):
File "eval.py", line 68, in
net_ig.load_state_dict(checkpoint['g'])
RuntimeError: Error(s) in loading state_dict for Generator:
Missing key(s) in state_dict: "feat_1024.1.weight_orig", "feat_1024.1.weight", "feat_1024.1.weight_u", "feat_1024.1.weight_orig", "feat_1024.1.weight_u", "feat_1024.1.weight_v", "feat_1024.2.weight", "feat_1024.2.bias", "feat_1024.2.running_mean", "feat_1024.2.running_var".
size mismatch for to_big.weight_orig: copying a param with shape torch.Size([3, 16, 3, 3]) from checkpoint, the shape in current model is torch.Size([3, 8, 3, 3]).
size mismatch for to_big.weight_v: copying a param with shape torch.Size([144]) from checkpoint, the shape in current model is torch.Size([72]).
File "FastGAN-pytorch/operation.py", line 16, in InfiniteSampler
yield order[i]
IndexError: index -1 is out of bounds for axis 0 with size 0
Thank you for your help.
There are about 20 images in the training set, and the number of iterations is set to 50000. Most of the output result images are distorted. If I want to improve the quality of the images in the results, may I ask from what aspects can I consider this problem?
Hi. Thanks for your excellent research work.
I have a question why training discriminator uses decoder_small, in addition to decoder_big and decoder_part?
In my opinion, reconstruction loss does not use rec_img_small accroding to paper. Maybe I missed important information.
rec_img_big = self.decoder_big(feat_last)
rec_img_small = self.decoder_small(feat_small)
rec_img_part = self.decoder_part(feat_32[:,:,:8,:8])
Hi,
This is a great and neat implementation. I was wondering if there is a simple way to implement conditioning (on image label) like a conditional GAN. I did try the standard approach used in cGAN but it was not successful. Could you please point me in the right direction?
Cheers,
I think the err_dr is just the loss of discriminator receiving real samples.
I think the err_dr is just the loss of discriminator receiving real samples. We should also add the loss of discriminator receiving fake samples. err_ discriminator_total=err_ real_samples+err_ fake_samples. Thank you!
Dear odegeasslbc, how to continue the training?
For example, yesterday I had stopped the training at iter 25000.
And today I wan to continue the training by following commands:
python train.py --name myImages --path "/myImages" --batch_size 16 --start_iter 25001
and it looks that "--start_iter 25001“ doesn't work. it still start the training from iter 0.
thanks.
Hi @odegeasslbc , Thank you so much for your awesome work.
Can you give some insights into why using the percept.sum() instead of percept.mean() ?
Thanks!
FileNotFoundError: [Errno 2] No such file or directory: './models/60000.pth'
For both
python hon eval.py --n_sample 1
and
python eval.py --n_sample 100 --start_iter 0 --end_iter 50000
Traceback (most recent call last):
File "/home/ml1/FastGAN-pytorch/train_results/test1/eval.py", line 67, in <module>
checkpoint = torch.load(ckpt, map_location=lambda a,b: a)
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 699, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/ml1/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 211, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: './models/60000.pth'
Appears to be a bug. No range check?
Just playing with this at the moment, but noticed that the easing functions referenced in generate_video.py don't appear to be included?
from easing_functions.easing import LinearInOut
Thanks!
I have a question w.r.t SEblcok.
Do you have any specific reason why bias option is set to "False" in conv layer of SEblock?
I know that the FastGAN and MobileNet also have the same option.
Thank you.
no problem...
Hi, my training has finished but I got an error though I followed your instructions.
I'm using MistGPU( a GPU server like Colab ).
--ckpt option unable to load trained models
RuntimeError: Error(s) in loading state_dict for DataParallel:
Missing key(s) in state_dict: "module.init.init.0.weight_orig", "module.init.init.0.weight", "module.init.init.0.weight_u", "module.init.init.0.weight_orig", "module.init.init.0.weight_u", "module.init.init.0.weight_v", "module.init.init.1.weight", "module.init.init.1.bias", "module.init.init.1.running_mean", "module.init.init.1.running_var", "module.feat_8.1.weight_orig", "module.feat_8.1.weight", "module.feat_8.1.weight_u", "module.feat_8.1.weight_orig", "module.feat_8.1.weight_u", "module.feat_8.1.weight_v", "module.feat_8.2.weight", "module.feat_8.3.weight", "module.feat_8.3.bias", "module.feat_8.3.running_mean", "module.feat_8.3.running_var", "module.feat_8.5.weight_orig", "module.feat_8.5.weight", "module.feat_8.5.weight_u", "module.feat_8.5.weight_orig", "module.feat_8.5.weight_u", "module.feat_8.5.weight_v", "module.feat_8.6.weight", "module.feat_8.7.weight", "module.feat_8.7.bias", "module.feat_8.7.running_mean", "module.feat_8.7.running_var", "module.feat_16.1.weight_orig", "module.feat_16.1.weight", "module.feat_16.1.weight_u", "module.feat_16.1.weight_orig", "module.feat_16.1.weight_u", "module.feat_16.1.weight_v", "module.feat_16.2.weight", "module.feat_16.2.bias", "module.feat_16.2.running_mean", "module.feat_16.2.running_var", "module.feat_32.1.weight_orig", "module.feat_32.1.weight", "module.feat_32.1.weight_u", "module.feat_32.1.weight_orig", "module.feat_32.1.weight_u", "module.feat_32.1.weight_v", "module.feat_32.2.weight", "module.feat_32.3.weight", "module.feat_32.3.bias", "module.feat_32.3.running_mean", "module.feat_32.3.running_var", "module.feat_32.5.weight_orig", "module.feat_32.5.weight", "module.feat_32.5.weight_u", "module.feat_32.5.weight_orig", "module.feat_32.5.weight_u", "module.feat_32.5.weight_v", "module.feat_32.6.weight", "module.feat_32.7.weight", "module.feat_32.7.bias", "module.feat_32.7.running_mean", "module.feat_32.7.running_var", "module.feat_64.1.weight_orig", "module.feat_64.1.weight", "module.feat_64.1.weight_u", "module.feat_64.1.weight_orig", "module.feat_64.1.weight_u", "module.feat_64.1.weight_v", "module.feat_64.2.weight", "module.feat_64.2.bias", "module.feat_64.2.running_mean", "module.feat_64.2.running_var", "module.feat_128.1.weight_orig", "module.feat_128.1.weight", "module.feat_128.1.weight_u", "module.feat_128.1.weight_orig", "module.feat_128.1.weight_u", "module.feat_128.1.weight_v", "module.feat_128.2.weight", "module.feat_128.3.weight", "module.feat_128.3.bias", "module.feat_128.3.running_mean", "module.feat_128.3.running_var", "module.feat_128.5.weight_orig", "module.feat_128.5.weight", "module.feat_128.5.weight_u", "module.feat_128.5.weight_orig", "module.feat_128.5.weight_u", "module.feat_128.5.weight_v", "module.feat_128.6.weight", "module.feat_128.7.weight", "module.feat_128.7.bias", "module.feat_128.7.running_mean", "module.feat_128.7.running_var", "module.feat_256.1.weight_orig", "module.feat_256.1.weight", "module.feat_256.1.weight_u", "module.feat_256.1.weight_orig", "module.feat_256.1.weight_u", "module.feat_256.1.weight_v", "module.feat_256.2.weight", "module.feat_256.2.bias", "module.feat_256.2.running_mean", "module.feat_256.2.running_var", "module.se_64.main.1.weight_orig", "module.se_64.main.1.weight", "module.se_64.main.1.weight_u", "module.se_64.main.1.weight_orig", "module.se_64.main.1.weight_u", "module.se_64.main.1.weight_v", "module.se_64.main.3.weight_orig", "module.se_64.main.3.weight", "module.se_64.main.3.weight_u", "module.se_64.main.3.weight_orig", "module.se_64.main.3.weight_u", "module.se_64.main.3.weight_v", "module.se_128.main.1.weight_orig", "module.se_128.main.1.weight", "module.se_128.main.1.weight_u", "module.se_128.main.1.weight_orig", "module.se_128.main.1.weight_u", "module.se_128.main.1.weight_v", "module.se_128.main.3.weight_orig", "module.se_128.main.3.weight", "module.se_128.main.3.weight_u", "module.se_128.main.3.weight_orig", "module.se_128.main.3.weight_u", "module.se_128.main.3.weight_v", "module.se_256.main.1.weight_orig", "module.se_256.main.1.weight", "module.se_256.main.1.weight_u", "module.se_256.main.1.weight_orig", "module.se_256.main.1.weight_u", "module.se_256.main.1.weight_v", "module.se_256.main.3.weight_orig", "module.se_256.main.3.weight", "module.se_256.main.3.weight_u", "module.se_256.main.3.weight_orig", "module.se_256.main.3.weight_u", "module.se_256.main.3.weight_v", "module.to_128.weight_orig", "module.to_128.weight", "module.to_128.weight_u", "module.to_128.weight_orig", "module.to_128.weight_u", "module.to_128.weight_v", "module.to_big.weight_orig", "module.to_big.weight", "module.to_big.weight_u", "module.to_big.weight_orig", "module.to_big.weight_u", "module.to_big.weight_v", "module.feat_512.1.weight_orig", "module.feat_512.1.weight", "module.feat_512.1.weight_u", "module.feat_512.1.weight_orig", "module.feat_512.1.weight_u", "module.feat_512.1.weight_v", "module.feat_512.2.weight", "module.feat_512.3.weight", "module.feat_512.3.bias", "module.feat_512.3.running_mean", "module.feat_512.3.running_var", "module.feat_512.5.weight_orig", "module.feat_512.5.weight", "module.feat_512.5.weight_u", "module.feat_512.5.weight_orig", "module.feat_512.5.weight_u", "module.feat_512.5.weight_v", "module.feat_512.6.weight", "module.feat_512.7.weight", "module.feat_512.7.bias", "module.feat_512.7.running_mean", "module.feat_512.7.running_var", "module.se_512.main.1.weight_orig", "module.se_512.main.1.weight", "module.se_512.main.1.weight_u", "module.se_512.main.1.weight_orig", "module.se_512.main.1.weight_u", "module.se_512.main.1.weight_v", "module.se_512.main.3.weight_orig", "module.se_512.main.3.weight", "module.se_512.main.3.weight_u", "module.se_512.main.3.weight_orig", "module.se_512.main.3.weight_u", "module.se_512.main.3.weight_v", "module.feat_1024.1.weight_orig", "module.feat_1024.1.weight", "module.feat_1024.1.weight_u", "module.feat_1024.1.weight_orig", "module.feat_1024.1.weight_u", "module.feat_1024.1.weight_v", "module.feat_1024.2.weight", "module.feat_1024.2.bias", "module.feat_1024.2.running_mean", "module.feat_1024.2.running_var".
Unexpected key(s) in state_dict: "init.init.0.weight_orig", "init.init.0.weight_u", "init.init.0.weight_v", "init.init.1.weight", "init.init.1.bias", "init.init.1.running_mean", "init.init.1.running_var", "init.init.1.num_batches_tracked", "feat_8.1.weight_orig", "feat_8.1.weight_u", "feat_8.1.weight_v", "feat_8.2.weight", "feat_8.3.weight", "feat_8.3.bias", "feat_8.3.running_mean", "feat_8.3.running_var", "feat_8.3.num_batches_tracked", "feat_8.5.weight_orig", "feat_8.5.weight_u", "feat_8.5.weight_v", "feat_8.6.weight", "feat_8.7.weight", "feat_8.7.bias", "feat_8.7.running_mean", "feat_8.7.running_var", "feat_8.7.num_batches_tracked", "feat_16.1.weight_orig", "feat_16.1.weight_u", "feat_16.1.weight_v", "feat_16.2.weight", "feat_16.2.bias", "feat_16.2.running_mean", "feat_16.2.running_var", "feat_16.2.num_batches_tracked", "feat_32.1.weight_orig", "feat_32.1.weight_u", "feat_32.1.weight_v", "feat_32.2.weight", "feat_32.3.weight", "feat_32.3.bias", "feat_32.3.running_mean", "feat_32.3.running_var", "feat_32.3.num_batches_tracked", "feat_32.5.weight_orig", "feat_32.5.weight_u", "feat_32.5.weight_v", "feat_32.6.weight", "feat_32.7.weight", "feat_32.7.bias", "feat_32.7.running_mean", "feat_32.7.running_var", "feat_32.7.num_batches_tracked", "feat_64.1.weight_orig", "feat_64.1.weight_u", "feat_64.1.weight_v", "feat_64.2.weight", "feat_64.2.bias", "feat_64.2.running_mean", "feat_64.2.running_var", "feat_64.2.num_batches_tracked", "feat_128.1.weight_orig", "feat_128.1.weight_u", "feat_128.1.weight_v", "feat_128.2.weight", "feat_128.3.weight", "feat_128.3.bias", "feat_128.3.running_mean", "feat_128.3.running_var", "feat_128.3.num_batches_tracked", "feat_128.5.weight_orig", "feat_128.5.weight_u", "feat_128.5.weight_v", "feat_128.6.weight", "feat_128.7.weight", "feat_128.7.bias", "feat_128.7.running_mean", "feat_128.7.running_var", "feat_128.7.num_batches_tracked", "feat_256.1.weight_orig", "feat_256.1.weight_u", "feat_256.1.weight_v", "feat_256.2.weight", "feat_256.2.bias", "feat_256.2.running_mean", "feat_256.2.running_var", "feat_256.2.num_batches_tracked", "se_64.main.1.weight_orig", "se_64.main.1.weight_u", "se_64.main.1.weight_v", "se_64.main.3.weight_orig", "se_64.main.3.weight_u", "se_64.main.3.weight_v", "se_128.main.1.weight_orig", "se_128.main.1.weight_u", "se_128.main.1.weight_v", "se_128.main.3.weight_orig", "se_128.main.3.weight_u", "se_128.main.3.weight_v", "se_256.main.1.weight_orig", "se_256.main.1.weight_u", "se_256.main.1.weight_v", "se_256.main.3.weight_orig", "se_256.main.3.weight_u", "se_256.main.3.weight_v", "to_128.weight_orig", "to_128.weight_u", "to_128.weight_v", "to_big.weight_orig", "to_big.weight_u", "to_big.weight_v", "feat_512.1.weight_orig", "feat_512.1.weight_u", "feat_512.1.weight_v", "feat_512.2.weight", "feat_512.3.weight", "feat_512.3.bias", "feat_512.3.running_mean", "feat_512.3.running_var", "feat_512.3.num_batches_tracked", "feat_512.5.weight_orig", "feat_512.5.weight_u", "feat_512.5.weight_v", "feat_512.6.weight", "feat_512.7.weight", "feat_512.7.bias", "feat_512.7.running_mean", "feat_512.7.running_var", "feat_512.7.num_batches_tracked", "se_512.main.1.weight_orig", "se_512.main.1.weight_u", "se_512.main.1.weight_v", "se_512.main.3.weight_orig", "se_512.main.3.weight_u", "se_512.main.3.weight_v", "feat_1024.1.weight_orig", "feat_1024.1.weight_u", "feat_1024.1.weight_v", "feat_1024.2.weight", "feat_1024.2.bias", "feat_1024.2.running_mean", "feat_1024.2.running_var", "feat_1024.2.num_batches_tracked".
how to load trained model?
According to the formula of your loss function, the loss of G is the opposite of the score given by D. Since D will only give a positive number, why does the loss of G need to be added with a negative sign?
What makes me even more strange is that when the G loss is printed in the code, a negative sign is added, but when I run it, the output is a negative number, indicating that the loss of G should be a positive number, but according to the formula, it cannot be a positive number.
if iteration % 100 == 0: print("GAN: loss d: %.5f loss g: %.5f " %(err_dr, -err_g.item()))
Hello,
First of all, congrats for your nice work.
Just to know what is the meaning of the saved models. I have trained with 50000 iterations and for each 10000 iterations a model is saved. Hence, we have 50000.pth and all_50000.pth. Whats is the difference between them when generating new fake samples?
Best Regards.
According to the download address of the datasets you provided, there was an error when decompressing the dataset after downloading it. Is there any other way to share the dataset? Thank you very much!
Why is the loss D for fake images min(0,-1-d(x_fake)) not min(0,-d(x_fake))?
I feel like min(0,-1-d(x_fake)) can try to converge to -1 when the D is fed fake images.
I thought the ideal output of D from real images to be 1 and from fake images to be 0.
When running this command python train.py --path=C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\custom-images --im_size=128 --name=voice
This is the output:
Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done Namespace(batch_size=8, ckpt='None', cuda=0, im_size=128, iter=50000, name='voice', path='C:\\Users\\Graham\\Desktop\\ICSCapstone\\FastGAN-pytorch-main\\custom-images', start_iter=0) Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done Setting up Perceptual loss... Loading model from: C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\lpips\weights\v0.1\vgg.pth ...[net-lin [vgg]] initialized ...Done 0%| | 0/50001 [00:02<?, ?it/s] Traceback (most recent call last): File "train.py", line 200, in <module> train(args) File "train.py", line 136, in train fake_images = netG(noise) File "C:\Users\Graham\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "C:\Users\Graham\Desktop\ICSCapstone\FastGAN-pytorch-main\models.py", line 171, in forward feat_512 = self.se_512( feat_32, self.feat_512(feat_256) ) File "C:\Users\Graham\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 779, in __getattr__ type(self).__name__, name)) torch.nn.modules.module.ModuleAttributeError: 'Generator' object has no attribute 'se_512'
Hi. Thanks for your excellent research work.
I have a question why the code use skip-layer channel-wise excitation to extracts feat_64, but the paper does not?
feat_64 = self.se_64( feat_4, self.feat_64(feat_32) )
Hi I am very new to coding and GAN's so please correct or help me out if I am doing something silly...
I Trained FastGAN until 30000.pth
When I go in to my training results and try to
python generate_video.py
I got:
python generate_video.py Traceback (most recent call last): File "generate_video.py", line 182, in <module> net.load_state_dict(torch.load(ckpt_path)['g']) File "/home/paperspace/miniconda3/envs/fastGAN/lib/python3.8/site-packages/torch/serialization.py", line 581, in load with _open_file_like(f, 'rb') as opened_file: File "/home/paperspace/miniconda3/envs/fastGAN/lib/python3.8/site-packages/torch/serialization.py", line 230, in _open_file_like return _open_file(name_or_buffer, mode) File "/home/paperspace/miniconda3/envs/fastGAN/lib/python3.8/site-packages/torch/serialization.py", line 211, in _init_ super(_open_file, self).__init__(open(name, mode)) FileNotFoundError: [Errno 2] No such file or directory: './time_1024_1/models/180000.pth'
I looked inside the code and changed './time_1024_1/models/180000.pth' to my file path.
But then this message came up:
python generate_video.py Traceback (most recent call last): File "generate_video.py", line 182, in <module> net.load_state_dict(torch.load(ckpt_path)['g']) File "/home/paperspace/miniconda3/envs/fastGAN/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Generator: size mismatch for init.init.0.weight_orig: copying a param with shape torch.Size([256, 2048, 4, 4]) from checkpoint, the shape in current model is torch.Size([100, 2048, 4, 4]). size mismatch for init.init.0.weight_v: copying a param with shape torch.Size([4096]) from checkpoint, the shape in current model is torch.Size([1600]).
I would really like to generate a video for a project I am doing. Again I am very new to this, I also not sure how I should find out what commands python generate_video.py needs....
All help would be very much appreciated, have a good day!
Hey, thanks for the code!
I had trouble with making the code run at some places due to package versioning problems. For example, the "compare_ssim" function from the skimage package is from an older version in lpips/__init__.py*. A requirements.txt and the python version that you have used during experiments would be very helpful. Thanks!
* For the people looking for a quick fix with a newer skimage version try:
from skimage.metrics import structural_similarity as compare_ssim
hello！
When I want to calculate fid, then run fid.py appears:
what are --path_a and--path_b in fid.py?Thank you!
when i set multi gpu True, an error occured
Traceback (most recent call last):
File "train.py", line 200, in <module>
train(args)
File "train.py", line 144, in train
err_dr, rec_img_all, rec_img_small, rec_img_part = train_d(netD, real_image, label="real")
File "train.py", line 38, in train_d
pred, [rec_all, rec_small, rec_part], part = net(data, label)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/data_parallel.py", line 162, in forward
return self.gather(outputs, self.output_device)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/data_parallel.py", line 174, in gather
return gather(outputs, output_device, dim=self.dim)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/scatter_gather.py", line 68, in gather
res = gather_map(outputs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/scatter_gather.py", line 63, in gather_map
return type(out)(map(gather_map, zip(*outputs)))
File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/scatter_gather.py", line 63, in gather_map
return type(out)(map(gather_map, zip(*outputs)))
TypeError: zip argument #1 must support iterationHi @odegeasslbc,
Thank you for your amazing work. I have a question about why you commented out the line of taking the model into eval mode. As I remember correctly, the eval mode is not used in test.py of pix2pixHD code.
Is it a good practice for gan models? If not, what is the specific reason of it since I really wonder about it? Also, new random noise is generated for each iteration in eval.py, so the generated images can be different even though model is in eval mode. Thank you for your answer :)
Thank you for your excellent work!
I have some questions about the Generator.
There are two implementation methods for upsampling in the code, "UpBlock" and "UpBlockComp", but the architecture in the paper does not mention the latter. I would like to ask when to use "UpBlock" and when to use "UpBlockComp"
https://github.com/odegeasslbc/FastGAN-pytorch/blob/d34b36642ca30e3c48dac294c5a831c12fc7bb75/models.py#L99C1-L105C17
and
https://github.com/odegeasslbc/FastGAN-pytorch/blob/d34b36642ca30e3c48dac294c5a831c12fc7bb75/models.py#L108C1-L119C17
Hope to get your response.
Hello,
I just have a question. Why are the reconstructed images obtained from the discriminator, while only garbage is obtained for me from the generator? I don't know if this is the intended functionality, but when I look at the image created by the gen, all I see is:
When I look at the outputted reconstruction from the discriminator, however, I see this:
Shouldnt the generator output the reconstruction, not discriminator?
In models.py on lines 168 - 178
if self.im_size == 256:
return [self.to_big(feat_256), self.to_128(feat_128)]
feat_512 = self.se_512( feat_32, self.feat_512(feat_256) )
if self.im_size == 512:
return [self.to_big(feat_512), self.to_128(feat_128)]
feat_1024 = self.feat_1024(feat_512)
im_128 = torch.tanh(self.to_128(feat_128))
im_1024 = torch.tanh(self.to_big(feat_1024))
It looks like when returning an image of size 1024, the output of modules to_128 and to_big are wrapped by a tanh, while for images 256 and 512 they are not. I guess they were all supposed to be wrapped in tanh?
Hi! Thank you for sharing the code of your great work!
Our team is enjoying the performance.
By the way, I wonder about the tanh functions applied only with the netG output of im_size==1024 (but not when im_size==256 or im_size==512.
Line 177 in 0e6ea56
Line 169 in 0e6ea56
I guess that this is not a critical issue but just wanted to let you know.
Running FastGan with latest CUDA and torchvision required for 3090x series card gives the following warnings.
/home/ml1/anaconda3/lib/python3.9/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.
/home/ml1/anaconda3/lib/python3.9/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
I would switch to weights=VGG16_Weights.DEFAULT for latest version or to weights=VGG16_Weights.IMAGENET1K_V1 for a stable fixed version.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.