znxlwm / ugatit-pytorch Goto Github PK

I do not understand why SN is used in the output layer of Patch Discriminator while it is not common to use any normalization functions for it in original implementation?

error imfoemation:number of epochs must be larger than or equal to one.

Can you provide me with your trained model, because I don't have enough GPU memory, so I can't train. I just want to test a few pictures to see the effect.

Hi guys!
I'm using RTX 2080Ti 11GB. At first I tried to train on a dataset of 100K images (1000px) with --light flag . And after the 1000th epoch I got the "CUDA out of memory" error. Then I tried a smaller dataset of 10K images (256px) and got the same error after the 1000th epoch. Finally I tried 3400 images (256px) and there were no changes.

Here is an output:

[  997/1000000] time: 582.6236 d_loss: 0.00474171, g_loss: 1344.68078613
[  998/1000000] time: 583.2094 d_loss: 0.00624988, g_loss: 1328.24572754
[  999/1000000] time: 583.7950 d_loss: 0.00641153, g_loss: 1374.71826172
[ 1000/1000000] time: 584.3810 d_loss: 0.00178387, g_loss: 1280.08032227
/home/p0wx/prj/UGATIT-pytorch/utils.py:46: RuntimeWarning: invalid value encountered in true_divide
  cam_img = x / np.max(x)
Traceback (most recent call last):
  File "main.py", line 83, in <module>
    main()
  File "main.py", line 75, in main
    gan.train()
  File "/home/p0wx/prj/UGATIT-pytorch/UGATIT.py", line 209, in train
    fake_B2B, fake_B2B_cam_logit, _ = self.genA2B(real_B)
  File "/home/p0wx/.local/share/virtualenvs/p0wx-AcgHNkMk/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
    result = self.forward(*input, **kwargs)
  File "/home/p0wx/prj/UGATIT-pytorch/networks.py", line 108, in forward
    out = self.UpBlock2(x)
  File "/home/p0wx/.local/share/virtualenvs/p0wx-AcgHNkMk/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
    result = self.forward(*input, **kwargs)
  File "/home/p0wx/.local/share/virtualenvs/p0wx-AcgHNkMk/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/home/p0wx/.local/share/virtualenvs/p0wx-AcgHNkMk/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
    result = self.forward(*input, **kwargs)
  File "/home/p0wx/prj/UGATIT-pytorch/networks.py", line 191, in forward
    out = self.rho.expand(input.shape[0], -1, -1, -1) * out_in + (1-self.rho.expand(input.shape[0], -1, -1, -1)) * out_ln
RuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 10.76 GiB total capacity; 9.32 GiB already allocated; 5.56 MiB free; 621.27 MiB cached)```

I tried to train your complete network under the nvidia GTX 1080Ti with 12G GPU memory.
But pytorch throw out a "out of cuda memory" error.
I just wonder how large gpu memory i have i can train the so big network?

Hi, in the paper, I find $L_{cam}^{D_{t}}$ have a log function, but in source code, only use MSE(UGATIT.py 182-189 lines). So, the paper adds unnecessarily log？

G_cam_loss_A = self.BCE_loss(fake_B2A_cam_logit, torch.ones_like(fake_B2A_cam_logit).to(self.device)) + self.BCE_loss(fake_A2A_cam_logit, torch.zeros_like(fake_A2A_cam_logit).to(self.device))
G_cam_loss_B = self.BCE_loss(fake_A2B_cam_logit, torch.ones_like(fake_A2B_cam_logit).to(self.device)) + self.BCE_loss(fake_B2B_cam_logit, torch.zeros_like(fake_B2B_cam_logit).to(self.device))

As for the generator, what does the cam want to classify?
Classify the real image to 0? Classify the fake image to 1?

As for the discriminator, we can see that the real images will be classified to 1 and the fake images will be classified to 0.
Therefore, I am confusing about the CAM for the generator.

Excuse me, can the training dataset and training model be downloaded? Thank you！

UGATIT 预训练模型和数据集百度云下载，为方便谷歌硬盘无法使用的人，不懂的可加Q群857449786 注明UGATIT 共同研究 ，谢谢！

官方已经开放的数据集和模型在google硬盘，比较大，可能无法下载，我分卷压缩了一下，分享到百度云，方便大家下载。
Pretrained model： selfie2anime checkpoint (100 epoch)
Dataset：selfie2anime dataset

目前win10下的测试训练自己的数据问题，都可以 ，UGATIT 预训练模型和数据集百度云下载，为方便谷歌硬盘无法使用的人，不懂的可加Q群857449786 注明UGATIT 共同研究 ，谢谢！

链接：https://pan.baidu.com/s/1dP1mXuU-rA9dPvFe8YS8jQ
提取码：k6rc

下载后合并01.02.03为一个文件。
目录结构如下：
B:\Tensorflow\UGATIT\checkpoint\UGATIT_selfie2anime_lsgan_4resblock_6dis_1_1_10_10_1000_sn_smoothing

Hi, thank you for sharing source code.

But, compared to tf version, ResNetAdaILNBlock layer doesn't seem to have a shortcut connection which adds the input of layer to output directory.

Hi, thanks for the code.
Did you try with U-net architecture ?

Hi, Thank you for release this project. Why does the training batchsize default to 1? Can it be a larger value?

If I need to train rectangular datasets, how to modify code and set parameters?

Refer to the following figure for details.

with the default config, i got 5.x gen_loss but got 1000.x dis_loss. Can you give me some tweak to balance it

GPU which I have is not enough memory to train with single GPU.

in networks.py 165 torch.mean(input, dim=[2, 3], keepdim=True) , why use list to dim

just like this, which D has a very small loss

Hi, I have noticed that the parameter Rho of AdaILN and ILN layer in pytorch version of UGATIT is not clipped or constrainted compared to your TF version and the function in paper.

Not quite sure what I am writing for the intended purpose. The fact is that I have no practical experience with python and areas devoted to machine learning. Quite by chance I read about your work and decided to deploy the code on my machine (OS Ubuntu 19.04). The test (python3 main.py --dataset selfie2anime --phase test) results in the specified error. A fragment of the console log:

[*] Reading checkpoints ...
W0826 19: 42: 05.169215 140086150489920 deprecation.py:323] From /home/praetorp/.local/lib/python3.7/site-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python .training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
2019-08-26 19: 42: 05.310732: W tensorflow / core / framework / allocator.cc: 107] Allocation of 1073741824 exceeds 10% of system memory.
2019-08-26 19: 42: 05.310732: W tensorflow / core / framework / allocator.cc: 107] Allocation of 1073741824 exceeds 10% of system memory.
2019-08-26 19: 42: 05.311256: W tensorflow / core / framework / op_kernel.cc: 1502] OP_REQUIRES failed at save_restore_v2_ops.cc:184: Out of range: Read less bytes than requested

I would be grateful if you could help me figure this out.

Hi, thank you for sharing your code with pytorch, and I find some importance trouble in your code. Maybe cause failed to reimplements model performance!!

In UGATIT.py -- train function:

1. when you update D , you do not use

self.genA2B.eval(), self.genB2A.eval() and in your code here

fake_B2A.detach(), fake_B2A.detach() ..... but only fake_B2A, fake_B2A. and so on...

so, when you want to only update D, but meanwhile, the G is also updated.
It is the same error in update G

you can reference SPADE code , if I am wrong ,please tell me

I could successfully train the model on my own dataset, and I can see that the trained models are saved in the model folder already. However, I got the following error when I ran test code "python main.py --dataset experiment1 --phase test --light True". Do you know where might go wrong?
Thanks!
The error is:
'''
[*] Load SUCCESS
Traceback (most recent call last):
File "main.py", line 83, in
main()
File "main.py", line 79, in main
gan.test()
File "/home/zoey/nas/zoey/github/UGATIT-pytorch/UGATIT.py", line 387, in test
A2B = np.concatenate((A2B, np.concatenate((RGB2BGR(tensor2numpy(denorm(real_A[0]))),
UnboundLocalError: local variable 'A2B' referenced before assignment
'''

Could you, please, clarify, why MSE loss in discriminator compares unnormalized logits with 0/1 values?

I've never write python code.So how can I run it?
I try to run python main.py --dataset selfie2anime in cmd.But some errors be displayed:
Traceback (most recent call last): File "main.py", line 1, in <module> from UGATIT import UGATIT File "/Users/yh/Personal/UGATIT-pytorch/UGATIT.py", line 2, in <module> from dataset import ImageFolder File "/Users/yh/Personal/UGATIT-pytorch/dataset.py", line 1, in <module> import torch.utils.data as data ImportError: No module named torch.utils.data

can the weights to be exported to onnx?

I test the model with images, some results have different face attributions with input images.

torch.mean(input, dim=[2, 3], keepdim=True), torch.var(input, dim=[2, 3], keepdim=True)

Modify the code so that it resembles the following code sample:

torch.mean(torch.mean(input, dim=2, keepdim=True), dim=3, keepdim=True), torch.var(torch.var(input, dim=2, keepdim=True), dim=3, keepdim=True)

I found that there was no activation function layer where each block was connected.
Such as conv--adaILN--conv--adaILN, Why do you do this? In order to achieve better performance?

为什么训练D的时候不把G的requires_grad设为False呢?
Why not set G's require_grad to False when training D?

两次使用了下面的代码，是否可以将下面代码简化？
Two times using the following code, can you simplify the following code?
fake_A2A, fake_A2A_cam_logit, _ = self.genB2A(real_A)
fake_B2B, fake_B2B_cam_logit, _ = self.genA2B(real_B)
@znxlwm

There's neither relu before the first conv

I failed to find design details of residual blocks from the paper, but I guess it's a bug, can you confirm? Thanks.

PS: Just checked the official tf code, it suffers from the same problem.

Hi, I found that the pytorch version is more unstable compared to tensorflow version of UGATIT. One difference I noticed in this version is the weight-decay is not used in the pytorch optimizer compared to TF version.

I ran python main.py --dataset selfie2anime --light True but I have this error

AssertionError: Torch not compiled with CUDA enabled

Hi, when I execute python main.py --dataset selfie2anime ,it tips RuntimeError:

number of epochs must be larger than or equal to one

##### Information #####
# light :  False
# dataset :  selfie2anime
# batch_size :  1
# iteration per epoch :  1000000

##### Generator #####
# residual blocks :  4

##### Discriminator #####
# discriminator layer :  6

##### Weight #####
# adv_weight :  1
# cycle_weight :  10
# identity_weight :  10
# cam_weight :  1000
Traceback (most recent call last):
  File "main.py", line 83, in <module>
    main()
  File "main.py", line 72, in main
    gan.build_model()
  File "H:\workspace\UGATIT-pytorch\UGATIT.py", line 100, in build_model
    self.trainA = ImageFolder(os.path.join('dataset', self.dataset, 'trainA'), train_transform)
  File "H:\workspace\UGATIT-pytorch\dataset.py", line 107, in __init__
    target_transform=target_transform)
  File "H:\workspace\UGATIT-pytorch\dataset.py", line 47, in __init__
    "Supported extensions are: " + ",".join(extensions)))
RuntimeError: Found 0 files in subfolders of: dataset\selfie2anime\trainA
Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif

but this folder hava default female_2321.jpg file

Hey professor. I remember that the 2 stride conv is more stable than up-pooling (according to DCGAN). However, in your model, the downsampling part is 2 stride conv, while up-pooling in upsampling part.

Could you give us a reason for this?
Looking forward to your reply

can you add pip requirements.txt file? thank you

When I pull new code in the repository I could not pretrain my model. I catch a exception

Is it possible that there is no explicit initialization for the created models? I cannot find it in UGATIT.py file or anywhere else. I would be grateful if you refer me to it or explain its absence.

run:
python main.py --dataset my_dataset --light True
error:
RuntimeError: CUDA error: out of memory

I had sucessfully installed pytorch with GPU on computer.

import torch
torch.cuda.is_available()
True

This works well,and I find no error when running other pytorch codes.

I don't have GPU on my laptop. So, I run the codes on the cloud(Google Drive). It always gives errors saying the gpu memory is not enough.

"cuda out of memory"

Hi, Thank you for sharing this great project. Would you mind sharing a trained model for me to test？ Or data set for training? Thx in advance

Hi, here is another issue that, the TF version of UGATIT has bias in most of the layers (including Linear and Conv), but the torch version has no bias at all.
Just wondering, if there is any difference?

Hi, thanks for this incredible net. I've already trained the dataset, but when I aim to test it and
I input the following command: python main.py --phrase test --light true
However, I got the error in the log:

##### Information #####
# light :  True
# dataset :  selfie2anime
# batch_size :  1
# iteration per epoch :  1000000

##### Generator #####
# residual blocks :  4

##### Discriminator #####
# discriminator layer :  6

##### Weight #####
# adv_weight :  1
# cycle_weight :  10
# identity_weight :  10
# cam_weight :  1000
 [*] Load FAILURE
 [*] Test finished!

It seems like your implementation of AdaILN/ILN layer in pytorch significantly differs from tf version (actually, tf version is the correct one).

in_mean, in_var = torch.mean(torch.mean(input, dim=2, keepdim=True), dim=3, keepdim=True), torch.var(torch.var(input, dim=2, keepdim=True), dim=3, keepdim=True)
ln_mean, ln_var = torch.mean(torch.mean(torch.mean(input, dim=1, keepdim=True), dim=2, keepdim=True), dim=3, keepdim=True), torch.var(torch.var(torch.var(input, dim=1, keepdim=True), dim=2, keepdim=True), dim=3, keepdim=True)

This block implies that when you are trying to evaluate variance of each tensor in the batch over appropriate dimension, instead of variation over axes [2, 3] in IN and axes [1, 2, 3] in LN you evaluate variation of variation in IN and variation of variation of variation in LN which is wrong