The srgan from tensorlayer

	SRGAN_g	SRGAN_d
TensorFlow	Baidu, Googledrive	Baidu, Googledrive
PaddlePaddle	Baidu, Googledrive	Baidu, Googledrive
MindSpore	🚧Coming soon!	🚧Coming soon!
PyTorch	🚧Coming soon!	🚧Coming soon!

Online discussion open

Please join Slack, the invite link as follow:

https://join.slack.com/t/tensorlayer/shared_invite/MjExMjMzODAwNjI0LTE0OTk4ODEzMjUtNmVjNjQ4ZjMzMw

A question about the weight of the loss

g_gan_loss = 1e-3 * tl.cost.sigmoid_cross_entropy(logits_fake, tf.ones_like(logits_fake), name='g')
mse_loss = tl.cost.mean_squared_error(net_g.outputs , t_target_image, is_mean=True)
vgg_loss = 2e-6 * tl.cost.mean_squared_error(vgg_predict_emb.outputs, vgg_target_emb.outputs, 
                              is_mean=True)

Could you pls tell me the reason that why you set 1e-3 and 2e-6 as the weight of the loss?
Thanks.

using only one subpixel layer?

i tried to modify the code to train a x2 model, and by using only one subpixel layer and set the scale=2,or using two subpixel layer set the scale_1=1, scale_2=2, but during training the results seems pretty bad and i don't know why,following are the generated images during training do you have any idears ? i had trained your x4 model and it seems to produce a better results from the begining @zsdonghao
epoch10

epoch20

epoch30

epoch270

Question about using my own dataset.

I only changed the (using the trained vgg19)
config.config.TRAIN.hr_img_path``config.config.TRAIN.lr_img_path
and validation path
and when I run main.py, it turned out like this
Loading fc8: (4096, 1000), (1000,) Exception in thread threading_and_return: Traceback (most recent call last): File "G:\anacondasetup\lib\threading.py", line 916, in _bootstrap_inner self.run() File "G:\anacondasetup\lib\threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "E:\datasets\srgan\SRGAN-master\tensorlayer\prepro.py", line 87, in apply_fn results[i] = fn(data, **kwargs) File "E:\datasets\srgan\SRGAN-master\utils.py", line 17, in crop_sub_imgs_fn x = crop(x, wrg=384, hrg=384, is_random=is_random) File "E:\datasets\srgan\SRGAN-master\tensorlayer\prepro.py", line 232, in crop h, w = x.shape[row_index], x.shape[col_index] AttributeError: 'NoneType' object has no attribute 'shape'
It seems I failed to load the images.
My image size is 256320(hl)
Is there anything else I should have done?
Thx =D

issues when training with my own image

Hi, @zsdonghao, Thank you for sharing your work. I have a problem need you help, thank you!
I occur a error when training with my own image, error information as follows:
Epoch [ 0/100] 28 time: 0.6905s, mse: 0.11056122
Epoch [ 0/100] 29 time: 0.6591s, mse: 0.09596997
Epoch [ 0/100] 30 time: 0.6747s, mse: 0.10335217
Traceback (most recent call last):
File "main.py", line 316, in
train()
File "main.py", line 175, in train
errM, _ = sess.run([mse_loss, g_optim_init], {t_image: b_imgs_96, t_target_image: b_imgs_384})
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 1096, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (4, 96, 96, 3) for Tensor 't_image_input_to_SRGAN_generator:0', which has shape '(16, 96, 96, 3)'

Seems not training on GPU

when I run the SRGAN code - main.py, it seems not training in gpu, how to solve this problem? My gpu is GTX 1080Ti.

SRGAN resolution result from .jpg LR and .png LR are so different.

I've trained the SRGAN model for a week, my training config is :

dataset : DVI2K + my own dataset, about 4000 training pics
epochs : 2000
n_epoch_init : 300
training time : 1 week

I tested some pics, if I use the DVI2K images, it gives pretty good result:

but when I test with my images, the result is not so good:

I start thinking if that's because the image format is jpg?

As we all know, jpg format picture is lossy compression picture, but png format is lossless compression picture. jpg throwing away many information for saving disk space, while png kept all information, so I did the fllowing experiment:

I chose a jpg format image as high resolution image, and then compressed them to jpg and png format low-resolution images, and put these two LR image into trained SRGAN model, let's see what I got:

left: jpg LR, right: generated:

left: png LR, right: generated:

It's obvious that the png format result is better than the jpg format, we can put them in one picture for comparing:

So the question is, are these STOA super resolution papers (including SRGAN) can only work on png format image super resolution? Are there any paper also doing well on jpg images?

Any ideas will be appreciated, I am working on solving the jpg format images super resolution problem recently.

Restart training from breakpoint

How can I start training from breakpoint ? @zsdonghao

ValueError: zero-size array to reduction operation minimum which has no identity

你好，我运行下面这句时，
sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True, log_device_placement=False))
出现
The TensorFlow library wasn't compiled to use SSE instructions, but these are available on your machine and could speed up CPU computations.The TensorFlow library wasn't compiled to use SSE2 instructions, but these are available on your machine and could speed up CPU computations.等问题。不过还可以继续运行。
运行npz = np.load(vgg19_npy_path, encoding='latin1').item()
出现[!] Load checkpoint/g_srgan.npz failed!
[!] Load checkpoint/g_srgan_init.npz failed!
[!] Load checkpoint/d_srgan.npz failed!
再往下运行就报错了：
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.1.3\helpers\pydev\pydevd.py", line 1585, in
globals = debugger.run(setup['file'], None, None, is_module)
File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.1.3\helpers\pydev\pydevd.py", line 1015, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.1.3\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "F:/人工神经网络学习/super Resolution/SRGAN-master/SRGAN-master/main.py", line 316, in
train()
File "F:/人工神经网络学习/super Resolution/SRGAN-master/SRGAN-master/main.py", line 141, in train
print('sample HR sub-image:',sample_imgs_384.shape, sample_imgs_384.min(), sample_imgs_384.max())
File "C:\Program Files\Anaconda3\lib\site-packages\numpy\core_methods.py", line 29, in _amin
return umr_minimum(a, axis, None, out, keepdims)
ValueError: zero-size array to reduction operation minimum which has no identity

If you could send me a model, that would be greatly appreciate! my email address is [email protected] Thanks!

trained SRGAN model : https://github.com/tensorlayer/srgan/releases/tag/1.2.0

Can you provide me your trained model? It takes me too much time to train the DIV2K dataset. @zsdonghao

How to input images with different sizes in the test stage?

I download 32622.zip and only find the g_srgan.npz,but where is d_srgan.npz?

training on 4K image (3840x2160)

I am getting following error while training my 4K image dataset
ValueError: Cannot feed value of shape (10, 96, 96, 3) for Tensor 't_image_input_to_SRGAN_generator:0', which has shape '(16, 96, 96, 3)'

A question about image preprocessing

x = x / (255. / 2.)
x = x - 1.

what's the meaning of this step. Why it can't converge without doing it.

speed of inference

Would you be able to give some timings of your inference with details of GPU?

ValueError: could not broadcast input array from shape (2040,1356,3) into shape (2040)

i'm new to GAN,
i set batchsize form 16 to 4 when training
and when epoch 35,
this issue occurs:
Traceback (most recent call last):
File "main.py", line 320, in
train()
File "main.py", line 167, in train
b_imgs = tl.prepro.threading_data(b_imgs_list, fn=get_imgs_fn, path=config.TRAIN.hr_img_path)
File "/home/wang/SRGAN-master/tensorlayer/prepro.py", line 105, in threading_data
return np.asarray(results)
File "/usr/local/lib/python2.7/dist-packages/numpy/core/numeric.py", line 531, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: could not broadcast input array from shape (2040,1356,3) into shape (2040)

then how to solve this question?
hope to get your reply!

Please download vgg19.npz from : https://github.com/machrisaa/tensorflow-vgg

[!] Load checkpoint/g_srgan.npz failed!
[!] Load checkpoint/g_srgan_init.npz failed!
[!] Load checkpoint/d_srgan.npz failed!
Please download vgg19.npz from : https://github.com/machrisaa/tensorflow-vgg

Why the HR training images use "crop_sub_imgs_fn" ?

I noticed that the training data preprocessing step, the HR images using "crop_sub_imgs_fn" to rescaled to the target size, while the LR images using "downsample_fn". Besides, the "crop_sub_imgs_fn" is randomly cropping the image, if we do so, the HR/LR pair will not match right? Since the corresponding training image region is different!

# from main.py line 226 - line 233
for idx in range(0, len(train_hr_imgs), batch_size):
            step_time = time.time()
            b_imgs_384 = tl.prepro.threading_data(
                    train_hr_imgs[idx : idx + batch_size],
                    fn=crop_sub_imgs_fn, is_random=True)
            b_imgs_96 = tl.prepro.threading_data(b_imgs_384, fn=downsample_fn)
            ## update D
            errD, _ = sess.run([d_loss, d_optim], {t_image: b_imgs_96, t_target_image: b_imgs_384})

# from utils.py line 16 - line 20
def crop_sub_imgs_fn(x, is_random=True):
    x = crop(x, wrg=640, hrg=640, is_random=is_random)
    x = x / (255. / 2.)
    x = x - 1.
    return x

# from prepro.py line 216 - line 244
def crop(x, wrg, hrg, is_random=False, row_index=0, col_index=1, channel_index=2):
    """Randomly or centrally crop an image.

    Parameters
    ----------
    x : numpy array
        An image with dimension of [row, col, channel] (default).
    wrg : float
        Size of weight.
    hrg : float
        Size of height.
    is_random : boolean, default False
        If True, randomly crop, else central crop.
    row_index, col_index, channel_index : int
        Index of row, col and channel, default (0, 1, 2), for theano (1, 2, 0).
    """
    h, w = x.shape[row_index], x.shape[col_index]
    assert (h > hrg) and (w > wrg), "The size of cropping should smaller than the original image"
    if is_random:
        h_offset = int(np.random.uniform(0, h-hrg) -1)
        w_offset = int(np.random.uniform(0, w-wrg) -1)
        # print(h_offset, w_offset, x[h_offset: hrg+h_offset ,w_offset: wrg+w_offset].shape)
        return x[h_offset: hrg+h_offset ,w_offset: wrg+w_offset]
    else:   # central crop
        h_offset = int(np.floor((h - hrg)/2.))
        w_offset = int(np.floor((w - wrg)/2.))
        h_end = h_offset + hrg
        w_end = w_offset + wrg
        return x[h_offset: h_end, w_offset: w_end]

Why do not use low resolution images in training?

First of all, thank you very much for your contribution.
I could not understand Why do not use low resolution images in training, but downsample the resolution of high resolution images?
I'm new. Thank you very much!

LR Image preprocessing did not rescale to [0, 1]

In the original paper, the author proposed scaling the LR input images to [0, 1]:

We scaled the range of the LR input images to [0, 1] and for the HR images to [−1, 1].

But in the source code main.py, in the training section line 224, LR input images are rescaled to [-1, 1] instead:

for idx in range(0, len(train_hr_imgs), batch_size):
            step_time = time.time()
            b_imgs_384 = tl.prepro.threading_data(
                    train_hr_imgs[idx : idx + batch_size],
                    fn=crop_sub_imgs_fn, is_random=True)
            b_imgs_96 = tl.prepro.threading_data(b_imgs_384, fn=downsample_fn)

The threading_data function is :

def crop_sub_imgs_fn(x, is_random=True):
    x = crop(x, wrg=384, hrg=384, is_random=is_random)
    x = x / (255. / 2.)
    x = x - 1.
    return x

def downsample_fn(x):
    # We obtained the LR images by downsampling the HR images using bicubic kernel with downsampling factor r = 4.
    x = imresize(x, size=[96, 96], interp='bicubic', mode=None)
    x = x / (255. / 2.)
    x = x - 1.
    return x

It seems both the HR images and LR images are rescaled to [-1, 1], if we follow the paper's advice, rescaling the LR images to [0, 1], would it be much better?

How to fine-tune the SRGAN model

hi @zsdonghao , I trained your SRGAN model for 2 days , when I evaluate the model with DIV2K images, it give pretty good result, but when I use my own image to test it , the result is not so good. So I want to use my own dataset to fine-tune the pre-train SRGAN model, should I freeze some layers and just train the last few layers? What's your suggestion?

Do someones use the SRGAN_g2 in model.py ?? Why "UpSampling2dLayer" do not work??

How to run SRGAN on multi-gpu？

2x scale photo

hi @zsdonghao , thanks for this great work, i have some photo want to upscal it to 2x , could you please explane to me ho to do that .

Why all PReLU layers are dropped?

Compared with the original paper, i found all PReLU layser have been dropped, why? It's better not using any activation function after conv through expriment?

Training image size

Thank you for publishing this work.
Currently it seems that the training image must be above 384*384 pixels to allow cropping function, however for faces that is too big. Is it possible to make the size a parameter?

download failed

When I download the code, it always displaying "download failed", so what should I do?
I want to ask that can i huave the authority to download the code? Thank you！ @zsdonghao

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[16,64,96,96]

Why does regularization appear twice?

Hello, zsdonghao,

I have a question while running your code, although it seems not affecting the result.

In the first step of training, the program crops images and regularize them to [-1,1], as the code saying,
sample_imgs_384 = tl.prepro.threading_data(sample_imgs, fn=crop_sub_imgs_fn, is_random=False)

In the next step, however, it downsamples the results and regularizes them again:
sample_imgs_96 = tl.prepro.threading_data(sample_imgs_384, fn=downsample_fn)

In both crop_sub_imgs_fn and downsample_fn, the data is divided by 255/2 and minus 1.

def crop_sub_imgs_fn(x, is_random=True):
    # x = crop(x, wrg=48, hrg=48, is_random=is_random)
    x = x / (255. / 2.)
    x = x - 1.
    return x

def downsample_fn(x):
    # We obtained the LR images by downsampling the HR images using bicubic kernel with downsampling factor r = 4.
    x = imresize(x, size=[24, 24], interp='bicubic', mode=None)
    x = x / (255. / 2.)
    x = x - 1.
    return x

Is that just a mistake, or it have some special meaning?

But the pictures saved from them are quiet normal. It is so weird...

issues when training

Thank you for publishing this work.
Currently when I am run the main.py when training,I met with such issues:

Traceback (most recent call last):
File "/home/weasly/\SRGAN/SRGAN-master/SRGAN-master/main.py", line 316, in
train()
File "/home/weasly/SRGAN/SRGAN-master/SRGAN-master/main.py", line 141, in train
print('sample HR sub-image:',sample_imgs_384.shape, sample_imgs_384.min(), sample_imgs_384.max())
File "/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py", line 29, in _amin
return umr_minimum(a, axis, None, out, keepdims)
ValueError: zero-size array to reduction operation minimum which has no identity

the issue seem to be related to:
[!] Load checkpoint/g_srgan.npz failed!
[!] Load checkpoint/g_srgan_init.npz failed!
[!] Load checkpoint/d_srgan.npz failed!

how can I solve the problem?thanks a lot.@zsdonghao

Same settings, different outputs.

hi, @zsdonghao . I'm a beginner in CV. I have trained the SRGAN using your settings and the same DIV2K dataset. But the size of my g_srgan.npz is 5952KB. The g_srgan.npz you sent to me is 8906KB. I used your npz file to test some images and I think the output is better than the model I trained. Is there any training advice for me to train a SRGAN or other GAN models? Thank you very much.

Could you please give out the PSNR & SSIM of SRResnet on Set5?

0w0

Significance of 2 Models

Hi,
I was wondering why there are two generator and discriminator models available? I see you actually only use one of each in the GAN, however these do not seem to be the models which align most closely with the original SRGAN architecture.
Thank you

where is 'easydict' and 'joblib' module

Why I get the ModuleNotFoundError: No module named 'easydict',and I am not found this two module in program.

Your hardware + GPU setup for training and average time to complete training

First off, thanks for publishing the code!
I've looked at a lot of SRGAN implementations on github but yours is the first with an understandable code structure for Tensorflow beginners.

I just wondered how your hardware setup looked like: CPU model, which GPU – and how many?

I'm using macOS 10.12.5 with an i7 6700k and two old crappy GPUs (for now, my GTX 1080 Ti arrives on Thursday! 🎉). Images get loaded from SSD (Read: ~600 MB/s).

Model	GPU Memory	CUDA Cores	CUDA V.	GPU Max Clock rate	Mem Clock rate	Mem Bus Width:
GTX 780	6144 MB	2304	3.5	902 MHz	3004 Mhz	384-bit
GTX 770	2048 MB	1536	3.0	1110 MHz	3505 Mhz	256-bit
SOON:
GTX 1080 Ti	11.264 MB	3584	6.1	~1700 MHz	11010 Mhz	352 bit

So for now i just have the two 700 series cards available. Since Tensorflow 1.2 doesn't offer GPU support for macOS i had to revert to 1.1.

Is it normal that i have to wait around 10min until the training actually starts?
It takes 10min from entering the cmd until i see the Epochs counting down:
Epoch [60/2000] 199 time: 2.3310s, d_loss: 0.13401291 g_loss: 0.03180185 (mse: 0.015155 vgg: 0.014024 adv: 0.002623)

I have completed the first 100 Epochs for SRGAN init, and am now @ Epoch Nr. 60/2000 of the actual training. But it would take me ~11 days to get to step 2000!

Do you think the 1080 Ti could cut the training time down to a bearable amount?

How long did it take for you to complete the 2000 Epochs?

And one more (so i don't have to open a separate issue):
When i'm done with training can i input an image that's not from the training set to upscale it? Could you please explain what i would need to change/add to the code?

ValueError: zero-size array to reduction operation minimum which has no identity

Traceback (most recent call last):
File "main.py", line 316, in
train()
File "main.py", line 141, in train
print('sample HR sub-image:',sample_imgs_384.shape, sample_imgs_384.min(), sample_imgs_384.max())
File "/usr/local/lib/python3.5/dist-packages/numpy/core/_methods.py", line 29, in _amin
return umr_minimum(a, axis, None, out, keepdims)
ValueError: zero-size array to reduction operation minimum which has no identity

ValueError: setting an array element with a sequence

I use the latest tensorflow and tensorlayer，I only changed batch_size from 16 to 4, but get the following error:

`Epoch [ 0/100] 0 time: 14.7791s, mse: 0.21392143
Epoch [ 0/100] 1 time: 0.8450s, mse: 0.26192695
Epoch [ 0/100] 2 time: 0.9554s, mse: 0.17628016
Epoch [ 0/100] 3 time: 0.6539s, mse: 0.26824623
Epoch [ 0/100] 4 time: 0.5328s, mse: 0.24897709
Epoch [ 0/100] 5 time: 1.0997s, mse: 0.27270195
Epoch [ 0/100] 6 time: 0.7496s, mse: 0.23530428
Epoch [ 0/100] 7 time: 0.3676s, mse: 0.25137147
Epoch [ 0/100] 8 time: 0.4393s, mse: 0.38204724
Epoch [ 0/100] 9 time: 0.4401s, mse: 0.21372676
Epoch [ 0/100] 10 time: 0.3587s, mse: 0.36686033
Epoch [ 0/100] 11 time: 0.5300s, mse: 0.18018314
Epoch [ 0/100] 12 time: 0.5174s, mse: 0.19175643
Epoch [ 0/100] 13 time: 0.4631s, mse: 0.16485015
Epoch [ 0/100] 14 time: 0.3546s, mse: 0.22096643
Epoch [ 0/100] 15 time: 0.4885s, mse: 0.20787539
Epoch [ 0/100] 16 time: 0.3442s, mse: 0.23604630
Epoch [ 0/100] 17 time: 0.3414s, mse: 0.24624050
Epoch [ 0/100] 18 time: 0.3520s, mse: 0.17463605
Epoch [ 0/100] 19 time: 0.4458s, mse: 0.13338993
Epoch [ 0/100] 20 time: 0.3602s, mse: 0.11997821
Epoch [ 0/100] 21 time: 0.4222s, mse: 0.24578686
Epoch [ 0/100] 22 time: 0.3538s, mse: 0.14088324
Epoch [ 0/100] 23 time: 0.3547s, mse: 0.21345212
Epoch [ 0/100] 24 time: 0.3659s, mse: 0.15475541
Epoch [ 0/100] 25 time: 0.3523s, mse: 0.19058520
Epoch [ 0/100] 26 time: 0.3555s, mse: 0.10877486
Epoch [ 0/100] 27 time: 0.4412s, mse: 0.15190440
Exception in thread threading_and_return:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/ldy/2017/SRGAN/tensorlayer/prepro.py", line 87, in apply_fn
results[i] = fn(data, **kwargs)
File "/home/ldy/2017/SRGAN/utils.py", line 17, in crop_sub_imgs_fn
x = crop(x, wrg=384, hrg=384, is_random=is_random)
File "/home/ldy/2017/SRGAN/tensorlayer/prepro.py", line 232, in crop
h, w = x.shape[row_index], x.shape[col_index]
IndexError: tuple index out of range

Exception in thread threading_and_return:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/ldy/2017/SRGAN/tensorlayer/prepro.py", line 87, in apply_fn
results[i] = fn(data, **kwargs)
File "/home/ldy/2017/SRGAN/utils.py", line 24, in downsample_fn
x = imresize(x, size=[96, 96], interp='bicubic', mode=None)
File "/home/ldy/2017/SRGAN/tensorlayer/prepro.py", line 834, in imresize
if x.shape[-1] == 1:
AttributeError: 'NoneType' object has no attribute 'shape'

Traceback (most recent call last):
File "main.py", line 341, in
train()
File "main.py", line 201, in train
errM, _ = sess.run([mse_loss, g_optim_init], {t_image: b_imgs_96, t_target_image: b_imgs_384})
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 968, in _run
np_val = np.asarray(subfeed_val, dtype=subfeed_dtype)
File "/usr/local/lib/python2.7/dist-packages/numpy/core/numeric.py", line 531, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.`

Error in testing own image

Hi @zsdonghao I downloaded the trained model you provided and used it to test it on my own image. But during the testing I encountered this error.

  File "main.py", line 318, in <module> 
      evaluate()
  File "main.py", line 284, in evaluate
    net_g = SRGAN_g(t_image, is_train=False, reuse=False)
  File "/home/lakshya/Downloads/SRGAN/model.py", line 42, in SRGAN_g
    n = SubpixelConv2d(n, scale=2, n_out_channel=None, act=tf.nn.relu, name='pixelshufflerx2/1')
  File "/home/lakshya/Downloads/SRGAN/tensorlayer/layers.py", line 2193, in SubpixelConv2d
    net_new.outputs = act(_PS(inputs, r=scale, n_out_channel=n_out_channel))
  File "/home/lakshya/Downloads/SRGAN/tensorlayer/layers.py", line 2175, in _PS
    Xs=tf.split(X,r,3) #b*h*w*r*r
  File "/home/lakshya/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1234, in split
    name=name)
  File "/home/lakshya/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3241, in _split
    num_split=num_split, name=name)
  File "/home/lakshya/anaconda2/lib/python2.7/sitepackages/tensorflow/python/framework/op_def_library.py", line 508, in apply_op
    (prefix, dtypes.as_dtype(input_arg.type).name))
TypeError: Input 'split_dim' of 'Split' Op has type float32 that does not match expected type of int32.```

How can I solve this error?

trian

following error occured training own image

Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/user01/anaconda2/envs/tensorflow/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/user01/anaconda2/envs/tensorflow/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/user01/SRGAN-master/tensorlayer/prepro.py", line 87, in apply_fn
results[i] = fn(data, **kwargs)
File "/home/user01/SRGAN-master/utils.py", line 17, in crop_sub_imgs_fn
x = crop(x, wrg=384, hrg=384, is_random=is_random)
File "/home/user01/SRGAN-master/tensorlayer/prepro.py", line 232, in crop
h, w = x.shape[row_index], x.shape[col_index]
AttributeError: 'NoneType' object has no attribute 'shape'

Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/user01/anaconda2/envs/tensorflow/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
('sample HR sub-image:', (16,), None, None)
File "/home/user01/anaconda2/envs/tensorflow/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/user01/SRGAN-master/tensorlayer/prepro.py", line 87, in apply_fn
results[i] = fn(data, **kwargs)
File "/home/user01/SRGAN-master/utils.py", line 17, in crop_sub_imgs_fn
x = crop(x, wrg=384, hrg=384, is_random=is_random)
File "/home/user01/SRGAN-master/tensorlayer/prepro.py", line 232, in crop
h, w = x.shape[row_index], x.shape[col_index]
AttributeError: 'NoneType' object has no attribute 'shape'

Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/user01/anaconda2/envs/tensorflow/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/user01/anaconda2/envs/tensorflow/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/user01/SRGAN-master/tensorlayer/prepro.py", line 87, in apply_fn
results[i] = fn(data, **kwargs)
File "/home/user01/SRGAN-master/utils.py", line 17, in crop_sub_imgs_fn
x = crop(x, wrg=384, hrg=384, is_random=is_random)
File "/home/user01/SRGAN-master/tensorlayer/prepro.py", line 232, in crop
h, w = x.shape[row_index], x.shape[col_index]
AttributeError: 'NoneType' object has no attribute 'shape'

Exception in thread threading_and_return:
Traceback (most recent call last):
File "/home/user01/anaconda2/envs/tensorflow/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/user01/anaconda2/envs/tensorflow/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/user01/SRGAN-master/tensorlayer/prepro.py", line 87, in apply_fn
results[i] = fn(data, **kwargs)
File "/home/user01/SRGAN-master/utils.py", line 24, in downsample_fn
x = imresize(x, size=[96, 96], interp='bicubic', mode=None)
File "/home/user01/SRGAN-master/tensorlayer/prepro.py", line 834, in imresize
if x.shape[-1] == 1:
AttributeError: 'NoneType' object has no attribute 'shape'

Traceback (most recent call last):
File "/home/user01/SRGAN-master/main.py", line 316, in
train()
File "/home/user01/SRGAN-master/main.py", line 144, in train
tl.vis.save_images(sample_imgs_96, [ni, ni], save_dir_ginit+'/_train_sample_96.png')
File "/home/user01/SRGAN-master/tensorlayer/visualize.py", line 54, in save_images
return imsave(images, size, image_path)
File "/home/user01/SRGAN-master/tensorlayer/visualize.py", line 51, in imsave
return scipy.misc.imsave(path, merge(images, size))
File "/home/user01/SRGAN-master/tensorlayer/visualize.py", line 42, in merge
h, w = images.shape[1], images.shape[2]
IndexError: tuple index out of range
('sample LR sub-image:', (16,), None, None)

Process finished with exit code 1

OOM when allocating tensor with shape[16,64,192,192]

If someone meet this problem during training.
It seems a memory problem. My GPU memory is 4G. I'm not sure if it's enough.

What do I need to do if I want to use the model to gray images?

Hi,
I want to use the SRGAN model to train my own images, but they are gray. So I want to know what I need to do to train these images. It seems that I need do a lot of changes.
Thank you very much!

Error happened when training

**Epoch [ 0/100] 49 time: 0.9090s, mse: 0.07975296
Traceback (most recent call last):
File "/home/hy/PycharmProjects/SRGAN-zsdonghao/main.py", line 316, in
train()
File "/home/hy/PycharmProjects/SRGAN-zsdonghao/main.py", line 175, in train
errM, _ = sess.run([mse_loss, g_optim_init], {t_image: b_imgs_96, t_target_image: b_imgs_384})
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 975, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (2, 96, 96, 3) for Tensor u't_image_input_to_SRGAN_generator:0', which has shape '(16, 96, 96, 3)'

Process finished with exit code 1**

When training , a error happened (Value error).
Could you please tell me why it happened?
thanks a lot.

Pre-train VGG on different application

Hi zsdonghao,
Thanks for sharing your work with tensorflow. On my application, my inputs do not have 3 channels like RGB images (let says they have 5 channels). I believe that I can not use the pre-trained VGG because my inputs have different standard.

So, my concern is that if I want to apply pre-trained VGG, I have to pre-train it first with my inputs (5 channels) until certain time, then use those weights as a pre-trained VGG like in the paper. I hope this is a correct way.

Bests,

training not using GPU

Thank you for your brilliant code. But when I train that code, it just use CPU other than GPU. Thus cause the process very slow. I changed "use_cudnn_on_gpu=None" to True in the file layers.py of the folder tensorlayer, but it was in vain.

So what should I do to run the code with GPU? Thank you for your reply.

Need help, I'm a newer

Please help me to handle this, I just dont know how to do with this.
I use the same dataset(DIV2K..) with author, and didnt change anything of any program.
But it just cant run...

errorTraceback (most recent call last):
File "main.py", line 316, in
train()
File "main.py", line 73, in train
net_g = SRGAN_g(t_image, is_train=True, reuse=False)
File "/home/zhangheng_li/SRGAN/model.py", line 42, in SRGAN_g
n = SubpixelConv2d(n, scale=2, n_out_channel=None, act=tf.nn.relu, name='pixelshufflerx2/1')
File "/home/zhangheng_li/SRGAN/tensorlayer/layers.py", line 2193, in SubpixelConv2d
net_new.outputs = act(_PS(inputs, r=scale, n_out_channel=n_out_channel))
File "/home/zhangheng_li/SRGAN/tensorlayer/layers.py", line 2175, in _PS
Xs=tf.split(X,r,3) #bhwrr
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1234, in split
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 3241, in _split
num_split=num_split, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 508, in apply_op
(prefix, dtypes.as_dtype(input_arg.type).name))
TypeError: Input 'split_dim' of 'Split' Op has type float32 that does not match expected type of int32.

Brightness difference between LR image and HR image

Thanks a lot for the impressed work !
And this condition happens in my experiment:
Scaling images to [-1, 1] is executed in both crop_sub_imgs_fn and downsample_fn. It caused the input of downsample_fn is not in [0,255]. But the input of scipy.misc.imresize need to be in [0,255] when the mode is set None, and this caused the brightness difference between LR image and HR image.
Scaling HR image after downsample_fn fixed it in my experiment.

Does the "batch_size" influence the effect of model?

@zsdonghao ，thanks for your code. I run the code, it works well when loading the images.But when running the conv2d,it raises "resource exhaust error".So I modify the
"batch_size" from "16" to "4",it works well。After finish the 2000 epoch training，I get the model.But when I evaluate my own images.it seems not good as I wished。My
question1：Does the “batch_size” will influence the effect of training？
question2：Could you send me the model you trained，I want to compare it with mine.Thank you very much. [email protected]

tensorlayer / srgan Goto Github PK

srgan's Introduction

Super Resolution Examples

SRGAN Architecture

Prepare Data and Pre-trained VGG

Run

Train

Evaluation.

Results

Reference

Citation

Other Projects

Discussion

License

srgan's People

Contributors

Stargazers

Watchers

Forkers

srgan's Issues

Recommend Projects

Recommend Topics

Recommend Org