atriumlts / subpixel Goto Github PK

When I specify --visualize True as an arg, I get the following

I tensorflow/stream_executor/dso_loader.cc:116] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:116] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:116] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:116] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:116] successfully opened CUDA library libcurand.so.8.0 locally
{'batch_size': 64,
 'beta1': 0.5,
 'checkpoint_dir': 'checkpoint',
 'dataset': 'celebC',
 'epoch': 2,
 'image_size': 400,
 'is_crop': True,
 'is_train': True,
 'learning_rate': 0.0002,
 'sample_dir': 'samples',
 'train_size': inf,
 'visualize': True}
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties: 
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 11.35GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id: 0000:01:00.0)

 [*] Reading checkpoints...
 [*] Load SUCCESS
Epoch: [ 0] [   0/   3] time: 3.6289, g_loss: 0.38425747
WARNING:tensorflow:*******************************************************
WARNING:tensorflow:TensorFlow's V1 checkpoint format is deprecated; V2 will become the default shortly after 10/31/2016.
WARNING:tensorflow:Consider switching to the more efficient V2 format now:
WARNING:tensorflow:   `tf.train.Saver(write_version=tf.train.SaverDef.V2)`
WARNING:tensorflow:to prevent breakage.
WARNING:tensorflow:*******************************************************
Epoch: [ 0] [   1/   3] time: 8.5043, g_loss: 0.34239292
Epoch: [ 0] [   2/   3] time: 10.3694, g_loss: 0.28749919
Epoch: [ 1] [   0/   3] time: 12.3451, g_loss: 0.35043076
Epoch: [ 1] [   1/   3] time: 14.3204, g_loss: 0.29150483
Epoch: [ 1] [   2/   3] time: 16.3195, g_loss: 0.23925565
Traceback (most recent call last):
  File "main.py", line 60, in <module>
    tf.app.run()
  File "/home/markb/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "main.py", line 49, in main
    to_json("./web/js/layers.js", [dcgan.h0_w, dcgan.h0_b, dcgan.g_bn0],
AttributeError: 'DCGAN' object has no attribute 'g_bn0'

Further, when I disable this call in hopes of getting sampler images to save, I get the following

Traceback (most recent call last):
  File "main.py", line 60, in <module>
    tf.app.run()
  File "/home/markb/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "main.py", line 57, in main
    visualize(sess, dcgan, FLAGS, OPTION)
  File "/home/markb/Dev/subpixel/utils.py", line 148, in visualize
    if option == 0:
AttributeError: 'DCGAN' object has no attribute 'sampler'

I can't find any mention of the dcgan's sampler in the code. Is there an easy way of fixing this?

Thanks!
Mark

Your script :

1 def pony2(I, r):
2 a, b, c = I.shape
3 B = I.reshape(a/r, b/r, r, r).transpose(0, 1, 3, 2)
4 B = np.concatenate([B[i] for i in range(8)], axis=1)
5 B = B.transpose(1, 2, 0)
6 B = np.concatenate([B[:, :, i] for i in range(8)], axis=1)
7 return B

changed code :
1 def pony2(I, r):
2 a, b, c = I.shape
3 B = I.reshape(a, b, r, r).transpose(0, 1, 3, 2)
4 B = np.concatenate([B[i] for i in range(8)], axis=1)
5 B = B.transpose(1, 2, 0)
6 B = np.concatenate([B[:, :, i] for i in range(8)], axis=1)
7 return B

The code in your third is a error. Please change it.

Sorry I am mistaken but I never see the phase shift called in your code. in the model.py file you make three deconv2d layers. Looking at the code for these its just regular conv transposes. Where do you use the phase shift stuff in your main code?

Probably a noob question but I have trained the model on the celebA dataset. How should I use this model to generate super-res versions of arbitrary images?

In main.py if the training mode is False, the model is simply loaded from checkpoints. How do I run the inference stage/forward pass of the loaded model?

Do you release the quantitative evaluation code?

The logs seem to show that the g_loss value is always nan for gpu, but strangely works fine on CPU. Running with CUDA 7.5, CudNN v4, GTX 1070 Tensorflow 0.10.0rc0.

Hi, thanks for your sharing, I try to run the code on cpu but failed. could you help me out?
I changed the link for installing tensorflow (on cpu)

I met this problem, when i run the code:
Traceback (most recent call last):
File "main.py", line 58, in
tf.app.run()
File "/root/anaconda3/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "main.py", line 39, in main
dataset_name=FLAGS.dataset, is_crop=FLAGS.is_crop, checkpoint_dir=FLAGS.checkpoint_dir)
File "/home/subpixel-master/model.py", line 58, in init
self.build_model()
File "/home/subpixel-master/model.py", line 75, in build_model
self.G = self.generator(self.inputs)
File "/home/subpixel-master/model.py", line 167, in generator
h2 = PS(h2, 4, color=True)
File "/home/subpixel-master/subpixel.py", line 21, in PS
X = tf.concat(3, [_phase_shift(x, r) for x in Xc])
File "/home/subpixel-master/subpixel.py", line 21, in
X = tf.concat(3, [_phase_shift(x, r) for x in Xc])
File "/home/subpixel-master/subpixel.py", line 7, in _phase_shift
bsize, a, b, c = I.get_shape().as_list()
ValueError: not enough values to unpack (expected 4, got 0)

May I suggest inclusion of the following link to explain the interlacing rainbow picture? I had a very hard time understanding it until I read this.

https://export.arxiv.org/ftp/arxiv/papers/1609/1609.07009.pdf

Hello,

In model.py there are several tabs instead of spaces, you should remove them.

Also, in the docker folder there is the vim backup file ;)

Cheers,
~Nico

HI, first a big thank you for publishing this work.
I am trying to use a trained model and query it with a new probe image.
It seems to me a very imprtant functionality , after all that is what you train the network for, right?
But I couldn't find it anywhere. I tried writing something, but I get poor results.
here is what I came up with:
any insights would be most appreciated.
thanks,
Omer

import os
import cv2
import numpy as np
from model import DCGAN
from utils import get_image, image_save, save_images
import tensorflow as tf
from scipy.misc import imresize

flags = tf.app.flags
flags.DEFINE_integer("batch_size", 64, "The size of batch images [64]")
flags.DEFINE_integer("image_size", 128, "The size of image to use")
flags.DEFINE_string("checkpoint_dir", "/home/omer/work/sub_pixel/models",
                    "Directory name to read the checkpoints [checkpoint]")
flags.DEFINE_string("test_image_dir", "/home/omer/work/sub_pixel/data/celebA/valid",
                    "Directory name of the images to evaluate")
flags.DEFINE_string("out_dir", "/home/omer/work/sub_pixel/out", "Directory name of to save results in")

FLAGS = flags.FLAGS


def doresize(x, shape):
    x = np.copy((x + 1.) * 127.5).astype("uint8")
    y = imresize(x, shape)
    return y


def main():
    with tf.Session() as sess:
        dcgan = DCGAN(sess, image_size=FLAGS.image_size, image_shape=[FLAGS.image_size, FLAGS.image_size, 3],
                      batch_size=FLAGS.batch_size,
                      dataset_name='celebA', is_crop=False, checkpoint_dir=FLAGS.checkpoint_dir)
        res = dcgan.load(FLAGS.checkpoint_dir)
        if not res:
            print ("failed loading model from path:" + FLAGS.checkpoint_dir)
            return

        i = 0
        files = []
        num_batches = len(os.listdir(FLAGS.test_image_dir)) / FLAGS.batch_size
        completed_batches = 0
        input_images = np.zeros(shape=(FLAGS.batch_size, FLAGS.image_size, FLAGS.image_size, 3))
        for f in os.listdir(FLAGS.test_image_dir):
            try:
                img_path = os.path.join(FLAGS.test_image_dir, f)
                if os.path.isdir(img_path):
                    i += 1
                    continue
                img = get_image(img_path, FLAGS.image_size, False)
                files.append(f)
                input_images[i] = img

                if i == FLAGS.batch_size - 1 or i == len(os.listdir(FLAGS.test_image_dir)) - 1:
                    batch_ready(dcgan, input_images, sess, files)

                    i = 0
                    input_images = np.zeros(shape=(FLAGS.batch_size, FLAGS.image_size, FLAGS.image_size, 3))
                    files = []
                    completed_batches += 1
                    print('done batch {0} out of {1}'.format(completed_batches, num_batches))
                else:
                    i += 1
            except Exception as e:
                print("problem working on:" + f)
                print (str(e))
                i += 1


def batch_ready(dcgan, input_images, sess, files):
    input_resized = [doresize(xx, (32, 32, 3)) for xx in input_images]
    sample_input_resized = np.array(input_resized).astype(np.float32)
    sample_input_images = np.array(input_images).astype(np.float32)
    output_images = sess.run(fetches=[dcgan.G],
                             feed_dict={dcgan.inputs: sample_input_resized, dcgan.images: sample_input_images})
    save_results(output_images, files)


def save_results(output_images, files):
    for k in range(0, len(files)):
        out_path = os.path.join(FLAGS.out_dir, files[k] + '_.png')
        out_img = output_images[0][k]

        # out_correct = ((out_img + 1) * 127.5).astype(np.uint8)
        # out_correct = cv2.cvtColor(out_correct, cv2.COLOR_RGB2BGR)
        # cv2.imshow('image', out_correct)
        # cv2.waitKey(0)

        image_save(out_img, out_path)


if __name__ == '__main__':
    main()

Hi,

Was amazed by this great idea compared to conv2_transpose. Looking at tensorflow documentation I found tf.depth_to_space, which seems to do something similar (at least the non-color version), am I mistaken? What's the difference? What's the reason you did not use this op?

I'm trying to understand the phase_shift code. But the following step doesn't make much sense to me.
X = tf.transpose(X, (0, 1, 2, 4, 3))
https://github.com/Tetrachrome/subpixel/blob/master/subpixel.py#L10

Why do we need this transpose here?

Have you looked at implementing this using built-in functions in TensorFlow? Should be much more efficient.

tf.split and tf.concat API changed from earlier version.

Error trying in grayscale images
ValueError: Error when checking target: expected SubPixel to have shape (250, 250, 3) but got array with shape (250, 250, 1)
Thank you in advance !!!!!!!!!!!

Traceback (most recent call last):
File "main.py", line 58, in
tf.app.run()
File "/home/yashiro32/virtualenvironment/neural_style_transfer/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "main.py", line 39, in main
dataset_name=FLAGS.dataset, is_crop=FLAGS.is_crop, checkpoint_dir=FLAGS.checkpoint_dir)
File "/home/yashiro32/virtualenvironment/neural_style_transfer/projects/subpixel/model.py", line 58, in init
self.build_model()
File "/home/yashiro32/virtualenvironment/neural_style_transfer/projects/subpixel/model.py", line 75, in build_model
self.G = self.generator(self.inputs)
File "/home/yashiro32/virtualenvironment/neural_style_transfer/projects/subpixel/model.py", line 167, in generator
h2 = PS(h2, 4, color=True)
File "/home/yashiro32/virtualenvironment/neural_style_transfer/projects/subpixel/subpixel.py", line 20, in PS
Xc = tf.split(3, 3, X)
File "/home/yashiro32/virtualenvironment/neural_style_transfer/local/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1214, in split
split_dim=axis, num_split=num_or_size_splits, value=value, name=name)
File "/home/yashiro32/virtualenvironment/neural_style_transfer/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3261, in _split
num_split=num_split, name=name)
File "/home/yashiro32/virtualenvironment/neural_style_transfer/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 513, in apply_op
(prefix, dtypes.as_dtype(input_arg.type).name))
TypeError: Input 'split_dim' of 'Split' Op has type float32 that does not match expected type of int32.

Running OSX, I get this error on any python file I try to run

{'batch_size': 64,
 'beta1': 0.5,
 'checkpoint_dir': 'checkpoint',
 'dataset': 'celebA',
 'epoch': 25,
 'image_size': 128,
 'is_crop': True,
 'is_train': False,
 'learning_rate': 0.0002,
 'sample_dir': 'samples',
 'train_size': inf,
 'visualize': False}
Traceback (most recent call last):
  File "main.py", line 58, in <module>
    tf.app.run()
  File "/Library/Python/2.7/site-packages/tensorflow/python/platform/default/_app.py", line 30, in run
    sys.exit(main(sys.argv))
  File "main.py", line 39, in main
    dataset_name=FLAGS.dataset, is_crop=FLAGS.is_crop, checkpoint_dir=FLAGS.checkpoint_dir)
  File "/Users/fraserhemp/Documents/subpixel/model.py", line 58, in __init__
    self.build_model()
  File "/Users/fraserhemp/Documents/subpixel/model.py", line 70, in build_model
    self.G = self.generator(self.inputs)
  File "/Users/fraserhemp/Documents/subpixel/model.py", line 155, in generator
    h2 = PS(h2, 4, color=True)
  File "/Users/fraserhemp/Documents/subpixel/subpixel.py", line 21, in PS
    X = tf.concat(3, [_phase_shift(x, r) for x in Xc])
  File "/Users/fraserhemp/Documents/subpixel/subpixel.py", line 9, in _phase_shift
    X = tf.reshape(I, (bsize, a, b, r, r))
  File "/Library/Python/2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 682, in reshape
    name=name)
  File "/Library/Python/2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 411, in apply_op
    as_ref=input_arg.is_ref)
  File "/Library/Python/2.7/site-packages/tensorflow/python/framework/ops.py", line 529, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/Library/Python/2.7/site-packages/tensorflow/python/ops/constant_op.py", line 178, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/Library/Python/2.7/site-packages/tensorflow/python/ops/constant_op.py", line 161, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape))
  File "/Library/Python/2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 319, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "/Library/Python/2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 259, in _AssertCompatible
    (dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

File "main.py", line 67, in
tf.app.run()
File "/home/fyh/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "main.py", line 51, in main
dcgan.train(FLAGS)
File "/home/fyh/ESPCN/subpixel-master/model.py", line 141, in train
save_images(sample_input_images, [8, 8], os.path.join(self.sample_dir, "inputs_small.png"))
File "/home/fyh/ESPCN/subpixel-master/utils.py", line 22, in save_images
return imsave(inverse_transform(images[:num_im]), size, image_path)
File "/home/fyh/ESPCN/subpixel-master/utils.py", line 41, in imsave
return scipy.misc.imsave(path, merge(images, size))
File "/home/fyh/ESPCN/subpixel-master/utils.py", line 31, in merge
h, w = images.shape[1], images.shape[2]

Hi, Thanks for the nice work. I was wondering if you are planning release keras code for subpixel layer nearby? I am currently working on a problem where I am using keras APIs. Thanks!

Thanks!

The following lines
line 12 X = tf.concat(2, [tf.squeeze(x) for x in X]) # bsize, b, a*r, r
line 14 X = tf.concat(2, [tf.squeeze(x) for x in X]) # bsize, a*r, b*r

in subpixel.py cause the first dimension to be dropped when the batch size is one. In fact, line 12 causes the dimension drop and line 14 throws an error.

I propose the following change:
X = tf.concat(2, [tf.squeeze(x, axis = 1) for x in X]) # bsize, b, a*r, r
X = tf.concat(2, [tf.squeeze(x, axis = 1) for x in X]) # bsize, a*r, b*r

Hi,

I really appreciate this work, the tensorflow code is nice and clean. Great job！
I'm wondering if there will be a keras version for this project in the future?

Thanks

@dribnet The last commit where things work for me is 0bee08b
After that I'm getting the following error:

I tensorflow/core/common_runtime/gpu/gpu_device.cc:839] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 680, pci bus id: 0000:04:00.0)
Traceback (most recent call last):
  File "main.py", line 58, in <module>
    tf.app.run()
  File "/home/eder/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv))
  File "main.py", line 42, in main
    dcgan.train(FLAGS)
  File "/home/eder/python/ponynet/model.py", line 108, in train
    save_images(sample_images, [8, 8], './samples/reference.png')
  File "/home/eder/python/ponynet/utils.py", line 21, in save_images
    return imsave(inverse_transform(images), size, image_path)
  File "/home/eder/python/ponynet/utils.py", line 40, in imsave
    return scipy.misc.imsave(path, merge(images, size))
  File "/home/eder/python/ponynet/utils.py", line 30, in merge
    h, w = images.shape[1], images.shape[2]
IndexError: tuple index out of range

I checked that images is an empty tensor. Do you know where that might have been introduced?

As title

Hi thank you for releasing this implementation.

I try to evaluate the model on images with different size, so I would like to define a placeholder as:

t_image = tf.placeholder('float32', [1, None, None, 3], name='input_image')

then when I use the subpixel layer, I will get:

    X = tf.reshape(I, (bsize, a, b, r, r))
  File "/ssd2/Workspace/env3/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2451, in reshape
    name=name)
  File "/ssd2/Workspace/env3/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 508, in apply_op
    (input_name, err))
ValueError: Tried to convert 'shape' to a tensor and failed. Error: None values not supported.

Do you have any idea to solve this problem? if we cannot use dynamic size, we need to define different inference for different image, which is resource and time consuming.

Many thanks in advance.

Hi,
I have trained the model using celebA dataset and now I'm going to test my own data. How should I do it. Where the super-resolution test data (the output) will be saved? In which file exactly?

Thanks for your help. :)

It seems like you highly recommend to use subpixel to replace deconv. But as introduced by Wenzhe Shi, subpixel upsampling is identical to deconv upsampling. So I don't understand why you do this while deconv layer has been well developed in existed deep learning tools.

when I do the command which is "python main.py --dataset mnist --is_train True --is_crop True",there are some errors.

tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 30, in run
sys.exit(main(sys.argv))
File "main.py", line 36, in main
dataset_name=FLAGS.dataset, is_crop=FLAGS.is_crop, checkpoint_dir=FLAGS.checkpoint_dir)
File "/home/qyq/q2015work/Image_Reconstruction/subpixel-master/model.py", line 58, in init
self.build_model()
File "/home/qyq/q2015work/Image_Reconstruction/subpixel-master/model.py", line 75, in build_model
self.G = self.generator(self.inputs)
File "/home/qyq/q2015work/Image_Reconstruction/subpixel-master/model.py", line 167, in generator
h2 = PS(h2, 4, color=True)
File "/home/qyq/q2015work/Image_Reconstruction/subpixel-master/subpixel.py", line 21, in PS
X = tf.concat(3, [_phase_shift(x, r) for x in Xc])
File "/home/qyq/q2015work/Image_Reconstruction/subpixel-master/subpixel.py", line 12, in _phase_shift
X = tf.concat(2, [tf.squeeze(x, axis=1) for x in X]) # bsize, b, a*r, r
TypeError: squeeze() got an unexpected keyword argument 'axis'

I'll let you know if make progress as well--thanks for the help!

thank you very much for this work
but would you release the pre-trained model ,thanks

Could you provide the model data and sample code for doing inference, i.e. generating supper-rosultion images that you provide as examples? Thank you.

Could you please tell me why do you use 32 in here and use 16*3 in here and use r=4 in here

As an example

        x3 = np.arange(10*528*528*48).reshape(10, 528, 528, 48)
        X3 = tf.placeholder("float32", shape=(10, 528, 528, 48), name="X")# tf.Variable(x, name="X")
        Y3 = PS(X3, 3, color=True)
        y3 = sess.run(Y3, feed_dict={X3: x3})

this works fine when the second arg to PS is, say, 4, but it chokes when it's 3

  File "/home/user/Dev/subpixel/model.py", line 75, in build_model
    self.G = self.generator(self.inputs)
  File "/home/user/Dev/subpixel/model.py", line 191, in generator
    h2 = PS(h2, 3, color=True)
  File "/home/user/Dev/subpixel/subpixel.py", line 21, in PS
    Xc = tf.split(3, 3, X)
  File "/home/user/Dev/subpixel/subpixel.py", line 9, in _phase_shift
    X = tf.reshape(I, (bsize, a, b, r, r))
  File "/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2166, in reshape
    name=name)
  File "/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 748, in apply_op
    op_def=op_def)
  File "/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2405, in create_op
    set_shapes_for_outputs(ret)
  File "/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1790, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1696, in _ReshapeShape
    (num_elements, known_elements))
ValueError: input has 44605440 elements, which isn't divisible by 2509056

Any ideas?

Please confirm that the function

def PS(I, r):
  assert len(I.shape) == 3
  assert r>0
  r = int(r)
  O = np.zeros((I.shape[0]*r, I.shape[1]*r, I.shape[2]/(r*2)))
  for x in range(O.shape[0]):
    for y in range(O.shape[1]):
      for c in range(O.shape[2]):
        c += 1
        a = np.floor(x/r).astype("int")
        b = np.floor(y/r).astype("int")
        d = c*r*(y%r) + c*(x%r)
        print a, b, d
        O[x, y, c-1] = I[a, b, d]
  return O

does not implement the PS(T)_{x,y,c} :

In particular, the line in PS(I,r)

d = c*r*(y%r) + c*(x%r)

does not implement the index in PS(T)_{x,y,c}

C * r * mod(y,r) + C * mod(x,r) + c

where capital letter C and small letter c are two different things.

Hi:
Good work for the implementation and could you please provide the trained model?

After training the model with the following line
python main.py --dataset celebA --is_train true --is_crop true
how can I test the model?

I've tried python main.py --dataset celebA --is_train false but it just print this
{'batch_size': 64, 'beta1': 0.5, 'checkpoint_dir': 'checkpoint', 'dataset': 'celebA', 'epoch': 25, 'gpu': '0', 'image_size': 128, 'is_crop': True, 'is_train': False, 'learning_rate': 0.0002, 'sample_dir': 'samples', 'train_size': inf, 'visualize': False} 2017-12-07 20:30:13.309369: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA 2017-12-07 20:30:13.415852: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2017-12-07 20:30:13.416285: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: name: GeForce 840M major: 5 minor: 0 memoryClockRate(GHz): 1.124 pciBusID: 0000:08:00.0 totalMemory: 1.96GiB freeMemory: 1.55GiB 2017-12-07 20:30:13.416310: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce 840M, pci bus id: 0000:08:00.0, compute capability: 5.0) [*] Reading checkpoints...
then it terminats.

Hi, Thank you for the Subpixel 2D implementation in Keras, I am able to run the 2D implementation.
I am currently working on a 3D Convolution and trying to implement Subpixel for 3D convolution. I was wondering if there was an implementation for 3D Convolution and / or is there any specific parameters that I need to take into account when implementing Subpixel 3D

I write the model ,and transfer to android ,but the output is dark.

I think there might be a problem with function _phase_shift(I,r). My code was crashing when image was resized from 1x1 to 2x2 . Probably it is because of some kind of dimension mismatch.
I changed it to :

def _phase_shift(I, r):
# Helper function with main phase shift operation
bsize, a, b, c = I.get_shape().as_list()
X = tf.reshape(I, (bsize, a, b, r, r))
X = tf.transpose(X, (0, 1, 2, 4, 3)) # bsize, a, b, 1, 1
X = tf.split(1, a, X) # a, [bsize, b, r, r]
X = tf.concat(2, [tf.squeeze(x, squeeze_dims=1) for x in X]) # bsize, b, ar, r
X = tf.split(1, b, X) # b, [bsize, ar, r]
X = tf.concat(2, [tf.squeeze(x, squeeze_dims=1) for x in X]) #
bsize, ar, br
return tf.reshape(X, (bsize, ar, br, 1))

Its just adding squeeze_dims=1 . Now it seems to work with batch size = 1

Hello, thanks for sharing the work. I want to know the first paper that proposed subpixel convolution, not efficient subpixel convolution that you discussed in the work. Please answer me.

Hi,i have some questions.
1.in the 1 epoch ,the loss gets lower from 0.1 to 0.05 . after 25 epoch ,no obvious change ..
it is normal?
2. the samples generated during training are like this:

some lines can be seen nearly the edge of images . it is something wrong?

btw, how to use trained model to generate sr image ?

thank you

Amazing to find this project and thank you for your work! I want to realize the project ,would you please tell me the request.txt?thanks!

Is there any specific reason you would prefer a transposed convolution to a normal convolution in the generator ?

such as an input images with size (?, 501, 301, ?).
considering a process:
images-> (501, 301)
pool ->(251, 151)
subpixel->(502, 302)

I think it might add a tf.image.resize_bilinear function at each subpixel convolution?

Hi, I am trying to compare the performance of deconvolution and subpixel convolution. I change the generator by following:
`
def generator(self, z):

    self.h0, self.h0_w, self.h0_b = deconv2d(z, [self.batch_size, 32, 32, self.gf_dim], k_h=1, k_w=1, d_h=1, d_w=1, name='g_h0', with_w=True)
    h0 = lrelu(self.h0)
    self.h1, self.h1_w, self.h1_b = deconv2d(h0, [self.batch_size, 32, 32, self.gf_dim], name='g_h1', d_h=1, d_w=1, with_w=True)
    h1 = lrelu(self.h1)
    h2, self.h2_w, self.h2_b = deconv2d(h1, [self.batch_size, 128, 128, 3], d_h=1, d_w=1, name='g_h2', with_w=True)
    return tf.nn.tanh(h2)`

But it doesn't works:
Traceback (most recent call last): File "main.py", line 58, in <module> tf.app.run() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 30, in run sys.exit(main(sys.argv[:1] + flags_passthrough)) File "main.py", line 42, in main dcgan.train(FLAGS) File "/home/zehaohuang/subpixel_sr/model.py", line 95, in train .minimize(self.g_loss, var_list=self.g_vars) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 196, in minimize grad_loss=grad_loss) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 253, in compute_gradients colocate_gradients_with_ops=colocate_gradients_with_ops) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gradients.py", line 491, in gradients in_grad.set_shape(t_in.get_shape()) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 408, in set_shape self._shape = self._shape.merge_with(shape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 583, in merge_with (self, other)) ValueError: Shapes (64, 128, 128, 64) and (64, 32, 32, 64) are not compatible
Is there something wrong in my changing?

BTW, h2 = tf.depth_to_space(h2, 4) works well, 'PS' function can be replaced by 'tf.depth_to_space'!

When I implement the subpixel layer in DCGAN, the output image is very strange. it full of grid. (sorry,I can't upload the image somehow)

The README.md says to run python main.py --dataset celebA --is_train True --is_crop True. But when I do this it crashes with the error:

Traceback (most recent call last):
  File "main.py", line 58, in <module>
    tf.app.run()
  File "/usr/local/anaconda2/envs/subpixel/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "main.py", line 39, in main
    dataset_name=FLAGS.dataset, is_crop=FLAGS.is_crop, checkpoint_dir=FLAGS.checkpoint_dir)
  File "/develop/nets/subpixel/model.py", line 58, in __init__
    self.build_model()
  File "/develop/nets/subpixel/model.py", line 64, in build_model
    self.up_inputs = tf.image.resize_images(self.inputs, self.image_shape[0], self.image_shape[1], tf.image.ResizeMethod.NEAREST_NEIGHBOR)
  File "/usr/local/anaconda2/envs/subpixel/lib/python2.7/site-packages/tensorflow/python/ops/image_ops.py", line 787, in resize_images
    raise ValueError('\'size\' must be a 1-D Tensor of 2 elements: '
ValueError: 'size' must be a 1-D Tensor of 2 elements: new_height, new_width

Running the same example from the carpedm20/DCGAN-tensorflow repo works fine for me.