Hi
When i typed python main.py --phase train
i got

I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.8095
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 6.98GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
Traceback (most recent call last):
File "main.py", line 58, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 50, in main
checkpoint_dir=args.checkpoint_dir, sample_dir=args.sample_dir)
File "/home/gilad/pix2pix-tensorflow/model.py", line 67, in init
self.build_model()
File "/home/gilad/pix2pix-tensorflow/model.py", line 78, in build_model
self.fake_B = self.generator(self.real_A)
File "/home/gilad/pix2pix-tensorflow/model.py", line 247, in generator
d1 = tf.concat([d1, e7], 3)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1075, in concat
dtype=dtypes.int32).get_shape(
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

what should i do?

batch_norm(
inputs,
decay=0.999,
center=True,
scale=False,
epsilon=0.001,
activation_fn=None,
param_initializers=None,
param_regularizers=None,
updates_collections=tf.GraphKeys.UPDATE_OPS,
is_training=True,
reuse=None,
variables_collections=None,
outputs_collections=None,
trainable=True,
batch_weights=None,
fused=None,
data_format=DATA_FORMAT_NHWC,
zero_debias_moving_mean=False,
scope=None,
renorm=False,
renorm_clipping=None,
renorm_decay=0.99
)

parameters are listed above, and is_training should be true when model is under traing ,false when testing, but why there is no such difference in these codes?

Hi,

I am trying to run this code on my own dataset of greyscale images (sample attached).
I replaced the images in facade/train and set input_c_dim=1, output_c_dim=1

But I get the following error message below. I suspect it has something to do with the greyscale because at the bottom of the error message it says "Assign requires shapes of both tensors to match. lhs shape= [1] rhs shape= [3]"

Any ideas?

It works when I run it on the original dataset by the way (with input_c_dim=3, output_c_dim=3)

Thanks,
Maria

~/git/pix2pix-tensorflow$ python main.py --phase train
[*] Reading checkpoint...
Traceback (most recent call last):
File "main.py", line 58, in
tf.app.run()
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "main.py", line 53, in main
model.train(args)
File "/home/hiro/git/pix2pix-tensorflow/model.py", line 150, in train
if self.load(self.checkpoint_dir):
File "/home/hiro/git/pix2pix-tensorflow/model.py", line 388, in load
self.saver.restore(self.sess, os.path.join(checkpoint_dir, ckpt_name))
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1802, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [1] rhs shape= [3]
[[Node: save/Assign_92 = Assign[T=DT_FLOAT, _class=["loc:@generator/g_d8/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](generator/g_d8/biases, save/RestoreV2:92)]]

Caused by op u'save/Assign_92', defined at:
File "main.py", line 58, in
tf.app.run()
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "main.py", line 50, in main
checkpoint_dir=args.checkpoint_dir, sample_dir=args.sample_dir)
File "/home/hiro/git/pix2pix-tensorflow/model.py", line 67, in init
self.build_model()
File "/home/hiro/git/pix2pix-tensorflow/model.py", line 109, in build_model
self.saver = tf.train.Saver()
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1338, in init
self.build()
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1347, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1384, in _build
build_save=build_save, build_restore=build_restore)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 835, in _build_internal
restore_sequentially, reshape)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 494, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 185, in restore
self.op.get_shape().is_fully_defined())
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/state_ops.py", line 283, in assign
validate_shape=validate_shape)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 60, in assign
use_locking=use_locking, name=name)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
op_def=op_def)
File "/home/hiro/git/pix2pix-tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1718, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1] rhs shape= [3]
[[Node: save/Assign_92 = Assign[T=DT_FLOAT, _class=["loc:@generator/g_d8/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](generator/g_d8/biases, save/RestoreV2:92)]]

Using the default tensorflow installation (ie, pip install tensorflow, as of 12/10/16, which yeilds 0.12.0RC0), training the pix2pix model fails with an error:

COMP:pix2pix-tensorflow usr$ python main.py --phase train
Traceback (most recent call last):
  File "main.py", line 58, in <module>
    tf.app.run()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "main.py", line 50, in main
    checkpoint_dir=args.checkpoint_dir, sample_dir=args.sample_dir)
  File "/Users/usr/Desktop/camera_vr/deep_pic/pix2pix-tensorflow/model.py", line 67, in __init__
    self.build_model()
  File "/Users/usr/Desktop/camera_vr/deep_pic/pix2pix-tensorflow/model.py", line 83, in build_model
    self.D_, self.D_logits_ = self.discriminator(self.fake_AB, reuse=True)
  File "/Users/usr/Desktop/camera_vr/deep_pic/pix2pix-tensorflow/model.py", line 205, in discriminator
    h1 = lrelu(self.d_bn1(conv2d(h0, self.df_dim*2, name='d_h1_conv')))
  File "/Users/usr/Desktop/camera_vr/deep_pic/pix2pix-tensorflow/ops.py", line 34, in __call__
    ema_apply_op = self.ema.apply([batch_mean, batch_var])
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/training/moving_averages.py", line 391, in apply
    self._averages[var], var, decay, zero_debias=zero_debias))
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/training/moving_averages.py", line 70, in assign_moving_average
    update_delta = _zero_debias(variable, value, decay)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/training/moving_averages.py", line 177, in _zero_debias
    trainable=False)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1024, in get_variable
    custom_getter=custom_getter)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 850, in get_variable
    custom_getter=custom_getter)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 346, in get_variable
    validate_shape=validate_shape)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 331, in _true_getter
    caching_device=caching_device, validate_shape=validate_shape)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 650, in _get_single_variable
    "VarScope?" % name)
ValueError: Variable d_bn1/d_bn1_2/d_bn1_2/moments/moments_1/mean/ExponentialMovingAverage/biased does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

using imsave on a float array stretches the image to [0,255]. we should convert to uint8 before saving.

When i run the code ,I got the error. Can you tell me how to fix it . Thanks.

wrong repo...

Hi just wondered if there is a reason that you still use the same batch normalisation layers when testing? Surely they should now have their is_training flag set to true?

I think the original pix2pix GAN is using patchGAN. Where is the code related to it in your Tensorflow version?

I tried the edges2shoes experiment, setting batch_size = 4, and
other parameters as default, after a few epochs the generator loss keeps, not decreasing.
Besides, after 80 epochs the images generated by the generator seemed almost the same as the input image. Could you give me some advices?

Thanks a lot!

both sampling and testing are done on the val/ folder
the test/ folder is never used
also not sure why we need to sample from a separate val folder as this is never used.

I am interested in the architecture used in this implementation. Does it use a U-Net for the generator and a PatchGAN for the discriminator, as reported in the image-to-image translation paper? By looking at the code, it seems that the model is a DCGAN adapted to 256x256 images.

After upgrading my tensorflow version to 1.0, this code will have Error even if I have transformed this code to 1.0 version.

The error is
ValueError: Variable d_h0_conv/w/Adam/ does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

Had anyone tried to reproduce results of day to night?

The problem is that i get very poor results. I take dataset mentioned in paper and make folders A and B in this way : take strong day photos(daylight > 0.8) for A and strong night photos (night > 0.6) for B for each webcam. Then i made some data augmentation through rotating/mirroring of images. After that I combined them with script and change batch_size from 1 to 4 and epochs from 200 to 17 as in paper.

Thank you!

hi,

i added a few features in my fork (eyaler/pix2pix-tensorflow)

please let me know if you are interested I can make PRs

1. support png
2. allow keeping aspect ratio by padding the shorter dimension
3. allow 90 deg rotations

thanks!

i am seeing errors when either input_c_dim=1 or output_c_dim=1 or both of them =1

only input or output ==1 and the other==3:
ValueError: Trying to share variable d_h0_conv/w, but specified shape (5, 5, 6, 64) and found shape (5, 5, 4, 64).

both ==1:
ValueError: Cannot feed value of shape (1, 256, 256, 1, 6) for Tensor 'real_A_and_B_images:0', which has shape '(1, 256, 256, 2)'

Curious what sort of train times you're seeing with this implementation.

I'm using a GRID K520 GPU (Amazon g2.2xlarge) -- i'm seeing each Epoch take around 1200 seconds, which seems wrong.

From the original paper:

"Data requirements and speed We note that decent results
can often be obtained even on small datasets. Our facade
training set consists of just 400 images (see results in
Figure 12), and the day to night training set consists of only
91 unique webcams (see results in Figure 13). On datasets
of this size, training can be very fast: for example, the results
shown in Figure 12 took less than two hours of training
on a single Pascal Titan X GPU."

Granted I'm not using a Pascal GPU -- which as 2496 CUDA cores, but the g2.2xlarge has around 1500 CUDA cores. At the current rate 200 epochs would take 3 days, as opposed to the 2 hours quoted in the original paper.

Are you seeing similar train times when running this code? Wondering why there is such a discrepancy compared to the original paper/Torch implementation

many args are not passed: flips, load_size, AtoB_direction, etc
also: type=bool in argparser is not supported and can give wrong value:
http://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse

Hi
I trained your code on the facades dataset. However the discriminator outputs for real and fake images i.e. self.D and selfD_ keeps on oscillating from 0 to 1 to 0. In ideal case, they should both converge to 0.5. However even after sufficient number of training epochs, these two outputs are not converging at all and keeps on oscillating.
Does this mean that discriminator is not getting trained at all?
If discriminator is not getting trained well, is the final image to image translation only due to L1 loss optimization?

Thanks

Hi...

I was wondering if adding Wasserstein distance to the code would help the GAN to stabilize and give better results... from what I've read, it has really nice properties and implementing it shouldn't be a problem since it only needs a couple of changes (described here). here's the link to the Github page of the paper : https://github.com/martinarjovsky/WassersteinGAN

I tried implementing it in Pix2Pix but it seems the whole network just ignores the new Wasserstein distance and only focuses on the MSE L2 loss thus giving averaged useless outputs... I'm pretty sure the problem lies in my implementation (as always!).

When I ran the script for dataset download. I got following error.

--2017-01-30 03:23:12-- https://people.eecs.berkeley.edu/~tinghuiz/projects/pix2pix/datasets/.tar.gz
Resolving people.eecs.berkeley.edu (people.eecs.berkeley.edu)... 128.32.189.73
Connecting to people.eecs.berkeley.edu (people.eecs.berkeley.edu)|128.32.189.73|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-01-30 03:23:14 ERROR 404: Not Found.

can we allow controlling the patch size explicitly?

how to add my own dataset and train my models?

Hi, Yen-Chen, first of all, this is really a great work :-)
I've used it and have an advice for restoring and saving checkpoints.

Instead of using counter, use globel_step to keep track of steps:

global_step = tf.Variable(0, name='global_step', trainable=False)
train_op = optimizer.minimize(loss, global_step=global_step)

Then you can save with:

saver.save(sess, save_path, global_step=global_step)

When you restore, the value of global_step is restored as well :-D

Hi,
while I trained the GAN using the facades database, I experienced a really high g_loss (around 30 @ 27 epoch) and quite low d_loss (around 5 @ 27 epoch). I am just running the code as it is, without changing anything. Is this behavior normal, or there is any issue I should tackle?

Here you can find the plots with tensorboard.

edit: Here some generated samples of various epochs.

I have trained hundreds of epochs and the loss of discriminator stay still at 1.38629460, loss of generator is also not decreasing. Samples given also shows no improvements at all. Could someone give me some suggestion?