thunil / tecogan Goto Github PK

Please make a google colab, thanks

python runGan.py 1
Testing test case 1
Traceback (most recent call last):
File "main.py", line 22, in
import tensorflow.contrib.slim as slim
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib_init_.py", line 37, in
from tensorflow.contrib import cudnn_rnn
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\cudnn_rnn_init_.py", line 38, in
from tensorflow.contrib.cudnn_rnn.python.layers import *
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\cudnn_rnn\python\layers_init_.py", line 23, in
from tensorflow.contrib.cudnn_rnn.python.layers.cudnn_rnn import *
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\cudnn_rnn\python\layers\cudnn_rnn.py", line 20, in
from tensorflow.contrib.cudnn_rnn.python.ops import cudnn_rnn_ops
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\cudnn_rnn\python\ops\cudnn_rnn_ops.py", line 22, in
from tensorflow.contrib.rnn.python.ops import lstm_ops
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\rnn_init_.py", line 91, in
from tensorflow.contrib.rnn.python.ops.lstm_ops import *
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\rnn\python\ops\lstm_ops.py", line 298, in
@ops.RegisterGradient("BlockLSTM")
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 2489, in call
_gradient_registry.register(f, self._op_type)
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow_core\python\framework\registry.py", line 61, in register
(self._name, name, function_name, filename, line_number))
KeyError: "Registering two gradient with name 'BlockLSTM'! (Previous registration was in register C:\ProgramData\Anaconda3\lib\site-packages\tensorflow_core\python\framework\registry.py:66)"

I have added a picture of my own to the calendar folder in the LR folder.

I have made my image have the same characteristics as the images in the calendar folder.

These are the characteristics of an original calendar image:
https://i.ibb.co/BPQ8FCv/calendarpng.png

And these are the ones in my own image:
https://i.ibb.co/YLnqmLb/mypng.png

When I execute the command 'python runGan.py 1' everything works correctly until it reaches my image, which is 0042.png and the next error is jumped:
https://i.ibb.co/bzT31MJ/Captura-de-pantalla-de-2020-03-14-14-45-28.png

How could I fix this to work with my own images?

While the code is available under Apache 2.0, what would be the license of the weights of the pre-trained models?

Thanks a lot in advance!

Hi, I have loaded all pre-trained model in the ./model folder, the console print
‘’variable not found in ckpt: generator/generator_unit/resblock_16/conv_2/Conv/weights:0
Assign Zero of (3, 3, 64, 64)
variable not found in ckpt: generator/generator_unit/resblock_16/conv_2/Conv/biases:0
Assign Zero of (64,)
Prepare to load 100 weights from the pre-trained model for generator and fnet
Prepare to load 0 weights from the pre-trained model for discriminator
Finish building the network.

Traceback (most recent call last):
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file E:\pythonLearning\TecoGAN-master\model\vgg_19.ckpt: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?‘’

when I run runGan 3. How can I solve it?
Thank you!

If i wanna froze the checkpoint to a .pb file
which node should i use?

When I try to run this code's trainnning part in google colab, after i change the direcotry to 'TrainingDataPath' ，the error is Exception: No frame files in the video input directory

the structure of my folders show in the picture

Hi,

First of all thanks for providing training code. I was about to retrain the model with my own dataset, without pretrained model. But starting from beginning of training cycle, some of the loss functions always output nan (like D_layer_0_loss, D_layer_1_loss, D_layer_2_loss, D_layer_3_loss and l2_warp_loss).

Is it normal to be like this? I didn't change any source code.

Is it possible to test pre-trained model with new video?

i up with the model to 280*4 = 1120 pixel but if video are in 360 or more il have a progam error
how should i do to be able to use larger image for do i have to train my own model?to get to 4k video ?

Hello. Why there is deprocess(gen_pre_output_warp) in training (line 143)? However, there is no this term in testing.
Thanks in advance.

Hi,
Thanks for your great job and share. To my opinion, your creation about PP Loss, Spatio-temporal discriminator would help the behavior of SR task intuitively.
I've tried run the inference process with the pre-trained model you guys supported, for the set calendar, it has not bad behavior, but for other set, like the video you showed on youtube, the result not satisfies me. I noticed that the compare is between LR and TecoGAN's output, and i tried to use method VESPCN, FRVSR... For the perceptual side, I've got Teco's output is no better than FRVSR(960x540-4k); For the artifacts side, I've got that the Teco's output is not better than VESPCN(when i saw your paper,i was excited,cause i think you might solved partly the problem of instable of GANS).
So,will you guys give me some advice about my confusion? Is unsatisfied behaviors cause of the influence of pre-train model? If you guys can give me some advice about the training step, i answer my question by myself.

Thanks in advance,
chuanjun

is the inference using GPU?
on my PC it is very slow - around 1 second per frame - despite having a GTX1080ti and the GPU utilization is very slow (always lower than 10%).

Hello,

I have a quick question about testing. I want to run the test for hundreds of scenes. Is it feasible to put all of the scenes together in test folder and run the test?

Thanks!

Following on from this thread, I'm still unable to get this to work. I think I'm messing something up with the code in dataloader.py but I still can't figure out what for the life of me.

Could someone please dumb it down a little more for me and if possible show which part of which line of the code provided in that thread, has to be changed? I really need help with this.

Thanks.

Can please somebody record a short video tut (just show what you doing, dosnt necessery to speak/write)

Hello,

I am trying to train the model with my own dataset. Instead of using bicubic, I generated some LR images using my own method. I want to train the model but I cannot find where to put the training LR data.

Thank you!

The last command in “1. Prepare the Training Data” in the readme,

python3 dataPrepare.py --start_id 2000 --duration 120 --REMOVE --disk_path TrainingDataPath

Is missing some slashes in the disk path flag, and causes the dataPrepare.py script to download mp4’s titled TrainingDataPathvimeoID.mp4 into the master folder instead of downloading them as vimeoID.mp4 in the actual TrainingDataPath folder, thus causing them to not be found later in the script & causing the script to fail to extract any scenes. This can be fixed by using the following command instead:

python3 dataPrepare.py --start_id 2000 --duration 120 --REMOVE --disk_path ./TrainingDataPath/

Which simply corrects the disk path flag and allows dataPrepare.py to work as intended

On linux, tensorflow 1.13.0rc1/2
tried Tensorflow 1.14
here is what i get..
$ python3 runGan.py 4
Testing test case 4
Using TensorFlow backend.
Preparing train_data
[Config] Use random crop
[Config] Use random crop
[Config] Use random flip
Sequenced batches: 27610, sequence length: 10
Preparing validation_data
[Config] Use random crop
[Config] Use random crop
[Config] Use random flip
Sequenced batches: 4400, sequence length: 10
tData count = 27610, steps per epoch 6902
Finish building the network.
Scope generator:
Variable: generator/generator_unit/input_stage/conv/Conv/weights:0
Shape: [3, 3, 51, 64]
Variable: generator/generator_unit/input_stage/conv/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_1/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_1/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_1/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_1/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_2/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_2/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_2/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_2/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_3/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_3/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_3/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_3/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_4/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_4/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_4/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_4/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_5/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_5/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_5/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_5/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_6/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_6/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_6/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_6/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_7/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_7/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_7/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_7/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_8/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_8/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_8/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_8/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_9/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_9/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_9/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_9/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_10/conv_1/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_10/conv_1/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/resblock_10/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/resblock_10/conv_2/Conv/biases:0
Shape: [64]
Variable: generator/generator_unit/conv_tran2highres/conv_tran1/Conv2d_transpose/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/conv_tran2highres/conv_tran1/Conv2d_transpose/biases:0
Shape: [64]
Variable: generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/weights:0
Shape: [3, 3, 64, 64]
Variable: generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/biases:0
Shape: [64]
Variable: generator/generator_unit/output_stage/conv/Conv/weights:0
Shape: [3, 3, 64, 3]
Variable: generator/generator_unit/output_stage/conv/Conv/biases:0
Shape: [3]
total size: 843587
Scope fnet:
Variable: fnet/autoencode_unit/encoder_1/conv_1/Conv/weights:0
Shape: [3, 3, 6, 32]
Variable: fnet/autoencode_unit/encoder_1/conv_1/Conv/biases:0
Shape: [32]
Variable: fnet/autoencode_unit/encoder_1/conv_2/Conv/weights:0
Shape: [3, 3, 32, 32]
Variable: fnet/autoencode_unit/encoder_1/conv_2/Conv/biases:0
Shape: [32]
Variable: fnet/autoencode_unit/encoder_2/conv_1/Conv/weights:0
Shape: [3, 3, 32, 64]
Variable: fnet/autoencode_unit/encoder_2/conv_1/Conv/biases:0
Shape: [64]
Variable: fnet/autoencode_unit/encoder_2/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: fnet/autoencode_unit/encoder_2/conv_2/Conv/biases:0
Shape: [64]
Variable: fnet/autoencode_unit/encoder_3/conv_1/Conv/weights:0
Shape: [3, 3, 64, 128]
Variable: fnet/autoencode_unit/encoder_3/conv_1/Conv/biases:0
Shape: [128]
Variable: fnet/autoencode_unit/encoder_3/conv_2/Conv/weights:0
Shape: [3, 3, 128, 128]
Variable: fnet/autoencode_unit/encoder_3/conv_2/Conv/biases:0
Shape: [128]
Variable: fnet/autoencode_unit/decoder_1/conv_1/Conv/weights:0
Shape: [3, 3, 128, 256]
Variable: fnet/autoencode_unit/decoder_1/conv_1/Conv/biases:0
Shape: [256]
Variable: fnet/autoencode_unit/decoder_1/conv_2/Conv/weights:0
Shape: [3, 3, 256, 256]
Variable: fnet/autoencode_unit/decoder_1/conv_2/Conv/biases:0
Shape: [256]
Variable: fnet/autoencode_unit/decoder_2/conv_1/Conv/weights:0
Shape: [3, 3, 256, 128]
Variable: fnet/autoencode_unit/decoder_2/conv_1/Conv/biases:0
Shape: [128]
Variable: fnet/autoencode_unit/decoder_2/conv_2/Conv/weights:0
Shape: [3, 3, 128, 128]
Variable: fnet/autoencode_unit/decoder_2/conv_2/Conv/biases:0
Shape: [128]
Variable: fnet/autoencode_unit/decoder_3/conv_1/Conv/weights:0
Shape: [3, 3, 128, 64]
Variable: fnet/autoencode_unit/decoder_3/conv_1/Conv/biases:0
Shape: [64]
Variable: fnet/autoencode_unit/decoder_3/conv_2/Conv/weights:0
Shape: [3, 3, 64, 64]
Variable: fnet/autoencode_unit/decoder_3/conv_2/Conv/biases:0
Shape: [64]
Variable: fnet/autoencode_unit/output_stage/conv1/Conv/weights:0
Shape: [3, 3, 64, 32]
Variable: fnet/autoencode_unit/output_stage/conv1/Conv/biases:0
Shape: [32]
Variable: fnet/autoencode_unit/output_stage/conv2/Conv/weights:0
Shape: [3, 3, 32, 2]
Variable: fnet/autoencode_unit/output_stage/conv2/Conv/biases:0
Shape: [2]
total size: 1745506
The first run takes longer time for training data loading...
Save initial checkpoint, before any training
[testWhileTrain] step 0:
python3 main.py --output_dir ex_FRVSR06-21-16/train/ --summary_dir ex_FRVSR06-21-16/train/ --mode inference --num_resblock 10 --checkpoint ex_FRVSR06-21-16/model-0 --cudaID 0 --input_dir_LR ./LR/calendar/ --output_pre --output_name 000000000 --input_dir_len 10
Using TensorFlow backend.
input shape: [1, 144, 180, 3]
output shape: [1, 576, 720, 3]
Finish building the network
Loading weights from ckpt model
Frame evaluation starts!!
Traceback (most recent call last):
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node generator/generator_unit/input_stage/conv/Conv/Conv2D}}]]
[[{{node generator/Assign_1}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 263, in
output_frame = sess.run(outputs, feed_dict=feed_dict)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node generator/generator_unit/input_stage/conv/Conv/Conv2D (defined at /media/coles/New Volume/TecoGAN-master/lib/ops.py:55) ]]
[[node generator/Assign_1 (defined at main.py:211) ]]

Caused by op 'generator/generator_unit/input_stage/conv/Conv/Conv2D', defined at:
File "main.py", line 208, in
gen_output = generator_F(inputs_all, 3, reuse=False, FLAGS=FLAGS)
File "/media/coles/New Volume/TecoGAN-master/lib/frvsr.py", line 62, in generator_F
net = conv2(gen_inputs, 3, 64, 1, scope='conv')
File "/media/coles/New Volume/TecoGAN-master/lib/ops.py", line 55, in conv2
activation_fn=None, weights_initializer=tf.contrib.layers.xavier_initializer())
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
return func(*args, **current_args)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1155, in convolution2d
conv_dims=2)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
return func(*args, **current_args)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1058, in convolution
outputs = layer.apply(inputs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1227, in apply
return self.call(inputs, *args, **kwargs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 530, in call
outputs = super(Layer, self).call(inputs, *args, **kwargs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 554, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/keras/layers/convolutional.py", line 194, in call
outputs = self._convolution_op(inputs, self.kernel)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 966, in call
return self.conv_op(inp, filter)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 591, in call
return self.call(inp, filter)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 208, in call
name=self.name)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1026, in conv2d
data_format=data_format, dilations=dilations, name=name)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
op_def=op_def)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1801, in init
self._traceback = tf_stack.extract_stack()

UnknownError (see above for traceback): Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node generator/generator_unit/input_stage/conv/Conv/Conv2D (defined at /media/coles/New Volume/TecoGAN-master/lib/ops.py:55) ]]
[[node generator/Assign_1 (defined at main.py:211) ]]

Traceback (most recent call last):
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.CancelledError: Dequeue operation was cancelled
[[{{node load_frame_cpu/validation_data/shuffle_batch}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 391, in
results = sess.run(fetches)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 676, in run
run_metadata=run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1171, in run
run_metadata=run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1270, in run
raise six.reraise(*original_exc_info)
File "/home/coles/.local/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1255, in run
return self._sess.run(*args, **kwargs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1327, in run
run_metadata=run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1091, in run
return self._sess.run(*args, **kwargs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.CancelledError: Dequeue operation was cancelled
[[node load_frame_cpu/validation_data/shuffle_batch (defined at /media/coles/New Volume/TecoGAN-master/lib/dataloader.py:270) ]]

Caused by op 'load_frame_cpu/validation_data/shuffle_batch', defined at:
File "main.py", line 284, in
rdata = frvsr_gpu_data_loader(FLAGS, useValidat)
File "/media/coles/New Volume/TecoGAN-master/lib/dataloader.py", line 297, in frvsr_gpu_data_loader
vald_batch_list, vald_num_image_list_HR_t_cur = loadHRfunc(valFLAGS, tar_size)
File "/media/coles/New Volume/TecoGAN-master/lib/dataloader.py", line 270, in loadHR
min_after_dequeue=FLAGS.video_queue_capacity, num_threads=FLAGS.queue_thread, seed = FLAGS.rand_seed)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 1346, in shuffle_batch
name=name)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 873, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/ops/data_flow_ops.py", line 488, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3645, in queue_dequeue_many_v2
timeout_ms=timeout_ms, name=name)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
op_def=op_def)
File "/home/coles/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1801, in init
self._traceback = tf_stack.extract_stack()

CancelledError (see above for traceback): Dequeue operation was cancelled
[[node load_frame_cpu/validation_data/shuffle_batch (defined at /media/coles/New Volume/TecoGAN-master/lib/dataloader.py:270) ]]

Hi, I'm trying to understand how TecoGAN works to improve the quality of a video from my childhood that is a little low quality. It's going to be a gift for my mother :)

The problem is that I can't figure out how to do it. Could someone tell me the steps to follow well explained?

Thank you very much for everything.

PS: I already have everything mounted on my pc, the only thing I can't do is to improve the quality of my video. Thank you.

Using Google Colab: 12.72 Gb RAM / Tesla P100-PCIE-16GB

I am using packages of 100 frames because more than that amount the model does not work, all in .png format extracted with ffmpeg, 720p resolution; the problem is that it cannot finish executing case 1 due to this error:
---------------------------------------------ERROR CODE
Testing test case 1
WARNING:tensorflow:From main.py:19: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

• https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
• https://github.com/tensorflow/addons
• https://github.com/tensorflow/io (for I/O related ops)
  If you depend on functionality not listed there, please file an issue.

Using TensorFlow backend.
WARNING:tensorflow:From main.py:138: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.

input shape: [1, 720, 1280, 3]
output shape: [1, 2880, 5120, 3]
WARNING:tensorflow:From main.py:195: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From main.py:201: The name tf.space_to_depth is deprecated. Please use tf.compat.v1.space_to_depth instead.

WARNING:tensorflow:From main.py:203: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From main.py:206: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

WARNING:tensorflow:From /content/TecoGAN/lib/frvsr.py:22: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

Finish building the network
WARNING:tensorflow:From main.py:221: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.

WARNING:tensorflow:From main.py:224: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From main.py:227: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From main.py:228: The name tf.local_variables_initializer is deprecated. Please use tf.compat.v1.local_variables_initializer instead.

WARNING:tensorflow:From main.py:230: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From main.py:239: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

Loading weights from ckpt model
Frame evaluation starts!!
Traceback (most recent call last):
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
(0) Resource exhausted: OOM when allocating tensor with shape[1,64,2881,5121] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

 [[generator/Assign_1/_203]]

Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

(1) Resource exhausted: OOM when allocating tensor with shape[1,64,2881,5121] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 259, in
output_frame = sess.run(outputs, feed_dict=feed_dict)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
(0) Resource exhausted: OOM when allocating tensor with shape[1,64,2881,5121] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[node generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1 (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

 [[generator/Assign_1/_203]]

Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

(1) Resource exhausted: OOM when allocating tensor with shape[1,64,2881,5121] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[node generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1 (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

Original stack trace for 'generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1':
File "main.py", line 204, in
gen_output = generator_F(inputs_all, 3, reuse=False, FLAGS=FLAGS)
File "/content/TecoGAN/lib/frvsr.py", line 76, in generator_F
net = conv2_tran(net, 3, 64, 2, scope='conv_tran2')
File "/content/TecoGAN/lib/ops.py", line 40, in conv2_tran
activation_fn=None, weights_initializer=tf.contrib.layers.xavier_initializer())
File "/tensorflow-1.15.2/python3.6/tensorflow_core/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
return func(*args, **current_args)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/contrib/layers/python/layers/layers.py", line 1417, in convolution2d_transpose
outputs = layer.apply(inputs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/keras/engine/base_layer.py", line 1700, in apply
return self.call(inputs, *args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/layers/base.py", line 548, in call
outputs = super(Layer, self).call(inputs, *args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/keras/engine/base_layer.py", line 854, in call
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/autograph/impl/api.py", line 234, in wrapper
return converted_call(f, options, args, kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/autograph/impl/api.py", line 439, in converted_call
return _call_unconverted(f, args, kwargs, options)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/autograph/impl/api.py", line 330, in _call_unconverted
return f(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/keras/layers/convolutional.py", line 835, in call
dilation_rate=self.dilation_rate)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/keras/backend.py", line 4823, in conv2d_transpose
data_format=tf_data_format)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/nn_ops.py", line 2204, in conv2d_transpose
name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/nn_ops.py", line 2275, in conv2d_transpose_v2
name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/gen_nn_ops.py", line 1407, in conv2d_backprop_input
name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

Could you help me with some suggestions for training TecoGAN model?(runGan.py 3)
Here is my running history of the last three lines:
total size: 20024384
VGG19 restored successfully!!
Loading weights from the pre-trained model to start a new training...

but it will return to terminal command line without any error showing up.
By the way, runGan.py 4 (FRVSR) run successfully.
Thank you in advance for your time.

I watched your video on youtube. That is very nice work.
However everyone is looking up at the higher level aspects of neural networks, and no one is looking down at the basic mechanisms. It could be you are using O(n^2) algorithms where you might use O(nlog(n)). That is a massive difference in computational requirements.
The fast Walsh Hadamard transform (WHT) is a set of n weighted sums with 2 main points:
A) The weights are fixed.
B) The cost is O(nlog(n)).
You can replace the n weighted sums in a neural network layer that take O(n^2) operations with a WHT and individually parameterize the nonlinear functions instead as a means of adjustment/training.
Or to say it again in a slightly different way:
In a conventional neural network there are n adjustable filters (weighted sums) and a fixed nonlinear function.
The alternative is to have n fixed filters (a WHT) and individually parameterized nonlinear functions.
https://github.com/S6Regen/Fixed-Filter-Bank-Neural-Networks

Hi everyone, I've tried to use the collab because a don have a powerful machine and I need to upscale some footage I thin is like 80 frames in separate images, named as Sintitulo11820.png and so on. the case is that I cannot get this done :( please someone help me.

this is the log where the process stops:
Testing test case 1
WARNING:tensorflow:From main.py:19: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

• https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
• https://github.com/tensorflow/addons
• https://github.com/tensorflow/io (for I/O related ops)
  If you depend on functionality not listed there, please file an issue.

Using TensorFlow backend.
WARNING:tensorflow:From main.py:138: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.

input shape: [1, 720, 1280, 3]
output shape: [1, 2880, 5120, 3]
WARNING:tensorflow:From main.py:195: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From main.py:201: The name tf.space_to_depth is deprecated. Please use tf.compat.v1.space_to_depth instead.

WARNING:tensorflow:From main.py:203: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From main.py:206: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

WARNING:tensorflow:From /content/TecoGAN/TecoGAN/lib/frvsr.py:22: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

Finish building the network
WARNING:tensorflow:From main.py:221: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.

WARNING:tensorflow:From main.py:224: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From main.py:227: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From main.py:228: The name tf.local_variables_initializer is deprecated. Please use tf.compat.v1.local_variables_initializer instead.

WARNING:tensorflow:From main.py:230: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From main.py:239: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

Loading weights from ckpt model
Frame evaluation starts!!
Traceback (most recent call last):
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
(0) Resource exhausted: OOM when allocating tensor with shape[1,64,2881,5121] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

 [[generator/Assign_1/_203]]

Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

(1) Resource exhausted: OOM when allocating tensor with shape[1,64,2881,5121] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 259, in
output_frame = sess.run(outputs, feed_dict=feed_dict)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
(0) Resource exhausted: OOM when allocating tensor with shape[1,64,2881,5121] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[node generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1 (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

 [[generator/Assign_1/_203]]

Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

(1) Resource exhausted: OOM when allocating tensor with shape[1,64,2881,5121] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[node generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1 (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

Original stack trace for 'generator/generator_unit/conv_tran2highres/conv_tran2/Conv2d_transpose/conv2d_transpose_1':
File "main.py", line 204, in
gen_output = generator_F(inputs_all, 3, reuse=False, FLAGS=FLAGS)
File "/content/TecoGAN/TecoGAN/lib/frvsr.py", line 76, in generator_F
net = conv2_tran(net, 3, 64, 2, scope='conv_tran2')
File "/content/TecoGAN/TecoGAN/lib/ops.py", line 40, in conv2_tran
activation_fn=None, weights_initializer=tf.contrib.layers.xavier_initializer())
File "/tensorflow-1.15.2/python3.6/tensorflow_core/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
return func(*args, **current_args)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/contrib/layers/python/layers/layers.py", line 1417, in convolution2d_transpose
outputs = layer.apply(inputs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/keras/engine/base_layer.py", line 1700, in apply
return self.call(inputs, *args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/layers/base.py", line 548, in call
outputs = super(Layer, self).call(inputs, *args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/keras/engine/base_layer.py", line 854, in call
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/autograph/impl/api.py", line 234, in wrapper
return converted_call(f, options, args, kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/autograph/impl/api.py", line 439, in converted_call
return _call_unconverted(f, args, kwargs, options)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/autograph/impl/api.py", line 330, in _call_unconverted
return f(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/keras/layers/convolutional.py", line 835, in call
dilation_rate=self.dilation_rate)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/keras/backend.py", line 4823, in conv2d_transpose
data_format=tf_data_format)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/nn_ops.py", line 2204, in conv2d_transpose
name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/nn_ops.py", line 2275, in conv2d_transpose_v2
name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/gen_nn_ops.py", line 1407, in conv2d_backprop_input
name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

Hello,

I am currently using the TecoGAN to perform super-resolution on a video. I find that there is pixel displacement(maybe two pixels) between the corresponding GT images and HR images if I use the bicubic method rather than the equal interval sample method to downsample the GT images(video frames). On this occasion, the quantitative index such as PSNR cannot be calculated. And I really want to know why this happened.

Thanks in advance!

Testing test case 1
WARNING:tensorflow:From main.py:19: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

• https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
• https://github.com/tensorflow/addons
• https://github.com/tensorflow/io (for I/O related ops)
  If you depend on functionality not listed there, please file an issue.

Using TensorFlow backend.
WARNING:tensorflow:From main.py:138: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.

input shape: [1, 144, 180, 3]
output shape: [1, 576, 720, 3]
WARNING:tensorflow:From main.py:195: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From main.py:201: The name tf.space_to_depth is deprecated. Please use tf.compat.v1.space_to_depth instead.

WARNING:tensorflow:From main.py:203: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From main.py:206: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

WARNING:tensorflow:From /content/TecoGAN/lib/frvsr.py:22: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

Finish building the network
WARNING:tensorflow:From main.py:221: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.

WARNING:tensorflow:From main.py:224: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From main.py:227: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From main.py:228: The name tf.local_variables_initializer is deprecated. Please use tf.compat.v1.local_variables_initializer instead.

WARNING:tensorflow:From main.py:230: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From main.py:239: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

Loading weights from ckpt model
Traceback (most recent call last):
File "main.py", line 245, in
weight_initiallizer.restore(sess, FLAGS.checkpoint)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py", line 1280, in restore
if not checkpoint_management.checkpoint_exists_internal(checkpoint_prefix):
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/checkpoint_management.py", line 366, in checkpoint_exists_internal
if file_io.get_matching_files(pathname):
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/lib/io/file_io.py", line 363, in get_matching_files
return get_matching_files_v2(filename)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/lib/io/file_io.py", line 384, in get_matching_files_v2
compat.as_bytes(pattern))
tensorflow.python.framework.errors_impl.NotFoundError: ./model; No such file or directory

when i run
$ python runGan.py 1
it return
Testing test case 1 File "main.py", line 56 os.mkdir(FLAGS.output_dubuntu-drivers devicesir) ^ SyntaxError: invalid syntax
Is some thing wrong I made?

As far as I see in runGan.py download these files: model.zip vid3_LR.zip vid3_HR.zip tos_HR.zip & tos_LR.zip but the trained model itself that should be placed in model/TecoGAN.

Where can I get that pre-trained files?

Regards.

Hi, I was wondering why you normalize HR images to [-1, 1] whereas LR images were normalized to [0, 1] ?

Is this even possible with a single GPU (2070S)?

Using my own dataset, 32 works, 64 works but is very memory intensive, but 128 and 256 flatout don't work at all (can't create shape, or OOM). Are there other settings I could look at adjusting to be able to train with a higher crop size apart from batch size, or is it simply impossible without multiple cards + 32gb system memory?

Thanks

Hello.
When I run python runGan.py 1 to test, I encountered the error below.

Testing test case 1
Traceback (most recent call last):
File "runGan.py", line 90, in
mycall(cmd1).communicate()
File "runGan.py", line 21, in mycall
return subprocess.Popen(cmd)
File "C:\Users\user\Anaconda3\envs\tensorflow\lib\subprocess.py", line 775, in init
restore_signals, start_new_session)
File "C:\Users\user\Anaconda3\envs\tensorflow\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

Please give me any solution or hint to solve this problem.

I generate or edit some works of mine via GAN or other generative models, but as they generally work on small resolution I wanted to rely on secondary models for the upscaling part. However I obtain very poor results.

See for example how this 256x256 image

Results in this poor 1024x1024 version

Curious to hear any explanations or speculation about this, and if there is any way around it.

i have run > conda create -n tecogan python = 3.5 anaconda on terminal
and conda install tensorflow-gpu keras-gpu opencv

i have a tecogan on my anaconda

when i try to run cd C：/ TecoGAN-master

it showed cannot find the path

When I run the command of "python RunGan.py 0", the download speed is only 1kB/s. Not a good experience. I am not sure why. I have tested in Win10and linux. I tried "curl" to replace "wget". The same result.

Could someone help me have tecogan please? I don't know what to do. Where do I begin?

Hi,

Anyone has tried to train a new TecoGAN model? My training processing gives an increasing loss, can you give me any hint by inspecting the log file?

logfile.txt

This lets me subscribe to the repo via GitHub's 'watch releases' feature.

I must be doing something wrong.

I setup TecoGAN on Windows, use the TecoGAN model, run inference mode on an SD progressive video converted to PNG images.
The resulting PNG's are dreadful. "Staircases", horizontal stripes and less detail.
There must be something I am missing. Is there some resolution and scaling setting I perhaps am missing?

Notice, the stair casing in the red-lighted hair on the left of the image and the horizontal stripes on the eyelids. Also, the overall detail seems to be reduced.

``
(tecogan) E:\TECOGAN_APP>python .\main\main.py --cudaID 0 --output_dir png-out/gpu0 --summary_dir log/ --mode inference --input_dir_LR png-in/gpu0 --num_resblock 16 --checkpoint ./main/model/TecoGAN --output_ext png
Using TensorFlow backend.
input shape: [1, 576, 720, 3]
output shape: [1, 2304, 2880, 3]
Finish building the network
Loading weights from ckpt model
Frame evaluation starts!!
Warming up 5
Warming up 4
Warming up 3
Warming up 2
Warming up 1
saving image output_0001
saving image output_0002
...... etc.

Traceback (most recent call last):
File "main.py", line 284, in
Net = TecoGAN( rdata.s_inputs, rdata.s_targets, FLAGS )
File "/home/ksjin/바탕화면/TecoGAN-master/lib/Teco.py", line 500, in TecoGAN
gif_sum = [ gif_summary('LR', r_inputs, max_outputs=max_outputs, fps=3),
File "/home/ksjin/바탕화면/TecoGAN-master/lib/ops.py", line 507, in gif_summary
if summary_op_util.skip_summary():
AttributeError: module 'tensorflow.python.ops.summary_op_util' has no attribute 'skip_summary'

how can i fix this error?

tensorflow = 1.14
python =3.7.3

I've tried run the inference process with the pre-trained model you guys supported, for the set calendar, it has not bad behavior, but for other set, some videos i dowoload from net, the result not satisfies me.
so i test some picture of the set. when the LR is downsle from the HR with cubic algorithms，the output is very good. but when the LR is downscale from the HR with othe algorithms likes convert to jpg->downscaling->convert to LR png,the output not satisfies me.
Is unsatisfied behaviors cause of the influence of pre-train model? If you guys can give me some advice about the training step, i answer my question by myself.
Thanks in advance,

I can't understand how the video data were processed in this way, You mentioned in your paper that it is benefcial to improve temporal coherence of the output.

I have a quick question. Why not direct use tf.space_to_depth() function in training?

Thanks in advance.

Could you comment whether it's possible to modify only a bit of the code and continue using your pre-trained model to "upscale_two"?

Thank you so much for TecoGAN 🙇
I've successfully upscaled sequences of images with your pre-trained model 🚀

The current code hard-codes upscale_four - quadrupling the width and quadrupling the height.

I have attempted several code changes, starting with changing output_shape to be *2 (not *4) of the input shape.

Needless to say, minor tinkering keeps resulting in incompatible shapes, e.g:

Dimension 1 in both shapes must be equal, but are 288 and 576. Shapes are [1,288,360,3] and [1,576,720,3]. for 'generator/Assign_1' (op: 'Assign') with input shapes: [1,288,360,3], [1,576,720,3].

I suspect I might be trying a fool's errand (since the model was trained to upsample x4 rather than x2). Please let me know 🙇

ps - In your paper you write:

In our case, the outputs have four times the resolution of the inputs.

But based on the code, it seems that you may have increased the resolution 16 times?

pps - minor grammatical error in the paper:

In order to avoid the this undesirable case

Thanks for your code. I have some questions. Fistly, how could i train this model. And how did you convert the video into an image for super resolution? Finally, is it a composite video of super-resolution images?

I have been learning image super resolution before, knowing nothing about video super resolution, I hope to receive your reply in your busy schedule. Thank you very much.

!python dataPrepare.py --start_id 2000 --duration 120 --REMOVE --disk_path TrainingDataPath

[Configurations]:
start_id: 2000
duration: 120
disk_path: TrainingDataPath
summary_dir: TrainingDataPath/log/
REMOVE: True
TEST: False
End of configuration
Try loading 308x120.
https://vimeo.com/121649159
[vimeo] 121649159: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
youtube_dl error:https://vimeo.com/121649159
Skipped invalid link or other error:https://vimeo.com/121649159
https://vimeo.com/40439273
[vimeo] 40439273: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
youtube_dl error:https://vimeo.com/40439273
Skipped invalid link or other error:https://vimeo.com/40439273
https://vimeo.com/87389090
[vimeo] 87389090: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
youtube_dl error:https://vimeo.com/87389090
Skipped invalid link or other error:https://vimeo.com/87389090
https://vimeo.com/335874600
[vimeo] 335874600: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
youtube_dl error:https://vimeo.com/335874600
Skipped invalid link or other error:https://vimeo.com/335874600
https://vimeo.com/114053015
[vimeo] 114053015: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
youtube_dl error:https://vimeo.com/114053015
Skipped invalid link or other error:https://vimeo.com/114053015