Hi Guys, impressive work!
I am trying to recreate your results, I am running the
python genDeepLight.py alice.npy original.mp4 test.avi 500
and then feeding it to deeplight.py as stated in your readme:
python3 deeplight.py -v test/test.avi -ltn weights/LightNet.h5 -scn weights/ScreenNet.json -t 30 -d log -o 1 -n -1
I was expecting to see some decoded data, but all I got is:
_Model: "deeplight"
Layer (type) Output Shape Param #
input (InputLayer) [(None, 299, 299, 3)] 0
1st_conv (Conv2D) (None, 299, 299, 16) 48
1st_bn (BatchNormalization) (None, 299, 299, 16) 48
1st (Activation) (None, 299, 299, 16) 0
2nd_conv (Conv2D) (None, 97, 97, 32) 41472
2nd_bn (BatchNormalization) (None, 97, 97, 32) 96
2nd (Activation) (None, 97, 97, 32) 0
3rd_conv (Conv2D) (None, 31, 31, 64) 100352
3rd_bn (BatchNormalization) (None, 31, 31, 64) 192
3rd (Activation) (None, 31, 31, 64) 0
4th_conv (Conv2D) (None, 14, 14, 100) 160000
4th_bn (BatchNormalization) (None, 14, 14, 100) 300
4th (Activation) (None, 14, 14, 100) 0
5th_conv (Conv2D) (None, 12, 12, 100) 90000
5th_bn (BatchNormalization) (None, 12, 12, 100) 300
5th (Activation) (None, 12, 12, 100) 0
dropout (Dropout) (None, 12, 12, 100) 0
conv2d (Conv2D) (None, 1, 1, 100) 1440100
activation (Activation) (None, 1, 1, 100) 0
flatten (Flatten) (None, 100) 0
Total params: 1,832,908
Trainable params: 1,832,284
Non-trainable params: 624
None
================================= SCREENNET =================================
2021-11-01 22:13:00.074416: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open weights/ScreenNet.json: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
Model: "functional_1"
Layer (type) Output Shape Param # Connected to
input_1 (InputLayer) [(None, 256, 256, 3) 0
conv2d_1 (Conv2D) (None, 256, 256, 64) 1792 input_1[0][0]
conv2d_2 (Conv2D) (None, 256, 256, 64) 36928 conv2d_1[0][0]
max_pooling2d (MaxPooling2D) (None, 128, 128, 64) 0 conv2d_2[0][0]
conv2d_3 (Conv2D) (None, 128, 128, 128 73856 max_pooling2d[0][0]
conv2d_4 (Conv2D) (None, 128, 128, 128 147584 conv2d_3[0][0]
max_pooling2d_1 (MaxPooling2D) (None, 64, 64, 128) 0 conv2d_4[0][0]
conv2d_5 (Conv2D) (None, 64, 64, 256) 295168 max_pooling2d_1[0][0]
conv2d_6 (Conv2D) (None, 64, 64, 256) 590080 conv2d_5[0][0]
max_pooling2d_2 (MaxPooling2D) (None, 32, 32, 256) 0 conv2d_6[0][0]
conv2d_7 (Conv2D) (None, 32, 32, 512) 1180160 max_pooling2d_2[0][0]
conv2d_8 (Conv2D) (None, 32, 32, 512) 2359808 conv2d_7[0][0]
dropout_1 (Dropout) (None, 32, 32, 512) 0 conv2d_8[0][0]
max_pooling2d_3 (MaxPooling2D) (None, 16, 16, 512) 0 dropout_1[0][0]
conv2d_9 (Conv2D) (None, 16, 16, 1024) 4719616 max_pooling2d_3[0][0]
conv2d_10 (Conv2D) (None, 16, 16, 1024) 9438208 conv2d_9[0][0]
dropout_2 (Dropout) (None, 16, 16, 1024) 0 conv2d_10[0][0]
up_sampling2d (UpSampling2D) (None, 32, 32, 1024) 0 dropout_2[0][0]
conv2d_11 (Conv2D) (None, 32, 32, 512) 2097664 up_sampling2d[0][0]
concatenate (Concatenate) (None, 32, 32, 1024) 0 dropout_1[0][0]
conv2d_11[0][0]
conv2d_12 (Conv2D) (None, 32, 32, 512) 4719104 concatenate[0][0]
conv2d_13 (Conv2D) (None, 32, 32, 512) 2359808 conv2d_12[0][0]
up_sampling2d_1 (UpSampling2D) (None, 64, 64, 512) 0 conv2d_13[0][0]
conv2d_14 (Conv2D) (None, 64, 64, 256) 524544 up_sampling2d_1[0][0]
concatenate_1 (Concatenate) (None, 64, 64, 512) 0 conv2d_6[0][0]
conv2d_14[0][0]
conv2d_15 (Conv2D) (None, 64, 64, 256) 1179904 concatenate_1[0][0]
conv2d_16 (Conv2D) (None, 64, 64, 256) 590080 conv2d_15[0][0]
up_sampling2d_2 (UpSampling2D) (None, 128, 128, 256 0 conv2d_16[0][0]
conv2d_17 (Conv2D) (None, 128, 128, 128 131200 up_sampling2d_2[0][0]
concatenate_2 (Concatenate) (None, 128, 128, 256 0 conv2d_4[0][0]
conv2d_17[0][0]
conv2d_18 (Conv2D) (None, 128, 128, 128 295040 concatenate_2[0][0]
conv2d_19 (Conv2D) (None, 128, 128, 128 147584 conv2d_18[0][0]
up_sampling2d_3 (UpSampling2D) (None, 256, 256, 128 0 conv2d_19[0][0]
conv2d_20 (Conv2D) (None, 256, 256, 64) 32832 up_sampling2d_3[0][0]
concatenate_3 (Concatenate) (None, 256, 256, 128 0 conv2d_2[0][0]
conv2d_20[0][0]
conv2d_21 (Conv2D) (None, 256, 256, 64) 73792 concatenate_3[0][0]
conv2d_22 (Conv2D) (None, 256, 256, 64) 36928 conv2d_21[0][0]
conv2d_23 (Conv2D) (None, 256, 256, 2) 1154 conv2d_22[0][0]
conv2d_24 (Conv2D) (None, 256, 256, 1) 3 conv2d_23[0][0]
Total params: 31,032,837
Trainable params: 31,032,837
Non-trainable params: 0
None_
am I doing something wrong?
one of the inputs is deeplight\stable\utils\VideoGen\alice.npy, how is this file generated?
thanks!