Hi Guys, impressive work!

I am trying to recreate your results, I am running the

python genDeepLight.py alice.npy original.mp4 test.avi 500

and then feeding it to deeplight.py as stated in your readme:

python3 deeplight.py -v test/test.avi -ltn weights/LightNet.h5 -scn weights/ScreenNet.json -t 30 -d log -o 1 -n -1

I was expecting to see some decoded data, but all I got is:

_Model: "deeplight"

---

Layer (type) Output Shape Param #

input (InputLayer) [(None, 299, 299, 3)] 0

---

1st_conv (Conv2D) (None, 299, 299, 16) 48

---

1st_bn (BatchNormalization) (None, 299, 299, 16) 48

---

1st (Activation) (None, 299, 299, 16) 0

---

2nd_conv (Conv2D) (None, 97, 97, 32) 41472

---

2nd_bn (BatchNormalization) (None, 97, 97, 32) 96

---

2nd (Activation) (None, 97, 97, 32) 0

---

3rd_conv (Conv2D) (None, 31, 31, 64) 100352

---

3rd_bn (BatchNormalization) (None, 31, 31, 64) 192

---

3rd (Activation) (None, 31, 31, 64) 0

---

4th_conv (Conv2D) (None, 14, 14, 100) 160000

---

4th_bn (BatchNormalization) (None, 14, 14, 100) 300

---

4th (Activation) (None, 14, 14, 100) 0

---

5th_conv (Conv2D) (None, 12, 12, 100) 90000

---

5th_bn (BatchNormalization) (None, 12, 12, 100) 300

---

5th (Activation) (None, 12, 12, 100) 0

---

dropout (Dropout) (None, 12, 12, 100) 0

---

conv2d (Conv2D) (None, 1, 1, 100) 1440100

---

activation (Activation) (None, 1, 1, 100) 0

---

flatten (Flatten) (None, 100) 0

Total params: 1,832,908
Trainable params: 1,832,284
Non-trainable params: 624

---

None
================================= SCREENNET =================================
2021-11-01 22:13:00.074416: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open weights/ScreenNet.json: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
Model: "functional_1"

---

Layer (type) Output Shape Param # Connected to

input_1 (InputLayer) [(None, 256, 256, 3) 0

---

conv2d_1 (Conv2D) (None, 256, 256, 64) 1792 input_1[0][0]

---

conv2d_2 (Conv2D) (None, 256, 256, 64) 36928 conv2d_1[0][0]

---

max_pooling2d (MaxPooling2D) (None, 128, 128, 64) 0 conv2d_2[0][0]

---

conv2d_3 (Conv2D) (None, 128, 128, 128 73856 max_pooling2d[0][0]

---

conv2d_4 (Conv2D) (None, 128, 128, 128 147584 conv2d_3[0][0]

---

max_pooling2d_1 (MaxPooling2D) (None, 64, 64, 128) 0 conv2d_4[0][0]

---

conv2d_5 (Conv2D) (None, 64, 64, 256) 295168 max_pooling2d_1[0][0]

---

conv2d_6 (Conv2D) (None, 64, 64, 256) 590080 conv2d_5[0][0]

---

max_pooling2d_2 (MaxPooling2D) (None, 32, 32, 256) 0 conv2d_6[0][0]

---

conv2d_7 (Conv2D) (None, 32, 32, 512) 1180160 max_pooling2d_2[0][0]

---

conv2d_8 (Conv2D) (None, 32, 32, 512) 2359808 conv2d_7[0][0]

---

dropout_1 (Dropout) (None, 32, 32, 512) 0 conv2d_8[0][0]

---

max_pooling2d_3 (MaxPooling2D) (None, 16, 16, 512) 0 dropout_1[0][0]

---

conv2d_9 (Conv2D) (None, 16, 16, 1024) 4719616 max_pooling2d_3[0][0]

---

conv2d_10 (Conv2D) (None, 16, 16, 1024) 9438208 conv2d_9[0][0]

---

dropout_2 (Dropout) (None, 16, 16, 1024) 0 conv2d_10[0][0]

---

up_sampling2d (UpSampling2D) (None, 32, 32, 1024) 0 dropout_2[0][0]

---

conv2d_11 (Conv2D) (None, 32, 32, 512) 2097664 up_sampling2d[0][0]

---

concatenate (Concatenate) (None, 32, 32, 1024) 0 dropout_1[0][0]
conv2d_11[0][0]

---

conv2d_12 (Conv2D) (None, 32, 32, 512) 4719104 concatenate[0][0]

---

conv2d_13 (Conv2D) (None, 32, 32, 512) 2359808 conv2d_12[0][0]

---

up_sampling2d_1 (UpSampling2D) (None, 64, 64, 512) 0 conv2d_13[0][0]

---

conv2d_14 (Conv2D) (None, 64, 64, 256) 524544 up_sampling2d_1[0][0]

---

concatenate_1 (Concatenate) (None, 64, 64, 512) 0 conv2d_6[0][0]
conv2d_14[0][0]

---

conv2d_15 (Conv2D) (None, 64, 64, 256) 1179904 concatenate_1[0][0]

---

conv2d_16 (Conv2D) (None, 64, 64, 256) 590080 conv2d_15[0][0]

---

up_sampling2d_2 (UpSampling2D) (None, 128, 128, 256 0 conv2d_16[0][0]

---

conv2d_17 (Conv2D) (None, 128, 128, 128 131200 up_sampling2d_2[0][0]

---

concatenate_2 (Concatenate) (None, 128, 128, 256 0 conv2d_4[0][0]
conv2d_17[0][0]

---

conv2d_18 (Conv2D) (None, 128, 128, 128 295040 concatenate_2[0][0]

---

conv2d_19 (Conv2D) (None, 128, 128, 128 147584 conv2d_18[0][0]

---

up_sampling2d_3 (UpSampling2D) (None, 256, 256, 128 0 conv2d_19[0][0]

---

conv2d_20 (Conv2D) (None, 256, 256, 64) 32832 up_sampling2d_3[0][0]

---

concatenate_3 (Concatenate) (None, 256, 256, 128 0 conv2d_2[0][0]
conv2d_20[0][0]

---

conv2d_21 (Conv2D) (None, 256, 256, 64) 73792 concatenate_3[0][0]

---

conv2d_22 (Conv2D) (None, 256, 256, 64) 36928 conv2d_21[0][0]

---

conv2d_23 (Conv2D) (None, 256, 256, 2) 1154 conv2d_22[0][0]

---

conv2d_24 (Conv2D) (None, 256, 256, 1) 3 conv2d_23[0][0]

Total params: 31,032,837
Trainable params: 31,032,837
Non-trainable params: 0

---

None_

am I doing something wrong?

one of the inputs is deeplight\stable\utils\VideoGen\alice.npy, how is this file generated?

thanks!