chshin10 / epinet Goto Github PK

Hello Changha Shin, is this code that you published only test code? Will the training code be announced? Still training code has been released, just because I didn't see it.

Hi Changha,

I found that in your real-world result, you captured light-field images by a Lyrto Illum camera. And in your data dir, there are 81 images from different perspective. How did you generate these 81 images from Lytro file *.lfr or *.lfp? Is there any code or tools for me to generate these images on my own data?

Hi Dr. Shin,
Thank you very much for your excellent works, it really helps me a lot. But I got a little confused when I tested HCI light field scene (boxes, cotton, dino, sideboard) with offered model (pretrained model 9x9 and 5x5), I found that the performances of estimates (MSE and Bad pixel ratio) is different from that published on HCI benchmark website, I guess maybe that the models you offered is different from that you used for HCI benchmark test? Do you have any idea about the difference? Thank you for your attention!
Yours sincerely,
Jinglei

I have a problem when I try to run the code:
Traceback (most recent call last):
File "EPINET_train.py", line 278, in
initial_epoch=iter00, verbose=1,workers=1)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training.py", line 1426, in fit_generator
initial_epoch=initial_epoch)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training_generator.py", line 115, in model_iteration
shuffle=shuffle)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training_generator.py", line 377, in convert_to_generator_like
num_samples = int(nest.flatten(data)[0].shape[0])
AttributeError: 'threadsafe_iter' object has no attribute 'shape'

Anyone may give me some suggestions about it?
Thanks a lot !!

Hello,

I have a feeling that the ix_rd and iy_rd in these two lines (135-136) should be switched. Is it really the case or am I missing something?

In "epinet_fun/func_generate_traindata.py", line 171 ~ 175:
"traindata_batch_label[ii,:,:]=(1.0/scale)*....."
why should it divide by scale after using scale augmentation.

line 121 in func_generate_traindata.py
if(image_id==4 or 6 or 15): this statement will always be True.
Maybe you want something like this? if(image_id==4 or image_id==6 or image_id==15):

And for line 128-132,
if( np.sum(a_tmp[idx_start+scale*crop_half1: idx_start+scale*crop_half1+scale*label_size:scale, idy_start+scale*crop_half1: idy_start+scale*crop_half1+scale*label_size:scale])>0 or np.sum(a_tmp[idx_start: idx_start+scale*input_size:scale, idy_start: idy_start+scale*input_size:scale])>0 ): valid=0
Should this code fragment be also in if(image_id==4): and if(image_id==6): ?

Hi,

I found that using the default code and settings. After 6 days of training, the accuracy on validation set is still worse than the provided pre-trained model. Did you exclude some special training settings or strategies in the provided code?

Thank you!

Hi Changha,

Very thankful for your github contribution code. Can you confirm the resolution of disparity map ?
In your code, it generates 490x490 one but 512x512 is more correct, isnt it?
I ask this question because the ground truth of synthetic LF has resolution 512x512. I want to make a fair comparison.

Have a good day, btw.

Hi Changha,

Very thankful for your github contribution code. In your code, your output is 490x490. I want to know how to use the evaluation-toolkit from HCI benchmark to evaluate 490x490 output, their source code can only be used to evaluate 512x512 output, thank you!

Have a good day, btw

we select the 7×7 views and disparity image of its center view to train our network.

The paper mentioned the disparity label should respond to its center view after random shift augmentation. But in the current code, load_LFdata function introduces the code:
traindata_label=np.zeros((len(dir_LFimages), 512, 512),np.float32), it doesn't include all view disparity.
Although the generate_traindata_for_train function assumes the situation, the disparity image of its center view can't be produced. So I want to clarify the detail of whether to use the center disparity label during the training process.

these lines should be taken out of the for loop of kkk.

I use imgaug to do data_augmentation, and train this model (input 9x9 images), and noticed that every epoch in model.fit() cost 100+ seconds and I run this on 2 TITAN V,and after 2600 epochs the results are bad, so I just want to know how you train this model, do you have a plan to release the full code? Since I was in this trouble for almost one month, more details would be appreciated.

hello,can you provide me with the pytorch version of the code?thank you very much。

Hello, I read your paper about the color scale, but the code didn't include the part, can you explain it to us?Thank you very much.

Hello Changha Shin.
In the paper, you mentioned that methods such as randomly converting color to gray scale from [0,1] is used. However, by reviewing the "epinet_fun/func_makeinput.py" I notice that it've already converted the image into gray scale before returning the data.
Is the training data all gray or just part of it?
Is this function only applied in the test process or in the training as well?
More details in augmentation would be appreciated.
Thank you for your great work.

We extract epipolar image stacks like this.
If you use your LF dataset, LF image ordering may be different, so be careful.

seq90d [76, 67, 58, 49, 40, 31, 22, 13, 4 ] --> image_stack_90d(dimension:NxNx9)
seq0d [36, 37, 38, 39, 40, 41, 42, 43, 44 ] --> image_stack_0d(dimension:NxNx9)
seq45d [72, 64, 56, 48, 40, 32, 24, 16, 8 ] --> image_stack_45d(dimension:NxNx9)
seqM45d [ 0, 10, 20, 30, 40, 50, 60, 70, 80 ] --> image_stack_minus45d(dimension:NxNx9)

The text "Initalization" should be "Initialization".