This is the repository of the paper "A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views" (TIP 2019).

By Jinglei Shi, Xiaoran Jiang and Christine Guillemot

<Project page>, <Paper link>

python==2.X or <=3.6
tensorflow==1.2.1
cuda version==8.0.27 compatible GPU (tested on NVIDIA Tesla P-100)

Folder 'fn2': the code of FlowNet 2.0 and some tool functions.

Folder 'models': two well trained models (one for densely sampled light fields, the other for sparsely sampled light fields).

Download links: dense model & sparse model

refinement.py: the code for the refinement network.

warper.py: it inclueds functions that warp images towards desired position.

pipeline.py: our proposed pipeline, which integrates both FlowNet 2.0 and refinement network together.

test.py: applying our framework to estimate depth map. In its main function, input parameters are:

• checkpoint: the path to the trained model;
• lf_file_path: the path to a .h5 file containing the target light field, and this light field should be stored as an array with shape [H,W,C,U,V] and with type unit8, and the name of this array is 'image'
• row & column: the row & column index for the target sub-apeture view, whose disparity map will be estimated.
• min_radius & max_radius: the two parameters that decide which views are used as 'stereo views' for estimation, those views falling into the range [min_radius, max_radius] and in the same row & column of the target view will be stereo views.
• warping_view_positions: a list containing the positions of the 'warping views'.

After configurating all parameters, we can simply launch the simulation by:

python test.py

We have created the "INRIA Synthetic Light Field Datasets" tailored for diverse light field processing tasks: Dense Light Field Dataset (DLFD, captcha "lfcc") and Sparse Light Field Dataset (SLFD), captcha "lfcc"), each dataset in a .zip format. Every light field included in the datasets boasts an angular resolution of $9 \times 9$ and a spatial resolution $512 \times 512$. Within each scene folder, we provide all sub-aperture images in .PNG format, alongside disparity maps available in both .npy and .mat file formats, and a .cfg file containing camera parameters.

Please consider citing our work if you find it useful.

@article{shi2019depth,
    title={A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views},
    author={Jinglei Shi and Xiaoran Jiang and Christine Guillemot},
    journal={IEEE Transactions on Image Processing},
    volume={28},
    number={12},
    pages={5867-5880},
    month={Dec},
    year={2019}}