Python Jupyter notebook with TensorFlow Keras implementation for the mono image depth prediction.

Kaggle Notebook

Local Notebook

We can achieve a depth map from a pair of stereo images with the SGBM algorithm. Let's feed stereo_depth_estimator with KITTI dataset to achieve depth maps for stereo image pairs. Then we can generate pairs of input -> depth by uniformly taking left or right image from the input pair and the corresponding depth map.

Full dataset available here. Example pair:

Let's train a GAN model that learns image to image translations. We are going to use Pix2Pix model architecture by Isola et al.

We will feed the model with source -> destination pairs and expect it to learn the translation between them. In our case, we are going to train the network on input -> depth pairs and expect it to generate depth maps for unseen inputs.

With this approach we are going to train the network that is capable of generating a depth map for a single image.

Left column shows source inputs, middle column model generated outputs and right column destination depth maps.

This is how the training progress looks over time:

We can see that over time model generated outputs are getting closer to the destination depth maps generated with stereo matching.

Inputs on the left and model generated depth maps on the right:

Greg (Grzegorz) Surma

PORTFOLIO

GITHUB

BLOG