Using pre-trained Deep Learning models and Transformations for generating occupancy maps.

If you find any difficulty in using some notebook or script, please feel free to create an issue. This is a work in progress and I will keep making changes to the repository for a while.

Occupancy Grid Mapping refers to a family of computer algorithms which address the problem of generating maps from noisy and uncertain data.

The system takes a stereo pair and generates a depth map(using PSMNet) and instance segmented scene(using maskrcnn). We then use these to get a 3D Model of the scene. This 3d model is projected to the ground to get the occupancy grid.

In CV, a depth image contains information about depth of surfaces presents in the image. Some methods to get the depth image.

• Stereo Methods.
• Monocular Methodss.
• Network (PSMNET)
  • PSMNet actually gives disparity map, which can be converted to a depth map.

We identify each instance of each object featured in the image instead of categorizing each pixel like in semantic segmentation.

• We can use pretrained models of detectron2 model-zoo.

The inputs are

• Image or scene of a self driving car.
• Resolution Size for occupancy grid(to be done).
• Size of image input.

The output consists of

• Semantic Segmentation in one folder.
• Depth Image in another folder.
• Occupancy Grid in another.
• The output images should have a proper name.

• KITTI Dataset is used for testing in the project.

• Debug PSMNet to get the depth image.
• Using detectron2 to get the instance segmented image(without text).
• Using depth image to obtain a 3d visualisation.
• Getting occupancy maps from 3d visualisation.
• Format in the input/output format mentioned above.
• Making the occupancy map better.
• Writing scripts so that the system is easily usable.
• Adding the Monocular depth maps.
• Stitching the point clouds to get a map of the environment(a little ambitious for now).
  • If you are a beginner and want to team up on this, please contact.

• Time Series Prediction.
• ResNet 34/ Resnet 50 Implementations.
• GANs Implementation.
• Sequence to Sequence model.

• Photogometry Lectures

Shashank Srikanth