Figure 1. The AdobeIndoorNav Dataset and other 3D scene datasets. Our dataset supports research on robot visual navigation in real-world scenes. It provides visual inputs given a robot position: (a) the original 3D point cloud reconstruction; (b) the densely sampled locations shown on 2D scene map; (c) four examples RGB images captured by robot camera and their corresponding locations and poses. Sample views from 3D synthetic and real-world recontructed scene datasets: (d) Observation images from two synthetic datasets: SceneNet RGB-D and AI2-THOR; (e) Rendered images from two real-world scene datasets: Stanford 2D-3D-S and ScanNet.

Arxiv Version: https://arxiv.org/abs/1802.08824

Project Page: https://cs.stanford.edu/~kaichun/adobeindoornav/

Video: https://youtu.be/iqo1ihr_qXI

Contact: [email protected]

This repository contains the AdobeIndoorNav dataset and the relevant codes for visualization. The dataset is proposed and used in the paper The AdobeIndoorNav Dataset: Towards Deep Reinforcement Learning based Real-world Indoor Robot Visual Navigation by Kaichun Mo, Haoxiang Li, Zhe Lin, Joon-Young Lee. We design a semi-automatic pipeline to collect a new dataset for robot indoor visual navigation. Our dataset includes 3D reconstruction for real-world scenes as well as densely captured real 2D images from the scenes. It provides high-quality visual inputs with real-world scene complexity to the robot at dense grid locations.

All the code is tested in Python2.7. Please run the following commands to install the dependencies.

       pip install -r requirements.txt

Please check the README.md under folder datasets to download the dataset.

The first-version dataset contains 24 scenes (i.e. 15 office rooms, 5 conference rooms, 2 open spaces, 1 kitchen, 1 storage room). For each scene, we propose the raw 3D point cloud in ply format, the 2D obstacle map and laser-scan map, the ground-truth world graph map and a set of densely captured panoramic 360 images at each observation location.

The dataset splits are in splits folder. It contains the train/test split and all scene sub-category splits.

The dataset statistics are in stats folder. It contains the sparse landmark location ids (stats/landmark_targets) and the dense SIFT-featureful location ids (stats/landmark_targets), as introduced in the paper.

You can run the following command to quickly browse in et12-kitchen scene.

        bash quick_browse.sh

To prepare all the 24 scenes for visualization, run the following command. This will take a while. Be patient. Please make sure you have downloaded the dataset and put it under folder datasets/adobeindoornav_dataset.

        bash prepare_all_scenes.sh

To prepare the scenes with random camera jitters and visual noises, please run

        bash prepare_all_scenes_with_jitters.sh

You need to first crop regular images from the panoramic 360 images. To process each scene, go to scripts folder and run

        python crop_panorama_images.py [scene_name]

We also provide the functionality to add camera jitters and random noises to the visual inputs, check the following for more details.

        python crop_panorama_images.py --help

To run batch generation for all 24 scenes, please run

        bash run_crop_all_scenes_without_jitters.sh
        bash run_crop_all_scenes_with_jitters.sh

Then, we dump the data into HDF5 files. To process each scene, go to scripts folder and run

        python gen_visu_h5.py [scene_name]

This commands defaults to load the images from data/panorama_images_cropped_rgb_images folder and generate one image per location. To load the cropped images from different folder and to load more images per location, use the following

        python gen_visu_h5.py [scene_name] --data_dir [data_dir] --num_imgs_per_loc [num_imgs_per_loc]

To run in batch, please use

        bash run_gen_visu_h5_without_jitters.sh
        bash run_gen_visu_h5_with_jitters.sh

Finally, we are ready to use a keyboard controlled agent to visualze the scene. Go to visu folder and run

        python keyboard_agent.py [scene_name]

Run the following to see more options. You can disable the overhead map, or specify a target observation (shown as red arrow in the map).

        python keyboard_agent.py --help

If the dataset is useful for your research, please consider cite the following paper:

    @article{Mo18AdobeIndoorNav,
        Author = {Kaichun Mo and Haoxiang Li and Zhe Lin and Joon-Young Lee},
        Title = {The AdobeIndoorNav Dataset: Towards Deep Reinforcement Learning based Real-world Indoor Robot Visual Navigation},
        Year = {2018},
        Eprint = {arXiv:1802.08824},
    }

MIT Licence