Code for reproducing all results in our paper, which can be found here
Additional results can be found here

• Pytorch 0.4.1/0.4.0
• TensorboardX
• Mayavi (for display only)

├── Evaluation 
    ├── eval.py             # Evaluation of conditional generation, using Chamfer and EMD metrics
    ├── generate.py         # Generate point clouds from a pretrained model, both for clean and corrupted input        
├── Launch Scripts          
    ├── vae_rs.py           # launch the hyperparameter search used for the proposed VAE models
    ├── baseline_rs.py      # launch the hyperparameter search used for the baseline models
├── Trained Models  
    ├── ....                # contains the weights of the models (ours and baseline) used in the paper
├── Training Logs
    ├── ....                # Contains all the tensorboard logs for all the model hyperparameter searches
├── gan_2d.py               # Training file for GAN model
├── vae_2d.py               # Training file for the VAE models
├── models.py               # Implementation of all the model architectures used in the paper
├── utils.py                # Utilies for preprocessing and visualitation 

Models are logged according to the following file sturcture, where the root is specified using the --base_dir flag

├── <base_dir>
    ├── TB/
        ├── ...             # Contains the Tensorboard logs 
    ├── models/  
        ├── ...             # Contains the saved model weights, stored as `.pth` files.
├── args.json               # List of all the hyperparameters used for training.     

We provide the full list of commands to replicate all of our results.

The general command is python vae_2d.py <flags> e.g.
python vae_2d.py --z_dim=256 --batch_size=64 --kl_warmup_epochs=100

To get more information regarding the different flags, you can run python vae_2d.py --help
To replicate the results from a specific model, simply provide the hyperparemeter values listed in trained_models/<model_you_want>/args.json

Similarly, you can train a GAN using the following command python gan_2d.py <flags> e.g.
python gan_2d.py --optim=Adam --batch_size=64 --loss=1

To get more information regarding the different flags, you can run python gan_2d.py --help

To evaluate a AE/VAE on the clean, noisy and corrupted tasks, run
python evaluation/eval.py <path_to_base_dir_of_trained_model> <epoch_#_to_load> <emd/chamfer> e.g.
python evaluation/eval.py trained_models/ae_xyz 209 chamfer,
which will print the reconstruction results and the noise std /missing data percentage

Additionally, we provide the tensorboard logs for all models trained in the hyperparameter search. To see them in tensorboard, run
tensorboard --logdir=trained_models/

Our model learns a compressed representation of the input. Here we encode into a 512 dimensional vector, (60x smaller than original input) and decode it back.

Original Lidar	Reconstruction from compressed (60x smaller) encoding

Surprisingly, our approch is highly robust to noisy input. Here we added gaussian noise on the input (shown on the left) rendering the lidar uninformative to the human eye. Yet, the model is able to reconstruct the point cloud with little performance loss. We note that the model was not trained on noisy data :O

Noisy input	Reconstruction from noisy input

Here we repeat the same corruption process, but with even more noise

Noisy input	Reconstruction from noisy input

Thanks to Fxia2 for his NNDistance module. Thanks to Thibault GROUEIX and Panos Achlioptas for open sourcing their code.
Thanks to Alexia JM for her open source code on Relativistic GANs. Please check out her work if you are working with GANs!
Thanks to Alexandre Bachaalani for his video editing help!

If you find this code useful please cite us in your work.

@article{caccia2018deep,
  title={Deep Generative Modeling of LiDAR Data},
  author={Caccia, Lucas and van Hoof, Herke and Courville, Aaron and Pineau, Joelle},
  journal={2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year={2018}
}