Giter VIP home page Giter VIP logo

sparse-segmentation's Introduction

Semantic Segmentation of Sparse Point Clouds using SqueezeSeg - RISS 2018

This repository contains:

  • A C++ ground annotator of point clouds based on: Fast segmentation of 3D point clouds: a paradigm on LIDAR data By Dimitris Zermas Izzat Izzat and Nikolas Papanikolopoulos
@inproceedings{7989591, 
	author={D. Zermas and I. Izzat and N. Papanikolopoulos}, 
	booktitle={2017 IEEE International Conference on Robotics and Automation (ICRA)}, 
	title={Fast segmentation of 3D point clouds: A paradigm on LiDAR data for autonomous vehicle applications}, 
	year={2017}, 
	pages={5067-5073}, 
	doi={10.1109/ICRA.2017.7989591}, 
	ISSN={}, 
	month={May}
}

  • Data downsampler to simulate VLP16 LIDAR scans from VLP64.
  • Data upsampler to simulate VLP32 LIDAR scans from VLP16.

These scans were obtained from: Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud By Bichen Wu, Alvin Wan, Xiangyu Yue, Kurt Keutzer SqueezeSeg

@article{wu2017squeezeseg,
    title={Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud},
    author={Wu, Bichen and Wan, Alvin and Yue, Xiangyu and Keutzer, Kurt},
    journal={arXiv preprint arXiv:1710.07368},
    year={2017}
}

Prerequisites

For the ground extraction algorithm (C++)
- Eigen3
- Boost
For the downsampler / upsampler converter (Python)
- Python 2.7

Process (example)

1. Convert .npy data to .txt

The ground annotation tool is implemented in c++ and currently it only takes text files as input, so it is necessary to convert the SqueezeSeg data which is in .npy format.

To do so, run:

cd $SPARSEG_ROOT/
python scripts/convert.py --inpath data/sample/squeeze/ --outpath data/sample/ --outdir textfiles --conv txt

It should create a folder called textfiles_v1 in data/sample/ where the converted point clouds are saved.

2. Annotate ground

Compile

mkdir & cd build
cmake ..
make 

To test if it works, run the following command:

cd $SPARSEG_ROOT/build
./extractGround

It should create a folder called g_textfiles1_1 in ./data/sample/ where it will save the annotated point clouds. The 'g' means that the files in the folder have been annotated.

Modifying parameters:

cd $SPARSEG_ROOT/build
./extractGround --inpath ../data/sample/textfiles_v1 --outpath ../data/sample/ --seg 4 --lpr 20 --iter 3 --thseed 0.8 --thdist 0.5 --method 0

Those are my prefered parameter values. However, if not modified, it will use the default values based on the original implementation. Check either the paper or the code to see what each parameter does.

Some results

Tested on a pointcloud with 64 scanlines (obtained from the KITTI dataset):

Tested on a pointclouds with 16 scanlines

3. Convert .txt data to .npy

After annotating the point clouds it is necessary to convert them back to the SqueezeSeg format (.npy) to train the network.

To do so, run:

cd $SPARSEG_ROOT/
python scripts/convert.py --inpath data/sample/g_textfiles1_1/ --outpath data/sample/ --outdir g_squeeze --conv npy

It should create a folder called g_squeeze1 in ./data/sample/ where it will save the converted files in the SqueezeSeg format.

4. Downsample data (pipeline 1)

The original point clouds from the SqueezeSeg approach were recorded using a Velodyne 64. The downsampler creates a point cloud scan with 16 and 32 rings based on the scans with 64 rings.

To downsample the point cloud, run:

cd $SPARSEG_ROOT/
python scripts/convert.py --inpath data/sample/g_squeeze1/ --outpath data/sample/ --outdir downsampled --conv down

It should create a folder called downsampler in ./data/sample/ with 4 subdirectories: 3 that represen 16-ring scans and 1 that represent the 32-ring scans.

Some results

Original 64-ring scan:

Downsampled 16-ring scan:

5. Upsample data (pipeline 2)

The upsampler creates a 32-ring scan based on a 16-ring scan.

To upsample the point cloud, run:

cd $SPARSEG_ROOT/
python scripts/convert.py --inpath data/sample/downsampled/vlp16m1/ --outpath data/sample/ --outdir upsampled --conv up

Some results

Original 16-ring scan:

Upsampled 32-ring scan:

It should create a folder called upsample1 in ./data/sample/ where the 32-ring point clouds are saved.

NOTE

More details about the pipelines can be found on my modified version of SqueezeSeg MySqueezeSeg.

TO DO:

  • Python version of the ground annotation algorithm
  • Upsample to 64-ring scans

sparse-segmentation's People

Contributors

navarrs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

sparse-segmentation's Issues

Problem in the upsampling process for computing Y and Z

Hi, thanking you for your great work!
I have some problems regarding the upsampling process.
In the line 212 to line 215 in scripts/convert.py, to compute the Y and Z you wrote:
`# get Y
interm[:,1] = xyzirl1[:,1] + d*(xyzirl2[:,1]-xyzirl1[:,1]) / D # y1 + d * (y2-y1) / D
'# get Z
interm[:,2] = xyzirl1[:,2] + d*(xyzirl2[:,2]-xyzirl1[:,2]) / D # z1 + d * (z2-z1) / D
I am wondering why don't you just use the average number instead?
interm[:,1] = (xyzirl1[:,1] + xyzirl2[:,1]) / 2
interm[:,2] = (xyzirl1[:,2] + xyzirl2[:,2]) / 2
Why it is not in the way that you compute the X, I, R?

Problem of applying the project to our own lidar data

Hi, I'm interested in the great project. I want to segment the ground rings of my own lidar data, but I can only acquire the xyz of the lidar data, and the label in another file, I don"t have the i,r value in the txt file, can the code handle this kind of the data? And I don't understand how to modify the src to fit my own data, could you please give me some suggestions? And I saw the below of the project show that it will come the python version of the project, if it completed, could you please provide me? Thank you very much for your kind help!

query regarding downsampling data from 64 to 32

Hi, I was using the convert.py/downSample utility to convert KITTI 64 resolution data to 32 resolution. If I am not mistaken, this function removes alternate(odd numbered) scanlines to convert. Is that right ?

I tried doing that and trained a model for semantic segmentation on 32 res data, converted as above. Although the model works on KITTI validation set. However, when I test on lidar data from veloview(I downloaded HDL-32 from here), the model does not generalize.

Can you please let me know if squeezeseg can work, in general, being trained on 32 res data obtained this way? Is this something you have already tried? Thanks for your help. :-)

Prateep

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.