Peng Jiang¹, Philip Osteen², Maggie Wigness² and Srikanth Saripalli¹
1. Texas A&M University;  2. CCDC Army Research Laboratory
[Website] [Paper] [Github]

Semantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data; however, existing autonomy datasets represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in an off-road environment containing annotations for 13,556 LiDAR scans and 6,235 images. The data was collected on the Rellis Campus of Texas A&M University and presents challenges to existing algorithms related to class imbalance and environmental topography. Additionally, we evaluate the current state of the art deep learning semantic segmentation models on this dataset. Experimental results show that RELLIS-3D presents challenges for algorithms designed for segmentation in urban environments. Except for the annotated data, the dataset also provides full-stack sensor data in ROS bag format, including RGB camera images, LiDAR point clouds, a pair of stereo images, high-precision GPS measurement, and IMU data. This novel dataset provides the resources needed by researchers to develop more advanced algorithms and investigate new research directions to enhance autonomous navigation in off-road environments.

• Clearpath Robobtics Warthog

• 64 channels Lidar: Ouster OS1
• 32 Channels Lidar: Velodyne Ultra Puck
• 3D Stereo Camera: Nerian Karmin2 + Nerian SceneScan
• RGB Camera: Basler acA1920-50gc + Edmund Optics 16mm/F1.8 86-571
• Inertial Navigation System (GPS/IMU): Vectornav VN-300 Dual Antenna GNSS/INS

With the goal of providing multi-modal data to enhance autonomous off-road navigation, we defined an ontology of object and terrain classes, which largely derives from the RUGD dataset but also includes unique terrain and object classes not present in RUGD. Specifically, sequences from this dataset includes classes such as mud, man-made barriers, and rubble piles. Additionally, this dataset provides a finer-grained class structure for water sources, i.e., puddle and deep water, as these two classes present different traversability scenarios for most robotic platforms. Overall, 20 classes (including void class) are present in the data.

Ontology Definition (Download 18KB)

Image with Annotation Examples (Download 3MB)

Full Images (Download 11GB)

Full Image Annotations (Download 94MB)

Image Split File (44KB)

LiDAR with Annotation Examples (Download 24MB)

LiDAR with Color Annotation PLY Format (Download 26GB)

The header of the PLY file is described as followed:

element vertex
property float x
property float y
property float z
property float intensity
property uint t
property ushort reflectivity
property uchar ring
property ushort noise
property uint range
property uchar label
property uchar red
property uchar green
property uchar blue

To visualize the color of the ply file, please use CloudCompare or Open3D. Meshlab has problem to visualize the color.

LiDAR SemanticKITTI Format (Download 14GB)

To visualize the datasets using the SemanticKITTI tools, please use this fork: https://github.com/unmannedlab/point_labeler

LiDAR Annotation SemanticKITTI Format (Download 174MB)

LiDAR Scan Poses files (Download 174MB)

LiDAR Split File (75KB)

Camera Instrinsic (Download 2KB)

Camera to LiDAR (Download 3KB)

To reproduce the results, please refer to here

Data included in raw ROS bagfiles:

ROS Bag Examples (2GB)

Sequence 00000: Synced data: (12GB) Filtered data: (23GB) Full-stack Raw data: (29GB)

Sequence 00001: Synced data: (8GB) Filtered data: (16GB) Full-stack Raw data: (22GB)

Sequence 00002: Synced data: (14GB) Filtered data: (28GB) Full-stack Raw data: (37GB)

Sequence 00003:Synced data: (8GB) Filtered data: (15GB) Full-stack Raw data: (19GB)

Sequence 00004:Synced data: (7GB) Filtered data: (14GB) Full-stack Raw data: (17GB)

The ROS workspace includes a plaftform description package which can provide rough tf tree for running the rosbag.

Access Link

@misc{jiang2020rellis3d,
      title={RELLIS-3D Dataset: Data, Benchmarks and Analysis}, 
      author={Peng Jiang and Philip Osteen and Maggie Wigness and Srikanth Saripalli},
      year={2020},
      eprint={2011.12954},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

All datasets and code on this page are copyright by us and published under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.

• 11/26/2020 v1.0 release

SemanticUSL: A Dataset for Semantic Segmentation Domain Adatpation

LiDARNet: A Boundary-Aware Domain Adaptation Model for Lidar Point Cloud Semantic Segmentation