A multi-modal Human Activities of Daily Living dataset and Data Collection Tool.

• Dataset
  • Dataset properties
  • Playing depth video
  • Skeleton data stream
• Data Collection Tool
  • Using the tool
    • Setting up dependencies
    • Running the tool
• Licence
• Citaton
• Collaborators

Mora-ADL Dataset

7 data streams:

• Depth image stream
• RGB image stream
• Skeleton data stream
• Silhouette stream
• 3 audio streams

24 Activities of Daily Living:

• Making a phone call, Clapping, Drinking, Eating, Entering from door, Exiting from door, Falling, Lying down, Opening pill container, Picking object, Reading, Sit still, Sitting down, Sleeping, Standing up, Sweeping, Using laptop, Using phone, Wake up, Walking, Washing hand, Watching TV, Water pouring and Writing

17 subjects:

• 11 males
• 6 females

Continuous dataset to test continuous activity classification.

• Performed by 3 subjects
• In 2 different environments

Run playDepthVideo.py with the following arguments to play a depth video file.

• -d depth file path
• -s skeleton file path
• -sil (Optional) True/False [True to view silhouette view. False to view depth image view]

The skeleton data stream has 5 values per row and for each frame it has 15 such rows. The 15 rows gives the positions of the 15 joint positions as in the below figure.

The 5 values of the row stands for:

• X coordinate using the "real world" coordinate system
• Y coordinate using the "real world" coordinate system
• Z coordinate using the "real world" coordinate system
• X coordinates of the depth map
• Y coordinates of the depth map

Data collection tool written using python to collect depth, RGB, audio, silhouette and skeleton data using the Microsoft Kinect device and the OpenNI/NiTE tool.

Please follow the following steps if you have not installed Kinect SDK, OpenNI or NiTE tool.

1. Download & install Kinect SDK 1.8 or higher version from here or another source.
2. Download & install OpenNI 2.2 or higher version from here or another source.
3. To verify the setup run SimpleViewer in Program files -> OpenNI2 -> Samples -> Bin
4. Download & install Nite 2.2 or higher version from here or another source.
5. To verify the setup run UserViewer in Program files -> PrimeSense -> NiTE2 ->Samples -> Bin

1. Download openni-python repository from here.
2. Extract it to a preferred location.
3. Copy NiTE2 and OpenNI2 folders in Program files -> PrimeSense -> NiTE2 -> Samples to the openni-python.

1. Copy the dataset tool (dataCollectionTool.py) and audioDevices.py to the openni-python folder.

First, run the audioDevices.py to detect the port numbers of the microphones.

Then run dataCollectionTool.py in the command line using the following command.

$ python audioDevices.py -d <<microphone_ports>> -p <<location_to_save_data>> -s <<subject_name>> -a <<act_name>> 

To stop recording press Ctrl + c in the command line.

Creative Commons Zero v1.0 Universal

If you use the Mora-ADL dataset or the Mora-ADL Data Collection Tool please cite:

Madhuranga, D., Madushan, R., Siriwardane, C. et al. Real-time multimodal ADL recognition using convolution neural networks. Vis Comput (2020). https://doi.org/10.1007/s00371-020-01864-y

C. Siriwardhana, D. Madhuranga, R. Madushan and K. Gunasekera, "Classification of Activities of Daily Living Based on Depth Sequences and Audio," 2019 14th Conference on Industrial and Information Systems (ICIIS), Kandy, Sri Lanka, 2019, pp. 278-283, doi: 10.1109/ICIIS47346.2019.9063306.

Rivindu Madushan

Danushkka Madhuranga

Chathuranga Siriwardhana