Giter VIP home page Giter VIP logo

ndx-pose's Introduction

ndx-pose Extension for NWB

PyPI version

ndx-pose is a standardized format for storing pose estimation data in NWB. It was developed initially to store the output of DeepLabCut in NWB, but is also designed to store the output of general pose estimation tools. Please post an issue or PR to suggest or add support for your favorite pose estimation tool.

This extension consists of two new neurodata types:

  • PoseEstimationSeries which stores the estimated positions (x, y) or (x, y, z) of a body part over time as well as the confidence/likelihood of the estimated positions.
  • PoseEstimation which stores the estimated position data (PoseEstimationSeries) for multiple body parts, computed from the same video(s) with the same tool/algorithm.

Installation

pip install ndx-pose

Usage

import datetime
import numpy as np
from pynwb import NWBFile, NWBHDF5IO
from ndx_pose import PoseEstimationSeries, PoseEstimation

nwbfile = NWBFile(
    session_description='session_description',
    identifier='identifier',
    session_start_time=datetime.datetime.now(datetime.timezone.utc)
)

camera1 = nwbfile.create_device(
    name='camera1',
    description='left camera',
    manufacturer='my manufacturer'
)
camera2 = nwbfile.create_device(
    name='camera2',
    description='right camera',
    manufacturer='my manufacturer'
)

data = np.random.rand(100, 3)  # num_frames x (x, y, z)
timestamps = np.linspace(0, 10, num=100)  # a timestamp for every frame
confidence = np.random.rand(100)  # a confidence value for every frame
front_left_paw = PoseEstimationSeries(
    name='front_left_paw',
    description='Marker placed around fingers of front left paw.',
    data=data,
    unit='pixels',
    reference_frame='(0,0,0) corresponds to ...',
    timestamps=timestamps,
    confidence=confidence,
    confidence_definition='Softmax output of the deep neural network.',
)

data = np.random.rand(100, 2)  # num_frames x (x, y)
timestamps = np.linspace(0, 10, num=100)  # a timestamp for every frame
confidence = np.random.rand(100)  # a confidence value for every frame
front_right_paw = PoseEstimationSeries(
    name='front_right_paw',
    description='Marker placed around fingers of front right paw.',
    data=data,
    unit='pixels',
    reference_frame='(0,0,0) corresponds to ...',
    timestamps=front_left_paw,  # link to timestamps of front_left_paw
    confidence=confidence,
    confidence_definition='Softmax output of the deep neural network.',
)

pose_estimation_series = [front_left_paw, front_right_paw]

pe = PoseEstimation(
    pose_estimation_series=pose_estimation_series,
    description='Estimated positions of front paws using DeepLabCut.',
    original_videos=['camera1.mp4', 'camera2.mp4'],
    labeled_videos=['camera1_labeled.mp4', 'camera2_labeled.mp4'],
    dimensions=np.array([[640, 480], [1024, 768]], dtype='uint8'),
    scorer='DLC_resnet50_openfieldOct30shuffle1_1600',
    source_software='DeepLabCut',
    source_software_version='2.2b8',
    nodes=['front_left_paw', 'front_right_paw'],
    edges=np.array([[0, 1]], dtype='uint8'),
    # devices=[camera1, camera2],  # this is not yet supported
)

behavior_pm = nwbfile.create_processing_module(
    name='behavior',
    description='processed behavioral data'
)
behavior_pm.add(pe)

path = 'test_pose.nwb'
with NWBHDF5IO(path, mode='w') as io:
    io.write(nwbfile)

with NWBHDF5IO(path, mode='r', load_namespaces=True) as io:
    read_nwbfile = io.read()
    read_pe = read_nwbfile.processing['behavior']['PoseEstimation']
    print(read_pe)

Contributors

  • @rly
  • @bendichter
  • @AlexEMG

This extension was created using ndx-template.

ndx-pose's People

Contributors

rly avatar bendichter avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.