Source: Segway

We implemented a person tracking pipeline based on video. This pipeline was used in a human-robot race with a Loomo Robot following a person around a track based on the camera feed. A specific hand gesture is all that is needed to be selected as the person of interest to be then detected, tracked and followed by the robot. The main challenge consists of guiding the robot through many people without loosing the person of interest.

The complete pipeline and implementation details are explained in the report.

The first milestone implements a detector, for each frame it detects specific hand gestures as well as people in the frame.

The second milestone add memory between frame by tracking a person that made the specific hand gesture in a previous frame (green vs white bounding box)

The third and last milestone adapts the previous milestone to be used as the driving unit for the Loomo robot. The tracking pipeline is running on the camera feed of the robot. The bounding box is then used as a target for the control unit of the robot.

The first two milestones have a noteboob version. Please note that most of the delay comes from rendering the frame using javascript, indeed the local python script is smoother.

For all the milestones the specific hand gesture is a triangle with both hands.