cs3431-18s2-project's Introduction

COMP3431/COMP9431 18s2 Project

Speech and Person Recognition

Platform: TurtleBot3, ROS Kinetic, Ubuntu 16.04

Team Members: Ella, Hoang, Robin, Faris

Installation

The program requires some (very) small modification on TurtleBot3 Nagigation (ACML), and TurtleBot3 Application Follower packages, which you might already have in your catkin workspace.

If you want to keep the original TurtleBot3 pakages, you might want to check the modification. Otherwise, you could remove your TurtleBot3 packages and clone this repo --recursive (with the submodules):

git clone --recursive https://github.com/hoangp/cs3431-18s2-project.git

It also requires OpenPose installed on the Remote PC.

The TurtleBot3 need a Lidar and a Camera

For other installation instructions (TurtleBot3, ROS Kinetic, Ubuntu 16.04) Please refer to Robotis TurtleBot3 e-Manual

Setup

Below is the steps and command lines in our PC. It might be different in your PC.

Run roscore
Bringup TurtleBot3 Lidar and Camera

roslaunch turtlebot3_bringup turtlebot3_robot.launch
roslaunch realsense_camera r200_nodelet_default.launch

Run TurtleBot3 SLAM and Navigation

roslaunch turtlebot3_slam turtlebot3_slam.launch slam_methods:=gmapping
roslaunch turtlebot3_navigation person_recognition_navigation.launch

Run OpenPose listener

rosrun person_recognition listener

Run Follower scripts

roslaunch turtlebot3_follow_filter turtlebot3_follow_filter.launch
roslaunch turtlebot3_follower turtlebot3_follower.launch

Run our main scripts

cd scripts
python speech_cmd.py # speech recognition and send command to 'pr/cmd'
python run.py # excute command from topic 'pr/cmd'

cs3431-18s2-project's People

Contributors

Stargazers

Watchers

cs3431-18s2-project's Issues

Body Detect

Face Detect

Given few pictures of people with faces. We can identify how many faces

Speech Understanding

Order: "Meet xxx【name of someone】" (when training it to recognition)
Listen to trigger keywords "How are you"
Reply that the robot is listening
Listen to introduction "My name is ____" or "I am ____"
Record the name, and say "Hello ___. How are you"
During this: Trigger of remember the name of the person(match pictures: face+colour diagram)
Order: "Find xxx【name of someone】"
During this: Trigger