It is a sad result for our team.
One of Udacity's requirements, "Stop at traffic lights as needed," was not evaluated.
The reason is that the traffic light in the testing lot did not work as shown in the figure below.
Video | rosbag /image_color |
---|---|
But that is true, and it is a sad result.
We limit the velocity toward the stop line to approximately 10 km/h and improve the detection performance for blurred images.
- Change
MAX_DECEL
from 5.0 to 0.5 inwaypoint_updater.py
. - Change
real_model.h5
used bytl_detector.py
(but not included in repository).
MAX_DECEL and decelerate_vel |
real_model.h5 (1.1.0) |
---|---|
The car did not stop even though the traffic light was red.
The cause is that the blurred traffic lights are can not be detected when the velocity exceeds about 10 km/h on the way to /traffic_waypoint
.
/current_velocity and /traffic_waypoint |
real_model.h5 (1.0.0) |
---|---|
Night | Day |
---|---|
- Hidetoshi Furukawa: xdymg975 (at) gmail.com (team lead)
- João Gonçalves: miguel.joao.goncalves (at) gmail.com
- Liangli Fei: fei.liangli.info (at) gmail.com
- Yuji Kawamura: yujika2019 (at) gmail.com
- Junxun Luo: luojunxun (at) gmail.com
- Launch correctly using the launch files provided in the capstone repo.
- Smoothly follow waypoints in the simulator.
- Respect the target top speed set for the waypoints'
twist.twist.linear.x
inwaypoint_loader.py
. - Stop at traffic lights when needed.
- Stop and restart PID controllers depending on the state of
/vehicle/dbw_enabled
. - Publish throttle, steering, and brake commands at 50Hz.
The following is a system architecture diagram showing the ROS nodes and topics used in the project. The ROS nodes and topics shown in the diagram are described briefly in the Code Structure section below.
Below is a brief overview of the repo structure, along with descriptions of the ROS nodes.
This package contains the traffic light detection node: tl_detector.py
.
This node takes in data from the /image_color
, /current_pose
, and /base_waypoints
topics and publishes the locations to stop for red traffic lights to the /traffic_waypoint
topic.
The /current_pose
topic provides the vehicle's current position, and /base_waypoints
provides a complete list of waypoints the car will be following.
We build both a traffic light detection node and a traffic light classification node. Traffic light detection should take place within tl_detector.py
, whereas traffic light classification should take place within ../tl_detector/light_classification_model/tl_classfier.py
.
See code in /ros/src/tl_detector/.
This package contains the waypoint updater node: waypoint_updater.py
.
See code in /ros/src/waypoint_updater/.
Carla is equipped with a drive-by-wire (DBW) system, meaning the throttle, brake, and steering have electronic control.
This package contains the files that are responsible for control of the vehicle: the node dbw_node.py
and the file twist_controller.py
.
See code in /ros/src/twist_controller/.
A package that contains a server for communicating with the simulator, and a bridge to translate and publish simulator messages to ROS topics.
A package which includes definitions of the custom ROS message types used in the project.
A package which loads the static waypoint data and publishes to /base_waypoints
.
A package containing code from Autoware which subscribes to /final_waypoints
and publishes target vehicle linear and angular velocities in the form of twist commands to the /twist_cmd
topic.
We use Tiny YOLOv3 for traffic light detection and classification.
- Download Tiny YOLOv3 (also called YOLOv3-tiny) weight from YOLO website
- Convert the Darknet YOLO model to a Keras model using keras-yolo3.
- Transfer learning for traffic light detection.
Loss | Val Loss |
---|---|
Graph of the Tiny YOLOv3 model:
We use YOLOv3 for traffic light detection and classification.
YOLOv3 (open youtube video on click):
Tiny YOLOv3 (open youtube video on click):
This is the project repo for the final project of the Udacity Self-Driving Car Nanodegree: Programming a Real Self-Driving Car. For more information about the project, see the project introduction here.
Please use one of the two installation options, either native or docker installation.
-
Be sure that your workstation is running Ubuntu 16.04 Xenial Xerus or Ubuntu 14.04 Trusty Tahir. Ubuntu downloads can be found here.
-
If using a Virtual Machine to install Ubuntu, use the following configuration as minimum:
- 2 CPU
- 2 GB system memory
- 25 GB of free hard drive space
The Udacity provided virtual machine has ROS and Dataspeed DBW already installed, so you can skip the next two steps if you are using this.
-
Follow these instructions to install ROS
- ROS Kinetic if you have Ubuntu 16.04.
- ROS Indigo if you have Ubuntu 14.04.
-
- Use this option to install the SDK on a workstation that already has ROS installed: One Line SDK Install (binary)
-
Download the Udacity Simulator.
Build the docker container
docker build . -t capstone
Run the docker file
docker run -p 4567:4567 -v $PWD:/capstone -v /tmp/log:/root/.ros/ --rm -it capstone
To set up port forwarding, please refer to the instructions from term 2
- Clone the project repository
git clone https://github.com/udacity/CarND-Capstone.git
- Install python dependencies
cd CarND-Capstone
pip install -r requirements.txt
- Make and run styx
cd ros
catkin_make
source devel/setup.sh
roslaunch launch/styx.launch
- Run the simulator
- Download training bag that was recorded on the Udacity self-driving car.
- Unzip the file
unzip traffic_light_bag_file.zip
- Play the bag file
rosbag play -l traffic_light_bag_file/traffic_light_training.bag
- Launch your project in site mode
cd CarND-Capstone/ros
roslaunch launch/site.launch
- Confirm that traffic light detection works on real life images