Behavioral Cloning Project

The goals / steps of this project are the following:

• Use the simulator to collect data of good driving behavior
• Build, a convolution neural network in Keras that predicts steering angles from images
• Train and validate the model with a training and validation set
• Test that the model successfully drives around track one without leaving the road
• Summarize the results with a written report

Using the Udacity provided simulator and my drive.py file, the car can be driven autonomously around the track by executing

python drive.py model.h5

The model.py file contains the code for training and saving the convolution neural network. The file shows the pipeline I used for training and validating the model, and it contains comments to explain how the code works.

The data Preprocessing and argumentation are in data generator.ipynb

The distribution of data is not good to use to train as the histogram

At the beginning, I throw it into training. The car's behavior don't know how to afford the sharp turn. Therefore, I do some modification for the distribution of data.

1. add the different camera view including the correction of steering
2. throw out the data of the steering in zero (not including number of close zero )
3. throw out the data of the car speed less than 15
4. blur the steering by multiplying a random number 0-0.1 based on the steering value itself.

The reason is the data of the steering in zero is very non-useful to learn, and, in playing, the actual steering is not only one solution. We may choose blue direction to keep or slightly modify the direction or maybe the red direction making the car back to road center. Both of the solution are the "right" direction in playing.

The result looks very good:

Before augmentation I play again with inverse race. Applying some augmentations like:

1. flip the image
2. random the brightness and vertical shift two times. (mean the data would be double)

So the data size become the 4 times of the original size. The final size of X_train_easy data for simple race is up to 1.6G and for complicated race is up to 10 G!

My first step was to use a LeNnet convolution neural network that we used before, but it doen't work well enough. Sometimes, it doen't distinguish the edge well. Then I apply the NVIDIA self driving car network model. Initially, it already work very well in the simple race. And, I try it into complicated race. I found it still doen't distinguish the special stuffs that around the road as:

I added two dropout layers, and cropped the image from 160x320 to 76x320 leaving the useful range. The model become very reliable and also in the simple race.

Here is my visualization of the architecture which manly referred NVIDIA self driving car network model

I completed the complicated race, though it still not very stable in some corner I think. The validation accuracy only 0.9. I have tried more epochs, but it didn't give significant improvements.

The result are in the video of run1.mp4 for simple race and run2.mp4 for the complicated race. (I didn't upload the pictures of run2)

I also upload the video in player perspective to youtube as: https://youtu.be/ely6-2Rqak0