Below steps are performed to achieve vehicle detection and tracking in a video stream .

• Feature Extraction - apply five techniques, including histograms of color, color distribution, spatial binning, gradient magnitude, and Histogram of Oriented Gradients (HOG), on the labeled training set of images to create a feature vector.
• Preprocessing Data - normalize, randomize, and split the labeled data into a training set, a validation set, and a testing set.
• Training - train a Linear SVM classifier on the labeled training set
• Sliding Windows - implement a technique to search an image for vehicles using the trained classifier, and optimize the algorithm's efficiency by limiting the search area of the image and/or using heat maps that reject outliers of the positive windows.
• Video - run a function using moviepy that estimates a bounding box for detected vehicles frame by frame.

Detailed description is given below .

Several different techniques for feature extraction were used in Section 2 of this project, including histograms of color, color distribution, spatial binning, gradient magnitude, and Histogram of Oriented Gradients (HOG). Each has its own effect on the feature vector that is produced, and when combined the techniques tend to improve the chosen classifier's performance.

Histograms of Color

Histograms of color are used to analyze the raw pixel intensities of each color channel as features in the image. Given an image in RGB color space, color histograms of features return a concatenated feature vector based on the number of intensity bins and pixel intensity ranges.

Color Distribution

Color distribution maps are useful in studying color values and locating clusters of color that correspond to objects of interest in an image. When plotting these maps in 3D, often the color distribution of objects of interest is conveniently clustered along a plane.

Spatial Binning

Raw pixel values are useful to include in the feature vector when the desired objects in an image remain identifiable at low image resolutions, such as a car object.

Gradient Magnitude

Gradient magnitude is a technique used in previous computer vision projects (Projects 1 & 4) that applies a filter that represents the magnitude of the sobel-x and sobel-y gradients of odd-numbered pixel squares, such as 3x3 or 5x5.

Histogram of Oriented Gradients (HOG)

HOG feature extraction is the most important technique utilized in this project. The scikit-image package has a built in function to handle HOG extraction, which is tuned by parameters including orientations, pixels_per_cell, and cells_per_block.

orientations : I tried with both 8 and 9 , later find there is no such huge difference , so kept it with 8 . pixels_per_cell and cells_per_block : I kept these values unchanged what I tried earlier in scikit-image HOG tutorial .This gave a good result .

The final feature extraction method that was implemented includes color histograms, spatial binning, and HOG . I have converted BGR to YUV representation in the final step of feature extraction .I choose it over other because , image/video codecs are YUV so they can reduce the resolution of the U and V channels while keeping Y at full resolution, because luminance is more important than color .

The training data was normalized, randomized, and splitted into training and testing sets, with 20% of the data segregated for the testing set.

After exploring feature extraction techniques, I trained a SVC classifier to classify the dataset as "vehicle" or "not-vehicle". The accuracy for this classifier was above 94% for multiple iterations . I analyzed two different feature vectors and ran them through the classfier to produce the following results:

Classification by Color and Spatial Binning

Classification by HOG

After comparing these two results, I decided to encorporate both of them as one feature vector that I retrained prior to running sliding windows on the test image, as explained next.

P5_Vehicle_Detection_and_Tracking.ipynb outlines the sliding windows approach to search for vehicles in a test image with trained SVC classifier. Scale at 1 seems to be performed well. I limited the search area to the bottom half of the image, minus the bottom 10%, for a total of 40% of the image to lower the total number of windows I needed to make.

The parameters I chose for the sliding windows themselves are as follows:

These parameters produced the following results: The overlap of 0.8 allowed for a strong signature of positive window values on the actual vehicle that simplified the process of ruling out false positives with heat maps, as shown in the image below:

After recording the positions of positive detections in each frame of the video, I created a heatmap and thresholded it to identify vehicle positions. I then used scipy.ndimage.measurements.label() to identify individual blobs in the heatmap.

Below is the url to the final video output .

https://youtu.be/9golUDcCFx0

Getting the right combination of parameters to make the code work on the video pipeline was a bit challenging .Feature extraction played a major role and combining results of different feature extractors was a good learning point .I observed HOG and spatial binning are very crucial for this project .

Two more things are the sliding window technique and removing false positives from the video frames also very important .

My future course of action would be to implement the same with deep learning object detection and recognition techniques , which is more robust .

It's a new learning experience all together with different types of features from the images .

Thank you !