Giter VIP home page Giter VIP logo

ndrplz / self-driving-car Goto Github PK

View Code? Open in Web Editor NEW
2.7K 149.0 1.4K 152.66 MB

Udacity Self-Driving Car Engineer Nanodegree projects.

Python 0.66% Jupyter Notebook 1.93% CMake 1.50% C++ 84.33% C 1.85% Shell 0.14% Makefile 0.05% Cuda 0.86% Fortran 8.61% JavaScript 0.05% CSS 0.03%
self-driving-car udacity-self-driving-car deep-learning deep-neural-networks kalman-filter lecture-material particle-filter lane-finding computer-vision vehicle-detection convolutional-neural-networks detect-lane-lines lane-detection pid-control traffic-sign-classification tensorflow

self-driving-car's Introduction

self-driving-car

In this repository I will share the source code of all the projects of Udacity Self-Driving Car Engineer Nanodegree.

Hope this might be useful to someone! :-)

Overview

Projects

Overview
P1: Basic Lane Finding
(code)

Overview
P2: Traffic Signs
(code)

Overview
P3: Behavioral Cloning
(code)

Overview
P4: Adv. Lane Finding
(code)

Overview
P5: Vehicle Detection
(code)

Overview
P6: Ext. Kalman Filter
(code)

Overview
P7: Unsc. Kalman Filter
(code)

Overview
P8: Kidnapped Vehicle
(code)

Overview
P9: PID Controller
(code)

Overview
P10: MPC Controller
(code)

Overview
P11: Path Planning
(code)

Overview
P12: Road Segmentation
(code)

Capstone


Table of Contents

  • Summary: Detected highway lane lines on a video stream. Used OpencV image analysis techniques to identify lines, including Hough Transforms and Canny edge detection.
  • Keywords: Computer Vision
  • Summary: Built and trained a deep neural network to classify traffic signs, using TensorFlow. Experimented with different network architectures. Performed image pre-processing and validation to guard against overfitting.
  • Keywords: Deep Learning, TensorFlow, Computer Vision
  • Summary: Built and trained a convolutional neural network for end-to-end driving in a simulator, using TensorFlow and Keras. Used optimization techniques such as regularization and dropout to generalize the network for driving on multiple tracks.
  • Keywords: Deep Learning, Keras, Convolutional Neural Networks
  • Summary: Built an advanced lane-finding algorithm using distortion correction, image rectification, color transforms, and gradient thresholding. Identified lane curvature and vehicle displacement. Overcame environmental challenges such as shadows and pavement changes.
  • Keywords: Computer Vision, OpenCV
  • Summary: Created a vehicle detection and tracking pipeline with OpenCV, histogram of oriented gradients (HOG), and support vector machines (SVM). Implemented the same pipeline using a deep network to perform detection. Optimized and evaluated the model on video data from a automotive camera taken during highway driving.
  • Keywords: Computer Vision, Deep Learning, OpenCV
  • Summary: Implement the extended Kalman filter in C++. Simulated lidar and radar measurements are used to detect a bicycle that travels around your vehicle. Kalman filter, lidar measurements and radar measurements are used to track the bicycle's position and velocity.
  • Keywords: C++, Kalman Filter
  • Summary: Utilize an Unscented Kalman Filter to estimate the state of a moving object of interest with noisy lidar and radar measurements. Kalman filter, lidar measurements and radar measurements are used to track the bicycle's position and velocity.
  • Keywords: C++, Kalman Filter
  • Summary: Your robot has been kidnapped and transported to a new location! Luckily it has a map of this location, a (noisy) GPS estimate of its initial location, and lots of (noisy) sensor and control data. In this project you will implement a 2 dimensional particle filter in C++. Your particle filter will be given a map and some initial localization information (analogous to what a GPS would provide). At each time step your filter will also get observation and control data.
  • Keywords: C++, Particle Filter
  • Summary: Implement a PID controller for keeping the car on track by appropriately adjusting the steering angle.
  • Keywords: C++, PID Controller
  • Summary: Implement an MPC controller for keeping the car on track by appropriately adjusting the steering angle. Differently from previously implemented PID controller, MPC controller has the ability to anticipate future events and can take control actions accordingly. Indeed, future time steps are taking into account while optimizing current time slot.
  • Keywords: C++, MPC Controller
  • Summary: The goal in this project is to build a path planner that is able to create smooth, safe trajectories for the car to follow. The highway track has other vehicles, all going different speeds, but approximately obeying the 50 MPH speed limit. The car transmits its location, along with its sensor fusion data, which estimates the location of all the vehicles on the same side of the road.
  • Keywords: C++, Path Planning
  • Summary: Implement the road segmentation using a fully-convolutional network.
  • Keywords: Python, TensorFlow, Semantic Segmentation

self-driving-car's People

Contributors

ndrplz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

self-driving-car's Issues

project_4 : unsupported pickle protocol: %d" % proto

Hello,

I'm stuck in this issue when I run project 4:

$ python ./main.py
Loading cached camera calibration... Traceback (most recent call last):
File "./main.py", line 135, in
ret, mtx, dist, rvecs, tvecs = calibrate_camera(calib_images_dir='camera_cal')
File "/home/nxnam/DigitalRace/hanson/self-driving-car/project_4_advanced_lane_finding/calibration_utils.py", line 20, in wrapper
calibration = pickle.load(dump_file)
File "/usr/lib/python2.7/pickle.py", line 1384, in load
return Unpickler(file).load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatchkey
File "/usr/lib/python2.7/pickle.py", line 892, in load_proto
raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3

How can I fix this?

Thank you.

Under Lane-changing situations

Hi, Thank you for sharing this wonderful repo!

Just one question: will the lane-detection work the same when the vehicle is lane-changing? how can the left and right lane be detected?

Also, if the road's lane mark is a little bit unclear, will this framework still works?

Advance Lane Detection - TypeError: 'NoneType' object is not subscriptable

def draw_back_onto_the_road(img_undistorted, Minv, line_lt, line_rt, keep_state):
    height, width, _ = img_undistorted.shape

    left_fit = line_lt.average_fit if keep_state else line_lt.last_fit_pixel
    right_fit = line_rt.average_fit if keep_state else line_rt.last_fit_pixel
    left_fit.sort()
    right_fit.sort()
    # Generate x and y values for plotting
    ploty = np.linspace(0, height - 1, height)
    left_fitx = left_fit[0] * ploty ** 2 + left_fit[1] * ploty + left_fit[2]
    right_fitx = right_fit[0] * ploty ** 2 + right_fit[1] * ploty + right_fit[2]

Error

left_fitx = left_fit_pixel[0] * ploty ** 2 + left_fit_pixel[1] * ploty + left_fit_pixel[2]
TypeError: 'NoneType' object is not subscriptable

Someone please help me with this

Question 1: Data augmentation

Hello sir,
Here shouldn't be X_train instead of X_train_norm inside train_test_split()?
X_train_norm, X_val_norm, y_train, y_val = train_test_split(X_train_norm, y_train, test_size=VAL_RATIO, random_state=0)
Anyway, here img_rgb = X_train[0], you used X_train[0] which will always give me the first picture of the training dataset! will never be random.
And here: plt.title('Example of RGB image (class = {})'.format(y_train[0])), you used the new training labels (after splitation).

Not sure if you got my point, but it is not correct, I am new to python so I couldn't figure out how to fix the issue here, so any help?

Trained model

Do you have some trained model for project 12 I can use directly?

How to run project 3 with my own images and control

I would like to run Project 3 - Behavioral Cloning but with my own images and control. How can I do it? It was not very clear which is the training steep, and which is the inference step. Also, what is the best way to replace the data with my own images and control? My control data also have throttle data, but I can start with just steering.

project_12_road_segmentation -- RuntimeError: MetaGraphDef associated with tags 'vgg16' could not be found in SavedModel

Hello,

When I run project_12_road_segmentation with python main.py, I can't get the expected results.
It prompts as

Traceback (most recent call last):
  File "main.py", line 225, in <module>
    run()
  File "main.py", line 176, in run
    image_input, keep_prob, vgg_layer3_out, vgg_layer4_out, vgg_layer7_out = load_vgg(sess, vgg_path)
  File "main.py", line 39, in load_vgg
    tf.saved_model.loader.load(sess, ['vgg16'], vgg_path)
  File "/Users/xxx/.pyenv/versions/3.6.6/lib/python3.6/site-packages/tensorflow/python/saved_model/loader_impl.py", line 209, in load
    "[]") + " could not be found in SavedModel")
RuntimeError: MetaGraphDef associated with tags 'vgg16' could not be found in SavedModel

What is vgg16? How can I get it?
Thank you so much.

Advanced Lane Finding Lane Offset

I have worked for hours and hours with your code. It is wonderful! Thanks for sharing it. However, one thing that makes it hard to implement in an actual bot is the fact that the road curvature and offset values are always positive. Could you please help me make them negative if the road curves, for example, left, and positive if it curves right? Same for the offset within the lane. My self-driving motercycle needs to know which way it is drifting.
Thanks so much!

After using the command cmake -G "Unix Makefiles" && make in the build folder

After using the command cmake -G "Unix Makefiles" && make in the build path at the cmd, I have the following error. I have CMake, Make installed and g++ with MinGW
Error:

CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

Distinguishing between dashed and solid lanes

In project_4_advanced_lane_finding, it shows the blue line is for the dashed lane and the red line is for the solid lane. Is the code able to distinguish between the dashed and solid street lanes ?????

ENet-Label-Torch is available now (a light-weight and effective lane detection model)

Our ENet-Label-Torch has been released. More details can be found in my repo.

Key features:

(1) ENet-label is a light-weight lane detection model based on ENet and adopts self attention distillation (more details can be found in our paper which will be published soon).

(2) It has 20 ร— fewer parameters and runs 10 ร— faster compared to the state-of-the-art SCNN, and achieves 72.0 (F1-measure) on CULane testing set (better than SCNN which achieves 71.6).

(Do not hesitate to try our model!!!)

Performance on CULane testing set (F1-measure):

Category SCNN-Torch SCNN-Tensorflow ENet-Label-Torch
Normal 90.6 90.2 90.7
Crowded 69.7 71.9 70.8
Night 66.1 64.6 65.9
No line 43.4 45.8 44.7
Shadow 66.9 73.8 70.6
Arrow 84.1 83.8 85.8
Dazzle light 58.5 59.5 64.4
Curve 64.4 63.4 65.4
Crossroad 1990 4137 2729
Total 71.6 71.3 72.0
Runtime(ms) 133.5 -- 13.4
Parameter(M) 20.72 -- 0.98

Following datas are needed.

You have used train.p and test.p. But I did not find your repository.
Please, give these datasets.

# Load pickled data
train, test = load_traffic_sign_data('../traffic_signs_data/train.p', '../traffic_signs_data/test.p')

Where is the VGG pretrained model in Project 12

I got this error when I try to run the project 12

Traceback (most recent call last):
File "main_27.py", line 284, in
run()
File "main_27.py", line 257, in run
image_input, keep_prob, vgg_layer3_out, vgg_layer4_out, vgg_layer7_out = load_vgg(sess, vgg_path)
File "main_27.py", line 132, in load_vgg
tf.saved_model.loader.load(sess, ['vgg16'], vgg_path)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/saved_model/loader_impl.py", line 200, in load
saved_model = _parse_saved_model(export_dir)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/saved_model/loader_impl.py", line 78, in _parse_saved_model
constants.SAVED_MODEL_FILENAME_PB))
IOError: SavedModel file does not exist at: /home/jack/code/self-driving-car/project_12_road_segmentation/data/vgg/{saved_model.pbtxt|saved_model.pb}
`

project_5_vehicle_detection -- Got TypeError: 'module' object is not callable when run main_ssd.py

When I run main_ssd.py I got this:

Traceback (most recent call last):
  File "/home/binli/self-driving-car/project_5_vehicle_detection/main_ssd.py", line 10, in <module>
    ssd_model, bbox_helper, color_palette = get_SSD_model()
  File "/home/binli/self-driving-car/project_5_vehicle_detection/SSD.py", line 805, in get_SSD_model
    model_ssd = SSD300(input_shape=(300, 300, 3), num_classes=NUM_CLASSES, pretrained=True)
  File "/home/binli/self-driving-car/project_5_vehicle_detection/SSD.py", line 679, in SSD300
    mode='concat', concat_axis=1, name='mbox_loc')
TypeError: 'module' object is not callable

I tried a lot but I cannot fix this? Any suggestions about this? @ndrplz thanks a lot

Webcam image processing

Hello,

I've been trying to implement your code with a webcam I have so I can use it to create an autonomous vehicle that would be able to travel between two white lines. The only issue I have is being able to capture frames from a live camera feed and have the coding process the images. I wouldn't think it would be much harder to do since you have implemented the use of images and video, but I haven't been able to figure out how to do this yet. Also, this will be done from a laptop so no Raspberry Pi will be involved. Any direction or input would be helpful.

project 4

i m unable to access video.
error: from moviepy.editor import VideoFileClip
ModuleNotFoundError: No module named 'moviepy'

i have installed moviepy package but its still not working

CarND-Controls-PID

How to run the carnd-controls-pid program?Can you give me the software and complete instructions?

SystemError: Unknown opcode

Try to run project 3 on ubuntu18: python3 drive.py got error: SystemError: Unknown opcode

seems it is python version mismatch, ubuntu 18 is python3.6, what version of python do you use to train the model?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.