Self Driving Car
Attempting to build a self driving car….
To fully build a self-driving car, I would need to build 1. Self-driving car software, and 2. Self-driving car hardware. The software would use sensor and input data to algorithmically generate driving instructions, and the hardware would execute these driving instruction within the actual car.
Since much of the technical challenge is actually in the software part, I will be exclusively focusing on the software part. In particular, I want to build self-driving software that can do two things:
- Based on video input of the road, the software can determine how to safely and effectively steer the car.
- Based on video input of the road, the software can determine how to safely and effectively use the car’s acceleration and braking mechanisms.
I may attempt to tackle each of these pieces separately or together. I’m not sure yet. I’m also not completely sure the specific details of each of these sub-challenges, but I will flesh these details out more seriously once I’ve done some initial research.
Suggested by a friend to install Ubuntu 16.04 LTS instead of Windows 10 Pro which was running on my computer. So, instead of removing Windows 10 from scratch, I decided to install Ubuntu 16.04 LTS in dual boot with my Windows 10 Pro. This was a decision that wasted much of my time but thanks to Stackoverflow, I was able to complete it without wasting any more time.
Made my system computation ready by installing Python3 and other required libraries on Ubuntu 16.04 LTS. Installed Anaconda and related libraries from the Web. Also studied Anaconda and learnt how to make an environment and use it for installing particular packages for a project. At the end of Week 1, I was able to run Python programs on my Ubuntu 16.04 LTS and was able to create environments and use them. I had Windows 10 Pro and Ubuntu 16.04 LTS running side by side. Further, I prepared an environment for the use of my project and installed various packages in it like numpy, matplotlib, h5py, scipy, scikit-learn, scikit-image, pillow, opencv3, jupyter, eventlet, imageio, pandas, seaborn, moviepy, tensorflow and keras.
Installed Udacity Self Driving Car Simulator on my machine and started to build my model. First, I have to train the model and then build it to drive on its own. For training, in Self Driving Car Simulator, there were two tracks. I chose one and started the procedure for training. I needed various examples comprising of steering angle, throttle, reverse, speed, left, right and center position of the car. There were three cameras located on the car. First on the windshield, second on the lower side above the front left tyre and third one above the front right side tyre. I drove the car on the track for about two minutes and recorded all the various training examples comprising of the values that I needed. After driving myself for about two minutes, I was able to record around 4,000 training examples. After recording, a new folder “data” was created which contains nearly 4,000 images of different camera angles of the car. Also, a “driving_log.csv” file was created, which contained all the data of steering, throttle, reverse, speed, left, right, center position of the car in a .csv file. Next step was to write code so as to train the model using these 4,000 training examples. I was able to record my training set and write a suitable for training of my car using tensorflow and keras by implementing a convolutional neural network.
After writing a suitable code to train my model for the 4,000 training examples, my code yielded a lot of errors. Thanks to the web that I was able to solve all of them and came to a cleaner implementation of the code that ran without any errors. This was the task that took most of my time in my project. At the week 2, I was able to train my model using the code that I had written and implemented. The training took about 10 to 15 minutes and was successful. This was the reason I shifted to Ubuntu 16.04 LTS from my Windows 10 as in Windows 10 when I implemented the same training process, it took about 40 to 45 minutes. The training generated a file model-<epoch>.h5 whenever the performance in the epoch is better than the previous best. For example, the first epoch will generate a file called model-000.h5. For example, further if the performance is better in the fifth epoch than a new file model-005.h5 will be created with a better performance than the previous one.
In week 3, I implemented the code to let the car drive on its own. I implemented it in no time and used it to test my model. But when I used that to drive, the car was not steering properly though I had done all the procedure correctly. I train the model again and again and ran it multiple times to see if it’s working (or steering) properly. But to my disappointment, my model had failed to drive on its own. Either it moved out of the track or it steered abnormally. After a hard work for about two days of finding a way out, I decided to proceed from the basic approach i.e. increase the training data size. I deleted the “data” folder which contained 4,000 images and also the “driving_log.csv” file. And trained the model once again.
I followed the basic approach of machine learning of increasing the training data size so decided to train my model for the second time. This time with a larger training data. I chose the same track as before and this time I decided to drive the car for about 10 minutes. Due to this, after recording, a “data” folder was created which contained nearly 12,000 images of different camera angles of the car. Corresponding to the “data” folder, “driving_log.csv” file was also created that contain all the data in the form of a database. After recording, I ran the script to train my model. This time my model took more time to train as there were nearly three times more training data than the previous one. At the end of Week 3, I trained my model for the second time with nearly 12,000 training images and I was able to find a way out of improving my model and started working in that direction.
In these days, I started to experiment my trained model with code for driving that I had previously written. To my surprise, my approach of increasing the training data size proved to be success and I was able to see my car steer on its own. I waited and observed that the car is now able to steer, brake and accelerate on its own and is also capable of outstanding steep curves in the track.
I had completed my project by this time and learnt many new things. Got stuck in a lot of errors but thanks to my mentor and the web that cleared all my doubts and helped me to proceed further. Till now I implemented the model on one track only. Later ideas for improving the project are the ways to improve the accuracy of the car a little more by implementing certain more techniques and then implementing the self-driving car on the second track.
The project can be found on my GitHub Repository with detailed instructions. It contains detailed code about the functioning of the python code and the deep learning model used.
And with that, we have come to the end of this article. Bundle of thanks for reading it!
