This is our bachelor project at the Chair for Autonomous Intelligent Systems at the University of Freiburg for a self-driving car based on the paper "End-to-End Driving via Conditional Imitation Learning" by Codevilla et al. Here we implement the architecture given in the paper and attempt to create variations and improvements, including a version which first attempts to first perform semantic segmentation on the camera image first.
This architecture is designed to run a simulated vehicle within Unreal Engine with the AirSim plugin.
The different network variations are:
| Network | Description |
|---|---|
| standard | Uses only one forward facing camera to drive |
| segmented | Uses only the ground truth segmentation of a forward facing camera |
| seg and normal | Uses both the image and ground truth segmentation of a forward facing camera |
| last image too | Uses both the current and previous image from a forward facing camera |
| two cams | Uses a forward facing and a camera angled towards the right |
| self segmentation | Uses the forward facing camera and segments the image with a network first |
Our architecture performs well and generalizes to new unseen environments quite well. The best performing variation is segmented, although we believe with more training, the self segmentation variation can perform equally well. We were unable to train the self segmentation network fully due to time constraints.
To use our network, run the following command in a command line interpreter:
python src/main.py [weight_path] -eAlternatively, to train the network, the following can be used:
python src/main.py [weight_path] -t [training_data_directory] -b [batch_size] -p [number of epochs]Our command line tool can unfortunately only run the standard network (i.e. with a single forward facing camera).
We have also created a GUI for simpler usage. Open it with:
python src/project.pyIt also allows the selection of the different network variations.
This project is released under the GNU General Public License v3.0. Please review the License file for more details.