PROJECT REPORT
Presentation slide
Semester project of ANU ENGN6528/ENGN4528(computer vision)
In this project, we use an end to end two-stream CNN model to predict the steering angle of a self-driving car. We first train the model on the dataset which is collected by SullyChen in LA, USA and then test it on the dataset which is collected on the campus of the Australian National University, Canberra, Australia.
Pytorch 1.0
torchvision
OpenCV(cv2)
The dataset can be downloaded here
The model takes the RGB information and temporal information as the inputs of the two stream respectively and finally output the steering angle, which is a regression probelm. The temporal information can be optical-flow or dynamic images.
The network architecture is shown below.Detalis can be referred to the technical report.
https://www.youtube.com/watch?v=7juEYI-gGKw&feature=youtu.be
For the split of the datset, please refer to the report.
| Model | City | Hill |
|---|---|---|
| single stream single RGB | 19.04 | 23.89 |
| single stream 3 x RGB | 7.23 | 13.91 |
| 3 x RGB +1 x optical flow | 7.10 | 10.92 |
| 3x RGB +2 x optical flow | 6.09 | 9.73 |
| 3 x RGB +dynamic image of 3 frames | 6.93 | 10.15 |
| 3 x RGB +dynamic image of 5 frames | 5.99 | 11.51 |
NOTE: The code shall be slightly modified accroding to your own directory
Run the frame2flow.py to generate the frame.
You can test the ANU campus driving in the test/ANU_test file and run the test.py.
You will need to download the dataset here:https://drive.google.com/file/d/1Zr0wgDWOhRNm-C8eovcpUSzxLtQSKqBm/view?usp=sharing
Unzip the file and place the two files that contain frames and optical-flow in the same directory of the code.