Deep Learning-Based Driver Behavior Detection on Simulated and Real Data

Abstract

Driver behaviour has a significant influence on vehicle accidents. Measuring and providing feedback on driver behaviour can provide significant benefits for understanding and improving road safety. Mobile phones can be leveraged for the detection of driver actions and characteristics from the broadest population of drivers. Mobile phones also offer easy accessibility for cost-effective and reliable information with the built-in sensors available on them, such as the Global Positional System (GPS) and Inertial Measurement Unit (IMU). However, when it comes to a larger scale, obtaining labelled data from these mobile devices is still far from optimal for low-cost and reliable applications due to noise and missing data. In this study, data obtained from mobile phone sensors is simulated as a time series dataset using a traffic simulator and a robotic simulator. Then, the dataset is used with deep learning methods to classify both manoeuvres and driver behaviours, focusing specifically on aggressive driver behaviour. We propose a novel method using two Convolutional Neural Networks Convolutional Neural Networks (CNN) working in parallel to classify driver behaviours while classifying manoeuvres (i.e., aggressive right lane-change). We claim that the Parallel Convolutional Neural Network (PCNN) not only speeds up training time but also increases performance since having information about the manoeuvre helps improve behaviour classification performance. To validate this, first, a single task CNN for manoeuvre classification and a single task CNN aggressive/non-aggressive behaviour classifiers were built separately. The utility of the classifiers was demonstrated on a large simulated dataset created using the Sumo and Webots simulators. Subsequently, the PCNN classifier has been trained and validated on the big simulated dataset and a small driven dataset. We have also collected a dataset driven on real road by using GPS and IMU sensors and the PCNN model has been tested on this real dataset to investigate whether a classifier trained on simulated data can generalize to real data. In addition to this method, we propose a method of Spatial CNN with Attention (SCNN-A) layer to apply to our time series data with extracting more high-level features from spatial data for classification purposes.