Towards Accurate Vehicle Behaviour Classification
With Multi Relational Graph Convolutional Networks

Link to our paper on arxiv : https://arxiv.org/abs/2002.00786
For any queries mail : [email protected] (or) [email protected]

Our method detects and classifies objects of interest (vehicles) in a video scene into 6 classes moving away, moving towards us, parked, lane change(L->R), lane change(R->L), overtake. Note that our method is not based on classifying the ego-vehicle.

Apart from comparisons in the paper, we compare with Structural-RNN, a LSTM based graph network. Since the tasks in their paper confine only to driver-anticipation, we use one of their methods similar to our task. Specifically, we use the detection method of activity-anticipation mentioned in the paper due to the similarity in the architecture and task . We use Vehicles as Humans and Lane Markings as Objects in their architecture for our purpose. Similar to the Human-Object, Human-Human and Object-Object interactions, we observe the Vehicle-Lane, Vehicle-Vehicle and Lane-Lane interactions for all time-steps. We give features similar to the baselines in the paper.

We selected 3 main datasets to perform the experiments.

1. Apollo
2. Kitti
3. Indian

Graphs for all datasets can be downloaded from graphs.
For information on how each graph is stored as a npz file, go through the README file in the same link.

On apollo we have selected sequences from scene-parsing dataset and picked around 70 small sequences(each containing aroud 100 images) manually that include behaviours of our interest. Similarly on Kitti, we use tracking sequences 4,5,10 which are in line with our class requirement.

dgl
pytorch == 1.2.0
pandas
numpy
tqdm

pip3 install requirements.txt

To install and use GPU for dgl, cuda support can be installed from their official website, dgl .
And set use_cuda = 1 in training/testing codes.

git clone https://github.com/ma8sa/temporal-MR-GCN.git
cd temporal-MR-GCN
--testing
python3 lstm_rgcn_test_apollo.py
--training
python3 lstm_rgcn_train_apollo.py

NOTE : Make sure to extract the corresponding graphs(lstm_graphs_apollo) and place it in the same folder where you are running the code from.

In training, lstm_rgcn_train_apollo has all the parameters in main function to tune. main_model.py contains the complete model. rgcn_layer.py contains the dgl format of MR-GCN. graphs_preproc_apollo.py conatains all the data preprocessing methods used.

for indian,
python3 lstm_rgcn_test_ind_kitti.py indian
for kitti,
python3 lstm_rgcn_test_ind_kitti.py kitti

NOTE : Make sure to extract the corresponding graphs (lstm_graphs_kitti for kitti and lstm_graphs_indian for indian) and place it in the same folder where you are running the code from.

0->Move forward
1->Moving towards us
2->Parked
3-> lane-change(L->R)
4-> lane-change(R->L)
5-> Overtake

Since the number of cars showing overtake behaviour are less, we augmented and added few synthetic-graphs and augmented data to train data only for apollo. With a little more data, the model can clearly learn overtake too above 80%, as number of synthetic graphs added were too low 76.