DAMO-StreamNet is a novel streaming perception framework for real-time video object detection in autonomous driving scenarios. It builds upon state-of-the-art models like YOLO and LongShortNet to achieve optimized accuracy under strict latency constraints.

• Robust Neck Design: Incorporates deformable convolution to enhance receptive fields and feature alignment.

• Dual-Branch Structure: Fuses semantic and temporal features for accurate motion prediction.

• Asymmetric Distillation: Distills future knowledge from teacher to student network during training for performance gains.

• Real-time Forecasting: Continuously updates support frames for seamless streaming.

For more details, please see our full IJCAI 2023 paper.

DAMO-StreamNet supports real-time detection of 8 classes relevant to autonomous driving:

• Person, Bicycle, Car, Motorcycle, Bus, Truck, Traffic Light, Stop Sign

See ModelScope Documentation for code examples to run inference using our pretrained models.

Teacher models available here.

Follow install guidelines from StreamYOLO and LongShortNet.

Follow Argoverse-HD setup instructions.

Organize downloaded models:

./models
├── checkpoints
│   ├── streamnet_l_1200x1920.pth
│   ├── streamnet_l.pth
│   ├── streamnet_m.pth
│   └── streamnet_s.pth
├── coco_pretrained_models
│   ├── yolox_l_drfpn.pth
│   ├── yolox_m_drfpn.pth
│   └── yolox_s_drfpn.pth  
└── teacher_models
    └── l_s50_still_dfp_flip_ep8_4_gpus_bs_8
        └── best_ckpt.pth

bash run_train.sh

bash run_eval.sh 

• 8 Epochs on Argoverse-HD
• SGD Optimizer with Linear LR Schedule
• Random Flip Augmentation
• Multi-Scale Training

Please cite our paper:

@article{DAMO_StreamNet,
  title={DAMO-StreamNet: Optimizing Streaming Perception in Autonomous Driving},
  author={Jun-Yan He, Zhi-Qi Cheng, Chenyang Li, Wangmeng Xiang, Binghui Chen, Bin Luo, Yifeng Geng, Xuansong Xie},
  journal={IJCAI},  
  year={2023}
}

DAMO-StreamNet builds on YOLO, LongShortNet and StreamYOLO.

For academic research only. Please contact authors for commercial licensing.