Seungjun Lee · Desmond Tan · Thomas Liang

Overfitting to the training dataset is a common problem in the fully supervised setting. Specifically in weapon detection task, this overfitting issue might manifest itself in two ways: 1) Model lacks the ability to detect the unseen weapons from the source dataset. 2) Model is unable to identify danger if the images come from different domain than the source dataset. To mitigate this issue, we propose DangerCLIP, the novel Open-Domain Weapon Detection method with leveraging the generality and robustness of foundation model like CLIP. Our contribution is three-fold:

1. We propose novel open-domain weapon detection method with leveraging CLIP that can detect weapon or other forms of danger within the images from open-domain.
2. We show that giving dense supervision to the model can improve the classification performance of the model. By giving pseudo-mask generated from pretrained segmentation model as supervision, model can understand image more locally with being able to identify more smaller weapons.
3. Our method surpasses the baseline models with large margin in precision, recall and f1-score metrics, and also shows better robustness to the open-domain dataset.

If you are more interested in, check out our poster in here.