Awesome Temporal Action Localization:
A curated list of temporal action localization/detection and related area (e.g. temporal action proposal) resources.
Contributors:
SCUT: Runhao Zeng, Zeng You, Xinyu Sun
NPU: Le Yang
Temporal Action Localization
[RTD-Net] Relaxed Transformer Decoders for Direct Action Proposal Generation - Jing Tan et al, ICCV 2021
. [code]
[LoFi] Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization - Mengmeng Xu et al, NIPS 2021
.
[ATAG] Augmented Transformer with Adaptive Graph for Temporal Action Proposal Generation - Shuning Chang et al, arXiv 2021
.
[AEI] AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation - Khoa Vo et al, BMVC 2021
.
[GCM] Graph Convolutional Module for Temporal Action Localization in Videos - Runhao Zeng et al, TPAMI 2021
. [code]
[AVFusion] Hear Me Out: Fusional Approaches for AudioAugmented Temporal Action Localization - Bagchi et al, arXiv 2021
. [code]
[ContextLoc] Enriching Local and Global Contexts for Temporal Action Localization - Zixin Zhu et al, ICCV 2021
.
[CSA] Class Semantics-based Attention for Action Detection - Deepak Sridhar et al, ICCV 2021
.
[TCANet] Temporal Context Aggregation Network for Temporal Action Proposal Refinement - Zhiwu Qing et al, CVPR 2021
.
[Multi-Task TAD] Three Birds with One Stone: Multi-Task Temporal Action Detection via Recycling Temporal Annotations - Zhihui Li et al, CVPR 2021
.
[Coarse-Fine Networks] Coarse-Fine Networks for Temporal Activity Detection in Videos - Kahatapitiya et al, CVPR 2021
.
[AFSD] Learning Salient Boundary Feature for Anchor-free Temporal Action Localization - Chuming Lin et al, CVPR 2021
. [code]
[MUSEs] Multi-shot temporal event localization: A Benchmark - Xiaolong Liu et al, CVPR 2021
[SALAD] SALAD: Self-Assessment Learning for Action Detection - Guillaume Vaudaux-Ruth et al, WACV 2021
[RTD-Net] Relaxed Transformer Decoders for Direct Action Proposal Generation - Jing Tan et al, arxiv 2021
. [code]
[AGT] Activity Graph Transformer for Temporal Action Localization - Megha Nawhal et al, arxiv 2021
[VSGN] Video Self-Stitching Graph Network for Temporal Action Localization - Chen Zhao et al, ICCV 2021
[UFA] Temporal Action Detection with Multi-level Supervision - Baifeng Shi et al, arxiv 2020
[TSP] TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Tasks - Humam Alwassel et al, arxiv 2020
[BSP] Boundary-sensitive Pre-training for Temporal Localization in Videos - Mengmeng Xu et al, arxiv 2020
[VAN] Temporal Action Localization with Variance-Aware Networks - Ting-Ting Xie et al, arxiv 2020
[TSI] TSI: Temporal Scale Invariant Network for Action Proposal Generation - Shuming Liu et al, ACCV 2020
. [code]
[BU-TAL] Bottom-Up Temporal Action Localization with Mutual Regularization - Peisen Zhao et al, ECCV 2020
.
[DBG] Fast Learning of Temporal Action Proposal via Dense Boundary Generator - Chuming Lin et al, AAAI 2020
. [code]
[G-TAD] G-TAD: Sub-Graph Localization for Temporal Action Detection - Mengmeng Xu et al, CVPR 2020
. [code]
[PBRNet] Progressive Boundary Refinement Network for Temporal Action Detection - Qinying Liu et al, AAAI 2020
.
[AGCN] Graph Attention based Proposal 3D ConvNets for Action Detection - Jun Li et al, AAAI 2020
.
Method
Conference
IoU=0.1
IoU=0.2
IoU=0.3
IoU=0.4
IoU=0.5
IoU=0.6
IoU=0.7
DAPs
ECCV-2016
-
-
-
-
13.9
-
-
SLM
CVPR-2016
39.7
35.7
30.0
23.2
15.2
-
-
FG
CVPR-2016
48.9
44.0
36.0
26.4
17.1
-
-
SMS
CVPR-2017
51.0
45.2
36.5
27.8
17.8
-
-
PSDF
CVPR-2016
51.4
42.6
33.6
26.1
18.8
-
-
S-CNN
CVPR-2016
47.7
43.5
36.3
28.7
19.0
10.3
5.3
SST
ICCV-2017
-
-
-
-
23.0
-
-
CDC
CVPR-2017
-
-
40.1
29.4
23.3
13.1
7.9
TURN
ICCV-2017
54.0
50.9
44.1
34.9
25.6
-
-
TCN
ICCV-2017
-
-
-
33.3
25.6
15.9
9.0
Self-Ad
AAAI-2018
-
-
-
-
27.7
-
-
TPC
AAAI-2018
-
-
44.1
37.1
28.2
20.6
12.7
R-C3D
ICCV-2017
54.5
51.5
44.8
35.6
28.9
-
-
SSN
ICCV-2017
66.0
59.4
51.9
41.0
29.8
-
-
Action-Search
ECCV-2018
-
-
51.8
42.4
30.8
20.2
11.1
DBS
AAAI-2019
56.7
54.7
50.6
43.1
34.3
24.4
14.7
BSN
ECCV-2018
-
-
53.5
45.0
36.9
28.4
20.0
AGCN
AAAI-2020
59.3
59.6
57.1
51.6
38.6
28.9
17.0
GTAN
CVPR-2019
69.1
63.7
57.8
47.2
38.8
-
-
BMN
ICCV-2019
-
-
56.0
47.4
38.8
29.7
20.5
DBG
AAAI-2020
-
-
57.8
49.4
39.8
30.2
21.7
TSI
ACCV-2020
-
-
61.0
52.1
42.6
33.2
22.4
TAL-Net
CVPR-2018
59.8
57.1
53.2
48.5
42.8
33.8
20.8
RAM
TMM-2019
65.4
63.1
58.8
52.7
43.7
-
-
TCANet
CVPR-2021
-
-
60.6
53.2
44.6
36.8
26.7
SALAD
WACV-2021
73.3
70.7
65.7
57.0
44.6
-
-
AEI
BMVC-2021
-
-
58.7
52.7
44.7
35.9
23.4
RTD-Net
ICCV-2021
-
-
58.5
53.1
45.1
36.4
25.0
BU-TAL
ECCV-2020
-
-
53.9
50.7
45.4
38.0
28.5
PGCN
ICCV-2019
69.5
67.8
63.6
57.8
49.1
-
-
CSA
ICCV-2021
-
-
64.4
58.0
49.2
38.2
27.8
PBRNet
AAAI-2020
-
-
58.5
54.6
51.3
41.8
29.5
G-TAD
CVPR-2020
-
-
66.4
60.4
51.6
37.6
22.9
GCM
TPAMI-2021
72.5
70.9
66.5
60.8
51.9
-
-
VSGN
ICCV-2021
-
-
66.7
60.4
52.4
41.0
30.4
RCL
CVPR-2022
-
-
70.1
62.3
52.9
42.7
30.7
DCAN
AAAI-2022
-
-
68.2
62.7
54.1
43.9
32.6
ContextLoc
ICCV-2021
-
-
68.3
63.8
54.3
41.8
26.2
Multi-Task TAD
CVPR-2021
-
-
63.2
58.5
54.8
44.3
32.4
AFSD
CVPR-2021
-
-
67.3
62.4
55.5
43.7
31.1
MUSES
CVPR-2021
-
-
68.9
64.0
56.9
46.3
31.0
TALLFormer
ECCV-2022
-
-
68.4
-
57.6
-
30.8
TadTR
TIP-2022
-
-
74.8
69.1
60.1
46.6
32.8
ActionFormer
ECCV-2022
-
-
82.1
77.8
71.0
59.4
43.9
Method
Conference
IoU=0.1
IoU=0.2
IoU=0.3
IoU=0.4
IoU=0.5
IoU=0.6
IoU=0.7
UFA
arXiv
-
-
45.6
36.4
26.2
15.5
7.1
VAN
arXiv
-
-
55.0
48.6
39.2
26.9
15.0
ATAG
arXiv
-
-
62.0
53.1
47.3
38.0
28.0
AGT
arXiv
72.1
69.8
65.0
58.1
50.2
-
-
RTD-Net
arXiv
-
-
68.3
62.3
51.9
38.8
23.7
C-TCN
arXiv
72.2
71.4
68.0
62.3
52.1
-
-
TSP
arXiv
-
-
69.1
63.3
53.5
40.4
26.0
AVFusion
arXiv
-
-
70.2
65.0
57.2
45.4
28.9
Method
Conference
IoU=0.5
IoU=0.75
IoU=0.95
Avg
R-C3D
ICCV-2017
26.8
-
-
-
AGCN
AAAI-2020
30.4
-
-
-
SCC
CVPR-2017
39.9
18.7
4.7
19.3
TAL-Net
CVPR-2018
38.23
18.30
1.30
20.22
RAM
TMM-2019
36.99
23.10
3.34
23.03
TCN
ICCV-2017
37.49
23.47
4.47
23.58
CDC
CVPR-2017
45.3
26.0
0.2
23.8
DBS
CVPR-2019
43.2
25.8
6.1
26.1
PGCN
ICCV-2019
42.90
28.14
2.47
26.99
SSN
ICCV-2017
43.26
28.70
5.63
28.28
BU-TAL
ECCV-2020
43.47
33.91
9.21
30.12
BSN
ECCV-2018
46.45
29.96
8.02
30.03
RTD-Net
ICCV-2021
47.21
30.68
8.61
30.83
SALAD
WACV-2021
51.72
31.21
3.33
31.02
BMN
ICCV-2019
50.07
34.78
8.29
33.85
MUSES
CVPR-2021
50.02
34.97
6.57
33.99
G-TAD
CVPR-2020
50.36
34.60
9.02
34.09
TSI
ACCV-2020
51.18
35.02
6.59
34.15
ContextLoc
ICCV-2021
56.01
35.19
3.55
34.23
GCM
TPAMI-2021
51.03
35.17
7.44
34.24
LoFi
NIPS-2021
50.68
35.16
8.16
34.49
GTAN
CVPR-2019
52.61
34.14
8.91
34.31
RCL
CVPR-2022
51.74
35.27
8.03
34.39
AFSD
CVPR-2021
52.38
35.27
6.47
34.39
AEI
BMVC-2021
52.3
34.5
9.7
34.7
PBRNet
AAAI-2020
53.96
34.97
8.98
35.01
Multi-Task TAD
CVPR-2021
57.8
37.6
9.6
35.0
DCAN
AAAI-2021
51.78
35.98
9.45
35.39
TCANet
CVPR-2021
52.27
36.73
6.86
35.52
CSA
ICCV-2021
51.88
36.88
8.74
35.69
Method
Conference
IoU=0.5
IoU=0.75
IoU=0.95
IoU=Avg
RTD-Net
arXiv
46.4
30.4
8.6
30.5
C-TCN
arXiv
47.6
31.9
6.2
31.1
TadTR
arXiv
47.57
31.65
7.98
31.32
BSP
arXiv
50.1
34.7
7.9
34.0
ATAG
arXiv
50.92
35.35
9.71
34.68
VSGN
arXiv
52.4
36.0
8.4
35.1
ActionFormer
arXiv
53.5
36.2
7.7
35.6
TALLFormer
arXiv
54.1
36.2
7.9
35.6
TSP
arXiv
51.3
37.2
9.3
35.8
AVFusion
arXiv
52.73
37.78
9.39
36.63
Weakly Supervised Temporal Action Localization
[BackTAL] Background-Click Supervision for Temporal Action Localization - Le Yang et al, TPAMI 2021
. [code]
[ACSNet] ACSNet: Action-Context Separation Network for Weakly Supervised Temporal Action Localization - Ziyi Liu et al, AAAI 2021
.
[AMS] Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization - Chen Ju et al, arXiv 2021
.
[AUMN] Action Unit Memory Network for Weakly Supervised Temporal Action Localization - Wang Luo et al, CVPR 2021
.
[CSCL] Weakly-Supervised Temporal Action Localization via Cross-Stream Collaborative Learning - Yuan Ji et al, ACM MM 2021
.
[RefineLoc] RefineLoc: Iterative Refinement for Weakly-Supervised Action Localization - Alejandro Pardo et al, WACV 2021
. [code]
[UM-Net] Weakly-supervised Temporal Action Localization by Uncertainty Modeling - Pilhyeon Lee et al, AAAI 2021
.
[CoLA] CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning - Can Zhang et al, CVPR 2021
.
[ActShufNet] Action Shuffling for Weakly Supervised Temporal Localization - Xiao-Yu Zhang et al, arXiv 2021
.
[$\mathrm{CO_2-Net}$] Cross-modal Consensus Network for Weakly Supervised Temporal Action Localization - Fa-Ting Hong et al, ACM MM 2021
.
[HAM-Net] A Hybrid Attention Mechanism for Weakly-Supervised Temporal Action Localization - Ashraful Islam et al, AAAI 2021
. [code]
[ECM] Equivalent Classification Mapping for Weakly Supervised Temporal Action Localization - Tao Zhao et al, arxiv 2020
[TCA] Learning Temporal Co-Attention Models for Unsupervised Video Action Localization - Guoqiang Gong et al, CVPR 2020
[EM-MIL] Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance - Zhekun Luo et al, ECCV 2020
.
[SF-Net] SF-Net: Single-Frame Supervision for Temporal Action Localization - Fan Ma et al, ECCV 2020
. [code]
[A2CL-PT] Adversarial Background-Aware Loss for Weakly-supervised Temporal Activity Localization - Kyle Min et al, ECCV 2020
.
[TSCN] Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization - Yuanhao Zhai et al, ECCV 2020
.
[ActionBytes] ActionBytes: Learning from Trimmed Videos to Localize Actions - Mihir Jain et al, CVPR 2020
.
[DGAM] Weakly-Supervised Action Localization by Generative Attention Modeling - Baifeng Shi et al, CVPR 2020
.
[RPN] Relational Prototypical Network for Weakly Supervised Temporal Action Localization - Linjiang Huang et al, AAAI 2020
.
[BaSNet] Background Suppression Network for Weakly-supervised Temporal Action Localization - Pilhyeon Lee et al, AAAI 2020
.
[DML] Weakly Supervised Temporal Action Localization Using Deep Metric Learning - Ashraful Islam et al, WACV 2020
.
[MCASL] Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks - Maheen Rashid et al, WACV 2020
.
[WSGN] Weakly Supervised Gaussian Networks for Action Detection - Basura Fernando et al, WACV 2020
.
Method
Conference
IoU=0.1
IoU=0.2
IoU=0.3
IoU=0.4
IoU=0.5
IoU=0.6
IoU=0.7
H&S
ICCV-2017
36.44
27.84
19.49
12.66
6.84
-
-
UNet
CVPR-2017
44.4
37.7
28.2
21.1
13.7
-
-
One-Shot
CVPR-2018
-
-
-
-
14.7
-
-
STPN
CVPR-2018
52.0
44.7
35.5
25.8
16.9
9.9
4.3
MAAN
ICLR-2019
59.8
50.8
41.1
30.6
20.3
12.0
6.9
IWO-Net
TIP-2019
57.6
48.9
38.9
29.3
20.5
-
-
WSGN
WACV-2020
55.3
47.6
38.9
30.0
21.1
-
-
AutoLoc
ECCV-2018
-
-
35.8
29.0
21.2
13.4
5.8
W-TAL
ECCV-2018
55.2
49.6
40.1
31.1
22.8
-
7.6
STAR
AAAI-2019
68.8
60.0
48.7
34.7
23.0
-
-
CMCS
WACV-2021
-
-
40.8
32.7
23.1
13.3
5.3
CMCS
CVPR-2019
57.4
50.8
41.2
32.1
23.1
15.0
7.0
CleanNet
ICCV-2019
-
-
44.4
36.3
27.1
17.3
7.3
TSM
ICCV-2019
-
-
39.5
-
24.5
-
7.1
MCASL
WACV-2020
63.7
56.9
47.3
36.4
26.1
-
-
3C-Net
ICCV-2019
59.1
53.5
44.2
34.1
26.6
-
8.1
BM
ICCV-2019
60.4
56.0
46.6
37.5
26.8
17.6
9.0
BaSNet
AAAI-2020
58.2
52.3
44.6
36.0
27.0
18.6
10.4
RPN
AAAI-2020
62.3
57.0
48.2
37.2
27.9
16.7
8.1
TSCN
ECCV-2020
63.4
57.6
47.8
37.7
28.7
19.4
10.2
DGAM
CVPR-2020
60.0
54.2
46.8
38.2
28.8
19.8
11.5
ActionBytes
CVPR-2020
-
-
43.0
35.8
29.0
-
9.5
SF-Net
ECCV-2020
71.0
63.4
53.2
40.7
29.3
18.4
9.6
DML
AAAI-2020
62.3
-
46.8
-
29.6
-
9.7
A2CL-PT
ECCV-2020
61.2
56.1
48.1
39.0
30.1
19.2
10.6
TCA
CVPR-2020
-
-
46.9
38.9
30.1
19.8
10.4
EM-MIL
ECCV-2020
59.1
52.7
45.5
36.8
30.5
22.7
16.4
HAM-Net
AAAI-2021
65.4
59.0
50.3
41.1
31.0
20.7
11.2
CoLA
CVPR-2021
66.2
59.5
51.5
41.9
32.2
22.0
13.1
ACSNet
AAAI-2021
-
-
51.4
42.7
32.4
22.0
11.7
AUMN
CVPR-2021
66.2
61.9
54.9
44.4
33.3
20.5
9.0
CSCL
ACM MM-2021
68.0
61.8
52.7
43.3
33.4
21.8
12.3
UM-Net
AAAI-2021
67.5
61.2
52.3
43.4
33.7
22.9
12.1
BackTAL
TPAMI-2021
-
-
54.4
45.5
36.3
26.2
14.8
$\mathrm{CO_2-Net}$
ACM MM-2021
70.1
63.6
54.5
45.7
38.3
26.4
13.4
Method
Conference
IoU=0.1
IoU=0.2
IoU=0.3
IoU=0.4
IoU=0.5
IoU=0.6
IoU=0.7
ECM
arXiv
62.6
55.1
46.5
38.2
29.1
19.5
10.9
ActShufNet
arXiv
63.44
57.92
48.46
40.01
31.12
22.01
11.26
AMS
arXiv
69.1
62.3
52.7
42.8
33.1
23.1
13.0
Method
Conference
IoU=0.5
IoU=0.75
IoU=0.95
IoU=Avg
STPN
CVPR-2018
29.3
16.9
2.6
20.07
IWO-Net
TIP-2019
29.8
17.6
4.7
-
TSM
ICCV-2019
30.3
19.0
4.5
-
STAR
AAAI-2019
31.1
18.8
4.7
-
CMCS
CVPR-2019
34.0
20.9
5.7
21.2
CleanNet
ICCV-2019
36.7
20.4
4.5
21.4
TSCN
ECCV-2020
35.3
21.4
5.3
21.7
BaSNet
AAAI-2019
34.5
22.5
4.9
22.2
MAAN
ICLR-2019
33.7
21.9
5.5
-
BM
ICCV-2019
36.4
19.2
2.9
-
A2CL-PT
ECCV-2020
36.8
22.0
5.2
22.5
AUMN
CVPR-2021
38.3
23.5
5.2
23.5
UM-Net
AAAI-2021
37.0
23.9
5.7
23.7
Method
Conference
IoU=0.5
IoU=0.75
IoU=0.95
IoU=Avg
ECM
arxiv
36.7
23.6
5.9
23.5
ActShufNet
arxiv
36.3
23.5
5.8
23.6
Method
Conference
IoU=0.5
IoU=0.75
IoU=0.95
IoU=Avg
UNet
CVPR-2017
7.4
3.2
0.7
-
AutoLoc
ECCV-2018
27.3
15.1
3.3
-
TSM
ICCV-2019
28.3
17.0
3.5
-
MCASL
AAAI-2020
29.4
-
-
-
STAR
AAAI-2019
31.1
18.8
4.7
-
DML
AAAI-2020
35.2
-
-
-
W-TALC
ECCV-2018
37.0
-
-
18.0
3C-Net
ICCV-2019
37.2
-
-
-
CMCS
CVPR-2019
36.8
22.0
5.6
22.4
RefineLoc
WACV-2021
38.7
22.6
5.5
23.2
RPN
AAAI-2020
37.6
23.9
5.4
23.3
CleanNet
ICCV-2019
40.5
22.3
5.2
23.4
TSCN
ECCV-2020
37.6
23.7
5.7
23.6
ACSNet
AAAI-2021
36.3
24.2
5.8
23.9
BaSNet
AAAI-2020
38.5
24.2
5.6
24.3
ActionBytes
CVPR-2020
39.4
-
-
-
EM-MIL
ECCV-2020
37.4
-
-
-
TCA
CVPR-2020
40.0
25.0
4.6
24.6
HAM-Net
AAAI-2021
41.0
24.8
5.3
25.1
AUMN
CVPR-2021
42.0
25.0
5.6
25.5
UM-Net
AAAI-2021
41.2
25.6
6.0
25.9
CoLA
CVPR-2021
42.7
25.7
5.8
26.1
$\mathrm{CO_2-Net}$
ACM MM-2021
43.3
26.3
5.2
26.4
CSCL
ACM MM-2021
43.8
26.9
5.6
26.9
BackTAL
TPAMI-2021
41.5
27.3
4.7
27.0
Method
Conference
IoU=0.5
IoU=0.75
IoU=0.95
IoU=Avg
AMS
arxiv
40.7
23.7
5.8
24.6
ActShufNet
arxiv
41.2
24.9
5.9
25.0