willprice / two-stream-action-cnn-analysis Goto Github PK

View Code? Open in Web Editor NEW

:mortar_board: Undergraduate thesis on analysing two stream CNNs for action recognition using excitation backpropagation

Python 1.51% Jupyter Notebook 97.92% Shell 0.44% Makefile 0.07% MATLAB 0.05%

two-stream-action-cnn-analysis's Introduction

2SCNN Feature analysis

Working repository for feature analysis of 2SCNN

Models

CUHK VGG16-2SCNN models trained on UCF101
UoB VGG16-2SCNN trained on BEOID (not publicly available)

Dependencies

Python 3
- matplotlib
- numpy
- scipy
- pandas
- seaborn
- click
- skimage
Jupyter notebook (6.0)
IPykernel for Jupyter (5.0)
Bash
Caffe (Excitation BP fork), preferably built with GPU support.

Setup

Edit lib/config.py to point to your local excitation bp caffe installation, nets path (e.g. caffe/models, I like to keep mine in my home directory), and data root.

Repository overview

lib: Most of the meaty code lives in here, code for handling datasets, performing contrastive EBP, mapping from class ids to class names.
net_configs: Python library of configuration scripts defining settings peculiar to models.
scripts: Python CLI scripts for generating attention maps, stitching videos, graphing smoothness
generated: A few bash scripts live in here to call the python scripts that do most of the work.
notebooks: All the Jupyter notebooks used for experimental analysis reside here. They're meant to be run in numerical order. Notebooks with the same numerical prefix can be run in any order.
./run_jupyter.sh is a helper script to setup my environment with the correct LD_LIBRARY_PATH and PATH env variables necessary to find Caffe, CuDNN etc.

Acknowledgments

The code for performing contrastive excitation backprop is from https://github.com/jimmie33/Caffe-ExcitationBP, I also took inspiration from the code used to overlay attention maps on images.

Recommend Projects