mostafa-saad / deep-activity-rec Goto Github PK

Hi Mostafa,
Some queries:

1. For the image classification baseline B1: do you just remove the last layer with 1k neurons and add a new output layer layer with 8 neurons(one for each group activity category) , or do you add any more hidden layer between fc7 and the 8 neuron output layer ?
   2)Also, do you take a center crop 227 * 227 or do you resize the frames instead of cropping?
   Thanks

For the volleyball activity dataset, the training and testing of group activity is per seq or per frame?

Thanks
Fabio

Hi @mostafa-saad
In the experiment below:
exePhase1_2.cpp
exePhase3.cpp
exePhase4.cpp

Can you point out or indicate which experiment belong to the baseline B, based on the Journal paper?

Thanks.

mkdir -p release
mkdir -p apps-release
make exePhase1_2
make[1]: Entering directory `/home/chetan/first/caffe/examples/deep-activity-rec'
Building file: src/leveldb-reader.cpp
Invoking: GCC C++ Compiler
g++ -fPIC -std=gnu++0x -Wall -c -fmessage-length=0 -O3 -DCPU_ONLY=0 -I/home/chetan/first/caffe/include -I/home/chetan/first/caffe/build/src -I/home/chetan/dlib-19.6 -fPIC -MMD -MP -MF"release/leveldb-reader.d" -MT"release/leveldb-reader.d" -o "release/leveldb-reader.o" "src/leveldb-reader.cpp"
In file included from src/leveldb-reader.cpp:12:0:
src/leveldb-reader.h:44:35: fatal error: caffe/vision_layers.hpp: No such file or directory
#include "caffe/vision_layers.hpp"
^
compilation terminated.
make[1]: *** [release/leveldb-reader.o] Error 1

Iam trying to run make all and this error shows up.
I have a ubuntu 14 with python 2.7.I have checked the caffe directory and i couldn't find the header file vision_header.hpp.

how to solve this problem?

Could you please provide a simple benchmark for this awesome work? and is there any cause to not wrote this code using CUDA?

Hi @mostafa-saad ,

I would like to request the raw video files for Youtube volleyball. Is it possible to include the raw videos and how to extract the keyframes from them (before manual selection)?

Thanks,
Peratham

I can't find file annotations.txt for folder 9

Hi @mostafa-saad,

Could you re-run your algorithm using pre-trained Inception-V3. Since, in this paper: https://arxiv.org/abs/1611.09078, they use the Inception-V3 and outperform your algorithm.

Thanks

At Phase 3, the program crashes with "Program received signal SIGSEGV, Segmentation fault."
I run the program with GDB. And I find that it crashed in caffe::Caffe::RNG::generator().
Has anyone met this problem?
the call stack is listed below.

#0 0x00007ffff5da7560 in caffe::Caffe::RNG::generator() ()
from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#1 0x00007ffff5d625e0 in caffe::caffe_rng_rand() ()
from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#2 0x00007ffff5c52e58 in caffe::InternalThread::StartInternalThread() ()
from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#3 0x00007ffff5d4c96e in caffe::DataReader::Body::Body(caffe::LayerParameter const&) ()
from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#4 0x00007ffff5d4cbc8 in caffe::DataReader::DataReader(caffe::LayerParameter const&) ()
from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#5 0x00007ffff5ca81fb in caffe::DataLayer::DataLayer(caffe::LayerParameter const&) ()
from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#6 0x00007ffff5ca8242 in boost::shared_ptr<caffe::Layer > caffe::Creator_DataLayer(caffe::LayerParameter const&) () from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#7 0x00007ffff5d954cc in caffe::LayerRegistry::CreateLayer(caffe::LayerParameter const&) ()
from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#8 0x00007ffff5d9b395 in caffe::Net::Init(caffe::NetParameter const&) ()
from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#9 0x00007ffff5d9dac8 in caffe::Net::Net(std::string const&, caffe::Phase, caffe::Net const*) () from /home/time/group_action/caffe-lstm/build/lib/libcaffe.so
#10 0x0000000000460998 in void feature_extraction_pipeline(int&, char**&) ()
#11 0x0000000000407ff0 in main ()

While testing the single frame baseline models on Collective activity dataset, do you consider only the 10th frame in a sequence ? For some videos, for examples Seq01, annotations are provided for all the frames. While testing, do you consider all the 10 labeled instances in a sequence as separate test cases, or do you discard the first 9 frames ?
Thanks

Hi @mostafa-saad,

I would like to know how do you perform person detection given a frame, I already read the paper and understood the person tracking using "Dlib Video Tracking Object". However to perform feature extraction and classification in the test step you need firstly detect people in a given video frame.

The Download Link does not seem to work.
http://vml.cs.sfu.ca/wp-content/uploads/volleyballdataset/volleyball.zip

Hi @mostafa-saad,

Is there the code how to split the collective activity dataset into testing/training set?.

Thanks

@mostafa-saad
I tried to run “script-simple.sh” as you told, then I found Phase1 and Phase 2 is ok, but Phase 3 reports “p4-network2/trainval-leveldb: does not exist”.

I tried to run “script-clean.sh”, but the same problem exists.
Could you please help me how to fix it? Thank you very much.

Best wishes
Binbing

script-simple-expected-log.txt
z_log_dataset_net2.txt
z_trainval-test-log.txt
z_trainval-test-window-evaluation-log-prop.txt

Hello,
The paper mentioned that the model relies on the visual tracking provided by DLib. Is it maybe possible to have those tracks available within the dataset? That would be very useful.

Thanks!

Hi,

Thanks for sharing the dataset! However, are some of the labels wrong in this Volleyball dataset? I randomly checked the labels of some clips in the test set, and I found their labels to be wrong. Here are the clips that I checked and the annotated group activity label is shown in the folder name: https://www.dropbox.com/sh/651zc5lvooe9k33/AADjUZjoGL1GR2ZDLlqmpl7pa?dl=0

For example, clip 63975 of video 5 has "l_set" as the label, but from the frames clearly it's the right group performing set, so the label should be "r_set".

@mostafa-saad Could you confirm this? Thank you very much!

Best,
Honglu

thanks for your wonderful work and share the code . I am reading your paper and your code . I have a puzzle , how to build new activity dataset , for example , a game video activity dataset . I know you build the dataset by dlib ,but I cannot find step by step guideline to build a new activity dataset employing the dlib.
can you list your dataset building step ? I appreciate it ! Thank you !

It always said something wrong in the file. Does the dataset still all right? Thx

Hi

I can run the ./script now. And use the db from following link you provided. But I got fowllowing error.
Maybe there is some wrong in the db, how do I debug the issue? Thanks a lot.

http://vml.cs.sfu.ca/wp-content/uploads/volleyballdataset/volleyball.zip
ERROR. Every 1 batch should have the same value. Inconsistent batch # 992-th
Overall unique labels are: 2. The appeared labels are: 3 6
exePhase3: ../apps/exePhase3.cpp:265: void feature_extraction_pipeline(int&, char**&) [with Dtype = float]: Assertion `false' failed.

Hi,
I'm afraid there is a discrepancy over the manual annotation files. If we take for example dir 0/3596 the folder contains 41 images as it should according to the README file.
If I open file 3596 in folder 0 of the manual annotation folder I can see the following columns (please correct me if I'm wrong):

There are more columns (classification labels) and more rows of course, But if we look at player 0 he has only 20 frames annotated for video 0 instead of 41. Player 0 is annotated for frames 3586-3605 while the dataset folder contains frames 3576-3616.

What am I missing here?
@mostafa-saad

Thanks!

Hi Mostafa,

I encounter following issue when run the installation guide you wrote.

run ./script-simple.sh
exePhase1_2:error while loading shared libraries: libcaffe.so : cannot open shared object file: no such file or directory

Thanks a lot,
Jc

Hey! Thank you for the detailed README and great paper! However, I was wondering where I can find a trained model I can use?
I don't really know C++ and find it hard follow through the instructions on how to run it etc.

I would really appreciate any help or pointers

An annotation text file is missing in Update 3 manual annotations for the trajectories.

Missing file directory structure : 4/10465/10465.txt

@mostafa-saad
I tried to run “script-simple.sh”, and I found Phase1 and Phase 2 is ok, but Phase 3 and Phase report the same error when creating the test network, here is what I got:

E0205 16:20:49.591593 6865 exePhase3.cpp:194] Creating the test network
Segmentation fault (core dumped)
E0205 16:21:00.454159 6886 exePhase4.cpp:169] Creating the test network
Segmentation fault (core dumped)

Do you have any idea? Thank you very much.

Best wishes
Yichao

Does anybody know some alternative links to that dataset ?
http://vml.cs.sfu.ca/wp-content/uploads/volleyballdataset/volleyball.zip

Can I ask what framerate is used to extract image frames? I want to have an idea of the amount of time (in seconds) for each activity. Thank you!