can't download models via https://github.com/yjxiong/anet2016-cuhk/blob/master/models/get_reference_models.sh.

Does your released model trained using the temporal annotations, or just the video level labels?
Thanks.

Hello everyone,
Has anyone successfully converted ActivityNet pre-trained weights into Pytorch TSN?

When I run python examples/classify_video.py --use_flow data/plastering.avi, I got following error:

Traceback (most recent call last):
File "examples/classify_video.py", line 17, in
from pyActionRec.action_classifier import ActionClassifier
File "/home/user/anet2016-cuhk/pyActionRec/action_classifier.py", line 3, in
from action_flow import FlowExtractor
File "/home/user/anet2016-cuhk/pyActionRec/action_flow.py", line 7, in
from libpydenseflow import TVL1FlowExtractor
ImportError: /home/user/anet2016-cuhk/3rd-party/opencv-2.4.13/build/lib/libopencv_ml.so.2.4: undefined symbol: _ZN2cv13AlgorithmInfo8addParamERNS_9AlgorithmEPKcRibMS1_FivEMS1_FviERKSs

Is there anyone can help me, thanks very much!

I have installed caffe-action using the "build-caffe" provided in build_all.sh. But it seems python caffe has not been compiled because '_caffe.so' does not exist at python branch and only its c++ file is there. Should I have to install pycaffe separately like with make pycaffe? Please advice!

I just did line below:

build caffe

echo "Building Caffe"
cd ../../caffe-action
mkdir build
cd build
OpenCV_DIR=../../../3rd-party/opencv-$version/build/ cmake ..
make -j32

Thanks for your help!

In ActivityNet Challenge for detection, you need to submit some segment that predicted as action,like this.

{
version: "VERSION 1.3",
results: {
  5n7NCViB5TU: [
      {
      label: "Discus throw",
      score: 0.64,
      segment: [24.25,38.08]
      },
      {
      label: "Shot put".
      score: 0.77,
      segment: [11.25, 19.37]
      }
  ]
}
external_data: {
  used: true, # Boolean flag. True indicates used of external data.
  details: "First fully-connected layer from VGG-16 pre-trained on ILSVRC-2012 training set", # String with details of your external data.
}
}

However, the demo with classify_video.py or with web demo I can only get the classification result.
How could I get such segment information?

Hi @yjxiong,
Great repository!
However, I failed to get features and to run classification due to incompatibility between Cuda8.0 (required) and cuda 9.2 (which I have on my machine). Do you have any idea on how to solve this problem ?

Thanks in advance,

I have build OpenCV, denseflow and caffe successfully. When I run python examples/classify_video.py data/plastering.avi, the output is:

----------------Classification Results----------------------
Sumo 0.901648
Playing violin 0.056127
Chopping wood 0.0172338
Shot put 0.00949485
Volleyball 0.00876805
Playing accordion 0.00340133
Playing lacrosse 0.000825542
Baking cookies 0.000779136
Surfing 0.000494444
Grooming horse 0.000358299

I am using cuda-8. @yjxiong Can you please look into this?

This is same issue as yjxiong/temporal-segment-networks#237
However, I am not sure how I would be able to get the docker build AND use the denseflow lib provided.
Shortened error below

-- Found CUDA: /usr/local/cuda-8.0 (found suitable exact version "8.0") 
-- Could NOT find LIBZIP (missing:  LIBZIP_LIBRARY) 
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   python
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
LIBZIP_LIBRARY
    linked by target "denseflow" in directory /home/ubuntu/anet2016-cuhk/lib/dense_flow
    linked by target "extract_cpu" in directory /home/ubuntu/anet2016-cuhk/lib/dense_flow
    linked by target "extract_warp_gpu" in directory /home/ubuntu/anet2016-cuhk/lib/dense_flow
    linked by target "extract_gpu" in directory /home/ubuntu/anet2016-cuhk/lib/dense_flow

-- Configuring incomplete, errors occurred!
See also "/home/ubuntu/anet2016-cuhk/lib/dense_flow/build/CMakeFiles/CMakeOutput.log".
See also "/home/ubuntu/anet2016-cuhk/lib/dense_flow/build/CMakeFiles/CMakeError.log".
make: *** No targets specified and no makefile found.  Stop.
Failed to build Dense Flow. Please check the logs above.

The last part of Dense Flow failing was added from https://github.com/yjxiong/temporal-segment-networks/blob/012104d0e3104d25689ae9c4fdccffd4e818a1bb/build_all.sh#L56

Hello! Firstly, I want to congratulate you for this project.

I wanted to know if it is possible to give us a Dockerfile for building a docker image suited for your project since the current build script give us many errors.

Thanks!

Hi everyone!

I was trying to make the code run on my machine and I faced significant trouble installing the correct versions of packages because they became obsolete. Then I tried docker. The experience with it wasn't smooth either because dense_flow installation wasn't compatible with my GPU. So, I tried the docker container provided for the TSN network. I tested it with 1080Ti (worked) and 2080Ti (didn't work w/ CUDNN_STATUS_EXECUTION_FAILED) with 64GB RAM.

docker pull iashin/anet2016-cuhk:latest

or build on top of it by specifying the header of your Dokerfile:

FROM iashin/anet2016-cuhk:latest

The output from the examples:

• python examples/classify_video.py --use_flow data/plastering.avi

frame sample 53: 0.322223 second
frame sample 54: 0.322393 second
total time: 20.200887 second
----------------Classification Results----------------------
Plastering 0.9559713
Rock climbing 0.020194435
Painting fence 0.011123132
Hanging wallpaper 0.010004199
Painting furniture 0.0007557427
Hand washing clothes 0.00065550825
Laying tile 0.00040866304
Throwing darts 0.00033438313
Paintball 0.00019233384
Hitting a pinata 0.000125195

• python examples/classify_video.py --use_flow https://www.youtube.com/watch?v=QkuC0lvMAX0

total time: 91.161525 second
----------------Classification Results----------------------
Windsurfing 1.0
Sailing 6.5195724e-16
Surfing 4.3629453e-17
Kite flying 2.1080195e-18
Wakeboarding 5.306861e-19
Snowboarding 6.466279e-21
Plataform diving 3.3573223e-22
Doing motocross 1.9524087e-22
Scuba diving 1.3819956e-22
Fixing bicycle 1.2276588e-22

Here is the Dockerfile I used to build it (also might be useful if you are not familiar with docker and would like to install it on your machine or as a reference for the package version)

FROM bitxiong/tsn:cuda9_cudnn7

RUN git clone --recursive https://github.com/yjxiong/anet2016-cuhk
WORKDIR /app/anet2016-cuhk
RUN bash models/get_reference_models.sh

RUN cp -r ../lib/ .
RUN cp ../cv2.so .

RUN pip install easydict==1.6
# YouTube API changes often so you may want to install a more recent version of youtube_dl:
RUN pip install youtube_dl==2020.9.20

RUN echo "export ANET_HOME="/app/anet2016-cuhk"" >> ~/.bashrc

Hi Professor,
I have CUDA8 installed so I changed opencv to 2.4.13 in build_all.sh. It built up without errors. But python examples/classify_video.py data/plastering.avi gets:
----------------Classification Results----------------------
Sumo 0.901697
Playing violin 0.0560898
Chopping wood 0.0172752
Shot put 0.00946948
Volleyball 0.00875752
Playing accordion 0.00338519
Playing lacrosse 0.000827064
Baking cookies 0.000784449
Surfing 0.000487826
Grooming horse 0.00035665

And it output similar things with other videos.
Do you think of anything that might cause this?

I can not open the demo website, anyone has the same issue ?
So i try to run it locally, but i came across an error :

(base) weidawang@weidawang-TUF-Gaming-FX506LU-FX506LU:/media/weidawang/DATA/Repo/anet2016-cuhk$ python demo_server.py
Traceback (most recent call last):
  File "demo_server.py", line 7, in <module>
    from pyActionRec.action_classifier import ActionClassifier
  File "/media/weidawang/DATA/Repo/anet2016-cuhk/pyActionRec/__init__.py", line 1, in <module>
    from config import ANET_CFG
ModuleNotFoundError: No module named 'config'

I have trained the model with my dataset on TSN. I want to use this project for testing, so I would like to ask what needs to be modified?

After building everything, I tried to test the code python examples/classify_video.py data/plastering.avi.
The error information shown as following:

Traceback (most recent call last):
  File "examples/classify_video.py", line 17, in <module>
    from pyActionRec.action_classifier import ActionClassifier
  File "/home/changmao/actionRec/anet2016-cuhk/pyActionRec/action_classifier.py", line 3, in <module>
    from action_flow import FlowExtractor
  File "/home/changmao/actionRec/anet2016-cuhk/pyActionRec/action_flow.py", line 7, in <module>
    from libpydenseflow import TVL1FlowExtractor
ImportError: No module named libpydenseflow

I think that denseflow project should be added. But I built it with segmentation fault errors...

I was attempting to have one of the models on 1 gpu, while the other model on another gpu, however, I get a crash. I was setting to dev_ids 0, 1, and I verified nvidia-smi shows multiple gpus. Also verified that gpus work with tensorflow.
This happens after forward is called in action_caffe.py on line

out = self._net.forward(blobs=[score_name,], data=data)

Error:

 F0630 03:45:26.391547     1 cudnn_conv_layer.cu:34] Check failed: status == CUDNN_STATUS_SUCCESS (8 vs. 0)  CUDNN_STATUS_EXECUTION_FAILED

Hello, when compiling opencv2.4.13 in runing build_all.sh, it shows error:

[ 34%] Built target opencv_video
In file included from /usr/include/x86_64-linux-gnu/qt5/QtGui/QSurfaceFormat:1:0,
                 from /usr/include/x86_64-linux-gnu/qt5/QtWidgets/qopenglwidget.h:42,
                 from /usr/include/x86_64-linux-gnu/qt5/QtWidgets/QtWidgets:57,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGLDepends:5,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGL:3,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/../../../modules/highgui/src/window_QT.h:46,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/moc_window_QT.cpp:9,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/opencv_highgui_automoc.cpp:2:
/usr/include/x86_64-linux-gnu/qt5/QtGui/qsurfaceformat.h:123:24: error: missing binary operator before token "("
 #if QT_DEPRECATED_SINCE(5, 2)
                        ^
In file included from /usr/include/x86_64-linux-gnu/qt5/QtGui/QSurfaceFormat:1:0,
                 from /usr/include/x86_64-linux-gnu/qt5/QtWidgets/qopenglwidget.h:42,
                 from /usr/include/x86_64-linux-gnu/qt5/QtWidgets/QtWidgets:57,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGLDepends:5,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGL:3,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/modules/highgui/src/window_QT.h:46,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/modules/highgui/src/window_QT.cpp:47:
/usr/include/x86_64-linux-gnu/qt5/QtGui/qsurfaceformat.h:123:24: error: missing binary operator before token "("
 #if QT_DEPRECATED_SINCE(5, 2)
                        ^
In file included from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/qglfunctions.h:39:0,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGL:8,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/../../../modules/highgui/src/window_QT.h:46,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/moc_window_QT.cpp:9,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/opencv_highgui_automoc.cpp:2:
/usr/include/x86_64-linux-gnu/qt5/QtGui/qopenglfunctions.h:253:24: error: missing binary operator before token "("
 #if QT_DEPRECATED_SINCE(5, 0)
                        ^
In file included from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/qglfunctions.h:39:0,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGL:8,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/modules/highgui/src/window_QT.h:46,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/modules/highgui/src/window_QT.cpp:47:
/usr/include/x86_64-linux-gnu/qt5/QtGui/qopenglfunctions.h:253:24: error: missing binary operator before token "("
 #if QT_DEPRECATED_SINCE(5, 0)
                        ^
In file included from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/opencv_highgui_automoc.cpp:2:0:
/home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/moc_window_QT.cpp:15:2: error: #error "This file was generated using the moc from 5.5.1. It"
 #error "This file was generated using the moc from 5.5.1. It"
  ^
/home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/moc_window_QT.cpp:16:2: error: #error "cannot be used with the include files from this version of Qt."
 #error "cannot be used with the include files from this version of Qt."
  ^
/home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/moc_window_QT.cpp:17:2: error: #error "(The moc has changed too much.)"
 #error "(The moc has changed too much.)"
  ^
In file included from /usr/include/x86_64-linux-gnu/qt5/QtWidgets/QtWidgets:57:0,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGLDepends:5,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGL:3,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/../../../modules/highgui/src/window_QT.h:46,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/moc_window_QT.cpp:9,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/build/modules/highgui/opencv_highgui_automoc.cpp:2:
/usr/include/x86_64-linux-gnu/qt5/QtWidgets/qopenglwidget.h:49:38: error: expected initializer before ‘:’ token
 class Q_WIDGETS_EXPORT QOpenGLWidget : public QWidget
                                      ^
In file included from /usr/include/x86_64-linux-gnu/qt5/QtWidgets/QtWidgets:57:0,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGLDepends:5,
                 from /usr/include/x86_64-linux-gnu/qt5/QtOpenGL/QtOpenGL:3,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/modules/highgui/src/window_QT.h:46,
                 from /home/ztgong/work/video/actionRecognition/anet2016-cuhk/3rd-party/opencv-2.4.13/modules/highgui/src/window_QT.cpp:47:
/usr/include/x86_64-linux-gnu/qt5/QtWidgets/qopenglwidget.h:49:38: error: expected initializer before ‘:’ token
 class Q_WIDGETS_EXPORT QOpenGLWidget : public QWidget
                                      ^
modules/highgui/CMakeFiles/opencv_highgui.dir/build.make:605: recipe for target 'modules/highgui/CMakeFiles/opencv_highgui.dir/opencv_highgui_automoc.cpp.o' failed
make[2]: *** [modules/highgui/CMakeFiles/opencv_highgui.dir/opencv_highgui_automoc.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
modules/highgui/CMakeFiles/opencv_highgui.dir/build.make:197: recipe for target 'modules/highgui/CMakeFiles/opencv_highgui.dir/src/window_QT.cpp.o' failed
make[2]: *** [modules/highgui/CMakeFiles/opencv_highgui.dir/src/window_QT.cpp.o] Error 1
CMakeFiles/Makefile2:1955: recipe for target 'modules/highgui/CMakeFiles/opencv_highgui.dir/all' failed
make[1]: *** [modules/highgui/CMakeFiles/opencv_highgui.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2
cp: cannot stat 'lib/cv2.so': No such file or directory

is this error about qt?

the cmake content about QT is as follows:

--   GUI: 
--     QT 5.x:                      YES (ver 5.5.1)
--     QT OpenGL support:           YES (Qt5::OpenGL 5.5.1)
--     OpenGL support:              YES (/usr/lib/x86_64-linux-gnu/libGLU.so /usr/lib/x86_64-linux-gnu/libGL.so)
--     VTK support:                 NO

how to fix it?
looking forward to your reply.

I would like to know the original results on 'plastering.avi' for the rgb and the flow stream.
Also, Im trying to use this code with python 3.6 and opencv 3.4. Is this possible to compile?

When I am trying to execute classify.py, I get the following error:

File "examples/classify_video.py", line 13, in
anet_home = os.environ['ANET_HOME']
File "/Users/Kousik/anaconda/envs/py27/lib/python2.7/UserDict.py", line 40, in getitem
raise KeyError(key)
KeyError: 'ANET_HOME'

Hi ,Thanks for sharing your resnet model! I want to finetune the resnet 200 model on my own rgb dataset which consists of 20 classes. i change the deploy file to train-val.prototxt, add 4 layers: Train VideoData layer，Test VideoData layer, prob layer , perbt layer.
This twoVideoData layers crop_size is 224. I think they are ok.
Details are:
layer {
name: "lkaction"
type: "InnerProduct"
bottom: "caffe.Flatten_673"
top: "lkaction"
inner_product_param {
num_output: 20
axis: -1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
bottom: "lkaction"
bottom: "label"
top: "prob"
name: "prob"
type: "SoftmaxWithLoss"
include {
phase: TRAIN
}
}

layer {
name: "probt"
type: "Softmax"
bottom: "lkaction"
top: "probt"
include {
phase: TEST
}
}

The log says that
I1111 12:46:10.121731 38914 caffe.cpp:86] Finetuning from /home/lk/anet2016-cuhk/models/resnet200_anet_2016.caffemodel
F1111 12:46:10.761404 38915 blob.cpp:455] Check failed: ShapeEquals(proto) shape mismatch (reshape not set)

Could you please tell me how to debug it?

hi. i'm trying to run your code on linux. I see there are 2 enviorment variables that are not configured anywhere.
ANET_HOME and ANET_CFG. to what folders should I point them too?

I am getting the following error - Cannot use GPU in CPU-only Caffe: check mode.. Can it be corrected by modifying classify_video.py? If possible can somebody suggest the modifications I need to make to run the example successfully without GPU.

Does anyone build the code on Ubuntu 16.04?

Thanks for your sharing!
Congratulations on the 1st in the ActivityNet competition! Could you please answer the following
2 questions.I did not find answers according to your notepaper(http://wanglimin.github.io/contests/XiongW_Anet16.pdf):

1. The model of rgb and optical flow is resnet and inceptionv3. I want to know what model you used for the audio cnn? alexnet?Train the audio cnn from scratch or finetune the model which pretrained on the imagenet
2. What is your sample rate when you construct your train dataset: for rgb cnn, 1fps? What about the optical flow cnn?

after run docker_server build_all.sh, opencv is installed successfully, but when build dense_flow ,I meet the question, my cuda version is 9.0
[ 50%] Building CXX object CMakeFiles/extract_warp_gpu.dir/tools/extract_warp_flow_gpu.cpp.o
[ 64%] Building CXX object CMakeFiles/extract_gpu.dir/tools/extract_flow_gpu.cpp.o
[ 64%] Building CXX object CMakeFiles/extract_cpu.dir/tools/extract_flow.cpp.o
[ 71%] Building CXX object CMakeFiles/pydenseflow.dir/src/py_denseflow.cpp.o
[ 78%] Linking CXX executable extract_gpu
[ 85%] Linking CXX executable extract_warp_gpu
[ 92%] Linking CXX executable extract_cpu
/usr/bin/ld: cannot find -lcudart
/usr/bin/ld: cannot find -lnppc
/usr/bin/ld: cannot find -lnppial
/usr/bin/ld: cannot find -lnppicc
/usr/bin/ld: cannot find -lnppicom
/usr/bin/ld: cannot find -lnppidei
/usr/bin/ld: cannot find -lnppif
/usr/bin/ld: cannot find -lnppig
/usr/bin/ld: cannot find -lnppim
/usr/bin/ld: cannot find -lnppist
/usr/bin/ld: cannot find -lnppisu
/usr/bin/ld: cannot find -lnppitc
/usr/bin/ld: cannot find -lnpps
/usr/bin/ld: cannot find -lcufft
collect2: error: ld returned 1 exit status
CMakeFiles/extract_cpu.dir/build.make:117: recipe for target 'extract_cpu' failed
make[2]: *** [extract_cpu] Error 1
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/extract_cpu.dir/all' failed
make[1]: *** [CMakeFiles/extract_cpu.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/usr/bin/ld: cannot find -lcudart
CMakeFiles/extract_gpu.dir/build.make:117: recipe for target 'extract_gpu' failed
/usrmake[2]: *** [extract_gpu] Error 1

Hi,
I am trying to install caffe using your build_all.sh. I have GPU with CUDA support the installed version of CUDA on my machine is 7.0. During caffe installation I get many cuda related errors such as :

/data05/A/ActivityNet/anet2016-cuhk/lib/caffe-action/include/caffe/neuron_layers.hpp(503): error: identifier "cudnnActivationDescriptor_t" is undefined

/data05/A/ActivityNet/anet2016-cuhk/lib/caffe-action/include/caffe/neuron_layers.hpp(587): error: identifier "cudnnActivationDescriptor_t" is undefined

/data05/A/ActivityNet/anet2016-cuhk/lib/caffe-action/include/caffe/neuron_layers.hpp(673): error: identifier "cudnnActivationDescriptor_t" is undefined

Is there any specific version of CUDA I need to install on machine? Could you please help me out to install caffe-action?

I get the following error while I am trying to execute classify_video.py:

File "examples/classify_video.py", line 18, in
from pyActionRec.action_classifier import ActionClassifier
File "/home/user/koushik/anet2016-cuhk-master/pyActionRec/action_classifier.py", line 3, in
from action_flow import FlowExtractor
File "/home/user/koushik/anet2016-cuhk-master/pyActionRec/action_flow.py", line 7, in
from libpydenseflow import TVL1FlowExtractor
ImportError: No module named libpydenseflow

Build of denseflow was successful.
Got the following while trying to build denseflow:

-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/gcc-4.8
-- Check for working C compiler: /usr/bin/gcc-4.8 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++-4.8
-- Check for working CXX compiler: /usr/bin/g++-4.8 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Performing Test COMPILER_SUPPORTS_CXX0X
-- Performing Test COMPILER_SUPPORTS_CXX0X - Success
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.26")
-- Found LIBZIP: /usr/lib/libzip.so
-- Boost version: 1.54.0
-- Found the following Boost libraries:
-- python
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found version "2.7.6")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/koushik/anet2016-cuhk-master/lib/dense_flow/build

Your Action Recognition Web Demo is not working.
It shows

504 Gateway Time-out
The server didn't respond in time.

As an academic researcher, I must unequivocally state that the code is non-functional and video classification cannot be achieved with it.

I succesfully built caffe but I get the following error:

Traceback (most recent call last):
File "examples/classify_video.py", line 12, in
import caffe
File "/home/user/koushik/anet2016-cuhk/lib/caffe-action/python/caffe/init.py", line 1, in
from .pycaffe import Net, SGDSolver
File "/home/user/koushik/anet2016-cuhk/lib/caffe-action/python/caffe/pycaffe.py", line 13, in
from ._caffe import Net, SGDSolver
ImportError: No module named _caffe

I am using the caffe given by this git repo.

Much appreciation for making the framework publicly available.
However, due to the resources and time required for training resnet, inappropriate parameter settings could be time-consuming and futile. I have been trying to reproduce the training process of resnet. As the ActivityNet Challenge is over, is it ok to release your training settings for resnet beforehand? Thanks

Hi, yuanjun,
Thanks for your code and models.
I run your code and models on activitynet1.3, but the top1 accuracy is just 75.6% (RGB stream), I have no idea why.
I feed your code with the video path.
What maybe the reasons, can you give me some ideas?
Thanks very much.

I am curious about what is the intuition behind the last line of pyActionRec/utils/io.py ? It doesn't make sense to me. Is this just a trick you happened to find working well in practice?

crops[5:, range(0, stack_depth, 2), ...] = 255 - crops[5:, range(0, stack_depth, 2), ...]

Run example python examples/classify_video.py data/plastering.avi with error:

libprotobuf ERROR google/protobuf/text_format.cc:172] Error parsing text-format caffe.NetParameter: 52:12: Message type "caffe.LayerParameter" has no field named "bn_param".
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0901 16:20:27.977401 28675 upgrade_proto.cpp:79] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: models/resnet200_anet_2016_deploy.prototxt
*** Check failure stack trace: ***
Aborted (core dumped)

I am receiving this error while trying to run the classify_video.py. I had to modify the build_all.sh file to make it work. Specifically libboost1.55-all-dev couldn't be installed, so I installed the 1.58 version using libboost-all-dev. Could this error be possibly linked to it, since the undefined symbol seems related to boost. I am running the code on Ubuntu 16.04. Should I run this on Ubuntu 14.04 or some other version?

On running -
$ ANET_HOME="/home/user/Documents/anet2016-cuhk" python examples/classify_video.py --use_flow data/plastering.avi

Traceback (most recent call last):
File "examples/classify_video.py", line 17, in
from pyActionRec.action_classifier import ActionClassifier
File "/home/user/Documents/anet2016-cuhk/pyActionRec/action_classifier.py", line 3, in
from action_flow import FlowExtractor
File "/home/user/Documents/anet2016-cuhk/pyActionRec/action_flow.py", line 7, in
from libpydenseflow import TVL1FlowExtractor
ImportError: lib/dense_flow//build/libpydenseflow.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE

The script file I used to bulid the code.
modified_build_all.txt

When I run the code, it reports that
"
[libprotobuf ERROR google/protobuf/text_format.cc:274] Error parsing text-format caffe.NetParameter: 52:12: Message type "caffe.LayerParameter" has no field named "bn_param".

F0820 16:12:43.238001 8162 upgrade_proto.cpp:90] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: models/resnet200_anet_2016_deploy.prototxt
"
It seems that there are some problems with the model file.. Hou can I deal with it...

Hello!

I'm a Teaching Assistant at the University of Toronto interested in demoing your algorithm to students of Prof. Sven Dickinson's course "PMU199Y Can we get a robot to see like a human?"

It would be nice if we could use your web demo for a live demonstration as part of a lab. However, your link seems to be down.

Any help would be appreciated.

David

I build the codebase successfully by running sh build_all.sh，but when running ANET_HOME="/root/caffe_video_end2end" python examples/classify_video.py data/plastering.avi
error happens as fellow：
Traceback (most recent call last):
File "examples/classify_video.py", line 17, in
from pyActionRec.action_classifier import ActionClassifier
File "/root/caffe_video_end2end/pyActionRec/action_classifier.py", line 3, in
from action_flow import FlowExtractor
File "/root/caffe_video_end2end/pyActionRec/action_flow.py", line 7, in
from libpydenseflow import TVL1FlowExtractor
ImportError: /root/caffe_video_end2end/3rd-party/opencv-2.4.13/build/lib/libopencv_gpu.so.2.4: undefined symbol: _ZN2cv3gpu9convertToERKNS0_6GpuMatERS1_ddP11CUstream_st

above says the libopencv_gpu.so.2.4 is an undefined symbol，but in 3rd-party/opencv-2.4.13/build/lib by using ls -l，i can see that：
libopencv_gpu.so -> libopencv_gpu.so.2.4
libopencv_gpu.so.2.4 -> libopencv_gpu.so.2.4.13
libopencv_gpu.so.2.4.13

which means the libopencv_gpu.so.2.4 has already linked to libopencv_gpu.so.2.4.13.

By the way，i have tested your caffe_tsn codebase，the linkage relation is the same like above，but i can run extract features、test、train correctly.

and i also compile the anet2016-cuhk code by using the caffe_tsn build_all.sh, the linkage relation also shows libopencv_gpu.so.2.4 has already linked to libopencv_gpu.so.2.4.13. but it still shows the above bug.

/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:38:25: error: ‘class caffe::TransformationParameter’ has no member named ‘max_distort’
max_distort_ = param_.max_distort();
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:40:30: error: ‘class caffe::TransformationParameter’ has no member named ‘scale_ratios_size’
for (int i = 0; i < param_.scale_ratios_size(); ++i){
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:41:43: error: ‘class caffe::TransformationParameter’ has no member named ‘scale_ratios’
custom_scale_ratios_.push_back(param_.scale_ratios(i));
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:43:26: error: ‘const class caffe::TransformationParameter’ has no member named ‘original_image’
org_size_proc_ = param.original_image();
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp: In member function ‘void caffe::DataTransformer::Transform(const caffe::Datum&, Dtype*)’:
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:163:38: error: ‘class caffe::TransformationParameter’ has no member named ‘multi_scale’
const bool do_multi_scale = param_.multi_scale();
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:226:18: error: ‘class caffe::TransformationParameter’ has no member named ‘fix_crop’
if (param_.fix_crop()){
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:228:30: error: ‘class caffe::TransformationParameter’ has no member named ‘more_fix_crop’
param_.more_fix_crop(), offset_pairs);
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:282:23: error: ‘class caffe::TransformationParameter’ has no member named ‘is_flow’
if (param_.is_flow() && do_mirror && c%2 == 0)
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:287:23: error: ‘class caffe::TransformationParameter’ has no member named ‘is_flow’
if (param_.is_flow() && do_mirror && c%2 == 0)
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:294:23: error: ‘class caffe::TransformationParameter’ has no member named ‘is_flow’
if (param_.is_flow() && do_mirror && c%2 == 0)
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:299:23: error: ‘class caffe::TransformationParameter’ has no member named ‘is_flow’
if (param_.is_flow() && do_mirror && c%2 == 0)
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp: In member function ‘void caffe::DataTransformer::Transform(const caffe::Datum&, const caffe::Datum&, caffe::Blob, caffe::Blob)’:
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:344:14: error: ‘class caffe::TransformationParameter’ has no member named ‘scale_ratios_size’
if (param_.scale_ratios_size() == 2)
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:346:26: error: ‘class caffe::TransformationParameter’ has no member named ‘scale_ratios’
lower_scale = param_.scale_ratios(0);
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:347:26: error: ‘class caffe::TransformationParameter’ has no member named ‘scale_ratios’
upper_scale = param_.scale_ratios(1);
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:353:29: error: ‘class caffe::TransformationParameter’ has no member named ‘stride’
const int stride = param_.stride();
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:380:14: error: ‘class caffe::TransformationParameter’ has no member named ‘has_upper_size’
if (param_.has_upper_size())
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:382:48: error: ‘class caffe::TransformationParameter’ has no member named ‘upper_size’
crop_height = std::min(crop_height, param_.upper_size());
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:383:46: error: ‘class caffe::TransformationParameter’ has no member named ‘upper_size’
crop_width = std::min(crop_width, param_.upper_size());
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:385:19: error: ‘class caffe::TransformationParameter’ has no member named ‘has_upper_height’
else if (param_.has_upper_height() && param_.has_upper_width())
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:385:48: error: ‘class caffe::TransformationParameter’ has no member named ‘has_upper_width’
else if (param_.has_upper_height() && param_.has_upper_width())
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:387:48: error: ‘class caffe::TransformationParameter’ has no member named ‘upper_height’
crop_height = std::min(crop_height, param_.upper_height());
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:388:46: error: ‘class caffe::TransformationParameter’ has no member named ‘upper_width’
crop_width = std::min(crop_width, param_.upper_width());
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp: In member function ‘void caffe::DataTransformer::Transform(const cv::Mat&, caffe::Blob)’:
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:582:38: error: ‘class caffe::TransformationParameter’ has no member named ‘multi_scale’
const bool do_multi_scale = param_.multi_scale();
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:635:20: error: ‘class caffe::TransformationParameter’ has no member named ‘fix_crop’
if (param_.fix_crop()) {
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:637:32: error: ‘class caffe::TransformationParameter’ has no member named ‘more_fix_crop’
param_.more_fix_crop(), offset_pairs);
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:714:22: error: ‘class caffe::TransformationParameter’ has no member named ‘is_flow’
if (param_.is_flow() && do_mirror && c % 2 == 0)
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:722:24: error: ‘class caffe::TransformationParameter’ has no member named ‘is_flow’
if (param_.is_flow() && do_mirror && c % 2 == 0)
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:729:24: error: ‘class caffe::TransformationParameter’ has no member named ‘is_flow’
if (param_.is_flow() && do_mirror && c % 2 == 0)
^
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp: In member function ‘void caffe::DataTransformer::Transform(caffe::Blob, caffe::Blob*)’:
/home/frank/PycharmProjects/end/caffe-action_recog/src/caffe/data_transformer.cpp:838:23: error: ‘class caffe::TransformationParameter’ has no member named ‘is_flow’
if (param_.is_flow() && c%2 == 0)
^
src/caffe/CMakeFiles/caffe.dir/build.make:81480: recipe for target 'src/caffe/CMakeFiles/caffe.dir/data_transformer.cpp.o' failed
make[2]: *** [src/caffe/CMakeFiles/caffe.dir/data_transformer.cpp.o] Error 1
CMakeFiles/Makefile2:240: recipe for target 'src/caffe/CMakeFiles/caffe.dir/all' failed
make[1]: *** [src/caffe/CMakeFiles/caffe.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2