stevenliuwen / ano_pred_cvpr2018 Goto Github PK

Hey! How did you create the .mat file that is being used. As far as I know .mat files are used in matlab.

Hi, I'm curious about that why you comment out this line of code: https://github.com/StevenLiuWen/ano_pred_cvpr2018/blob/master/Codes/evaluate.py#L334 ? I think it should be right with it added.

Thanks for sharing the code... Can you please explain how to run it on the live video feed

Dear Dr. LiuWen,
I am trying to train from scratch following your instruction. But I encountered the following error:

Traceback (most recent call last): File "train.py", line 94, in <module> height=flow_height, width=flow_width, reuse=None) File "/data1/ShanghaiTech-FutureFramePred/ano_pred_cvpr2018-master/Codes/models.py", line 25, in flownet net = FlowNetSD(mode=Mode.TEST) TypeError: Can't instantiate abstract class FlowNetSD with abstract methods loss

Could you help to check this ? Thank you very much

please can anyone explain how can I evaluate the result on videos to get frame by frame label for normal and abnormal??

The moving objets in videos of ShanghaiTech dataset are blurry.

你好
我拿了17个正常的地铁（包括高峰期和正常人流）视频，数据集均是按照跳2帧切一张图，17个文件夹接近20w的图片库，按照说明进行一些参数设置
--iters = 80000
--batch = 16
损失图如下，adversarial loss和train_psnr_error generator loss均是越来越大

Hi,thanks for your inspiring work. And I want to know how to run demo like in https://www.youtube.com/watch?v=M--wv-Y_h0A

你好，非常感谢你的开源代码！我想请教一下，在论文中，训练时你们使用了分块识别器(patch discriminator)，但在源码中，你们好像没有分块，而是pix2pix的识别器，所以想请教一下是我理解错了代码还是pix2pix的效果会更好？谢谢不吝指教～

Hi,

What are kind of video Anamolies that are detected in the pretrained model?

Thanks
Guru

您好，很高兴您能够开源这么优秀的代码。想请教您一个问题，如何训练自己的数据集呢？我想用您的方法做视频预测，下载ped1数据集，有个mat文件？请问mat文件如何生成，或者含义是什么？
期待您的回复！

First thank you for your interesting publication and well documented code example.
I had some path errors, which I bandaid-fixed by hard-coding relative paths.

  File "/home/Code/ano_pred_cvpr2018-master/Codes/evaluate.py", line 126, in __load_ucsd_avenue_subway_gt abnormal_events = scio.loadmat(mat_file, squeeze_me=True)['gt']

  File "/home/anaconda3/envs/deeplearning/lib/python3.6/site-packages/scipy/io/matlab/mio.py", line 141, in loadmat 

MR, file_opened = mat_reader_factory(file_name, appendmat, **kwargs)

  File "/home/anaconda3/envs/deeplearning/lib/python3.6/site-packages/scipy/io/matlab/mio.py", line 64, in mat_reader_factory 

byte_stream, file_opened = _open_file(file_name, appendmat)

TypeError: 'NoneType' object is not iterable

Worked after replacing it with:

abnormal_events = scio.loadmat('../Data/avenue/avenue.mat', squeeze_me=True)['gt']

Next path error afterwards was:

FileNotFoundError: [Errno 2] No such file or directory: '/public/home/gaoshenghua/liuwen/datasets/avenue/testing/frames'

Replacing all absolute paths with relative paths fixed this one.

TypeError: Can't instantiate abstract class FlowNetSD with abstract methods loss

刘博士，你好，谢谢你分享代码资源给我们。我在运行程序train.py过程中出现以下错误，找不到解决的办法，想麻烦你百忙之中帮我看下是什么问题。tensorflow.python.framework.errors_impl.InvalidArgumentError: ValueError: low >= high
[[Node: PyFunc = PyFuncTin=[DT_INT64], Tout=[DT_FLOAT], token="pyfunc_1"]]
[[Node: dataset/IteratorGetNext = IteratorGetNextoutput_shapes=[[?,256,256,15]], output_types=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
[[Node: dataset/IteratorGetNext/_117 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_44_dataset/IteratorGetNext", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

I would like to output all prediction and difference images during inference and save them as jpgs. Is there a suggested or intended way of doing that with your code? Was diff_mask intended for storing difference images?

Thanks!

for chinese students, onedrive is out of reach and I can not download the resources from the links you offerred. can you offer a new link for BaiduYunPan? thanks!

Hi, Dr. Liu Wen. Could you share a BaiduYun Link about the pretrained models and datasets of this paper ? There are some difficulties about downloading these files from OneDrive. Thanks a lot.

Thanks for sharing Code, I have few questions
How to test the code on new data(new video)?, I am not able to check output in tensorboard.
How to use it on live feed?
I tried training on avenue data, it was very slow..what was training time for you guys(I am using p2.x large instance of aws)

It would be great if you could demo/explain code somewhere..probably a jupyter notebook.

Traceback (most recent call last):
File "inference.py", line 149, in
inference_func(snapshot_dir, dataset_name, evaluate_name)
File "inference.py", line 88, in inference_func
psnrs[0:num_his] = psnrs[num_his]
IndexError: index 4 is out of bounds for axis 0 with size 0

The input to the two flownet models is distorted from aspect ratio 1:1(256x256) to 384x512. I understand that flownet needs inputs sizes dividable by 64.
I am trying to understand the reason for this? Why not just pad with zeros to avoid the distortion?

https://github.com/StevenLiuWen/ano_pred_cvpr2018/blob/master/Codes/models.py#L24

Tested it on some new data out of interest. Is this a behavior that you saw during your training https://imgur.com/a/fiSc0iQ ? Did you implement any loss balancing strategies for the generator and the discriminator, e.g. only train one, if it isn't too much stronger than its adversary etc.?

博士您好：
如果我想要在新的数据集上训练，那我应该怎么构建我的数据集呢，我看您生成了.MAT文件。有相应的参考或者制作训练数据集的方法吗？

Hi, nice to meet you. I would like to ask you to improve README file to be more practical and intuitive. Based on the issues list, it's possible to notice a lot of mistakes due to incomplete information. Please.

filter_x = tf.expand_dims(tf.stack([neg, pos]), 0) # [-1, 1]
filter_y = tf.stack([tf.expand_dims(pos, 0), tf.expand_dims(neg, 0)]) # [[1],[-1]]
filter_x like follow
[[[[-1. -0. -0.]
[-0. -1. -0.]
[-0. -0. -1.]]
[[ 1. 0. 0.]
[ 0. 1. 0.]
[ 0. 0. 1.]]]]

and how to get Operator [-1, 1] from 4-D filter_x ?

As I retrained the model on UCSD and avenue datasets, the result of AU of ROC on UCSD looks fine, but the AU of ROC on avenue is only 0.67 but AU of PR on avenue reached 0.8493.
Having made sure the evaluate_name is correctly. BTW, the psnr value is about 10 for avenue, but about 37 using your provided pretrained model.
Wonder if there is somewhere different for the setting to train on avenue? Thx.

Hello, could you help me answer some questions below?

1. What should I do if I want to train on my own data?
2. I wonder how I discovered the abnormal situation in the company after I ran inference. Py?Should there be a crest when something unusual happens?I hope to get your reply. Thank you.

Hi, Dr. Liu Wen. I have a problem about the last activation function of the generator. Why dose the last activation function in your code choose tanh rather than Relu? To my knowledge, the original U-net chose ReLu as the last activation function. I'm confused about this change. Could you give me some explanations about this problem? Thanks in advance.

Question1
I had few questions on the evaluation schemes provided in the paper. The PSNR evaluation scheme says that you have to calculate the regularity score by using the PSNR values of every test frame and then set a threshold. If my test video changes will the same threshold work? Can the same threshold value be used for similar type of videos?

Question2
In the paper where AUC of different methods is given. Could you please share how exactly this is calculated?

Question3
In the section of paper you describe "We first compute the gap between average score of normal frames and that of abnormal frames, denoted as ∆s". What is average score here?

Please provide the pre-trained models for ShanghaiTech dataset

could you give me the about Comparative experiment code which the paper named Arevisitofsparsecodingbased anomaly detection in stacked rnn framework.Tanks very much.

On CHUK Avenue clip 18, according to the pixel-level-label, all frames are abnormal, but in your offering label file it's not. So why?

I do step 3. Testing on saved models（I have download pretraines model and tar -xvf it into Codes file)

Then run
<<<python inference.py --dataset ped1
--test_folder ../Data/ped1/testing/frames
--gpu 1
--snapshot_dir checkpoints/pretrains/ped1

the problem comes like bellow：
<<<video = 36 / 36, i = 199 / 200, psnr = 36.892002
total time = 225.26929545402527, fps = 31.961745987124253
Traceback (most recent call last):
File "inference.py", line 149, in
inference_func(snapshot_dir, dataset_name, evaluate_name)
File "inference.py", line 101, in inference_func
results = evaluate.evaluate(evaluate_name, pickle_path)
File "/home/huangpan/git_download/Anomaly_Detection/Codes/evaluate.py", line 560, in evaluate
optimal_results = eval_func(save_file)
File "/home/huangpan/git_download/Anomaly_Detection/Codes/evaluate.py", line 405, in compute_auc
dataset, psnr_records, gt = load_psnr_gt(loss_file=sub_loss_file)
File "/home/huangpan/git_download/Anomaly_Detection/Codes/evaluate.py", line 256, in load_psnr_gt
gt = gt_loader(dataset=dataset)
File "/home/huangpan/git_download/Anomaly_Detection/Codes/evaluate.py", line 118, in call
gt = self.__load_ucsd_avenue_subway_gt(dataset)
File "/home/huangpan/git_download/Anomaly_Detection/Codes/evaluate.py", line 137, in __load_ucsd_avenue_subway_gt
.format(num_video, len(video_list))
AssertionError: ground true does not match the number of testing videos. 21 != 36

Who can tell me how to solve it？

the provided shanghaitech model cant reach 0.72 auc. use the default training setting also cant reach 0.72 auc on shanghaitech dataset.

Any idea?

i successfully run train.py in ped1 and ped2 datasets.but when i try to run train.py in avenue and shanghai tech datasets .there have been some errors as follows .and i don't know how to solve it .Could you help me ?Thank you (because of they are color dataset?)

Traceback (most recent call last):

File "/home/cvzw/anaconda3/envs/ALOCC-CVPR2018-master/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 209, in call
ret = func(*args)

File "/home/cvzw/anaconda3/envs/ALOCC-CVPR2018-master/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 514, in generator_py_func
values = next(generator_state.get_iterator(iterator_id))

File "/home/cvzw/PycharmProjects/zhouwei/future frame prediction/Codes/utils.py", line 54, in video_clip_generator
video_clip.append(np_load_frame(video_info['frame'][frame_id], resize_height, resize_width))

File "/home/cvzw/PycharmProjects/zhouwei/future frame prediction/Codes/utils.py", line 24, in np_load_frame
image_resized = cv2.resize(image_decoded, (resize_width, resize_height))

cv2.error: /io/opencv/modules/imgproc/src/imgwarp.cpp:3492: error: (-215) ssize.width > 0 && ssize.height > 0 in function resize

Thank you for your great work!

I am wondering exact hyperparameters when you get the best AUC score on Ped1 dataset.
Maybe the hyperparameters aside from iterations are written in your original paper.
Can you provide us with the iterations number you used as long as other parameters have been set as the same as paper?
Otherwise, can you provide us with the values of other parameters?

Thank you so much!

Hi, Steven. Thanks for your inspiring paper and code.
I want to know why you choose flownet-SD rather than flownet2(the final fused model) to generate optical flow. Did flownet2 show poor performance in your experiments?
Do you think a more accurate optical flow prediction will improve the final detection results?

Looking forward to your insight.

刘博士您好！我想要跑一下您的代码，在python3.6的环境下按您的txt安装了相关库，然后出现了如下报错：
/home/srp/anaconda3/envs/srp/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
Traceback (most recent call last):
File "/home/srp/anaconda3/envs/srp/lib/python3.6/configparser.py", line 1138, in _unify_values
sectiondict = self._sections[section]
KeyError: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "inference.py", line 10, in
from constant import const
File "/home/srp/chengduo/ano_pred_cvpr2018-master/Codes/constant.py", line 104, in
const.L_NUM = config.getint(const.DATASET, 'L_NUM')
File "/home/srp/anaconda3/envs/srp/lib/python3.6/configparser.py", line 819, in getint
fallback=fallback, **kwargs)
File "/home/srp/anaconda3/envs/srp/lib/python3.6/configparser.py", line 809, in _get_conv
**kwargs)
File "/home/srp/anaconda3/envs/srp/lib/python3.6/configparser.py", line 803, in _get
return conv(self.get(section, option, **kwargs))
File "/home/srp/anaconda3/envs/srp/lib/python3.6/configparser.py", line 781, in get
d = self._unify_values(section, vars)
File "/home/srp/anaconda3/envs/srp/lib/python3.6/configparser.py", line 1141, in _unify_values
raise NoSectionError(section)
configparser.NoSectionError: No section: None

其中第一段报错像是库和python3.6不兼容，所以我又在python3.5的环境下运行，报错如下：
Traceback (most recent call last):
File "/home/srp/anaconda3/envs/srpp/lib/python3.5/configparser.py", line 1135, in _unify_values
sectiondict = self._sections[section]
KeyError: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "inference.py", line 10, in
from constant import const
File "/home/srp/chengduo/ano_pred_cvpr2018-master/Codes/constant.py", line 104, in
const.L_NUM = config.getint(const.DATASET, 'L_NUM')
File "/home/srp/anaconda3/envs/srpp/lib/python3.5/configparser.py", line 816, in getint
fallback=fallback, **kwargs)
File "/home/srp/anaconda3/envs/srpp/lib/python3.5/configparser.py", line 806, in _get_conv
**kwargs)
File "/home/srp/anaconda3/envs/srpp/lib/python3.5/configparser.py", line 800, in _get
return conv(self.get(section, option, **kwargs))
File "/home/srp/anaconda3/envs/srpp/lib/python3.5/configparser.py", line 778, in get
d = self._unify_values(section, vars)
File "/home/srp/anaconda3/envs/srpp/lib/python3.5/configparser.py", line 1138, in _unify_values
raise NoSectionError(section)
configparser.NoSectionError: No section: None

看起来版本问题像是解决了，但第二段的bug依然存在。
这时候我注意到constant.py文件中的第30、31行：
parser.add_argument('-d', '--dataset', type=str,
help='the name of dataset.')

再对比上下文，发现漏掉了default参数，但我加上default=''之后依然报错。
因为我也只是个初学者，对代码不是很熟悉，所以想请教您问题出在哪里呢？

As the code shows, the scores are normalized every video clips, which makes the anomaly standard vary at each clips.
I make it norm for only one time for all the SCORES, and the result better than the paper with 85%+ AUC
my_re-implementation

    `for i in range(num_videos):
        distance = psnr_records[i]
        #if NORMALIZE:
            #distance = (distance-distance.min())/distance.max()  # distances = (distance - min) / (max - min)
            #distance /= distance.max()
            # distance = 1 - distance

        scores = np.concatenate((scores, distance[DECIDABLE_IDX:]), axis=0)
        labels = np.concatenate((labels, gt[i][DECIDABLE_IDX:]), axis=0)

    if NORMALIZE:
        scores-=scores.min()
        scores/=scores.max()

`

How do I make online inference?

Following command does not show anything. Is missing some step?

tensorboard --logdir=./summary --port=10086

Hi,
Many thanks for your great work!
Can you please help with code if i want to train your algorithm on my own dataset?

tahmidzbr

The link doesn't work, could you please upload a new link

The inference code works with batch size =1 ? is it written in batch mode to compute faster for larger dataset ?