foolwood / siammask Goto Github PK

load pretrained model from SiamMask_DAVIS.pth
[2019-03-10 04:08:45,841-rk0-load_helper.py# 25] remove prefix 'module.'
[2019-03-10 04:08:45,843-rk0-load_helper.py# 18] used keys:356
: cannot connect to X server

It seems that in the Refine function, the target feature is not center placed in the cropped patch but why?

The evaluation code of VOT2016 may be set to low=108, high=371 and peak=168. I used the matlab vot-toolkit to get above data.

(siammask) [liqiang@inspur siammask]$ bash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2016 0
[2019-03-14 19:42:16,619-rk0-test.py#551] Namespace(arch='Custom', config='config_vot.json', dataset='VOT2016', gt=False, log='log_test.txt', mask=True, refine=True, resume='SiamMask_VOT.pth', save_mask=False, visualization=False)
[2019-03-14 19:42:17,087-rk0-load_helper.py# 31] load pretrained model from SiamMask_VOT.pth
[2019-03-14 19:42:20,119-rk0-load_helper.py# 25] remove prefix 'module.'
[2019-03-14 19:42:20,121-rk0-load_helper.py# 18] used keys:356
[2019-03-14 19:42:20,184-rk0-test.py#595] Total Lost: 0
/export/userhome/liqiang/liqiang/Anaconda/ENTER/envs/siammask/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2925: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/export/userhome/liqiang/liqiang/Anaconda/ENTER/envs/siammask/lib/python3.6/site-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
[2019-03-14 19:42:20,185-rk0-test.py#597] Mean Speed: nan FPS

when I run this file as what you write on readme.md, the result is like this
what's wrong with this error?

I met the following trouble when i running the demo.py, and I've made a lot of effort to solve it, but can't.

ImportError: Building module region failed: ["distutils.errors.CompileError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2\n"]

Could anybody give me a hand? Thank you very much in advance!

D:\anaconda\envs\pytorch\python.exe D:/python_work/SiamMask-master/tools/demo.py
usage: demo.py [-h] --resume PATH [--config CONFIG] [--base_path BASE_PATH]
demo.py: error: the following arguments are required: --resume

run python ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json and get a error:
Traceback (most recent call last):
File "../tools/demo.py", line 7, in
from tools.test import *
ModuleNotFoundError: No module named 'tools'
Can someone help me solve this problem? Thank you.

Thanks for your great work! I have a difficulty to understand why and how to use COCO dataset to train the SiamMask?
Thanks

So far, only get (0.421/0.598/0.234) based on SiamMask_VOT_LD.pth and still have large gap@Accuracy (0.598 vs. 0.615). Would like to know the possible root cause, thanks ~

Hello.
I met the following trouble when i running the demo.py, and I've made a lot of effort to solve it, but can't.

Traceback (most recent call last):
  File "/home/aibc/Wen/Siamese/SiamMask/tools/demo.py", line 12, in <module>
    from tools.test import siamese_track,siamese_init
ModuleNotFoundError: No module named 'tools'

Thanks for your help.

Sorry，if I want to train this network, can i use only 1 dataset from Youtube-VOS， COCO， ImageNet-DET and ImageNet-VID, or all. thanks!

Hi, I'm wondering siammask's performance on OTB. I evaluated on OTB100 using siammask-3B bbox branch output with siammask_vot.pth with tracking parameters same as DaSiamRPN but only got 0.629/0.844 with AUC/Prec. Is this intrinsic or because of sensitivity to the hyper-paramerters?

Thanks for your time.

This is really a great project
But can you open up its training code and training details, I want to know more about SiamMask, it is very helpful to me.

when i download VOT2019, I got this issues, Why?

would you like share your three pretrained model on baiduyun or googledrive when you are free, thank you!!

python ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json
Traceback (most recent call last):
File "../../tools/demo.py", line 20, in
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
AttributeError: 'module' object has no attribute 'device'

How can I solve the problem? Thank you very much.

Hello sir!
I want to use uav123 to evaluate it.But siammask use 8 parameters to describe the rectangle . But like Siam-RPN just use 4 . So if I want to evaluate the precision and success of it ,what conversion of rectangle should I do?
Thanks a lot for your reply!

Hi,Qiang
Thks for your contribution!
       I trained and tested the SiamRPN++ network you implemented and studied the code. Some questions now:

1. The output model structure (only layer3 in the custom version of resnet50, why?) is different from the SiamRPN++ model in pysot.

2. SiamRPN++ indicates in paper that the feature fusion of layer2_RPN, layer3_RPN and layer4_RPN in ResNet50 is not found in your code.

look forward to your reply.

I'm really interested in your amazing work and I've tried it ,here I would want to ask for your advice:
Are there any methods for scale variation? Because I found it not so effective to face the challenge,can you give some advice?.

This problem arises when testing vot. I hope you can help me with it. Thanks

File "tools/demo.py", line 25, in
cfg = load_config(args)
File "/home/zz/SiamMask/utils/config_helper.py", line 11, in load_config
assert exists(args.config), '"{}" not exists'.format(args.config)
AssertionError: "config_davis.json" not exists
but there is config_davis.json in siammask,what should i do?Thanks for your reply!

python3 ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json
[2019-05-14 16:31:21,238-rk0-load_helper.py# 31] load pretrained model from SiamMask_DAVIS.pth
[2019-05-14 16:31:23,679-rk0-load_helper.py# 25] remove prefix 'module.'
[2019-05-14 16:31:23,680-rk0-load_helper.py# 18] used keys:356
Xlib: extension "RANDR" missing on display ":0".
Select a ROI and then press SPACE or ENTER button!
Cancel the selection process by pressing c button!
/home/sdu/.local/lib/python3.5/site-packages/torch/nn/functional.py:1890: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
Traceback (most recent call last):
File "../../tools/demo.py", line 54, in
state = siamese_track(state, im, mask_enable=True, refine_enable=True) # track
File "/home/sdu/SiamMask/tools/test.py", line 285, in siamese_track
_, contours, _ = cv2.findContours(target_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
ValueError: not enough values to unpack (expected 3, got 2)

I haven't corrected the code in it, but after drawing the area and entering the "Enter" key, the above error occurred. Do you have a solution? Thank you very much.

我想用下 DAVIS model，但是想放在 pysot 下使用，请问该怎么转换？

Hello, anyone try this SiamMask project in ios? Can it is real-time and have a good result?

Hello.
I met the following trouble when i running the demo.py.

Thanks for your help.

As the title, sir.

What are the data format to consider for training SiamMask? And what are the steps to follow to train it on this data?
Many thanks for your interesting work.

Hi Qiang,
After training siamrpn for about 200 iterations I spotted nan in the loss. Do you think it is the problem of data preprocessing or somewhere in the code? Thank you.
Full training log attached below.
Best wishes,
Yiming

[2019-05-30 20:07:36,388-rk0-train_siamrpn.py#281] Epoch: [1][140/2343] lr: 0.001000    batch_time: 0.447263 (0.852683) data_time: 0.000222 (0.096190)  rpn_cls_loss: 0.348680 (0.452594)       rpn_loc_loss: 0.327530 (0.406526)       siamrpn_loss: 0.676210 (0.859119)
[2019-05-30 20:07:36,389-rk0-log_helper.py# 97] Progress: 140 / 46860 [0%], Speed: 0.853 s/iter, ETA 0:11:03 (D:H:M)

[2019-05-30 20:07:41,014-rk0-train_siamrpn.py#281] Epoch: [1][150/2343] lr: 0.001000    batch_time: 0.428898 (0.826568) data_time: 0.000247 (0.089822)  rpn_cls_loss: 0.330786 (0.445250)       rpn_loc_loss: 0.339954 (0.400595)       siamrpn_loss: 0.670740 (0.845844)
[2019-05-30 20:07:41,014-rk0-log_helper.py# 97] Progress: 150 / 46860 [0%], Speed: 0.827 s/iter, ETA 0:10:43 (D:H:M)

[2019-05-30 20:07:45,451-rk0-train_siamrpn.py#281] Epoch: [1][160/2343] lr: 0.001000    batch_time: 0.491518 (0.802561) data_time: 0.000388 (0.084235)  rpn_cls_loss: 0.332730 (0.438211)       rpn_loc_loss: 0.321817 (0.395307)       siamrpn_loss: 0.654547 (0.833518)
[2019-05-30 20:07:45,451-rk0-log_helper.py# 97] Progress: 160 / 46860 [0%], Speed: 0.803 s/iter, ETA 0:10:24 (D:H:M)

[2019-05-30 20:07:50,009-rk0-train_siamrpn.py#281] Epoch: [1][170/2343] lr: 0.001000    batch_time: 0.449069 (0.782116) data_time: 0.000453 (0.079310)  rpn_cls_loss: 0.321972 (0.431562)       rpn_loc_loss: 0.312462 (0.390696)       siamrpn_loss: 0.634434 (0.822258)
[2019-05-30 20:07:50,009-rk0-log_helper.py# 97] Progress: 170 / 46860 [0%], Speed: 0.782 s/iter, ETA 0:10:08 (D:H:M)

[2019-05-30 20:07:54,762-rk0-train_siamrpn.py#281] Epoch: [1][180/2343] lr: 0.001000    batch_time: 0.511367 (0.764989) data_time: 0.000424 (0.074935)  rpn_cls_loss: 0.293845 (0.425333)       rpn_loc_loss: 0.274334 (nan)    siamrpn_loss: 0.568179 (nan)
[2019-05-30 20:07:54,763-rk0-log_helper.py# 97] Progress: 180 / 46860 [0%], Speed: 0.765 s/iter, ETA 0:09:55 (D:H:M)

[2019-05-30 20:07:59,350-rk0-train_siamrpn.py#281] Epoch: [1][190/2343] lr: 0.001000    batch_time: 0.401666 (0.748833) data_time: 0.000397 (0.071019)  rpn_cls_loss: 0.301740 (0.419603)       rpn_loc_loss: 0.295769 (nan)    siamrpn_loss: 0.597508 (nan)

Hello, thanks for sharing your code. I encounted a problem when running 'demo.py'. I did all accoding to the Readme.
(siammask) bill@bill-QiTianM4550-N000:~/Documents/SiamMask/experiments/siammask$ python ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json
Traceback (most recent call last):
File "../../tools/demo.py", line 7, in
from tools.test import *
File "/home/bill/Documents/SiamMask/tools/test.py", line 30, in
from utils.pyvotkit.region import vot_overlap, vot_float2str
File "/home/bill/Documents/SiamMask/utils/pyvotkit/init.py", line 9, in
from . import region
ImportError: cannot import name 'region'
Then I deleted this project and reinstall it. However it shows again.

Did you also train SiamMask using random pairs of the same sequence from DAVIS dataset? I'm just curious because DAVIS has much much more occlusions than VOT. I doubt if siamese network works well on DAVIS. It's too difficult if the template is one part of the object and the target is another part of the object.

Could you share the training code?

The server runs the program. When running the demo, the following thread problems occur. How to solve them：

QObject::moveToThread: Current thread (0x2b3fd7fb8ff0) is not the object's thread (0x2b3fde28b410).
Cannot move to target thread (0x2b3fd7fb8ff0)

QPixmap: Must construct a QApplication before a QPaintDevice
Aborted (core dumped)

If training with the refinement module, what is the loss for mask you use? a) Eq. 3 in the paper; b) Eq.3 plus something like binary cross entropy for the final mask; c)binary cross entropy for the final mask?

Sir ,I'm very interesting in your thought about Siamese network.I want to run this code(SiamMask) in the win10.But there are some wrong in the VScode when it's running.The problem showed is that " ImportError
Building module utils.pyvotkit.region failed: ["TypeError: '>=' not supported between instances of 'NoneType' and 'str'\n"]".
And all package have been installed.It's running in VScode ,python 3.6.What should I do?I'm looking forward to your reply.
Thanks a lot.

Can you share where I can access the 'resnet.model' weights, the pretrained resnet weights. And where the load_pretrain method is?

From line 37 of custom.py

@foolwood I am trying to understand the pipeline of the Siammask from the code. I found that you have instantiated multiple objects of "DepthCorr"-- separate for classification, regression and masking. Though in paper, I see the figure where correlation is shared by all the three head blocks.
Can you explain me if I am missing something or not understanding it properly?

(siammask) [liqiang@inspur trackdat]$ VOT_YEAR=2016 bash scripts/download_vot.sh dl/vot2016
--2019-03-14 16:18:38-- http://data.votchallenge.net/vot2016/main/description.json
Resolving data.votchallenge.net (data.votchallenge.net)... 194.249.1.179
Connecting to data.votchallenge.net (data.votchallenge.net)|194.249.1.179|:80... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

The file is already fully retrieved; nothing to do.

scripts/download_vot.sh: line 19: jq: command not found
scripts/download_vot.sh: line 20: jq: command not found
scripts/download_vot.sh: line 21: jq: command not found

@foolwood where is this problem come from? anybody please help me to locate it

Before that, I read a paper on deep learning in target tracking called MDnet. MDnet seems to track only specific categories. I don't know if the same is true of Siamese, who is trained to track only certain categories of objects.

Traceback (most recent call last):
File "../../tools/demo.py", line 52, in
state = siamese_init(im, target_pos, target_sz, siammask, cfg['hp']) # init tracker
File "/home/lut/SiamMask/tools/test.py", line 157, in siamese_init
z_crop = get_subwindow_tracking(im, target_pos, p.exemplar_size, s_z, avg_chans)
File "/home/lut/SiamMask/tools/test.py", line 109, in get_subwindow_tracking
im_patch = cv2.resize(im_patch_original, (model_sz, model_sz))
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/resize.cpp:4044: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

if my object out of my window,it always have bbox.Can you solve it??thanks

select multi ROI?

flake8 testing of https://github.com/foolwood/SiamMask on Python 3.7.1

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./utils/pysot/utils/statistics.py:57:21: F821 undefined name 'x'
        overlaps = [x if u else 0 for u in unkown]
                    ^
1     F821 undefined name 'x'
1

E901,E999,F821,F822,F823 are the "showstopper" flake8 issues that can halt the runtime with a SyntaxError, NameError, etc. These 5 are different from most other flake8 issues which are merely "style violations" -- useful for readability but they do not effect runtime safety.

• F821: undefined name name
• F822: undefined name name in __all__
• F823: local variable name referenced before assignment
• E901: SyntaxError or IndentationError
• E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree

Hello, I run the project with only 10 fps on a titan xp, which is much slower than mentioned in README.md. Does anyone know how to accelerate? How to achieve a speed like real-time tracking?

There are some postprocessing methods? I can not obtain the results shown in the paper. The segmentation results use CRF?

Traceback (most recent call last):
File "../../tools/demo.py", line 56, in
state = siamese_track(state, im, mask_enable=True, refine_enable=True) # track
File "/home/zhouwenzhang/SiamMask-master/tools/test.py", line 254, in siamese_track
mask = net.track_refine((delta_y, delta_x)).cuda().sigmoid().squeeze().view(
File "/home/zhouwenzhang/SiamMask-master/experiments/siammask/custom.py", line 157, in track_refine
pred_mask = self.refine_model(self.feature, self.corr_feature, pos=pos)
File "/opt/conda/envs/pytorch-py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 468, in call
result = self.forward(*input, **kwargs)
File "/home/zhouwenzhang/SiamMask-master/experiments/siammask/custom.py", line 123, in forward
out = self.post0(F.upsample(self.h2(out) + self.v2(p2), size=(31, 31)))
File "/opt/conda/envs/pytorch-py3.6/lib/python3.6/site-packages/torch/nn/functional.py", line 1797, in upsample
return torch._C._nn.upsample_nearest2d(input, _scale_factor(2))
File "/opt/conda/envs/pytorch-py3.6/lib/python3.6/site-packages/torch/nn/functional.py", line 1769, in _scale_factor
'x'.join(map(str, input.size()))))
RuntimeError: output size specified in UpsamplingNearest (31x31) has to be divisible by the input size, but got: 1x32x15x15

I run this code （我运行这个指令）
-->conda create -n siammask python=3.6
then i find this problem （发现如下问题）

Solving environment: failed

CondaHTTPError: HTTP 404 NOT FOUND for url https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/mian/noarch/repodata.json
Elapsed: 00:00.046715

The remote server could not find the noarch directory for the
requested channel with url: https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/mian

As of conda 4.3, a valid channel must contain a noarch/repodata.json and
associated noarch/repodata.json.bz2 file, even if noarch/repodata.json is
empty. please request that the channel administrator create
noarch/repodata.json and associated noarch/repodata.json.bz2 files.
$ mkdir noarch
$ echo '{}' > noarch/repodata.json
$ bzip2 -k noarch/repodata.json

You will need to adjust your conda configuration to proceed.
Use conda config --show channels to view your configuration's current state.
Further configuration help can be found at https://conda.io/docs/config.html.

If I have multiple targets in the videos, could I track each of them using siamMask ?

Thanks !