torrvision / siamfc-tf Goto Github PK

The bounding boxes returned by the model are not concise especially when object scale is changing in the video stream(specially when object becomes smaller). Any hyper-parameters to tune to dampen this issue.

When i tried data from validation set everything is ok.
But i tried fast flying object(Fly) and tracker can even catch.
No success, what can be the reason?

/opt/github2/siamfc-tf# python3 run_tracker_evaluation.py
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/usr/lib/python3.5/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: libcublas.so.8.0: cannot open shared object file: No such file or directory

Hi, can anyone tell me what the coordinates in the ground truth file correspond to? Does it contain the top left and bottom right coordinates osomething else? opening the image in GIMP shows it might not the the coordinates. if not, what are thy?

Also what do the parameters inn_att.txt file mean?

Hi there, I am trying to run the code on Titanx, after using the command
export LD_LIBRARY_PATH=/usr/local/cuda/lib64 I got an error below

F tensorflow/core/kernels/conv_ops.cc:659] Check failed: stream->parent()->GetConvolveAlgorithms(&algorithms) 
Aborted (core dumped)

Can any give some suggestions? Thanks.

Hi @bertinetto, @jvlmdr ,
Could you help me out please?

I am getting following error in windows 10

Command "python setup.py egg_info" failed with error code 1 in c:\users\jumab\appdata\local\temp\pip-build-etluxy\matplotlib\
when I run
pip install -r requirements.txt in cmd (on administrator mode) in step 6

This is complete error message:

  Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "c:\users\jumab\appdata\local\temp\pip-build-etluxy\matplotlib\setup.py", line 268, in <module>
        **extra_args
      File "c:\anaconda2\Lib\distutils\core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "e:\code\ve-tracking\lib\site-packages\setuptools\dist.py", line 320, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "e:\code\ve-tracking\lib\site-packages\setuptools\dist.py", line 377, in fetch_build_eggs
        replace_conflicting=True,
      File "e:\code\ve-tracking\lib\site-packages\pkg_resources\__init__.py", line 852, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "e:\code\ve-tracking\lib\site-packages\pkg_resources\__init__.py", line 1124, in best_match
        return self.obtain(req, installer)
      File "e:\code\ve-tracking\lib\site-packages\pkg_resources\__init__.py", line 1136, in obtain
        return installer(requirement)
      File "e:\code\ve-tracking\lib\site-packages\setuptools\dist.py", line 445, in fetch_build_egg
        return cmd.easy_install(req)
      File "e:\code\ve-tracking\lib\site-packages\setuptools\command\easy_install.py", line 673, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "e:\code\ve-tracking\lib\site-packages\setuptools\command\easy_install.py", line 699, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "e:\code\ve-tracking\lib\site-packages\setuptools\command\easy_install.py", line 880, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "e:\code\ve-tracking\lib\site-packages\setuptools\command\easy_install.py", line 1119, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "e:\code\ve-tracking\lib\site-packages\setuptools\command\easy_install.py", line 1107, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: Microsoft Visual C++ 9.0 is required. Get it from http://aka.ms/vcpython27

Thx

Using Tensorflow 1.1.0

Traceback (most recent call last):
  File "/data/siamfc-tf/siamfc-tf-master/run_tracker_evaluation.py", line 174, in <module>
    sys.exit(main())
  File "/data/siamfc-tf/siamfc-tf-master/run_tracker_evaluation.py", line 16, in main
    hp, evaluation, run, env, design = parse_arguments()
  File "/data/siamfc-tf/siamfc-tf-master/src/parse_arguments.py", line 10, in parse_arguments
    evaluation = json.load(json_file)
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 380, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting object: line 8 column 1 (char 188)

what should I do?

In the training time, they mentioned images are extracted from two frames of a video that both contain the object and are at most T frames apart.
Is there any specific way to select T, or these 2 frames?

The link to the pretrained weights file in the ReadMe.md, step #8, is incorrect. It should point to (I believe) an archive containing "baseline-conv5_e55.mat". However, the link is a duplicate of the link pointing to "cfnet_validation_set.zip" (which is given in step #9, regarding the video sequences).

Since this version is not made to train yet, the project cannot be used at all until this file is provided.

the web can't open

I was trying to get this to work with a webcam...by using pyyolo. but could not get the tracker to work by repeated calls by image. Box would either track to image center or remain fixed and not track object. Any help how to set up tracker so to call it in a loop over images.

Problem solved

Thanks for this amazing work :)

I am currently using tfserving for the object tracking. Is there any way to convert this network to tensorflow model?

Thank you.

Thank you for releasing this code. But I have some problems. Could you help me?

Hello，I use validation data "tc_Airport_ce" and enable visualization, and run it successfully, but if some shallow occlusion occur, it will fail.
More precisely, occlusion haven't occur, but it already failed.
Is that normal?
Thank you~~

I have run your tracker . It shows an amazing result. But In my
environment，I still lose my target . I have not installed matlab. Can you provide a train script. Thanks.

During the definition of the function:
def build_tracking_graph(final_score_sz, design, env):
filename = tf.placeholder(tf.string, [], name='filename')
image_file = tf.read_file(filename)

But, what is the filename on earth?

Compiler error when pip install matplotlib 1.3.1:

npy_PyFile_DupClose was not declared in this scope

and I find no npy_PyFile_DupClose in numpy's .h file, but just npy_PyFile_DupClose2 in it.

I also find that npy_PyFile_DupClose was deprecated in numpy 1.9.0:https://github.com/numpy/numpy/blob/7cfec2403486456b52b525eccf7541e1562d9ab3/doc/release/1.9.0-notes.rst

So, I think numpy's version(1.11.1) is too high for matplotlib 1.3.1?

hello, when i see your codes,i dont know why u set up this number---evaluation.n_subseq?could u explain it?

I try to implement training code by tf, but i am stuck, have anyone implemented training code by tf.
Thanks

Dear @bertinetto, @jvlmdr & @PeterHuang2015
Thank you for your fantastic work.
I want to change the Base CNN model of your code via another one (e.g., ResNet50). Do you have any idea that how can I do that?

Hi,
What changes to be made to search over 5 scales? Am I right in thinking that changing 'scale_num' in hyperparams to 5 would be enough? But that gives (-3. , -1.5, 0 , 1.5, 3) as scales here in contrary to paper(Page-9) where (-2, -1, 0, 1, 2) scale factors are used.

Also, threshold for scales calculated here are not used ahead for thresholding? This scenario may occur when Using siamfc-5s, the scale of Search Image may shrink quickly and go below threshold in cases when tracker loses the template set initially(Eg - 'tc_Toyplane_ce' in VOT test-sequences).

when I run the python:
python run_tracker_evaluation.py
occur this error:
AttributeError: 'module' object has no attribute 'layers'
is anyone know how to solve it? thank you.

pretrained networks link (ftp://ftp.robots.ox.ac.uk/pub/outgoing/tvg/cfnet/cfnet-networks.zip) seems to be down.

I encountered this error while running, I don't know how to solve it.

runfile('E:/0目标跟踪/siamfc-tf-master/run_tracker_evaluation.py', wdir='E:/0目标跟踪/siamfc-tf-master')
C:\ProgramData\Anaconda3\lib\site-packages\h5py_init_.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Using Tensorflow 1.13.1
WARNING:tensorflow:From C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py:3632: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.

Layer 1
CONV: setting br_conv1f br_conv1b
CONV: stride 2, filter-group False
BNORM: setting br_bn1b br_bn1m br_bn1x
Traceback (most recent call last):

File "", line 1, in
runfile('E:/0目标跟踪/siamfc-tf-master/run_tracker_evaluation.py', wdir='E:/0目标跟踪/siamfc-tf-master')

File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)

File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "E:/0目标跟踪/siamfc-tf-master/run_tracker_evaluation.py", line 174, in
sys.exit(main())

File "E:/0目标跟踪/siamfc-tf-master/run_tracker_evaluation.py", line 24, in main
filename, image, templates_z, scores = siam.build_tracking_graph(final_score_sz, design, env)

File "E:\0目标跟踪\siamfc-tf-master\src\siamese.py", line 58, in build_tracking_graph
template_z, templates_x, p_names_list, p_val_list = _create_siamese(os.path.join(env.root_pretrained,design.net), x_crops, z_crops)

File "E:\0目标跟踪\siamfc-tf-master\src\siamese.py", line 102, in _create_siamese
scope='conv'+str(i+1), reuse=False)

File "E:\0目标跟踪\siamfc-tf-master\src\convolutional.py", line 10, in set_convolutional
W = tf.get_variable("W", W.shape, trainable=False, initializer=tf.constant_initializer(W))

File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1480, in get_variable
aggregation=aggregation)

File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1221, in get_variable
aggregation=aggregation)

File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 547, in get_variable
aggregation=aggregation)

File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 485, in _true_getter
return self._get_single_variable(

AttributeError: '_VariableStore' object has no attribute '_get_single_variable'

cannot work on python2

My questions may be dumb. I want to know how this tracker finds the person to track in the very first frame? Why it needs groundtruth.txt for all the frames as input? I want to use this tracker on my own video, how could I generate the groundtruth.txt? Or if I have already known the ground truth for every frame, why do I still need this tracker? Thanks! Jan

Hi there, I have noticed that when trying to run the model on my own images, I need to put both images and ground truth in the data folder, which seems just like ploting ground truth on the image. So how does the model track the target? Does the ground truth put in only for IoU computing? Thanks.

I find that the link of pretrained model can't been open.Could you update the link?

Hi,
I am trying to reproduce the siamfc net with pytorch.
I have built the whole net, but I encounter some difficulty in building loss function.
I always get a loss around 0.693 with the formula in paper and it seems that no matter what the score you get, you will get 0.693 in the end. For example:

import numpy as np

score = np.random.randint(-5,5,size=(1,5,5))
label = np.ones((1,5,5))

for i in range(1, 4):
    for j in range(1,4):
        label[0, i, j] = -1

weight = np.ones((1,5,5))
indexP = np.where(label == 1)
indexF = np.where(label == -1)

weight[indexP] = 0.5 / indexP[0].shape[0]
weight[indexF] = 0.5 / indexF[0].shape[0]

label = weight * label
print(np.sum(np.log(1+np.exp(-label*score)))/25)

you will get a value of 0.693. You can try any other score or label, initializing it with random number and you will get 0.693 again.

I wonder is there something not mentioned in paper?

Thanks for your work, I has implemented the inference demo on my machine, but I have some questions confusing me。

1. In paper，what's the meaning of "the stride of the final representation is eight"， how to control the stride, is that the sum of stride of conv and pool? however it`s not equal。
2. In paper, the network structure is [2, 2, 1, 2, 1, 1, 1]， but the pretrained model "baseline-conv5_e55.mat" structure is [2, 2, 1, 1, 1, 1, 1]，the second pooling layer stride is 1 which different from the paper， and different size of the final feature map, is it a skillful design？
3. I want to convert "baseline-conv5_e55.mat" to caffemodel, i can implement the alexnet part by https://github.com/ecoto/dagnn_caffe_deploy , but the correlation layer has not in caffe layer, so my question is the correlation layer have weights or other parameters? must I convert to this layer?

use gpu:k40 and tensorflow 1.1.0

I think the method you're using to visualize the network output is too slow. I'm sure you've seen this, but using the vot2016_ball1 video without visualization gets speed of 46.70 FPS and with visualization it gets speed of 6.39 FPS.

I have found the output channel is 32 ,but it is 128 in the paper, why are they not equal? Will this affect the accuracy?

Hi there, I just got an error below when trying to run the code on the mac. Can you give some hints? Thanks.

Using Tensorflow 1.1.0
Traceback (most recent call last):
  File "run_tracker_evaluation.py", line 174, in <module>
    sys.exit(main())
  File "run_tracker_evaluation.py", line 16, in main
    hp, evaluation, run, env, design = parse_arguments()
  File "/Users/zhaomingchuan/siamfc-tf/src/parse_arguments.py", line 10, in parse_arguments
    evaluation = json.load(json_file)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Does the output tensor values represent confidence in any way? Or is there anyway to get a confidence value per frame?

Thanks

Hi,

I'm working on a command line interface to make demoing the object tracking network easier. Would anyone like to see it merged back into master?

In tracker.py, why do you need to perform the following:

score_ = score_ - np.min(score_)
score_ = score_/np.sum(score_)

If I have multiple targets , how should track all of them simultaneously?

Thanks

I wonder about the running time of this tensorflow codes. It's a big difference compared with the original matlab one.
With GTX 1060 3GB GPU in my computer, I got about 5fps, is that correct?

In step 3 : It should be
source ~/ve-tracking/bin/activate

Thanks

I have download the pretrained model: baseline-conv5_e55.mat, and I'd like to finetune on my own dataset based on your older repo: https://github.com/bertinetto/siamese-fc, but I encountered problem when loading the model:
A dot name structure assignment is illegal when the structure is empty. Use a subscript on the structure
Error in dagnn.DagNN/rebuild (line 23)
[obj.vars.fanin] = tolist(num2cell(varFanIn)) ;

Error in dagnn.DagNN.loadobj (line 38)
obj.rebuild();

Error in load_pretrained (line 25)
net = dagnn.DagNN.loadobj(net);

Error in tracker (line 54)
net_z = load_pretrained([p.net_base_path p.net], p.gpus);

Error in run_tracker (line 11)
tracker(params);

Error in run (line 63)
evalin('caller', [script ';']);

I don't know whether it is about the edition of matlab because I am using matlab 2014b, which is not the preferred 2015b