Giter VIP home page Giter VIP logo

pysot's Introduction

PySOT

PySOT is a software system designed by SenseTime Video Intelligence Research team. It implements state-of-the-art single object tracking algorithms, including SiamRPN and SiamMask. It is written in Python and powered by the PyTorch deep learning framework. This project also contains a Python port of toolkit for evaluating trackers.

PySOT has enabled research projects, including: SiamRPNDaSiamRPNSiamRPN++, and SiamMask.

Example SiamFC, SiamRPN and SiamMask outputs.

Introduction

The goal of PySOT is to provide a high-quality, high-performance codebase for visual tracking research. It is designed to be flexible in order to support rapid implementation and evaluation of novel research. PySOT includes implementations of the following visual tracking algorithms:

using the following backbone network architectures:

Additional backbone architectures may be easily implemented. For more details about these models, please see References below.

Evaluation toolkit can support the following datasets:

📎 OTB2015 📎 VOT16/18/19 📎 VOT18-LT 📎 LaSOT 📎 UAV123

Model Zoo and Baselines

We provide a large set of baseline results and trained models available for download in the PySOT Model Zoo.

Installation

Please find installation instructions for PyTorch and PySOT in INSTALL.md.

Quick Start: Using PySOT

Add PySOT to your PYTHONPATH

export PYTHONPATH=/path/to/pysot:$PYTHONPATH

Download models

Download models in PySOT Model Zoo and put the model.pth in the correct directory in experiments

Webcam demo

python tools/demo.py \
    --config experiments/siamrpn_r50_l234_dwxcorr/config.yaml \
    --snapshot experiments/siamrpn_r50_l234_dwxcorr/model.pth
    # --video demo/bag.avi # (in case you don't have webcam)

Download testing datasets

Download datasets and put them into testing_dataset directory. Jsons of commonly used datasets can be downloaded from Google Drive or BaiduYun. If you want to test tracker on new dataset, please refer to pysot-toolkit to setting testing_dataset.

Test tracker

cd experiments/siamrpn_r50_l234_dwxcorr
python -u ../../tools/test.py 	\
	--snapshot model.pth 	\ # model path
	--dataset VOT2018 	\ # dataset name
	--config config.yaml	  # config file

The testing results will in the current directory(results/dataset/model_name/)

Eval tracker

assume still in experiments/siamrpn_r50_l234_dwxcorr_8gpu

python ../../tools/eval.py 	 \
	--tracker_path ./results \ # result path
	--dataset VOT2018        \ # dataset name
	--num 1 		 \ # number thread to eval
	--tracker_prefix 'model'   # tracker_name

Training 🔧

See TRAIN.md for detailed instruction.

Getting Help 🔨

If you meet problem, try searching our GitHub issues first. We intend the issues page to be a forum in which the community collectively troubleshoots problems. But please do not post duplicate issues. If you have similar issue that has been closed, you can reopen it.

  • ModuleNotFoundError: No module named 'pysot'

🎯Solution: Run export PYTHONPATH=path/to/pysot first before you run the code.

  • ImportError: cannot import name region

🎯Solution: Build region by python setup.py build_ext —-inplace as decribled in INSTALL.md.

References

Contributors

License

PySOT is released under the Apache 2.0 license.

pysot's People

Contributors

103yiran avatar aryanpandeyy avatar foolwood avatar lb1100 avatar siyuanren avatar strangerzhang avatar vladz avatar zhiyuanchen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pysot's Issues

SiamRPN++ only got 0.36 EAO if dropping out smoothness during testing?

I evaluated the official checkpoint siamrpn_r50_l234_dwxcorr on VOT2018, I found that if i drop out bounding box smoothness , it only got about 0.36 EAO (fast away from 0.41 using smoothness). It shows that this post-processing refinement is important to boost the performance. But, why it was not mentioned at all in the paper?

Setup buildup problem

Hi, as I followed the instruction of install.md, I encountered the c1.exe failed with exit status 2 problem when I run python setup.py build_ext --inplace. How can I fix it?
Thanks a lot.

about train

Hi, I want to train the alexnet with your training code on VID dataset, is there any backbone use the same structure as SiamRPN++? Means random shift and depth-wise cross correlation used even in alexnet.

KeyError(key) from None KeyError: 'RANK'

请问如何不通过命令行运行train.py?目前我直接在运行test.py 可以,但是直接运行train.py:
File "/home/public/anaconda3/lib/python3.6/os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'RANK'

请问如何在代码中设置:
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python -m torch.distributed.launch
--nproc_per_node=8
--master_port=2333
../../tools/train.py --cfg config.yaml
中的值,使得可以直接在pycharm运行train.py,
谢谢您的回复。

Training data sources

Would you mind, if possible, indicate in pysot/MODEL_ZOO.md the training data sources of each reported model for fair comparison purpose? Thanks a lot in advance.

class UPChannelRPN error

self.search_cls_conv = nn.Conv2d(feature_in,
feature_in * cls_output, kernel_size=3)
self.search_loc_conv = nn.Conv2d(feature_in,
feature_in * loc_output, kernel_size=3)
should be :
self.search_cls_conv = nn.Conv2d(feature_in,
feature_in, kernel_size=3)
self.search_loc_conv = nn.Conv2d(feature_in,
feature_in, kernel_size=3)

Training issues

Thank you for your work, Could I run train.py in single GPU?what should I do? When I run 'train.py' directly, I get the following error:Traceback (most recent call last):
File "../../tools/train.py", line 314, in
main()
File "../../tools/train.py", line 256, in main
rank, world_size = dist_init()
File "/home/db/Subject/pysot/pysot/utils/distributed.py", line 104, in dist_init
rank, world_size = _dist_init()
File "/home/db/Subject/pysot/pysot/utils/distributed.py", line 83, in _dist_init
rank = int(os.environ['RANK'])
File "/home/db/anaconda3/envs/pysot/lib/python3.7/os.py", line 678, in getitem
raise KeyError(key) from None
KeyError: 'RANK'

ImportError: No module named 'Cython'

conda list:

cython   0.29.7   py37he6710b0_0   defaults

我执行:sudo python setup.py build_ext --inplace
出现错误:
Traceback (most recent call last):
File "setup.py", line 3, in
from Cython.Build import cythonize
ImportError: No module named 'Cython'

我明明已经装了Cython了,为什么还报错呢?

Why the name SiamRPN?

Hi, I'm wondering why it is named "SiamRPN", where I think it's more like "SiamSSD" since it is single staged. Just curious.

评价多个算法性能

eval.py载入多个算法时
trackers = glob(os.path.join(args.tracker_path, args.dataset, args.tracker_prefix+'*'))
trackers = [x.split('/')[-1] for x in trackers]
这样语法会存在错误吧,是不是应该改成
trackers = args.tracker_prefix.split(" ")

Train on pretrained baseline models

Hi,
Is there a way to use the pretrained baseline models as base for the training, rather than just the pretrained backbones?
I want to train a model on a new dataset and don't want to retrain it with the provided four.

RuntimeError: Error(s) in loading state_dict for ModelBuilder

May I ask how to solve this error when running the demo.py.
File "/home/shiyuanyuan/anaconda3/envs/pysot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 719, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for ModelBuilder:
Missing key(s) in state_dict:

conda create --name pysot python=3.7 conda activate pysot

why I run it and false
conda create --name pysot python=3.7
conda activate pysot

CondaHTTPError: HTTP 404 NOT FOUND for url https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/noarch/repodata.json
Elapsed: 00:09.424438

The remote server could not find the noarch directory for the
requested channel with url: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge

As of conda 4.3, a valid channel must contain a noarch/repodata.json and
associated noarch/repodata.json.bz2 file, even if noarch/repodata.json is
empty. please request that the channel administrator create
noarch/repodata.json and associated noarch/repodata.json.bz2 files.
$ mkdir noarch
$ echo '{}' > noarch/repodata.json
$ bzip2 -k noarch/repodata.json

You will need to adjust your conda configuration to proceed.
Use conda config --show channels to view your configuration's current state.
Further configuration help can be found at https://conda.io/docs/config.html.

.json file for GOT-10k is missing

The following error is encountered during (file pysot/testing_dataset/GOT-10k/GOT-10k_new.json, line 60)
with open(os.path.join(dataset_root, name+'_new.json'), 'r') as f:
This is due to the missing file GOT-10k_new.json which can neither be found on BaiduYun disk nor Google Drive. Would you mind upload this missing file? Thanks in advance.

能否提供百度云链接下载

您好!
非常感谢你们开源工作,不知道能否提供百度云链接下载;国内谷歌云下载很不稳定,经常下载失败。

rank

when use one node
File "/home/public/anaconda3/lib/python3.6/os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'RANK'

请教

importError: cannot import name 'region',总是出现该错误,使用pip install region也显示安装成功了,我想咨询下这个是什么原因

TRAIN.md

TRAIN.md
Testing
python -u ../tools/test.py
--snapshot {}
--config config.py \

should be:
python -u ../../tools/test.py
--snapshot {}
--config config.yaml \

TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

Thanks for your work.

I used the following command under ''/experiments/siammask_r50_l3/'':

CUDA_VISIBLE_DEVICES=0 # change your cuda visible devices here
python -m torch.distributed.launch --nproc_per_node=1 --master_port=2333 ../../tools/train.py --cfg config.yaml

But I encountered an error when I trained siammask:

Traceback (most recent call last):
  File "../../tools/train.py", line 316, in <module>
    main()
  File "../../tools/train.py", line 311, in main
    train(train_loader, dist_model, optimizer, lr_scheduler, tb_writer)
  File "../../tools/train.py", line 211, in train
    outputs = model(data)
  File "/data/anaconda3/envs/tang0.4.1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/data/anaconda3/envs/tang0.4.1/lib/python3.7/site-packages/pysot/utils/distributed.py", line 43, in forward
    return self.module(*args, **kwargs)
  File "/data/anaconda3/envs/tang0.4.1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/data/anaconda3/envs/tang0.4.1/lib/python3.7/site-packages/pysot/models/model_builder.py", line 113, in forward
    outputs['total_loss'] += cfg.TRAIN.MASK_WEIGHT * mask_loss
TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

Could you please tell me how to run train.py correctly?

Thank you very much!

training

请问如何不通过命令行运行train.py?目前我直接在运行test.py 可以,但是直接运行train.py:
File "/home/public/anaconda3/lib/python3.6/os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'RANK'

请问如何在代码中设置:
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python -m torch.distributed.launch
--nproc_per_node=8
--master_port=2333
../../tools/train.py --cfg config.yaml
中的值,使得可以直接在pycharm运行train.py,
谢谢您的回复。

cfg.TRAIN.LOG_DIR

Traceback (most recent call last):
File "../../tools/train.py", line 378, in
main()
File "../../tools/train.py", line 327, in main
os.makedirs(cfg.TRAIN.LOG_DIR)
File "/home/public/anaconda3/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: './logs'

there is no cfg.TRAIN.LOG_DIR in the config.yaml
why os.makedirs(cfg.TRAIN.LOG_DIR)?

Training dataset

When processing training dataset, how to generate corresponding annotations after cropping the training dataset?

Shape Error in Training

Hi,
when running the training for siamrpn_alex_dwxcorr as described, I always run into an error that seems to be related to tensor shapes:

Traceback (most recent call last):
File "../../tools/train.py", line 316, in
main()
File "../../tools/train.py", line 311, in main
train(train_loader, dist_model, optimizer, lr_scheduler, tb_writer)
File "../../tools/train.py", line 211, in train
outputs = model(data)
File "/home/users/kob/mastera/pysot/venv/lib/python3.5/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/users/kob/mastera/pysot/pysot/utils/distributed.py", line 43, in forward
return self.module(*args, **kwargs)
File "/home/users/kob/mastera/pysot/venv/lib/python3.5/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/users/kob/mastera/pysot/pysot/models/model_builder.py", line 103, in forward
loc_loss = weight_l1_loss(loc, label_loc, label_loc_weight)
File "/home/users/kob/mastera/pysot/pysot/models/loss.py", line 33, in weight_l1_loss
diff = (pred_loc - label_loc).abs()
RuntimeError: The size of tensor a (17) must match the size of tensor b (25) at non-singleton dimension 4

关于画面切换的目标追踪目前支持吗

提个外行的问题😁尝试了一下,剪了一段视频最初ROI选定人脸,由于画面有切换,不是一直追踪拍摄,切换后目标就乱了,使用SiamMask算法,SiamRPN稍好一点。有时会再捕捉到原目标,有时就走远了~~

test1

The realization of weighted version of MultiRPN

class MultiRPN(RPN):
    def __init__(self, anchor_num, in_channels, weighted=False):
        super(MultiRPN, self).__init__()
        self.weighted = weighted
        for i in range(len(in_channels)):
            self.add_module('rpn'+str(i+2), 
                    DepthwiseRPN(anchor_num, in_channels[i], in_channels[i]))
        if self.weighted:
            self.cls_weight = nn.Parameter(torch.ones(len(in_channels)))
            self.loc_weight = nn.Parameter(torch.ones(len(in_channels)))

    def forward(self, z_fs, x_fs):
        cls = []
        loc = []
        for idx, (z_f, x_f) in enumerate(zip(z_fs, x_fs), start=2):
            rpn = getattr(self, 'rpn'+str(idx))
            c, l = rpn(z_f, x_f)
            cls.append(c)
            loc.append(l)

        if self.weighted:
            cls_weight = F.softmax(self.cls_weight, 0)
            loc_weight = F.softmax(self.loc_weight, 0)

        def avg(lst):
            return sum(lst) / len(lst)

        def weighted_avg(lst, weight):
            s = 0
            for i in range(len(weight)):
                s += lst[i] * weight[i]
            return s

        if self.weighted:
            return weighted_avg(cls, cls_weight), weighted_avg(loc, loc_weight)
        else:
            return avg(cls), avg(loc)

where did you change the weight? It seems

cls_weight = F.softmax(self.cls_weight, 0)
loc_weight = F.softmax(self.loc_weight, 0)

will get a mean version.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.