timmeinhardt / trackformer Goto Github PK

Implementation of "TrackFormer: Multi-Object Tracking with Transformers”. [Conference on Computer Vision and Pattern Recognition (CVPR), 2022]

Home Page: https://arxiv.org/abs/2101.02702

License: Apache License 2.0

Python 92.16% Shell 0.01% C++ 0.75% Cuda 7.08%

trackformer's People

Contributors

Stargazers

Watchers

Forkers

lsabrinax mrtornado24 tangzixia qkgia veredeps acaelles97 azerogyh evendrow uspectacle phantask kiwicodesstuff trendingtechnology bokorn bjuncek agirbau haojunyu1998 pkurainbow verigle siyisan selvamarul collector-m kinsonjul fungungun sulinyu 39239580 dominikandreas atlasgooo2 tachikakamin deeptracker jackwoo0831 aschortgen lvpengfeinj healthonrails goyallon atousaz howyoungchen meihuanshan mpfuff amitgalor18 abrahamon theodore-xs vvvityaaa cocoen12 chisyliu asmiftekhar shuowang-ai luczot larrylikestocode iamweiweishi cv-tracking finkbeiner-lab sylee-skhu stddef aimotive-legacy alessandromagliola zhiguozhao sirish-gambhira harkiratbehl peterzs wnakano tutu-xl infzhou yulincheng00 dendorferpatrick hububu9673 dlee960504 jinuhwang tostenzel jiaoaoshirenjinbu jaswanth-0821 kevp0 weijian-li lucifer1702 shuvo001 yjjian5 hustmx721 lmp-off serkansatak tadeegan syo093c henryzhou19 ricardo-yu mlee2025 keti-multi dkhanna511 satyabikash hudiework lennart-maack tuandung222 kiyoshikawasaki solra-yuan sched088 chamathabeysinghe sdehury34 dinonair chen-si-jia can-song williamflinchbaugh noyzzz pamelaosuna

trackformer's Issues

Ways to increase fps

I am getting 5.17 fps on a V100. Do you have any suggestions for increasing this? The command I am running:

%run ./src/track.py with
reid
dataset_name=DEMO
data_root_dir=/data/track_data/black_hat/images/black_hat_small_images
output_dir=/data/track_data/black_hat\reid_ip5_trk5
write_images=debug

Only detection without Deformable issue

Hello,
I tried running the training code without tracking and only using detection.

"program": "trackformer/src/train.py",
"args": [
            "with", 
            "crowdhuman", 
        ],

Unfortunately while testing all the results are zeros. I made no modifications to the code.

Does DETR(without tracking) not work on this repo?

RuntimeError: Error compiling objects for extension

Hello, when i finished all the requirements installed,when compile the module i met this error.

_`/usr/include/c++/7/bits/basic_string.tcc:1067:16: error: cannot call member function ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_set_sharable() [with _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’ without object
__p->_M_set_sharable();
error.txt

   ~~~~~~~~~^~

/usr/include/c++/7/bits/basic_string.tcc:1067:16: error: cannot call member function ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_set_sharable() [with _CharT = char32_t; _Traits = std::char_traits<char32_t>; Alloc = std::allocator<char32_t>]’ without object
ninja: build stopped: subcommand failed.`
i am not sure what i can do to address this issue so ask help for people who are solve this problem correctly,Thank you!
and my system configuration is as follows:
Ubuntu 18.04 LTS
cuda 10.1
pytorch 1.5

TypeError: _safe_repr() missing 1 required positional argument: 'sort_dicts'

Instructions To Reproduce the 🐛 Bug:

what changes you made (git diff) or what code you wrote
None
what exact command you run:
Copy pasted from MOT17 Section of README.md:

python src/track.py with \
    reid \
    tracker_cfg.public_detections=min_iou_0_5 \
    obj_detect_checkpoint_file=models/mot17_deformable_multi_frame/checkpoint_epoch_50.pth

what you observed (including full logs):

WARNING - root - Changed type of config entry "tracker_cfg.public_detections" from bool to str
WARNING - track - No observers have been added to this run
INFO - track - Running command 'main'
INFO - track - Started
ERROR - track - Failed after 0:00:00!
Exception originated from within Sacred.
Traceback (most recent calls):
  File "/home/abhik/anaconda3/envs/trackformer/lib/python3.8/site-packages/sacred/commands.py", line 61, in _non_unicode_repr
    repr_string, isreadable, isrecursive = pprint._safe_repr(
TypeError: _safe_repr() missing 1 required positional argument: 'sort_dicts'
srun: error: node001: task 0: Exited with exit code 1

please simplify the steps as much as possible so they do not require additional resources to

Expected behavior:

Environment:

Solution

This bug
IDSIA/sacred#780

Simply update requirements.txt to sacred==0.8.2 or higher instead of sacred ==0.8.2

Training Custom Dataset

Hi,

Thanks for the great paper !

I try to train on a custom dataset (of one sample, attached to this issue), the data is not temporal.
I run the following command:

python src/train.py
    with \
    deformable \
    mot17 \
    resume=models/crowdhuman_train_val_deformable/checkpoint.pth \
    output_dir=models/boxy_train_deformable \
    mot_path=my_custom_dataset \
    train_split=train\ 
    val_split=val \
    epochs=50 \

As I train on static data (not temporal) I don't use the 'tracking' option.
Upon running the code I face an assert inside the generalized_box_iou function:

trackformer/src/trackformer/util/box_ops.py

Line 51 in 686fe60

assert (boxes1[:, 2:] >= boxes1[:, :2]).all()

In fact, the boxes1 and boxes2 are full of nans.

Following the mentioned COCO style in the TRAIN.md, I followd the suggested structure and here is my sample of my train.json file:

{
    "type": "instances",
    "images": [
        {
            "file_name": "601038faba553c0015ac7c00_00000.png",
            "id": 0
        }
    ],
    "annotations": [
        {
            "id": 0,
            "bbox": [
                464.0,
                0.0002879999999976235,
                241.000448,
                228.99974399999996
            ],
            "image_id": 0,
            "segmentation": [],
            "ignore": 0,
            "visibility": 1.0,
            "area": 55189.04089588531,
            "iscrowd": 0,
            "category_id": 0
        }
    ],
    "categories": [
        {
            "supercategory": "human",
            "name": "person",
            "id": 0
        }
    ]
}

After investigation, I notice that the COCO annotation format uses box coordinates: [xtl, ytl, w, h],
where:

xtl: the top left x
ytl: the top left y
w: width
h: height

I'm I doing anything wrong ?

Best,
Axel

Error when using tracking = true

In both cases, when I enable the tracking by either tracking=true in track.yaml or using python src/train.py with deformable tracking, the program gives me an error saying

Traceback (most recent call last):
File "src/train.py", line 341, in
train(args)
File "src/train.py", line 78, in train
model, criterion, postprocessors = build_model(args)
File "/media/overlord/Project/trackformer/src/trackformer/models/init.py", line 57, in build_model
model = DeformableDETRTracking(tracking_kwargs, detr_kwargs)
File "/media/overlord/Project/trackformer/src/trackformer/models/detr_tracking.py", line 145, in init
DETRTrackingBase.init(self, **tracking_kwargs)
File "/media/overlord/Project/trackformer/src/trackformer/models/detr_tracking.py", line 19, in init
super().init()

TypeError: init() missing 5 required positional arguments: 'backbone', 'transformer', 'num_classes', 'num_queries', and 'num_feature_levels'

Can you kindly tell me where I am going wrong? And what is the possible solution?
Thanks

cell tracking dataset as the custom dataset

This project is very helpful for me I guess!!! When I clone it to my own directory and try to train my own model, I find something wrong...
I have a cell tracking dataset and convert it into coco-format, then how could I use the trackformer to train my tracking model?
When I use :
python src/train.py with
deformable
tracking
mot17
full_res
resume=models/mot17_train_deformable_private/checkpoint.pth
output_dir=models/custom_dataset_train_deformable
mot_path=data/custom_dataset
train_split=train
val_split=val
epochs=20 \

I got the FileNotFoundError: [Errno 2] No such file or directory: 'models/mot17_train_deformable_private/checkpoint.pth' error...

Pre-Trained models have no attribute for generating attention maps

Hello,

I try to get attention maps from pre-trained models updated 1 hours ago.
Using a virtual environment with all the requirements from INSTALL.md in the same version, I tryed executing the following :

python src/track.py with \ 
dataset_name=DEMO \
data_root_dir=data/snakeboard \
output_dir=snake_results \
write_images=pretty \
generate_attention_maps=True \
verbose=True \
obj_detect_checkpoint_file=models/mot17_train_deformable_public/checkpoint.pth

I got :

WARNING - root - Changed type of config entry "write_images" from bool to str
WARNING - track - No observers have been added to this run
INFO - track - Running command 'main'
INFO - track - Started
Configuration (modified, added, typechanged, doc):
  data_root_dir = 'data/snakeboard'
  dataset_name = 'DEMO'
  generate_attention_maps = True
  interpolate = False
  load_results_dir = None
  obj_detect_checkpoint_file = 'models/mot17_train_deformable_public/checkpoint.pth'
  output_dir = 'snake_results'
  seed = 666
  verbose = True
  write_images = 'pretty'
  frame_range:
    end = 1.0
    start = 0.0
  tracker_cfg:
    detection_nms_thresh = 0.9
    detection_obj_score_thresh = 0.9
    inactive_patience = -1
    public_detections = False
    reid_greedy_matching = False
    reid_score_thresh = 0.8
    reid_sim_only = False
    reid_sim_threshold = 0.0
    track_nms_thresh = 0.9
    track_obj_score_thresh = 0.8
INFO - main - INIT object detector [EPOCH: 40]
ERROR - track - Failed after 0:00:12!
Traceback (most recent calls WITHOUT Sacred internals):
  File "src/track.py", line 103, in main
    generate_attention_maps, track_logger)
  File "/trackformer/src/trackformer/models/tracker.py", line 51, in __init__
    multihead_attn = self.obj_detector.transformer.decoder.layers[-1].multihead_attn
  File "/miniconda3/envs/venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 594, in __getattr__
    type(self).__name__, name))
AttributeError: 'DeformableTransformerDecoderLayer' object has no attribute 'multihead_attn'

With the same but with obj_detect_checkpoint_file=models/mots20_train_masks/checkpoint.pth instead I got :

[...]
Traceback (most recent calls WITHOUT Sacred internals):
  File "src/track.py", line 69, in main
    obj_detector, _, obj_detector_post = build_model(obj_detect_args)
  File "/data/mlarsen/trackformer/src/trackformer/models/__init__.py", line 28, in build_model
    matcher = build_matcher(args)
  File "/data/mlarsen/trackformer/src/trackformer/models/matcher.py", line 132, in build_matcher
    focal_loss=args.focal_loss,
AttributeError: 'Namespace' object has no attribute 'focal_loss'

By adding focal_loss=false and focal_alpha=0.25 to models/mots20_train_masks/config.yaml, I now get :

[...]
Traceback (most recent calls WITHOUT Sacred internals):
  File "src/track.py", line 132, in main
    tracker.step(frame_data)
  File "/trackformer/src/trackformer/models/tracker.py", line 288, in step
    outputs, *_ = self.obj_detector(img, target)
  File "/miniconda3/envs/venv-fb/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/trackformer/src/trackformer/models/detr_segmentation.py", line 42, in forward
    out, targets, features, memory, hs = super().forward(samples, targets)
  File "/trackformer/src/trackformer/models/detr_tracking.py", line 129, in forward
    out, targets, features, memory, hs  = super().forward(samples, targets)
  File "/trackformer/src/trackformer/models/detr.py", line 113, in forward
    src, mask, query_embed, pos, tgt)
  File "/miniconda3/envs/venv-fb/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/trackformer/src/trackformer/models/transformer.py", line 76, in forward
    prev_frame=prev_frame)
  File "/miniconda3/envs/venv-fb/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/trackformer/src/trackformer/models/transformer.py", line 154, in forward
    pos=pos, query_pos=query_pos)
  File "/miniconda3/envs/venv-fb/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/trackformer/src/trackformer/models/transformer.py", line 308, in forward
    tgt_key_padding_mask, memory_key_padding_mask, pos, query_pos)
  File "/trackformer/src/trackformer/models/transformer.py", line 266, in forward_post
    key_padding_mask=memory_key_padding_mask)[0]
  File "/miniconda3/envs/venv-fb/lib/python3.7/site-packages/torch/nn/modules/module.py", line 552, in __call__
    hook_result = hook(self, input, result)
  File "/trackformer/src/trackformer/models/tracker.py", line 47, in add_attention_map_to_data
    attention_maps = output[1].view(-1, height, width)
RuntimeError: shape '[-1, 188, 334]' is invalid for input of size 100800

Thanks you for this excellent paper, for releasing the code and for your time.

Smaller Backbone Models

Hi Tim Meinhardt thanks for the great work!

I'm trying to adapt trackformer in my own research, however I don't have enough GPU memory. Do you have private mode models with smaller backbones(resnet18/34) pretrained on coco/crowdedhuman detection task or scripts for coco detection pretraining? Thanks!

Track_ids matching problem between different frames

Hello，I found something while training that I don't quite understand

I printed the data preprocessing results about track_ids, the id of the previous frame and the id of the next frame will be misaligned,

` def _getitem_from_id(self, image_id):
img, target = super(CocoDetection, self).getitem(image_id)
image_id = self.ids[image_id]
target = {'image_id': image_id,
'annotations': target}

    img, target = self.prepare(img, target)
    print("track_ids: ", target['track_ids'])
    target['track_ids'] = torch.arange(len(target['labels']))
    print("track_ids: ", target['track_ids'])

    if self._transforms is not None:
        img, target = self._transforms(img, target)
    print("track_ids: ", target['track_ids'])`

in an image of 55 frames
track_ids: tensor([ 3, 9, 10, 14, 15, 17, 18, 19, 20, 21, 23, 26, 27, 31, 36, 37, 39, 40, 57, 59, 61, 68, 69, 70, 72])
track_ids: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24])
track_ids: tensor([ 0, 7, 8, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 24])
in an image of 57 frames
track_ids: tensor([ 2, 3, 9, 10, 14, 15, 17, 18, 19, 20, 21, 23, 26, 27, 31, 36, 39, 40, 57, 59, 61, 68, 69, 70, 72])
track_ids: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24])
track_ids: tensor([ 1, 8, 9, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 24])

Obviously, in 2, 3, 9, 10, 14, 15, 17, 18, 19, 20, 21, 23, 26, 27, 31, 36, two frames are misaligned， 40, 57, 59, 61, 68, 69, 70, 72 are normal

Also I print the result that matches in detr_tracking after

` print("prev_track_ids: ", target['prev_track_ids'])
# print("prev_target_ind", prev_target_ind)
print("track_ids: ", target['track_ids'])
# print(target)

            # match track ids between frames
            # print(prev_track_ids.unsqueeze(dim=1))
            target_ind_match_matrix = prev_track_ids.unsqueeze(dim=1).eq(target['track_ids'])
            print(target_ind_match_matrix)
            target_ind_matching = target_ind_match_matrix.any(dim=1)
            print(target_ind_matching)
            target_ind_matched_idx = target_ind_match_matrix.nonzero()[:, 1]
            print('track_query_match_ids: ', target_ind_matched_idx)`

prev_track_ids: tensor([ 0, 7, 8, 13, 14, 15, 21, 22, 24], device='cuda:0')
track_ids: tensor([ 1, 8, 9, 14, 15, 21, 22, 24], device='cuda:0')
tensor([[False, True, False, False, False, False, False, False],
[False, False, False, True, False, False, False, False],
[False, False, False, False, True, False, False, False],
[False, False, False, False, False, False, False, True],
[False, False, False, False, False, False, False, False],
[False, False, False, False, False, False, False, False],
[False, False, False, False, False, True, False, False],
[False, False, False, False, False, False, True, False]],
device='cuda:0')
tensor([ True, True, True, True, False, False, True, True],
device='cuda:0')
track_query_match_ids: tensor([1, 3, 4, 7, 5, 6], device='cuda:0')

As a result, there seems to be a problem。
So, will there be effects in these places?
Thanks。

How to train with distributed mode

I want to train trackformer on one machine with multi GPUs using DDP? So how can I enable distributed mode in the command line, when I add python -m torch.distributed.launch --nproc_per_node=4 before src/train.py then it asserts subprocess.CalledProcessError, can anyone help me?. Thanks firstly.

how to add reid

hi
I want to add some reid network.But I do not know how to do it .could you help me? or give me some advice.
Thank you!

Great paper!

Grat paper Tim, waiting already for code... Best regards!

pre-trained deformable DETR

It is said in the paper that "We follow [58] and pre-train the model for 50 epochs without track queries on COCO [26]."
Do you use pre-trained models from https://github.com/fundamentalvision/Deformable-DETR, or do you pre-train it yourself?
In case you use models from https://github.com/fundamentalvision/Deformable-DETR, which one exactly do you use?
If you pre-train it yourself on COCO, is the COCO pre-training script also included in the repository?

Thanks

Masks in deformable transformer and transformer

Hi,

I have a question regarding deformable transformer and transformer. I see the variable Mask is passed to the encoder and I am not sure what is the usage? Why are you using masks there?
I am testing transformer without any masks and I want to ask if it is possible to use deformable transformer without any masks as well? Looking at the code seems to me it is not possible. Any thought?

I would appreciate your feedback.
Thanks,

settings of multi-frame and number of classes (20)

Hi @timmeinhardt , thanks for your great work!

After checking the code, I found that (1) the number of clases is set to 20 even only the person is tracked; (2) multi-frame attention is performed, but no discussion is provided in the paper or this repo.. Here come the questions: why you set the number of classes to 20? Does the multi-frame attention contributs to the perfomance gain a lot?

Thanks.

Not able to install MultiscaleDeformableAttention

Hi,

I am trying to run a test for training trackformer. I am looking at INSTALL.md to build the environment. When I try python src/trackformer/models/ops/setup.py build --build-base=src/trackformer/models/ops/ install I get an error saying:

Traceback (most recent call last):
  File "src/trackformer/models/ops/setup.py", line 62, in <module>
    setup(
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 148, in setup
    return run_commands(dist)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
    dist.run_commands()
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
    self.run_command(cmd)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/dist.py", line 1214, in run_command
    super().run_command(command)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/dist.py", line 1214, in run_command
    super().run_command(command)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
    build_ext.build_extensions(self)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
    _build_ext.build_extension(self, ext)
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 528, in build_extension
    objects = self.compiler.compile(sources,
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 414, in unix_wrap_ninja_compile
    _write_ninja_file_and_compile_objects(
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1135, in _write_ninja_file_and_compile_objects
    _run_ninja_build(
  File "/z/home/mahzad-khosh/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error compiling objects for extension

How can I get this package installed?

Thanks,

Class error is very high when using COCO pretrained model with Custom Datasets(VisDrone2019)

Hello, Firstly thanks for your brilliant work.
I want to train the TrackFormer in VisDrone2019, which contains objects of vehicles instead of Pedestrians.
Although the object is different, considering that Deformable DETR's pre-trained model (i.e. r50_deformable_detr-checkpoint.pth) should have been pre-trained on the COCO dataset, which contains vehicles. So I think it works.

In converting the VisDrone dataset into COCO format, I only kept the vehicle categories. However, after training two epochs based on r50_deformable_detr-checkpoint.pth model, it was found that class error was basically 100, and various losses did not improve significantly in the later training period. But other evaluation index(such as loss_bbox and loss_ce etc.) seems reasonable,.

During training, it acts like this:

And after two epoch, the average status is:

What is the possible reason for this? or Could you give me some advice?

Thank you very much.

Cuda is not available

hallo timmeinhardt

When I install MultiScaleDeformableAttention package: python src/trackformer/models/ops/setup.py build --build-base=src/trackformer/models/ops/ install :
I met an error.

(tfr) C:\Users\ztmar\mots\trackformer>python src/trackformer/models/ops/setup.py build --build-base=src/trackformer/models/ops/ install
Traceback (most recent call last):
  File "src/trackformer/models/ops/setup.py", line 64, in <module>
    ext_modules=get_extensions(),
  File "src/trackformer/models/ops/setup.py", line 41, in get_extensions
    raise NotImplementedError('Cuda is not available')
NotImplementedError: Cuda is not available

Environment:

Provide your environment information using the following command:

PyTorch version: 1.7.1+cu101
Is debug build: False
CUDA used to build PyTorch: 10.1
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Home
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.22.5

Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] torch==1.7.1+cu101
[pip3] torchaudio==0.7.2
[pip3] torchfile==0.1.0
[pip3] torchvision==0.8.2+cu101
[conda] Could not collect

Some confusion about the paper

Hi, thanks for you great job!
I have a question about your paper, In the MOT17 experiment section of the paper, The dataset you used for the test is the MOT17 test dataset or a part of training dataset as the test dataset?

Is there a pretrained weight file based on vanilla DETR?

The pretrained weight files of Trackformer currently are all based on Deformable DETR. However, my current environment does not support Deformable DETR due to some physical limitations.

Could you please provide a pretrained weight file based on vanilla DETR (e.g., DETR-ResNet-50)? Thanks!

When I was implementing python3 src/track.py with dataset_name=DEMO data_root_dir=data/snakeboard output_dir=data/snakeboard write_images=pretty,I encountered the following problems

0%| | 0/131 [00:02<?, ?it/s]
ERROR - track - Failed after 0:00:06!
Traceback (most recent calls WITHOUT Sacred internals):
File "src/track.py", line 132, in main
tracker.step(frame_data)
File "/home/lihenian/trackformer-main/src/trackformer/models/tracker.py", line 306, in step
outputs, _, features, _, _ = self.obj_detector(img, target, self._prev_features[0])
File "/home/lihenian/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/lihenian/trackformer-main/src/trackformer/models/detr_tracking.py", line 275, in forward
out, targets, features, memory, hs = super().forward(samples, targets, prev_features)
File "/home/lihenian/trackformer-main/src/trackformer/models/deformable_detr.py", line 227, in forward
self.transformer(src_list, mask_list, pos_list, query_embeds, targets)
File "/home/lihenian/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/lihenian/trackformer-main/src/trackformer/models/deformable_transformer.py", line 166, in forward
mask_flatten[:, :src_flatten.shape[1] // 2])
File "/home/lihenian/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/lihenian/trackformer-main/src/trackformer/models/deformable_transformer.py", line 325, in forward
output = layer(output, pos, reference_points, spatial_shapes, padding_mask)
File "/home/lihenian/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/lihenian/trackformer-main/src/trackformer/models/deformable_transformer.py", line 290, in forward
src2 = self.self_attn(self.with_pos_embed(src, pos), reference_points, src, spatial_shapes, padding_mask)
File "/home/lihenian/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/lihenian/trackformer-main/src/trackformer/models/ops/modules/ms_deform_attn.py", line 87, in forward
value, input_spatial_shapes, sampling_locations, attention_weights, self.im2col_step)
File "/home/lihenian/trackformer-main/src/trackformer/models/ops/functions/ms_deform_attn_func.py", line 19, in forward
value, value_spatial_shapes, sampling_locations, attention_weights, ctx.im2col_step)
RuntimeError: Unrecognized tensor type ID: PythonTLSSnapshot

Why the outputs from decoder layers are concatenated while this is not the case for the encoder

Hi,

I have a question regarding the code for the transformer encoder & decoder. I am looking into the case args.deformable=False and args.Tracking=False. Looking into transformer.py I am a bit confused about the code. I check the self.num_layers and they are 6 for both encoder and decoder. However, looking at the code, In the decoder forward, output from each layer is stored, and then they are stacked for the final output.

for i, layer in enumerate(self.layers):
            if self.track_attention:
                track_output = output[:-100].clone()

                track_output = self.layers_track_attention[i](
                    track_output,
                    src_mask=tgt_mask,
                    src_key_padding_mask=tgt_key_padding_mask,
                    pos=track_query_pos)

                output = torch.cat([track_output, output[-100:]])

            output = layer(output, memory, tgt_mask=tgt_mask,
                           memory_mask=memory_mask,
                           tgt_key_padding_mask=tgt_key_padding_mask,
                           memory_key_padding_mask=memory_key_padding_mask,
                           pos=pos, query_pos=query_pos)
            
            if self.return_intermediate:
                intermediate.append(output)

if self.return_intermediate:
       output = torch.stack(intermediate)

if self.norm is not None:
    return self.norm(output), output

for the encoder however the loop is done but the output is the result of the last layer only.

       for layer in self.layers:
            output = layer(output, src_mask=mask,
                           src_key_padding_mask=src_key_padding_mask, pos=pos)
            print(output.shape,'enc-layer')

        if self.norm is not None:
            output = self.norm(output)

Is this a bug or am I misunderstood?

Multi-Class MOT

Hello,

Does TrackFormer support multi-class multi-object tracking? If so how do I go about training this?

Thanks :)

The setting of `prev_query_embed` in `DeformableTransformer`

Hi, thanks for your great works!

I found that prev_query_embed of track query in deformable_transformer.py

trackformer/src/trackformer/models/deformable_transformer.py

Line 214 in df70fef

prev_query_embed = torch.zeros_like(prev_hs_embed)

is set to zeros. However, the query_embed of detection query is learned end-to-end, which is in fact the postional embeddings. Why you do such settings? From the commented lines (line 215-220), it seems that you have tried different settings of prev_tgt and prev_query_embed. Does the performance differ a lot with these different settings?

Error while using DETR instead of deformable DETR

Hi.
I was trying to run trackformer using DETR instead of deformable DETR, I strictly need the first for my application.
I was indeed try to use the pretrained weights for mots20 dataset, as suggested in this issue:
The problem arise at the beginning when trying to run the following commands

python src/track.py with \ dataset_name=MOTS20-ALL \ obj_detect_checkpoint_file=models/mots20_train_masks/checkpoint.pth

with this additional fields
write_images="pretty", seed=666, interpolate=False,\ verbose=True, load_results_dir=None, generate_attention_maps=False,\ tracker_cfg=tracker_cfg, \ frame_range={"start":0.0, "end":1.0}, _config="cfgs/track.yaml", _log=None, _run=None, obj_detector_model=None

This is the error that I got...

raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for DETRSegmTracking: size mismatch for class_embed.weight: copying a param with shape torch.Size([2, 256]) from checkpoint, the shape in current model is torch.Size([21, 256]). size mismatch for class_embed.bias: copying a param with shape torch.Size([2]) from checkpoint, the shape in current model is torch.Size([21]).

My questions on the topic are the following.

why mots20_train_masks_checkpoint uses a class_embed dimension of 2 ? Is this the expected behavior, or they should be 21 as for the class_embedding of the other models ?
My specific application requires to use a pretrained DETR on COCO dataset for panoptic-segmentation on your trackformer. Is there a way to achieve such a thing? Use DETR instead of Deformable DETR

EDIT
I suspect the error is something related to number of classes. Indeed changing in build_model of models/init.py the number of class to 1, the error goes away, but something else arise :

File "src/new_track.py", line 211, in main tracker.step(frame_data) File "/trackformer/src/trackformer/models/tracker.py", line 308, in step outputs, _, features, _, _ = self.obj_detector(img, target, self._prev_features[0]) TypeError: forward() takes from 2 to 3 positional arguments but 4 were given

My additional question is ...

Since DETR can take typically as input an image without target and self._prev_features[0], how we should modify the models to use again DETR ?

ms_deform_attn_cpu.o: No such file or directory

g++: error: /home/lh/trackformer-main/src/trackformer/models/ops/temp.linux-x86_64-3.7/home/lh/trackformer-main/src/trackformer/models/ops/src/cpu/ms_deform_attn_cpu.o: No such file or directory
g++: error: /home/lh/trackformer-main/src/trackformer/models/ops/temp.linux-x86_64-3.7/home/lh/trackformer-main/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.o: No such file or directory
g++: error: /home/lh/trackformer-main/src/trackformer/models/ops/temp.linux-x86_64-3.7/home/lh/trackformer-main/src/trackformer/models/ops/src/vision.o: No such file or directory
error: command '/opt/rh/devtoolset-7/root/usr/bin/g++' failed with exit code 1

Using the DEMO code

This paper is very good! When I use the code of demo interface,I always get mistakes like this:

$ python src/track.py with \

dataset_name=DEMO \
data_root_dir=data/snakeboard \
output_dir=data/snakeboard \
write_images=pretty

WARNING - root - Changed type of config entry "write_images" from bool to str
WARNING - track - No observers have been added to this run

In the end, I'll get something like this again:
if (isinstance(colors[0], Sized) and len(colors[0]) == 2
IndexError: list index out of range

I have created a DEMO folder and put the video I'm going to demonstrate into it, but it still reports this error. I guess it's because the video I put in hasn't been converted to coco format?

But when I demonstrate, I should be able to enter any video and get visual results rather than having to convert it to Coco format.

I'm very confused.How can I solve the problem?

Some installation issue regarding c++

Can't execute Installation.md step 2.iv: Install MultiScaleDeformableAttention package: python src/trackformer/models/ops/setup.py build --build-base=src/trackformer/models/ops/ install

Changed this command from python to python3.

Trying to install into conda environment.
Tryed all cuda combinations for pytorch 1.5
gcc --version -> gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
python3 --version -> Python 3.8.10

Getting: RuntimeError: Error compiling objects for extension

Grep'ing for error results in:

python3 src/trackformer/models/ops/setup.py build --build-base=src/trackformer/models/ops/ install | grep error
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
Traceback (most recent call last):
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1395, in _run_ninja_build
    subprocess.run(
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/trackformer/models/ops/setup.py", line 62, in <module>
    setup(
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
    build_ext.build_extensions(self)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
    _build_ext.build_extension(self, ext)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
    objects = self.compiler.compile(sources,
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 414, in unix_wrap_ninja_compile
    _write_ninja_file_and_compile_objects(
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1135, in _write_ninja_file_and_compile_objects
    _run_ninja_build(
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error compiling objects for extension

Complete Error Report:

python3 src/trackformer/models/ops/setup.py build --build-base=src/trackformer/models/ops/ install
running build
running build_ext
building 'MultiScaleDeformableAttention' extension
Emitting ninja build file /home/matthias/trackformer/src/trackformer/models/ops/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/3] c++ -MMD -MF /home/matthias/trackformer/src/trackformer/models/ops/temp.linux-x86_64-3.8/home/matthias/trackformer/src/trackformer/models/ops/src/cpu/ms_deform_attn_cpu.o.d -pthread -B /home/matthias/miniconda3/envs/trackformer/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/matthias/trackformer/src/trackformer/models/ops/src -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/TH -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/THC -I/home/matthias/miniconda3/envs/trackformer/include/python3.8 -c -c /home/matthias/trackformer/src/trackformer/models/ops/src/cpu/ms_deform_attn_cpu.cpp -o /home/matthias/trackformer/src/trackformer/models/ops/temp.linux-x86_64-3.8/home/matthias/trackformer/src/trackformer/models/ops/src/cpu/ms_deform_attn_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
[2/3] c++ -MMD -MF /home/matthias/trackformer/src/trackformer/models/ops/temp.linux-x86_64-3.8/home/matthias/trackformer/src/trackformer/models/ops/src/vision.o.d -pthread -B /home/matthias/miniconda3/envs/trackformer/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/matthias/trackformer/src/trackformer/models/ops/src -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/TH -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/THC -I/home/matthias/miniconda3/envs/trackformer/include/python3.8 -c -c /home/matthias/trackformer/src/trackformer/models/ops/src/vision.cpp -o /home/matthias/trackformer/src/trackformer/models/ops/temp.linux-x86_64-3.8/home/matthias/trackformer/src/trackformer/models/ops/src/vision.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /home/matthias/trackformer/src/trackformer/models/ops/src/vision.cpp:11:0:
/home/matthias/trackformer/src/trackformer/models/ops/src/ms_deform_attn.h: In function ‘at::Tensor ms_deform_attn_forward(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, int)’:
/home/matthias/trackformer/src/trackformer/models/ops/src/ms_deform_attn.h:29:20: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     if (value.type().is_cuda())
                    ^
In file included from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Tensor.h:11:0,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Context.h:4,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/ATen.h:5,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/extension.h:4,
                 from /home/matthias/trackformer/src/trackformer/models/ops/src/cpu/ms_deform_attn_cpu.h:12,
                 from /home/matthias/trackformer/src/trackformer/models/ops/src/ms_deform_attn.h:13,
                 from /home/matthias/trackformer/src/trackformer/models/ops/src/vision.cpp:11:
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:30: note: declared here
   DeprecatedTypeProperties & type() const {
                              ^~~~
In file included from /home/matthias/trackformer/src/trackformer/models/ops/src/vision.cpp:11:0:
/home/matthias/trackformer/src/trackformer/models/ops/src/ms_deform_attn.h: In function ‘std::vector<at::Tensor> ms_deform_attn_backward(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, int)’:
/home/matthias/trackformer/src/trackformer/models/ops/src/ms_deform_attn.h:51:20: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     if (value.type().is_cuda())
                    ^
In file included from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Tensor.h:11:0,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Context.h:4,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/ATen.h:5,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                 from /home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/extension.h:4,
                 from /home/matthias/trackformer/src/trackformer/models/ops/src/cpu/ms_deform_attn_cpu.h:12,
                 from /home/matthias/trackformer/src/trackformer/models/ops/src/ms_deform_attn.h:13,
                 from /home/matthias/trackformer/src/trackformer/models/ops/src/vision.cpp:11:
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:30: note: declared here
   DeprecatedTypeProperties & type() const {
                              ^~~~
[3/3] /usr/bin/nvcc -DWITH_CUDA -I/home/matthias/trackformer/src/trackformer/models/ops/src -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/TH -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/THC -I/home/matthias/miniconda3/envs/trackformer/include/python3.8 -c -c /home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu -o /home/matthias/trackformer/src/trackformer/models/ops/temp.linux-x86_64-3.8/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14
FAILED: /home/matthias/trackformer/src/trackformer/models/ops/temp.linux-x86_64-3.8/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.o 
/usr/bin/nvcc -DWITH_CUDA -I/home/matthias/trackformer/src/trackformer/models/ops/src -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/TH -I/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/THC -I/home/matthias/miniconda3/envs/trackformer/include/python3.8 -c -c /home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu -o /home/matthias/trackformer/src/trackformer/models/ops/temp.linux-x86_64-3.8/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1945): warning: extra ";" ignored

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(261): warning: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_im2col_cuda(cudaStream_t, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *) [with scalar_t=double]" 
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu(64): here

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(762): warning: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu(134): here

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(872): warning: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu(134): here

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(331): warning: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu(134): here

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(436): warning: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu(134): here

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(544): warning: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu(134): here

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(649): warning: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu(134): here

/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h(1945): warning: extra ";" ignored

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(261): warning: variable "q_col" was declared but never referenced

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(331): warning: variable "q_col" was declared but never referenced

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(436): warning: variable "q_col" was declared but never referenced

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(544): warning: variable "q_col" was declared but never referenced

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(649): warning: variable "q_col" was declared but never referenced

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(762): warning: variable "q_col" was declared but never referenced

/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_im2col_cuda.cuh(872): warning: variable "q_col" was declared but never referenced

/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorMethods.h:5293:139:   required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorMethods.h:5293:139:   required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor>&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorMethods.h:5293:139:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor>&&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorMethods.h:5293:139:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:1418:446:   required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:1418:446:   required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:1418:446:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>&&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:1418:446:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> >}]’:
/usr/include/c++/6/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:1437:111:   required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> >}]’:
/usr/include/c++/6/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:1437:111:   required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> >}]’:
/usr/include/c++/6/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> >}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:1437:111:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >&&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> >}]’:
/usr/include/c++/6/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> >}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > >::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:1437:111:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int}]’:
/usr/include/c++/6/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:2315:280:   required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int}]’:
/usr/include/c++/6/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:2315:280:   required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int}]’:
/usr/include/c++/6/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:2315:280:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>&&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int}]’:
/usr/include/c++/6/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, long int>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:2315:280:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:3228:268:   required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:3228:268:   required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor>&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:3228:268:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor>&&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:3228:268:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, double, long int>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, double, long int}]’:
/usr/include/c++/6/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, double, long int>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, double, long int>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, double, long int>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:5162:99:   required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, double, long int>}; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, double, long int}]’:
/usr/include/c++/6/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, double, long int>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, double, long int>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, double, long int>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:5162:99:   required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, double, long int>&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, double, long int}]’:
/usr/include/c++/6/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, double, long int>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, double, long int}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, double, long int>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:5162:99:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, double, long int>&&; bool <anonymous> = true; _Elements = {at::Tensor, at::Tensor, double, long int}]’:
/usr/include/c++/6/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, double, long int>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, double, long int}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, double, long int>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:5162:99:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>}; bool <anonymous> = true; _Elements = {at::Tensor&, at::Tensor&, at::Tensor&}]’:
/usr/include/c++/6/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor&, at::Tensor&, at::Tensor&>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor&, at::Tensor&, at::Tensor&>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:6418:303:   required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>}; bool <anonymous> = true; _Elements = {at::Tensor&, at::Tensor&, at::Tensor&}]’:
/usr/include/c++/6/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor&, at::Tensor&, at::Tensor&>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor&, at::Tensor&, at::Tensor&>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:6418:303:   required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>&; bool <anonymous> = true; _Elements = {at::Tensor&, at::Tensor&, at::Tensor&}]’:
/usr/include/c++/6/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor&, at::Tensor&, at::Tensor&>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor&, at::Tensor&, at::Tensor&}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor&, at::Tensor&, at::Tensor&>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:6418:303:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>&&; bool <anonymous> = true; _Elements = {at::Tensor&, at::Tensor&, at::Tensor&}]’:
/usr/include/c++/6/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor&, at::Tensor&, at::Tensor&>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor&, at::Tensor&, at::Tensor&}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor&, at::Tensor&, at::Tensor&>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Functions.h:6418:303:   required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^    
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In function ‘at::Tensor ms_deform_attn_cuda_forward(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, int)’:
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:34:62: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(value.type().is_cuda(), "value must be a CUDA tensor");
                                                              ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:35:71: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(spatial_shapes.type().is_cuda(), "spatial_shapes must be a CUDA tensor");
                                                                       ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:36:74: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(level_start_index.type().is_cuda(), "level_start_index must be a CUDA tensor");
                                                                          ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:37:69: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(sampling_loc.type().is_cuda(), "sampling_loc must be a CUDA tensor");
                                                                     ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:38:68: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(attn_weight.type().is_cuda(), "attn_weight must be a CUDA tensor");
                                                                    ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:64:43: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         AT_DISPATCH_FLOATING_TYPES(value.type(), "ms_deform_attn_forward_cuda", ([&] {
                                           ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:64:98: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
         AT_DISPATCH_FLOATING_TYPES(value.type(), "ms_deform_attn_forward_cuda", ([&] {
                                                                                                  ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Dispatch.h:46:1: note: declared here
 inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties &t) {
 ^~~~~~~~~~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:66:13: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 value.data<scalar_t>() + n * im2col_step_ * per_value_size,
             ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:66:99: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 value.data<scalar_t>() + n * im2col_step_ * per_value_size,
                                                                                                   ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:67:14: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 spatial_shapes.data<int64_t>(),
              ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:67:47: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 spatial_shapes.data<int64_t>(),
                                               ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:68:2: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 level_start_index.data<int64_t>(),
  ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:69:32: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
                                ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:70:67: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size,
                                                                   ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:71:25: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
                         ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:71:68: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
                                                                    ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:71:100: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
                                                                                                    ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:72:54: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                 columns.data<scalar_t>());
                                                      ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:73:83: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
 
                                                                                   ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In function ‘std::vector<at::Tensor> ms_deform_attn_cuda_backward(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, int)’:
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:100:62: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(value.type().is_cuda(), "value must be a CUDA tensor");
                                                              ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:101:71: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(spatial_shapes.type().is_cuda(), "spatial_shapes must be a CUDA tensor");
                                                                       ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:102:74: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(level_start_index.type().is_cuda(), "level_start_index must be a CUDA tensor");
                                                                          ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:103:69: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(sampling_loc.type().is_cuda(), "sampling_loc must be a CUDA tensor");
                                                                     ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:104:68: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(attn_weight.type().is_cuda(), "attn_weight must be a CUDA tensor");
                                                                    ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:105:68: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
     AT_ASSERTM(grad_output.type().is_cuda(), "grad_output must be a CUDA tensor");
                                                                    ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:134:43: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         AT_DISPATCH_FLOATING_TYPES(value.type(), "ms_deform_attn_backward_cuda", ([&] {
                                           ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
   DeprecatedTypeProperties & type() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:134:98: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
         AT_DISPATCH_FLOATING_TYPES(value.type(), "ms_deform_attn_backward_cuda", ([&] {
                                                                                                  ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/Dispatch.h:46:1: note: declared here
 inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties &t) {
 ^~~~~~~~~~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:136:21: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     grad_output_g.data<scalar_t>(),
                     ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:136:47: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     grad_output_g.data<scalar_t>(),
                                               ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:137:5: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     value.data<scalar_t>() + n * im2col_step_ * per_value_size,
     ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:137:48: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     value.data<scalar_t>() + n * im2col_step_ * per_value_size,
                                                ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:137:81: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     value.data<scalar_t>() + n * im2col_step_ * per_value_size,
                                                                                 ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:138:36: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     spatial_shapes.data<int64_t>(),
                                    ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:139:69: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     level_start_index.data<int64_t>(),
                                                                     ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:140:25: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
                         ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:140:113: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
                                                                                                                 ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:142:80: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
                                                                                ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:142:105: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
                                                                                                         ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:143:63: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     grad_value.data<scalar_t>() +  n * im2col_step_ * per_value_size,
                                                               ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:143:106: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     grad_value.data<scalar_t>() +  n * im2col_step_ * per_value_size,
                                                                                                          ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:144:10: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     grad_sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
          ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:144:92: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     grad_sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
                                                                                            ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:145:124: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
                                     grad_attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size);
                                                                                                                            ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:146:79: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
 
                                                                               ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
/home/matthias/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:147:38: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
         }));
                                      ^
/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
   T * data() const {
 ^ ~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1395, in _run_ninja_build
    subprocess.run(
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/trackformer/models/ops/setup.py", line 62, in <module>
    setup(
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
    build_ext.build_extensions(self)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
    _build_ext.build_extension(self, ext)
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
    objects = self.compiler.compile(sources,
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 414, in unix_wrap_ninja_compile
    _write_ninja_file_and_compile_objects(
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1135, in _write_ninja_file_and_compile_objects
    _run_ninja_build(
  File "/home/matthias/miniconda3/envs/trackformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error compiling objects for extension

Please read & provide the following

If you do not know the root cause of the problem, and wish someone to help you, please
post according to this template:

Instructions To Reproduce the Issue:

what changes you made (git diff) or what code you wrote

<put diff or code here>

what exact command you run:
what you observed (including full logs):

<put logs here>

please simplify the steps as much as possible so they do not require additional resources to
run, such as a private dataset.

Expected behavior:

If there are no obvious error in "what you observed" provided above,
please tell us the expected behavior.

If you expect the model to converge / work better, note that we do not give suggestions
on how to train a new model.
Only in one of the two conditions we will help with it:
(1) You're unable to reproduce the results in DETR model zoo.
(2) It indicates a DETR bug.

Environment:

Provide your environment information using the following command:

python -m torch.utils.collect_env

wrongly load prev_frame image and target in `MOT.getitem`

because no_obj_imgs removed from self.ids, the ids has a new map relation between img_id. the above code will get wrong prev_frame image_id

Train on custom dataset

Dear author,
I am trying to train your model on my custom dataset i am using following command
!python src/train.py with deformable tracking resume=r50_deformable_detr-checkpoint.pth output_dir=models/custom_dataset_train_deformable mot_path=data/custom_dataset train_split=train val_split=val epochs=20
i get following error message

frame_id = self.coco.imgs[idx]['frame_id']
KeyError: 'frame_id'

Does the model load the whole image sequence into memorry?

I have a question regarding the way data is loaded into memory for this code.
Since you are using auto-regressive track queries does that mean all the images on a video sequence are loaded simultaneously into memory?
Or do you load only a specific frame and its previous one into memory at each iteration? I would really appreciate it if you could clarify this for me?

Thanks in advance,

ImportError: cannot import name '_new_empty_tensor' from 'torchvision.ops'

Camera shift

Hi,

thanks for your amazing work!

I have noticed that the results on flashmob.mp4 are really good and robust until there is a camera shift, it seems that the reid is getting lost. (frames 222, and 223).

I have used the demo:

python src/track.py with \
    reid \
    dataset_name=DEMO \
    data_root_dir=data/flashmob \
    output_dir=data/flashmob

Could it be that some of the parameters in cfgs/track.yaml should be updated:

...
    public_detections: False
    # score threshold for detections
    detection_obj_score_thresh: 0.9
    # score threshold for keeping the track alive
    track_obj_score_thresh: 0.8
    # NMS threshold for detection
    detection_nms_thresh: 0.9
    # NMS theshold while tracking
    track_nms_thresh: 0.9
    # motion model settings
    # How many timesteps inactive tracks are kept and cosidered for reid
    inactive_patience: -1
    # How similar do image and old track need to be to be considered the same person
    reid_sim_threshold: 0.2
    reid_sim_only: false
    reid_score_thresh: 0.8
    reid_greedy_matching: false

Cheers!

undefined symbol: THPVariableClass

Hello,

I want to run the code. So, at first I built the MultiScaleDeformableAttention Module and it is installed successfully.

(virenv) mohammadh@ut-cavlab:/hdd/nasseri/trackformer-main/src/trackformer/models/ops$ python3 setup.py build install --install-lib='/hdd/nasseri/virenv/bin'
running build
running build_py
running build_ext
running install
running bdist_egg
running egg_info
writing MultiScaleDeformableAttention.egg-info/PKG-INFO
writing dependency_links to MultiScaleDeformableAttention.egg-info/dependency_links.txt
writing top-level names to MultiScaleDeformableAttention.egg-info/top_level.txt
reading manifest file 'MultiScaleDeformableAttention.egg-info/SOURCES.txt'
writing manifest file 'MultiScaleDeformableAttention.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/functions
copying build/lib.linux-x86_64-3.6/functions/ms_deform_attn_func.py -> build/bdist.linux-x86_64/egg/functions
copying build/lib.linux-x86_64-3.6/functions/__init__.py -> build/bdist.linux-x86_64/egg/functions
creating build/bdist.linux-x86_64/egg/modules
copying build/lib.linux-x86_64-3.6/modules/ms_deform_attn.py -> build/bdist.linux-x86_64/egg/modules
copying build/lib.linux-x86_64-3.6/modules/__init__.py -> build/bdist.linux-x86_64/egg/modules
copying build/lib.linux-x86_64-3.6/MultiScaleDeformableAttention.cpython-36m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
byte-compiling build/bdist.linux-x86_64/egg/functions/ms_deform_attn_func.py to ms_deform_attn_func.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/functions/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/modules/ms_deform_attn.py to ms_deform_attn.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/modules/__init__.py to __init__.cpython-36.pyc
creating stub loader for MultiScaleDeformableAttention.cpython-36m-x86_64-linux-gnu.so
byte-compiling build/bdist.linux-x86_64/egg/MultiScaleDeformableAttention.py to MultiScaleDeformableAttention.cpython-36.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying MultiScaleDeformableAttention.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying MultiScaleDeformableAttention.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying MultiScaleDeformableAttention.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying MultiScaleDeformableAttention.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
__pycache__.MultiScaleDeformableAttention.cpython-36: module references __file__
creating 'dist/MultiScaleDeformableAttention-1.0-py3.6-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing MultiScaleDeformableAttention-1.0-py3.6-linux-x86_64.egg
removing '/hdd/nasseri/virenv/bin/MultiScaleDeformableAttention-1.0-py3.6-linux-x86_64.egg' (and everything under it)
creating /hdd/nasseri/virenv/bin/MultiScaleDeformableAttention-1.0-py3.6-linux-x86_64.egg
Extracting MultiScaleDeformableAttention-1.0-py3.6-linux-x86_64.egg to /hdd/nasseri/virenv/bin
MultiScaleDeformableAttention 1.0 is already the active version in easy-install.pth

Installed /hdd/nasseri/virenv/bin/MultiScaleDeformableAttention-1.0-py3.6-linux-x86_64.egg
Processing dependencies for MultiScaleDeformableAttention==1.0
Finished processing dependencies for MultiScaleDeformableAttention==1.0

and after that the pip can find the MultiScaleDeformableAttention Module:

pip show MultiScaleDeformableAttention
Name: MultiScaleDeformableAttention
Version: 1.0
Summary: Multi-Scale Deformable Attention Module in Deformable DETR
Home-page: xxx
Author: Weijie Su
Author-email: UNKNOWN
License: UNKNOWN
Location: /hdd/nasseri/virenv/bin
Requires:
Required-by:

But when I run the track.py, I get the following error:

python3 track.py
Traceback (most recent call last):
  File "track.py", line 16, in <module>
    from trackformer.models import build_model
  File "/hdd/nasseri/trackformer-main/src/trackformer/models/__init__.py", line 6, in <module>
    from .deformable_transformer import build_deforamble_transformer
  File "/hdd/nasseri/trackformer-main/src/trackformer/models/deformable_transformer.py", line 17, in <module>
    from .ops.modules import MSDeformAttn
  File "/hdd/nasseri/trackformer-main/src/trackformer/models/ops/modules/__init__.py", line 9, in <module>
    from .ms_deform_attn import MSDeformAttn
  File "/hdd/nasseri/trackformer-main/src/trackformer/models/ops/modules/ms_deform_attn.py", line 21, in <module>
    from ..functions import MSDeformAttnFunction
  File "/hdd/nasseri/trackformer-main/src/trackformer/models/ops/functions/__init__.py", line 9, in <module>
    from .ms_deform_attn_func import MSDeformAttnFunction
  File "/hdd/nasseri/trackformer-main/src/trackformer/models/ops/functions/ms_deform_attn_func.py", line 18, in <module>
    import MultiScaleDeformableAttention as MSDA
ImportError: /hdd/nasseri/trackformer-main/src/trackformer/models/ops/MultiScaleDeformableAttention.cpython-36m-x86_64-linux-gnu.so: undefined symbol: THPVariableClass

How can I solve the problem?

Error while installing MultiScaleDeformableAttention package

🐛 Bug:
Hi, after cloning the lat commit of your repo, and applying all the step to install the dependencies on a conda environment, I got the following error. Specifically when executing python src/trackformer/models/ops/setup.py build --build-base=src/trackformer/models/ops/ install

`python src/trackformer/models/ops/setup.py build --build-base=src/trackformer/models/ops/ install
running build
running build_ext
building 'MultiScaleDeformableAttention' extension

...

/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, double, long int>&&; bool = true; _Elements = {at::Tensor, at::Tensor, double, long int}]’:
/usr/include/c++/6/tuple:686:422: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, double, long int>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type > constexpr std::tuple< >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, double, long int}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, double, long int>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, double, long int>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type = ]’
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/Functions.h:5162:99: required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
return _and<_not<is_same<tuple<_Elements...>,
^
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
struct is_convertible
^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>}; bool = true; _Elements = {at::Tensor&, at::Tensor&, at::Tensor&}]’:
/usr/include/c++/6/tuple:626:248: required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor&, at::Tensor&, at::Tensor&>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type > constexpr std::tuple< >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor&, at::Tensor&, at::Tensor&>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type = ]’
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/Functions.h:6418:303: required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
return _and<is_constructible<_Elements, _UElements&&>...>::value;
^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>}; bool = true; _Elements = {at::Tensor&, at::Tensor&, at::Tensor&}]’:
/usr/include/c++/6/tuple:626:362: required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor&, at::Tensor&, at::Tensor&>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type > constexpr std::tuple< >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor&, at::Tensor&, at::Tensor&>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type = ]’
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/Functions.h:6418:303: required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
return _and<is_convertible<_UElements&&, _Elements>...>::value;
^~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>&; bool = true; _Elements = {at::Tensor&, at::Tensor&, at::Tensor&}]’:
/usr/include/c++/6/tuple:662:419: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor&, at::Tensor&, at::Tensor&>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type > constexpr std::tuple< >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor&, at::Tensor&, at::Tensor&}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor&, at::Tensor&, at::Tensor&>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type = ]’
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/Functions.h:6418:303: required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
return _and<_not<is_same<tuple<_Elements...>,
^
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
struct is_convertible
^~~~~~~~~~~~~~
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor&, at::Tensor&, at::Tensor&>&&; bool = true; _Elements = {at::Tensor&, at::Tensor&, at::Tensor&}]’:
/usr/include/c++/6/tuple:686:422: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor&, at::Tensor&, at::Tensor&>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type > constexpr std::tuple< >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor&, at::Tensor&, at::Tensor&}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor&, at::Tensor&, at::Tensor&>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor&, at::Tensor&, at::Tensor&>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type = ]’
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/Functions.h:6418:303: required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
return _and<not<is_same<tuple<Elements...>,
^
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class From, class To> struct std::is_convertible’
struct is_convertible
^~~~~~~~~~~~~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In function ‘at::Tensor ms_deform_attn_cuda_forward(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, int)’:
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:31:62: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_ASSERTM(value.type().is_cuda(), "value must be a CUDA tensor");
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:32:71: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_ASSERTM(spatial_shapes.type().is_cuda(), "spatial_shapes must be a CUDA tensor");
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:33:69: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_ASSERTM(sampling_loc.type().is_cuda(), "sampling_loc must be a CUDA tensor");
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:34:68: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_ASSERTM(attn_weight.type().is_cuda(), "attn_weight must be a CUDA tensor");
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:69:43: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_DISPATCH_FLOATING_TYPES(value.type(), "ms_deform_attn_forward_cuda", ([&] {
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:69:98: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
AT_DISPATCH_FLOATING_TYPES(value.type(), "ms_deform_attn_forward_cuda", ([&] {
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/Dispatch.h:46:1: note: declared here
inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties &t) {
^~~~~~~~~~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:71:13: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
value.data<scalar_t>() + n * im2col_step * per_value_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:71:99: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
value.data<scalar_t>() + n * im2col_step * per_value_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:72:14: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
spatial_shapes.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:72:47: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
spatial_shapes.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:73:2: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
level_start_index.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:74:32: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
sampling_loc.data<scalar_t>() + n * im2col_step * per_sample_loc_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:75:67: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
attn_weight.data<scalar_t>() + n * im2col_step * per_attn_weight_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:76:25: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:76:68: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:76:100: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:77:54: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
columns.data<scalar_t>());
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:78:83: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]

/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In function ‘std::vectorat::Tensor ms_deform_attn_cuda_backward(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, int)’:
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:100:62: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_ASSERTM(value.type().is_cuda(), "value must be a CUDA tensor");
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:101:71: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_ASSERTM(spatial_shapes.type().is_cuda(), "spatial_shapes must be a CUDA tensor");
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:102:69: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_ASSERTM(sampling_loc.type().is_cuda(), "sampling_loc must be a CUDA tensor");
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:103:68: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_ASSERTM(attn_weight.type().is_cuda(), "attn_weight must be a CUDA tensor");
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:139:43: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
AT_DISPATCH_FLOATING_TYPES(value.type(), "deform_conv_backward_cuda", ([&] {
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:262:1: note: declared here
DeprecatedTypeProperties & type() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:139:98: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
AT_DISPATCH_FLOATING_TYPES(value.type(), "deform_conv_backward_cuda", ([&] {
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/Dispatch.h:46:1: note: declared here
inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties &t) {
^~~~~~~~~~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:141:27: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
// gradient w.r.t. sampling location & attention weight
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:141:53: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
// gradient w.r.t. sampling location & attention weight
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:142:11: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
ms_deformable_col2im_coord_cuda(at::cuda::getCurrentCUDAStream(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:142:54: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
ms_deformable_col2im_coord_cuda(at::cuda::getCurrentCUDAStream(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:142:87: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
ms_deformable_col2im_coord_cuda(at::cuda::getCurrentCUDAStream(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:143:42: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
grad_output_g.data<scalar_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:144:82: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
value.data<scalar_t>() + n * im2col_step_ * per_value_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:145:42: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
spatial_shapes.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:146:61: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
level_start_index.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:146:101: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
level_start_index.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:147:16: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:147:49: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:148:4: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:149:37: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu: In lambda function:
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:151:4: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
grad_attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size);
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:151:29: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
grad_attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size);
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:151:115: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
grad_attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size);
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:152:30: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
// gradient w.r.t. value
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:152:62: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
// gradient w.r.t. value
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:153:16: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
ms_deformable_col2im_cuda(at::cuda::getCurrentCUDAStream(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:154:55: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
grad_output_g.data<scalar_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:155:14: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
spatial_shapes.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:156:32: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
level_start_index.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:156:72: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
level_start_index.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:156:115: warning: ‘T* at::Tensor::data() const [with T = long int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
level_start_index.data<int64_t>(),
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:157:19: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:157:101: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
/home/roberto/tratrackformer/trackformer/src/trackformer/models/ops/src/cuda/ms_deform_attn_cuda.cu:159:5: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,
^
/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h:341:1: note: declared here
T * data() const {
^ ~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1395, in _run_ninja_build
subprocess.run(
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "src/trackformer/models/ops/setup.py", line 56, in
setup(
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/init.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 177, in setup
return run_commands(dist)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
dist.run_commands()
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
self.run_command(cmd)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/command/build.py", line 24, in run
super().run()
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
self.distribution.run_command(command)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
build_ext.build_extensions(self)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 459, in build_extensions
self._build_extensions_serial()
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 485, in _build_extensions_serial
self.build_extension(ext)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
_build_ext.build_extension(self, ext)
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 540, in build_extension
objects = self.compiler.compile(
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 414, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1135, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/home/roberto/anaconda3/envs/track_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
raise RuntimeError(message)
RuntimeError: Error compiling objects for extension
`

Pretrained Model

I couldn't find any pretrained model anywhere. Would it be possible to provide one? I'd be happy to help in order to add support in torch.hub, I think that would be a great contribution.

Error to generate_coco_from_crowdhuman.py

Hi.
I'm getting a error when I try to run generate_coco_from_crowdhuman.py.
When creating CrowdHuman Dataset, KeyError: '284193,faa9000f2678b5e' is displayed.
How should I resolve the error?

Expected results after training on the joint set of CrowdHuman and MOT17

Hey, thank you for you excellent work!

I train TrackFormer on your default setting (load from pretrained CrowdHuaman checkpoint) on the joint set of CrowdHuman and MOT17, but get result on MOT17 73.3 MOTA (I think it is validated on your default dataset mot17_train_cross_val_frame_0_5_to_1_0_coco).

Is the result corresponds to the training set result (74.2 MOTA provided in ReadMe) or the cross-validate result (71.3 MOTA provided in the paper)?

Thank you in advance.

solved

In src/trackformer/datasets/coco.py:(_getitem_from_id method)

line 45: target['track_ids'] = torch.arange(len(target['labels']))

This will produce wrong track_ids for loading MOT dataset.

Fix: just remove this code line.

Changing the backbone

Hi Tim,

Thanks for the great work.
Could you possibly help me with changing the backbone neural net to a specific pre-trained neural net of my own?
Also, you mentioned that for MOT20, you have fine-tuned the MOT17 private detection model. Could you explain the steps of fine-tuning in more detail?

Thank you for your time.

Why different between train and inference during the date association？

I found that the data association methods used in the training process and the inference process are different. Why they are different?

Private vs Public Detection

I have a question regarding this project:

what is the difference between public and private detection? How this is going to affect the final results?

train error

when I run
python src/train.py with deformable tracking mot17 full_res resume=models/crowdhuman_train_val_deformable/checkpoint.pth output_dir=models/mot17_train_deformable_private

ERROR - train - Failed after 0:00:05!
Traceback (most recent calls WITHOUT Sacred internals):
File "src/train.py", line 332, in main
train(args)
File "src/train.py", line 265, in train
visualizers['train'], args)
File "/home/ls/code/Tracking/trackformer/src/trackformer/engine.py", line 110, in train_one_epoch
loss_dict = criterion(outputs, targets)
File "/home/ls/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/ls/code/Tracking/trackformer/src/trackformer/models/detr.py", line 332, in forward
indices = self.matcher(outputs_without_aux, targets)
File "/home/ls/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/ls/anaconda3/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 49, in decorate_no_grad
return func(*args, **kwargs)
File "/home/ls/code/Tracking/trackformer/src/trackformer/models/matcher.py", line 121, in forward
for i, c in enumerate(cost_matrix.split(sizes, -1))]
File "/home/ls/code/Tracking/trackformer/src/trackformer/models/matcher.py", line 121, in
for i, c in enumerate(cost_matrix.split(sizes, -1))]
File "/home/ls/anaconda3/lib/python3.7/site-packages/scipy/optimize/_hungarian.py", line 93, in linear_sum_assignment
raise ValueError("matrix contains invalid numeric entries")
ValueError: matrix contains invalid numeric entries

How to do multiple distributed training at the same time?

Hi, I'm trying to use distributed training. However I'm unable to run more than 1 experiment at the same time cuz the 'address already in use' error. What should I change the dist_url to be in order to launch multiple distributed training experiments at the same time?

AttributeError: 'ReadOnlyList' object has no attribute 'message'

Hi, when I load the trained checkpoint, I got this error:
AttributeError: 'ReadOnlyList' object has no attribute 'message'

I think this mistake is related to Sacred.

when reading config.yaml , i got this message:
message: The configuration is read-only in a captured function!

the config.yaml :

lr_linear_proj_names:  !!python/object/new:sacred.config.custom_containers.ReadOnlyList
  listitems:
  - reference_points
  - sampling_offsets
  state:
    message: The configuration is read-only in a captured function!

I'm just following the training instructions. And I don't know much about Sacred. Can anyone help me? Thanks firstly.

Not setting random state for torch in dataset _add_frame_to_target

Hi, in _add_frame_to_target function in dataset file (mot.py), the current code only set random state with random.setstate(random_state), but not for torch. And it cause the sample images in the same batch to be cropped potentially from different areas of the original image, which is not the desired behavior I guess? Since drastic drift in training image pairs can contaminate the training data and teach the model the wrong thing.

By changing random_state = random.getstate() in getitem to
random_state = np.random.randint(some number)
random.seed(random_state)
torch.manual_seed(random_state)
and changing random.setstate(random_state) to
random.seed(random_state)
torch.manual_seed(random_state)
the drifting in image pairs disappears, and I was able to improve the MOTA and IDF1 score by 2~3 when training on resolution 600 (compare to the original code). Can you please try this on full resolution and see if can also boost the performance? Thanks!

Evaluate TrackFormer on MOT17 with the problem with numpy

Hello, when I tried to Evaluate TrackFormer on MOT17 with python src/track.py with \ reid \ tracker_cfg.public_detections=min_iou_0_5 \ obj_detect_checkpoint_file=models/mot17_deformable_multi_frame/checkpoint_epoch_50.pth, I got the problem.

INFO - main - TRACK SEQ: MOT17-02-DPM
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 600/600 [02:15<00:00,  4.41it/s]
INFO - main - NUM TRACKS: 96 ReIDs: 13
INFO - main - RUNTIME: 135.96 s
ERROR - track - Failed after 0:08:13!
Traceback (most recent calls WITHOUT Sacred internals):
  File "src/track.py", line 153, in main
    mot_accum = get_mot_accum(results, seq_loader)
  File "/media/HardDisk_new/wh/second_code/trackformer/src/trackformer/util/track_utils.py", line 397, in get_mot_accum
    distance)
  File "/home/wh/anaconda3/envs/trackformer/lib/python3.7/site-packages/motmetrics/mot.py", line 252, in update
    rids, cids = linear_sum_assignment(dists)
  File "/home/wh/anaconda3/envs/trackformer/lib/python3.7/site-packages/motmetrics/lap.py", line 73, in linear_sum_assignment
    rids, cids = solver(costs)
  File "/home/wh/anaconda3/envs/trackformer/lib/python3.7/site-packages/motmetrics/lap.py", line 288, in lsa_solve_lapjv
    from lap import lapjv
  File "/home/wh/anaconda3/envs/trackformer/lib/python3.7/site-packages/lap/__init__.py", line 25, in <module>
    from ._lapjv import (
  File "__init__.pxd", line 199, in init lap._lapjv
ValueError: numpy.ndarray has the wrong size, try recompiling. Expected 80, got 88

used python src/track.py with reid and got the same problem

INFO - main - TRACK SEQ: MOT17-02-DPM
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 600/600 [02:15<00:00,  4.44it/s]
INFO - main - NUM TRACKS: 133 ReIDs: 25
INFO - main - RUNTIME: 135.04 s
ERROR - track - Failed after 0:07:17!
Traceback (most recent calls WITHOUT Sacred internals):
  File "src/track.py", line 153, in main
    mot_accum = get_mot_accum(results, seq_loader)
  File "/media/HardDisk_new/wh/second_code/trackformer/src/trackformer/util/track_utils.py", line 397, in get_mot_accum
    distance)
  File "/home/wh/anaconda3/envs/trackformer/lib/python3.7/site-packages/motmetrics/mot.py", line 252, in update
    rids, cids = linear_sum_assignment(dists)
  File "/home/wh/anaconda3/envs/trackformer/lib/python3.7/site-packages/motmetrics/lap.py", line 73, in linear_sum_assignment
    rids, cids = solver(costs)
  File "/home/wh/anaconda3/envs/trackformer/lib/python3.7/site-packages/motmetrics/lap.py", line 288, in lsa_solve_lapjv
    from lap import lapjv
  File "/home/wh/anaconda3/envs/trackformer/lib/python3.7/site-packages/lap/__init__.py", line 25, in <module>
    from ._lapjv import (
  File "__init__.pxd", line 199, in init lap._lapjv
ValueError: numpy.ndarray has the wrong size, try recompiling. Expected 80, got 88

pretrained model

Hi @timmeinhardt ,

Thanks for your great works! I want to produce the results in the paper with your pretrained models. However, I do not find the link to download the models. Are you going to release the models used in your paper?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.