hustvl / tevit Goto Github PK

Hi, vealocia,
Maybe we need to do it first to run this code.
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI
So I think you can add it to requirements.🥰

Thanks for your excellent work. Do you have any plans to release the code of the online reference recently?

Thanks for your great work. I am not familiar with the youtubevis API for bbox evaluation and want to learn about the evaluation procedure base on the provided trainset annotation. Besides the evaluation code on segmentation, can you provide a code that can store the bbox prediction result in standard format for evaluation? Thanks.

Traceback (most recent call last):
File "./tools/train.py", line 17, in
from mmdet.apis import init_random_seed, set_random_seed, train_detector
File "/home/hss/TeViT-main/mmdet/apis/init.py", line 2, in
from .inference import (async_inference_detector, inference_detector,
File "/home/hss/TeViT-main/mmdet/apis/inference.py", line 12, in
from mmdet.datasets import replace_ImageToTensor
File "/home/hss/TeViT-main/mmdet/datasets/init.py", line 18, in
Traceback (most recent call last):
File "./tools/train.py", line 17, in
from .youtubevis import YoutubeVISDataset
File "/home/hss/TeViT-main/mmdet/datasets/youtubevis.py", line 9, in
from pycocotools.ytvos import YTVOS
ModuleNotFoundError: No module named 'pycocotools.ytvos'
from mmdet.apis import init_random_seed, set_random_seed, train_detector
File "/home/hss/TeViT-main/mmdet/apis/init.py", line 2, in
from .inference import (async_inference_detector, inference_detector,
File "/home/hss/TeViT-main/mmdet/apis/inference.py", line 12, in
from mmdet.datasets import replace_ImageToTensor
File "/home/hss/TeViT-main/mmdet/datasets/init.py", line 18, in
from .youtubevis import YoutubeVISDataset
File "/home/hss/TeViT-main/mmdet/datasets/youtubevis.py", line 9, in
from pycocotools.ytvos import YTVOS

Thanks for your work. I'm struggling for days with this error. Can you please provide some solutions to overcome it.

(base) root@78845bc2a82a:/mmdetection/Tevit# python tools/test_vis.py configs/tevit/tevit_msgshift.py checkpoint/tevit_r50.pth
load checkpoint from local path: checkpoint/tevit_r50.pth
Traceback (most recent call last):
File "tools/test_vis.py", line 137, in
main(args)
File "tools/test_vis.py", line 54, in main
cfg_options=args.cfg_options)
File "/opt/conda/lib/python3.7/site-packages/mmdet/apis/inference.py", line 45, in init_detector
checkpoint = load_checkpoint(model, checkpoint, map_location='cpu')
File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 581, in load_checkpoint
checkpoint = _load_checkpoint(filename, map_location, logger)
File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 520, in _load_checkpoint
return CheckpointLoader.load_checkpoint(filename, map_location, logger)
File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 285, in load_checkpoint
return checkpoint_loader(filename, map_location)
File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 302, in load_from_local
checkpoint = torch.load(filename, map_location=map_location)
File "/opt/conda/lib/python3.7/site-packages/torch/serialization.py", line 585, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/opt/conda/lib/python3.7/site-packages/torch/serialization.py", line 755, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
EOFError: Ran out of input

Thanks for your excellent work.
For the test of image_demo.py(or video_demo.py) in /demo, use the demo.jpg(or demo.mp4) as input, there is a problem.
Could you please provide some advice? Looking forward to your reply.

/TeViT/mmdet/apis/inference.py:50: UserWarning: Class names are not saved in the checkpoint's meta data, use COCO classes by default.
warnings.warn('Class names are not saved in the checkpoint's '

Traceback (most recent call last):
File "image_demo.py", line 65, in
main(args)
File "image_demo.py", line 35, in main
result = inference_detector(model, args.img)
File "/TeViT/mmdet/apis/inference.py", line 137, in inference_detector
data['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']]
TypeError: 'DataContainer' object is not iterable

File "video_demo.py", line 61, in
main()
File "video_demo.py", line 47, in main
result = inference_detector(model, frame)
File "/TeViT/mmdet/apis/inference.py", line 137, in inference_detector
data['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']]
TypeError: 'DataContainer' object is not iterable

Wonderful work!
We're very interested in your work. VIS is a future key development direction in mmtracking. We'll appreciate it if you can create a PR about this work in mmtracking.

Hi,

Thanks for uploading the code and for a great paper. I have a few questions about the method as I've been reading the paper but found it difficult to understand from the codebase its implementation.

1. The STQI decoder has a DynConv layer for all N_H STQI heads. Is this DynConv layer within each STQI head the same as in QueryInst? i.e. q_t <-- DynConv_box(p_box, q_t-1). where p_box are ROI-pooled instance features
2. In the STQI figure in the paper (Figure 1) - there is just 1 Dynamic Conv per head. In QueryInst there are both dynamic mask and dynamic box layers for each stage. Can you confirm there is only DynConv_box in STQI?
3. The features from either MsgShiftT or Swin are multi-scale. How are the multi-resolution features dealt with in DynConv_box or DynConv_mask. I can't find this information in the manuscript. Do you make predictions for every scale like in an FPN network?
4. Do the N_H STQI-heads replace the 6 stages you might have in QueryInst?

Many thanks!

Thanks for this work.

Can you please share how we can test the model on one image or video.

Thanks in advance

Hello,
Thank you for publishing this awesome work. I'm struggling with this error while testing tevit on one image.

result = inference_detector(model, img) File "/mmdetection/tevit/mmdet/apis/inference.py", line 137, in inference_detector data['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']] TypeError: 'DataContainer' object is not iterable

Thanks in advance.

Hi, I am trying to run this repo and i have followed all the steps mentioned in the description.

While running the inference code, I am getting the following error.
python tools/test_vis.py configs/tevit/tevit_msgshift.py checkpoints/tevit_r50.pth Traceback (most recent call last): File "tools/test_vis.py", line 130, in <module> main(args) File "tools/test_vis.py", line 53, in main cfg_options=args.cfg_options) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmdet/apis/inference.py", line 43, in init_detector model = build_detector(config.model, test_cfg=config.get('test_cfg')) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmdet/models/builder.py", line 59, in build_detector cfg, default_args=dict(train_cfg=train_cfg, test_cfg=test_cfg)) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmcv/utils/registry.py", line 212, in build return self.build_func(*args, **kwargs, registry=self) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg return build_from_cfg(cfg, registry, default_args) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmcv/utils/registry.py", line 45, in build_from_cfg f'{obj_type} is not in the {registry.name} registry') KeyError: 'TeViT is not in the models registry'

Kindly, could you guide me if I want to create a file that directly runs this algorithm on a video or set of images and provide an out in the form of images and json format which gets saved in some folder?

Hello,

I am encountering an issue with the TeViT model and need your assistance. I am using the video_demo.py script for the inference part of the TeViT model, but when I run the script, I get the following error:

KeyError: 'TeViT is not in the mmdet::model registry. Please check whether the value of TeViT is correct or it was registered as expected.'

Could you please help me understand the root cause of this error and how it can be fixed? I need to ensure that I have correctly added the TeViT model to the mmdet::model registry, but I am having trouble identifying what might have gone wrong.

If there are any additional details I need to add or any other corrections to the command-line script I'm using, please let me know.

Thank you!

!python video_demo.py video.mp4 config_file.py checkpoint.pth --out output.mp4 --show