kakaobrain / hotr Goto Github PK

hello, when i'm going to reproduce the hicodet of hotr, i encounter the error follows:
FileNotFoundError: [Errno 2] No such file or directory: 'hico_20160224_det/list_action.txt'

could you provide the list_action.txt for me? thanks! i use the hicodet datasets i had download before, not your repo

looking forward to your reply, thanks!

Dear author:
I noticed that there is at least one hoi annotaion for each image in HICO dataset. and when i try to reduce the hoi categoreies number, it will create some images without any hoi annotation. Thus when training, the Matcher will trigger errors. I wonder How to deal with zero-hoi images when traininig your model. I suppose zsero-hoi sample setting is general in real scenario. Thank you.

While trying to convert to ONNX from pytorch model I face issue
RuntimeError: output 1 (0.209699
[ CPUDoubleType{} ]) of traced region did not have observable data dependence with trace inputs; this probably indicates your program cannot be understood by the tracer.

Could you check if there's some issue and if so could you suggest a fix

안녕하세요, paper 잘 읽었습니다.
궁금한게 있어서 issue에 글 남깁니다.

shared encoder로 부터 두개의 query가 나와 각각 instance decoder와 interaction decoder로 들어간다고 나오는데
하나의 input (CNN feature + positional encoding) 으로부터 어떻게 2개의 output (instance query, interaction query)가 나오는지 궁금합니다.

다시한번 훌륭한 연구 결과물 공유해주셔서 감사드립니다.

Thanks for your nice work. When evaluating HOTR on vcoco dataset (vcoco_multi_train) on a server with 8 GeForce RTX 2080Ti Cards, I encountered the CUDA out of memory problem.

File "/project/HOI/HOTR-main/hotr/engine/evaluator_vcoco.py", line 53, in vcoco_evaluate
gather_res = utils.all_gather(res)
File "/project/HOI/HOTR-main/hotr/util/misc.py", line 129, in all_gather
data_list.append(pickle.loads(buffer))
File "/anaconda3/envs/kakaobrain/lib/python3.7/site-packages/torch/storage.py", line 141, in _load_from_bytes
return torch.load(io.BytesIO(b))
File "/anaconda3/envs/kakaobrain/lib/python3.7/site-packages/torch/serialization.py", line 595, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/anaconda3/envs/kakaobrain/lib/python3.7/site-packages/torch/serialization.py", line 774, in _legacy_load
result = unpickler.load()
File "/anaconda3/envs/kakaobrain/lib/python3.7/site-packages/torch/serialization.py", line 730, in persistent_load
deserialized_objects[root_key] = restore_location(obj, location)
File "/anaconda3/envs/kakaobrain/lib/python3.7/site-packages/torch/serialization.py", line 175, in default_restore_location
result = fn(storage, location)
File "/anaconda3/envs/kakaobrain/lib/python3.7/site-packages/torch/serialization.py", line 155, in _cuda_deserialize
return storage_type(obj.size())
File "/anaconda3/envs/kakaobrain/lib/python3.7/site-packages/torch/cuda/init.py", line 462, in _lazy_new
return super(_CudaBase, cls).new(cls, *args, **kwargs)
RuntimeError: CUDA error: out of memory

This problem seems happens on line 53 of evaluator_vcoco.py, utils.all_gather(res) . Any suggestions how to solve this problem? Thanks a lot.

Hello
Thanks for your implementation.
Is your architecture end-to-end HOI? Does it mean that it does not require any features and feature extraction?
For example, can I feed my features to your network?

hello, could you please provide the detr pretrain model finetuned on hicodet, looking forward to your reply, thanks!

Hello,
Where can I find the pretrained weights hico_q16.pth and vcoco_q16.pth, please ?
Thank you?

thank you very much!

hello，in main.py, there is a question that what does base_ds mean, I can not find the definition. Thanks!

Thanks for your great work.
hico det dataset 's "annotation files " download link is same as "HICO-DET" .it lead to can not download “annotation files”

hello, what a wonderful work, could you provide the hicodet version, thanks!

Nice work.

I make the hico_single_test, but why i get map all zero, and my test image has 4773 images, while yours 573 in readme file.

Thank you.

Line41

Why did you use "else:" in line 41?
Doesn't it work the same without using "else:"?

Is this optional "else:" clause by "if:" statement of Line39?

Could you provide the training log files? Thanks

hello, i don't understand the meaning of target classification in hicodet, could you please explain it? looking forward to your reply

hello, i read the code of loss calculation, you take loss_value and loss_dict_reduced_scaled to metric_logger as input. but your metric_logger class is difficult for me to understand. so i have three questions about the loss log. as is shown in figure

1. i saw each loss have two items such as: loss_act:1.3329(1.5253), how the meaning of 1.3329 and 1.5253
2. the loss_value is sum of loss_dict_reduced_scaled's values. but when i add the each loss respectively, as is shown in figure
   
   the sum 21.6664 != 24.1092, but the other sum is correct, can you explain it?
3. when i saw the loss in the training process, which loss should i pay attention to? the loss in the parentheses or loss is outside parenthneses.

looking foward to your reply! thanks!

Nice work.

I make the hico_single_test, but why i get map all zero, and my test image has 4773 images, while yours 573 in readme file.

Thank you.

Dear author:
I tested some influential HOI algorithms, such as iCAN, and found these algorithms may perform good in public dataset. However, when transfered to real scenario images, such as the photos taken by my mobile phone, i found the performance dropped severely. My team are trying to make HOI work on some real scenario, therefore I would invite you to reccomend a neat and effective algorithm to use. Which algorithm will you reccommend,? Thank you.

Dear @meliketoy,
We attempt to draw the interaction results in the images, could you please tell us that whether the inst_target is the interaction actions?

Hello I am currently writing my master thesis in the field of HOI detection. I would like to use custom data for this. However, I still lack any clues on how to annotate this data.
Can you recommend a tool for this? I would also like to investigate HOTR in more detail in my thesis and write a paper about it. I would be very happy to receive a response.

Best regards

Thank you for sharing the code! I am trying to set up HICO-DET but the Google links to pre-processed annotations and list_action.txt are invalid. I can find the annotations files in PPDM repo, but can't find the list_action.txt.

hello, have you tried to finetune the whole structure of detr on hicodet datasets later? i want to know the performance gap with detr without finetune.

hello, i found that when your code are training, the gpu memory usage is small, but after testing, the gpu memory usage becomes large, could you please explain it? thanks!

I have an issue with the training, for the 2 datasets, I get the same error: Loss is nan, stopping training.

Dear author:
Thanks for sharing the training and validation code. A lot of researchers, just like me, want to quickly try your work and test it on custom images. Would you kindly share the demo code to inference on single image? Thank you very much.

Dear author:
I understand the meaning of keys: roles, roles_bbox, roles_object_id in v-coco annotation files. But when i try to follow your code in vcoco dataset dataloader, I found these new variable, such as inst_action and action, pair_actitons, are really confusing for me. Could you kindly illustrate these variables? thank you.

hello, in your readme, you said the temperature is relative with number of decoder. but you don't tell us how to adjust the temperature according the number of decoders. looking forward to your reply, thanks!

dear @meliketoy
I'm trying to run demo code. In environment setup, There is no activate kakaobrain. could you add that line? It is good for junior developer like me. Thanks!

Thanks for the nice work. Could you please provide the download link?

I don't completely understand the outputs that hotr outputs. For example, if I want to visualize from this output, how can I do that?

Hello and thank you for your great work!

I am currently training your model with custom data, but I would like to run the predictions on a test data set and visualise them.

In a previous post you said that you will soon provide the code for this. However, I haven't found anything in the repository yet. If the code is not yet available, can you tell me how and where I can use the predications to visualise them on test images?

hello, i spend some time to read the code of hotr_matcher.py, but i couldn't understand it clearly. when i debug the code, the output of hotr_matcher.py is [(tensor([1]), tensor([0])), (tensor([4]), tensor([0]))] with bs=2, could you explain the meaning of this output? each element represents what? thanks very much, looking forward to your reply!