fawnliu / tris Goto Github PK

Thank you for your great work. I have a question for you. In the first stage of training, we have divided a positive sample of a text image pair and N negative samples of other text and this image, so why do we still need to select the response graph generated by these sample pairs in stage 1? Why not just select the response graph of the positive sample？

Hi,

Thank you for sharing your work!

I noticed in demo.py, the get_transform() method does not respect the size argument. It is hardcoded to resize to (224,224):

I believe this should be (size,size), which will take 320px by default. I can confirm that this change results in a heatmap that is a lot closer to the example in your readme.

Before fix

After fix

I'm still not sure why my heatmap doesn't match your example 100%, but the results are impressive nonetheless. 🙂

Thanks authors for sharing the code. I have a following question:

When computing the attention map for visual features, is Av in the below line a all-one tensor? Only one language vector is used as key and the softmax is applied on the last dimension which is 1.

When will your group release the paper and code?

I would like to know how long your model was trained on a single RTX 3090?

Thank you for your outstanding paper! I tried to retrain your model to use it as my baseline model.

This is my current state on training stage1.

I just checked that about 33seconds consumed on loss.backword().

Is it right that all_eta is logged as 5 days?

run in the order you specified but run these code :

Train IRNet and generate pseudo masks.
cd IRNet

dir=../output
CUDA_VISIBLE_DEVICES=0,1,2,3 python run_sample_refer.py --cam_out_dir $dir/refcocog_umd/cam --ir_label_out_dir $dir/refcocog_umd/ir_label --ins_seg_out_dir $dir/refcocog_umd/ins_seg --train_list $dir/refcocog_umd/refcocog_train_names.json --cam_eval_thres 0.15 --work_space output_refer/refcocog_umd --num_workers 8 --irn_batch_size 96 --cam_to_ir_label_pass True --train_irn_pass True --make_ins_seg_pass True

error: No such file or directory: '../output/refcocog_umd/refcocog_train_names.json'