Hi!

I have 4 2080TI GPUs and want to use them to train the superglue model.
I try to run the following command:
python3 -m torch.distributed.launch --nproc_per_node=4 train_superglue.py --config_path configs/coco_config.yaml

But I get errors:

Traceback (most recent call last):
  File "train_superglue.py", line 252, in <module>
    train(config, opt.local_rank)
  File "train_superglue.py", line 165, in train
    total_loss, pos_loss, neg_loss = superglue_model.forward_train(superglue_input)
  File "/home/zhy/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 772, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'DistributedDataParallel' object has no attribute 'forward_train'

How to fix this problem?

Hello. I'm trying to annotate my custom dataset (images only) on COCO format so I can use your script for training SuperGlue. My dataset consists RGB images and I want to extract keypoints from both viewpoints to perform image matching.
My main goal is to annotate keypoints only. But I'm stuck on what to include in my annotations file. I understand that I need to focus on the Keypoint Detection task of COCO dataset. This is the description of the annotations for keypoint detection i could find: "A keypoint annotation contains all the data of the object annotation (including id, bbox, etc.) and two additional fields.
For Object Detection:
annotation{"id" : int, "image_id" : int, "category_id" : int, "segmentation" : RLE or [polygon], "area" : float, "bbox" : [x,y,width,height], "iscrowd" : 0 or 1,} categories[{ "id" : int, "name" : str, "supercategory" : str, }]

Additional fields for keypoint detection:
annotation{ "keypoints" : [x1,y1,v1,...], "num_keypoints" : int, "[cloned]" : ..., } categories[{ "keypoints" : [str], "skeleton" : [edge], "[cloned]" : ...,}]"

My primary concerns are:
i. What categories and supercategories should i define?
ii. What do i include in 'bbox', 'segmentation', 'num_keypoints' and 'skeleton'?

I would be very grateful if you can usher me in the right direction.

The recall of the model I trained with the default yaml file is 10% lower than COCO_homo, are you using any pretrained weights for training COCO_homo? How to reach the recall of COCO_homo? Thanks.

Homography using RANSAC
Indoor test set (Mean over 144 pairs)

Outdoor test set (Mean over 145 pairs)

COCO2017 test set (Mean over 199 pairs)

This code seems to only support COCO 2017 dataset. How to train in another dataset? Can you give some advice?

Hi,
I want to train superglue on my custom dataset so as far as I understood from implementation only supervision required is homography between images. Please correct me otherwise or approve.
Thanks in advance

Hi.
I have been following the code that has been shared in the respective github repository. But on executing the code I am ending with an error as follow:

Traceback (most recent call last):
  File "/home/shuhulh/SuperGlue_training/train_superglue.py", line 252, in <module>
    train(config, opt.local_rank)
  File "/home/shuhulh/SuperGlue_training/train_superglue.py", line 139, in train
    superpoint_results = superpoint_model.forward_train({'homography': homographies, 'image': orig_warped})
  File "/home/shuhulh/SuperGlue_training/models/superpoint.py", line 260, in forward_train
    results.append(remove_borders_projected(k, s, self.config['remove_borders'], h*8, w*8, homo_matrix))
  File "/home/shuhulh/SuperGlue_training/models/superpoint.py", line 73, in remove_borders_projected
    projected_keypoints = warp_keypoints(keypoints, torch.inverse(homo_matrix))
  File "/home/shuhulh/SuperGlue_training/models/superpoint.py", line 103, in warp_keypoints
    dest /= dest[:, 2:3]
RuntimeError: unsupported operation: some elements of the input tensor and the written-to tensor refer to a single memory location. Please clone() the tensor before performing the operation.

How to overcome this error?

Hi, nice work here
I found in your code one can choose between LayerNorm and BatchNorm. Did you find any performance difference here?

Thanks

First, thank you for open code engineering. I want to test the code on other data sets, but I find that Homography Estimation is mainly included in the project. There is no pose estimation. Can I test the code as provided by Superglue official? Meanwhile, I found that your code can load indoor. pth and outdoor. pth provided by the official code, but the official code cannot load the training model of your code? Thanks for your reply.

May I ask whether this project can carry out a complete training and testing process?

What is this file used for?
Downloading coco homography model.

Hi,

I am training on my dataset. And I am using the same loss function. I saw the loss begin at 3.xxx and gradually drop to 1.9xxx. What is your experience with the scope of loss values?

Best regards

hi,professor:
i want to realize superglue in C++, so it can run on nvidia jetson platform, that is a huge project, so first step, i got a problem, emm, i can not convert superglue official example pth models to c++ pt models or onnx models, so ,please help me! how can i do it?

Hi @gouthamvgk nice work, can I change the batch size to large values?

I have tried training the model for my own custom dataset, but the problem that I facing is that in class COCO_valloader it is calling a file coco_val_images_homo.txt which I don't actually understand what is it consisting of. Can you help me out how this file is being generated or what does the file consists of so that I can generate one for my own custom dataset?

torch>=1.8.1 ？？ the lastest is 1.8.0

Hi, I am trying to train superglue using hfnet keypoints instead of superpoint.

Can you please point me to the code that I need to change for this? Where is superpoint run?

Thanks!

First, thank you for open code engineering. But there is a big gap between my final result and your result.Evaluation results of your model (coco_homo) : accuracy(80.87), recall(96.22). I used your model (models/weights/superglue_cocohomo.pt) to get the final result : acc(81.43),recall(96.25).The results are very good.But I got the final result（acc : 76.70 recall : 89.20）when I trained the superglue with your superpoint model(models/weights/superpoint_v1.pth).This is a lot worse than your results.

I used the coco(2017) dataset and the other parameters have not changed , but I used DDP to train. Why is there a discrepancy in the evaluation results? I look forward to your response.

When I am training the SuperGlue model with a custom dataset, following is the warning (or maybe an error, I guess) that pops up after every epoch
Rare condition executed
What does this error/warning indicates? Should I be worried regarding the same as I observe that my loss value is decreasing as usual (not much but still it does).

Hi, thank you for your great work! i want to train a new CNN feature map in superpoint, the loss function is useful for training the superpoint and superglue?