xuelunshen / rfnet Goto Github PK

Traceback (most recent call last):
File "C:\Users\86132\Desktop\rfnet-master\example.py", line 61, in
kp1, des1, img1 = model.detectAndCompute(img1_path, device, (240, 320))
File "C:\Users\86132\Desktop\rfnet-master\model\rf_net_so.py", line 216, in detectAndCompute
_, kp, des = self.inference(img, img_info, img_raw)
File "C:\Users\86132\Desktop\rfnet-master\model\rf_net_so.py", line 161, in inference
im_score = self.det.process(im_rawsc)[0]
File "C:\Users\86132\Desktop\rfnet-master\model\rf_det_module.py", line 209, in process
padding=self.GAUSSIAN_KSIZE // 2,
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

I started to train the same sample data on machine with the same configuration, but it always reported "CUDA out of memory" error. And I've changed the size of the input image, but nothing has changed. Looking forward to your reply as soon as possible！

What's the meaning of imgBatchXYZ?

Hi,

Thanks for providing the code and making it so user-friendly! I had a small query regarding the name of the pretrained weight file shared. e121_NN_0.480_NNT_0.655_NNDR_0.813_MeanMS_0.649.pth.tar Am I right in assuming they're the accuracies using those evaluation metrics? If so, this is on which data exactly? (HPatches? Validation? Test? V/I?) Also what is MeanMS?

你好，我在下载预训练模型时发现压缩包已损坏，可以重新更新一下链接吗

Hello, first of all thank you for your work, here I would like to ask a little question.The comparison algorithm lfnet used in your article mentioned. You retrained it with the hpatch dataset, but lfnet training seems to require a depth map. How did you solve this problem?

Do I misunderstand the concept of over-fitting?

How I can match detectAndCompute inference results on two images?
I have this question because I try use default OpenCV matching approaches such as BFMatcher and FLANN with kNN method, but I got bad results.

Code:

base_kp1, base_des1 = feature_extractor.detectAndCompute(
            img1, None)
warp_kp2, warp_des2 = feature_extractor.detectAndCompute(img2, None)
bf = cv2.BFMatcher()
matches = bf.knnMatch(warp_des2, base_des1, k=2)

Note: detectAndCompute is changed for another input.

And I try use next approach:

def nearest_neighbor_match_score(des1, des2, kp1, kp2, COO_THRSH):
    des_dist_matrix = distance_matrix_vector(des1, des2)
    nn_value, nn_idx = des_dist_matrix.min(dim=-1)

    kp1w = kp1
    nn_kp2 = kp2.index_select(dim=0, index=nn_idx)

    coo_dist_matrix = pairwise_distances(
        kp1w[:, 1:3].float(), nn_kp2[:, 1:3].float()
    )

    return list(
        zip(
            *np.where(coo_dist_matrix.le(COO_THRSH).to('cpu').numpy() > 0)
        )
    )

Please share your matching code.

@Xylon-Sean
Thank you for sharing a great work.
But this model seems not good at unseen images.
I had tested some images ( not in Hpatches ) and the results are pretty bad.

Hi,

Can you please share or tell test sequences that you used to compute the repeatability for Table 4 in the paper.

Hi!

I want to train your model on my own dataset that is similar to COCO, with only one image of each area. Would it be possible to train your model on this type of dataset ? would love some of your input.

My idea is to generate/sample a homography of my training image and generate a patch of the same area based on this homography. Similar to what is done in SuperPoint model. And then use this connection between my original image, the generated patch and the now known sample homography between them as training. I think this would be enough to train RF-net, but there might be something i'm missing.

Is there any other parts i need as labels or known facts about my data that i need to train ?
Hope i made myself understandable.

Thanks for any input!

Sincerely

Error result display：
D:/金林/pytorch/rfnet-master/rfnet/train.py:310: UserWarning: Anomaly Detection has been enabled. This mode will increase the runtime and should only be enabled for debugging.
with autograd.detect_anomaly():
D:\pytools\Anaconda3\envs\rfnet\lib\site-packages\torch\nn\functional.py:3826: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
"Default grid_sample and affine_grid behavior has changed "
D:\金林\pytorch\rfnet-master\rfnet\model\rf_det_module.py:202: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than tensor.new_tensor(sourceTensor).
None, None, :, :
Traceback (most recent call last):
File "D:/金林/pytorch/rfnet-master/rfnet/train.py", line 422, in
train()
File "D:/金林/pytorch/rfnet-master/rfnet/train.py", line 314, in train
endpoint = model(batch)
File "D:\pytools\Anaconda3\envs\rfnet\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\金林\pytorch\rfnet-master\rfnet\model\rf_net_so.py", line 48, in forward
im2_rawsc, homo12
File "D:\金林\pytorch\rfnet-master\rfnet\model\rf_net_module.py", line 39, in gtscore
im1gt_score, topk_mask, topk_value = self.det.process(im1w_score)
File "D:\金林\pytorch\rfnet-master\rfnet\model\rf_det_module.py", line 209, in process
padding=self.GAUSSIAN_KSIZE // 2,
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

Process finished with exit code 1

RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same。

what's the average match quantity?

11-11 02:03:53 : Prepare for repetition
True
11-11 02:03:53 : Build the model
cuda:0
True
11-11 02:03:56 : Loading traning data
11-11 02:03:56 : /mnt/rfnet/data/hpatch_v_sequence has 235 pair images
11-11 02:04:00 : Loading evaluation data
11-11 02:04:00 : /mnt/rfnet/data/hpatch_v_sequence has 30 pair images
11-11 02:04:01 : Loading testing data
11-11 02:04:01 : /mnt/rfnet/data/hpatch_v_sequence has 30 pair images
11-11 02:04:02 : Start training
train.py:318: UserWarning: Anomaly Detection has been enabled. This mode will increase the runtime and should only be enabled for debugging.
with autograd.detect_anomaly():
/opt/conda/envs/rfnet/lib/python3.7/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.)
return VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/mnt/rfnet/model/rf_net_module.py:53: UserWarning: floordiv is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
half_scale = im2_scale // 2
/opt/conda/envs/rfnet/lib/python3.7/site-packages/torch/nn/functional.py:4004: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
"Default grid_sample and affine_grid behavior has changed "
/mnt/rfnet/model/rf_det_module.py:202: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad(True), rather than tensor.new_tensor(sourceTensor).
None, None, :, :
Traceback (most recent call last):
File "train.py", line 430, in
train()
File "train.py", line 322, in train
endpoint = model(batch).cuda()
File "/opt/conda/envs/rfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/rfnet/model/rf_net_so.py", line 48, in forward
im2_rawsc, homo12
File "/mnt/rfnet/model/rf_net_module.py", line 39, in gtscore
im1gt_score, topk_mask, topk_value = self.det.process(im1w_score).cuda()
File "/mnt/rfnet/model/rf_det_module.py", line 209, in process
padding=self.GAUSSIAN_KSIZE // 2,
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

When I run this:

python example.py --imgpath /home/lz/PycharmProjects/rfnet-master/data/hpatch_v_sequence/v_abstract/1.ppm@/home/lz/PycharmProjects/rfnet-master/data/hpatch_v_sequence/v_abstract/2.ppm --resume home/lz/PycharmProjects/rfnet-master/runs/10_24_09_25/model/e121_NN_0.480_NNT_0.655_NNDR_0.813_MeanMS_0.649.pth.tar

I will see a error: Segmentation fault (core dumped)
After locating, this happen in:

model = RFNetSO(
        det, des, cfg.LOSS.SCORE, cfg.LOSS.PAIR, cfg.PATCH.SIZE, cfg.TRAIN.TOPK
    )

What's the problem? How can I fix it?

when I change the train.batchsize in config.py from 1 to 2 and change cuda_visibled_device to 2GPU,there is a attributeError:'float' object has no attribute 'mean'. PLT_SCALAR[key]=PLT_SCALAR[key].mean()

Hello
Thanks for sharing the great code
I couldn't get my image pair to match with this code in Google Colab
could you please guide me how to run my image pair in colab with RF-net code (step - by - step)?
With respect

Hello, I'm sorry to bother you. I want to learn, but I can't open the link between data and pretrained model.Could you tell me how else to download it?Or could you give me another link?thank you

Sorry, I try to run train.sh,but got this errorRuntimeError: CUDA out of memory. Tried to allocate 64.00 MiB (GPU 0; 7.77 GiB total capacity; 6.85 GiB already allocated; 59.12 MiB free; 148.04 MiB cached)
My computer has two 2080 and I try to downsample images fed to model and also failed ,Would you please tell me how to modify the program to reduce the memory usage.
Thanks a lot!

sorry to bother you ,but I got some problems while running the code.Have you ever meet problems like this:
RuntimeError : Function 'SqrtBackward' returned nan values in its 0th output.

How I should preprocess image in order to get the parameters (im_data, im_info, im_raw) to feed to the RFNetSO.inference input?

I try use next preprocessing in (change) ms.py:

`model.eval()

img = np.array(Image.open(args.image).convert('LA'))[..., 0]

im_data = torch.Tensor(cv2.resize(img, (720, 960))).to('cuda').unsqueeze(0).unsqueeze(0) / 255
raw_data = torch.Tensor(cv2.resize(img, (240, 320))).to('cuda').unsqueeze(0).unsqueeze(0) / 255

#sh sw
image_info = torch.FloatTensor([[720 / img.shape[0], 960 / img.shape[1]]]).cuda()
#
im_scale, kpts, im_des = model.inference(im_data, image_info, raw_data)`