xuelunshen / rfnet Goto Github PK
View Code? Open in Web Editor NEWRF-Net: An End-to-End Image Matching Network based on Receptive Field
License: MIT License
RF-Net: An End-to-End Image Matching Network based on Receptive Field
License: MIT License
Do I misunderstand the concept of over-fitting?
How I should preprocess image in order to get the parameters (im_data, im_info, im_raw) to feed to the RFNetSO.inference input?
I try use next preprocessing in (change) ms.py:
`model.eval()
img = np.array(Image.open(args.image).convert('LA'))[..., 0]
im_data = torch.Tensor(cv2.resize(img, (720, 960))).to('cuda').unsqueeze(0).unsqueeze(0) / 255
raw_data = torch.Tensor(cv2.resize(img, (240, 320))).to('cuda').unsqueeze(0).unsqueeze(0) / 255
#sh sw
image_info = torch.FloatTensor([[720 / img.shape[0], 960 / img.shape[1]]]).cuda()
#
im_scale, kpts, im_des = model.inference(im_data, image_info, raw_data)`
Hello
Thanks for sharing the great code
I couldn't get my image pair to match with this code in Google Colab
could you please guide me how to run my image pair in colab with RF-net code (step - by - step)?
With respect
when I change the train.batchsize in config.py from 1 to 2 and change cuda_visibled_device to 2GPU,there is a attributeError:'float' object has no attribute 'mean'. PLT_SCALAR[key]=PLT_SCALAR[key].mean()
When I run this:
python example.py --imgpath /home/lz/PycharmProjects/rfnet-master/data/hpatch_v_sequence/v_abstract/1.ppm@/home/lz/PycharmProjects/rfnet-master/data/hpatch_v_sequence/v_abstract/2.ppm --resume home/lz/PycharmProjects/rfnet-master/runs/10_24_09_25/model/e121_NN_0.480_NNT_0.655_NNDR_0.813_MeanMS_0.649.pth.tar
I will see a error: Segmentation fault (core dumped)
After locating, this happen in:
model = RFNetSO(
det, des, cfg.LOSS.SCORE, cfg.LOSS.PAIR, cfg.PATCH.SIZE, cfg.TRAIN.TOPK
)
What's the problem? How can I fix it?
Hello, first of all thank you for your work, here I would like to ask a little question.The comparison algorithm lfnet used in your article mentioned. You retrained it with the hpatch dataset, but lfnet training seems to require a depth map. How did you solve this problem?
Traceback (most recent call last):
File "C:\Users\86132\Desktop\rfnet-master\example.py", line 61, in
kp1, des1, img1 = model.detectAndCompute(img1_path, device, (240, 320))
File "C:\Users\86132\Desktop\rfnet-master\model\rf_net_so.py", line 216, in detectAndCompute
_, kp, des = self.inference(img, img_info, img_raw)
File "C:\Users\86132\Desktop\rfnet-master\model\rf_net_so.py", line 161, in inference
im_score = self.det.process(im_rawsc)[0]
File "C:\Users\86132\Desktop\rfnet-master\model\rf_det_module.py", line 209, in process
padding=self.GAUSSIAN_KSIZE // 2,
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
what's the average match quantity?
Hello, I'm sorry to bother you. I want to learn, but I can't open the link between data and pretrained model.Could you tell me how else to download it?Or could you give me another link?thank you
11-11 02:03:53 : Prepare for repetition
True
11-11 02:03:53 : Build the model
cuda:0
True
11-11 02:03:56 : Loading traning data
11-11 02:03:56 : /mnt/rfnet/data/hpatch_v_sequence has 235 pair images
11-11 02:04:00 : Loading evaluation data
11-11 02:04:00 : /mnt/rfnet/data/hpatch_v_sequence has 30 pair images
11-11 02:04:01 : Loading testing data
11-11 02:04:01 : /mnt/rfnet/data/hpatch_v_sequence has 30 pair images
11-11 02:04:02 : Start training
train.py:318: UserWarning: Anomaly Detection has been enabled. This mode will increase the runtime and should only be enabled for debugging.
with autograd.detect_anomaly():
/opt/conda/envs/rfnet/lib/python3.7/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.)
return VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/mnt/rfnet/model/rf_net_module.py:53: UserWarning: floordiv is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
half_scale = im2_scale // 2
/opt/conda/envs/rfnet/lib/python3.7/site-packages/torch/nn/functional.py:4004: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
"Default grid_sample and affine_grid behavior has changed "
/mnt/rfnet/model/rf_det_module.py:202: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad(True), rather than tensor.new_tensor(sourceTensor).
None, None, :, :
Traceback (most recent call last):
File "train.py", line 430, in
train()
File "train.py", line 322, in train
endpoint = model(batch).cuda()
File "/opt/conda/envs/rfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/rfnet/model/rf_net_so.py", line 48, in forward
im2_rawsc, homo12
File "/mnt/rfnet/model/rf_net_module.py", line 39, in gtscore
im1gt_score, topk_mask, topk_value = self.det.process(im1w_score).cuda()
File "/mnt/rfnet/model/rf_det_module.py", line 209, in process
padding=self.GAUSSIAN_KSIZE // 2,
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
What's the meaning of imgBatchXYZ?
Error result display:
D:/金林/pytorch/rfnet-master/rfnet/train.py:310: UserWarning: Anomaly Detection has been enabled. This mode will increase the runtime and should only be enabled for debugging.
with autograd.detect_anomaly():
D:\pytools\Anaconda3\envs\rfnet\lib\site-packages\torch\nn\functional.py:3826: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
"Default grid_sample and affine_grid behavior has changed "
D:\金林\pytorch\rfnet-master\rfnet\model\rf_det_module.py:202: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than tensor.new_tensor(sourceTensor).
None, None, :, :
Traceback (most recent call last):
File "D:/金林/pytorch/rfnet-master/rfnet/train.py", line 422, in
train()
File "D:/金林/pytorch/rfnet-master/rfnet/train.py", line 314, in train
endpoint = model(batch)
File "D:\pytools\Anaconda3\envs\rfnet\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\金林\pytorch\rfnet-master\rfnet\model\rf_net_so.py", line 48, in forward
im2_rawsc, homo12
File "D:\金林\pytorch\rfnet-master\rfnet\model\rf_net_module.py", line 39, in gtscore
im1gt_score, topk_mask, topk_value = self.det.process(im1w_score)
File "D:\金林\pytorch\rfnet-master\rfnet\model\rf_det_module.py", line 209, in process
padding=self.GAUSSIAN_KSIZE // 2,
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
Process finished with exit code 1
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same。
sorry to bother you ,but I got some problems while running the code.Have you ever meet problems like this:
RuntimeError : Function 'SqrtBackward' returned nan values in its 0th output.
@Xylon-Sean
Thank you for sharing a great work.
But this model seems not good at unseen images.
I had tested some images ( not in Hpatches ) and the results are pretty bad.
Hi,
Thanks for providing the code and making it so user-friendly! I had a small query regarding the name of the pretrained weight file shared. e121_NN_0.480_NNT_0.655_NNDR_0.813_MeanMS_0.649.pth.tar
Am I right in assuming they're the accuracies using those evaluation metrics? If so, this is on which data exactly? (HPatches? Validation? Test? V/I?) Also what is MeanMS?
I started to train the same sample data on machine with the same configuration, but it always reported "CUDA out of memory" error. And I've changed the size of the input image, but nothing has changed. Looking forward to your reply as soon as possible!
How I can match detectAndCompute inference results on two images?
I have this question because I try use default OpenCV matching approaches such as BFMatcher and FLANN with kNN method, but I got bad results.
Code:
base_kp1, base_des1 = feature_extractor.detectAndCompute(
img1, None)
warp_kp2, warp_des2 = feature_extractor.detectAndCompute(img2, None)
bf = cv2.BFMatcher()
matches = bf.knnMatch(warp_des2, base_des1, k=2)
Note: detectAndCompute is changed for another input.
And I try use next approach:
def nearest_neighbor_match_score(des1, des2, kp1, kp2, COO_THRSH):
des_dist_matrix = distance_matrix_vector(des1, des2)
nn_value, nn_idx = des_dist_matrix.min(dim=-1)
kp1w = kp1
nn_kp2 = kp2.index_select(dim=0, index=nn_idx)
coo_dist_matrix = pairwise_distances(
kp1w[:, 1:3].float(), nn_kp2[:, 1:3].float()
)
return list(
zip(
*np.where(coo_dist_matrix.le(COO_THRSH).to('cpu').numpy() > 0)
)
)
Please share your matching code.
Sorry, I try to run train.sh,but got this errorRuntimeError: CUDA out of memory. Tried to allocate 64.00 MiB (GPU 0; 7.77 GiB total capacity; 6.85 GiB already allocated; 59.12 MiB free; 148.04 MiB cached)
My computer has two 2080 and I try to downsample images fed to model and also failed ,Would you please tell me how to modify the program to reduce the memory usage.
Thanks a lot!
Hi!
I want to train your model on my own dataset that is similar to COCO, with only one image of each area. Would it be possible to train your model on this type of dataset ? would love some of your input.
My idea is to generate/sample a homography of my training image and generate a patch of the same area based on this homography. Similar to what is done in SuperPoint model. And then use this connection between my original image, the generated patch and the now known sample homography between them as training. I think this would be enough to train RF-net, but there might be something i'm missing.
Is there any other parts i need as labels or known facts about my data that i need to train ?
Hope i made myself understandable.
Thanks for any input!
Sincerely
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.