nianticlabs / map-free-reloc Goto Github PK

Hi,

This benchmark looks really interesting!

I was wondering if you also have a leaderboard for the validation set based on your baselines?
I'm trying to run some of the default configs and would like to check if the results match up.

Kind regards,
Jaime

I can help but notice that each scene has a lot of images that are very correlated. This leads to evaluation on my local machine taking multiple hours (about 1 sec for matching, and 1 sec for ransac).

Considering the images in the query sequences are very correlated, simply taking every 10th frame ought to give almost identical results.

I get that I could do this manually for the train/val sets, but I'm simply interested in eval for benchmarking.

https://research.nianticlabs.com/mapfree-reloc-benchmark/benchmark

Doesn't seem to exist?

Should it be: https://research.nianticlabs.com/mapfree-reloc-benchmark/submit ?

Hi
Thanks for your greate work! I want to reproduce the results on 7scenes dataset but there are many config settings and I am a little confused. How can I run as the paper, e.g. using all mapping frames and get the results in Figure 6?

Best regards,
Peng

Hello,

I've made a private submission to the website, uploaded two days ago. It still says "processing" under status. This is surprising since the page indicates the results should be ready within a few minutes. Could someone please assist with this? It is for an upcoming deadline.

Thanks so much!
Chris

Hi! thanks for your great work!
The README.md says:

The pose is given in world-to-camera format, i.e. $R(q), t$ transform a world point $p$ to the camera coordinate system as $Rp + t$.

But what does the camera coordinate system look like? In detail, what is the direction of x,y,z axis of camera?

For example, this is for s00535. It seems strange to me that the camera parameters are not consistent, since it seems to be the same camera taking the images? The variation is quite small, and as I've understood the evaluation thresholds are mild so maybe exact ground truth is not that important, still interested in the reasoning though.

In uploading my solution to the submission site, generated by running submission.py, the above error is raised.
I'm pretty sure the file uploaded is correctly formatted.

In case I've done something stupid, I've also uploaded the submission so you can check if its somehow weird
submission.zip

From a high level, what is the difference between map-free relocalization and a SLAM system? From my understanding, a SLAM system estimates the relative pose to a map and a map-free relocalization task estimates the metric relative pose to a reference image. Is this correct? Both methods also incrementally build a map from estimated camera poses. If this is incorrect, please let me know.

I use 3d3d method with or without dptkitti to run the submission.py on test dataset and modify ESTIMATED_DEPTH: dptkitti in mapfree.yaml
python submission.py config/regression/mapfree/3d3d.yaml --checkpoint mapfree/3d3d.ckpt --split test -o results/

and submit, but get same result.

Thanks for your great dataset, it's really awesome and I want to use the dataset to train a RPR network modified from LoFTR.

However, training LoFTR requires correspondences between image pairs, I find that you provided precomputed correspondences for val and test splits. From what I understand, these correspondences are used for produce same pose estimation results as in your paper but not groundtruth.

I have produced some gt correspondences for other datasets by get match pairs with image matching network and filter these match pairs using epipolar distance. But this would be difficult for this dataset, for training set is too large to calculate and filter correspondences from scratch. Is there any method to produce correspondences of training set quickly?