Comments (9)
-
In the current version (a new preprint will be released soon) we train either on only megadepth (like LoFTR), or on a combination of MegaDepth and a synthetic dataset (that we made ad-hoc). For megadepth the reference warp is constructed like in loftr (we use the warp_kpts function https://github.com/zju3dv/LoFTR/blob/2122156015b61fbb650e28b58a958e4d632b1058/src/loftr/utils/geometry.py#L5 with the input being created from a meshgrid to be dense). The depth is provided in megadepth, so no additional method is needed. Please refer to to the LoFTR (or perhaps even D2Net) for how to download and process megadepth. The synthetic dataset is very similar to the one presented in PDCNet or PDCNet+, although we have our own implementation. There there the flow is computed by a set of homographies and the confidence comes from covisibility (which is easy to compute given that we generate the warps).
-
See above. Note that we found using the confidence loss exclusively from megadepth to yield better estimation results. I.e. we only use the confidence in the synthetic dataset to zero out the regression loss in certain regions.
-
I don't remember L_swap, if you mean L_warp then it is simply done by computing warp_kpts with the grid as described above, and comparing it with the estimated dense warp. This gives an error for each pixel, we take the 2-norm in each pixel,multiply with the reference confidence(so as to remove the loss from non-matching pairs) and then take the mean. For the confidence loss we simply use binary cross entropy in each pixel.
from dkm.
@KakueiTanaka
I updated the codebase now, and there is some (hopefully working) training code.
Note that the code we provide here is adapted from our internal training framework that is quite messy, hence there might have been some translation errors. Let me know if there are some issues!
from dkm.
We do not plan to release the training code in the near future, however, if you have any questions regarding the training I'm happy to answer. Probably the training code release will be sometime after summer.
I'll leave this issue open in case others have the same question.
from dkm.
Hi, parskatt. Thank you for the excellent work. I'm trying to train it from scratch. However, I still can not understand the loss functions. As mentioned in Sec 3.4, "The reference warps can come from projected depths like in (Sarlin et al., 2020; Sun et al., 2021) or from synthetic homographies, and the reference confidence p indicates, e.g., covisibility or consistent depth.", so
- What is the choice in this paper? depth or synthetic homographies ? It seems for different dataset, the strategy is different. If depth policy used, which method is used to generate the depth maps?
- If the reference confidence p indicates covisibility, is is a scalar or a tensor with the same shape as referenced warps ? how to obtain p ?
- would you please help to give a detailed illustration about the two loss functions L_swap and L_conf ?
Thank you again.
from dkm.
Thank you for your patient illustration.
Looking forward to the new version. :)
from dkm.
Hello, Parskatt.
Could you tell me the size of the images used when training the model?
from dkm.
@KakueiTanaka Hi, we use height = 384 width=512 images, we will release preliminary training code, loss functions etc in a few days :)
from dkm.
Thanks a lot!
from dkm.
I realized that in our internal training we actually freeze the batchnorm statistics in the resnet backbone during training, which is not done here. This might cause some small discrepancies if trying to reproduce results. I'll make some updates to this code when I get back from vacation :)
from dkm.
Related Issues (20)
- DKM:sample HOT 2
- How many GPUs and consumed memory for your training? HOT 7
- Inference time HOT 2
- Can DKM run on CPU only? HOT 9
- 3d point projection, best way to fetch matches HOT 6
- `numpy.random.choice` to pytorch? HOT 6
- run on batch data HOT 5
- the structure and information of 'warp' HOT 5
- what does low_res_certainty do in dkm.py? HOT 3
- e_R reaches 180° HOT 15
- About the pretrained model HOT 3
- Questions about global matcher HOT 3
- About the pretrained model with resnet18 HOT 1
- When using multi-GPU training, there is additional memory occupancy on GPU 0 HOT 1
- About testing results HOT 2
- DKMv3 and DKMv2 HOT 4
- Pretrained weights licensing HOT 2
- Questions about the key points on Megadepth test images HOT 2
- About previous sota pdc-net+ HOT 5
- torch.linalg.inv HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dkm.