ucuapps / openglue Goto Github PK
View Code? Open in Web Editor NEWOpen Source Graph Neural Net Based Pipeline for Image Matching
License: MIT License
openglue's People
Contributors
Stargazers
Watchers
Forkers
leilegelei1 areslp bluedream1121 gfloros peterzs redabuspatrol bnapora jlunder00 christianmarzahl cyrilyang lvyong1943 kongan presunshine kevinjinm davorjordacevic hellovuong zebin-dm xuyuting45 doublepoints wpacholarz angelababypro keunmo vincentqyw chengwei920412 marsbzp zwl995 brahimmade waicool20 perseusdg linzgood jaafarmahmoud1 datomi79 cuistiano notgooood gongxi1994 ragnariock flyinggh cocohao715 imagecn deyh2020 chenzhy223 whuhxbopenglue's Issues
Info regarding number of keypoints for training
Hello,
Thanks for the work and great paper.
While training, I noticed that the number of keypoints for img1 and img2 is constant. The loss function is also built on this assumption.
Can you please let me know what may happen if I use a different number of keypoints for img1 and img2?
Would it have an adverse effect on the training?
For example: If I use 500 keypoints for Img1 and 1000 keypoints for Img2.
Thank you
Typical Training Time
Hi, nice work there. But I could not find any training time info in the repo or in the paper. Eg: OpenGlue w/ magic leap superpoint on X GPU for 1 epoch .etc.
Will you kindly share more information so one can know everything is working as expected ?
Out of bounds error in run_nms
HI,
I am trying to get openGlue running. I start training like this:
python train.py --config='config/config.yaml' --features_config='config/features_online/sift.yaml'
However during training superglue with sift I encounter an error.
Epoch 0: 3%|▉ | 313/10851 [05:52<3:18:00, 1.13s/it, loss=7.73, v_num=0s2i, Train NLL loss=8.400, Train Metric loss=0.000]../aten/src/ATen/native/cuda/IndexKernel.cu:91: operator(): block: [1,0,0], thread: [41,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:91: operator(): block: [1,0,0], thread: [62,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
Running the debugger I get the error:
CUDA error: device-side assert triggered
File "/media/ssd4TB/software/OpenGlue/models/matching_module.py", line 81, in training_step
lafs1, responses1, desc1 = self.local_features_extractor(batch['image1'])
File "/media/ssd4TB/software/OpenGlue/models/features/base.py", line 79, in forward
lafs, scores = self.run_nms(lafs, scores, image.size())
File "/media/ssd4TB/software/OpenGlue/models/features/base.py", line 49, in run_nms
mask[0, 0, kpts_[:, 1], kpts_[:, 0]] = scores_
RuntimeError: CUDA error: device-side assert triggered
During handling of the above exception, another exception occurred:
File "/media/ssd4TB/software/OpenGlue/train.py", line 86, in main
trainer.fit(model, datamodule=dm, ckpt_path=config.get('checkpoint'))
File "/media/ssd4TB/software/OpenGlue/train.py", line 90, in <module>
main()
I use pytroch 1.11.0 with cuda 11.3 and pytorch lightning 1.6
GPU is a 3090 and operating system is Ubunutu.
# packages in environment at /home/wga2hi/anaconda3/envs/openGlue_2:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main defaults
_openmp_mutex 5.1 1_gnu defaults
absl-py 1.4.0 pypi_0 pypi
aiohttp 3.8.4 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
albumentations 1.3.0 pypi_0 pypi
antlr4-python3-runtime 4.9.3 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
async-timeout 4.0.2 pypi_0 pypi
asynctest 0.13.0 pypi_0 pypi
attrs 23.1.0 pypi_0 pypi
bosch-ca 1.0 1 defaults
ca-certificates 2022.10.11 boschca_h06a4308_0 [bosch-ca] defaults
cachetools 5.3.0 pypi_0 pypi
certifi 2022.12.7 boschca_py37h06a4308_0 [bosch-ca] defaults
charset-normalizer 3.1.0 pypi_0 pypi
click 8.1.3 pypi_0 pypi
cycler 0.11.0 pypi_0 pypi
deepdish 0.3.7 pypi_0 pypi
docker-pycreds 0.4.0 pypi_0 pypi
fonttools 4.38.0 pypi_0 pypi
frozenlist 1.3.3 pypi_0 pypi
fsspec 2023.1.0 pypi_0 pypi
gitdb 4.0.10 pypi_0 pypi
gitpython 3.1.31 pypi_0 pypi
google-auth 2.18.1 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
grpcio 1.54.2 pypi_0 pypi
idna 3.4 pypi_0 pypi
imageio 2.29.0 pypi_0 pypi
importlib-metadata 6.6.0 pypi_0 pypi
joblib 1.2.0 pypi_0 pypi
kiwisolver 1.4.4 pypi_0 pypi
kornia 0.6.12 pypi_0 pypi
kornia-moons 0.2.6 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1 defaults
libffi 3.4.4 h6a678d5_0 defaults
libgcc-ng 11.2.0 h1234567_1 defaults
libgomp 11.2.0 h1234567_1 defaults
libstdcxx-ng 11.2.0 h1234567_1 defaults
markdown 3.4.3 pypi_0 pypi
markupsafe 2.1.2 pypi_0 pypi
matplotlib 3.5.3 pypi_0 pypi
multidict 6.0.4 pypi_0 pypi
ncurses 6.4 h6a678d5_0 defaults
networkx 2.6.3 pypi_0 pypi
numexpr 2.8.4 pypi_0 pypi
numpy 1.21.6 pypi_0 pypi
oauthlib 3.2.2 pypi_0 pypi
omegaconf 2.3.0 pypi_0 pypi
opencv-python 4.7.0.72 pypi_0 pypi
opencv-python-headless 4.7.0.72 pypi_0 pypi
openssl 1.1.1t h7f8727e_0 defaults
packaging 23.1 pypi_0 pypi
pathtools 0.1.2 pypi_0 pypi
pillow 9.5.0 pypi_0 pypi
pip 22.3.1 py37h06a4308_0 defaults
protobuf 3.20.3 pypi_0 pypi
psutil 5.9.5 pypi_0 pypi
pyasn1 0.5.0 pypi_0 pypi
pyasn1-modules 0.3.0 pypi_0 pypi
pydeprecate 0.3.2 pypi_0 pypi
pyparsing 3.0.9 pypi_0 pypi
python 3.7.16 h7a1cb2a_0 defaults
python-dateutil 2.8.2 pypi_0 pypi
pytorch-lightning 1.6.0 pypi_0 pypi
pywavelets 1.3.0 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
qudida 0.0.4 pypi_0 pypi
readline 8.2 h5eee18b_0 defaults
requests 2.31.0 pypi_0 pypi
requests-oauthlib 1.3.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
scikit-image 0.19.3 pypi_0 pypi
scikit-learn 1.0.2 pypi_0 pypi
scipy 1.7.3 pypi_0 pypi
sentry-sdk 1.24.0 pypi_0 pypi
setproctitle 1.3.2 pypi_0 pypi
setuptools 65.6.3 py37h06a4308_0 defaults
shutup 0.2.0 pypi_0 pypi
six 1.16.0 pypi_0 pypi
smmap 5.0.0 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0 defaults
tables 3.7.0 pypi_0 pypi
tensorboard 2.11.2 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
threadpoolctl 3.1.0 pypi_0 pypi
tifffile 2021.11.2 pypi_0 pypi
tk 8.6.12 h1ccaba5_0 defaults
torch 1.11.0+cu113 pypi_0 pypi
torchmetrics 0.11.4 pypi_0 pypi
tqdm 4.65.0 pypi_0 pypi
typing-extensions 4.6.1 pypi_0 pypi
urllib3 1.26.16 pypi_0 pypi
wandb 0.15.3 pypi_0 pypi
werkzeug 2.2.3 pypi_0 pypi
wheel 0.38.4 py37h06a4308_0 defaults
xz 5.4.2 h5eee18b_0 defaults
yarl 1.9.2 pypi_0 pypi
zipp 3.15.0 pypi_0 pypi
zlib 1.2.13 h5eee18b_0 defaults
I also tried with different lightning and torch versions, however the issue still occurs, but at other iterations during training.
Are you aware of the issue? Are there fixes for it.
Could you please help? Thanks!
where is the .pth of the superglue,the originnal superglue .pth can't fit your code ,thank you
i can't find the .pth '/home/ostap/projects/superglue-lightning/logs/pretrain_superpoint_magicleap/Pretraining_SuperPointNet__attn_softmax__laf_none__2022-03-07-20-34-43/superglue_sp_magicleap.pth'
The function draw_LAF_matches() not defined
Hello,
First of all, thank you for sharing your work!
Unfortunately, when trying to use inference.py, the function draw_LAF_matches() doesn't seem to be defined anywhere.
Could you provide it please?
How to generate the paris.txt
I want to train the superglue on my custom dataset, I already have the picture but I have no idea about how to generate the paris.txt in "sparse-txt" folder. could someone show me the code about generate this? much appreciate!
another work of training your worn graph matcher
We also release the code of our CVPR work 'IMP: Iterative Matching and Pose Estimation with Adaptive Pooling' with training and testing code available at https://github.com/feixue94/imp-release.
The weight of the SuperPoint
the superpoint weight you provided is not match with the name in both superpoint_coco.yaml and superpoint_kitti.yaml. And use the weight you provided for superpoint will raise the keyerror as follow:
File "d:\Develop\PythonProject\OpenGlue-main\models\features\superpoint\model.py", line 175, in _load_weights
state_dict = torch.load(str(weights), map_location='cpu')['model_state_dict']
KeyError: 'model_state_dict'
I can not train superpoint right now so could you please provide the weight for superpoint-coco or superpoint-kitti?
The first line directory in the screenshot is the default checkpoint path and the second line is the checkpoint path direct to the weight that you give. and it will cause the Keyerror.
AUC is always 0 during trainning
I tried to train a matcher for d2net using cached its features, but the AUC is always 0.
Could you please help? Thanks!
Where is "Image-matching.ipynb"?
As the title, I can not find the .ipynb file.
Would you please upload it?
Best regards
/examples folder missing
Hello. First of all, congrats for the nice work, and thank you very much for making the source code and pretrained models available to all. Apparently, the /examples folder which is referenced in the README is missing. Could you please check it?
Training on custom dataset
What are some things to consider when training on my own dataset ?
Not working with SIFT instead of OPENCV_SIFT
Hi,
I would like to use models.features.sift.SIFT instead of OPENCV_SIFTas a feature extractor but the superglue matcher doesn't work with this configuration. Even if the model is trained using the features from OPENCV_SIFT, shoudn't the two versions of sift compute the same features given the same parameters ? Here are the configurations i used :
features:
name: 'SIFT'
descriptor_dim: 128
parameters:
max_keypoints: 2048
nms_diameter: 9
rootsift: True
descriptor_dim: 128
With SIFT no matches are found.
problem
Hello, I have encountered some unknown errors in the process of reproduction. I can't solve them personally. I want to ask you about the problem. The command I entered is as follows.
python inference. py --image0_ path '/home/ch/OpenGlue-main1/OpenGlue-main/freiburg/1341847980.722988.png' --image1_ path '/home/ch/OpenGlue-main1/OpenGlue-main/freiburg/1341847981.726650.png' --experiment_ path '/home/ch/OpenGlue-main1/OpenGlue-main/freiburg' --checkpoint_ name '/home/ch/OpenGlue-main1/OpenGlue-main/freiburg/openglue-SIFT-rotation. ckpt' --output_ dir 'results.png'
The error message displayed is as follows:
File "inference.py", line 99
def __ init__ (self, local_feature: nn.Module, matcher: nn.Module, match_config: Dict = {}) -> None:
^
SyntaxError: invalid syntax
Missing Checkpoints
Hi,
Thanks for the great implementation. I was wondering when the rest of the checkpoints were going to be released.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.