whu-usi3dv / sghr Goto Github PK

您好，使用您提供的权重对3dmatch数据集进行test时，每次得到的RR数值变化非常大，从82%到95%不等，请问是哪里参数需要额外设置吗。



万分感激！！

Hi

I am trying to go through the testing step with my own data, when I run the "test.py", I got the following error, could you please provide some hint what caused this issue? thank you so much

Hello author

I would like to ask, how can I obtain the required key point text file when I only have point cloud files?

Hi, thank you for your great work, I am trying to reproduce your result on my local machine, but why the training take so long, does it even use GPU? How can I speed up the training if possible, Thank you

Hello, thanks for your work!

I am trying out your code and so far I've compiled the CUDA based KNN searcher (modified the code to work for cuda11.7).
When trying to proceed to the next steps, I am unable to find knn_search directory for the code below.

cd knn_search/
export CUDA_HOME=/usr/local/cuda-11.1 #We have checked cuda-11.1.
python setup.py build_ext --inplace
cd ..

Could you help specify how I can proceed or modify the commands to proceed?
Thank you!

Hello, I would like to use your model to train on my own dataset. My dataset consists of point clouds from different viewpoints of a statue. However, I do not have the corresponding rotation and translation matrices between the point clouds pairwise. How can I address this issue? Thank you!

Thank you for your work，Why is the GPU occupancy rate during training very low, only 10%？

Hello, thank your for your work!

I would like to test your model on my own dataset. I found this explanation in YOHO repository, but didn't understand some parts:

So, in my case, I acquired point clouds using 3D scanner. Therefore, I only have point clouds such as cloud_bin_0.ply, cloud_bin_1.ply, cloud_bin_2.ply, etc and I don't have "gt.log" and "gt.info" files. I want to merge them using your multiview registration method to have a better 3D recontruction.

1. I checked the links for "gt.log" and "gt.info", and as I understood these files should be created during acquisition of point clouds. However, in my case, I am missing these files. Is there a way to obtain these files after aqusition or can I use model without it?
   

2. I am also missing the "cloud_bin_0Keypoints.txt", but as I understood it contains 5000 randomly sampled points from point cloud_bin0.ply. So I can create this file by myself.

I hope you understand my quesitons. Thank you!

Best regards

感谢分享您的工作，
请问下点云下采样的leaf size和voxel size的设置有关系吗，比如为现在有一个近处的40cm左右尺寸的物体，点云下采样的voxel为5mm，那我在demo.py的设置，voxel size还需要考虑物体和3dmatch的场景尺寸之间的映射吗，还需要用0.4/3*0.025吗？还是设置为0.005就可以了。谢谢。

Thanks for sharing your work.
I am confused with the voxel size setting.
If the point cloud is downsampled with leaf size 0.005(m), the object's size in pointcloud is about 0.4m. how can i set the parameter value of voxel size in demo.py. it should be 0.005 or 0.4/3*0.025=0.003. Does it make sense to set a voxel size smaller than pointcloud resolution.

looking forward to your reply.

In the testset, I cannot find the 3DLoMatch dataset. There is only a 3DMatch dataset. Could you upload the dataset to the network disk？

Hi @HpWang-whu ,
Thanks for your great work. I have some confusion while reproducing some results from the paper.

1. In the experiment part, the comparison methods include some rotation averaging methods (e.g. HARA). I wonder how to calculate absolute t for these methods. Does it use the same least squares translation synchronization as the SGHR? If YES, how to determine the weight w for these rotation averaging methods?
2. For these optimization-based methods, whether the hyperparameters use the original default values or other adjusted values? For example, the RR results of EIGSE3 reported in the paper are 23.2%/60.4% (full/sparse graph with YOHO+3DMatch). However, my reproduced results using the below default EIGSE3 hyperparameters are only 15%/52%.

use_mex=0;               % use_mex = false -> do not use mex function to compute weights
use_sparse=1;            % use_sparse = true -> use sparse Matlab solvers (eigs/svds)
nmax_eig_irls=50;        % nmax = maximum number of iterations of IRLS
thresh_eig_irls=1e-5;    % thresh = thrsehold on relative error to check convergence of IRLS
method="top";            % method = 'top' -> compute top eigenvectors

Any help is appreciated!

Hello！
Your work is excellent, it happens to be the tool I need. I am a graduate student in civil engineering, using 3D scanning to reconstruct building scenes. However, during the scanning process, multiple locally overlapping scenes are usually used for registration. Currently, we still use manual methods for registration, and your work may greatly reduce this workload. I currently have about 10 different local point clouds in the same scene. Can I use your method for registration? How much time will this take approximately? Since I am not familiar with deep learning, could you provide a demo for us to learn？
Thanks！

Hello！
You're doing great work！I tried to use the demo to work on registering a local dataset, I tested it directly using the default parameters, however the results were not ideal. May I know which model parameters or parameters of the local data need to be adjusted? For example, do I need to ensure that the point cloud density is consistent with 3Dmatch? Does the number of point clouds affect the registration time?

Hello！
Your work is excellent and I would like to know how to view the visual results of the point cloud registration, as the code does not seem to provide visualization tools. The code seems to save the global pose in pose.txt, can I read the point cloud file and then read the pose matrix in turn for visualization after running Testcycle.py?Or can you provide a demo to view?
Thanks a lot!

你好，想请教一个问题：在使用IRLS with history reweighting计算统一坐标系时，输入的pairwise registration的变换矩阵T01、T02、T12分别如下所示：
T01 = [[1.0，-1.2149189387855586e-05， -0.00014383767847903073，-11.460397720336914],
[1.2157798664702568e-05, 1.0, 5.9850561228813604e-05, -1.2087631225585938],
[0.0001438369508832693, -5.9852311096619815e-05, 1.0, -0.43892526626586914],
[0.0, 0.0, 0.0, 1.0]];

T02 = [[1.0，-5.033568413637113e-06， 9.842186409514397e-05，-1.8885232210159302],
[5.042900284024654e-06, 1.0, -9.480899461777881e-05, -11.029406547546387],
[-9.842139115789905e-05, 9.480948938289657e-05, 1.0, -12.239741325378418],
[0.0, 0.0, 0.0, 1.0]];

T12 = [[1.0，-5.77977665727758e-11， 2.6372001804553236e-10，9.558798789978027],
[-5.779732248356595e-11, 1.0, 8.342294910423931e-11, -9.82029914855957],
[-2.637197404897762e-10, -8.342281032636123e-11, 1.0, -11.802338600158691],
[0.0, 0.0, 0.0, 1.0]];

这三个中旋转矩阵都近似于单位矩阵，按理说得到的global registration变换矩阵中旋转矩阵应该也是近似单位矩阵的。
但计算得到的global registration的变换矩阵是类似这个数值范围的：
T0 = [[-0.7116015140757603，-0.6879857888161842， 0.14247399605562958，-5.370030503337107],
[-0.10789030996916145, -0.09337589548789123, -0.9897679643010234, -5.043447974465397],
[0.6942499306300612, -0.7196919455772777, -0.007780571402392206, 0.1193426041857415],
[0.0, 0.0, 0.0, 1.0]];
从而导致最后配准后的点云和原始点云位置偏移很大，而且还有很大的旋转。

请问这是什么原因导致的？如何解决呢？十分感谢！

Amazing work, but there are some questions here.
1、What is the role of gt.info and test_3dmatch.pkl?I would like to use pre-trained models for testing, are these files mandatory? In the test flow, I get an error when I run the following command. The pkl file seems to be relevant to the dataset, but I don't know how to build it.
2.Report a bug, even if it's an easy bug to fix.
desc = np.load(f'{args.yoho_desc_dir}/{dataset.name}/YOHO_Output_Group_feature')
->
desc = np.load(f'{args.yoho_desc_dir}/{dataset.name}/YOHO_Output_Group_feature/{pid}.npy')

Thanks for the great work, I have a question for your implementation of translation synchronization.

In your implementation, the translation is computed using:

But the original paper "Learning Transformation Synchronization" computes the translation following :

Because I failed to understand your implementation and the original implementation is more clearer, so following your code, I compute the translation using:

b = np.zeros([3*N])
for pid in range(len(locws)):
    i, j, w = int(locws[pid,0]), int(locws[pid,1]), locws[pid,2]
    xs, xe = i * 3, i * 3 + 3
    b[xs:xe] -= w * Rs[pid] @ (-Rs[pid].T@ts[pid])
tpre = np.linalg.pinv(L) @ b
tpre = tpre.reshape(N, 3)

However, the result tpre does not stay the same with your implementation.

So the two questions are:

Q1. What is the difference between these two implementations?

Q2. Why the results of these two implementations are not the same?

Thank you so much for your help.

Hello，hai ping！
Did you use YOHO to extract features and RANSAC for registration in pairwise registration? Does the RANSAC method increase the time consumption? I tried to use the RANSAC-FREE method, but it requires an initial correspondence. How do I get the initial correspondence? Where should I replace the RANSAC?

Hi @HpWang-whu ,
Thanks for sharing the code of your amazing work!
I have some questions during reading the paper and code.
Q1: The dataset includes 5000 keypoints per point cloud frame; I'm curious how these keypoints were chosen; were they randomly picked or were they chosen using a specific sampling method?
Q2: I noticed that 1200 to 5000 points and descriptions will be selected for data augmentation. Rather than the more common FPS or random uniform sample, the sample method used here is to sample neighbours from a random center point. I'm interested, is there a specific reason for using this strategy or just empirical choice.
Q3: Released code using l2 mode when calculating the overlap from point global features, rather than the inner mode indicated in Eq.3 from main paper. I'm curious whether there's a significant difference in performance between these two modes？

Thanks for releasing your code. I'm wondering how can I generate yoho_desc files with my own point cloud files? It seems the scripts in simple_YOHO do not save anything.

Hi

thank you for your great work, I am trying to implement the test process of SGHR on my own data. However, it happens that building wheel for Minkowski Engine package always fails. Is it possible to skip the Engine and test the model on my data set? Do I have to generate YOHO_desc before testing? Thank you