Why is the name SGHR?

😍 SGHR

Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting

CVPR 2023

Haiping Wang^*,1, Yuan Liu^*,2, Zhen Dong^†,1, Yulan Guo³, Yu-Shen Liu⁴, Wenping Wang⁵ Bisheng Yang^†,1

¹Wuhan University    ²The University of Hong Kong    ³Sun Yat-sen University
⁴Tsinghua University    ⁵Texas A&M University
^*The first two authors contribute equally.    ^†Corresponding authors.

In this paper, we present a new method for the multiview registration of point cloud. Previous multiview registration methods rely on exhaustive pairwise registration to construct a densely-connected pose graph and apply Iteratively Reweighted Least Square (IRLS) on the pose graph to compute the scan poses. However, constructing a densely-connected graph is time-consuming and contains lots of outlier edges, which makes the subsequent IRLS struggle to find correct poses. To address the above problems, we first propose to use a neural network to estimate the overlap between scan pairs, which enables us to construct a sparse but reliable pose graph. Then, we design a novel history reweighting function in the IRLS scheme, which has strong robustness to outlier edges on the graph. In comparison with existing multiview registration methods, our method achieves $11$% higher registration recall on the 3DMatch dataset and $\sim13$% lower registration errors on the ScanNet dataset while reducing $\sim70$% required pairwise registrations. Comprehensive ablation studies are conducted to demonstrate the effectiveness of our designs.

| Paper | Poster | Video |

🆕 News

2023-05-13: An introduction video of SGHR on YouTube.
2023-04-04: Release SGHR on Arxiv.
2023-04-01: The code of SGHR is released.
2023-02-28: SGHR is accepted by CVPR 2023! 🎉🎉

✨ Pipeline

💻 Requirements

Here we offer the YOHO backbone SGHR. Thus YOHO requirements need to be met:

Ubuntu 14.04 or higher
CUDA 11.1 or higher
Python v3.7 or higher
Pytorch v1.6 or higher

Specifically, The code has been tested with:

Ubuntu 16.04, CUDA 11.1, python 3.7.10, Pytorch 1.7.1, GeForce RTX 2080Ti.
Ubuntu 20.04, CUDA 11.1, python 3.7.16, Pytorch 1.10.0, GeForce RTX 4090.

🔧 Installation

First, create the conda environment:

conda create -n sghr python=3.7
conda activate sghr

Second, intall Pytorch. We have checked version 1.7.1 and other versions can be referred to Official Set.
```
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch
```
Third, install other packages, here we use 0.8.0.0 version Open3d for Ubuntu 16.04:
```
pip install -r requirements.txt
```

Finally, compile the CUDA based KNN searcher:

cd knn_search/
export CUDA_HOME=/usr/local/cuda-11.1 #We have checked cuda-11.1.
python setup.py build_ext --inplace
cd ..

💾 Dataset & Pretrained model

The datasets are accessible in BaiduDesk(Code:oouk) and Google Cloud:

Trainset:

3DMatch_train;

Testset:

Datasets above contain the point clouds (.ply), keypoints (.txt, 5000 per point cloud), and rotation-invariant yoho-desc(.npy, extracted on the keypoints) files. Please place the data to ./data following the example data structure as:

data/
├── 3dmatch/
    └── kitchen/
        ├── PointCloud/
            ├── cloud_bin_0.ply
            ├── gt.log
            └── gt.info
        ├── yoho_desc/
            └── 0.npy
        └── Keypoints/
            └── cloud_bin_0Keypoints.txt
├── 3dmatch_train/
├── scannet/
└── ETH/

🚅 Train

You can train SGHR with the 3dmatch_train dataset downloaded above, where we offer the 32-dim rotation-invariant yoho-desc we extracted on 3dmatch_train and you can also extract 32-dim invariant yoho-desc(row-pooling on yoho-desc) yourself and save the features to '''data/3dmatch_train/<scene>/yoho_desc'''. Then, you can train SGHR with the following commond:

python Train.py

✏️ Test

To evalute SGHR on 3DMatch and 3DLoMatch, you can use the following commands:

# extract global features
python Test.py --dataset 3dmatch
# conduct multiview registration
python Test_cycle.py --dataset 3dmatch --rr

To evalute SGHR on ScanNet, you can use the following commands:

python Test.py --dataset scannet
python Test_cycle.py --dataset scannet --ecdf

To evalute SGHR on ETH, you can use the following commands:

python Test.py --dataset ETH
python Test_cycle.py --dataset ETH --topk 6 --inlierd 0.2 --tau_2 0.5 --rr

To evalute SGHR on your own dataset, you can follow here.

💡 Citation

Please consider citing SGHR if this program benefits your project

@inproceedings{
wang2023robust,
title={Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting},
author={Haiping Wang and Yuan Liu and Zhen Dong and Yulan Guo and Yu-Shen Liu and Wenping Wang and Bisheng Yang},
booktitle={Conference on Computer Vision and Pattern Recognition},
year={2023}
}

🔗 Related Projects

Take a look at our previous works on feature extraction and pairwise registration!

	def leastsquare(locws, ts, pcrs, N):
	'''
	base : RiPi+ti = RjPj+tj
	Pi = RijPj+tij
	--> Ri(RijPj+tij)+ti = RjPj+tj
	--> RiRij = Rj Ritij+ti = tj
	--> -ti + tj = Ritij (3 formular)
	least square:
	B: (p3)(N*3)
	P: (p3)(p*3)
	L: Ritij--> (p*3)
	'''
	tpre = np.zeros([N,3])
	p = locws.shape[0]
	B = np.zeros([p3, N3])
	P = np.eye(p*3)
	L = np.zeros([p*3])
	# get B and P
	for pid in range(locws.shape[0]):
	# the pid-th pair
	i, j, w = int(locws[pid,0]), int(locws[pid,1]), locws[pid,2]
	P[pid3:(pid+1)3] *= w
	B[pid3:(pid+1)3, i3:(i+1)3] = -np.eye(3)
	B[pid3:(pid+1)3, j3:(j+1)3] = np.eye(3)
	# get L
	for pid in range(locws.shape[0]):
	# the pid-th pair
	i, j, w = int(locws[pid,0]), int(locws[pid,1]), locws[pid,2]
	L[pid3:(pid+1)3] = pcrs[i]@ts[pid] - (tpre[j]-tpre[i])
	# delta
	deltat = np.linalg.pinv(B.T@P@B)@(B.T@P@L)
	tpre += deltat.reshape(N,3)

	# final T
	Tpre = []
	for i in range(N):
	r = pcrs[i]
	t = tpre[i]
	T = np.eye(4)
	T[0:3,0:3] = r
	T[0:3,-1] = t
	Tpre.append(T[None,:,:])
	Tpre = np.concatenate(Tpre, axis=0)
	return Tpre

	def _f2overlap(f, mode = 'l2'):
	# f: n_scan*c
	# normalization
	f = f/torch.norm(f,dim=1,keepdim=True)
	if mode == 'inner':
	overlap = (f @ f.T + 1)/2.0
	else:
	overlap = (2 - torch.norm(f[:,None,:] - f[None,:,:], dim = -1))/2.0

whu-usi3dv / sghr Goto Github PK

sghr's Introduction

😍 SGHR

CVPR 2023

🆕 News

✨ Pipeline

💻 Requirements

🔧 Installation

💾 Dataset & Pretrained model

🚅 Train

✏️ Test

💡 Citation

🔗 Related Projects

sghr's People

Contributors

Stargazers

Watchers

Forkers

sghr's Issues

Recommend Projects

Recommend Topics

Recommend Org