Skip to content

Commit 3f20101

Browse files
qinzheng93qinzheng93
committed
code release
1 parent 307b419 commit 3f20101

File tree

15 files changed

+1147
-5
lines changed

15 files changed

+1147
-5
lines changed

‎README.md

Lines changed: 116 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,126 @@
1-
# Deep Graph-based Spatial Consistency for Robust Non-rigid Point Cloud Registration
1+
# Deep Graph-Based Spatial Consistency for Robust Non-Rigid Point Cloud Registration
22

33
PyTorch implementation of the paper:
44

5-
Deep Graph-based Spatial Consistency for Robust Non-rigid Point Cloud Registration.
5+
[Deep Graph-Based Spatial Consistency for Robust Non-Rigid Point Cloud Registration](http://arxiv.org/abs/2303.09950)
66

7-
[Zheng Qin](https://scholar.google.com/citations?user=DnHBAN0AAAAJ), [Hao Yu](https://scholar.google.com/citations?user=g7JfRn4AAAAJ), Changjian Wang, Yuxing Peng, and [Kai Xu](https://scholar.google.com/citations?user=GuVkg-8AAAAJ).
7+
[Zheng Qin](https://scholar.google.com/citations?user=DnHBAN0AAAAJ), [Hao Yu](https://scholar.google.com/citations?user=g7JfRn4AAAAJ),
8+
Changjian Wang, Yuxing Peng, and [Kai Xu](https://scholar.google.com/citations?user=GuVkg-8AAAAJ).
89

910
## Introduction
1011

11-
We study the problem of outlier correspondence pruning for non-rigid point cloud registration. In rigid registration, spatial consistency has been a commonly used criterion to discriminate outliers from inliers. It measures the compatibility of two correspondences by the discrepancy between the respective distances in two point clouds. However, spatial consistency no longer holds in non-rigid cases and outlier rejection for non-rigid registration has not been well studied. In this work, we propose Graph-based Spatial Consistency Network (GraphSCNet) to filter outliers for non-rigid registration. Our method is based on the fact that non-rigid deformations are usually locally rigid, or local shape preserving. We first design a local spatial consistency measure over the deformation graph of the point cloud, which evaluates the spatial compatibility only between the correspondences in the vicinity of a graph node. An attention-based non-rigid correspondence embedding module is then devised to learn a robust representation of non-rigid correspondences from local spatial consistency. Despite its simplicity, GraphSCNet effectively improves the quality of the putative correspondences and attains state-of-the-art performance on three challenging benchmarks.
12+
We study the problem of outlier correspondence pruning for non-rigid point cloud registration. In rigid registration,
13+
spatial consistency has been a commonly used criterion to discriminate outliers from inliers. It measures the
14+
compatibility of two correspondences by the discrepancy between the respective distances in two point clouds. However,
15+
spatial consistency no longer holds in non-rigid cases and outlier rejection for non-rigid registration has not been
16+
well studied. In this work, we propose Graph-based Spatial Consistency Network (GraphSCNet) to filter outliers for
17+
non-rigid registration. Our method is based on the fact that non-rigid deformations are usually locally rigid, or local
18+
shape preserving. We first design a local spatial consistency measure over the deformation graph of the point cloud,
19+
which evaluates the spatial compatibility only between the correspondences in the vicinity of a graph node. An
20+
attention-based non-rigid correspondence embedding module is then devised to learn a robust representation of non-rigid
21+
correspondences from local spatial consistency. Despite its simplicity, GraphSCNet effectively improves the quality of
22+
the putative correspondences and attains state-of-the-art performance on three challenging benchmarks.
23+
24+
![](assets/teaser.png)
1225

1326
## News
1427

15-
2023.02.28: This work is accepted by CVPR 2023. Code and models will be released soon.
28+
2023.06.15: Code and models on 4DMatch released.
29+
30+
2023.02.28: This work is accepted by CVPR 2023.
31+
32+
## Installation
33+
34+
Please use the following command for installation:
35+
36+
```bash
37+
# 1. It is recommended to create a new environment
38+
conda create -n geotransformer python==3.8
39+
conda activate geotransformer
40+
41+
# 2. Install vision3d following https://github.com/qinzheng93/vision3d
42+
```
43+
44+
The code has been tested on Python 3.8, PyTorch 1.13.1, Ubuntu 22.04, GCC 11.3 and CUDA 11.7, but it should work with
45+
other configurations.
46+
47+
## 4DMatch
48+
49+
### Data preparation
50+
51+
The 4DMatch dataset can be downloaded from [DeformationPyramid](https://github.com/rabbityl/DeformationPyramid). We
52+
provide the correspondences extracted by GeoTransformer in the release page. The data should be organized as follows:
53+
54+
```text
55+
--data--4DMatch--train--abe_CoverToStand
56+
| |--...
57+
|--val--amy_Situps
58+
| |--...
59+
|--4DMatch-F--AJ_SoccerPass
60+
| |--...
61+
|--4DLoMatch-F--AJ_SoccerPass
62+
| |--...
63+
|--correspondences--val--amy_Situps
64+
| |--...
65+
|--4DMatch-F--AJ_SoccerPass
66+
| |--...
67+
|--4DLoMatch-F--AJ_SoccerPass
68+
|--...
69+
```
70+
71+
### Training
72+
73+
The code for 4DMatch is in `experiments/graphscnet.4dmatch.geotransformer`. Use the following command for training.
74+
75+
```bash
76+
CUDA_VISIBLE_DEVICES=0 python trainval.py
77+
```
78+
79+
### Testing
80+
81+
Use the following command for testing.
82+
83+
```bash
84+
# 4DMatch
85+
CUDA_VISIBLE_DEVICES=0 python test.py --test_epoch=EPOCH --benchmark=4DMatch-F
86+
# 4DLoMatch
87+
CUDA_VISIBLE_DEVICES=0 python test.py --test_epoch=EPOCH --benchmark=4DLoMatch-F
88+
```
89+
90+
`EPOCH` is the epoch id.
91+
92+
We also provide pretrained weights in `weights`, use the following command to test the pretrained weights.
93+
94+
```bash
95+
CUDA_VISIBLE_DEVICES=0 python test.py --checkpoint=/path/to/GraphSCNet/weights/graphscnet.pth --benchmark=4DMatch-F
96+
```
97+
98+
Replace `4DMatch` with `4DLoMatch` to evaluate on 4DLoMatch.
99+
100+
### Results
101+
102+
| Benchmark | Prec | Recall | EPE | AccS | AccR | OR |
103+
|:----------|:----:|:------:|:-----:|:----:|:----:|:----:|
104+
| 4DMatch | 92.2 | 96.9 | 0.043 | 72.3 | 84.4 | 9.4 |
105+
| 4DLoMatch | 82.6 | 86.8 | 0.121 | 41.0 | 58.3 | 21.0 |
106+
107+
## Citation
108+
109+
```bibtex
110+
@inproceedings{qin2023deep,
111+
title={Deep Graph-Based Spatial Consistency for Robust Non-Rigid Point Cloud Registration},
112+
author={Zheng Qin and Hao Yu and Changjian Wang and Yuxing Peng and Kai Xu},
113+
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
114+
month={June},
115+
year={2023},
116+
pages={5394-5403}
117+
}
118+
```
119+
120+
## Acknowledgements
121+
122+
- [vision3d](https://github.com/qinzheng93/vision3d)
123+
- [GeoTransformer](https://github.com/qinzheng93/GeoTransformer)
124+
- [PointDSC](https://github.com/XuyangBai/PointDSC)
125+
- [lepard](https://github.com/rabbityl/lepard)
126+
- [DeformationPyramid](https://github.com/rabbityl/DeformationPyramid)

‎assets/teaser.png

489 KB
Loading

‎experiments/graphscnet.4dmatch.geotransformer/__pycache__/config.cpython-38.pyc

2.85 KB
Binary file not shown.

‎experiments/graphscnet.4dmatch.geotransformer/__pycache__/dataset.cpython-38.pyc

5.07 KB
Binary file not shown.

‎experiments/graphscnet.4dmatch.geotransformer/__pycache__/graphsc.cpython-38.pyc

3.63 KB
Binary file not shown.

‎experiments/graphscnet.4dmatch.geotransformer/__pycache__/loss.cpython-38.pyc

3.45 KB
Binary file not shown.

‎experiments/graphscnet.4dmatch.geotransformer/__pycache__/model.cpython-38.pyc

4.49 KB
Binary file not shown.

‎experiments/graphscnet.4dmatch.geotransformer/config.py

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
import argparse
2+
import os
3+
import os.path as osp
4+
5+
from easydict import EasyDict as edict
6+
7+
from vision3d.utils.io import ensure_dir
8+
9+
_C = edict()
10+
11+
# exp
12+
_C.exp = edict()
13+
_C.exp.name = osp.basename(osp.dirname(osp.realpath(__file__)))
14+
_C.exp.working_dir = osp.dirname(osp.realpath(__file__))
15+
_C.exp.output_dir = osp.join("..", "..", "outputs", _C.exp.name)
16+
_C.exp.checkpoint_dir = osp.join(_C.exp.output_dir, "checkpoints")
17+
_C.exp.log_dir = osp.join(_C.exp.output_dir, "logs")
18+
_C.exp.event_dir = osp.join(_C.exp.output_dir, "events")
19+
_C.exp.cache_dir = osp.join(_C.exp.output_dir, "cache")
20+
_C.exp.result_dir = osp.join(_C.exp.output_dir, "results")
21+
_C.exp.seed = 7351
22+
23+
ensure_dir(_C.exp.output_dir)
24+
ensure_dir(_C.exp.checkpoint_dir)
25+
ensure_dir(_C.exp.log_dir)
26+
ensure_dir(_C.exp.event_dir)
27+
ensure_dir(_C.exp.cache_dir)
28+
ensure_dir(_C.exp.result_dir)
29+
30+
# data
31+
_C.data = edict()
32+
_C.data.dataset_dir = "../../data/4DMatch"
33+
34+
# train data
35+
_C.train = edict()
36+
_C.train.batch_size = 1
37+
_C.train.num_workers = 8
38+
_C.train.use_augmentation = True
39+
_C.train.return_corr_indices = True
40+
41+
# test data
42+
_C.test = edict()
43+
_C.test.batch_size = 1
44+
_C.test.num_workers = 8
45+
_C.test.return_corr_indices = True
46+
_C.test.shape_names = None
47+
48+
# evaluation
49+
_C.eval = edict()
50+
_C.eval.acceptance_score = 0.4
51+
_C.eval.acceptance_radius = 0.04
52+
_C.eval.distance_limit = 0.1
53+
54+
# trainer
55+
_C.trainer = edict()
56+
_C.trainer.max_epoch = 40
57+
_C.trainer.grad_acc_steps = 1
58+
59+
# optimizer
60+
_C.optimizer = edict()
61+
_C.optimizer.type = "Adam"
62+
_C.optimizer.lr = 1e-4
63+
_C.optimizer.weight_decay = 1e-6
64+
65+
# scheduler
66+
_C.scheduler = edict()
67+
_C.scheduler.type = "Step"
68+
_C.scheduler.gamma = 0.95
69+
_C.scheduler.step_size = 1
70+
71+
# model - Global
72+
_C.model = edict()
73+
_C.model.min_local_correspondences = 3
74+
_C.model.max_local_correspondences = 128
75+
76+
_C.model.deformation_graph = edict()
77+
_C.model.deformation_graph.num_anchors = 6
78+
_C.model.deformation_graph.node_coverage = 0.08
79+
80+
# model - transformer
81+
_C.model.transformer = edict()
82+
_C.model.transformer.input_dim = 6
83+
_C.model.transformer.hidden_dim = 256
84+
_C.model.transformer.output_dim = 256
85+
_C.model.transformer.num_heads = 4
86+
_C.model.transformer.num_blocks = 3
87+
_C.model.transformer.num_layers_per_block = 2
88+
_C.model.transformer.sigma_d = 0.08
89+
_C.model.transformer.dropout = None
90+
_C.model.transformer.activation_fn = "ReLU"
91+
_C.model.transformer.embedding_k = -1
92+
_C.model.transformer.embedding_dim = 1
93+
94+
# model - classifier
95+
_C.model.classifier = edict()
96+
_C.model.classifier.input_dim = 256
97+
_C.model.classifier.dropout = None
98+
99+
# Non-rigid ICP
100+
_C.model.nicp = edict()
101+
_C.model.nicp.corr_lambda = 5.0
102+
_C.model.nicp.arap_lambda = 1.0
103+
_C.model.nicp.lm_lambda = 0.01
104+
_C.model.nicp.num_iterations = 5
105+
106+
# loss
107+
_C.loss = edict()
108+
_C.loss.focal_loss = edict()
109+
_C.loss.focal_loss.weight = 1.0
110+
_C.loss.consistency_loss = edict()
111+
_C.loss.consistency_loss.weight = 1.0
112+
113+
114+
def make_cfg():
115+
return _C
116+
117+
118+
def parse_args():
119+
parser = argparse.ArgumentParser()
120+
parser.add_argument(
121+
"--link_output", dest="link_output", action="store_true", help="link output dir"
122+
)
123+
args = parser.parse_args()
124+
return args
125+
126+
127+
def main():
128+
cfg = make_cfg()
129+
args = parse_args()
130+
if args.link_output:
131+
os.symlink(cfg.output_dir, "output")
132+
133+
134+
if __name__ == "__main__":
135+
main()

0 commit comments

Comments
 (0)