Code for the ECCV 2020 paper: Detail Preserved Point Cloud Completion via Separated Feature Aggregation

Python 48.92% Makefile 0.42% C++ 27.14% Cuda 21.81% Shell 1.70%

detail-preserved-point-cloud-completion-via-sfa's Introduction

Detail Preserved Point Cloud Completion via Separated Feature Aggregation

Detail Preserved Point Cloud Completion via Separated Feature Aggregation ECCV 2020

Wenxiao Zhang, Qingan Yan, Chunxia Xiao

Wuhan University; JD.com American Technologies Corporation, CA

Update 2022/6/13

To test and generate the submission.zip file for completion3d leaderboard, run eval_completion3d.py. We follow the way of GRNet which directly use the pre-trained model on PCN dataset and downsample the results to 2048 points. You should also uncomment the L209 in rfa.py to perform the downsample.

Introduction

We propose a point cloud completion framework which leverages separated feature aggregation to reconstruct the known and the missing part separately. This work is based on PU-Net and PCN.

1) Pre-requisites

Python3
CUDA
tensorflow
tensorpack
matplotlib
lmdb
msgpack
pyarrow
open3d-python

This code is built using Tensorflow 1.14 with CUDA 10.0 and tested on Ubuntu 18.04 with Python 3.6.

2)Compile Customized TF Operators

The TF operators are included under tf_ops and pc_distance, you need to compile them. Check the makefile under pc_distance folder and compile.sh under tf_ops. (You need to check the the CUDA path and tensorflow path in each tf_xxx_compile_abs.sh under each tf_ops subfolder) first. Refer to Pointnet++ for more details.

Important: We modified the original source code gather_point function in tf_sampling.cpp and tf_sampling_g.cu to fit our needs. If you have already complied the sampling operator in other project, you need to recompile it.

3)Download pre-trained models

Download pre-trained models on trained_models folder from Google Drive and put them on data/trianed_models dir. The pre-trained models consist networks of RFA and GLFA.

4) Testing

Download ShapeNet test data on Google Drive. Specifically, this experiment requires test.lmdb and test_novel.lmdb. Put them on data/shapenet folder.
Run python3 eval.py. Use --model_type option to choose different model architectures. Use --save_path option to choose the folder to save the qualitative results. If you just want to see the quantitative results, you can set --plot to False for fast evaluation.

Note that we use the same testing data in PCN project but we create the lmdb file for faster evaluation. You can also use the original pcd file provided on PCN repository.

5) Traning

The training data are from PCN repository, you can download training (train.lmdb, train.lmdb-lock) and validation (valid.lmdb, valid.lmdb-lock) data from shapenet or shapenet_car directory on the provided training set link in PCN repository. Note that the training data for all 8 categories in shapenet takes up 49G of disk space. The training data for only the car category takes 9G instead.
Run python3 train.py. Use --model_type option to choose different model architectures. Type python3 train.py -h for more options.

detail-preserved-point-cloud-completion-via-sfa's People

Contributors

Stargazers

Watchers

Forkers

yanqingan sxy7147 risteon juanse-12138 one-of-student

detail-preserved-point-cloud-completion-via-sfa's Issues

Question for code

Hi, i have a question when i run python eval.py --model_type rfa

the log is:
`*** Received signal 11 ***
*** BEGIN MANGLED STACK TRACE ***
/home/tangjunshu/.virtualenvs/sfa/lib/python3.6/site-packages/tensorflow/python/../libtensorflow_framework.so.1(+0xfed8db)[0x7fd5650aa8db]
/lib/x86_64-linux-gnu/libc.so.6(+0x3f040)[0x7fd5e06d3040]
/home/tangjunshu/pc_proj/Detail-Preserved-Point-Cloud-Completion-via-SFA/tf_ops/sampling/tf_sampling_so.so(+0x5db7)[0x7fd553730db7]
/home/tangjunshu/pc_proj/Detail-Preserved-Point-Cloud-Completion-via-SFA/tf_ops/sampling/tf_sampling_so.so(ZNSt17_Function_handlerIFN10tensorflow6StatusEPNS0_15shape_inference16InferenceContextEEPS5_E9_M_invokeERKSt9_Any_dataOS4+0x20)[0x7fd553731810]
/home/tangjunshu/.virtualenvs/sfa/lib/python3.6/site-packages/tensorflow/python/../libtensorflow_framework.so.1(_ZN10tensorflow15shape_inference16InferenceContext3RunERKSt8functionIFNS_6StatusEPS1_EE+0x4d)[0x7fd564d9fa9d]
/home/tangjunshu/.virtualenvs/sfa/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow12ShapeRefiner10RunShapeFnEPKNS_4NodeEPKNS_18OpRegistrationDataEPNS_24ExtendedInferenceContextE+0x230)[0x7fd56e0143b0]
/home/tangjunshu/.virtualenvs/sfa/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow12ShapeRefiner7AddNodeEPKNS_4NodeE+0xcb8)[0x7fd56e015ed8]
/home/tangjunshu/.virtualenvs/sfa/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(TF_FinishOperation+0x44a)[0x7fd56ae637ba]
/home/tangjunshu/.virtualenvs/sfa/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(+0x29917d6)[0x7fd5686d67d6]
python[0x50a22f]
python(_PyEval_EvalFrameDefault+0x444)[0x50bf44]
python[0x5096c8]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x444)[0x50bf44]
python[0x507cd4]
python(_PyFunction_FastCallDict+0x2e2)[0x508fb2]
python[0x595221]
python[0x549b3f]
python[0x551391]
python(_PyObject_FastCallKeywords+0x19c)[0x5a9d2c]
python[0x50a533]
python(_PyEval_EvalFrameDefault+0x1215)[0x50cd15]
python[0x507cd4]
python[0x5893da]
python(PyObject_Call+0x3e)[0x5a058e]
python(_PyEval_EvalFrameDefault+0x19ff)[0x50d4ff]
python[0x507cd4]
python[0x509a00]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x1215)[0x50cd15]
python[0x507cd4]
python[0x509a00]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x1215)[0x50cd15]
python[0x507cd4]
python[0x509a00]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x444)[0x50bf44]
python[0x5096c8]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x444)[0x50bf44]
python[0x507cd4]
python[0x509a00]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x444)[0x50bf44]
python[0x507cd4]
python[0x509a00]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x1215)[0x50cd15]
python[0x5096c8]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x444)[0x50bf44]
python(_PyFunction_FastCallDict+0xf5)[0x508dc5]
python[0x595221]
python[0x549b3f]
python[0x551391]
python(_PyObject_FastCallKeywords+0x19c)[0x5a9d2c]
python[0x50a533]
python(_PyEval_EvalFrameDefault+0x444)[0x50bf44]
python[0x5096c8]
python[0x50a3fd]
python(_PyEval_EvalFrameDefault+0x444)[0x50bf44]
python[0x507cd4]
python(PyEval_EvalCode+0x23)[0x50ae13]
python[0x635262]
python(PyRun_FileExFlags+0x97)[0x635317]
python(PyRun_SimpleFileExFlags+0x17f)[0x638acf]
python(Py_Main+0x591)[0x639671]
python(main+0xe0)[0x4b0e40]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7fd5e06b5bf7]
python(_start+0x2a)[0x5b2f0a]
*** END MANGLED STACK TRACE ***

*** Begin stack trace ***
tensorflow::CurrentStackTrace()

std::_Function_handler<tensorflow::Status (tensorflow::shape_inference::InferenceContext*), tensorflow::Status (*)(tensorflow::shape_inference::InferenceContext*)>::_M_invoke(std::_Any_data const&, tensorflow::shape_inference::InferenceContext*&&)
tensorflow::shape_inference::InferenceContext::Run(std::function<tensorflow::Status (tensorflow::shape_inference::InferenceContext*)> const&)
tensorflow::ShapeRefiner::RunShapeFn(tensorflow::Node const*, tensorflow::OpRegistrationData const*, tensorflow::ExtendedInferenceContext*)
tensorflow::ShapeRefiner::AddNode(tensorflow::Node const*)
TF_FinishOperation


_PyEval_EvalFrameDefault


_PyEval_EvalFrameDefault

_PyFunction_FastCallDict



_PyObject_FastCallKeywords

_PyEval_EvalFrameDefault


PyObject_Call
_PyEval_EvalFrameDefault



_PyEval_EvalFrameDefault



_PyEval_EvalFrameDefault



_PyEval_EvalFrameDefault


_PyEval_EvalFrameDefault



_PyEval_EvalFrameDefault



_PyEval_EvalFrameDefault


_PyEval_EvalFrameDefault
_PyFunction_FastCallDict



_PyObject_FastCallKeywords

_PyEval_EvalFrameDefault


_PyEval_EvalFrameDefault

PyEval_EvalCode

PyRun_FileExFlags
PyRun_SimpleFileExFlags
Py_Main
main
__libc_start_main
_start

*** End stack trace ***
Aborted (core dumped)`

Could you please tell me how to fix it?

A question about the accuracy reported

The reported numbers are Chamfer Distance multiplied by 10^4.
However, compared to the numbers in the original PCN paper, say 0.009720 for FC, I think it should be Chamfer Distance multiplied by 10^3.
Did I miss anything? Or the metric is different from PCN?
Thanks for replying!

code

hello, I have the following question when I load the trained model and run the eval.py.

could help me , thanks

Problem during initialization in loading data

Traceback (most recent call last):
File "eval.py", line 109, in
train(args)
File "eval.py", line 33, in train
provider = TrainProvider(args, is_training_pl)
File "eval.py", line 17, in init
args.num_input_points, args.num_gt_points, is_training=False)
File "/all-data/linus-disk1/gongjingyu_home/gcode/others/Detail-Preserved-Point-Cloud-Completion-via-SFA/data_util.py", line 34, in lmdb_dataflow
df = dataflow.LMDBSerializer.load(lmdb_path, shuffle=False)
File "/all-data/linus-disk1/gongjingyu_home/.virtualenvs/SFA/lib/python3.6/site-packages/tensorpack/dataflow/serialize.py", line 114, in load
df = LMDBData(path, shuffle=shuffle)
File "/all-data/linus-disk1/gongjingyu_home/.virtualenvs/SFA/lib/python3.6/site-packages/tensorpack/dataflow/format.py", line 90, in init
self._set_keys(keys)
File "/all-data/linus-disk1/gongjingyu_home/.virtualenvs/SFA/lib/python3.6/site-packages/tensorpack/dataflow/format.py", line 110, in _set_keys
self.keys = loads(self.keys)
File "/all-data/linus-disk1/gongjingyu_home/.virtualenvs/SFA/lib/python3.6/site-packages/tensorpack/utils/serialize.py", line 93, in loads
return pickle.loads(buf)
_pickle.UnpicklingError: invalid load key, '\xdc'.

I think the question maybe because of the lmdb data or the version of tensorpack, can you give some suggestions?

Could you give a license to your codes?

As in the title, could you give a license (e.g MIT license) to your source codes?
Thank you.

Question about pretrained models

Hello! We try to use pretrained models given on google drive to evaluate the method. It works well on given test data. However, we can only get following results:

I think something may be wrong but we do not change any of the code. Could you help us?

xlechter / detail-preserved-point-cloud-completion-via-sfa Goto Github PK