yuxumin / pointr Goto Github PK

View Code? Open in Web Editor NEW

550.0 13.0 105.0 26.26 MB

[ICCV 2021 Oral] PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

License: MIT License

Python 77.14% Shell 0.21% Cuda 17.04% C++ 2.97% Jupyter Notebook 2.64%

deep-learning computer-vision pointcloud-completion vision-transformers 3dvision iccv2021

pointr's Introduction

PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

Created by Xumin Yu*, Yongming Rao*, Ziyi Wang, Zuyan Liu, Jiwen Lu, Jie Zhou

[arXiv] [Video] [Dataset] [Models] [supp]

This repository contains PyTorch implementation for PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers (ICCV 2021 Oral Presentation).

PoinTr is a transformer-based model for point cloud completion. By representing the point cloud as a set of unordered groups of points with position embeddings, we convert the point cloud to a sequence of point proxies and employ a transformer encoder-decoder architecture for generation. We also propose two more challenging benchmarks ShapeNet-55/34 with more diverse incomplete point clouds that can better reflect the real-world scenarios to promote future research.

🔥News

2023-9-2 AdaPoinTr accepted by T-PAMI, Projected-ShapeNet dataset see here
2023-1-11 Release AdaPoinTr (PoinTr + Adaptive Denoising Queries), achieving SOTA performance on various benchmarks. Arxiv.
2022-06-01 Implement SnowFlakeNet.
2021-10-07 Our solution based on PoinTr wins the Championship on MVP Completion Challenge (ICCV Workshop 2021). The code will come soon.
2021-09-09 Fix a bug in datasets/PCNDataset.py(#27), and update the performance of PoinTr on PCN benchmark (CD from 8.38 to 7.26).

Pretrained Models

We provide pretrained PoinTr models:

dataset	url	performance
ShapeNet-55	[Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:erdh)	CD = 1.09e-3
ShapeNet-34	[Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:atbb )	CD = 2.05e-3
PCN	[Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:9g79)	CD = 8.38e-3
PCN_new	[Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:aru3 )	CD = 7.26e-3
KITTI	[Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:99om)	MMD = 5.04e-4

We provide pretrained AdaPoinTr models (coming soon):

dataset	url	performance
ShapeNet-55	Tsinghua Cloud / Google Drive / BaiDuYun	CD = 0.81e-3
ShapeNet-34	Tsinghua Cloud / Google Drive / BaiDuYun	CD = 1.23e-3
Projected_ShapeNet-55	Tsinghua Cloud / Google Drive / [BaiDuYun](code:dycc)	CD = 9.58e-3
Projected_ShapeNet-34	Tsinghua Cloud / Google Drive / [BaiDuYun](code:dycc)	CD = 9.12e-3
PCN	[Tsinghua Cloud] / [Google Drive] / [BaiDuYun](code:rc7p)	CD = 6.53e-3

Usage

Requirements

PyTorch >= 1.7.0
python >= 3.7
CUDA >= 9.0
GCC >= 4.9
torchvision
timm
open3d
tensorboardX

pip install -r requirements.txt

Building Pytorch Extensions for Chamfer Distance, PointNet++ and kNN

NOTE: PyTorch >= 1.7 and GCC >= 4.9 are required.

# Chamfer Distance
bash install.sh

The solution for a common bug in chamfer distance installation can be found in Issue #6

# PointNet++
pip install "git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
# GPU kNN
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl

Note: If you still get ModuleNotFoundError: No module named 'gridding' or something similar then run these steps

    1. cd into extensions/Module (eg extensions/gridding)
    2. run `python setup.py install`

That will fix the ModuleNotFoundError.

Dataset

The details of our new ShapeNet-55/34 datasets and other existing datasets can be found in DATASET.md.

Inference

To inference sample(s) with pretrained model

python tools/inference.py \
${POINTR_CONFIG_FILE} ${POINTR_CHECKPOINT_FILE} \
[--pc_root <path> or --pc <file>] \
[--save_vis_img] \
[--out_pc_root <dir>] \

For example, inference all samples under demo/ and save the results under inference_result/

python tools/inference.py \
cfgs/PCN_models/AdaPoinTr.yaml ckpts/AdaPoinTr_PCN.pth \
--pc_root demo/ \ 
--save_vis_img  \
--out_pc_root inference_result/ \

Evaluation

To evaluate a pre-trained PoinTr model on the Three Dataset with single GPU, run:

bash ./scripts/test.sh <GPU_IDS>  \
    --ckpts <path> \
    --config <config> \
    --exp_name <name> \
    [--mode <easy/median/hard>]

Some examples:

Test the PoinTr (AdaPoinTr) pretrained model on the PCN benchmark or Projected_ShapeNet:

bash ./scripts/test.sh 0 \
    --ckpts ./pretrained/PoinTr_PCN.pth \
    --config ./cfgs/PCN_models/PoinTr.yaml \
    --exp_name example

bash ./scripts/test.sh 0 \
    --ckpts ./pretrained/PoinTr_ps55.pth \
    --config ./cfgs/Projected_ShapeNet55_models/AdaPoinTr.yaml \
    --exp_name example

Test the PoinTr pretrained model on ShapeNet55 benchmark (easy mode):

bash ./scripts/test.sh 0 \
    --ckpts ./pretrained/PoinTr_ShapeNet55.pth \
    --config ./cfgs/ShapeNet55_models/PoinTr.yaml \
    --mode easy \
    --exp_name example

Test the PoinTr pretrained model on the KITTI benchmark:

bash ./scripts/test.sh 0 \
    --ckpts ./pretrained/PoinTr_KITTI.pth \
    --config ./cfgs/KITTI_models/PoinTr.yaml \
    --exp_name example
CUDA_VISIBLE_DEVICES=0 python KITTI_metric.py \
    --vis <visualization_path>

Training

To train a point cloud completion model from scratch, run:

# Use DistributedDataParallel (DDP)
bash ./scripts/dist_train.sh <NUM_GPU> <port> \
    --config <config> \
    --exp_name <name> \
    [--resume] \
    [--start_ckpts <path>] \
    [--val_freq <int>]
# or just use DataParallel (DP)
bash ./scripts/train.sh <GPUIDS> \
    --config <config> \
    --exp_name <name> \
    [--resume] \
    [--start_ckpts <path>] \
    [--val_freq <int>]

Some examples:

Train a PoinTr model on PCN benchmark with 2 gpus:

CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
    --config ./cfgs/PCN_models/PoinTr.yaml \
    --exp_name example

Resume a checkpoint:

CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
    --config ./cfgs/PCN_models/PoinTr.yaml \
    --exp_name example --resume

Finetune a PoinTr on PCNCars

CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
    --config ./cfgs/KITTI_models/PoinTr.yaml \
    --exp_name example \
    --start_ckpts ./weight.pth

Train a PoinTr model with a single GPU:

bash ./scripts/train.sh 0 \
    --config ./cfgs/KITTI_models/PoinTr.yaml \
    --exp_name example

We also provide the Pytorch implementation of several baseline models including GRNet, PCN, TopNet and FoldingNet. For example, to train a GRNet model on ShapeNet-55, run:

CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
    --config ./cfgs/ShapeNet55_models/GRNet.yaml \
    --exp_name example

Completion Results on ShapeNet55 and KITTI-Cars

License

MIT License

Acknowledgements

Our code is inspired by GRNet and mmdetection3d.

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{yu2021pointr,
  title={PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers},
  author={Yu, Xumin and Rao, Yongming and Wang, Ziyi and Liu, Zuyan and Lu, Jiwen and Zhou, Jie},
  booktitle={ICCV},
  year={2021}
}

pointr's People

Contributors

Stargazers

Watchers

Forkers

johndpope jlqzzz siwang609 4564613 dirkkzzz cc945967091 metavai trendingtechnology cv-ip khuongnd stiphyjay collector-m lavenderla sheldontsui hmmpointcloud peterzhousz sergeyprokudin jumpxing fzh92128 solitude-zz9 kitsunetic sycasia topcomma tuananh1007 b1anchenv jiazewang vishalteli 5l1v3r1 xd-hkpc dinnno son-goku-gpu whuhxb golriz-code ebgu xiaoyanqian aycatakmaz sh4175515 anthonyrathe lan11312 pixas woodnurseryrhyme danhalp benygood pele324 jungseokhong phongnguyenai xuqihong1998 houda-pixel cmput-414 linlianjiang one-of-student seabird-go vivym qixuema blukaz eltociear niluk cuiruikai samdm encrypted-jpg ucprer arashjavan liqingque margaretya 0nedawn begin9 dimentary liangzhendong123 liuxinren456852 sashank-tirumala husnejahan dongjoocha 571061321 keneniwt alpha-davidson aofengbaobao pc-completion albro96 bjy12 ajaymopidevi qian5683 yaoleiqi hahahahaha233 simonschlaepfer curiocortex romantic-black nicolaistein wangyuqing0424 albrnd sharath-golluri97 wangjl1567 raember muzahidai dada308162 winerdecoder taiyehu x-web-7 srinik1 dhananjaysm fjl-projects

pointr's Issues

Question about dataset config and reproducing results

Hi Xumin,

Congrats on the great work and thank you for sharing your work with the community!

I have two questions, could you help me?

under the PoinTr.yaml, why val dataset is using "test" as a subset? (see the red circle in the picture below)

I am trying to reproduce the paper result of the PoinTr model on PCN dataset. I am using a single 3080Ti GPU, and I set the batch size to 26. In addition, I changed the "test" circled in the picture above into "val". Other than that, everything is following the configuration of the repo.

Here is my training curves for about 200 epoches):

Somehow, the validation curves seem to be not stable, is this normal?

Looking forward to your reply, and have a great weekend!

Barry,
Technical University of Munich

Support for CPU

Hi,
Macbook can't use cuda and the chamfer distance requires cuda to install. How can I use a cpu version?
Thank you!

Epoch 9/150 train.sh Line 8 ：30127 killed in training PCN methods（ShapeNet 55）

I'm sorry to bother you. Is there not enough memory? If so, how much memory do we need?

ModuleNotFoundError: No module named 'chamfer'

when I run the command,

bash ./scripts/test.sh 0 \         
    --ckpts ./pretrained/PoinTr_KITTI.pth \
    --config ./cfgs/KITTI_models/PoinTr.yaml \
    --exp_name example

I got this error.

  File "/home/city945/work/PoinTr/PoinTr/models/TopNet.py", line 6, in <module>
    from extensions.chamfer_dist import ChamferDistanceL2
  File "/home/city945/work/PoinTr/PoinTr/extensions/chamfer_dist/__init__.py", line 10, in <module>
    import chamfer
ModuleNotFoundError: No module named 'chamfer'

I think the installation of extensions is okay by run bash install.py, it prints

Extracting chamfer-2.0.0-py3.8-linux-x86_64.egg to /home/city945/work/PoinTr/PoinTr/.local/lib/python3.8/site-packages
chamfer 2.0.0 is already the active version in easy-install.pth

Installed /home/city945/work/PoinTr/PoinTr/.local/lib/python3.8/site-packages/chamfer-2.0.0-py3.8-linux-x86_64.egg
Processing dependencies for chamfer==2.0.0
Finished processing dependencies for chamfer==2.0.0

could you please help me ?

Visualization issue

Could you provide the code of visualization for ShapeNet? Thank you very much.

Question about Eval Chamfer Distance

Hello, when you use the Chamfer Distance to evaluate the model, you multiply its value by 1000. I was wondering what is the reason behind it. I know that Occupancy Network multiplies it by 10 because they are using 1/10 * (maximal edge length) as their unit of distance. Is it something similar?

multiprocessing errors

Hi!
I got an error when training with two GPUs, and the trained model is not very good.can you give me some advice.
I ran the code “CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 --config ./cfgs/PCN_models/PoinTr.yaml --exp_name example”
and got the following error:

Traceback (most recent call last):
File "main.py", line 68, in
main()
File "main.py", line 64, in main
run_net(args, config, train_writer, val_writer)
File "/home/uer/codes/PoinTr/tools/runner.py", line 159, in run_net
train_writer.close()
AttributeError: 'NoneType' object has no attribute 'close'

WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 13571 closing signal SIGTERM
/home/uer/anaconda3/envs/PoinTr/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 6 leaked semaphores to clean up at shutdown
len(cache))
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 1 (pid: 13572) of binary: /home/uer/anaconda3/envs/PoinTr/bin/python
Traceback (most recent call last):
File "/home/uer/anaconda3/envs/PoinTr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/uer/anaconda3/envs/PoinTr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/uer/anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/launch.py", line 193, in
main()
File "/home/uer/anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/home/uer/anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/home/uer/anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/run.py", line 713, in run
)(*cmd_args)
File "/home/uer/anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/uer/anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 261, in launch_agent
failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

main.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2022-02-14_01:37:10
host :uer
rank : 1 (local_rank: 1)
exitcode : 1 (pid: 13572)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

Which configuration parameter to change for saving memory?

Hi! Thank you for amazing your work. I was so impressed.

I've tried to train this network with own my point cloud data based on the PCN dataset.
But I conflicted memory problem.

Could you tell me which parameter to tweak in the settings?

Error message below...

+ GPUS=0
+ PY_ARGS='--config ./cfgs/PCN_models/PoinTr.yaml --exp_name example'
+ CUDA_VISIBLE_DEVICES=0
+ python main.py --config ./cfgs/PCN_models/PoinTr.yaml --exp_name example
2022-04-12 15:27:15,511 - PoinTr - INFO - Copy the Config file from ./cfgs/PCN_models/PoinTr.yaml to ./experiments/PoinTr/PCN_models/example/config.yaml
2022-04-12 15:27:15,511 - PoinTr - INFO - args.config : ./cfgs/PCN_models/PoinTr.yaml
2022-04-12 15:27:15,511 - PoinTr - INFO - args.launcher : none
2022-04-12 15:27:15,511 - PoinTr - INFO - args.local_rank : 0
2022-04-12 15:27:15,511 - PoinTr - INFO - args.num_workers : 4
2022-04-12 15:27:15,511 - PoinTr - INFO - args.seed : 0
2022-04-12 15:27:15,511 - PoinTr - INFO - args.deterministic : False
2022-04-12 15:27:15,511 - PoinTr - INFO - args.sync_bn : False
2022-04-12 15:27:15,511 - PoinTr - INFO - args.exp_name : example
2022-04-12 15:27:15,511 - PoinTr - INFO - args.start_ckpts : None
2022-04-12 15:27:15,511 - PoinTr - INFO - args.ckpts : None
2022-04-12 15:27:15,511 - PoinTr - INFO - args.val_freq : 1
2022-04-12 15:27:15,511 - PoinTr - INFO - args.resume : False
2022-04-12 15:27:15,511 - PoinTr - INFO - args.test : False
2022-04-12 15:27:15,511 - PoinTr - INFO - args.mode : None
2022-04-12 15:27:15,511 - PoinTr - INFO - args.experiment_path : ./experiments/PoinTr/PCN_models/example
2022-04-12 15:27:15,511 - PoinTr - INFO - args.tfboard_path : ./experiments/PoinTr/PCN_models/TFBoard/example
2022-04-12 15:27:15,511 - PoinTr - INFO - args.log_name : PoinTr
2022-04-12 15:27:15,511 - PoinTr - INFO - args.use_gpu : True
2022-04-12 15:27:15,512 - PoinTr - INFO - args.distributed : False
2022-04-12 15:27:15,512 - PoinTr - INFO - config.optimizer = edict()
2022-04-12 15:27:15,512 - PoinTr - INFO - config.optimizer.type : AdamW
2022-04-12 15:27:15,512 - PoinTr - INFO - config.optimizer.kwargs = edict()
2022-04-12 15:27:15,512 - PoinTr - INFO - config.optimizer.kwargs.lr : 0.0005
2022-04-12 15:27:15,512 - PoinTr - INFO - config.optimizer.kwargs.weight_decay : 0.0005
2022-04-12 15:27:15,512 - PoinTr - INFO - config.scheduler = edict()
2022-04-12 15:27:15,512 - PoinTr - INFO - config.scheduler.type : LambdaLR
2022-04-12 15:27:15,512 - PoinTr - INFO - config.scheduler.kwargs = edict()
2022-04-12 15:27:15,512 - PoinTr - INFO - config.scheduler.kwargs.decay_step : 21
2022-04-12 15:27:15,512 - PoinTr - INFO - config.scheduler.kwargs.lr_decay : 0.9
2022-04-12 15:27:15,512 - PoinTr - INFO - config.scheduler.kwargs.lowest_decay : 0.02
2022-04-12 15:27:15,512 - PoinTr - INFO - config.bnmscheduler = edict()
2022-04-12 15:27:15,512 - PoinTr - INFO - config.bnmscheduler.type : Lambda
2022-04-12 15:27:15,512 - PoinTr - INFO - config.bnmscheduler.kwargs = edict()
2022-04-12 15:27:15,512 - PoinTr - INFO - config.bnmscheduler.kwargs.decay_step : 21
2022-04-12 15:27:15,512 - PoinTr - INFO - config.bnmscheduler.kwargs.bn_decay : 0.5
2022-04-12 15:27:15,512 - PoinTr - INFO - config.bnmscheduler.kwargs.bn_momentum : 0.9
2022-04-12 15:27:15,512 - PoinTr - INFO - config.bnmscheduler.kwargs.lowest_decay : 0.01
2022-04-12 15:27:15,512 - PoinTr - INFO - config.dataset = edict()
2022-04-12 15:27:15,512 - PoinTr - INFO - config.dataset.train = edict()
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train._base_ = edict()
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train._base_.NAME : PCN
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train._base_.CATEGORY_FILE_PATH : data/PCN/PCN.json
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train._base_.N_POINTS : 16384
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train._base_.N_RENDERINGS : 8
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train._base_.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train._base_.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train._base_.CARS : False
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train.others = edict()
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train.others.subset : train
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.train.others.bs : 48
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val = edict()
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val._base_ = edict()
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val._base_.NAME : PCN
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val._base_.CATEGORY_FILE_PATH : data/PCN/PCN.json
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val._base_.N_POINTS : 16384
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val._base_.N_RENDERINGS : 8
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val._base_.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val._base_.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val._base_.CARS : False
2022-04-12 15:27:15,513 - PoinTr - INFO - config.dataset.val.others = edict()
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.val.others.subset : test
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test = edict()
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test._base_ = edict()
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test._base_.NAME : PCN
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test._base_.CATEGORY_FILE_PATH : data/PCN/PCN.json
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test._base_.N_POINTS : 16384
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test._base_.N_RENDERINGS : 8
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test._base_.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test._base_.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test._base_.CARS : False
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test.others = edict()
2022-04-12 15:27:15,514 - PoinTr - INFO - config.dataset.test.others.subset : test
2022-04-12 15:27:15,514 - PoinTr - INFO - config.model = edict()
2022-04-12 15:27:15,514 - PoinTr - INFO - config.model.NAME : PoinTr
2022-04-12 15:27:15,514 - PoinTr - INFO - config.model.num_pred : 14336
2022-04-12 15:27:15,514 - PoinTr - INFO - config.model.num_query : 224
2022-04-12 15:27:15,515 - PoinTr - INFO - config.model.knn_layer : 1
2022-04-12 15:27:15,515 - PoinTr - INFO - config.model.trans_dim : 384
2022-04-12 15:27:15,515 - PoinTr - INFO - config.total_bs : 48
2022-04-12 15:27:15,515 - PoinTr - INFO - config.step_per_update : 1
2022-04-12 15:27:15,515 - PoinTr - INFO - config.max_epoch : 300
2022-04-12 15:27:15,515 - PoinTr - INFO - config.consider_metric : CDL1
2022-04-12 15:27:15,515 - PoinTr - INFO - Distributed training: False
2022-04-12 15:27:15,515 - PoinTr - INFO - Set random seed to 0, deterministic: False
2022-04-12 15:27:15,520 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02691156, Name=airplane]
2022-04-12 15:27:15,540 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02933112, Name=cabinet]
2022-04-12 15:27:15,547 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02958343, Name=car]
2022-04-12 15:27:15,573 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=03001627, Name=chair]
2022-04-12 15:27:15,661 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=03636649, Name=lamp]
2022-04-12 15:27:15,671 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04256520, Name=sofa]
2022-04-12 15:27:15,684 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04379243, Name=table]
2022-04-12 15:27:15,713 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04530566, Name=watercraft]
2022-04-12 15:27:15,720 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=00000001, Name=house]
2022-04-12 15:27:15,721 - PCNDATASET - INFO - Complete collecting files of the dataset. Total files: 29050
2022-04-12 15:27:15,726 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02691156, Name=airplane]
2022-04-12 15:27:15,726 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02933112, Name=cabinet]
2022-04-12 15:27:15,727 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02958343, Name=car]
2022-04-12 15:27:15,727 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=03001627, Name=chair]
2022-04-12 15:27:15,727 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=03636649, Name=lamp]
2022-04-12 15:27:15,728 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04256520, Name=sofa]
2022-04-12 15:27:15,728 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04379243, Name=table]
2022-04-12 15:27:15,728 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04530566, Name=watercraft]
2022-04-12 15:27:15,728 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=00000001, Name=house]
2022-04-12 15:27:15,728 - PCNDATASET - INFO - Complete collecting files of the dataset. Total files: 1205
2022-04-12 15:27:15,729 - MODEL - INFO -  Transformer with knn_layer 1
2022-04-12 15:27:18,763 - PoinTr - INFO - Using Data parallel ...
Traceback (most recent call last):
  File "main.py", line 68, in <module>
    main()
  File "main.py", line 64, in main
    run_net(args, config, train_writer, val_writer)
  File "/home/takenobu/デスクトップ/study/PoinTr/tools/runner.py", line 98, in run_net
    ret = base_model(partial)
  File "/home/takenobu/miniconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/takenobu/miniconda3/envs/torch/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/takenobu/miniconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/takenobu/デスクトップ/study/PoinTr/models/PoinTr.py", line 92, in forward
    q, coarse_point_cloud = self.base_model(xyz) # B M C and B M 3
  File "/home/takenobu/miniconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/takenobu/デスクトップ/study/PoinTr/models/Transformer.py", line 389, in forward
    q = blk(q, x)
  File "/home/takenobu/miniconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/takenobu/デスクトップ/study/PoinTr/models/Transformer.py", line 160, in forward
    q_1 = self.self_attn(norm_q)
  File "/home/takenobu/miniconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/takenobu/デスクトップ/study/PoinTr/models/Transformer.py", line 76, in forward
    attn = (q @ k.transpose(-2, -1)) * self.scale
RuntimeError: CUDA out of memory. Tried to allocate 56.00 MiB (GPU 0; 7.79 GiB total capacity; 5.31 GiB already allocated; 71.88 MiB free; 5.41 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

and I run below...

bash ./scripts/train.sh 0 --config ./cfgs/PCN_models/PoinTr.yaml --exp_name example

Thank you in advance!

How to evaluate KITTI dataset under the Fidelity and MMD metrics?

Thanks for you work.
I have a question that how to evaluate KITTI dataset under the Fidelity and MMD metrics?

An error occurs when test with PoinTr on my own dataset:subprocess.CalledProcessError

Hi !
I am trying to train the PoinTr pretrained model on my own dataset use the method in #11 but after training to the last epoch I get these error :
` warnings.warn("Unable to load pointnet2_ops cpp extension. JIT Compiling.")
Traceback (most recent call last):
File "main.py", line 68, in
main()
File "main.py", line 64, in main
run_net(args, config, train_writer, val_writer)
File "/home/***/codes/PoinTr/tools/runner.py", line 162, in run_net
train_writer.close()
AttributeError: 'NoneType' object has no attribute 'close'

2022-04-27 12:12:53,925 - PoinTr - INFO - [Validation] EPOCH: 300 Metrics = ['0.3882', '14.8132', '0.7327']
2022-04-27 12:12:53,926 - PoinTr - INFO - ============================ TEST RESULTS ============================
2022-04-27 12:12:53,926 - PoinTr - INFO - Taxonomy #Sample F-Score CDL1 CDL2 #ModelName
2022-04-27 12:12:53,926 - PoinTr - INFO - 02691156 5 0.388 14.813 0.733 airplane
2022-04-27 12:12:53,927 - PoinTr - INFO - Overall 0.388 14.813 0.733
Killing subprocess 19184
Killing subprocess 19185
Traceback (most recent call last):
File "/home//anaconda3/envs/PoinTr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home//anaconda3/envs/PoinTr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home//anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/launch.py", line 340, in
main()
File "/home//anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/launch.py", line 326, in main
sigkill_handler(signal.SIGTERM, None) # not coming back
File "/home//anaconda3/envs/PoinTr/lib/python3.7/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/home//anaconda3/envs/PoinTr/bin/python', '-u', 'main.py', '--local_rank=1', '--launcher', 'pytorch', '--sync_bn', '--config', './cfgs/JYG_models/PoinTr.yaml', '--exp_name', 'example']' returned non-zero exit status 1.
by running this code:CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232
--config ./cfgs/MYDATA_models/PoinTr.yaml
--exp_name example`
I use two Nvidia 3060 GPUs
Could you please give some advice，thanks！

No moudle named 'gridding_distance'

Thanks for your code.
The code "bash install.sh" can be performed, and gridding-distance==1.0.0 can be succesfully installed. But when I test PoinTr pretrained model on the PCN benchmark, error occurred "No module named 'gridding_distance' in ./exensions/gridding_loss/init.py"

Loss function: additional robustness?

Hi @yuxumin, I've been wondering if you tried to combine both the CD_l1 & CD_l2 loss function during training? Maybe it would yield better results?

ImportError: /home/cky/anaconda3/envs/python37/lib/python3.7/site-packages/knn_cuda/csrc/_ext/knn/knn.so: cannot open shared object file: No such file or directory

Hi, I have run the building commands, but when I run the first example training command the error shows like this. The KNN cuda 0.2 is installed. Could you please give me some advice?

The program is killed by the system

What hardware do you use for this work?

Thank you for sharing. What hardware do you use for deep learning?

Visualisation

Hi, using the PCN dataset, how do I visualize the output. Is it possible to save them directly in .ply format?

Question about other models in ./models folder

Hi,
Thank you for the great work!
I've found some previous network frameworks in the ./models folder. I'm wondering what these networks are used for and how to use them?
I guess these networks may be used for comparative experiments.
Thank you !

How to test with PoinTr on my own dataset?

Thanks a lot for your commit, could you please give some ideas on how to test with PoinTr on my own dataset?

test

hi，when i test PoinTr on KITTI，i meet a error：worker（pid 27631）is killed by signal：killed. how should i solve it？Thank you very musch

Problem with multi-gpus

Hi,

Thanks for your great work.
I would like to train the model using multiple GPUs but I receive this error:
" RuntimeError: CUDA error: invalid device ordinal CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect."

by running this code:
CUDA_VISIBLE_DEVICES=0,1 singularity exec --nv --writable-tmpfs -B /work/myname/ /work/myname/pointr.sif bash ./scripts/dist_train.sh 2 13232 --config ./cfgs/PCN_models/PoinTr.yaml --exp_name example

Note that I do not have any problem when using single gpu

How can i fix the error clamed "ModuleNotFoundError: No module named 'chamfer'"?

HI! Thank you for your open source! I am trying to run examples you written in your document, but after input the code, it remind me an error named "ModuleNotFoundError: No module named 'chamfer'". How can I fix it? Looking for your help.

CUDA kernel failed : invalid device function

My env is RTX SUPER 2070,CUDA10.1,PYTORCH1.7.0,python3.7.
When I run bash ./scripts/train.sh 0 --config ./cfgs/KITTI_models/PoinTr.yaml --exp_name example
I got a error:
CUDA kernel failed : invalid device function
void furthest_point_sampling_kernel_wrapper(int, int, int, const float*, float*, int*) at L:228 in /tmp/pip-install-1fkhsme8/pointnet2-ops_2dcfc30f04014384afee68e794cd26b7/pointnet2_ops_lib/pointnet2_ops/_ext-src/src/sampling_gpu.cu
./scripts/train.sh: line 8: 2704 Segmentation fault (core dumped) CUDA_VISIBLE_DEVICES=${GPUS} python main.py ${PY_ARGS}
I tried to solve it,but failed.Can you help me?

How to collect the coordinates of the points from the completed point cloud?

Hi !

I would like to know how I could run the poinTr program on a personnal incompleted point cloud. Also could you help me on the way of collecting the coordinates of the points from the completed point cloud ?

Thank you

Error occured when testing the PoinTr pretrained model on the PCN benchmark : "Error building extension 'knn' "

Hi !

I am trying to test the PoinTr pretrained model on the PCN benchmark but I get this error :

`C:\Repos\PoinTr-master>bash ./scripts/test.sh 0
--ckpts ./pretrained/PoinTr_PCN.pth
--config ./cfgs/PCN_models/PoinTr.yaml
--exp_name example

GPUS=0
PY_ARGS=''
CUDA_VISIBLE_DEVICES=0
/mnt/c/Users/peter/AppData/Local/Programs/Python/Python39/python.exe main.py --test ''
Traceback (most recent call last):
File "C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\cpp_extension.py", line 1717, in _run_ninja_build
subprocess.run(
File "C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Repos\PoinTr-master\main.py", line 1, in
from tools import run_net
File "C:\Repos\PoinTr-master\tools_init_.py", line 1, in
from .runner import run_net
File "C:\Repos\PoinTr-master\tools\runner.py", line 5, in
from tools import builder
File "C:\Repos\PoinTr-master\tools\builder.py", line 8, in
from models import build_model_from_cfg
File "C:\Repos\PoinTr-master\models_init_.py", line 3, in
import models.PoinTr
File "C:\Repos\PoinTr-master\models\PoinTr.py", line 6, in
from .Transformer import PCTransformer
File "C:\Repos\PoinTr-master\models\Transformer.py", line 6, in
from .dgcnn_group import DGCNN_Grouper
File "C:\Repos\PoinTr-master\models\dgcnn_group.py", line 4, in
from knn_cuda import KNN
File "C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\knn_cuda_init_.py", line 38, in
knn = load_cpp_ext("knn")
File "C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\knn_cuda_init.py", line 26, in load_cpp_ext
ext = load(
File "C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\cpp_extension.py", line 1124, in load
return _jit_compile(
File "C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\cpp_extension.py", line 1337, in _jit_compile
_write_ninja_file_and_build_library(
File "C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\cpp_extension.py", line 1449, in _write_ninja_file_and_build_library
_run_ninja_build(
File "C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\cpp_extension.py", line 1733, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'knn': [1/1] "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64/link.exe" knn.o knn.cuda.o /nologo /DLL c10.lib c10_cuda.lib torch_cpu.lib torch
_cuda.lib -INCLUDE:?warp_size@cuda@at@@yahxz torch.lib /LIBPATH:C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\lib torch_python.lib /LIBPATH:C:\Users\peter\AppData\Local\Programs\Python\Python39\libs "/LIBP
ATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\lib/x64" cudart.lib /out:knn.pyd
FAILED: knn.pyd
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64/link.exe" knn.o knn.cuda.o /nologo /DLL c10.lib c10_cuda.lib torch_cpu.lib torch_cuda.lib -INCLUDE:?warp_size@cuda@at@@yahxz torch.l
ib /LIBPATH:C:\Users\peter\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\lib torch_python.lib /LIBPATH:C:\Users\peter\AppData\Local\Programs\Python\Python39\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CU
DA\v10.2\lib/x64" cudart.lib /out:knn.pyd
CrÚation de la bibliothÞque knn.lib et de l'objet knn.exp
knn.o : error LNK2019: symbole externe non rÚsolu "public: long * __cdecl at::TensorBase::data_ptr(void)const " (??$data_ptr@J@TensorBase@at@@QEBAPEAJXZ) rÚfÚrencÚ dans la fonction "public: long * __cdecl at::Tensor::data(voi
d)const " (??$data@J@Tensor@at@@QEBAPEAJXZ)
knn.pyd : fatal error LNK1120: 1 externes non rÚsolus
ninja: build stopped: subcommand failed.`

I have already done the solution for the commun bug in the chamfer distance installation (issue #6 ).

I am on Windows and I was wondering if this error could come from an error when building pytorch extensions.

Visualization

Can I get the code for Visualization?

setup.py issue

Hi, Thanks for your great paper and code, I bnefit a lot.

But when I run python setup.py install --user manually under each dependency directory ,some errors occur:

running install
running bdist_egg
running egg_info
writing chamfer.egg-info/PKG-INFO
writing dependency_links to chamfer.egg-info/dependency_links.txt
writing top-level names to chamfer.egg-info/top_level.txt
reading manifest file 'chamfer.egg-info/SOURCES.txt'
writing manifest file 'chamfer.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'chamfer' extension
Emitting ninja build file /home/car/chr/PoinTr/extensions/chamfer_dist/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] :/usr/local/cuda-11.1/bin/nvcc -I/home/car/anaconda3/lib/python3.8/site-packages/torch/include -I/home/car/anaconda3/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/car/anaconda3/lib/python3.8/site-packages/torch/include/TH -I/home/car/anaconda3/lib/python3.8/site-packages/torch/include/THC -I:/usr/local/cuda-11.1/include -I/home/car/anaconda3/include/python3.8 -c -c /home/car/chr/PoinTr/extensions/chamfer_dist/chamfer.cu -o /home/car/chr/PoinTr/extensions/chamfer_dist/build/temp.linux-x86_64-3.8/chamfer.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=chamfer -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_50,code=compute_50 -gencode=arch=compute_50,code=sm_50 -std=c++14
FAILED: /home/car/chr/PoinTr/extensions/chamfer_dist/build/temp.linux-x86_64-3.8/chamfer.o
:/usr/local/cuda-11.1/bin/nvcc -I/home/car/anaconda3/lib/python3.8/site-packages/torch/include -I/home/car/anaconda3/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/car/anaconda3/lib/python3.8/site-packages/torch/include/TH -I/home/car/anaconda3/lib/python3.8/site-packages/torch/include/THC -I:/usr/local/cuda-11.1/include -I/home/car/anaconda3/include/python3.8 -c -c /home/car/chr/PoinTr/extensions/chamfer_dist/chamfer.cu -o /home/car/chr/PoinTr/extensions/chamfer_dist/build/temp.linux-x86_64-3.8/chamfer.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=chamfer -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_50,code=compute_50 -gencode=arch=compute_50,code=sm_50 -std=c++14
/bin/sh: 1: :/usr/local/cuda-11.1/bin/nvcc: not found
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/car/anaconda3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1666, in _run_ninja_build
subprocess.run(
File "/home/car/anaconda3/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "setup.py", line 11, in
setup(name='chamfer',
File "/home/car/anaconda3/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/car/anaconda3/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/car/anaconda3/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/home/car/anaconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/car/anaconda3/lib/python3.8/site-packages/setuptools/command/install.py", line 67, in run
self.do_egg_install()
File "/home/car/anaconda3/lib/python3.8/site-packages/setuptools/command/install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "/home/car/anaconda3/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/car/anaconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/car/anaconda3/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/home/car/anaconda3/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "/home/car/anaconda3/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/car/anaconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/car/anaconda3/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/home/car/anaconda3/lib/python3.8/distutils/command/install_lib.py", line 107, in build
self.run_command('build_ext')
File "/home/car/anaconda3/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/car/anaconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/car/anaconda3/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/car/anaconda3/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/home/car/anaconda3/lib/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/home/car/anaconda3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 709, in build_extensions
build_ext.build_extensions(self)
File "/home/car/anaconda3/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/home/car/anaconda3/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/home/car/anaconda3/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/home/car/anaconda3/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
_build_ext.build_extension(self, ext)
File "/home/car/anaconda3/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
objects = self.compiler.compile(sources,
File "/home/car/anaconda3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 530, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/home/car/anaconda3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1355, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/home/car/anaconda3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

This is my envs:
-- pytorch 1.9.1
-- python 3.8
-- cudnn 8.05
-- g++ 7.5.0
-- gcc 7.5.0
-- nvcc 11.1
-- ninja 1.10.2.git
I make sure cuda and cudnn both are appropriate, ninja is the latest version.Can you give me some advice? thank you a lot.

TypeError

I used the pretrained model of PCN to test on PCN and got some issue:

Traceback (most recent call last):
  File "main.py", line 68, in <module>
    main()
  File "main.py", line 62, in main
    test_net(args, config)
  File "/media/zt/KINGSTON/PoinTr-master/tools/runner.py", line 304, in test_net
    test(base_model, test_dataloader, ChamferDisL1, ChamferDisL2, args, config, logger=logger)
  File "/media/zt/KINGSTON/PoinTr-master/tools/runner.py", line 330, in test
    sparse_loss_l1 =  ChamferDisL1(coarse_points, gt)
  File "/home/zt/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/zt/KINGSTON/PoinTr-master/extensions/chamfer_dist/__init__.py", line 79, in forward
    dist1, dist2 = ChamferFunction.apply(xyz1, xyz2)
  File "/media/zt/KINGSTON/PoinTr-master/extensions/chamfer_dist/__init__.py", line 16, in forward
    dist1, dist2, idx1, idx2 = chamfer.forward(xyz1, xyz2)
TypeError: forward(): incompatible function arguments. The following argument types are supported:
    1. (arg0: at::Tensor, arg1: at::Tensor, arg2: at::Tensor, arg3: at::Tensor, arg4: at::Tensor, arg5: at::Tensor) -> int

Invoked with: tensor([[[ 0.3372, -0.0633,  0.0071],
         [-0.3084, -0.0587, -0.0534],
         [ 0.1694, -0.0381,  0.0665],
         ...,
         [ 0.0708, -0.0256,  0.0818],
         [-0.2394,  0.0460,  0.0049],
         [-0.0334, -0.1057, -0.0396]]], device='cuda:0'), tensor([[[-0.0469, -0.0480,  0.1070],
         [-0.2379, -0.0410,  0.0122],
         [ 0.0307, -0.0627,  0.0817],
         ...,
         [ 0.0234, -0.0052,  0.0366],
         [-0.1879, -0.0876, -0.0684],
         [-0.0315, -0.0052, -0.0005]]], device='cuda:0')

How to change the GPU device?

Hi, how to change the GPU device in the code? The default is GPU:0.

PCN + shapeNet-34 dataset URLs are not available

Hi there,

Could u update the urls for the datasets mentioned above?

Much appreciated!
Nan

'XXdataset is not in the dataset registry'

hi,I want to use PoinTr in other dataset, and I have already modified the related code but got an error "XXdataset(the new dataset) is not in the dataset registry"？
Could you help me?

Opinion about point cloud completion

Hi @yuxumin !

I had a question about point cloud completion for a real application. I have created a point cloud from a real car and I am trying to complete it thanks to the pretrained model on ShapeNet-55 dataset. However, the result is not really satisfactory, as shown with the pictures below :

And this is the result after completion :

I have checked the input format and also the config for poinTr. Do you know what is wrong or what I could do to improve the completion ? Have you ever had experiences or feedback with point cloud completion for real applications ?

Thanks

pre-traind model

In you experiments, you have compared your model with 'PF-Net' and 'GRNet'. Could you provide the pre-trained models of 'PF-Net' and 'GRNet' on PCN and KITTI datasets? We don't have enough resources to trian the model. Thanks a lot.

falling to recover detailed local shapes at predicted points, Single categorie ShapeNet-55

Dear Xumin Yu.

First thanks for this such nice work and paper.

Firstly, I was able to replicate your results using ShapeNet-55, and the pretrained model.

But secondly, I performed the training of the network just using one of the categories of ShapeNet-55, specifically “bottles” that totally contains 498 .npy files. and training the network using just a single GPU.

I used the default configuration from the PoinTr.yaml replacing the batch size from 96 to 24 And max epoch to 600.

However, for this bottle class I was able only to reach CLD2 Min 0.87 and F-Score max 0.52,

After some analysis it seems that model is doing good the center points predictions, but it is falling to recover detailed local shapes at these predicted points. Because mostly of this predicted local shapes are flat surfaces. ( see next picture)

For me it is very important to know I can work with only one category and fewer samples as a previous step before of testing with my own data set, so do you have some previous experience training the network in a single category? it is possible that the DGCNN might not be extracting the features for a category with such a low number of samples?

Thanks in advance for your comments or any advice.

Questions about kNN query and query generator

Hi,
thank you for the great paper. I have some questions about the kNN query and the query generator:

What exactly is the operation in the kNN query layer? In the paper you write "Given the query coordinates p_Q, we query the features of the nearest keys according to the key coordinates p_k." Could you describe this in more detail? Are you still computing attention weights on the nearest neighbors?
In my understanding, the job of the query generator is to make sure that M missing point proxies are produced by the decoder. They are generated from the output of the encoder using a linear projection and the max-pooling operation. But then these coordinates are concatenated with the global features of the encoder. Why do this step? The query of the previous decoder self-attention is not used at all? Or are the coordinates generated from the previous decoder queries?

Thank you for your help.

OOM even when reducing batch size!

Hello and congrats on the excellent work!

I am trying to train the model with 2 gpus:

CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232
--config ./cfgs/PCN_models/PoinTr.yaml
--exp_name example

I am using two Nvidia 1080Tis, it would give me the error of running out of memory. I tried reducing the batchsize from 48 to 24 and even 12, but still the problem remains. do you have any solutions in mind?
thnx

setup.py issue

Hi, Thanks for your great paper and code, I bnefit a lot.

But when I run python setup.py install --user manually under each dependency directory ,some errors occur:

running install
/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
setuptools.SetuptoolsDeprecationWarning,
/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/easy_install.py:147: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
EasyInstallDeprecationWarning,
running bdist_egg
running egg_info
writing chamfer.egg-info/PKG-INFO
writing dependency_links to chamfer.egg-info/dependency_links.txt
writing top-level names to chamfer.egg-info/top_level.txt
reading manifest file 'chamfer.egg-info/SOURCES.txt'
writing manifest file 'chamfer.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'chamfer' extension
Emitting ninja build file /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/build/temp.linux-x86_64-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: warning: bad deps log signature or version; starting over
[1/2] c++ -MMD -MF /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/build/temp.linux-x86_64-3.7/chamfer_cuda.o.d -pthread -B /home/zhang/anaconda3/envs/lws/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/TH -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-10.1/include -I/home/zhang/anaconda3/envs/lws/include/python3.7m -c -c /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/chamfer_cuda.cpp -o /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/build/temp.linux-x86_64-3.7/chamfer_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=chamfer -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/ATen/Parallel.h:149:0,
from /home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
from /home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
from /home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
from /home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:12,
from /home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
from /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/chamfer_cuda.cpp:9:
/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
#pragma omp parallel for if ((end - begin) >= grain_size)

[2/2] /usr/local/cuda-10.1/bin/nvcc -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/TH -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-10.1/include -I/home/zhang/anaconda3/envs/lws/include/python3.7m -c -c /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/chamfer.cu -o /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/build/temp.linux-x86_64-3.7/chamfer.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=chamfer -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -std=c++14
FAILED: /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/build/temp.linux-x86_64-3.7/chamfer.o
/usr/local/cuda-10.1/bin/nvcc -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/TH -I/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-10.1/include -I/home/zhang/anaconda3/envs/lws/include/python3.7m -c -c /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/chamfer.cu -o /home/zhang/lws/PoinTr-master/extensions/chamfer_dist/build/temp.linux-x86_64-3.7/chamfer.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=chamfer -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -std=c++14
/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/ATen/core/boxing/impl/boxing.h(100): warning: integer conversion resulted in a change of sign

/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/ATen/core/op_registration/op_whitelist.h(39): warning: integer conversion resulted in a change of sign

/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/ATen/core/builtin_function.h(97): warning: statement is unreachable

/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/enum.h(191): warning: statement is unreachable

/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/ATen/core/boxing/impl/boxing.h(100): warning: integer conversion resulted in a change of sign

/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/ATen/core/op_registration/op_whitelist.h(39): warning: integer conversion resulted in a change of sign

/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/ATen/core/builtin_function.h(97): warning: statement is unreachable

/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/enum.h(191): warning: statement is unreachable

/usr/include/c++/7/bits/basic_string.tcc: In instantiation of ‘static std::basic_string<_CharT, _Traits, _Alloc>::_Rep* std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_S_create(std::basic_string<_CharT, _Traits, _Alloc>::size_type, std::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’:
/usr/include/c++/7/bits/basic_string.tcc:578:28: required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct(_InIterator, _InIterator, const _Alloc&, std::forward_iterator_tag) [with _FwdIterator = const char16_t*; _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’
/usr/include/c++/7/bits/basic_string.h:5042:20: required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct_aux(_InIterator, _InIterator, const _Alloc&, std::__false_type) [with _InIterator = const char16_t*; _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’
/usr/include/c++/7/bits/basic_string.h:5063:24: required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct(_InIterator, _InIterator, const _Alloc&) [with _InIterator = const char16_t*; _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’
/usr/include/c++/7/bits/basic_string.tcc:656:134: required from ‘std::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, std::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’
/usr/include/c++/7/bits/basic_string.h:6688:95: required from here
/usr/include/c++/7/bits/basic_string.tcc:1067:16: error: cannot call member function ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_set_sharable() [with _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’ without object
__p->_M_set_sharable();
~~~~~~~~~^~
/usr/include/c++/7/bits/basic_string.tcc: In instantiation of ‘static std::basic_string<_CharT, _Traits, _Alloc>::_Rep* std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_S_create(std::basic_string<_CharT, _Traits, _Alloc>::size_type, std::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’:
/usr/include/c++/7/bits/basic_string.tcc:578:28: required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct(_InIterator, _InIterator, const _Alloc&, std::forward_iterator_tag) [with _FwdIterator = const char32_t*; _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>]’
/usr/include/c++/7/bits/basic_string.h:5042:20: required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct_aux(_InIterator, _InIterator, const _Alloc&, std::__false_type) [with _InIterator = const char32_t*; _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>]’
/usr/include/c++/7/bits/basic_string.h:5063:24: required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct(_InIterator, _InIterator, const _Alloc&) [with _InIterator = const char32_t*; _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>]’
/usr/include/c++/7/bits/basic_string.tcc:656:134: required from ‘std::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, std::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’
/usr/include/c++/7/bits/basic_string.h:6693:95: required from here
/usr/include/c++/7/bits/basic_string.tcc:1067:16: error: cannot call member function ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_set_sharable() [with _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>]’ without object
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1539, in _run_ninja_build
env=env)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "setup.py", line 19, in
cmdclass={'build_ext': BuildExtension})
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/init.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 148, in setup
return run_commands(dist)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
dist.run_commands()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
self.run_command(cmd)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/dist.py", line 1214, in run_command
super().run_command(command)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
self.do_egg_install()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
self.run_command('bdist_egg')
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/dist.py", line 1214, in run_command
super().run_command(command)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
self.run_command(cmdname)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/dist.py", line 1214, in run_command
super().run_command(command)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 107, in build
self.run_command('build_ext')
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/dist.py", line 1214, in run_command
super().run_command(command)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 670, in build_extensions
build_ext.build_extensions(self)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
_build_ext.build_extension(self, ext)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 534, in build_extension
depends=ext.depends)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 500, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1255, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/home/zhang/anaconda3/envs/lws/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1555, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

This is my envs:
-- pytorch 1.7.1
-- python 3.7
-- cuda 10.1
-- nvcc 10.1

I downloaded PyTorch and CUDa from the appropriate link on the PyTorch website ,but it still has NVCC errors

Can you give me some advice? thank you a lot.

Import Error about "chamfer" Package

Hi, I downloaded your code on my 2 Ubuntu servers and google colab, but after I get the success of running these commands:

pip install -r requirements.txt
# Chamfer Distance
bash install.sh
# PointNet++
pip install "git+git://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
# GPU kNN
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl

Then I run your provided test example:

bash ./scripts/test.sh 0 \
    --ckpts ./pretrained/PoinTr_PCN.pth \
    --config ./cfgs/PCN_models/PoinTr.yaml \
    --exp_name example

but got:

bash ./scripts/test.sh 0 \
>     --ckpts ./pretrained/PoinTr_ShapeNet55.pth \
>     --config ./cfgs/ShapeNet55_models/PoinTr.yaml \
>     --mode easy \
>     --exp_name example
+ GPUS=0
+ PY_ARGS='--ckpts ./pretrained/PoinTr_ShapeNet55.pth --config ./cfgs/ShapeNet55_models/PoinTr.yaml --mode easy --exp_name example'
+ CUDA_VISIBLE_DEVICES=0
+ python main.py --test --ckpts ./pretrained/PoinTr_ShapeNet55.pth --config ./cfgs/ShapeNet55_models/PoinTr.yaml --mode easy --exp_name example

Traceback (most recent call last):
  File "main.py", line 1, in <module>
    from tools import run_net
  File "/userhome/chenbin/PoinTr/tools/__init__.py", line 1, in <module>
    from .runner import run_net
  File "/userhome/chenbin/PoinTr/tools/runner.py", line 5, in <module>
    from tools import builder
  File "/userhome/chenbin/PoinTr/tools/builder.py", line 8, in <module>
    from models import build_model_from_cfg
  File "/userhome/chenbin/PoinTr/models/__init__.py", line 2, in <module>
    import models.TopNet
  File "/userhome/chenbin/PoinTr/models/TopNet.py", line 6, in <module>
    from extensions.chamfer_dist import ChamferDistanceL2
  File "/userhome/chenbin/PoinTr/extensions/chamfer_dist/__init__.py", line 10, in <module>
    import chamfer
ModuleNotFoundError: No module named 'chamfer'

I tried for a long time to fix this problem, but still failed, so I have to ask you for help.
Could you please tell me how to fix it (or provide a simple colab example)? I just want to test and reproduce the results.
Looking forwarding to your response :) !

Attention Map

Hello,

First of all, thank you very much for your work.
I'd want to know if the attention map can be seen in any way.

Thanks in advance,
Benjamin

Some questions about Paper

Hi, Xumin. It's a great paper and I'm inspired a lot. I have a question about the ablation experiment in the paper. When using the baseline test, the result generated by the query generator is replaced, how do the Dynamic Queries of the Transformer decoder be generated? Thanks a lot.

Evaluation example on the KITTI benchmark

When I try to run the evaluation example for the KITTI benchmark the training procedure starts. Shouldn't this start only the evaluation? It could be that I'm missing something out...

Note: I'm running the code in google colab.

Btw, I added the few lines in the runner.py script, as you suggested in a previous issue, on how to save the dense_points.

Thank you in advance!

Why use FoldingNet to do upsampling?

Hi, Xumin. It's a great paper and I'm inspired a lot. I have a question about upsampling part in the paper. Why use FoldingNet to do upsampling? Thanks a lot.

How to set encoder_feature in PCN_models/TopNet.yaml?

Hi, how to set encoder_feature in PCN_models/TopNet.yaml? As I run bash ./scripts/train.sh 2 --config ./cfgs/PCN_models/TopNet.yaml --exp_name example

I met the error like this:

in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
AttributeError: TopNet: 'EasyDict' object has no attribute 'encoder_feature'

Could not find a version that satisfies the requirement pointnet2-ops

Hi Yuxumin,
It is great work, indeed. i caught this error when I run the code of pip install "git+git://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib". I was wondering if you could help me in that regard. Thank you so much and any response will be highly appreciated. Stay safe

Error listed as follows:
WARNING: Ignoring invalid distribution -orch (/home/mnt/xyqian/miniconda3/envs/centernet/lib/python3.8/site-packages)
WARNING: Ignoring invalid distribution -orch (/home/mnt/xyqian/miniconda3/envs/centernet/lib/python3.8/site-packages)
Collecting pointnet2_ops
Cloning git://github.com/erikwijmans/Pointnet2_PyTorch.git to /tmp/pip-install-kg2drmx2/pointnet2-ops_31824b8057754e00b827bd29dca8a98c
Running command git clone -q git://github.com/erikwijmans/Pointnet2_PyTorch.git /tmp/pip-install-kg2drmx2/pointnet2-ops_31824b8057754e00b827bd29dca8a98c
fatal: remote error:
The unauthenticated git protocol on port 9418 is no longer supported.
Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information.
WARNING: Discarding git+git://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib. Command errored out with exit status 128: git clone -q git://github.com/erikwijmans/Pointnet2_PyTorch.git /tmp/pip-install-kg2drmx2/pointnet2-ops_31824b8057754e00b827bd29dca8a98c Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pointnet2-ops (unavailable) (from versions: none)
ERROR: No matching distribution found for pointnet2-ops (unavailable)
WARNING: Ignoring invalid distribution -orch (/home/mnt/xyqian/miniconda3/envs/centernet/lib/python3.8/site-packages)
WARNING: Ignoring invalid distribution -orch (/home/mnt/xyqian/miniconda3/envs/centernet/lib/python3.8/site-packages)
WARNING: Ignoring invalid distribution -orch (/home/mnt/xyqian/miniconda3/envs/centernet/lib/python3.8/site-packages)

A problem named "AssertionError"

Hi, sorry to bother you. I got another problem when I run
`bash ./scripts/test.sh 0 \

--ckpts ./pretrained/PoinTr_PCN.pth \
--config ./cfgs/PCN_models/PoinTr.yaml \
--exp_name example

The Terminal shows+ GPUS=0

PY_ARGS='--ckpts ./pretrained/PoinTr_PCN.pth --config ./cfgs/PCN_models/PoinTr.yaml --exp_name example'
CUDA_VISIBLE_DEVICES=0
python main.py --test --ckpts ./pretrained/PoinTr_PCN.pth --config ./cfgs/PCN_models/PoinTr.yaml --exp_name example
/home/jimmy/anaconda3/lib/python3.9/site-packages/pointnet2_ops/pointnet2_utils.py:15: UserWarning: Unable to load pointnet2_ops cpp extension. JIT Compiling.
warnings.warn("Unable to load pointnet2_ops cpp extension. JIT Compiling.")
2022-03-14 13:51:16,679 - PoinTr - INFO - Copy the Config file from ./cfgs/PCN_models/PoinTr.yaml to ./experiments/PoinTr/PCN_models/test_example/config.yaml
2022-03-14 13:51:16,679 - PoinTr - INFO - args.config : ./cfgs/PCN_models/PoinTr.yaml
2022-03-14 13:51:16,680 - PoinTr - INFO - args.launcher : none
2022-03-14 13:51:16,680 - PoinTr - INFO - args.local_rank : 0
2022-03-14 13:51:16,680 - PoinTr - INFO - args.num_workers : 4
2022-03-14 13:51:16,680 - PoinTr - INFO - args.seed : 0
2022-03-14 13:51:16,680 - PoinTr - INFO - args.deterministic : False
2022-03-14 13:51:16,680 - PoinTr - INFO - args.sync_bn : False
2022-03-14 13:51:16,680 - PoinTr - INFO - args.exp_name : test_example
2022-03-14 13:51:16,680 - PoinTr - INFO - args.start_ckpts : None
2022-03-14 13:51:16,680 - PoinTr - INFO - args.ckpts : ./pretrained/PoinTr_PCN.pth
2022-03-14 13:51:16,680 - PoinTr - INFO - args.val_freq : 1
2022-03-14 13:51:16,680 - PoinTr - INFO - args.resume : False
2022-03-14 13:51:16,680 - PoinTr - INFO - args.test : True
2022-03-14 13:51:16,680 - PoinTr - INFO - args.mode : None
2022-03-14 13:51:16,680 - PoinTr - INFO - args.experiment_path : ./experiments/PoinTr/PCN_models/test_example
2022-03-14 13:51:16,680 - PoinTr - INFO - args.tfboard_path : ./experiments/PoinTr/PCN_models/TFBoard/test_example
2022-03-14 13:51:16,680 - PoinTr - INFO - args.log_name : PoinTr
2022-03-14 13:51:16,680 - PoinTr - INFO - args.use_gpu : True
2022-03-14 13:51:16,680 - PoinTr - INFO - args.distributed : False
2022-03-14 13:51:16,680 - PoinTr - INFO - config.optimizer = edict()
2022-03-14 13:51:16,680 - PoinTr - INFO - config.optimizer.type : AdamW
2022-03-14 13:51:16,680 - PoinTr - INFO - config.optimizer.kwargs = edict()
2022-03-14 13:51:16,680 - PoinTr - INFO - config.optimizer.kwargs.lr : 0.0005
2022-03-14 13:51:16,680 - PoinTr - INFO - config.optimizer.kwargs.weight_decay : 0.0005
2022-03-14 13:51:16,680 - PoinTr - INFO - config.scheduler = edict()
2022-03-14 13:51:16,680 - PoinTr - INFO - config.scheduler.type : LambdaLR
2022-03-14 13:51:16,681 - PoinTr - INFO - config.scheduler.kwargs = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.scheduler.kwargs.decay_step : 21
2022-03-14 13:51:16,681 - PoinTr - INFO - config.scheduler.kwargs.lr_decay : 0.9
2022-03-14 13:51:16,681 - PoinTr - INFO - config.scheduler.kwargs.lowest_decay : 0.02
2022-03-14 13:51:16,681 - PoinTr - INFO - config.bnmscheduler = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.bnmscheduler.type : Lambda
2022-03-14 13:51:16,681 - PoinTr - INFO - config.bnmscheduler.kwargs = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.bnmscheduler.kwargs.decay_step : 21
2022-03-14 13:51:16,681 - PoinTr - INFO - config.bnmscheduler.kwargs.bn_decay : 0.5
2022-03-14 13:51:16,681 - PoinTr - INFO - config.bnmscheduler.kwargs.bn_momentum : 0.9
2022-03-14 13:51:16,681 - PoinTr - INFO - config.bnmscheduler.kwargs.lowest_decay : 0.01
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.base = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.base.NAME : PCN
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.base.CATEGORY_FILE_PATH : data/PCN/PCN.json
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.base.N_POINTS : 16384
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.base.N_RENDERINGS : 8
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.base.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.base.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.base.CARS : False
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.others = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.others.subset : train
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.train.others.bs : 48
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.val = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.val.base = edict()
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.val.base.NAME : PCN
2022-03-14 13:51:16,681 - PoinTr - INFO - config.dataset.val.base.CATEGORY_FILE_PATH : data/PCN/PCN.json
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.val.base.N_POINTS : 16384
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.val.base.N_RENDERINGS : 8
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.val.base.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.val.base.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.val.base.CARS : False
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.val.others = edict()
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.val.others.subset : test
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test = edict()
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.base = edict()
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.base.NAME : PCN
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.base.CATEGORY_FILE_PATH : data/PCN/PCN.json
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.base.N_POINTS : 16384
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.base.N_RENDERINGS : 8
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.base.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.base.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.base.CARS : False
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.others = edict()
2022-03-14 13:51:16,682 - PoinTr - INFO - config.dataset.test.others.subset : test
2022-03-14 13:51:16,682 - PoinTr - INFO - config.model = edict()
2022-03-14 13:51:16,682 - PoinTr - INFO - config.model.NAME : PoinTr
2022-03-14 13:51:16,682 - PoinTr - INFO - config.model.num_pred : 14336
2022-03-14 13:51:16,682 - PoinTr - INFO - config.model.num_query : 224
2022-03-14 13:51:16,682 - PoinTr - INFO - config.model.knn_layer : 1
2022-03-14 13:51:16,682 - PoinTr - INFO - config.model.trans_dim : 384
2022-03-14 13:51:16,682 - PoinTr - INFO - config.total_bs : 48
2022-03-14 13:51:16,682 - PoinTr - INFO - config.step_per_update : 1
2022-03-14 13:51:16,682 - PoinTr - INFO - config.max_epoch : 300
2022-03-14 13:51:16,682 - PoinTr - INFO - config.consider_metric : CDL1
2022-03-14 13:51:16,682 - PoinTr - INFO - Distributed training: False
2022-03-14 13:51:16,683 - PoinTr - INFO - Set random seed to 0, deterministic: False
2022-03-14 13:51:16,683 - PoinTr - INFO - Tester start ...
2022-03-14 13:51:16,688 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02691156, Name=airplane]
2022-03-14 13:51:16,688 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02933112, Name=cabinet]
2022-03-14 13:51:16,689 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02958343, Name=car]
2022-03-14 13:51:16,689 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=03001627, Name=chair]
2022-03-14 13:51:16,689 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=03636649, Name=lamp]
2022-03-14 13:51:16,689 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04256520, Name=sofa]
2022-03-14 13:51:16,689 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04379243, Name=table]
2022-03-14 13:51:16,689 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04530566, Name=watercraft]
2022-03-14 13:51:16,690 - PCNDATASET - INFO - Complete collecting files of the dataset. Total files: 1200
2022-03-14 13:51:16,690 - MODEL - INFO - Transformer with knn_layer 1
2022-03-14 13:51:19,295 - PoinTr - INFO - Loading weights from ./pretrained/PoinTr_PCN.pth...
2022-03-14 13:51:19,496 - PoinTr - INFO - ckpts @ 291 epoch( performance = {'F-Score': 0.7966486847844346, 'CDL1': 7.263054704332413, 'CDL2': 0.2274231300179963})
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/bb9ba5deec9e2a5fca8607f540cc62ba/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/bb9ba5deec9e2a5fca8607f540cc62ba.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/c510e14ac20edee814a46d7ec44fc258/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/e431f79ac9f0266bca677733d59db4df/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/c510e14ac20edee814a46d7ec44fc258.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/e431f79ac9f0266bca677733d59db4df.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/2a05d684eeb9c1cfae2ca9bb680dd18b/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/2c16ac865f06542381598514fe928082/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/2c16ac865f06542381598514fe928082.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/2a05d684eeb9c1cfae2ca9bb680dd18b.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/ec879d7360572a5db65bd29086fc2a67/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/ec879d7360572a5db65bd29086fc2a67.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/604392af2cbb7d1fe30ec10233e7931a/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/3c6c4be95d2dad2937b25178f54477f4/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/604392af2cbb7d1fe30ec10233e7931a.pcd
Traceback (most recent call last):
File "/home/jimmy/PointCloudProduct/PoinTr-master/main.py", line 68, in
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/3c6c4be95d2dad2937b25178f54477f4.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/21c7cc3e5f27540be6553f96496649c9/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/21c7cc3e5f27540be6553f96496649c9.pcd
main()
File "/home/jimmy/PointCloudProduct/PoinTr-master/main.py", line 62, in main
test_net(args, config)
File "/home/jimmy/PointCloudProduct/PoinTr-master/tools/runner.py", line 304, in test_net
test(base_model, test_dataloader, ChamferDisL1, ChamferDisL2, args, config, logger=logger)
File "/home/jimmy/PointCloudProduct/PoinTr-master/tools/runner.py", line 316, in test
for idx, (taxonomy_ids, model_ids, data) in enumerate(test_dataloader):
File "/home/jimmy/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 517, in next
data = self._next_data()
File "/home/jimmy/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
return self._process_data(data)
File "/home/jimmy/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
data.reraise()
File "/home/jimmy/anaconda3/lib/python3.9/site-packages/torch/_utils.py", line 429, in reraise
raise self.exc_type(msg)
AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/jimmy/anaconda3/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
data = fetcher.fetch(index)
File "/home/jimmy/anaconda3/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/jimmy/anaconda3/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/jimmy/PointCloudProduct/PoinTr-master/datasets/PCNDataset.py", line 103, in getitem
assert data['gt'].shape[0] == self.npoints
AssertionError
`
I have checked my pytorch, and it is running well. Is that something wrong in open3d?

May the author provide the source code and configuration files of other models, such as AtlasNet, MSN, PFNet, etc. Thank you!

Failed to load image Python extension, Open3D Warning

Hi !

After issue #21 I had a new error when running the same command (test PoinTr model on the PCN benchmark).

`C:\Repos\PoinTr-master>bash
pierrot@DESKTOP-FMMUKS2:/mnt/c/Repos/PoinTr-master$ bash ./scripts/test.sh 0
--ckpts ./pretrained/PoinTr_PCN.pth
--config ./cfgs/PCN_models/PoinTr.yaml
--exp_name example

GPUS=0
PY_ARGS='--ckpts ./pretrained/PoinTr_PCN.pth --config ./cfgs/PCN_models/PoinTr.yaml --exp_name example'
CUDA_VISIBLE_DEVICES=0
/mnt/c/Users/peter/AppData/Local/Programs/Python/Python36/python.exe main.py --test --ckpts ./pretrained/PoinTr_PCN.pth --config ./cfgs/PCN_models/PoinTr.yaml --exp_name example
C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\io\image.py:11: UserWarning: Failed
to load image Python extension: [WinError 126] Le module spécifié est introuvable
warn(f"Failed to load image Python extension: {e}")
'cp' n’est pas reconnu en tant que commande interne
ou externe, un programme exécutable ou un fichier de commandes.
2021-12-17 09:35:10,418 - PoinTr - INFO - Copy the Config file from ./cfgs/PCN_models/PoinTr.yaml to ./experiments\PoinTr\PCN_models\test_example\config.yaml
2021-12-17 09:35:10,419 - PoinTr - INFO - args.config : ./cfgs/PCN_models/PoinTr.yaml
2021-12-17 09:35:10,419 - PoinTr - INFO - args.launcher : none
2021-12-17 09:35:10,419 - PoinTr - INFO - args.local_rank : 0
2021-12-17 09:35:10,420 - PoinTr - INFO - args.num_workers : 4
2021-12-17 09:35:10,420 - PoinTr - INFO - args.seed : 0
2021-12-17 09:35:10,420 - PoinTr - INFO - args.deterministic : False
2021-12-17 09:35:10,421 - PoinTr - INFO - args.sync_bn : False
2021-12-17 09:35:10,421 - PoinTr - INFO - args.exp_name : test_example
2021-12-17 09:35:10,421 - PoinTr - INFO - args.start_ckpts : None
2021-12-17 09:35:10,422 - PoinTr - INFO - args.ckpts : ./pretrained/PoinTr_PCN.pth
2021-12-17 09:35:10,422 - PoinTr - INFO - args.val_freq : 1
2021-12-17 09:35:10,422 - PoinTr - INFO - args.resume : False
2021-12-17 09:35:10,423 - PoinTr - INFO - args.test : True
2021-12-17 09:35:10,423 - PoinTr - INFO - args.mode : None
2021-12-17 09:35:10,423 - PoinTr - INFO - args.experiment_path : ./experiments\PoinTr\PCN_models\test_example
2021-12-17 09:35:10,424 - PoinTr - INFO - args.tfboard_path : ./experiments\PoinTr\PCN_models\TFBoard\test_example
2021-12-17 09:35:10,424 - PoinTr - INFO - args.log_name : PoinTr
2021-12-17 09:35:10,424 - PoinTr - INFO - args.use_gpu : True
2021-12-17 09:35:10,425 - PoinTr - INFO - args.distributed : False
2021-12-17 09:35:10,425 - PoinTr - INFO - config.optimizer = edict()
2021-12-17 09:35:10,425 - PoinTr - INFO - config.optimizer.type : AdamW
2021-12-17 09:35:10,426 - PoinTr - INFO - config.optimizer.kwargs = edict()
2021-12-17 09:35:10,426 - PoinTr - INFO - config.optimizer.kwargs.lr : 0.0005
2021-12-17 09:35:10,426 - PoinTr - INFO - config.optimizer.kwargs.weight_decay : 0.0005
2021-12-17 09:35:10,426 - PoinTr - INFO - config.scheduler = edict()
2021-12-17 09:35:10,427 - PoinTr - INFO - config.scheduler.type : LambdaLR
2021-12-17 09:35:10,427 - PoinTr - INFO - config.scheduler.kwargs = edict()
2021-12-17 09:35:10,427 - PoinTr - INFO - config.scheduler.kwargs.decay_step : 21
2021-12-17 09:35:10,429 - PoinTr - INFO - config.scheduler.kwargs.lr_decay : 0.9
2021-12-17 09:35:10,430 - PoinTr - INFO - config.scheduler.kwargs.lowest_decay : 0.02
2021-12-17 09:35:10,430 - PoinTr - INFO - config.bnmscheduler = edict()
2021-12-17 09:35:10,430 - PoinTr - INFO - config.bnmscheduler.type : Lambda
2021-12-17 09:35:10,431 - PoinTr - INFO - config.bnmscheduler.kwargs = edict()
2021-12-17 09:35:10,431 - PoinTr - INFO - config.bnmscheduler.kwargs.decay_step : 21
2021-12-17 09:35:10,431 - PoinTr - INFO - config.bnmscheduler.kwargs.bn_decay : 0.5
2021-12-17 09:35:10,432 - PoinTr - INFO - config.bnmscheduler.kwargs.bn_momentum : 0.9
2021-12-17 09:35:10,432 - PoinTr - INFO - config.bnmscheduler.kwargs.lowest_decay : 0.01
2021-12-17 09:35:10,432 - PoinTr - INFO - config.dataset = edict()
2021-12-17 09:35:10,433 - PoinTr - INFO - config.dataset.train = edict()
2021-12-17 09:35:10,433 - PoinTr - INFO - config.dataset.train.base = edict()
2021-12-17 09:35:10,433 - PoinTr - INFO - config.dataset.train.base.NAME : PCN
2021-12-17 09:35:10,433 - PoinTr - INFO - config.dataset.train.base.CATEGORY_FILE_PATH : data/PCN/PCN.json
2021-12-17 09:35:10,434 - PoinTr - INFO - config.dataset.train.base.N_POINTS : 16384
2021-12-17 09:35:10,434 - PoinTr - INFO - config.dataset.train.base.N_RENDERINGS : 8
2021-12-17 09:35:10,434 - PoinTr - INFO - config.dataset.train.base.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2021-12-17 09:35:10,435 - PoinTr - INFO - config.dataset.train.base.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2021-12-17 09:35:10,436 - PoinTr - INFO - config.dataset.train.base.CARS : False
2021-12-17 09:35:10,436 - PoinTr - INFO - config.dataset.train.others = edict()
2021-12-17 09:35:10,436 - PoinTr - INFO - config.dataset.train.others.subset : train
2021-12-17 09:35:10,437 - PoinTr - INFO - config.dataset.train.others.bs : 48
2021-12-17 09:35:10,437 - PoinTr - INFO - config.dataset.val = edict()
2021-12-17 09:35:10,437 - PoinTr - INFO - config.dataset.val.base = edict()
2021-12-17 09:35:10,438 - PoinTr - INFO - config.dataset.val.base.NAME : PCN
2021-12-17 09:35:10,438 - PoinTr - INFO - config.dataset.val.base.CATEGORY_FILE_PATH : data/PCN/PCN.json
2021-12-17 09:35:10,438 - PoinTr - INFO - config.dataset.val.base.N_POINTS : 16384
2021-12-17 09:35:10,438 - PoinTr - INFO - config.dataset.val.base.N_RENDERINGS : 8
2021-12-17 09:35:10,439 - PoinTr - INFO - config.dataset.val.base.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2021-12-17 09:35:10,439 - PoinTr - INFO - config.dataset.val.base.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2021-12-17 09:35:10,440 - PoinTr - INFO - config.dataset.val.base.CARS : False
2021-12-17 09:35:10,440 - PoinTr - INFO - config.dataset.val.others = edict()
2021-12-17 09:35:10,441 - PoinTr - INFO - config.dataset.val.others.subset : test
2021-12-17 09:35:10,441 - PoinTr - INFO - config.dataset.test = edict()
2021-12-17 09:35:10,441 - PoinTr - INFO - config.dataset.test.base = edict()
2021-12-17 09:35:10,442 - PoinTr - INFO - config.dataset.test.base.NAME : PCN
2021-12-17 09:35:10,442 - PoinTr - INFO - config.dataset.test.base.CATEGORY_FILE_PATH : data/PCN/PCN.json
2021-12-17 09:35:10,442 - PoinTr - INFO - config.dataset.test.base.N_POINTS : 16384
2021-12-17 09:35:10,442 - PoinTr - INFO - config.dataset.test.base.N_RENDERINGS : 8
2021-12-17 09:35:10,445 - PoinTr - INFO - config.dataset.test.base.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2021-12-17 09:35:10,445 - PoinTr - INFO - config.dataset.test.base.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2021-12-17 09:35:10,446 - PoinTr - INFO - config.dataset.test.base.CARS : False
2021-12-17 09:35:10,446 - PoinTr - INFO - config.dataset.test.others = edict()
2021-12-17 09:35:10,447 - PoinTr - INFO - config.dataset.test.others.subset : test
2021-12-17 09:35:10,447 - PoinTr - INFO - config.model = edict()
2021-12-17 09:35:10,447 - PoinTr - INFO - config.model.NAME : PoinTr
2021-12-17 09:35:10,447 - PoinTr - INFO - config.model.num_pred : 14336
2021-12-17 09:35:10,448 - PoinTr - INFO - config.model.num_query : 224
2021-12-17 09:35:10,448 - PoinTr - INFO - config.model.knn_layer : 1
2021-12-17 09:35:10,448 - PoinTr - INFO - config.model.trans_dim : 384
2021-12-17 09:35:10,449 - PoinTr - INFO - config.total_bs : 48
2021-12-17 09:35:10,449 - PoinTr - INFO - config.step_per_update : 1
2021-12-17 09:35:10,449 - PoinTr - INFO - config.max_epoch : 300
2021-12-17 09:35:10,450 - PoinTr - INFO - config.consider_metric : CDL1
2021-12-17 09:35:10,450 - PoinTr - INFO - Distributed training: False
2021-12-17 09:35:10,450 - PoinTr - INFO - Set random seed to 0, deterministic: False
2021-12-17 09:35:10,451 - PoinTr - INFO - Tester start ...
2021-12-17 09:35:10,460 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02691156, Name=airplane]
2021-12-17 09:35:10,461 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02933112, Name=cabinet]
2021-12-17 09:35:10,461 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02958343, Name=car]
2021-12-17 09:35:10,462 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=03001627, Name=chair]
2021-12-17 09:35:10,463 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=03636649, Name=lamp]
2021-12-17 09:35:10,463 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04256520, Name=sofa]
2021-12-17 09:35:10,464 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04379243, Name=table]
2021-12-17 09:35:10,464 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=04530566, Name=watercraft]
2021-12-17 09:35:10,465 - PCNDATASET - INFO - Complete collecting files of the dataset. Total files: 1200
2021-12-17 09:35:10,466 - MODEL - INFO - Transformer with knn_layer 1
2021-12-17 09:35:11,071 - PoinTr - INFO - Loading weights from ./pretrained/PoinTr_PCN.pth...
2021-12-17 09:35:12,220 - PoinTr - INFO - ckpts @ 289 epoch( performance = No Metrics)
C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\io\image.py:11: UserWarning: Failed to load image Python extension: [WinError 126] Le module spécifié est introuvable
warn(f"Failed to load image Python extension: {e}")
C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\io\image.py:11: UserWarning: Failed to load image Python extension: [WinError 126] Le module spécifié est introuvable
warn(f"Failed to load image Python extension: {e}")
C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\io\image.py:11: UserWarning: Failed to load image Python extension: [WinError 126] Le module spécifié est introuvable
warn(f"Failed to load image Python extension: {e}")
C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\io\image.py:11: UserWarning: Failed to load image Python extension: [WinError 126] Le module spécifié est introuvable
warn(f"Failed to load image Python extension: {e}")
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/bb9ba5deec9e2a5fca8607f540cc62ba/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/2c16ac865f06542381598514fe928082/00.pcd[Open3D WARNING]
Read PCD failed: unable to open file: data/PCN/test/partial/02691156/e431f79ac9f0266bca677733d59db4df/00.pcd
[Open3D WARNING] [Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/e431f79ac9f0266bca677733d59db4df.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/2c16ac865f06542381598514fe928082.pcdRead PCD failed: unable to open file: data/PCN/test/complete/02691156/bb9ba5deec9e2a5fca8607f540cc62ba.pcd

[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/c510e14ac20edee814a46d7ec44fc258/00.pcd
[Open3D WARNING] [Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/ec879d7360572a5db65bd29086fc2a67/00.pcdRead PCD failed: unable to open file: data/PCN/test/partial/02691156/3c6c4be95d2dad2937b25178f5
4477f4/00.pcd

Traceback (most recent call last):
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/c510e14ac20edee814a46d7ec44fc258.pcd File "main.py", line 68, in

[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/3c6c4be95d2dad2937b25178f54477f4.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/ec879d7360572a5db65bd29086fc2a67.pcdmain()

File "main.py", line 62, in main
test_net(args, config)
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/2a05d684eeb9c1cfae2ca9bb680dd18b/00.pcd
File "C:\Repos\PoinTr-master\tools\runner.py", line 304, in test_net
[Open3D WARNING] [Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/2a05d684eeb9c1cfae2ca9bb680dd18b.pcd
Read PCD failed: unable to open file: data/PCN/test/partial/02691156/604392af2cbb7d1fe30ec10233e7931a/00.pcd
test(base_model, test_dataloader, ChamferDisL1, ChamferDisL2, args, config, logger=logger)[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/604392af2cbb7d1fe30ec10233e7931a.pcd

File "C:\Repos\PoinTr-master\tools\runner.py", line 316, in test
for idx, (taxonomy_ids, model_ids, data) in enumerate(test_dataloader):
File "C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 521, in next
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/partial/02691156/21c7cc3e5f27540be6553f96496649c9/00.pcd
[Open3D WARNING] Read PCD failed: unable to open file: data/PCN/test/complete/02691156/21c7cc3e5f27540be6553f96496649c9.pcd
data = self._next_data()
File "C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 1229, in _process_data
data.reraise()
File "C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torch_utils.py", line 434, in reraise
raise exception
AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data_utils\worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data_utils\fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\peter\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data_utils\fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Repos\PoinTr-master\datasets\PCNDataset.py", line 103, in getitem
assert data['gt'].shape[0] == self.npoints
AssertionError`

The error seems to come from Open3D but I have no idea how to fix it.

pyexr问题

您好，requirements.txt中的pyexr安装报错，我尝试了很多网上安装方式，都未成功，请问这个库怎么安装，非常感谢。

Here is are some additional steps for others who are trying to train the PoinTr model on their own (custom) dataset:

Here is are some additional steps for others who are trying to train the PoinTr model on their own (custom) dataset:

add this line in the ".datasets/init.py" script: import datasets.CUSTOMDATASETNAME
add the support for your dataset in the ".tools/runner.py" script: take a look at line 82 (def run_net) and line 178 (def validate) in the code
optional: if you need support for ".ply" files then you have to add this in ".datasets/io.py" script
important: don't forget to center and normalize your point cloud data

Originally posted by @blukaz in #11 (comment)

the fixed dataloader bug (PCN)

You have fixed the bug in datasets/PCNDataset.py(#27), and updated the performance of PoinTr on PCN benchmark (CD from 8.38 to 7.26). But I can't find the fixed code in your code "PCNDataset.py", could you show me? Thanks in advance.

Input and predicted car scaling is off

Hi,

I ran PoinTr for the KITTI dataset as provided and visualized the input.npy and pred.npy with open3d. I'm aware that the car's pose has been normalized and therefore it seems odd to me that the scaling of the input.npy and pred.npy point clouds appear different? Does anyone else have this problem?

This was the command I ran. I did a fresh pull from the repo with no code modifications.

bash ./scripts/test.sh 0     
--ckpts ./pretrained/PoinTr_KITTI.pth     
--config ./cfgs/KITTI_models/PoinTr.yaml     
--exp_name

Purple is the input.npy and grey is the pred.npy. Predicted point cloud looks smaller than the input point cloud.

yuxumin / pointr Goto Github PK

pointr's Introduction

PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

🔥News

Pretrained Models

Usage

Requirements

Building Pytorch Extensions for Chamfer Distance, PointNet++ and kNN

Dataset

Inference

Evaluation

Some examples:

Training

Some examples:

Completion Results on ShapeNet55 and KITTI-Cars

License

Acknowledgements

Citation

pointr's People

Contributors

Stargazers

Watchers

Forkers

pointr's Issues

main.py FAILED

Failures: <NO_OTHER_FAILURES>

Root Cause (first observed failure): [0]: time : 2022-02-14_01:37:10 host :uer rank : 1 (local_rank: 1) exitcode : 1 (pid: 13572) error_file: <N/A> traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

Recommend Projects

Recommend Topics

Recommend Org

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2022-02-14_01:37:10
host :uer
rank : 1 (local_rank: 1)
exitcode : 1 (pid: 13572)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html