Giter VIP home page Giter VIP logo

pointmlp-pytorch's Introduction

Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework (ICLR 2022)

PWC PWC

github

open review | arXiv | Primary contact: Xu Ma

Overview of one stage in PointMLP. Given an input point cloud, PointMLP progressively extracts local features using residual point MLP blocks. In each stage, we first transform the local point using a geometric affine module, and then local points are extracted before and after aggregation, respectively. By repeating multiple stages, PointMLP progressively enlarges the receptive field and models entire point cloud geometric information.

BibTeX

@article{ma2022rethinking,
    title={Rethinking network design and local geometry in point cloud: A simple residual MLP framework},
    author={Ma, Xu and Qin, Can and You, Haoxuan and Ran, Haoxi and Fu, Yun},
    journal={arXiv preprint arXiv:2202.07123},
    year={2022}
}

Model Zoo

Questions on ModelNet40 classification results (a common issue for ModelNet40 dataset in the community)

The performance on ModelNet40 of almost all methods are not stable, see (CVMI-Lab/PAConv#9 (comment)).
If you run the same codes for several times, you will get different results (even with fixed seed).
The best way to reproduce the results is to test with a pretrained model for ModelNet40.
Also, the randomness of ModelNet40 is our motivation to experiment on ScanObjectNN, and to report the mean/std results of several runs.


The codes/models/logs for submission version (without bug fixed) can be found here commit:d2b8dbaa.

On ModelNet40, fixed pointMLP achieves a result of 91.5% mAcc and 94.1% OA without voting, logs and pretrained models can be found [here].

On ScanObjectNN, fixed pointMLP achieves a result of 84.4% mAcc and 86.1% OA without voting, logs and pretrained models can be found [here]. Fixed pointMLP-elite achieves a result of 81.7% mAcc and 84.1% OA without voting, logs and pretrained models can be found [here].

Stay tuned. More elite versions and voting results will be uploaded.

News & Updates:

  • Apr/24/2024: University server is down. Update the ScanobjectNN dataset link.
  • fix the uncomplete utils in partseg by Mar/10, caused by error uplaoded folder.
  • upload test code for ModelNet40
  • update std bug (unstable testing in previous version)
  • paper/codes release

👉👉👉NOTE: The codes/models/logs for submission version (without bug fixed) can be found here commit:d2b8dbaa.

Install

# step 1. clone this repo
git clone https://github.com/ma-xu/pointMLP-pytorch.git
cd pointMLP-pytorch

# step 2. create a conda virtual environment and activate it
conda env create
conda activate pointmlp
# Optional solution for step 2: install libs step by step
conda create -n pointmlp python=3.7 -y
conda activate pointmlp
conda install pytorch==1.10.1 torchvision==0.11.2 cudatoolkit=10.2 -c pytorch -y
# if you are using Ampere GPUs (e.g., A100 and 30X0), please install compatible Pytorch and CUDA versions, like:
# pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
pip install cycler einops h5py pyyaml==5.4.1 scikit-learn==0.24.2 scipy tqdm matplotlib==3.4.2
pip install pointnet2_ops_lib/.

Useage

Classification ModelNet40

Train: The dataset will be automatically downloaded, run following command to train.

By default, it will create a folder named "checkpoints/{modelName}-{msg}-{randomseed}", which includes args.txt, best_checkpoint.pth, last_checkpoint.pth, log.txt, out.txt.

cd classification_ModelNet40
# train pointMLP
python main.py --model pointMLP
# train pointMLP-elite
python main.py --model pointMLPElite
# please add other paramemters as you wish.

To conduct voting testing, run

# please modify the msg accrodingly
python voting.py --model pointMLP --msg demo

Classification ScanObjectNN

The dataset will be automatically downloaded

  • Train pointMLP/pointMLPElite
cd classification_ScanObjectNN
# train pointMLP
python main.py --model pointMLP
# train pointMLP-elite
python main.py --model pointMLPElite
# please add other paramemters as you wish.

By default, it will create a fold named "checkpoints/{modelName}-{msg}-{randomseed}", which includes args.txt, best_checkpoint.pth, last_checkpoint.pth, log.txt, out.txt.

Part segmentation

  • Make data folder and download the dataset
cd part_segmentation
mkdir data
cd data
wget https://shapenet.cs.stanford.edu/media/shapenetcore_partanno_segmentation_benchmark_v0_normal.zip --no-check-certificate
unzip shapenetcore_partanno_segmentation_benchmark_v0_normal.zip
  • Train pointMLP
# train pointMLP
python main.py --model pointMLP
# please add other paramemters as you wish.

Acknowledgment

Our implementation is mainly based on the following codebases. We gratefully thank the authors for their wonderful works.

CurveNet, PAConv, GDANet, Pointnet2_PyTorch

LICENSE

PointMLP is under the Apache-2.0 license.

pointmlp-pytorch's People

Contributors

canqin001 avatar jacklangerman avatar ma-xu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

pointmlp-pytorch's Issues

Please provide test.py for part segmentation experiment

Hello!
Thanks for shairing the code.
Please provide the test/evaluation code for part segmentation experiment, or please share the method to evaluate the pre-trained model for this particular experiment.

Look forward to your reply.
Thanks

farthest_point_sample

I found in your pointmlp model that you have a function defined farthest_point_sample but you still use .furthest_point_sample function in the pointnet2_utils, and I want to ask, is there a difference between the two functions?

Some questions about the code

in LocalGrouper.forward

std = torch.std((grouped_points-mean).reshape(B,-1),dim=-1,keepdim=True).unsqueeze(dim=-1).unsqueeze(dim=-1)

It would seem not consistent with standard deviation of local k neighborhood points in the paper?

Impossible shape during ablation study

Hello, authors! I am confused about Table 5, where you did an ablation study including the scenario with Φpos and Affine but no Φpre.

Based on lines 337-340, for the complete model, the shape of x should look like follow during processing.
[b,g,k,d] -> Affine -> [b,g,k,d] -> Φpre -> [b,d,g] -> Φpos -> [b,d,g]

If we remove Affine, the following still works:
[b,g,k,d] -> Φpre -> [b,d,g] -> Φpos -> [b,d,g]

If we remove Φpos, the following also works:
[b,g,k,d] -> Affine -> [b,g,k,d] -> Φpre -> [b,d,g]

However, if we remove Φpre, the following would be impossible since Φpos cannot process an input x with a shape of [b,g,k,d].
[b,g,k,d] -> Affine -> [b,g,k,d] -> Φpos -> [b,d,g]

I hope you can clarify my doubts.
Thanks!
Shen Zheng

GEOMETRIC AFFINE MODULE

May I ask what part of the GEOMETRIC AFFINE MODULE code is? Looking forward to the author's answer

Hi,thanks for your great work.But I have some questions ,and I'd like to ask you.Thank you su much.

(pointmlp) D:\Downloads\pointmlp\pointnet2_ops_lib>python setup.py install
running install
D:\Anaconda\envs\pointmlp\lib\site-packages\setuptools\command\install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
setuptools.SetuptoolsDeprecationWarning,
D:\Anaconda\envs\pointmlp\lib\site-packages\setuptools\command\easy_install.py:147: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
EasyInstallDeprecationWarning,
running bdist_egg
running egg_info
writing pointnet2_ops.egg-info\PKG-INFO
writing dependency_links to pointnet2_ops.egg-info\dependency_links.txt
writing requirements to pointnet2_ops.egg-info\requires.txt
writing top-level names to pointnet2_ops.egg-info\top_level.txt
reading manifest file 'pointnet2_ops.egg-info\SOURCES.txt'
writing manifest file 'pointnet2_ops.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
running build_ext
error: [WinError 2] The system could not find the specified file.。

CUDA capability sm_86 is not compatible with the current PyTorch installation.

Hello.
I'd like to request about environment settings.

I set up conda env following README.md like below

# Optional solution for step 2: install libs step by step
conda create -n pointmlp python=3.7 -y
conda activate pointmlp
conda install pytorch==1.10.1 torchvision==0.11.2 cudatoolkit=10.2 -c pytorch -y
pip install cycler einops h5py pyyaml==5.4.1 scikit-learn==0.24.2 scipy tqdm matplotlib==3.4.2
pip install pointnet2_ops_lib/.

Then, I met CUDA capability sm_86 is not compatible with the current PyTorch installation. and code did not work anymore.

I checked environment.yml and I found that pytorch=1.8.1 and cudatoolkit=11.1. These are different from versions in README.md. I set up env following environment.yml and then everything goes well.

Thus, I think above installation guide should be modified like this.

# Optional solution for step 2: install libs step by step
conda create -n pointmlp python=3.7 -y
conda activate pointmlp
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
pip install cycler einops h5py pyyaml==5.4.1 scikit-learn==0.24.2 scipy tqdm matplotlib==3.4.2
pip install pointnet2_ops_lib/.

About Train speed and Test speed in Table 2

Hi,

Thanks for the excellent job. I'm confusing on how to obtain the speed in Table 2. When you get the train speed, is batch_size set to 1 or 32? If 1, how do you solve the problem caused by BN?

How to obtain the speed in Table 2?

Hi,

thanks for the excellent job. I'm confusing on how to obtain the speed in Table 2. For pointMLPElite, I set the test batch_size = 16, and it takes 5s to process the whole 2468 test samples(~494samples/s). With test batch_size=1, it takes about 24s -> ~103samples/s. Both results are far different from the 176samples/s in Table 2. Would you mind explaining the speedtest operation in details?

Add Skip connection in the segmentation task

Hi!Thank you for the great work. Now I want to use your net for the Semantic Segmentation task, so I want to modify the Part Segmentation model. It seems like your model architecture only replaced the PointNet's backbone. Can I add skip connection after the set abstraction operation(like pointnet++)? Have you tried the Semantic Segmentation?

pointnet2_ops_lib

Dear author, first of all thank you for your efforts, this is a good project. But I have some questions to ask you. I want to run pointMLP.py first. But when running the code, it often reports an error:
ModuleNotFoundError: No module named 'pointnet2_ops'
Then I read the Readme. Which
pip install pointnet2_ops_lib/. can not run.
There will be ERROR: Invalid requirement: 'pointnet2_ops_lib/pointMLP2_ops'
Hint: It looks like a path. File 'pointnet2_ops_lib/pointMLP2_ops' does not exist.
hint.
How can I solve it please? ? Looking forward to your perfect answer.

about multi gpu support

Hello !
Thank you for really nice work.
Please tell me how to run your code on a multi gpu machine? secondly, how much gpu memory is needed for segmentation's task? I have 12 gb gpu memory but when I put the model on training it gives cuda out of memory error right in the beginning, although the classification model trains fine on my machine,
Please guide.
Thanks

GEOMETRIC AFFINE MODULE

Regarding the part of the geometric mapping module, why does the point cloud show a normal distribution after such an operation?

can you all get paper's result?

I tried to collect deep learning on point cloud, I tried to run the model in my env, however I just got 92.99 acc on test set which is also a high score, but I cant get what paper says. So what's the reason of this phenomena

part segmentation

Hi,
Thank you for sharing your code! But I have some difficult in training ShapeNetPart segmentation. I found that there is no folder named 'util' under 'part_segmentation' , so I can't find 'to_categorical', 'PartNormalDataset' etc. And I guess Class 'ShapeNetPart' might be another version of 'PartNormalDataset', but it seems to have different parameter.

Use without CUDA?

Hello!
Would it be possible to use your code without the need of CUDA?

Install ''pip install pointnet2_ops_lib/.''

Thanks to the author for the code.
In the Install section of the Github page
''pip install pointnet2_ops_lib/.''
for what purpose? Is it a package or file?
Terminal display:
''ERROR: Invalid requirement: 'pointnet2_ops_lib/.'
Hint: It looks like a path. File 'pointnet2_ops_lib/.' does not exist.''

Thanks a lot for sharing the code! I have some questions about part segment.

Are the best parameters is the default parameters?Also I noticed that in the classification you use the default initialization, but in this task you use the xavier initialization method, is there any intuitive explanation for this? I hope to get your reply, maybe the parameter settings. Thank you very much for your contribution!

About ScanObjectNN

When i python main.py in classification_ScanObjectNN,show
File "D:\software\Anaconda\envs\pct\lib\site-packages\h5py_hl\files.py", line 199, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py\h5f.pyx", line 100, in h5py.h5f.open
FileNotFoundError: [Errno 2] Unable to open file (unable to open file: name = 'F:\Project\pointMLP-pytorch-main\classification_ScanObjectNN/data/h5_files/main_split/training_objectdataset_augmentedrot_scale75.
h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

Please how should i solve this problem

FPS and KNN are not hardware friendly and efficient

Thanks for your great work!
Although the proposed network is simple with just residual MLPs, Geometric Affine Module cannot be executed efficiently in hardware. It is because when I design ASIC for PointMLP, I find in FPS and KNN phases, points must be processed serially not in parallel. Can you provide some advice about the efficient process of FPS and KNN in hardware? Or, how FPS and KNN can be modified to be hardware friendly?

Thank you very much!

About partseg

Hello, is there any test code for component splitting? How should I see the visual result of component segmentation?

CUDA out of memory

Hi, thanks for your great work.

I'm trying to run PointMLP onto a RTX 3080 Ti, with 12 Gb of vRAM. After each stage the memory consumption of PointMLP increases a lot. I logged the sizes and memory used at each stages, this is what i got:

Stage n°1
 Input points...................:  [32, 1024, 3]
 Input features.................:  [32, 64, 1024]
 CUDA memory allocated..........:  67,613,184
 After geometric affine points..:  [32, 512, 3]
 After geometric affine features:  [32, 512, 24, 128]
 CUDA memory allocated..........:  578,729,472
 After pre extraction points....:  [32, 128, 512]
 CUDA memory allocated..........:  2,617,166,336
 After pos extraction features..:  [32, 128, 512]
 CUDA memory allocated..........:  2,684,279,296
------------------------------------------------------------
Stage n°2
 Input points...................:  [32, 512, 3]
 Input features.................:  [32, 128, 512]
 CUDA memory allocated..........:  2,684,279,296
 After geometric affine points..:  [32, 256, 3]
 After geometric affine features:  [32, 256, 24, 256]
 CUDA memory allocated..........:  3,190,775,296
 After pre extraction points....:  [32, 256, 256]
 CUDA memory allocated..........:  5,229,217,280
 After pos extraction features..:  [32, 256, 256]
 CUDA memory allocated..........:  5,296,334,336
------------------------------------------------------------
Stage n°3
 Input points...................:  [32, 256, 3]
 Input features.................:  [32, 256, 256]
 CUDA memory allocated..........:  5,296,334,336
 After geometric affine points..:  [32, 128, 3]
 After geometric affine features:  [32, 128, 24, 512]
 CUDA memory allocated..........:  5,801,241,088
 After pre extraction points....:  [32, 512, 128]
 CUDA memory allocated..........:  7,839,693,312
 After pos extraction features..:  [32, 512, 128]
 CUDA memory allocated..........:  7,906,818,560
------------------------------------------------------------
Stage n°4
 Input points...................:  [32, 128, 3]
 Input features.................:  [32, 512, 128]
 CUDA memory allocated..........:  7,906,818,560
 After geometric affine points..:  [32, 64, 3]
 After geometric affine features:  [32, 64, 24, 1024]
 CUDA memory allocated..........:  8,410,930,688

Traceback (most recent call last):
  File "original_respointMLP.py", line 389, in <module>
    out = model(data)
  File "/home/vedrenne/miniconda3/envs/dlgpucu113/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "original_respointMLP.py", line 343, in forward
    x = self.pre_blocks_list[i](x)  # [b,d,g]
  File "/home/vedrenne/miniconda3/envs/dlgpucu113/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "original_respointMLP.py", line 252, in forward
    x = self.operation(x)  # [b, d, k]
  File "/home/vedrenne/miniconda3/envs/dlgpucu113/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/vedrenne/miniconda3/envs/dlgpucu113/lib/python3.8/site-packages/torch/nn/modules/container.py", line 141, in forward
    input = module(input)
  File "/home/vedrenne/miniconda3/envs/dlgpucu113/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "original_respointMLP.py", line 224, in forward
    return self.act(self.net2(self.net1(x)) + x)
RuntimeError: CUDA out of memory. Tried to allocate 192.00 MiB (GPU 0; 11.77 GiB total capacity; 9.71 GiB already allocated; 147.56 MiB free; 9.81 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Note that the code run just fine on CPU, and that the Elite version works on GPU.
Is this the expected behavior ? If so how much vRAM is required with a batch size of 32 ? Also, it does seem weird to have such a high consumption for a rather tiny model; how would you explain that ?

PointMLP-Elite in part segmentation?

Hi and Thanks for your great work.

I'm just wondering if you have had or tested PointMLP-'Elite' version for part segmentation while you have them for classification works.
And if so, do you have any plan for sharing the codes as well? 🙏

Thank you. 😊

about visualization

Hi~ Can you provide the point cloud visualization code in Figure 2? Or tell me what tool was used to visualize? Thanks very much!!!

Screenshot from 2022-02-28 21-04-07

Number of parameters in the paper

Hi, thanks again for your work and your responsiveness in responding to issues.

I've got another minor issue, I'm sure it's a small misunderstanding from me, sorry in advance.
In your paper on Table 2 you claim that PointMLP and PointMLP-Elite have respectively 12.6 and 0.68 M parameters.
However if i run something like

print_num_params = lambda net: print(f"{sum(p.numel() for p in net.parameters() if p.requires_grad):,}")
print_num_params(pointMLP())
print_num_params(pointMLPElite())

I get 13,238,504 and 716,232 respectively.

Where does the difference come from ?

Thanks in advance !

Questions about training materials

  Whether it is ModelNet40 or objectdateset, their xyz starting coordinates seem to have been adjusted, such as taking the center of the object as (0,0,0) and normalizing, but pointnet++ is based on Farthest Point Sampling for sampling, so in fact our The coordinate axis, whether it is set to any point in space as the starting point (0, 0, 0) relative to the change of x, y, z, will not affect the model, because in terms of classification, the model is based on the 3D object The outer contour of the point cloud to determine what the object is (MLP based invariance)? (Of course, no matter how the coordinate axes are converted, they must be normalized)

  I would like to ask the boss, what do you think about this and whether it will affect PointMLP. Looking forward to your reply.

Missing keys while trying to load pretrained models

Hello,

I tried to load a pretrained models from here
https://web.northeastern.edu/smilelab/xuma/pointMLP/checkpoints/fixstd/modelnet40/pointMLP-20220209053148-404/
using the mlp_nocuda code shared here
https://web.northeastern.edu/smilelab/xuma/pointMLP/pointmlp_nocuda.py

I get this error

RuntimeError: Error(s) in loading state_dict for Model:
	Missing key(s) in state_dict: "embedding.net.0.weight", "embedding.net.1.weight", "embedding.net.1.bias", "embedding.net.1.running_mean", "embedding.net.1.running_var", "local_grouper_list.0.affine_alpha", "local_grouper_list.0.affine_beta", "local_grouper_list.1.affine_alpha", "local_grouper_list.1.affine_beta", "local_grouper_list.2.affine_alpha", "local_grouper_list.2.affine_beta", "local_grouper_list.3.affine_alpha", "local_grouper_list.3.affine_beta", "pre_blocks_list.0.transfer.net.0.weight", "pre_blocks_list.0.transfer.net.1.weight", "pre_blocks_list.0.transfer.net.1.bias", "pre_blocks_list.0.transfer.net.1.running_mean", "pre_blocks_list.0.transfer.net.1.running_var", "pre_blocks_list.0.operation.0.net1.0.weight", "pre_blocks_list.0.operation.0.net1.1.weight", "pre_blocks_list.0.operation.0.net1.1.bias", "pre_blocks_list.0.operation.0.net1.1.running_mean", "pre_blocks_list.0.operation.0.net1.1.running_var", "pre_blocks_list.0.operation.0.net2.0.weight", "pre_blocks_list.0.operation.0.net2.1.weight", "pre_blocks_list.0.operation.0.net2.1.bias", "pre_blocks_list.0.operation.0.net2.1.running_mean", "pre_blocks_list.0.operation.0.net2.1.running_var", "pre_blocks_list.0.operation.1.net1.0.weight", "pre_blocks_list.0.operation.1.net1.1.weight", "pre_blocks_list.0.operation.1.net1.1.bias", "pre_blocks_list.0.operation.1.net1.1.running_mean", "pre_blocks_list.0.operation.1.net1.1.running_var", "pre_blocks_list.0.operation.1.net2.0.weight", "pre_blocks_list.0.operation.1.net2.1.weight", "pre_blocks_list.0.operation.1.net2.1.bias", "pre_blocks_list.0.operation.1.net2.1.running_mean", "pre_blocks_list.0.operation.1.net2.1.running_var", "pre_blocks_list.1.transfer.net.0.weight", "pre_blocks_list.1.transfer.net.1.weight", "pre_blocks_list.1.transfer.net.1.bias", "pre_blocks_list.1.transfer.net.1.running_mean", "pre_blocks_list.1.transfer.net.1.running_var", "pre_blocks_list.1.operation.0.net1.0.weight", "pre_blocks_list.1.operation.0.net1.1.weight", "pre_blocks_list.1.operation.0.net1.1.bias", "pre_blocks_list.1.operation.0.net1.1.running_mean", "pre_blocks_list.1.operation.0.net1.1.running_var", "pre_blocks_list.1.operation.0.net2.0.weight", "pre_blocks_list.1.operation.0.net2.1.weight", "pre_blocks_list.1.operation.0.net2.1.bias", "pre_blocks_list.1.operation.0.net2.1.running_mean", "pre_blocks_list.1.operation.0.net2.1.running_var", "pre_blocks_list.1.operation.1.net1.0.weight", "pre_blocks_list.1.operation.1.net1.1.weight", "pre_blocks_list.1.operation.1.net1.1.bias", "pre_blocks_list.1.operation.1.net1.1.running_mean", "pre_blocks_list.1.operation.1.net1.1.running_var", "pre_blocks_list.1.operation.1.net2.0.weight", "pre_blocks_list.1.operation.1.net2.1.weight", "pre_blocks_list.1.operation.1.net2.1.bias", "pre_blocks_list.1.operation.1.net2.1.running_mean", "pre_blocks_list.1.operation.1.net2.1.running_var", "pre_blocks_list.2.transfer.net.0.weight", "pre_blocks_list.2.transfer.net.1.weight", "pre_blocks_list.2.transfer.net.1.bias", "pre_blocks_list.2.transfer.net.1.running_mean", "pre_blocks_list.2.transfer.net.1.running_var", "pre_blocks_list.2.operation.0.net1.0.weight", "pre_blocks_list.2.operation.0.net1.1.weight", "pre_blocks_list.2.operation.0.net1.1.bias", "pre_blocks_list.2.operation.0.net1.1.running_mean", "pre_blocks_list.2.operation.0.net1.1.running_var", "pre_blocks_list.2.operation.0.net2.0.weight", "pre_blocks_list.2.operation.0.net2.1.weight", "pre_blocks_list.2.operation.0.net2.1.bias", "pre_blocks_list.2.operation.0.net2.1.running_mean", "pre_blocks_list.2.operation.0.net2.1.running_var", "pre_blocks_list.2.operation.1.net1.0.weight", "pre_blocks_list.2.operation.1.net1.1.weight", "pre_blocks_list.2.operation.1.net1.1.bias", "pre_blocks_list.2.operation.1.net1.1.running_mean", "pre_blocks_list.2.operation.1.net1.1.running_var", "pre_blocks_list.2.operation.1.net2.0.weight", "pre_blocks_list.2.operation.1.net2.1.weight", "pre_blocks_list.2.operation.1.net2.1.bias", "pre_blocks_list.2.operation.1.net2.1.running_mean", "pre_blocks_list.2.operation.1.net2.1.running_var", "pre_blocks_list.3.transfer.net.0.weight", "pre_blocks_list.3.transfer.net.1.weight", "pre_blocks_list.3.transfer.net.1.bias", "pre_blocks_list.3.transfer.net.1.running_mean", "pre_blocks_list.3.transfer.net.1.running_var", "pre_blocks_list.3.operation.0.net1.0.weight", "pre_blocks_list.3.operation.0.net1.1.weight", "pre_blocks_list.3.operation.0.net1.1.bias", "pre_blocks_list.3.operation.0.net1.1.running_mean", "pre_blocks_list.3.operation.0.net1.1.running_var", "pre_blocks_list.3.operation.0.net2.0.weight", "pre_blocks_list.3.operation.0.net2.1.weight", "pre_blocks_list.3.operation.0.net2.1.bias", "pre_blocks_list.3.operation.0.net2.1.running_mean", "pre_blocks_list.3.operation.0.net2.1.running_var", "pre_blocks_list.3.operation.1.net1.0.weight", "pre_blocks_list.3.operation.1.net1.1.weight", "pre_blocks_list.3.operation.1.net1.1.bias", "pre_blocks_list.3.operation.1.net1.1.running_mean", "pre_blocks_list.3.operation.1.net1.1.running_var", "pre_blocks_list.3.operation.1.net2.0.weight", "pre_blocks_list.3.operation.1.net2.1.weight", "pre_blocks_list.3.operation.1.net2.1.bias", "pre_blocks_list.3.operation.1.net2.1.running_mean", "pre_blocks_list.3.operation.1.net2.1.running_var", "pos_blocks_list.0.operation.0.net1.0.weight", "pos_blocks_list.0.operation.0.net1.1.weight", "pos_blocks_list.0.operation.0.net1.1.bias", "pos_blocks_list.0.operation.0.net1.1.running_mean", "pos_blocks_list.0.operation.0.net1.1.running_var", "pos_blocks_list.0.operation.0.net2.0.weight", "pos_blocks_list.0.operation.0.net2.1.weight", "pos_blocks_list.0.operation.0.net2.1.bias", "pos_blocks_list.0.operation.0.net2.1.running_mean", "pos_blocks_list.0.operation.0.net2.1.running_var", "pos_blocks_list.0.operation.1.net1.0.weight", "pos_blocks_list.0.operation.1.net1.1.weight", "pos_blocks_list.0.operation.1.net1.1.bias", "pos_blocks_list.0.operation.1.net1.1.running_mean", "pos_blocks_list.0.operation.1.net1.1.running_var", "pos_blocks_list.0.operation.1.net2.0.weight", "pos_blocks_list.0.operation.1.net2.1.weight", "pos_blocks_list.0.operation.1.net2.1.bias", "pos_blocks_list.0.operation.1.net2.1.running_mean", "pos_blocks_list.0.operation.1.net2.1.running_var", "pos_blocks_list.1.operation.0.net1.0.weight", "pos_blocks_list.1.operation.0.net1.1.weight", "pos_blocks_list.1.operation.0.net1.1.bias", "pos_blocks_list.1.operation.0.net1.1.running_mean", "pos_blocks_list.1.operation.0.net1.1.running_var", "pos_blocks_list.1.operation.0.net2.0.weight", "pos_blocks_list.1.operation.0.net2.1.weight", "pos_blocks_list.1.operation.0.net2.1.bias", "pos_blocks_list.1.operation.0.net2.1.running_mean", "pos_blocks_list.1.operation.0.net2.1.running_var", "pos_blocks_list.1.operation.1.net1.0.weight", "pos_blocks_list.1.operation.1.net1.1.weight", "pos_blocks_list.1.operation.1.net1.1.bias", "pos_blocks_list.1.operation.1.net1.1.running_mean", "pos_blocks_list.1.operation.1.net1.1.running_var", "pos_blocks_list.1.operation.1.net2.0.weight", "pos_blocks_list.1.operation.1.net2.1.weight", "pos_blocks_list.1.operation.1.net2.1.bias", "pos_blocks_list.1.operation.1.net2.1.running_mean", "pos_blocks_list.1.operation.1.net2.1.running_var", "pos_blocks_list.2.operation.0.net1.0.weight", "pos_blocks_list.2.operation.0.net1.1.weight", "pos_blocks_list.2.operation.0.net1.1.bias", "pos_blocks_list.2.operation.0.net1.1.running_mean", "pos_blocks_list.2.operation.0.net1.1.running_var", "pos_blocks_list.2.operation.0.net2.0.weight", "pos_blocks_list.2.operation.0.net2.1.weight", "pos_blocks_list.2.operation.0.net2.1.bias", "pos_blocks_list.2.operation.0.net2.1.running_mean", "pos_blocks_list.2.operation.0.net2.1.running_var", "pos_blocks_list.2.operation.1.net1.0.weight", "pos_blocks_list.2.operation.1.net1.1.weight", "pos_blocks_list.2.operation.1.net1.1.bias", "pos_blocks_list.2.operation.1.net1.1.running_mean", "pos_blocks_list.2.operation.1.net1.1.running_var", "pos_blocks_list.2.operation.1.net2.0.weight", "pos_blocks_list.2.operation.1.net2.1.weight", "pos_blocks_list.2.operation.1.net2.1.bias", "pos_blocks_list.2.operation.1.net2.1.running_mean", "pos_blocks_list.2.operation.1.net2.1.running_var", "pos_blocks_list.3.operation.0.net1.0.weight", "pos_blocks_list.3.operation.0.net1.1.weight", "pos_blocks_list.3.operation.0.net1.1.bias", "pos_blocks_list.3.operation.0.net1.1.running_mean", "pos_blocks_list.3.operation.0.net1.1.running_var", "pos_blocks_list.3.operation.0.net2.0.weight", "pos_blocks_list.3.operation.0.net2.1.weight", "pos_blocks_list.3.operation.0.net2.1.bias", "pos_blocks_list.3.operation.0.net2.1.running_mean", "pos_blocks_list.3.operation.0.net2.1.running_var", "pos_blocks_list.3.operation.1.net1.0.weight", "pos_blocks_list.3.operation.1.net1.1.weight", "pos_blocks_list.3.operation.1.net1.1.bias", "pos_blocks_list.3.operation.1.net1.1.running_mean", "pos_blocks_list.3.operation.1.net1.1.running_var", "pos_blocks_list.3.operation.1.net2.0.weight", "pos_blocks_list.3.operation.1.net2.1.weight", "pos_blocks_list.3.operation.1.net2.1.bias", "pos_blocks_list.3.operation.1.net2.1.running_mean", "pos_blocks_list.3.operation.1.net2.1.running_var", "classifier.0.weight", "classifier.0.bias", "classifier.1.weight", "classifier.1.bias", "classifier.1.running_mean", "classifier.1.running_var", "classifier.4.weight", "classifier.4.bias", "classifier.5.weight", "classifier.5.bias", "classifier.5.running_mean", "classifier.5.running_var", "classifier.8.weight", "classifier.8.bias". 
	Unexpected key(s) in state_dict: "net", "epoch", "acc", "best_test_acc", "best_train_acc", "best_test_acc_avg", "best_train_acc_avg", "best_test_loss", "best_train_loss", "optimizer". 

Would appreciate any help on how to solve this (if possible).
Thanks in advance!

Alternate to pointnet2_ops_lib

Hi,

First of all, thank you for sharing the code for this paper.

I was not able to install pointnet2_ops_lib because of cuda version conflict in my local PC.
But I have found that the only place I need this is 'pointnet2_utils.furthest_point_sample'.
There is a commented alternate to this function in the code in the following line.

# fps_idx = torch.multinomial(torch.linspace(0, N - 1, steps=N).repeat(B, 1).to(xyz.device), num_samples=self.groups, replacement=False).long()

Could you please let me know what will happen if I use this ensted? Will this lower the accuracy of the model?

How to get Loss landscape

Hello authors! I really appreciate your excellent work. For Fig. 4 (the Loss landscape) in your paper, I want to know the following information:
(1) The plotting code script, package, or repository
(2) The testing dataset
(3) The model version
Thanks!
Shen Zheng

运行main.py出现错误

昨天跑起来一次,但是今天来了看到中途报错卡顿了(未拍照),重新运行时便提醒该错误,请问作者遇到过吗?该如何解决呢,百度了一下,但是并未找到解决方案
报错:KeyError:'PointMLP'
IMG_9530

ScanObjectNN evaluation

Hi,

Thanks for your great work! I really like this idea and your brilliant paper-writing! I have a quick question about ScanObjectNN evaluation.

I notice this dataset has many variants shown in their paper, such as *objectdataset.h5, *objectdataset_augmentedrot_scale75.h5. And I think the result in PointMLP is based on *objectdataset_augmentedrot_scale75.h5.

May I ask is this the default setting to use this variant for ScanObjectNN classification task? And do you think all the results here use the same variant?

Thank you!

RTX 3090 not compatible

I've set up the environment according to the instructions. But when I train the model on my dataset with this script:
python main.py --num_classes 5 --num_points 200 --model pointMLP --workers 1
An error occurs:

Traceback (most recent call last):
  File "main.py", line 273, in <module>
    main()
  File "main.py", line 84, in main
    net = net.to(device)
  File "/home/ccbien/.conda/envs/point-mlp/lib/python3.7/site-packages/torch/nn/modules/module.py", line 899, in to
    return self._apply(convert)
  File "/home/ccbien/.conda/envs/point-mlp/lib/python3.7/site-packages/torch/nn/modules/module.py", line 570, in _apply
    module._apply(fn)
  File "/home/ccbien/.conda/envs/point-mlp/lib/python3.7/site-packages/torch/nn/modules/module.py", line 570, in _apply
    module._apply(fn)
  File "/home/ccbien/.conda/envs/point-mlp/lib/python3.7/site-packages/torch/nn/modules/module.py", line 570, in _apply
    module._apply(fn)
  File "/home/ccbien/.conda/envs/point-mlp/lib/python3.7/site-packages/torch/nn/modules/module.py", line 593, in _apply
    param_applied = fn(param)
  File "/home/ccbien/.conda/envs/point-mlp/lib/python3.7/site-packages/torch/nn/modules/module.py", line 897, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA error: the provided PTX was compiled with an unsupported toolchain.
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

The following is the output of command 'nvidia-smi':

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.45.01    Driver Version: 455.45.01    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 3090    On   | 00000000:01:00.0 Off |                  N/A |
| 45%   53C    P0   141W / 350W |      1MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

I've tried with different versions of pytorch, cudatoolkit (10.2, 11.3) but got the same error.

@qingfeng0830

@qingfeng0830
Please check if 'F:\Project\pointMLP-pytorch-main\classification_ScanObjectNN/data/h5_files/main_split/training_objectdataset_augmentedrot_scale75.h5' exist.
Maybe path issue cause by Windows

Originally posted by @ma-xu in #13 (comment)

What is the computation complexity of PointMLP-elite?

Hi and Thanks for your great work very much.
I want to deploy the elite version on mobile devices. I have two questions:

  1. What is the computation complexity of PointMLP-elite? (FLOPS)
  2. How much accuracy loss if halfing the size of PointMLP-elite? (I want a more lightweight version)
    Thank you very much!

About ShapeNet Part evalaution

Hi,

Thank you for sharing the code.

I'm intersted in the segmentation experiment.

In fact, in the part_segmentation/main.py file, in line 13, you import this:

from util.util import to_categorical, compute_overall_iou, IOStream

But I could not find it in the repository, where can I get these functions?

Thank you in advance.

CUDA kernel failed

Hello! when I run the 'main.py', there is an error occured as following:

CUDA kernel failed : invalid device function
void furthest_point_sampling_kernel_wrapper(int, int, int, const float*, float*, int*) at L:228 in /home/deepmodel/lwq/pointMLP-pytorch/pointnet2_ops_lib/pointnet2_ops/_ext-src/src/sampling_gpu.cu

Before that, I have installed all the packages you provided. Do you know how to deal with this error above? Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.