Giter VIP home page Giter VIP logo

3dgcn's Introduction

Convolution in the Cloud: Learning Deformable Kernels in 3D Graph Convolution Networks for Point Cloud Analysis

created by Zhi-Hao Lin, Sheng-Yu Huang, Yu-Chiang Frank Wang from National Taiwan University, Taiwan.

This repository is based on our work at IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020, and paper is provided here: [paper]

teaser

Introduction

Point clouds are among the popular geometry representations for 3D vision applications. However, without regular structures like 2D images, processing and summarizing information over these unordered data points are very challenging. Although a number of previous works attempt to analyze point clouds and achieve promising performances, their performances would degrade significantly when data variations like shift and scale changes are presented. To this end, we propose 3D Graph Convolution Networks (3D-GCN), which is designed to extract local 3D features from point clouds across scales, while shift and scale-invariance properties are introduced.

Environment

Linux Ubuntu 18.04
Python 3.6.9
PyTorch 1.2.0

Installation

To run the code provided in this work, the following packages are required:

numpy
torch
pandas
matplotlib
pyntcloud

Please install packages with the command if necessary:

pip3 install -r requirements.txt

3D Graph Convolutional Network (3D-GCN)

Our algorithm is implemented in gcn3d.py, including convolutional and pooling layers. These modules are used as basic blocks for all network structures in different tasks. Please refer to model_gcn3d.py under each folder to understand the usage.

Shape Classification

3D-GCN is able to perform shape classification on point cloud objects. Please refer to classification/README for more training and testing details.

Shape Part-segmentation

3D-GCN is able to perform part-segmentation on point cloud objects. Please refer to segmentation/README for more training and testing details.

3dgcn's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

3dgcn's Issues

Visualization Issues

Hello!

Thanks for open-sourcing this amazing work! Now I have a question about the figures in the paper.

Could you tell me how to get the visualization results in Figure-5 of the paper? Specifically, how to draw the transparent sphere with a discrete point cloud inside the sphere? Which tool did you use to render the point clouds to get that visualization results?

Thank you so much for your help!

Performance analysis

Hi,

Thank you for the good paper, and for sharing the code.

I'd like to know if you have done a performance analysis in term of execution time. For example, how much does it take to train the classification or the segmentation method, or how mush does it take to do one iteration over the data?

Thank you in advance!

Validation dataset are need ?

Hello @j1a0m0e4sNTU ,
I successfully executed this program, but I have some questions about the allocation of datasets.
Under normal circumstances, the training dataset and the validation dataset are carried out together.
And in your program it seems that the test dataset is used for training and also for testing.

# classification/main.py
# Line 55~72
if args.mode == "train":
    print('Trianing ...')
    train_data = ModelNet_pointcloud(args.dataset, 'train', transform= transform)
    train_loader = DataLoader(train_data, shuffle= True, batch_size= args.bs)
    test_data = ModelNet_pointcloud(args.dataset, 'test', transform= transform)
    test_loader = DataLoader(test_data, shuffle= False, batch_size= args.bs)

    # I have a problem here
    manager.train(train_loader, test_loader)

else:
    print('Testing ...')
    test_data = ModelNet_pointcloud(args.dataset, 'test', transform= transform)
    test_loader = DataLoader(test_data, shuffle= False, batch_size= args.bs)
    
    # I have a problem here 
    test_loss, test_acc = manager.test(test_loader)

    print('Test Loss: {:.5f}'.format(test_loss))
    print('Test Acc:  {:.5f}'.format(test_acc))

Degradation of results when rotation

Hi,

Thank you for the great publication.

In fact, I'm a bit surprised that the results degrade if you rotate the objects, because rotation is a rigid transformation like the shift. If you rotate, the neighbor of the points doesn't change, as well as the distance between them, so how can you explain this degradation?

Thank you in advance for your answer!

AttributeError: 'NoneType' object has no attribute 'axis'

dear,when add the -output in the train.sh or test.sh, I meet the problem. Could you help me? Thanks.

epoch 1 step 1000 | avg loss: 1.747 | miou(c): 0.440 | miou(i): 0.563
epoch 1 step 2000 | avg loss: 1.232 | miou(c): 0.498 | miou(i): 0.631
epoch 1 step 3000 | avg loss: 1.001 | miou(c): 0.532 | miou(i): 0.663
Traceback (most recent call last):
File "main.py", line 67, in
main()
File "main.py", line 56, in main
manager.train(train_loader, test_loader)
File "=/Graph/3dgcn-master/segmentation/manager.py", line 118, in train
test_loss, test_table_str = self.test(test_data, self.out_dir)
File "/Graph/3dgcn-master/segmentation/manager.py", line 151, in test
self.save_visualizations(out_dir, cat_name, obj_ids, points, labels, pred)
File "/Graph/3dgcn-master/segmentation/manager.py", line 81, in save_visualizations
visualize(point, label, gt_fig_name)
File "/Graph/3dgcn-master/segmentation/visualize.py", line 26, in visualize
ax.axis('off')
AttributeError: 'NoneType' object has no attribute 'axis'

hardware problem

Hi, thank you very much for your contribution to this project. I have a question to ask: what does your project hardware look like. What is the graphics card memory for the training project? thanks

gcn3d : getting nan value

Hi,

Thanks for providing the code. I was trying to setting up this repository with my another dataset where a single point cloud has 30k points. But in forward pass in first layer itself ( gcn3d.Conv_surface) the data value are getting nan at some positions , and resulting in loss as nan. So I wonder where I am going wrong ?

Looking forward to reply.

Thanks,
Charvi

Segmentation results

I tried to recreate segmentation results but could achieve only
Best mIoU(c): 0.811, Best mIoU (i): 0.846

Please let me know if any changes need to be done to achieve the results in the paper
i.e. mIoU(c): 0.821, mIoU (i): 0.851

Thanks in Advance.

Classification results

Congrats on the paper.

I downloaded the code and trained for classification as suggested in the Readme file.
However, upon the test, I could only get a 90.8% classification accuracy.

Please let us know how to achieve 92.1% accuracy on classification as stated in the paper.

Thanks in advance.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.