I am currently attempting to implement a reinforcement learning task using the OCNN_PyTorch framework. I have noticed that the input to the network created by OCNN includes both features extracted from an octree and the octree itself. In the semantic segmentation example provided in OCNN, I observed the use of ocnn.octree.merge_octrees(octrees) followed by octree.construct_all_neigh() to aggregate a batch of octree data for processing by the network.

My task involves reinforcement learning, where each observation should be independent and not interfere with others. Does the merging operation of octrees still satisfy the requirements for reinforcement learning, where each step's observations are expected to be independent?

Thank you for your assistance.

This is a great implementation and I appreciate how easy you have made it to set up!

I currently have the nempty LeNet model parameter set to True. If I set nempty to False I get the following error.

model = LeNet(in_channels=3, out_channels=4, stages=3, nempty=False)

AssertionError: The shape of input data is wrong.

I have two questions:

1. What is the purpose of the nempty parameter?
2. Why is the input shape wrong when I enable nempty?

Thanks!

I notice that the downsampling layers in the ounet and unet models use ConvBnRelu as the downsamplers rather than pooling, which is typically used in such dense prediction/reconstruction networks to extract hierarchical features. Is there some relation between the two operations due to the structure of the octree itself?

it is weird that when I run the command :python classification.py --config configs/cls_m40_hrnet.yaml SOLVER.alias time
Pin memory thread exited unexpectedly,
but when I change the Lenet, everything is ok.

As I am seeing, this library only uses the 1D Conv layer in the module.
Is there any other implementation with higher dimensions for the layers?

Could you please send the documentation link or some basic examples of the model?

Thank you

Sorry for long message: I want to send my own data into the Lenet:
def genOct():
pod = np.loadtxt('/mnt/zxj/datasets/20221030chuchai/Bianxie_stl2xyz_train/0015/Upper/0.txt')
xyz, normal = pod[:, 0:3][0:4096], pod[:, 3:][0:4096]
centroid = np.mean(xyz, axis=0)
xyz = xyz - centroid
m = np.max(np.sqrt(np.sum(xyz ** 2, axis=1)))
xyz = xyz / m

xyz = xyz.clip(min=-1, max=1)

po = Points(torch.from_numpy(xyz), torch.from_numpy(normal)) # .permute(0,2,1)

octree = Octree(5, 2)
octree.build_octree(po)

octree.construct_all_neigh()
return octree

def main():
o1 =genOct() -->(for test) change the cloud point into <class 'ocnn.octree.octree.Octree'>
o2 = genOct()
o3 = genOct()
octree = ocnn.octree.merge_octrees([o1,o2,o3])
data = self.get_input_feature(octree).cuda()
logits = self.model(data, octree, octree.depth)

Traceback (most recent call last):
File "classification__.py", line 152, in
ClsSolver.main()
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/thsolver/solver.py", line 411, in main
cls.worker(0, FLAGS)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/thsolver/solver.py", line 400, in worker
the_solver.run()
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/thsolver/solver.py", line 375, in run
eval('self.%s()' % self.FLAGS.SOLVER.run)
File "", line 1, in
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/thsolver/solver.py", line 298, in train
self.train_epoch(epoch)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/thsolver/solver.py", line 156, in train_epoch
output = self.train_step(batch)
File "classification__.py", line 142, in train_step
loss, accu = self.forward(batch)
File "classification__.py", line 89, in forward
logits = self.model(data, octree, octree.depth)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/ocnn-2.1.9-py3.7.egg/ocnn/models/lenet.py", line 42, in forward
data = self.convs[i](data, octree, d)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/ocnn-2.1.9-py3.7.egg/ocnn/modules/modules.py", line 72, in forward
out = self.conv(data, octree, depth)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/ocnn-2.1.9-py3.7.egg/ocnn/nn/octree_conv.py", line 351, in forward
self.max_buffer)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/ocnn-2.1.9-py3.7.egg/ocnn/nn/octree_conv.py", line 223, in forward
octree_conv.setup(octree, depth)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/ocnn-2.1.9-py3.7.egg/ocnn/nn/octree_conv.py", line 86, in setup
self.buffer_h = self.neigh.shape[0]
AttributeError: 'NoneType' object has no attribute 'shape'
Exception in thread Thread-2:
Traceback (most recent call last):
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/torch/utils/data/_utils/pin_memory.py", line 28, in _pin_memory_loop
r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/site-packages/torch/multiprocessing/reductions.py", line 297, in rebuild_storage_fd
fd = df.detach()
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/multiprocessing/resource_sharer.py", line 57, in detach
with _resource_sharer.get_connection(self._id) as conn:
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/multiprocessing/resource_sharer.py", line 87, in get_connection
c = Client(address, authkey=process.current_process().authkey)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 498, in Client
answer_challenge(c, authkey)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 746, in answer_challenge
response = connection.recv_bytes(256) # reject large message
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 216, in recv_bytes
buf = self._recv_bytes(maxlength)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/home/kexin/anaconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

I 'm sure the cloud point is not None. Is my operation procedure wrong? If so, what are the correct steps

Considering the following written ResNet class;

import torch
import ocnn
from ocnn.octree import Octree


class ResNet(torch.nn.Module):
    r''' Octree-based ResNet for classification.
    '''

    def __init__(self, in_channels: int, out_channels: int, resblock_num: int,
                 stages: int, nempty: bool = False):
        super().__init__()
        self.in_channels = in_channels
        self.out_channels = out_channels
        self.resblk_num = resblock_num
        self.stages = stages
        self.nempty = nempty
        channels = [2 ** max(i+9-stages, 2) for i in range(stages)]

        self.conv1 = ocnn.modules.OctreeConvBnRelu(
            in_channels, channels[0], nempty=nempty)
        self.pool1 = ocnn.nn.OctreeMaxPool(nempty)
        self.resblocks = torch.nn.ModuleList([ocnn.modules.OctreeResBlocks(
            channels[i], channels[i+1], resblock_num, nempty=nempty)
            for i in range(stages-1)])
        self.pools = torch.nn.ModuleList([ocnn.nn.OctreeMaxPool(
            nempty) for i in range(stages-1)])
        self.global_pool = ocnn.nn.OctreeGlobalPool(nempty)
        # self.header = torch.nn.Linear(channels[-1], out_channels, bias=True)
        self.header = torch.nn.Sequential(
            ocnn.modules.FcBnRelu(channels[-1], 512),
            torch.nn.Dropout(p=0.5),
            torch.nn.Linear(512, out_channels))

    def forward(self, data: torch.Tensor, octree: Octree, depth: int):
        r''''''

        data = self.conv1(data, octree, depth)
        data = self.pool1(data, octree, depth)
        for i in range(self.stages-1):
            d = depth - i - 1
            data = self.resblocks[i](data, octree, d)
            data = self.pools[i](data, octree, d)
        data = self.global_pool(data, octree, depth-self.stages)
        data = self.header(data)
        return data

and the input ply file with the shape of torch.Size([841348, 3]).
And the calling ResNet is as follows;

from ocnn.octree import Octree

model = ResNet(in_channels=3, out_channels=3, resblock_num=1,stages=2)
model.forward(data=tensor_cloud, octree= Octree, depth=8)

We will have the following AttributeError;

AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_22/1135799818.py in <module>
      2 
      3 model = ResNet(in_channels=3, out_channels=3, resblock_num=1,stages=2)
----> 4 model.forward(data=tensor_cloud, octree= Octree, depth=8)

/tmp/ipykernel_22/2884427206.py in forward(self, data, octree, depth)
     36         r''''''
     37 
---> 38         data = self.conv1(data, octree, depth)
     39         data = self.pool1(data, octree, depth)
     40         for i in range(self.stages-1):

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1108         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1109                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1110             return forward_call(*input, **kwargs)
   1111         # Do not call functions when jit is used
   1112         full_backward_hooks, non_full_backward_hooks = [], []

/opt/conda/lib/python3.7/site-packages/ocnn/modules/modules.py in forward(self, data, octree, depth)
     70     r''''''
     71 
---> 72     out = self.conv(data, octree, depth)
     73     out = self.bn(out)
     74     out = self.relu(out)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1108         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1109                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1110             return forward_call(*input, **kwargs)
   1111         # Do not call functions when jit is used
   1112         full_backward_hooks, non_full_backward_hooks = [], []

/opt/conda/lib/python3.7/site-packages/ocnn/nn/octree_conv.py in forward(self, data, octree, depth)
    366           data, self.weights, octree, depth, self.in_channels,
    367           self.out_channels, self.kernel_size, self.stride, self.nempty,
--> 368           self.max_buffer)
    369 
    370     if self.use_bias:

/opt/conda/lib/python3.7/site-packages/ocnn/nn/octree_conv.py in forward(ctx, data, weights, octree, depth, in_channels, out_channels, kernel_size, stride, nempty, max_buffer)
    238     octree_conv = _OctreeConv(
    239         in_channels, out_channels, kernel_size, stride, nempty, max_buffer)
--> 240     octree_conv.setup(octree, depth)
    241     out = octree_conv.check_and_init(data)
    242     out = octree_conv.forward_gemm(out, data, weights)

/opt/conda/lib/python3.7/site-packages/ocnn/nn/octree_conv.py in setup(self, octree, depth)
     86       self.out_h = octree.nnum_nempty[self.out_depth]
     87     else:
---> 88       self.in_h = octree.nnum[self.in_depth]
     89       self.out_h = octree.nnum[self.out_depth]
     90       if self.stride == 2:

AttributeError: type object 'Octree' has no attribute 'nnum'

And I think we have to call the OCtree properly not the object!

Any idea?
Thanks

Hello,

I am currently working on a project that involves merging multiple octrees to create a complete map. I am encountering an issue with the consistency of the points data in the octree and would appreciate some guidance.

Problem Description
I initially believed that the points data in the octree referred to the center coordinates of the occupied nodes. To ensure consistent space partitioning in the world coordinate system across different octrees, I implemented the following steps:

Filtered point cloud data within a specified bounded space in the world coordinate system.
Added boundary extreme points to the point cloud data to ensure that the octree partitions the same space.
I expected that with identical point cloud data inputs, the center coordinates of the nodes in the octree (points data) would be consistent. However, I observed that even with manually input identical point cloud data, the points data from the octree differ each time.

Steps to Reproduce
Filter point cloud data within a bounded space.
Add boundary extreme points to the point cloud data.
Generate octrees from the identical point cloud data inputs.
Observations
Despite using identical point cloud data, the points data from the octree are not consistent across different runs.

Questions
Is my understanding of the points data in the octree correct? Does it represent the center coordinates of the occupied nodes?
Is there a flaw in my approach to ensuring consistent space partitioning across different octrees?
How can I achieve consistent points data for identical point cloud inputs in the octree?
Thank you for your assistance.

What are the key advantages of your library in comparison to the microsoft/O-CNN?
I could not find any related description in the documentation that tells the difference between these two libraries.

Thank you

Hi author,
Thanks for your pure torch implementation! Your recent work DualOctreeGNN uses ocnn version from microsoft/O-CNN. Can I replace it with this repo?

When I try to build an OCTree, it returns the following error.

oc_tree = Octree(depth=8, full_depth=2)
oc_tree.xyzb(8)

'NoneType' object has no attribute 'device'

The oc_tree has the following type

ocnn.octree.octree.Octree

Thanks for your great job!
I'm working on model deployment on jetson agx orin by tensorRT, and some octree ops (octree-conv/octree-deconv etc.) are used in my model. And I'm wondering that are those ops having a CUDA version?
Thanks again!

Hello,

I am currently using a batch size of 8 with around 2000 samples in my training set. I have noticed that when keeping all other hyperparameter static, if I increase the size of my training dataset to 4000 samples, I run into out of memory errors near the very end of the epoch (95% finished).

Do you have any idea why increase the number of samples may result in OOM even when the batch size remains the same?

I am using a DGCNN model with the same training loop and hyperparameters and do not run into this issue, so I am wondering if it is OCNN specific.

Let me know what you think.

Thanks,

Harry

I am interested in using transfer learning for OCNN-LeNet. Would you be able to provide a pre-trained model (e.g., .t7, .pth) file? I can't seem to find one online...

Getting an error when testing HRNet (ResNet and LeNet both work with exact same code):

out = ocnn.nn.octree_upsample(out, octree, d, self.nempty) AttributeError: module 'ocnn.nn' has no attribute 'octree_upsample'

It looks like octree_upsample does not exist in ocnn.nn

I think this might be a typo because I can't find any other instances of octree_upsample in the source code beyond its mention in the code shown above within the HRNet.

Hopefully this is an easy fix as I am keen to try out HRNet!

Thanks!

Summary

I was comparing the two main ways to handle batched points (creating batched points or merging octrees of different points) and realized a peculiarity where the members octre.nnum and octree.nnum_empty change from int32 to int64.

Question

It looks that the two methods are equivalent value-wise. Would the long type make a difference on the bit-operations?

Why it happens

Reproduce

import torch
import ocnn
from ocnn.octree import Points

def process_batch(batched_points, batched_features):
    def points2octree(points):
        octree = ocnn.octree.Octree(5, 2, batch_size=1, device=points.device)
        octree.build_octree(points)
        return octree

    # batched_points: (B, N, 3)
    B = batched_points.shape[0]
    batch_ids = torch.arange(B, device=batched_points.device).view(-1, 1).repeat(1, batched_points.shape[1])  # (B, N)
    list_points = [Points(points=batched_points[i], features=batched_features[i], batch_id=batch_ids[i], batch_size=1) for i in range(B)]
    points = [pts.cuda(non_blocking=True) for pts in list_points]
    octrees = [points2octree(pts) for pts in points]
    octree = ocnn.octree.merge_octrees(octrees)
    octree.construct_all_neigh()
    new_points = ocnn.octree.merge_points(points)
    return octree, new_points




def process_batch2(batched_xyz, batched_features):
    # batched_points: (B, N, 3)
    B = batched_xyz.shape[0]
    batch_ids = torch.arange(B, device=batched_xyz.device).view(-1, 1).repeat(1, batched_xyz.shape[1])  # (B, N)
    batched_points = Points(points=batched_xyz.reshape(-1, 3), features=batched_features.reshape(-1, 3), batch_id=batch_ids.view(-1), batch_size=B)
    octree = ocnn.octree.Octree(5, 2, batch_size=B, device=batched_xyz.device)
    octree.build_octree(batched_points)
    octree.construct_all_neigh()
    return octree, batched_points




al = torch.randn(4, 100, 3).cuda()
octree1, new_pts1 = process_batch(al, al)
octree2, new_pts2 = process_batch2(al, al)

print(torch.allclose(new_pts2.points, new_pts1.points))
print([torch.allclose(c1, c2) for c1, c2 in zip(octree1.children, octree2.children)])
print([torch.allclose(c1, c2) for c1, c2 in zip(octree1.keys, octree2.keys)])

print([torch.allclose(c1, c2) for c1, c2 in zip(octree1.points, octree2.points) if c1 is not None])
print([torch.allclose(c1, c2) for c1, c2 in zip(octree1.features, octree2.features) if c1 is not None])
print([torch.allclose(c1, c2) for c1, c2 in zip(octree1.nnum.to(dtype=torch.int32), octree2.nnum)])
print([torch.allclose(c1, c2) for c1, c2 in zip(octree1.nnum_nempty.to(dtype=torch.int32), octree2.nnum_nempty)])
print(octree1.nnum_nempty)
print(octree2.nnum_nempty)


torch.sum(octree2.nnum_nempty, dim=0).dtype # !!!

Output

True
[True, True, True, True, True, True]
[True, True, True, True, True, True]
[True]
[True]
[True, True, True, True, True, True]
[True, True, True, True, True, True]
tensor([  4,  32, 190, 359, 396, 400])
tensor([  4,  32, 190, 359, 396, 400], dtype=torch.int32)

Does this implement support adaptive O-CNN?

I am trying to increase the octree depth for LeNet from 5 (depth used in example), but I keep getting an input shape error as soon as the data hits the first linear layer in the network. With the same data and code, increasing the depth from 5 to 6 works with HRNet.

line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (16x32768 and 4096x128)

I am using a dataset with 4096 points and a batch size of 16. Any idea what might be causing this?

Hello I am using pytorch lightning 16-bit automatic mixed precision training (AMP) with OCNN HRNet and I am running into the error traceback:

grad_out = octree_conv.backward_gemm(grad_out, grad, weights) buffer = torch.mm(grad[start:end], weights.flatten(0, 1).t()) RuntimeError: expected scalar type Half but found Float

However, when I run the same code with OCNN-LeNet, it works just fine. Any idea why this might be happening?

I am trying organize my own dataset to fit the Net

xyz = np.loadtxt('/mnt/zxj/datasets/cangkou/cangsao_test_yazulen2/78248/Lower/45.xyz')  ------>>1. contain (x,y,z)
X_std = (xyz - xyz.min(axis=0)) / (xyz.max(axis=0) - xyz.min(axis=0))
xyz = X_std / (1+1)  -1
print('\nxyzxyzxyzxyzxyzxyzxyz', xyz.shape)    ------------------------------------------------------->2.(n,3)
po = Points(torch.from_numpy(xyz))#.permute(0,2,1)    ----------------------------------------->3. <class cnn.octree.points.Points'>       

li =[po]
li = ocnn.octree.merge_points(li)----------------------------------------------------------------->4.<class cnn.octree.points.Points'>   
octree = Octree(6).build_octree(li)------------------------------------------------------------> 5.<class 'torch.Tensor'>
data = self.get_input_feature(octree)

how can I convert the 4.<class cnn.octree.points.Points'> to not tensor? I am just following the doc ( following the the documen: thttps://ocnn-pytorch.readthedocs.io/en/latest/modules/octree.html)

@wang-ps I have a point cloud with N points and 3 channel features. Creating an octree of depth 4 with this point cloud creates an octree with features only at depth 4. How do I compute the features at a non-empty node in depth 3,2 and 1, for example, by setting the features at the parent node as the average of the feature values of the child nodes (without using a OctreeConvBnRelu block as a downsampler)?

The OCNN paper says that pooling is equivalent to downsampling the features to lower depths, so I tried just average pooling the features at depth 4 with a kernel 222 and stride 2, but the output doesn't have the same size as the number of nodes in depth 3...
E.g. I want to convolve the features at all layers of an octree separately, but the convolution cannot be run after the pooling since the number of nodes generated by pooling at depth d is not the same as nnum[d-1]:

self.pool = ocnn.nn.OctreeAvgPool(kernel_size=[2,2,2],stride=2,nempty=False)
convs = dict()
depth, full_depth = self.depth, self.full_depth
data = dict()
data[depth] = self.get_input_feature(octree,'ND')
for i,d in enumerate(range(depth, full_depth-1, -1)):
    convs[d] = self.encoder_blks[i](data[d],octree,d)
    if d>full_depth:
      data[d-1] = self.pool(data[d],octree,d)

where, the encoder blocks are OctreeResBlocks with fixed input size (channel size) and different output sizes.
Thanks in advance!

Hello, Professor Wang,
I noticed that in the code, the comments suggest using 0.1 and 1e-5 for certain parameters in PyTorch. However, it seems that in the actual usage, values like 0.01 and 0.001 are used. I was wondering about the reasoning behind this difference?

KeyError: 'Non-existent config key: DATA.test.flip'

Hi, I'm facing an issue about generating octree feature. I've already generated Octree upon point cloud successfully, with attributions children, neighbor etc, but the attribution features is a list of None.

When I try to generate feature data using ocnn.modules.InputFeature, it claims an error:

Exception has occurred: TypeError
unsupported operand type(s) for *: 'Tensor' and 'NoneType'
  File "/root/OpenPCDet/pcdet/models/backbones_3d/vfe/pillarmamba_vfe.py", line 805, in <module>
    data = octree_feature(octree)
TypeError: unsupported operand type(s) for *: 'Tensor' and 'NoneType'

I think these NoneType might come up due to features and normals, but I have o idea how should these two be generated. Thanks in advance.

Hi, thank you very much for your work! As we all know, the deployment of sparse convolution consistently is a very difficult problem.But in o-cnn all the ops are pytorch supported, so I would like to ask you if you have tried to export o-cnn to onnx? and if so, I would very much appreciate if you could provide an example, thanks!

Hello,

I am trying to use the HRNet adaptation with the following setup:

model = HRNet(dropout=0.5, in_channels=3, out_channels=4, stages=3, interp='linear', nempty=True)

I am also using an octree depth of 5 and a full depth of 2.

I am running into the following error:

Traceback (most recent call last): File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\multiprocessing\spawn.py", line 69, in _wrap fn(i, *args) File "D:\Sync\DL_Development\Scripts\DL_Biomass\Pytorch\utils\train_dl.py", line 235, in train_model pred = forward_pass(model, batch, rank, cfg) File "D:\Sync\DL_Development\Scripts\DL_Biomass\Pytorch\utils\train_dl.py", line 147, in forward_pass pred = model(features, octree, octree.depth) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\parallel\distributed.py", line 1040, in forward output = self._run_ddp_forward(*inputs, **kwargs) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\parallel\distributed.py", line 1000, in _run_ddp_forward return module_to_run(*inputs[0], **kwargs[0]) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "D:\Sync\DL_Development\Scripts\DL_Biomass\Pytorch\pytorch_models\ocnn_hrnet.py", line 192, in forward logits = self.cls_header(convs, octree, depth) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "D:\Sync\DL_Development\Scripts\DL_Biomass\Pytorch\pytorch_models\ocnn_hrnet.py", line 153, in forward logit = self.header(out) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\modules\container.py", line 204, in forward input = module(input) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\ocnn\modules\modules.py", line 171, in forward out = self.fc(out) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\hseely\Miniconda3\envs\pytorch\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x1024 and 512x256)

I am using pytorch Distributed Data Parallel (DDP). I am able to run LeNet with no issues.

Any idea what might be causing this?

Thanks!

-Harry

Greetings,

Thanks for the incredible work!

I'm experimenting with a setting where I would like to first downscale input voxel grid to /8 and then perform shape reconstruction from that point. For the encoding layer, I would prefer if network behaved like a Sparse net (nempty=True) but during upscaling, since I want it to perform reconstruction it should perform convolution on non-empty as well (nempty=False).

I have tried to test this approach with a simple script, but it is giving me shape error.

if __name__ == '__main__':
    conv = ocnn.nn.octree_conv.OctreeConv(3, 3, [3], 2, True, False, False)
    upconv = ocnn.nn.octree_conv.OctreeDeconv(3, 3, [2], 2, False)

    small_tree = ocnn.octree.Octree(3, full_depth=1)
    point = [0.2, 0.5, 0.5]
    point = torch.tensor(point).unsqueeze(0)
    small_tree.build_octree(ocnn.octree.points.Points(point))
    small_tree.construct_all_neigh()
    print("Octree shape:", octree2voxel(small_tree.get_input_feature(), small_tree, 3, True)[0].shape)
    encoding = ocnn.nn.octree_max_pool(conv(small_tree.get_input_feature(), small_tree, 3), small_tree, 3, nempty=True)
    print("Octree shape:", octree2voxel(encoding, small_tree, 2, True)[0].shape)

    # OK to this point

    reconstruction = upconv(encoding, small_tree, 2)  # Crash

I am trying to increase the depth of my octree to greater than 5, but I an running out of memory. As a workaround I would like to use the Pytorch multi GPU DataParallel wrapper. For However, I am running into these two errors when wrapping any OCNN model in nn.DataParallel:

AssertionError: The shape of input data is wrong.

DataParallel splits the data across the GPUs, which is in my case 2. The error is traced to this line of code in octree_conv.py:

check = tuple(data.shape) == self.in_shape

The tuple(data.shape) is (15646, 3) whereas self.in_shape is (31291, 3).

It appears that DataParallel is half working, because data.shape[0] * 2 = 31292 (1 different from self.in_shape). This means that the data is being split across GPUs, but the self.in_shape is not updated to match...

Any idea what might be stopping DataParallel from working in this case? Could you test this?

Thanks!

Hello,

I noticed that OCNN HRNet is more prone to overfitting the the LeNet implementation. This can be seen by the attached learning curve. I am assuming that this is in part due to the lack of dropout layers in the network. The dropout section of code in HRNet is commented out in the new release:

# self.header = torch.nn.Sequential( # ocnn.modules.FcBnRelu(512, 256), # torch.nn.Dropout(p=0.5), # torch.nn.Linear(256, out_channels))

Should I uncomment this code to initiate dropout? Why is it commented out?

Thanks!

@wang-ps, Referring to the following lines in octree.py code (in function octree_grow_full):

self.nnum[depth] = num * self.batch_size
self.nnum_nempty[depth] = num * self.batch_size

I was wondering why the number of non-empty nodes is updated when the octree is grown fully at a certain depth..
The number of non-empty nodes should depend on only the points input right?
As I understand it, the number of nodes (nnum) are the total number of nodes expanded/initialized at a particular depth and nnum_nempty is the number of non-empty nodes at a particular depth.
If I run the following code (adds just 1 point to octree at the center of a cube at resolution 4),

oct = ocnn.octree.Octree(4,4)
pts = np.array([[0,0,0]],dtype=np.float32)
pts+=0.5
pts*=1/(2**4)
points = ocnn.octree.Points(torch.from_numpy(pts))
oct.build_octree(points)
print(oct.nnum)
print(oct.nnum_nempty)

I would expect to see only 1 non-empty node at each depth, but instead I observe the following:

tensor([   1,    8,   64,  512, 4096], dtype=torch.int32)
tensor([  1,   8,  64, 512,   1], dtype=torch.int32)

As such, I am quite confused as to what the difference between nnum and nnum_nempty is..
Similarly, I also see that the number of children at depth d is 8^d. I would expect the number of nodes at depth d to be 8^d and the number of children to be 8^(d+1).
Please let me know if I'm misunderstanding something.
Thanks in advance!

Thank you for the great library. :)

Is there any script that could be used for visualizing the octree? That would be very useful for debugging purposes!

O-CNN constructs an octree based on point cloud. From my perspective, any high-dimensional data (e.g. dim>=3) could also has its octree through a somewhat different construction. For example, my data is a three-dimensional computed tomography image which has a size of 512^3. I want to borrow the idea of O-CNN and construct an octree for CT images. Is it possible?