jonkhler / s2cnn Goto Github PK

Hi,
Thank you so much for your wonderful work, really appreciate it! They are really easy to use. However, I did encounter some problems when trying to calculate the rotation R in SO(3) between two rotated spheres.

Basically, I followed the s2cnn/s2cnn/soft/s2_conv.py, and changed the torch kernel y with another sphere.
Now that Sphere1 and Sphere2 are passed into S2_fft_real.apply() with the results of Sphere1_FFT and Sphere2_FFT. The correlation result is then calculated by s2_mm(Sphere1, Sphere2) with slight modification in channels and shape. Then the correlation is passed to the SO3_ifft_real.apply() with the result argmax of the ZYZ angles.

I was wondering if this is the correct way of using the code to calculate rotations between two rotated spheres because by now the result seems incorrect.

Thanks in advance!!!

The computational cost of spherical convolution is too great. Is there any measure that makes spherical convolution can handle large images?

I'm trying to make the equivariance_plot script work on my own weather data of size (721, 1440).
When it runs the so3_rotation function I have the following error:

Traceback (most recent call last):
  File "/home/owen/anaconda3/envs/cupy-test/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3441, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-678a6fff5083>", line 1, in <module>
    runfile('/home/owen/PycharmProjects/cranfield/irp/ai.py', wdir='/home/owen/PycharmProjects/cranfield/irp')
  File "/home/owen/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/211.7628.24/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/home/owen/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/211.7628.24/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/owen/PycharmProjects/cranfield/irp/ai.py", line 109, in <module>
    main(date, cycle)
  File "/home/owen/PycharmProjects/cranfield/irp/ai.py", line 80, in main
    y1 = phi(s2_rotation(x, *abc))
  File "/home/owen/PycharmProjects/cranfield/irp/ai.py", line 15, in s2_rotation
    x = so3_rotation(x.view(*x.size(), 1).expand(*x.size(), x.size(-1)), a, b, c)
  File "/home/owen/PycharmProjects/cranfield/irp/s2cnn/soft/so3_rotation.py", line 18, in so3_rotation
    Us = _setup_so3_rotation(b, alpha, beta, gamma, device_type=x.device.type, device_index=x.device.index)
  File "/home/owen/PycharmProjects/cranfield/irp/s2cnn/soft/so3_rotation.py", line 67, in _setup_so3_rotation
    Us = __setup_so3_rotation(b, alpha, beta, gamma)
  File "/home/owen/PycharmProjects/cranfield/irp/s2cnn/utils/decorator.py", line 97, in wrapper
    result = func(*args)
  File "/home/owen/PycharmProjects/cranfield/irp/s2cnn/soft/so3_rotation.py", line 55, in __setup_so3_rotation
    Us = [wigner_D_matrix(l, alpha, beta, gamma,
  File "/home/owen/PycharmProjects/cranfield/irp/s2cnn/soft/so3_rotation.py", line 55, in <listcomp>
    Us = [wigner_D_matrix(l, alpha, beta, gamma,
  File "/home/owen/anaconda3/envs/cupy-test/lib/python3.8/site-packages/lie_learn/representations/SO3/wigner_d.py", line 63, in wigner_D_matrix
    D = rot_mat(alpha=alpha, beta=beta, gamma=gamma, l=l, J=Jd[l])
IndexError: index 151 is out of bounds for axis 0 with size 151

After trying to debug it, I have found that the issue is with the Jd matrix of size 151, defined in lie_learn when calling the line 55 in s2cnn/soft/so3_rotation.py

Us = [wigner_D_matrix(l, alpha, beta, gamma,
                          field='complex', normalization='quantum', order='centered', condon_shortley='cs')
          for l in range(b)]

b is set as b = x.size()[-1] // 2 so 720 in my case

the winger_D_matrix calls the rot_mat function which uses Jd[l], but since Jd is size 151, it throws the error.

Is there a way to redefine the Jd matrix or do I need to use a different function?

Hello

I did a simple experiment where I created a random signal x on SO(3) (I tried multiple bandwidths = 20,30,40). Then I computed the FFT using

x_fft = SO3_fft_real.apply(x)

Finally, I computed the IFFT using

x_hat = SO3_ifft_real.apply(x_fft)

I found that x and x_hat are very different from each other and appear to have no patterns that I can identify. For example, one is not a scaled version of the other.

Any help here is appreciated, It's possible I am just using the functions incorrectly.

Thanks
Suhas

Sorry to spam you all with multiple issues, but is there any usage documentation associated with your s2cnn package? In particular, I'm curious when to use the different grid types and convolution types. Things like so3_equatorial_grid() vs. so3_soft_grid(), etc.

I also notice you explicitly call so3_integrate() in the MNIST example. I am wondering why there is a need for explicit integration, and what the operation is doing (I couldn't find that in the papers).

Why does the S2Convolution layer require square inputs? Is there a way around this to explore the use of your convolutions on equirectangular images?

Thanks for your work. I know the differences between different grid. But I don't know how to choose different grid. It both seems Ok when I use it. Could you give me some advice about when should I use near identity grid or equatorial grid?

Hi, I'm trying to do something on my mac and it doesn't have any GPUs.

Could you please release a version that only to use the CPU?

Thank you so much!

The MNIST example depend on tensorflow to load the MNIST dataset.
With pytorch, the MNIST dataset can be loaded using torchvision.

When I run gendata.py with no rotation in training data and test data with the command :

        python3 gendata.py --no_rotate_train --no_rotate_test                

And I visualize some results, but some images generated seem to has nothing to do with the original images. Some results are like these:

I think the spherical images(right) are strange, there has nothing to do with the original images(left). I also test the algorithm on some other image:

Obviously, there is something wrong with gendata.py, but I can not figure out what is wrong, so can you explain it?

Hello,

I am trying to run mnist example, but i am getting many errors running script /examples/mnist/run.py. Seems like i have wrong version of some packages but cant figure out which versions are needed. Fixing one error leads to another error..

Some errors:
x = torch.fft(torch.stack((x, torch.zeros_like(x)), dim=-1), 2) TypeError: 'module' object is not callable

RuntimeError: view_as_complex is only supported for half, float and double tensors, but got a tensor of scalar type: ComplexFloat

I have installed all the dependencies:

conda create --name cuda9 python=3.6 
conda activate cuda9

# s2cnn deps
#conda install pytorch torchvision cuda90 -c pytorch # get correct command line at http://pytorch.org/
conda install -c anaconda cupy  
pip install pynvrtc joblib

# lie_learn deps
conda install -c anaconda cython  
conda install -c anaconda requests  

# shrec17 example dep
conda install -c anaconda scipy  
conda install -c conda-forge rtree shapely  
conda install -c conda-forge pyembree  
pip install "trimesh[easy]"  

python setup.py install

Any help? Thanks.

I am sorry to bother you many times.What is the role of the "wigner_D_matrix" function in the code? I can't get it based on contextual information.

This problem has been asked in issue 3, but I can't fix it.
After adding , encoding='utf-8', now the open() function in setup.py is like this:
long_description=open(os.path.join(os.path.dirname(__file__), "README.md"), encoding='utf-8').read(),

Then after runing python setup.py install, the encoding issue :
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1264: ordinal not in range(128)
is gone.
But when I want to run the example of shrec17, the following problem still remains:
module 's2cnn.ops.gpu.lib_cufft' has no attribute 'destroy'

I think may be the real problem occurs in the version of CUDA or pytorch.
Here's my system environment:

• Python 3.6
• CUDA 9.1.85
• pytorch 0.4.0

Can you help me?
Thanks a lot!

Hi,

Thank you for the amazing work!
I was wondering, just to be sure: you use spherical harmonics to handle some of your computations, right? I'm searching for a way to compute the spherical spectrum with torch bindings, and I thought that your library may already do the stuff.

Hi, I tried to use your s2conv/so3conv in multi model like following.
(Model includes your s2conv/so3conv)

def train(epoch):
    model.train()
    for batch_idx, (image,target) in enumerate(train_loader):
        image = image.to(device)
        optimizer.zero_grad()
       
        # multi model
        re_image1 = model(image)
        re_image2 = model(image)
        loss = re_image1.abs().mean() + re_image2.abs().mean()

        loss.backward()
        optimizer.step()

Then I got following error.

  File "main.py", line 66, in <module>
    main()
  File "main.py", line 62, in main
    train(epoch)
  File "main.py", line 53, in train
    loss.backward()
  File "/home/hayashi/.python-venv/lib/python3.5/site-packages/torch/tensor.py", line 93, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/hayashi/.python-venv/lib/python3.5/site-packages/torch/autograd/__init__.py", line 89, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

There are no error when I use mono-model like following

def train(epoch):
    model.train()
    for batch_idx, (image,target) in enumerate(train_loader):
        image = image.to(device)
        optimizer.zero_grad()
       
        # mono model
        image1 = model(image)
        loss = image1.abs().mean() 

        loss.backward()
        optimizer.step()

So I think this error is not caused from inplace operation.
Do you know this error's detail?

P.S.
I found this error doesn't occur when I use past version of your s2conv/so3conv.
(maybe this is for Pytorch v0.3.1)
If you can, please republish past version of s2cnn (for Pytorch v0.3.1).

hello, I find the dataset has been removed when i run example 'shrec17' project. where could i download the dataset? thanks.

Could you please clarify the difference between S2 convolutions and SO3 convolutions? From what I gather from the two papers, an S2 convolution should happen on a spherical input (e.g. an equirectangular image), but all subsequent convolutions should be SO3 convolutions. This is because, while the S2 convolution is an operation on the sphere (i.e. in S2), it involves a rotation in SO3. Hence, the output is a function of SO3 and requires a slightly different convolution operation. Is that accurate?

Nonetheless, in practice, does this just mean that we apply an S2 convolution to the input and SO3 convolutions to the subsequent feature maps?

Since the update for pytorch 0.4.0, the gradient of the layers S2Convolution and SO3Convolution are no longer computed.

model.parameters() returns all the parameters but list(model.parameters())[0].grad is None

Hi, I was going to run a simple example main.py.

The environment is: Ubuntu 16.04, python2.7.12 in Anaconda. cuda9.0.

And I have installed s2cnn successfully by

python setup.py install

The output of this command is:

running install
running bdist_egg
running egg_info
writing requirements to s2cnn.egg-info/requires.txt
writing s2cnn.egg-info/PKG-INFO
writing top-level names to s2cnn.egg-info/top_level.txt
writing dependency_links to s2cnn.egg-info/dependency_links.txt
reading manifest file 's2cnn.egg-info/SOURCES.txt'
writing manifest file 's2cnn.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying s2cnn/ops/gpu/lib_cufft/init.py -> build/lib.linux-x86_64-2.7/s2cnn/ops/gpu/lib_cufft
running build_ext
generating cffi module 'build/temp.linux-x86_64-2.7/s2cnn.ops.gpu.lib_cufft._lib_cufft.c'
already up-to-date
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/s2cnn
creating build/bdist.linux-x86_64/egg/s2cnn/utils
copying build/lib.linux-x86_64-2.7/s2cnn/utils/complex_utils.py -> build/bdist.linux-x86_64/egg/s2cnn/utils
copying build/lib.linux-x86_64-2.7/s2cnn/utils/init.py -> build/bdist.linux-x86_64/egg/s2cnn/utils
copying build/lib.linux-x86_64-2.7/s2cnn/utils/cuda_utils.py -> build/bdist.linux-x86_64/egg/s2cnn/utils
creating build/bdist.linux-x86_64/egg/s2cnn/nn
creating build/bdist.linux-x86_64/egg/s2cnn/nn/soft
copying build/lib.linux-x86_64-2.7/s2cnn/nn/soft/so3_rotation.py -> build/bdist.linux-x86_64/egg/s2cnn/nn/soft
copying build/lib.linux-x86_64-2.7/s2cnn/nn/soft/s2_conv.py -> build/bdist.linux-x86_64/egg/s2cnn/nn/soft
copying build/lib.linux-x86_64-2.7/s2cnn/nn/soft/so3_conv.py -> build/bdist.linux-x86_64/egg/s2cnn/nn/soft
creating build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu
copying build/lib.linux-x86_64-2.7/s2cnn/nn/soft/gpu/s2_fft.py -> build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu
copying build/lib.linux-x86_64-2.7/s2cnn/nn/soft/gpu/init.py -> build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu
copying build/lib.linux-x86_64-2.7/s2cnn/nn/soft/gpu/so3_fft.py -> build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu
copying build/lib.linux-x86_64-2.7/s2cnn/nn/soft/so3_integrate.py -> build/bdist.linux-x86_64/egg/s2cnn/nn/soft
copying build/lib.linux-x86_64-2.7/s2cnn/nn/soft/init.py -> build/bdist.linux-x86_64/egg/s2cnn/nn/soft
copying build/lib.linux-x86_64-2.7/s2cnn/nn/init.py -> build/bdist.linux-x86_64/egg/s2cnn/nn
creating build/bdist.linux-x86_64/egg/s2cnn/ops
copying build/lib.linux-x86_64-2.7/s2cnn/ops/s2_localft.py -> build/bdist.linux-x86_64/egg/s2cnn/ops
copying build/lib.linux-x86_64-2.7/s2cnn/ops/so3_localft.py -> build/bdist.linux-x86_64/egg/s2cnn/ops
creating build/bdist.linux-x86_64/egg/s2cnn/ops/gpu
copying build/lib.linux-x86_64-2.7/s2cnn/ops/gpu/torchcufft.py -> build/bdist.linux-x86_64/egg/s2cnn/ops/gpu
creating build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/lib_cufft
copying build/lib.linux-x86_64-2.7/s2cnn/ops/gpu/lib_cufft/_lib_cufft.so -> build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/lib_cufft
copying build/lib.linux-x86_64-2.7/s2cnn/ops/gpu/lib_cufft/init.py -> build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/lib_cufft
copying build/lib.linux-x86_64-2.7/s2cnn/ops/gpu/so3_mm.py -> build/bdist.linux-x86_64/egg/s2cnn/ops/gpu
copying build/lib.linux-x86_64-2.7/s2cnn/ops/gpu/s2_mm.py -> build/bdist.linux-x86_64/egg/s2cnn/ops/gpu
copying build/lib.linux-x86_64-2.7/s2cnn/ops/gpu/init.py -> build/bdist.linux-x86_64/egg/s2cnn/ops/gpu
copying build/lib.linux-x86_64-2.7/s2cnn/ops/init.py -> build/bdist.linux-x86_64/egg/s2cnn/ops
copying build/lib.linux-x86_64-2.7/s2cnn/init.py -> build/bdist.linux-x86_64/egg/s2cnn
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/utils/complex_utils.py to complex_utils.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/utils/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/utils/cuda_utils.py to cuda_utils.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/soft/so3_rotation.py to so3_rotation.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/soft/s2_conv.py to s2_conv.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/soft/so3_conv.py to so3_conv.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu/s2_fft.py to s2_fft.pyc
File "build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu/s2_fft.py", line 28
output = output.view(-1, *batch_size, 2) # [l * m, ..., complex] (nspec, ..., 2)
SyntaxError: only named arguments may follow *expression
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu/so3_fft.py to so3_fft.pyc
File "build/bdist.linux-x86_64/egg/s2cnn/nn/soft/gpu/so3_fft.py", line 29
output = output.view(-1, *batch_size, 2) # [l * m * n, ..., complex]
SyntaxError: only named arguments may follow *expression
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/soft/so3_integrate.py to so3_integrate.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/soft/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/nn/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/s2_localft.py to s2_localft.pyc
File "build/bdist.linux-x86_64/egg/s2cnn/ops/s2_localft.py", line 29
result = result.view(*sz[:-1], n_spectral, 2) # shape (..., n_spectral, 2)
SyntaxError: only named arguments may follow *expression
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/so3_localft.py to so3_localft.pyc
File "build/bdist.linux-x86_64/egg/s2cnn/ops/so3_localft.py", line 39
result = result.view(*sz[:-1], n_spectral, 2) # shape (..., n_spectral, 2)
SyntaxError: only named arguments may follow *expression
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/torchcufft.py to torchcufft.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/lib_cufft/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/so3_mm.py to so3_mm.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/s2_mm.py to s2_mm.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/init.py to init.pyc
creating stub loader for s2cnn/ops/gpu/lib_cufft/_lib_cufft.so
byte-compiling build/bdist.linux-x86_64/egg/s2cnn/ops/gpu/lib_cufft/_lib_cufft.py to _lib_cufft.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying s2cnn.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying s2cnn.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying s2cnn.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying s2cnn.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying s2cnn.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating 'dist/s2cnn-1.0.0-py2.7-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing s2cnn-1.0.0-py2.7-linux-x86_64.egg
Removing /home/zfq/anaconda2/lib/python2.7/site-packages/s2cnn-1.0.0-py2.7-linux-x86_64.egg
Copying s2cnn-1.0.0-py2.7-linux-x86_64.egg to /home/zfq/anaconda2/lib/python2.7/site-packages
s2cnn 1.0.0 is already the active version in easy-install.pth
Installed /home/zfq/anaconda2/lib/python2.7/site-packages/s2cnn-1.0.0-py2.7-linux-x86_64.egg
Processing dependencies for s2cnn==1.0.0
Searching for cffi==1.7.0
Best match: cffi 1.7.0
Adding cffi 1.7.0 to easy-install.pth file
Using /home/zfq/anaconda2/lib/python2.7/site-packages
Searching for pycparser==2.14
Best match: pycparser 2.14
Adding pycparser 2.14 to easy-install.pth file
Using /home/zfq/anaconda2/lib/python2.7/site-packages
Finished processing dependencies for s2cnn==1.0.0

And then I run

python examples/equivariance_error/main.py 

the error is:

Traceback (most recent call last):
File "examples/equivariance_error/main.py", line 12, in
from s2cnn.ops.s2_localft import equatorial_grid as s2_equatorial_grid
File "/home/zfq/s2cnn/s2cnn/ops/s2_localft.py", line 29
result = result.view(*sz[:-1], n_spectral, 2) # shape (..., n_spectral, 2)
SyntaxError: only named arguments may follow *expression

So could anyone please give me some suggestions about how to solve this problem? Thank you very much!

@mariogeiger In your provided MNIST example, running S2CNN encounters the following problem about lie_learn module.

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\Users\peng\anaconda3\envs\pytorch_1.8\lib\site-packages\lie_learn\representations\SO3\wigner_d.py", line 5, in <module>
    from lie_learn.representations.SO3.irrep_bases import change_of_basis_matrix
ModuleNotFoundError: No module named 'lie_learn.representations.SO3.irrep_bases'

Originally posted by @EricPengShuai in #52 (comment)

Hi,

Thanks for your great work! Your repo really helps me a lot! However, i am encountering some theoretical problems about the derivative of SO(3) fourier transformation when i tries to write the back propagation formula related to SO(3) fourier transformation. To be more specified, if i have

In your code, you write like this:

class SO3_fft_real(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x, b_out=None):  # pylint: disable=W
        ctx.b_out = b_out
        ctx.b_in = x.size(-1) // 2
        return so3_rfft(x, b_out=ctx.b_out)

    @staticmethod
    def backward(self, grad_output):  # pylint: disable=W
        # ifft of grad_output is not necessarily real, therefore we cannot use rifft
        return so3_ifft(grad_output, for_grad=True, b_out=self.b_in)[..., 0], None

It seems very intuitive that the partial derivative of SO(3) fft is SO(3) ifft, but i still wonders how you can get this. Can you help to illustrate this in details or is there any paper deriving this formula or discussing about this?

Thanks a lot for your help!!

Is there a way to visualize features maps of MNIST?
Thanks

The new release of pytorch changes lot of things.

Also now they provide FFT, so maybe we no longer need to compile you own FFT wrapper.

Hello,

Thanks for the great work! I meet some problem when I tried to specify a certain GPU on the server to run s2cnn, I encountered some issues. I found that s2cnn can only run successfully when I specify cuda:0 when using GPU. When I try to specify other GPU devices, such as cuda:1, the following error occurs:
File "/home/ly/.conda/envs/vcnn/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/soft/s2_conv.py", line 42, in forward
File "/home/ly/.conda/envs/vcnn/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/s2_mm.py", line 28, in s2_mm
File "/home/ly/.conda/envs/vcnn/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/s2_mm.py", line 65, in forward
File "/home/ly/.conda/envs/vcnn/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/s2_mm.py", line 127, in _cuda_s2_mm
File "/home/ly/.conda/envs/vcnn/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/s2_mm.py", line 198, in _setup_s2mm_cuda_kernel
File "/home/ly/.conda/envs/vcnn/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/utils/cuda.py", line 24, in compile_kernel
File "cupy/cuda/function.pyx", line 241, in cupy.cuda.function.Module.load
File "cupy/cuda/function.pyx", line 243, in cupy.cuda.function.Module.load
File "cupy_backends/cuda/api/driver.pyx", line 253, in cupy_backends.cuda.api.driver.moduleLoadData
File "cupy_backends/cuda/api/driver.pyx", line 125, in cupy_backends.cuda.api.driver.check_status
cupy_backends.cuda.api.driver.CUDADriverError: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered
Error in sys.excepthook:
How can I solve this?

Awesome work!

Just wanted to make a suggestion. I think that if you added some simple visualizations in your README like starting from the simple ones which symbol (alpha, beta, ...) corresponds to what, visually, it would be so much easier to parse the code and the paper.

Performance information across different resolutions would be invaluable as well.

Research like this where the paper is super math heavy would go a long way if the authors could create an accompanying blog where you could go past the X-page limit and try and intuitively explain it.

I personally, for example, am capable of understanding (which is different from saying that I actually understood it) every single equation in the paper, but since it's out of context for me I lose a lot of time just to understand something which I'd understand with a single look at a good visualization.

Again awesome/innovative work, and if you ever find some time to explain this a bit more visually it would be an amazing service to the community!

Hi! Thank you for this work!

Following paragraph 5.1 of your paper, I am trying to check the equivariance of the convolution layers. I have a question about Figure 3. Is the error in 10^-6 magnitude for the four plots?

MAIN ISSUE:
I tried to run the code below. It synthesizes 500 random feature maps and 500 random rotation around the Z-axis. Then, it measures the equivariance error:
mean(STD(Rotation(Layer(DATA)) - Layer(Rotation(DATA))) / STD(Layer(DATA)))

import torch
import numpy as np
from s2cnn import S2Convolution
from s2cnn import s2_near_identity_grid


device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")  # Device
b_in = 26  # Bandwidth of the input data
c = 10  # Number of feature maps
grid_s2 = s2_near_identity_grid(n_alpha=6, max_beta=np.pi/32, n_beta=1)  # S2 convolution
conv_S2 = S2Convolution(c, c, b_in, b_in, grid_s2)
conv_S2.to(device)


def phi(input, conv_S2):
    input = conv_S2(input)
    return input


def rot(input, angle):
    n = round(input.size(3) * angle / 360)  # rotate the signal around the Z axis
    return torch.cat([input[:, :, :, n:], input[:, :, :, :n]], dim=3)


relative_error = 0
n = 500
for i in range(n):
    with torch.no_grad():
        x = torch.randn(1, c, b_in*2, b_in*2).to(device)  # Random input [batch, feature, beta, alpha]
        angle = np.random.randint(0, 360)  # Random rotation around the Z-axis

        y = phi(x, conv_S2)  # Layer(DATA)
        y1 = rot(phi(x, conv_S2), angle=angle)  # Rotation(Layer(DATA))
        y2 = phi(rot(x, angle=angle), conv_S2)  # Layer(Rotation(DATA))
        relative_error += torch.std(y1.data - y2.data) / torch.std(y.data)

relative_error /= n
print('relative error = {}'.format(relative_error))

The layer is an S2 convolution. I fixed the input and output bandwidth to be equal. I observed that the equivariance error depends a lot on that bandwidth.

• For b_in<25, the error has a magnitude of 10^-7.

• For b_in>25, the error has a magnitude of 10^-1.

I understand that the 10^-7 error is due to numerical precision and means that the layer is equivariant. But I don't know what I do wrong to get a 10^-1 error for a larger bandwidth. I currently work with data with bandwidth 32 and my model doesn't seem to be equivariant due to this problem. Does it come from numerical precision? Can I do something to solve it?

NOTE:
I tried to run it on my CPU or GPU, but the problem is still here. I tried to change the size of the float but with no effect.

I also tried to change the output bandwidth to b_out=b_in//2.

• For b_in<4, the error has a magnitude of 10^-7.

• For b_in>4, the error has a magnitude of 10^-1.

First, thanks for the great work. Super interesting.

Wondering what node version to use? I had to edit the evaluator script as some things don't work with node v0.10.25.

Main issue:
After running test.py with no errors, summary.csv is empty. Furthermore, running ./evaluate.js ../myrun/ also doesn't work. The final printout is "Loaded Evaluator using file all.csv.gz
../my_run/val_perturbed/...". Any advice on what I should do?

NOTE:

I ran test.py with the -augmentations set to 1. Don't know if this would effect anything? I'll run it again with higher number of augmentations if necessary. Thanks!

Trying to run shrec17 example in a conda3 docker image with Python 3.6 [cuda-9.0 NVIDIA drivers 384.111] I faced the following error:

{'num_workers': 1, 'batch_size': 32, 'dataset': 'train', 'augmentation': 4, 'model_path': 'model.py', 'log_dir': 'my_run', 'learning_rate': 0.5}
Downloading http://3dvision.princeton.edu/ms/shrec17-data/train_perturbed.zip
Unzip data/train_perturbed.zip
Fix obj files
Downloading http://3dvision.princeton.edu/ms/shrec17-data/train.csv
Done!
402955 paramerters in total
5555 paramerters in the last layer
learning rate = 1 and batch size = 32
transform data/train_perturbed/046114.obj...
/root/miniconda3/lib/python3.6/site-packages/trimesh/triangles.py:188: RuntimeWarning: divide by zero encountered in true_divide
  center_mass = integrated[1:4] / volume
/root/miniconda3/lib/python3.6/site-packages/trimesh/triangles.py:188: RuntimeWarning: invalid value encountered in true_divide
  center_mass = integrated[1:4] / volume
transform data/train_perturbed/005351.obj...
transform data/train_perturbed/019736.obj...
transform data/train_perturbed/018758.obj...
transform data/train_perturbed/029336.obj...
transform data/train_perturbed/012867.obj...
transform data/train_perturbed/045223.obj...
transform data/train_perturbed/025009.obj...
transform data/train_perturbed/048329.obj...
transform data/train_perturbed/038370.obj...
transform data/train_perturbed/037326.obj...
transform data/train_perturbed/025172.obj...
transform data/train_perturbed/015628.obj...
transform data/train_perturbed/038990.obj...
transform data/train_perturbed/040417.obj...
transform data/train_perturbed/044571.obj...
transform data/train_perturbed/038458.obj...
transform data/train_perturbed/048180.obj...
transform data/train_perturbed/033437.obj...
transform data/train_perturbed/030847.obj...
transform data/train_perturbed/050627.obj...
transform data/train_perturbed/005628.obj...
transform data/train_perturbed/045656.obj...
transform data/train_perturbed/008172.obj...
transform data/train_perturbed/010100.obj...
transform data/train_perturbed/024292.obj...
transform data/train_perturbed/038671.obj...
transform data/train_perturbed/025215.obj...
transform data/train_perturbed/032604.obj...
transform data/train_perturbed/048823.obj...
transform data/train_perturbed/018781.obj...
transform data/train_perturbed/040830.obj...
transform data/train_perturbed/007740.obj...
Traceback (most recent call last):
  File "train.py", line 135, in <module>
    main(**args.__dict__)
  File "train.py", line 105, in main
    loss, correct = train_step(data, target)
  File "train.py", line 74, in train_step
    prediction = model(data)
  File "/root/miniconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "my_run/model.py", line 47, in forward
    x = self.sequential(x)  # [batch, feature, beta, alpha, gamma]
  File "/root/miniconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/miniconda3/lib/python3.6/site-packages/torch/nn/modules/container.py", line 67, in forward
    input = module(input)
  File "/root/miniconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/miniconda3/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6-linux-x86_64.egg/s2cnn/nn/soft/s2_conv.py", line 40, in forward
    x = S2_fft_real(b_out=self.b_out)(x) # [l * m, batch, feature_in, complex]
  File "/root/miniconda3/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6-linux-x86_64.egg/s2cnn/nn/soft/gpu/s2_fft.py", line 231, in forward
    return s2_fft(as_complex(x), b_out=self.b_out)
  File "/root/miniconda3/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6-linux-x86_64.egg/s2cnn/nn/soft/gpu/s2_fft.py", line 27, in s2_fft
    output = _s2_fft(x, for_grad=for_grad, b_in=b_in, b_out=b_out) # [l * m, batch, complex]
  File "/root/miniconda3/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6-linux-x86_64.egg/s2cnn/nn/soft/gpu/s2_fft.py", line 43, in _s2_fft
    plan = _setup_fft_plan(b_in, nbatch)
  File "/root/miniconda3/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6-linux-x86_64.egg/s2cnn/nn/soft/gpu/s2_fft.py", line 146, in _setup_fft_plan
    plan = Plan1d_c2c(N=2 * b, batch=nbatch * 2 * b)
  File "/root/miniconda3/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6-linux-x86_64.egg/s2cnn/ops/gpu/torchcufft.py", line 12, in __init__
    self.handler = cufft.plan1d_c2c(N, istride, idist, ostride, odist, batch)
AttributeError: module 's2cnn.ops.gpu.lib_cufft' has no attribute 'plan1d_c2c'
Exception ignored in: <bound method Plan1d_c2c.__del__ of <s2cnn.ops.gpu.torchcufft.Plan1d_c2c object at 0x7efc0f667438>>
Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6-linux-x86_64.egg/s2cnn/ops/gpu/torchcufft.py", line 18, in __del__
    cufft.destroy(self.handler)
AttributeError: module 's2cnn.ops.gpu.lib_cufft' has no attribute 'destroy'

Hello,

I'm trying to run the equivariance_plot but come up with this error, I am using CUDA 11 with pytorch and cupy for cuda 11:

 File "/home/owen/anaconda3/envs/s2cnn/lib/python3.8/site-packages/torch/functional.py", line 344, in einsum
    return _VF.einsum(equation, operands)  # type: ignore
RuntimeError: expected scalar type Float but found ComplexFloat

Full error log:

runfile('/home/owen/PycharmProjects/s2cnn/examples/equivariance_plot/main.py', wdir='/home/owen/PycharmProjects/s2cnn/examples/equivariance_plot')
/home/owen/anaconda3/envs/s2cnn/lib/python3.8/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370172916/work/c10/cuda/CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
compute 0.pkl.gz... save 0.pkl.gz... done
compute 0.pkl.gz... save 0.pkl.gz... done
Traceback (most recent call last):
  File "/home/owen/anaconda3/envs/s2cnn/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3437, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-34cedf137222>", line 1, in <module>
    runfile('/home/owen/PycharmProjects/s2cnn/examples/equivariance_plot/main.py', wdir='/home/owen/PycharmProjects/s2cnn/examples/equivariance_plot')
  File "/home/owen/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/211.7142.13/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/home/owen/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/211.7142.13/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/owen/PycharmProjects/s2cnn/examples/equivariance_plot/main.py", line 102, in <module>
    main()
  File "/home/owen/PycharmProjects/s2cnn/examples/equivariance_plot/main.py", line 76, in main
    y1 = phi(s2_rotation(x, *abc))
  File "/home/owen/PycharmProjects/s2cnn/examples/equivariance_plot/main.py", line 14, in s2_rotation
    x = so3_rotation(x.view(*x.size(), 1).expand(*x.size(), x.size(-1)), a, b, c)
  File "/home/owen/PycharmProjects/s2cnn/s2cnn/soft/so3_rotation.py", line 21, in so3_rotation
    x = SO3_fft_real.apply(x)  # [l * m * n, ..., complex]
  File "/home/owen/PycharmProjects/s2cnn/s2cnn/soft/so3_fft.py", line 453, in forward
    return so3_rfft(x, b_out=ctx.b_out)
  File "/home/owen/PycharmProjects/s2cnn/s2cnn/soft/so3_fft.py", line 110, in so3_rfft
    out = torch.einsum("bmn,zbmnc->mnzc", (wigner[:, s].view(-1, 2 * l + 1, 2 * l + 1), xx))
  File "/home/owen/anaconda3/envs/s2cnn/lib/python3.8/site-packages/torch/functional.py", line 342, in einsum
    return einsum(equation, *_operands)
  File "/home/owen/anaconda3/envs/s2cnn/lib/python3.8/site-packages/torch/functional.py", line 344, in einsum
    return _VF.einsum(equation, operands)  # type: ignore
RuntimeError: expected scalar type Float but found ComplexFloat

I was not able to find the TEST label for the SHREC17 Shape Retrieval and wondering how have you test your results?

When I run the command python setup.py install, I got output as follows

running install
running bdist_egg
running egg_info
writing s2cnn.egg-info/PKG-INFO
writing dependency_links to s2cnn.egg-info/dependency_links.txt
writing requirements to s2cnn.egg-info/requires.txt
writing top-level names to s2cnn.egg-info/top_level.txt
reading manifest file 's2cnn.egg-info/SOURCES.txt'
writing manifest file 's2cnn.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying s2cnn/ops/gpu/lib_cufft/init.py -> build/lib.linux-x86_64-3.6/s2cnn/ops/gpu/lib_cufft
running build_ext
generating cffi module 'build/temp.linux-x86_64-3.6/s2cnn.ops.gpu.lib_cufft._lib_cufft.c'
already up-to-date
building 's2cnn.ops.gpu.lib_cufft._lib_cufft' extension
gcc -pthread -B /home/whu/apps/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/whu/apps/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/whu/apps/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/whu/apps/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/whu/apps/anaconda3/include/python3.6m -c build/temp.linux-x86_64-3.6/s2cnn.ops.gpu.lib_cufft._lib_cufft.c -o build/temp.linux-x86_64-3.6/build/temp.linux-x86_64-3.6/s2cnn.ops.gpu.lib_cufft._lib_cufft.o
gcc -pthread -B /home/whu/apps/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/whu/apps/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/whu/apps/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/whu/apps/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/whu/apps/anaconda3/include/python3.6m -c /home/whu/Documents/s2cnn/s2cnn/ops/gpu/plan_cufft.c -o build/temp.linux-x86_64-3.6/home/whu/Documents/s2cnn/s2cnn/ops/gpu/plan_cufft.o
gcc -pthread -shared -B /home/whu/apps/anaconda3/compiler_compat -L/home/whu/apps/anaconda3/lib -Wl,-rpath=/home/whu/apps/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/build/temp.linux-x86_64-3.6/s2cnn.ops.gpu.lib_cufft._lib_cufft.o build/temp.linux-x86_64-3.6/home/whu/Documents/s2cnn/s2cnn/ops/gpu/plan_cufft.o -lcufft -o build/lib.linux-x86_64-3.6/s2cnn/ops/gpu/lib_cufft/_lib_cufft.abi3.so
/home/whu/apps/anaconda3/compiler_compat/ld: cannot find -lcufft
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

It seems that the current SO3Convolution and S2Convolution only support small bandwidth, e.g., b_in = 64.
Because when I change the bandwidth to 128 or 256, it's either memory error or overflow error.
Am I doing something wrong?

Please help.

Hi, I am a student now work on omnidirectional image processing. I am interested in this work. I have two questions about this. First I print the tensor shape after processing from the SO3conv layers, but as I know it should be like [batch size, channels, 2b, 2b] because the output of SO3conv should in the S2 domain? But what I got is [batch size, channels, 2b, 2b, 2*b]. And another question is that I want to use these layers to build an autoencoder, how can I project the signal back to the input image domain? I means from SO3 project back to S2. @jonas-koehler @mariogeiger @tscohen @AndreaPi @pochoi

I'm trying to modify the train.py of shrec17 example to multi-gpu version.

Here's the code:

loader = importlib.machinery.SourceFileLoader('model', os.path.join(log_dir, "model.py"))
mod = types.ModuleType(loader.name)
loader.exec_module(mod)
model = mod.Model(60)  
if os.path.exists("./my_run/"+mode+"_"+"state.pkl"):
        model.load_state_dict(torch.load("./my_run/"+mode+"_"+"state.pkl"))
        print("Loading pkl successfully!!!!")
model.cuda()
logger.info("{} paramerters in total".format(sum(x.numel() for x in model.parameters())))
logger.info("{} paramerters in the last layer".format(sum(x.numel() for x in model.out_layer.parameters())))
bw = model.bandwidths[0]

I don't know whether s2cnn is supporting multi-gpu now, but this error AttributeError: module 's2cnn.utils' has no attribute 'cuda' occured when I run the following command: CUDA_VISIBLE_DEVICES=7,6,5,4 python train.py --model_path model.py --log_dir my_run --mode train --batch_size 72 --learning_rate 0.01 --augmentation 5 --num_workers 1 --evalutation cv

Could you please give me some help, thank you so much!!!!
@mariogeiger @tscohen @jonas-koehler

Thank you for publishing such a great code !
I have a question.

When I use this Spherical convolution in our network,
I tried to train our model using on multi GPU like torch.nn.DataParallel(model).cuda()
However I got following error message.

  File "/home/users/.python_venv_3/lib/python3.5/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/users/.python_venv_3/lib/python3.5/site-packages/s2cnn-1.0.0-py3.5.egg/s2cnn/soft/s2_conv.py", line 40, in forward
  File "/home/users/.python_venv_3/lib/python3.5/site-packages/s2cnn-1.0.0-py3.5.egg/s2cnn/soft/gpu/s2_fft.py", line 225, in forward
  File "/home/users/.python_venv_3/lib/python3.5/site-packages/s2cnn-1.0.0-py3.5.egg/s2cnn/soft/gpu/s2_fft.py", line 27, in s2_fft
  File "/home/users/.python_venv_3/lib/python3.5/site-packages/s2cnn-1.0.0-py3.5.egg/s2cnn/soft/gpu/s2_fft.py", line 51, in _s2_fft
  File "cupy/cuda/function.pyx", line 147, in cupy.cuda.function.Function.__call__
  File "cupy/cuda/function.pyx", line 129, in cupy.cuda.function._launch
  File "cupy/cuda/driver.pyx", line 195, in cupy.cuda.driver.launchKernel
  File "cupy/cuda/driver.pyx", line 75, in cupy.cuda.driver.check_status
cupy.cuda.driver.CUDADriverError: CUDA_ERROR_INVALID_HANDLE: invalid resource handle

Can we use DataParallel for this spherical convolution?
Or is there a future plan to implement for this?

Hi all, thanks for your great effort in this research! I have a question on how to get the feature map of an equirectangular image input.

For example, I used the model S2ConvNet_deep in the MNIST example code to get feature map of an equirectangular image by cutting out the linear block at the end. I resized input panoramic image to 100x100 and the initial bandwith is 50. But the model returns an output in shape (1, 64). How can I get larger feature map like a VGG feature extractor?

Can you please share your precious comments?

Firstly, thank you for your great work!

Following your README.md, I installed the s2cnn successfully.

However, when I run example/equivariance_error.py, error occurs:

(cuda9) jiankai@fibonacci:~/data/Applications/s2cnn/examples/equivariance_error$ python main.py 
compute 0.pkl.gz... save 0.pkl.gz... done
Traceback (most recent call last):
  File "main.py", line 40, in <module>
    y = phi(x)
  File "main.py", line 27, in phi
    x = s2_conv(x)
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/soft/s2_conv.py", line 40, in forward
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/soft/s2_fft.py", line 251, in forward
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/soft/s2_fft.py", line 28, in s2_fft
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/soft/s2_fft.py", line 52, in _s2_fft
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/soft/s2_fft.py", line 201, in _setup_s2fft_cuda_kernel
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/s2cnn-1.0.0-py3.6.egg/s2cnn/utils/cuda.py", line 20, in compile_kernel
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/pynvrtc/compiler.py", line 52, in __init__
    include_names)
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/pynvrtc/interface.py", line 200, in nvrtcCreateProgram
    c_char_p(encode_str(src)), c_char_p(encode_str(name)),
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/pynvrtc/interface.py", line 54, in encode_str
    return s.encode("utf-8")
AttributeError: 'bytes' object has no attribute 'encode'
Exception ignored in: <bound method Program.__del__ of <pynvrtc.compiler.Program object at 0x7f9070bdb550>>
Traceback (most recent call last):
  File "/home/jiankai/miniconda3/envs/cuda9/lib/python3.6/site-packages/pynvrtc/compiler.py", line 56, in __del__
    self._interface.nvrtcDestroyProgram(self._program)
AttributeError: 'Program' object has no attribute '_program'

It seems that this problem is caused by pynvrtc, do you have any suggestions?

Thank you!

What is the actual meaning of bandwidth in the paper and how it should be set?What can I set and adjust based on?

Dear Jonas,

I request to kindly help me on how to run the code in MAC. As I'm not having a CUDA version for MAC.

Regards,
Jaideep Bommidi

Firstly, thank you for the great work!

As the paper mentioned The maximum frequency b is known as the bandwidth, and is related to the resolution of the spatial grid (Kostelec and Rockmore, 2007).

I notice that the bandwidth is set to 30 when you generate the new MNIST dataset. For each S2Convolution and SO3Convolution, the bandwidth is different.

I wonder how to decide the parameter bandwidth when we use S2Convolution or SO3Convolution? Is it also a super parameter (empirical value) or need to be calculated meticulously?

Why you set bandwidth = 30 when you generate the new MNIST dataset?

Thank you!

Hello,

Thanks for the great work! I have some issues with getting this code to run, starting with the example given in the repository. Basically, when I try to do python run.py in the mnist folder, I get a bunch of different errors that I feel like I shouldn't have to fix in order for the code to work since this is the basic example. The first error is this:

Traceback (most recent call last):
File "run.py", line 257, in
main(args.network)
File "run.py", line 221, in main
outputs = classifier(images)
File "/home/kiran/anaconda3/envs/OCT_latent_space/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "run.py", line 89, in forward
x = self.conv1(x)
File "/home/kiran/anaconda3/envs/OCT_latent_space/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/kiran/Desktop/Dev/s2cnn/mnist/s2cnn/soft/s2_conv.py", line 40, in forward
x = S2_fft_real.apply(x, self.b_out) # [l * m, batch, feature_in, complex]
File "/home/kiran/Desktop/Dev/s2cnn/mnist/s2cnn/soft/s2_fft.py", line 233, in forward
return s2_fft(as_complex(x), b_out=ctx.b_out)
File "/home/kiran/Desktop/Dev/s2cnn/mnist/s2cnn/soft/s2_fft.py", line 56, in s2_fft
output[s] = torch.einsum("bm,zbmc->mzc", (wigner[:, s], xx))
File "/home/kiran/anaconda3/envs/OCT_latent_space/lib/python3.8/site-packages/torch/functional.py", line 297, in einsum
return einsum(equation, *_operands)
File "/home/kiran/anaconda3/envs/OCT_latent_space/lib/python3.8/site-packages/torch/functional.py", line 299, in einsum
return _VF.einsum(equation, operands) # type: ignore[attr-defined]
RuntimeError: expected scalar type Float but found ComplexFloat

If I try to fix this, by setting xx to a real tensor then I get a bunch of errors down the line. I was wondering if anyone had any advice with this.

Great work! Thanks for releasing the code.

I was reading code about projection of meshes input onto sphere but I could understand the following preprocessing code:

        im[0] -= 0.75
        im[0] /= 0.26
        im[1] -= 0.59
        im[1] /= 0.50
        im[2] -= 0.54
        im[2] /= 0.29
        im[3] -= 0.52
        im[3] /= 0.19
        im[4] -= 0.80
        im[4] /= 0.18
        im[5] -= 0.51
        im[5] /= 0.25

I wonder where these constants come from and what's the meaning of this kind of translation and scaling?
Are these constants empirical mean and std of each channel? If it is, will these kinds of data preprocessing help a lot w.r.t. empirical performance/convergence?

Besides, I wonder if you have done any ablation study about the importance of each channel of input? For example, will the performance degrades much without convex hull input? What if without sin input channel(I feel like cos is enough to represent the angle between normal and ray)?

Hello,

There are some questions that confused me for weeks and I cannot solve them on my own, hope you guys can help me:

1. how the kernel rotate over the sphere, in which direction? (by the way, is there any gif to see or understand that?)
2. How many times does the filter rotate?
3. I know that the output of s2convolution is a signal on SO(3), but why? how to visualize or explain that?

Thanks, a lot!

Hi, thank you for sharing the code! I'm trying to generate spherical map from ShapeNetCore, and I have one question when running your shrec17 example, in file dataset.py in shrec17 in examples, from line 167 to 178
im[0] -= 0.75 im[0] /= 0.26 im[1] -= 0.59 im[1] /= 0.50 im[2] -= 0.54 im[2] /= 0.29 im[3] -= 0.52 im[3] /= 0.19 im[4] -= 0.80 im[4] /= 0.18 im[5] -= 0.51 im[5] /= 0.25
Are these lines a special case designed to normalize data on shrec17? Or what does these lines means? So I can move it safely. Thank you very much!

Can you add an example of implementation of the 5.4 part of your article please?

First thank you for publishing this useful code!
However, when I was implementing this code, I found that it run very slow with high CPU usage and no GPU usage. I finally find out that in some functions, such as _setup_s2_ft and _setup_s2_fft, there are for loops so that if the given bandwidth is a large number (I want to let it be 256), it takes very long time to run the function, like:

_setup_s2_fft(256, 256, True)... done in 10535 seconds
_setup_s2_ft(256, ((1.5707963267948966, 0.0), (1.5707963267948966, 0.01227184630308513), ...), device_type='cuda', device_index=0)... done in 11894 seconds

Did you ever tried a large bandwidth in practice? Is there any better way to obtain the results of these functions? (Maybe with utilization of GPU?)