Giter VIP home page Giter VIP logo

e3nn's Introduction

Euclidean neural networks

Coverage Status DOI

Documentation | Code | CHANGELOG | Colab

The aim of this library is to help the development of E(3) equivariant neural networks. It contains fundamental mathematical operations such as tensor products and spherical harmonics.

Installation

Important: install pytorch and only then run the command

pip install --upgrade pip
pip install --upgrade e3nn

For details and optional dependencies, see INSTALL.md

Breaking changes

e3nn is under development. It is recommanded to install using pip. The main branch is considered as unstable. The second version number is incremented every time a breaking change is made to the code.

0.(increment when backwards incompatible release).(increment for backwards compatible release)

Help

We are happy to help! The best way to get help on e3nn is to submit a Question or Bug Report.

Want to get involved? Great!

If you want to get involved in and contribute to the development, improvement, and application of e3nn, introduce yourself in the discussions.

Code of conduct

Our community abides by the Contributor Covenant Code of Conduct.

Citing

  • Euclidean Neural Networks
@misc{thomas2018tensorfieldnetworksrotation,
      title={Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds}, 
      author={Nathaniel Thomas and Tess Smidt and Steven Kearnes and Lusann Yang and Li Li and Kai Kohlhoff and Patrick Riley},
      year={2018},
      eprint={1802.08219},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1802.08219}, 
}

@misc{weiler20183dsteerablecnnslearning,
      title={3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data}, 
      author={Maurice Weiler and Mario Geiger and Max Welling and Wouter Boomsma and Taco Cohen},
      year={2018},
      eprint={1807.02547},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.02547}, 
}

@misc{kondor2018clebschgordannetsfullyfourier,
      title={Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network}, 
      author={Risi Kondor and Zhen Lin and Shubhendu Trivedi},
      year={2018},
      eprint={1806.09231},
      archivePrefix={arXiv},
      primaryClass={stat.ML},
      url={https://arxiv.org/abs/1806.09231}, 
}
  • e3nn
@misc{e3nn_paper,
    doi = {10.48550/ARXIV.2207.09453},
    url = {https://arxiv.org/abs/2207.09453},
    author = {Geiger, Mario and Smidt, Tess},
    keywords = {Machine Learning (cs.LG), Artificial Intelligence (cs.AI), Neural and Evolutionary Computing (cs.NE), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {e3nn: Euclidean Neural Networks},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

@software{e3nn,
  author       = {Mario Geiger and
                  Tess Smidt and
                  Alby M. and
                  Benjamin Kurt Miller and
                  Wouter Boomsma and
                  Bradley Dice and
                  Kostiantyn Lapchevskyi and
                  Maurice Weiler and
                  Michał Tyszkiewicz and
                  Simon Batzner and
                  Dylan Madisetti and
                  Martin Uhrin and
                  Jes Frellsen and
                  Nuri Jung and
                  Sophia Sanborn and
                  Mingjian Wen and
                  Josh Rackers and
                  Marcel Rød and
                  Michael Bailey},
  title        = {Euclidean neural networks: e3nn},
  month        = apr,
  year         = 2022,
  publisher    = {Zenodo},
  version      = {0.5.0},
  doi          = {10.5281/zenodo.6459381},
  url          = {https://doi.org/10.5281/zenodo.6459381}
}

Copyright

Euclidean neural networks (e3nn) Copyright (c) 2020, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy), Ecole Polytechnique Federale de Lausanne (EPFL), Free University of Berlin and Kostiantyn Lapchevskyi. All rights reserved.

If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Intellectual Property Office at [email protected].

NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit others to do so.

e3nn's People

Contributors

amorehead avatar bdice avatar bkmi avatar blondegeek avatar borisfom avatar colinkohler avatar dmadisetti avatar frellsen avatar jatentaki avatar jkhales avatar jnooree avatar joshrackers avatar l-sky avatar linux-cpp-lisp avatar marcelroed avatar mariogeiger avatar mister-bailey avatar mitkotak avatar mixarcid avatar mjwen avatar muhrin avatar r-barnes avatar sauravmaheshkar avatar simonbatzner avatar sophiaas avatar wouterboomsma avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

e3nn's Issues

`o3.selection_rule` should also include parity

Selection rules should also include whether change in parity is allowed. Then we can include this in rs.tensor_product.

e3nn/e3nn/o3.py

Lines 179 to 188 in 6d840ed

def selection_rule(l1, l2, lmax=None):
"""
selection rule
:return: list from |l1-l2|... to l1+l2
"""
if lmax is None:
l_max = l1 + l2
else:
l_max = min(lmax, l1 + l2)
return list(range(abs(l1 - l2), l_max + 1))

Add CG cache back

This feature seemed to disappear in the rewrite -- it would be nice to again default to a pre-computed cache.

Identifying pairwise distances in `RadialModel`

In my usage I need to back out which points a pairwise distance is associated with, for instance so that I can scale it with a pair-wise parameter that I optimize.
I noticed that with some of the recent changes, you started filtering out
radii <= eps here
https://github.com/e3nn/e3nn/blob/master/e3nn/kernel.py#L122
This way the RadialModel is not guaranteed to receive all pairwise distances,
and I can't trivially infer which i,j a distance corresponds to.

Two possible work-arounds come to mind:

  1. One could add the r_eps parameter to Convolution.forward so it can be passed
    on to Kernel.forward, I'd then run it with r_eps=-1
  2. One could pass the radii > r_eps mask to the RadialModel, as in
    R = self.R(radii[radii > r_eps], mask=(radii > r_eps))

Do you have any advice?

P.s.: absolutely <3 this work of yours.

Converting pytorch conv3d layers to e3nn layers

I am trying to understand how should I decide the Rs_in and Rs_out values for any specific PyTorch conv3d layer I want to convert to its equivariance version.

For eg: If I want to convert nn.conv3d(in_channels=4, out_channels=4, kernel_size=5, stride=1) to its closest equivalent e3nn version, how should I go about deciding Rs_in, Rs_out, size and steps parameters for e3nn.image Convolution.

Thanks

Resnet architecture using e3nn convolutions

Hi e3nn team,

I'm trying to build a resnet architecture on top of your convolutions, although i'm unsure if this is the right architecture as it's not giving me desirable results compared to normal convolution resnet. I have simply replaced the convolution of a normal resnet with your convolutions, but I'm unsure if that's the right way. Maybe I'm not using the suitable activation, or maybe should I make the network larger?

Can you please guide me on this, thanks

from e3nn.image.convolution import Convolution as  e3nn_conv3d
from e3nn.image.convolution import Transpose_Convolution as e3nn_transpose_conv3d

class Net3D_NOBN_Equi(nn.Module):
    def __init__(self, in_channel, pred_dim, chans=64):
        super(Net3D_NOBN_Equi, self).__init__()
        conv3d = []
        up_bn = [] #batch norm layer for deconvolution
        conv3d_transpose = []


        self.down_in_dims = [in_channel, chans, 2*chans]
        self.down_out_dims = [chans, 2*chans, 4*chans]
        self.down_ksizes = [4, 4, 4]
        self.down_strides = [2, 2, 2]
        padding = 1 
        print('down dims: ', self.down_out_dims)

        for i, (in_dim, out_dim, ksize, stride) in enumerate(zip(self.down_in_dims, self.down_out_dims, self.down_ksizes, self.down_strides)):
            stride_tup = (stride, stride, stride)
            in_tup = [(in_dim, 0)]
            out_tup = [(out_dim, 0)]
            conv3d.append(nn.Sequential(
                e3nn_conv3d(in_tup, out_tup,size=ksize, stride=stride_tup, padding=padding),
                nn.LeakyReLU(),
                # nn.BatchNorm3d(num_features=out_dim),
            ))
        self.conv3d = nn.ModuleList(conv3d)

        self.up_in_dims = [4*chans, 6*chans]
        self.up_out_dims = [4*chans, 2*chans]
        self.up_ksizes = [4, 4]
        self.up_strides = [2, 2]
        padding = 1 
        for i, (in_dim,  out_dim, ksize, stride) in enumerate(zip(self.up_in_dims,  self.up_out_dims, self.up_ksizes, self.up_strides)):
            stride_tup = (stride, stride, stride)
            in_tup = [(in_dim, 0)]
            out_tup = [(out_dim, 0)]             
            conv3d_transpose.append(nn.Sequential(
                e3nn_transpose_conv3d(out_tup,in_tup,size=ksize,stride=stride_tup, padding=padding),
                nn.LeakyReLU(),
            ))
        self.conv3d_transpose = nn.ModuleList(conv3d_transpose)
        stride_tup = (1, 1, 1)
        in_tup = [(3 * chans, 0)]
        out_tup = [(pred_dim, 0)]
        ksize = 3


        self.final_feature = nn.Conv3d(in_channels=3*chans, out_channels=pred_dim, kernel_size=1, stride=1, padding=0)

        
    def forward(self, inputs):
        feat = inputs
        skipcons = []
        for conv3d_layer in self.conv3d:
            feat = conv3d_layer(feat)
            skipcons.append(feat)
        skipcons.pop() # we don't want the innermost layer as skipcon

        for i, conv3d_transpose_layer in enumerate(self.conv3d_transpose):
            feat = conv3d_transpose_layer(feat)
            feat = torch.cat([feat, skipcons.pop()], dim=1) #skip connection by concatenation
        feat = self.final_feature(feat)
       return feat

Question about equivariance in point convolution layer

Hello, I just read paper about SE(3) equivariant neural network (3D steerable CNN, Tensor field network, etc) and test this repository.

First, I try to play with example code in the front page of this github.
What I tried is add random rotation in the example code like below.

# unmentioned part of example code is same with that in front page

from e3nn.o3 import rand_rot

Rs_in = [(3, 0)]  # Input = Three scalars
Rs_out = [(1, 1)]  # Output = One single vector

features = rs.randn(1, n, Rs_in, requires_grad=True)
in_geometry = torch.randn(1, n, 3)
out_geometry = torch.zeros(1, 1, 3)  # One point at the origin

rot_in_geometry = torch.bmm(rand_rot().unsqueeze(0), in_geometry.transpose(1,2)).transpose(1,2)

out = norm(conv(features, in_geometry, out_geometry)
out_rot = norm(conv(features, rot_in_geometry, out_geometry))

In that case, out and out_rot show same result.
However, when I try to add vector in Rs_in like below, I cannot get equivariant result (out != out_rot).

# unmentioned part of example code is same with that in front page

from e3nn.o3 import rand_rot

Rs_in = [(1, 1)]  # Input = One single vector
Rs_out = [(1, 1)]  # Output = One single vector

features = rs.randn(1, n, Rs_in, requires_grad=True)
in_geometry = torch.randn(1, n, 3)
out_geometry = torch.zeros(1, 1, 3)  # One point at the origin

rot_matrix = rand_rot()

rot_features = torch.bmm(rot_matrix.unsqueeze(0), features.transpose(1,2)).transpose(1,2)
rot_in_geometry = torch.bmm(rot_matrix.unsqueeze(0), in_geometry.transpose(1,2)).transpose(1,2)

out = norm(conv(features, in_geometry, out_geometry)
out_rot = norm(conv(rot_features, rot_in_geometry, out_geometry))

Is this result is correct? What I understand is the norm of the outputs from se(3)-equivariant layer with se(3) transformed inputs always same. However, the result confuses me. I know this question is rather basic but I need someone's help.

lie_learn error when calling o3.irr_repr

When I call o3.irr_repr, I get this error associate with lie_learn trying to grab Wigner D matrix.

`---------------------------------------------------------------------------
EOFError Traceback (most recent call last)
~/anaconda3/envs/e3nn/lib/python3.8/site-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
459 try:
--> 460 return pickle.load(fid, **pickle_kwargs)
461 except Exception:

EOFError: Ran out of input

During handling of the above exception, another exception occurred:

OSError Traceback (most recent call last)
in
11
12 # Wigner D matrix -- rotation matrix for irreducible representations
---> 13 wignerD = o3.direct_sum(*[o3.irr_repr(l, *angles) for mul, l, parity in Rs_3x3 for _ in range(mul)])
14 print("Shape of Wigner-D matrix:", wignerD.shape)
15

in (.0)
11
12 # Wigner D matrix -- rotation matrix for irreducible representations
---> 13 wignerD = o3.direct_sum(*[o3.irr_repr(l, *angles) for mul, l, parity in Rs_3x3 for _ in range(mul)])
14 print("Shape of Wigner-D matrix:", wignerD.shape)
15

~/anaconda3/envs/e3nn/lib/python3.8/site-packages/e3nn/o3.py in irr_repr(order, alpha, beta, gamma, dtype, device)
142 - compatible with compose and spherical_harmonics
143 """
--> 144 from lie_learn.representations.SO3.wigner_d import wigner_D_matrix
145 abc = [alpha, beta, gamma]
146 for i, x in enumerate(abc):

~/anaconda3/envs/e3nn/lib/python3.8/site-packages/lie_learn/representations/SO3/wigner_d.py in
2 import numpy as np
3
----> 4 from lie_learn.representations.SO3.pinchon_hoggan.pinchon_hoggan_dense import Jd, rot_mat
5 from lie_learn.representations.SO3.irrep_bases import change_of_basis_matrix
6

~/anaconda3/envs/e3nn/lib/python3.8/site-packages/lie_learn/representations/SO3/pinchon_hoggan/pinchon_hoggan_dense.py in
15 # https://iopscience.iop.org/article/10.1088/1751-8113/40/7/011/
16
---> 17 Jd = download('https://github.com/AMLab-Amsterdam/lie_learn/releases/download/v1.0/J_dense_0-278.npy')
18
19 def SO3_irreps(g, irreps):

~/anaconda3/envs/e3nn/lib/python3.8/site-packages/lie_learn/representations/SO3/pinchon_hoggan/download.py in download(url)
9 if not os.path.isfile(path):
10 open(path, 'wb').write(requests.get(url).content)
---> 11 return np.load(path, encoding='latin1', allow_pickle=True)

~/anaconda3/envs/e3nn/lib/python3.8/site-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
460 return pickle.load(fid, **pickle_kwargs)
461 except Exception:
--> 462 raise IOError(
463 "Failed to interpret file %s as a pickle" % repr(file))
464 finally:

OSError: Failed to interpret file '/Users/jracker/anaconda3/envs/e3nn/lib/python3.8/site-packages/lie_learn/representations/SO3/pinchon_hoggan/J_dense_0-278.npy' as a pickle`

Impose mul=1 in SphericalTensor

Everything can be put into the batch indices.

So far we allow signal of type [..., l * mul * m]
I propose to simply allow [..., l * m] to start with

Remove, update, test

  • util/plot.py looks deprecated to me
  • groupnorm.py needs tests
  • spherical_harmonics.py needs tests and maybe renaming

`SphericalTensor.plot` with `radius=True` is not plotting with correct magnitude

I'm re-running e3nn_tutorial and it seems that the plot method in SphericalTensor is not producing the correct magnitude of the signal (see attached example from simple_tasks_and_symmetry.ipynb). Could it be something is different between spherical harmonic conventions in s2grid and those of the spherical_tensor.projection?

simple_tasks_and_symmetry_-_Jupyter_Notebook

add functions for manipulating spherical tensors

add functions for rearranging spherical tensors such as rearrange, split, sort, simplify which will map an input Rs to a specified or default (in the case of split, sort, and simplify) Rs. These functions may be considered utilities.

Kernel not pickleable

In the process of integrating e3nn into schnetpack it came to my attention that the Kernel, Radial Model, and potentially more are not pickleable. The main reason for this is due to the definition of local functions within classes and use of lambdas.

I believe adding this feature would be worthwhile since it increases the usability of e3nn. In schnetpack, the best trained model is saved using pickle. To use a pickleable model in another context one merely has to unpickle the model. As e3nn currently works, one must keep a script around which sets the model up correctly then load the state dict.

Compatibility with pytorch 1.5.0

Here is some of the summary when using pytest with pytorch==1.5.0

FAILED equivariance_test.py::Tests::test_parity_gated_block_parity - RuntimeError: view size is not compatible with input tensor's size and stride (at...
FAILED equivariance_test.py::Tests::test_parity_gated_block_parity_mod - RuntimeError: view size is not compatible with input tensor's size and stride...
FAILED equivariance_test.py::Tests::test_parity_rotation_gated_block_parity - RuntimeError: view size is not compatible with input tensor's size and s...
FAILED equivariance_test.py::Tests::test_parity_rotation_gated_block_parity_mod - RuntimeError: view size is not compatible with input tensor's size a...
FAILED o3_test.py::Tests::test_sh_cuda_single - TypeError: 'int' object is not iterable
FAILED tensor_product_test.py::Tests::test_elementwise_tensor_product - RuntimeError: view size is not compatible with input tensor's size and stride ...
FAILED tensor_product_test.py::Tests::test_tensor_product - RuntimeError: view size is not compatible with input tensor's size and stride (at least on...
FAILED tensor_product_test.py::Tests::test_tensor_product_norm - RuntimeError: view size is not compatible with input tensor's size and stride (at lea...

Therefore tests are incompatible. I expect other things will be too since it seems like pytorch got more strict about views somehow.

rename `SelfInteraction` to `Linear`

Describe explicitly in the docstring that this is linear on the multiplicity of a given L.

Add to documentation or comments somewhere that this is equivalent to the SelfInteraction in Schnet.

cuda-10.1 python-3.6 (CentOS-7) py-torch-1.4.0

Does this error rings a bell?

(torchvision) [tru@ld19-1020 e3nn]$ python3 -m pip install  git+https://github.com/e3nn/e3nn
Collecting git+https://github.com/e3nn/e3nn
  Cloning https://github.com/e3nn/e3nn to /tmp/pip-req-build-os9t7y21
  Running command git clone -q https://github.com/e3nn/e3nn /tmp/pip-req-build-os9t7y21
Requirement already satisfied: scipy in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from e3nn==0.0.0) (1.4.1)
Requirement already satisfied: lie_learn in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from e3nn==0.0.0) (0.0.0)
Requirement already satisfied: appdirs in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from e3nn==0.0.0) (1.4.3)
Requirement already satisfied: numpy>=1.13.3 in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from scipy->e3nn==0.0.0) (1.18.1)
Requirement already satisfied: requests in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from lie_learn->e3nn==0.0.0) (2.23.0)
Requirement already satisfied: cython in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from lie_learn->e3nn==0.0.0) (0.29.15)
Requirement already satisfied: chardet<4,>=3.0.2 in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from requests->lie_learn->e3nn==0.0.0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from requests->lie_learn->e3nn==0.0.0) (2019.11.28)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from requests->lie_learn->e3nn==0.0.0) (1.25.8)
Requirement already satisfied: idna<3,>=2.5 in /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages (from requests->lie_learn->e3nn==0.0.0) (2.9)
Building wheels for collected packages: e3nn
  Building wheel for e3nn (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-os9t7y21/setup.py'"'"'; __file__='"'"'/tmp/pip-req-bui
ld-os9t7y21/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-
wheel-dc5r2azs
       cwd: /tmp/pip-req-build-os9t7y21/
  Complete output (288 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.6
  creating build/lib.linux-x86_64-3.6/e3nn
...
  creating build/temp.linux-x86_64-3.6/src/real_spherical_harmonics
  gcc -pthread -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -I/c7/shared/cuda/10.1.243_418.87.00/include -I/opt/rh/devtoolset-4/root/usr/include -I/c7/home/slurm/current/include -I/c7/shared/cuda/10.1.243_418.87.00/include -I/opt/rh/devtoolset-4/root/usr/include -I/c7/home/slurm/current/include -fPIC -I/c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include -I/c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/TH -I/c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/THC -I/c7/shared/cuda/10.1.243_418.87.00/include -I/usr/include/python3.6m -c src/real_spherical_harmonics/rsh_bind.cpp -o build/temp.linux-x86_64-3.6/src/real_spherical_harmonics/rsh_bind.o -std=c++14 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=real_spherical_harmonics -D_GLIBCXX_USE_CXX11_ABI=0
  /c7/shared/cuda/10.1.243_418.87.00/bin/nvcc -I/c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include -I/c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/TH -I/c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/THC -I/c7/shared/cuda/10.1.243_418.87.00/include -I/usr/include/python3.6m -c src/real_spherical_harmonics/rsh_cuda.cu -o build/temp.linux-x86_64-3.6/src/real_spherical_harmonics/rsh_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -std=c++14 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=real_spherical_harmonics -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75
  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/c10/util/ArrayRef.h(278): warning: attribute does not apply to any entity
  
  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/c10/core/TensorTypeSet.h(44): warning: integer conversion resulted in a change of sign
  
  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(14): warning: integer conversion resulted in a change of sign

 <lots of warning deleted>

  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:44:65: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:56:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
...
  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/modules/normalization.h:106:48:   required from here
  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:44:65: error: invalid static_cast from type ‘const torch::OrderedDict
<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:56:59: error: invalid static_cast from type ‘const torch::OrderedDict
<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
  /c7/home/tru/virtualenv/Python3.6-centos7/torchvision/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:68:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
  error: command '/c7/shared/cuda/10.1.243_418.87.00/bin/nvcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for e3nn
  Running setup.py clean for e3nn

(torchvision) [tru@ld19-1020 e3nn]$ pip list
Package     Version   
----------- ----------
appdirs     1.4.3     
certifi     2019.11.28
chardet     3.0.4     
Cython      0.29.15   
idna        2.9       
lie-learn   0.0.0     
numpy       1.18.1    
Pillow      7.0.0     
pip         20.0.2    
requests    2.23.0    
scipy       1.4.1     
setuptools  46.0.0    
torch       1.4.0     
torchvision 0.5.0     
urllib3     1.25.8    
wheel       0.34.2    
(torchvision) [tru@ld19-1020 e3nn]$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-4/root/usr/libexec/gcc/x86_64-redhat-linux/5.3.1/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-4/root/usr --mandir=/opt/rh/devtoolset-4/root/usr/share/man --infodir=/opt/rh/devtoolset-4/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --enable-plugin --with-linker-hash-style=gnu --enable-initfini-array --disable-libgcj --with-default-libstdcxx-abi=gcc4-compatible --with-isl=/builddir/build/BUILD/gcc-5.3.1-20160406/obj-x86_64-redhat-linux/isl-install --enable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 5.3.1 20160406 (Red Hat 5.3.1-6) (GCC) 
(torchvision) [tru@ld19-1020 e3nn]$ nvcc -v
nvcc fatal   : No input files specified; use option --help for more information
(torchvision) [tru@ld19-1020 e3nn]$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019

NaN value appears when I calculate a derivative of kernel with respect to relative position

Hello.

I found that derivative of kernel with respect to relative position shows NaN value in specific case.
(relative position of [0., 0., A]: vector which is parallel to z-axis)

Sample code of above problem is in below:

import torch
from e3nn import Kernel, o3
from e3nn.radial import GaussianRadialModel
from e3nn.non_linearities.rescaled_act import swish
from functools import partial

RadialModel = partial(GaussianRadialModel, max_radius=7.0, number_of_basis=3, h=100, L=1, act=swish)
mykernel = Kernel([(4,0)], [(1,1)], RadialModel=RadialModel)

xx = torch.randn(1,4)
yy = torch.tensor([[0.,0.,2.]], requires_grad=True)

out = torch.einsum('eij,ej->ei', mykernel(yy), xx)
loss = torch.sum(torch.norm(out, dim=1))

dyy = torch.autograd.grad(loss, yy)
print(dyy)

# $ (tensor([[nan, nan, nan]]),)

Derivative of network with respect of relative positions of atom is important for calculating atomic force of given structure.
Thus, above phenomenon (or bug?) is very critical for me.
I will try to fix that NaN value for my own but I think developers of e3nn can find the source of this phenomenon faster than me. :)
Please give me some help, thanks!

Benchmark timing of various similar functions

  • Test timing of linear_mod.Linear and kernel_mod.Kernel compared to those functions in linear and kernel respectively. This will help determine whether we should merge functions or keep separate due to memory / computation concerns.

Question for handling different size of input via masking

Hello.

I want to use the molecule/crystal training data.
As you already know, the number of atoms in molecules is not fixed.
And common method to generate batch from a dataset like this is masking.
Thus, I need to handle a dataset like below:

features=[ # [2 (batch size), 3 (# of atoms), # of features]
    [
        [...],
        [...],
        [empty space] # masked with some value
    ],
   [
        [...],
        [...],
        [...] # masked
   ]
]

There are two atoms in first molecule and three atoms in second molecule.
And what I want is generating model that provide same output regardless of the masked space.
Is it possible in e3nn module?
Please guide me for the question, thanks.

GaussianRadialModel is not scale invariant

In GaussianRadialModel, the basis function is

    def basis(x):
        return torch.exp(-gamma * x ** 2)

with gamma = 1. / spacing.

If max_radius is multiplied by 2, gamma is divided by 2 but should be divided by 4

bugs and confusion in radial model

I'm fairly sure line 37 is a bug

diff = x.unsqueeze(1) - self.position.unsqueeze(0)  # [batch, i, ...]
batch, n, *rest = diff.size()
x = self.basis(diff.view(-1, *rest)).view(batch, n)  # [batch, i]
return self.f(x)

since it will raise an error when rest is anything by an empty list.

The cosine basis needs some clarification.
Am I right in determining that its mathematical formula is

\begin{cases}
  cos^2 (\gamma (d_{ab} - \mu_{k}) \frac{\pi}{2}) & 1 \geq \gamma (d_{ab} - \mu_{k}) \geq -1 \\
  0 & otherwise
\end{cases}

CodeCogsEqn(1)

write user-friendly instructions for installing needed software for compiling cuda extensions

If possible, summarize in a brief paragraph to be put in the readme what is needed to install cuda extensions -- provide links to needed websites, but avoid relying on the user going to that site to get an overview of what is needed: e.g. provide a copy of the commands on the nvidia cuda install site. The users will thank you for an unintimidating, straightforward, and concise overview of what is needed.

reduce_tensor_product vs. tensor_product

In the switch from se3cnn to e3nn, it looks like the function reduce_tensor_product was moved to the new rs.py file and renamed tensor_product. However, the two do not give the same output for identical input. I have tried every permutation of indices and sign flips and cannot get the outputs to equal each other. Has the functionality changed or is there something I'm missing here?

Predicting L=1/2 output

I've had several requests over time for predicting magnetic moments. It seems to me that the simplest way to handle this would be to use spin spherical harmonics for kernels and expressing intermediate tensors. This is essentially adding a second representation index that has Rs = [(1, 1/2)]. We would still need Clebsch-Gordan coefficients to handle the second representation index (which is contracted separately from the first representation index), but we would only need it for 1/2 $\otimes$ 1/2 -> 0 $\oplus$ 1 as we can always readjust our intermediate tensors to have a second representation index solely made of L=0 and L=1/2.

Decide how to distribute Clebsch-Gordan coefs

obvious options:

  1. upload binaries to github and include the cg coefs in the MANIFEST.in.

  2. do not upload binaries to github, precompute them on CI and upload to pypi. Users downloading from github must compute CGs themselves and put them in the right place.

Slow tests

Some tests are much slower than others:

============================================================= slowest test durations =============================================================
65.03s call     tests/point/kernelconv_test.py::TestKernelConvFn::test1
15.04s call     tests/point/kernelconv_test.py::TestKernelConv::test_compare_custom_backward
14.97s call     tests/point/kernelconv_test.py::TestKernelConv::test_compare_backward
11.56s call     tests/point/kernelconv_test.py::TestKernelConv::test_compare_forward
9.14s call     tests/o3_test.py::Tests::test_sh_closure
8.43s call     tests/kernel_test.py::Tests::test1
1.80s call     tests/non_linearities/so3_test.py::Tests::test_equivariance
0.93s call     tests/normalization_test.py::Tests::test1
0.91s call     tests/normalization_test.py::Tests::test2
0.39s call     tests/non_linearities/s2_test.py::Tests::test_equivariance
0.35s call     tests/o3_test.py::Tests::test_clebsch_gordan_orthogonal
0.31s call     tests/kernel_test.py::TestCompare::test_compare_forward
0.22s call     tests/o3_test.py::Tests::test_reduce_tensor_product
0.17s call     tests/import_test.py::test_import
0.08s call     tests/point/equivariance_test.py::Tests::test6
0.08s call     tests/point/equivariance_test.py::Tests::test5
0.08s call     tests/o3_test.py::Tests::test_clebsch_gordan_sh_norm
0.03s call     tests/point/gated_block_test.py::Tests::test1
0.03s call     tests/point/equivariance_test.py::Tests::test1
0.02s call     tests/point/equivariance_test.py::Tests::test3
0.01s call     tests/point/equivariance_test.py::Tests::test2
0.01s call     tests/tensor_product_test.py::Tests::test_tensor_product
0.01s call     tests/o3_test.py::Tests::test_sh_norm
0.01s call     tests/point/equivariance_test.py::Tests::test4
0.01s call     tests/o3_test.py::Tests::test_spherical_harmonics
0.01s call     tests/o3_test.py::Tests::test_sh_dirac
0.01s call     tests/o3_test.py::Tests::test_wignerd
0.01s call     tests/o3_test.py::Tests::test_sh_parity
0.01s call     tests/tensor_product_test.py::Tests::test_elementwise_tensor_product

Performance of python-based KernelFn

I have compared using KernelFn (forwards and backwards in python) to using the automatic differentiation available by only coding the forward pass. The file I used to prepare the test is written here.

import torch

from e3nn.kernel import Kernel, KernelAutoBackward
from e3nn.radial import ConstantRadialModel


torch.set_default_dtype(torch.float64)
device = torch.device('cuda')

Rs_in = [(1, 0), (1, 1), (2, 0), (1, 2)]
Rs_out = [(2, 0), (1, 1), (1, 2), (3, 0)]
batch = 100
atoms = 40

k = Kernel(Rs_in, Rs_out, RadialModel=ConstantRadialModel).to(device)
kab = KernelAutoBackward(Rs_in, Rs_out, RadialModel=ConstantRadialModel).to(device)


def forward_backward(kernel):
    geometry = torch.rand(batch, atoms, 3).to(device)
    features = kernel(geometry)
    target = torch.rand_like(features).to(device)
    loss = torch.norm(features - target)
    loss.backward()

When using device = torch.device('cuda') I got the following results for speed:

In [1]: %timeit -n15 -r70 forward_backward(k)                                                                                                            
20.5 ms ± 575 µs per loop (mean ± std. dev. of 70 runs, 15 loops each)

In [2]: %timeit -n15 -r70 forward_backward(kab)                                                                                                          
22.7 ms ± 273 µs per loop (mean ± std. dev. of 70 runs, 15 loops each)

and cuda memory results were custom written by me comparing the max allocated memory before and after running the function.

In [1]: forward_backward(k)                                                                                                                              
allocated 19.730432 MB
cached 25.165824 MB

In [2]: forward_backward(kab)                                                                                                                            
allocated 30.536704 MB
cached 31.45728 MB

Now, when I used device = torch.device('cpu'), I got the following results for speed:

In [1]: %timeit forward_backward(k)                                                                                                                      
207 ms ± 2.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [2]: %timeit forward_backward(kab)                                                                                                                    
237 ms ± 1.82 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

and for memory

In [1]: %memit forward_backward(k)                                                                                                                       
peak memory: 220.74 MiB, increment: 33.52 MiB

In [2]: %memit forward_backward(kab)                                                                                                                     
peak memory: 275.71 MiB, increment: 56.20 MiB

--

I think the memory and speed usage implies that we should simply use the built-in/automatic differentiation rather than actually coding our own forward and backward functions. However keeping them in the code base might be useful since effort was made to write down the explicit derivative. If we ever convert the kernel to a cuda call it might be useful to have that code around. (also to compare to)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.