Paper list for equivariant neural network. Work-in-progress.
Feel free to suggest relevant papers in the following format.
**Group Equivariant Convolutional Networks**
Taco S. Cohen, Max Welling ICML 2016 [paper](https://arxiv.org/pdf/1602.07576.pdf)
Acknowledgement: I would like to thank Maurice Weiler, Fabian Fuchs, Tess Smidt, Rui Wang, David Pfau, Jonas Köhler, Taco Cohen, Gregor Simm, Erik J Bekkers, Jean-Baptiste Cordonnier, David W. Romero, Ivan Sosnovik, Kostas Daniilidis for paper suggestions! Thank Weihao Xia for helping out typesetting!
- Equivariance and Group convolution
- Theory
- Equivariant Density Estimation and Sampling
- Application
- Permutation Equivariance
- Talk and Tutorial
- TO READ
-
Group Equivariant Convolutional Networks
Taco S. Cohen, Max Welling ICML 2016 paper
Note: first paper; discrete group; -
Steerable CNNs
Taco S. Cohen, Max Welling ICLR 2017 paper -
Harmonic Networks: Deep Translation and Rotation Equivariance
Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, Gabriel J. Brostow CVPR 2017 paper -
Spherical CNNs
Taco S. Cohen, Mario Geiger, Jonas Koehler, Max Welling ICLR 2018 best paper paper
Note: use generalized FFT to speed up convolution on$S^2$ and$SO(3)$ -
Clebsch–Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network
Risi Kondor, Zhen Lin, Shubhendu Trivedi NeurIPS 2018 paper
Note: perform equivariant nonlinearity in Fourier space; -
General E(2)-Equivariant Steerable CNNs
Maurice Weiler, Gabriele Cesa NeurIPS 2019 paper
Note: nice benchmark on different reprsentations -
Learning Steerable Filters for Rotation Equivariant CNNs
Maurice Weiler, Fred A. Hamprecht, Martin Storath CVPR 2018 paper
Note: group convolutions, kernels parameterized in circular harmonic basis (steerable filters); -
Learning SO(3) Equivariant Representations with Spherical CNNs
Carlos Esteves, Christine Allen-Blanchette, Ameesh Makadia, Kostas Daniilidis ECCV 2018 paper
Note: SO(3) equivariance; zonal filter -
Polar Transformer Networks
Carlos Esteves, Christine Allen-Blanchette, Xiaowei Zhou, Kostas Daniilidis ICLR 2018 paper -
3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, Taco Cohen NeurIPS 2018 paper
Note: SE(3) equivariance; characterize the basis of steerable kernel -
Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, Patrick Riley paper
Note: SE(3) equivariance for point clouds -
Equivariant Multi-View Networks
Carlos Esteves, Yinshuang Xu, Christine Allen-Blanchette, Kostas Daniilidis ICCV 2019 paper -
Gauge Equivariant Convolutional Networks and the Icosahedral CNN
Taco S. Cohen, Maurice Weiler, Berkay Kicanaoglu, Max Welling ICML 2019 paper, talk
Note: gauge equivariance on general manifold -
Cormorant: Covariant Molecular Neural Networks
Brandon Anderson, Truong-Son Hy, Risi Kondor NeurIPS 2019 paper -
Deep Scale-spaces: Equivariance Over Scale
Daniel Worrall, Max Welling NeurIPS 2019 paper -
Scale-Equivariant Steerable Networks
Ivan Sosnovik, Michał Szmaja, Arnold Smeulders ICLR 2020 paper -
B-Spline CNNs on Lie Groups
Erik J Bekkers ICLR 2020 paper -
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
Fabian B. Fuchs, Daniel E. Worrall, Volker Fischer, Max Welling NeurIPS 2020 paper, blog
Note: TFN + equivariant self-attention; improved spherical harmonics computation -
Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
Pim de Haan, Maurice Weiler, Taco Cohen, Max Welling ICLR 2021 paper
Note: anisotropic gauge equivariant kernels + message passing by parallel transporting features over mesh edges -
Lorentz Group Equivariant Neural Network for Particle Physics
Alexander Bogatskiy, Brandon Anderson, Jan T. Offermann, Marwah Roussi, David W. Miller, Risi Kondor ICML 2020 paper
Note: SO(1, 3) equivariance -
Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
Marc Finzi, Samuel Stanton, Pavel Izmailov, Andrew Gordon Wilson ICML 2020 paper
Note: fairly generic architecture; use Monte Carlo sampling to achieve equivariance in expectation; -
Spin-Weighted Spherical CNNs
Carlos Esteves, Ameesh Makadia, Kostas Daniilidis NeurIPS 2020 paper
Note: anisotropic filter for vector field on sphere -
Learning Invariances in Neural Networks
Gregory Benton, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson NeurIPS 2020 paper
Note: very interesting approch; enfore "soft" invariance via learning over both model parameters and distributions over augmentations -
Lie Algebra Convolutional Neural Networks with Automatic Symmetry Extraction
Nima Dehmamy, Yanchen Liu, Robin Walters, Rose Yu paper
Note: very interesting paper; It’s unfortunate that it is rejected by ICLR 2021 -
LieTransformer: Equivariant self-attention for Lie Groups
Michael Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Teh, Hyunjik Kim paper
Note: equivariant self attention to arbitrary Lie groups and their discrete subgroups -
Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data
David W. Romero, Mark Hoogendoorn ICLR 2020 paper -
Attentive Group Equivariant Convolutional Networks
David W. Romero, Erik J. Bekkers, Jakub M. Tomczak, Mark Hoogendoorn ICML 2020 paper -
Wavelet Networks: Scale Equivariant Learning From Raw Waveforms
David W. Romero, Erik J. Bekkers, Jakub M. Tomczak, Mark Hoogendoorn paper -
Group Equivariant Stand-Alone Self-Attention For Vision
David W. Romero, Jean-Baptiste Cordonnier ICLR 2021 paper -
MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning
Elise van der Pol, Daniel E. Worrall, Herke van Hoof, Frans A. Oliehoek, Max Welling NeurIPS 2020 paper -
Isometric Transformation Invariant and Equivariant Graph Convolutional Networks
Masanobu Horie, Naoki Morita, Toshiaki Hishinuma, Yu Ihara, Naoto Mitsume ICLR 2021 paper -
E(n) Equivariant Graph Neural Networks
Victor Garcia Satorras, Emiel Hoogeboom, Max Welling ICML 2021 paper
Note: a simple alternative that achieves E(n) equivariance -
Vector Neurons: A General Framework for SO(3)-Equivariant Networks
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, Leonidas Guibas paper
Note: a simple MLP for type-1 features -
Equivariant message passing for the prediction of tensorial properties and molecular spectra
Kristof T. Schütt, Oliver T. Unke, Michael Gastegger ICML 2021 paper
-
On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
Risi Kondor, Shubhendu Trivedi ICML 2018 paper
Note: convolution is all you need (for scalar fields) -
A General Theory of Equivariant CNNs on Homogeneous Spaces
Taco Cohen, Mario Geiger, Maurice Weiler NeurIPS 2019 paper
Note: convolution is all you need (for general fields) -
Equivariance Through Parameter-Sharing
Siamak Ravanbakhsh, Jeff Schneider, Barnabas Poczos ICML 2017 paper -
Universal approximations of invariant maps by neural networks
Dmitry Yarotsky paper -
A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
Leon Lang, Maurice Weiler ICLR 2021 paper
Note: steerable kernel spaces are fully understood and parameterized in terms of 1) generalized reduced matrix elements, 2) Clebsch-Gordan coefficients, and 3) harmonic basis functions on homogeneous spaces. -
On the Universality of Rotation Equivariant Point Cloud Networks
Nadav Dym, Haggai Maron ICLR 2021 paper,
Note: universality for TFN and se3-transformer -
Universal Equivariant Multilayer Perceptrons
Siamak Ravanbakhsh paper -
Provably Strict Generalisation Benefit for Equivariant Models
Bryn Elesedy, Sheheryar Zaidi ICML 2021 paper
- Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities
Jonas Köhler, Leon Klein, Frank Noé ICML 2020 paper
Note: general framework for constructing equivariant normalizing flows on euclidean spaces. Instantiation for particle systems/point clouds = simultanoues SE(3) and permutation equivariance. - Equivariant Hamiltonian Flows
Danilo Jimenez Rezende, Sébastien Racanière, Irina Higgins, Peter Toth NeurIPS 2019 ML4Phys workshop paper
Note: general framework for constructing equivariant normalizing flows in phase space utilizing Hamiltonian dynamics. Instantiation for SE(2) equivariance. - Sampling using SU(N) gauge equivariant flows
Denis Boyda, Gurtej Kanwar, Sébastien Racanière, Danilo Jimenez Rezende, Michael S. Albergo, Kyle Cranmer, Daniel C. Hackett, Phiala E. Shanahan paper
Note: normalizing flows for lattice gauge theory. Instantiation for SU(2)/SU(3) equivariance. - Exchangeable neural ode for set modeling
Yang Li, Haidong Yi, Christopher M. Bender, Siyuan Shan, Junier B. Oliva NeurIPS 2020 paper
Note: framework for permutation equivariant flows for set data. Instantiation for permutation equivariance. - Equivariant Normalizing Flows for Point Processes and Sets
Marin Biloš, Stephan Günnemann NeurIPS 2020 paper
Note: framework for permutation equivariant flows for set data. Instantiation for permutation equivariance. - The Convolution Exponential and Generalized Sylvester Flows
Emiel Hoogeboom, Victor Garcia Satorras, Jakub M. Tomczak, Max Welling NeurIPS 2020 paper
Note: invertible convolution operators. Instantiation for permutation equivariance. - Targeted free energy estimation via learned mappings
Peter Wirnsberger, Andrew J. Ballard, George Papamakarios, Stuart Abercrombie, Sébastien Racanière, Alexander Pritzel, Danilo Jimenez Rezende, Charles Blundell J Chem Phys. 2020 Oct 14;153(14):144112. paper
Note: normalizing flows for particle systems on a torus. Instantiation for permutation equivariance. - Temperature-steerable flows
Manuel Dibak, Leon Klein, Frank Noé NeurIPS 2020 ML4Phys workshops paper
Note: normalizing flows in phase space with equivariance with respect to changes in temperature.
- Trajectory Prediction using Equivariant Continuous Convolution
Robin Walters, Jinxi Li, Rose Yu ICLR 2021 paper - Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
Rui Wang, Robin Walters, Rose Yu ICLR 2021 paper - SE(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
Simon Batzner, Tess E. Smidt, Lixin Sun, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Boris Kozinsky paper - Finding Symmetry Breaking Order Parameters with Euclidean Neural Networks
Tess E. Smidt, Mario Geiger, Benjamin Kurt Miller paper - Group Equivariant Generative Adversarial Networks
Neel Dey, Antong Chen, Soheil Ghafurian ICLR 2021 paper - Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks
David Pfau, James S. Spencer, Alexander G. de G. Matthews, W. M. C. Foulkes paper - Symmetry-Aware Actor-Critic for 3D Molecular Design
Gregor N. C. Simm, Robert Pinsler, Gábor Csányi, José Miguel Hernández-Lobato ICLR 2021 paper - Roto-translation equivariant convolutional networks: Application to histopathology image analysis
Maxime W. Lafarge, Erik J. Bekkers, Josien P.W. Pluim, Remco Duits, Mitko Veta MedIA paper - Scale Equivariance Improves Siamese Tracking
Ivan Sosnovik*, Artem Moskalev*, Arnold Smeulders WACV 2021 paper - 3D G-CNNs for Pulmonary Nodule Detection Marysia Winkels, Taco S. Cohen paper International Conference on Medical Imaging with Deep Learning (MIDL), 2018.
- Roto-translation covariant convolutional networks for medical image analysis
Erik J. Bekkers, Maxime W. Lafarge, Mitko Veta, Koen A.J. Eppenhof, Josien P.W. Pluim, Remco Duits MICCAI 2018 Young Scientist Award paper - Equivariant Spherical Deconvolution: Learning Sparse Orientation Distribution Functions from Spherical Data
Axel Elaldi*, Neel Dey*, Heejong Kim, Guido Gerig, Information Processing in Medical Imaging (IPMI) 2021 paper - Rotation-Equivariant Deep Learning for Diffusion MRI
Philip Müller, Vladimir Golkov, Valentina Tomassini, Daniel Cremers paper - Equivariant geometric learning for digital rock physics: estimating formation factor and effective permeability tensors from Morse graph
Chen Cai, Nikolaos Vlassis, Lucas Magee, Ran Ma, Zeyu Xiong, Bahador Bahmani, Teng-Fong Wong, Yusu Wang, WaiChing Sun paper
Note: equivariant nets + Morse graph for permeability tensor prediction - Direct prediction of phonon density of states with Euclidean neural network Zhantao Chen, Nina Andrejevic, Tess Smidt, Zhiwei Ding, Yen-Ting Chi, Quynh T. Nguyen, Ahmet Alatas, Jing Kong, Mingda Li, Advanced Science (2021) paper arXiv
- SE(3)-equivariant prediction of molecular wavefunctions and electronic densities Oliver T. Unke, Mihail Bogojeski, Michael Gastegger, Mario Geiger, Tess Smidt, Klaus-Robert Müller paper
There are many paper on this topics. I only added very few of them.
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles R. Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas CVPR 2017 paper - Deep Sets
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, Alexander Smola NeurIPS 2017 paper - Invariant and Equivariant Graph Networks
Haggai Maron, Heli Ben-Hamu, Nadav Shamir, Yaron Lipman ICLR 2019 paper - Provably Powerful Graph Networks
Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, Yaron Lipman NeurIPS 2019 paper - Universal Invariant and Equivariant Graph Neural Networks
Nicolas Keriven, Gabriel Peyré NeurIPS 2019 paper - On Learning Sets of Symmetric Elements
Haggai Maron, Or Litany, Gal Chechik, Ethan Fetaya ICML 2020 best paper - On the Universality of Invariant Networks
Haggai Maron, Ethan Fetaya, Nimrod Segol, Yaron Lipman paper
IAS: Graph Nets: The Next Generation - Max Welling - YouTube
Equivariance and Data Augmentation workshop: many nice talks
IPAM: E(3) Equivariant Neural Network Tutorial
IPAM: Risi Kondor: "Fourier space neural networks"
NeurIPS 2020 tutorial: Equivariant Networks
Yaron Lipman - Deep Learning of Irregular and Geometric Data - YouTube
Math-ML: Erik J Bekkers: Group Equivariant CNNs beyond Roto-Translations: B-Spline CNNs on Lie Groups
Kostas Daniilidis: Geometry-aware deep learning: A brief history of equivariant representations and recent results
Andrew White: Deep Learning for Molecules and Materials.
I am by no means an expert in this field. Here are books and articles suggest by Taco Cohen when asked references to learn group theory and representation theory.
-
Carter, Visual Group Theory
Note: very basic intro to group theory -
Theoretical Aspects of Group Equivariant Neural Networks
Carlos Esteves
Note: covers all the math you need for equivariant nets in a fairly compact and accessible manner. -
Serre, Linear Representations of Finite Groups
Note: classic text on representations of finite groups. First few chapters are relevant to equivariant nets. -
G B Folland. A Course in Abstract Harmonic Analysis
Note: covers representations of locally compact groups; induced representations. -
Mark Hamilton. Mathematical Gauge Theory: With Applications to the Standard Model of Particle Physics
Note: covers fiber bundles, useful for understanding homogeneous G-CNNs and Gauge CNNs.
- Taco Cohen, Equivariant Convolutional Networks, PhD Thesis, University of Amsterdam, 2021 [pdf] (Note: Part II contains a lot of new material, not published before)
There are many paper I haven't read carefully yet.
- Making Convolutional Networks Shift-Invariant Again
Richard Zhang ICML 2019 paper - Probabilistic symmetries and invariant neural networks
Benjamin Bloem-Reddy, Yee Whye Teh JMLR paper - On Representing (Anti)Symmetric Functions
Marcus Hutter paper - PDE-based Group Equivariant Convolutional Neural Networks
Bart M.N. Smets, Jim Portegies, Erik J. Bekkers, Remco Duits paper