Name: Felix Dangel
Type: User
Company: @ProbabilisticNumerics @VectorInstitute
Bio: Postdoc at the Vector Institute, Toronto.
Interested in computing and using anything beyond the gradient for ML.
Twitter: f_dangel
Location: Toronto
Blog: https://fdangel.com
Felix Dangel's Projects
Use DeepOBS with BackPACK
BackPACK - a backpropagation package built on top of PyTorch which efficiently computes quantities other than the gradient.
Experiments code for "BackPACK: Packing more into Backrop" [ICLR 2020]
My global `.bib` file to store bibliographic records
Cockpit: A Practical Debugging Tool for Training Deep Neural Networks
Experiments for the NeurIPS 2021 paper "Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks"
scipy linear operators for the Hessian, Fisher/GGN, and more in PyTorch
DeepOBS: A Deep Learning Optimizer Benchmark Suite
Convolutions and more as einsum for PyTorch
Hessian backpropagation (HBP): PyTorch extension of backpropagation for block-diagonal curvature matrix approximations
My org-export settings
How to export Org mode files into awesome HTML in 2 minutes
Source code for my PhD thesis: Backpropagation Beyond the Gradient
LaTeX template for my PhD thesis at the University of Tuebingen
PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks
Python utility functions I often use
PyTorch implementation of the Hessian-free optimizer
[ICML 2024] SINGD: KFAC-like Structured Inverse-Free Natural Gradient Descent (http://arxiv.org/abs/2312.05705)
[ICML 2024] SIRFShampoo: Structured inverse- and root-free Shampoo in PyTorch (https://arxiv.org/abs/2402.03496)
Matrix-multiplication-only KFAC; Code for ICML 2023 paper on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
(N=1,2,3)-dimensional unfold (im2col) and fold (col2im) in PyTorch
[TMLR 2022] Curvature access through the generalized Gauss-Newton's low-rank structure: Eigenvalues, eigenvectors, directional derivatives & Newton steps
Experiments for the TMLR 2023 paper "ViViT: Curvature Access Through the Generalized Gauss-Newton’s Low-rank Structure"