Collection of algorithms for approximating Fisher Information Matrix for Natural Gradient. The problems used are the simplest possible (e.g. toy classification problem and MNIST), to highlight more on the algorithms themshelves.
- Python 3.5+
- Numpy
- Matplotlib
- Exact Fisher
- Diagonal Fisher
- KFAC (Kronecker-factored Approximate Curvature) [pdf]