A Pytorch-Lightning implementation of the Forward-Forward algorithm.
This started as a fork of https://github.com/mohammadpz/pytorch_forward_forward !!
Below is my understanding of the FF algorithm presented at Geoffrey Hinton's talk at NeurIPS 2022.
The conventional backprop computes the gradients by successive applications of the chain rule, from the objective function to the parameters. FF, however, computes the gradients locally with a local objective function, so there is no need to backpropagate the errors.
The local objective function is designed to push a layer's output to values larger than a threshold for positive samples and to values smaller than a threshold for negative samples.
A positive sample
A negative sample
Among the many ways of generating the positive/negative samples, for MNIST, we have:
Positive sample
Negative sample
After training all the layers, to make a prediction for a test image
To run the training:
> python train.py
Hyperparameters can surely be optimized!