Grokking is a new phenomenon in deep learning where some models exhibit a (very) late generalisation phase, after an initial phase of very strong over-fitting.
In the original publication, it has been demonstrated with modular arithmetic and a transformer architecture.
![grokking_openai](https://private-user-images.githubusercontent.com/15683540/266827001-d516f2a9-f805-4c61-887d-911baffab2bf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg0ODg3MTYsIm5iZiI6MTcxODQ4ODQxNiwicGF0aCI6Ii8xNTY4MzU0MC8yNjY4MjcwMDEtZDUxNmYyYTktZjgwNS00YzYxLTg4N2QtOTExYmFmZmFiMmJmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjE1VDIxNTMzNlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTE0NGJjZWY1NmZmMjZmMDJkNjUyYzA4M2UxODMwN2EyMTQ4ZWY2OTYxYmQ4ZTQxNDkyOGQyMjJhZDFiODc3ZjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.AR36ox3CEpCjH0coPFY17YzeJhfTimEtZxFQMRAyRmc)
POWER, Alethea, BURDA, Yuri, EDWARDS, Harri, et al. Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv preprint arXiv:2201.02177, 2022.
In this notebook, I show how to reproduce the grokking phenomenon in a minimalistic setting : modular addition and a model involving only linear embedding followed by a feedfoward (one hidden layer).
In particular, principal component analysis on the embedding layer weights shows how the model learnt to organize the embedding of the tokens in a circular way, similar to what happens with a clock.
Other relevant publications :
NANDA, Neel, CHAN, Lawrence, LIBERUM, Tom, et al. Progress measures for grokking via mechanistic interpretability. arXiv preprint arXiv:2301.05217, 2023.
LIU, Ziming, KITOUNI, Ouail, NOLTE, Niklas S., et al. Towards understanding grokking: An effective theory of representation learning. Advances in Neural Information Processing Systems, 2022, vol. 35, p. 34651-34663.
Liu, Z., Michaud, E. J., & Tegmark, M. (2022). Omnigrok: Grokking beyond algorithmic data. arXiv preprint arXiv:2210.01117.
ZHONG, Ziqian, LIU, Ziming, TEGMARK, Max, et al. The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks. arXiv preprint arXiv:2306.17844, 2023.