This repository contains the source code and videos for the paper: A spring–block theory of feature learning in deep neural networks. More details can be found here.

When slow and steady pulling:

and the corresponding lazy training of a non-linear DNN:

When quick and jerky pulling:

and the corresponding active training of a non-linear DNN:

./bash/test_pd_ns_nl.sh   # data noise
./bash/test_pd_lr_nl.sh   # learning rate
./bash/test_pd_dp_nl.sh   # drop out
./bash/test_pd_bm_nl.sh   # batch size

In the first notebook, we solve the first-order overdamping system by discrete iteration.

In the second notebook, we solve the second-order stochastic differential equation using sdeint.