sea-snell / grokking Goto Github PK

The value for p in the train grokk config file is 96, but should probably be 97. As far as I can tell the 96 then overrides the 97 from the dataset config default.

When I try to install this repo on my computer, I met some difficulties when installing requirements.txt.
The conflict is caused by:
The user requested numpy
mkl-fft 1.3.1 depends on numpy<1.23.0 and >=1.22.3
torchvision 0.11.1 depends on numpy
mkl-random 1.2.2 depends on numpy<1.25.0 and >=1.24.3

I will appreciate it if you can update your requirements.txt with more simplified package requirements and more detailed version information. Thanks a lot.

Why do you have (a * pow(b, self.p-2, self.p)) % self.p for ModDivisonDataset? Shouldn't it be (a / b) % self.p?

According to paper, the weight_decay should be 1. However, your config has it as 0. Is this correct? I am trying to reproduce Figure 1 and I thought that must have been produced with weight_Decay=1 per appendix A.1.2.