problem with reproducibility about cassle HOT 3 CLOSED

1. we ran all in100 the experiments with Dali, reproducibility without Dali is not guaranteed. What do you mean that you need to remove Dali for comparison? Dali has nothing to do with the methodology, it is just a data loading library.

2. I don't see any issues, it's just lr scaling based on the batch size. This is done in most of the repositories out there. It makes sense because when you increase the batch size you also need to increase the lr.

3. pytorch and pytorch lightning work like that, the batch size is always per GPU. Since this is the standard behavior I don't think it's worth mentioning.

from cassle.

1. Of course it is just data loading but I'm just saying simply by disabling dali, performance drops about 6-7% on barlow_distill (both dali/regular linear evaluation). I think there is a difference between dali and the regular dataloader. I tried finding the difference but I couldn't. I'll update here if I find it. Closing it for now.
   2,3. Thanks for the explanation.

from cassle.

Are you able to reproduce any results with/without dali? I am training with dali but I still get 5-6% lower than reported currently.

from cassle.