Comments (3)
-
we ran all in100 the experiments with Dali, reproducibility without Dali is not guaranteed. What do you mean that you need to remove Dali for comparison? Dali has nothing to do with the methodology, it is just a data loading library.
-
I don't see any issues, it's just lr scaling based on the batch size. This is done in most of the repositories out there. It makes sense because when you increase the batch size you also need to increase the lr.
-
pytorch and pytorch lightning work like that, the batch size is always per GPU. Since this is the standard behavior I don't think it's worth mentioning.
from cassle.
- Of course it is just data loading but I'm just saying simply by disabling dali, performance drops about 6-7% on barlow_distill (both dali/regular linear evaluation). I think there is a difference between dali and the regular dataloader. I tried finding the difference but I couldn't. I'll update here if I find it. Closing it for now.
2,3. Thanks for the explanation.
from cassle.
Are you able to reproduce any results with/without dali? I am training with dali but I still get 5-6% lower than reported currently.
from cassle.
Related Issues (18)
- How did you train the classifier? HOT 3
- About the Forward Transfer HOT 2
- Difficulty In Reproducing The Paper Results (e.g., Table 4., BYOL reported 66%, measured <<60%) HOT 13
- Forward Transfer Issue HOT 10
- Some questions about training and evaluation process HOT 5
- Some question about lower and upper bounds HOT 2
- train data on DomainNet HOT 1
- Question about contrastive distillation loss
- Clarification on the Role of the Predictor Network g in the CaSSLe Framework
- Why did you load the checkpoint of task 0 before training cassle? HOT 5
- DomainNet dataset version HOT 1
- ¿Bug in online KNN eval? HOT 3
- Need of checkpoints : BT and VicReg HOT 4
- KNN Classifier Issue HOT 8
- Task_id set to zero as first task SSL HOT 2
- The data for Linear Evaluation Accuracy. HOT 1
- The classifier for Linear Evaluation Accuracy HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cassle.