Comments (2)
Thanks for your comment. Yes, you are right, this is indeed a discrepancy between the code and our method-description in the paper. According to the method-description in our paper, the line of code that you refer to should indeed be left out.
It thus seems odd that this line of code ended up there. When I went back to the original paper proposing “online EWC” (https://arxiv.org/pdf/1805.06370.pdf) I realized that in their method-description they actually do include this additional scaling by gamma (Eq (8) in section 4) when computing the regularization loss.
Note that because this additional scaling by gamma has the same effect as the hyper-parameter lambda, whether or not this additional scaling is included should actually not have any real effect on the final result when lambda is selected by a hyperparameter search. (It should just result in a slightly different selected value for lambda.)
Thank you for pointing this out to me! Sorry for the confusion, hope this helped to clear it up. In the next update I will one way or the other make things consistent.
from continual-learning.
Thank you,
I now understand where it comes from.
Closing.
Best,
D
from continual-learning.
Related Issues (20)
- Empirical Fisher Estimation HOT 3
- Datasets more complicated than MNIST HOT 1
- Just a request
- Grad in SI HOT 4
- Wrong dataset? HOT 2
- why batch_size has to be 1 when update fisher? HOT 1
- Lower/Upper Bound Experiments HOT 2
- one little confusion about the loss_fn_kd function HOT 1
- Suspicious Precision HOT 3
- Link error HOT 2
- Reproducing BI+SI method HOT 9
- about kafc fisher infromation matrix HOT 1
- How to create Resnet34 HOT 2
- Joint training results different for different types of incremental learning? HOT 3
- Task-IL evaluation HOT 2
- Single head or multihead task incremental HOT 1
- 0 accuracy values for task-free setting HOT 9
- Whether context identity must be inferred in case of domain increment? HOT 1
- About printing results of experimental output
- Results for None ("lower target")
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from continual-learning.