Question about Online EWC about continual-learning HOT 2 CLOSED

Thanks for your comment. Yes, you are right, this is indeed a discrepancy between the code and our method-description in the paper. According to the method-description in our paper, the line of code that you refer to should indeed be left out.

It thus seems odd that this line of code ended up there. When I went back to the original paper proposing “online EWC” (https://arxiv.org/pdf/1805.06370.pdf) I realized that in their method-description they actually do include this additional scaling by gamma (Eq (8) in section 4) when computing the regularization loss.

Note that because this additional scaling by gamma has the same effect as the hyper-parameter lambda, whether or not this additional scaling is included should actually not have any real effect on the final result when lambda is selected by a hyperparameter search. (It should just result in a slightly different selected value for lambda.)

Thank you for pointing this out to me! Sorry for the confusion, hope this helped to clear it up. In the next update I will one way or the other make things consistent.

from continual-learning.

Thank you,

I now understand where it comes from.

Closing.
Best,
D

from continual-learning.