Score normalization in ReDo about dopamine HOT 2 OPEN

hi, apologies for the delay in the response.

you are correct in that it's not actually normalizing in the sense that the values add up to 1. we are in fact scaling the "normalized" values by the number of neurons in the layer.

this may have been an oversight on our end, and we will add a clarifying note to the paper and to this code. however, given that our experiments were run with this setup, we will keep the code as is!

if you decide to correct this and emit properly normalized scores, you will likely have to adjust the threshold as well. do let us know if you find anything interesting!

from dopamine.

Hi @yycho0108 @psc-g , I have some confusion about redo. In the paper, it says, reinitialize their incoming weights and zero out the outgoing weights. I'm confused since in my mind each layer of the network is just a matrix. I'm wondering what are the incoming weights and outgoing weights.
Could you give me some hints?
Thanks a lot!

from dopamine.