How to process with the salient map? about atari HOT 4 CLOSED

I don't know of a mathematical proof, but it seems to work well in practice, and I've not seen a normalisation step in other code I've looked at. An issue with normalisation is that you need to choose a method to use, whilst without you can directly observe changes in intensity over time or between models on the same domain.

from atari.

Thanks!!!
One more thing confuses me is whether to take the positive part or the absolute value of the gradients.

I've tried the two both and found that using abs value could result some noisy part while the positive value is more clear. However, the Dueling DQN paper uses absolute value. If positive value of the gradients indicate that part is important for the agent to make decisions, and zero means the agent does not care that part, then what negative value might indicate?

Many thanks.

from atari.

I did not realise that they updated the paper with details of the saliency - but it doesn't appear that they use normalisation either.

The absolute value makes sense as the magnitude is important. Positive values indicate areas that should be increased (if you could change the image as if it were a weight matrix) to increase the error, whilst negative areas indicate areas that should be decreased to increase the error. Hence doing the opposite of this (gradient descent) decreases the error in neural network training.

from atari.

Thanks!!! It is really helpful for me!!!

from atari.