You code is outdated! about keras-attention HOT 5 CLOSED

This is not replacing the original purpose of the code I think. Instead of using multiply, you should using Concatenate or Add.

from keras-attention.

what version of keras should replace these points?

from keras-attention.

This is not replacing the original purpose of the code I think. Instead of using multiply, you should using Concatenate or Add.

If you look at the mode params of the merge call it is mul which makes multiplication.
So the answer from @korolaab is correct (also if you look at the paper / implementations it is pretty clear that there's a multiplication there as you combine attention and input, attention is a prob. distribution (as is the output of a softmax activation).
So addition or concatenation would not make much sense.

from keras-attention.

I agree with @korolaab and @JunkyByte

from keras-attention.

Code has been updated. Thanks guys :)

from keras-attention.