about normalization about transmatcher HOT 2 CLOSED

Hi, thank you for the interest. A good question. In QAConv L2 norm is applied in feature extraction before QAConv, but that's for efficiency to avoid redundant computation. For a better understanding, it should be applied in QAConv to normalize the two input features, to perform cosine similarity measure, which, if I remember correctly, performs better than dot product.

In TransMatcher, however, in the feature extraction module the L2 norm should not be pre-applied, because there is a stack of Transformer encoders there that requires original unscaled features. Besides, in the decoder the first unit is FC which also expects unscaled features. On the other hand, in the decoder after FC, at that time I was considering how to adapt the original decoder in Transformers for matching, so the dot product is re-used. I cannot remember if the L2 norm was tried here.

However, indeed it is a good question. If you are interested in, I would suggest you do a further ablation study to see what's the effect of the L2 norm, in both QAConv and Transmatcher, and I would appreciate if you could post your findings here. Thanks.

from transmatcher.

ok, thanks for you reply. I will do a further ablation stydy~

from transmatcher.