Problems about masking details. about perturbed-masking HOT 1 CLOSED

I remember that we also tried the setting in which we do not mask x_i, and it performs poorly on syntactic tasks.
We suspect the reason is that if you do not mask x_i, x_i+1 will only have a very small impact on x_i's representation, and make parsing difficult (since the impact values are quite close to each other).
But yeh, it's also possible only masking once can also do well on sentence segmentation task.
Would be great if you would like to share some of your results, e.g., performance of masking x_i and x_i+1 vs masking x_i+1

from perturbed-masking.