Sentence delimiter <s> or </s> about xlm HOT 3 CLOSED

No, this is correct. In the case of language model training, sentences are attached one after each other in a stream. A stream looks like:

</s> sent_1_word_1 sent_1_word_2 ... sent_1_word_n </s> sent_2_word_1 sent_2_word_2 ... sent_2_word_m </s> sent_3_word_1 ...

In that setting, the model learns that the is used to both delimitate the beginning and the end of a sentence.

from xlm.

@glample Hi, what are the advantages of doing this ？Thank you.

from xlm.

I think this was the simplest way to combine MLM / CLM and MT training. It probably does not make a big difference as it should be easy for the model to understand the meaning of these symbols.

from xlm.