use Multilingual pretrain model Bert about albert_zh HOT 6 OPEN

you can have a try.
and be aware that there are some differences between bert and albert in modelling.py
why do you want to train multillingual model?

from albert_zh.

i want to use model with vietnamese language. The important of change is share parameters, i know. how i can train with my language. Thanks for support =)

from albert_zh.

1、you can change vocab.txt in ./albert_config, then set non_chinese to True when create pretrain data using create_pretraining_data.py
2、then do pre train using run_pretraining.py

from albert_zh.

okay. Thanks for support. Best repo =)

from albert_zh.

I have tried to pretrain with my dataset, but i see the loss is very small but accuracy is not improve. How i can improve result

from albert_zh.

@brightmart Can we have a multilingual model for just Chinese and English? Cause in practical scenerios we may meet many english words in APP names, music names, Apple's all products's name and so on, and Google's multilingual model has too many languages. Our daliy life cannot leave English, you can see that Apple try to use purely Chinese in its products, such as replace Finder with 访达 which I think is totally a mess.
Maybe a language model for just Chinese and English can have huge impact on both research and industry and many multilingual tasks can benefit from it.

from albert_zh.