goru001 / nlp-for-hindi Goto Github PK

In the Jupyter Notebook, I am unable to understand a few things:

1. What does the path/'hindi_transformer' refer to? Is it referring to the Training Corpus? Also, Is it required to have separate training and validation corpus?
   In [19]:
   data_lm = TextLMDataBunch.from_folder(path=path/'hindi_transformer', tokenizer=tokenizer, vocab=hindi_vocab)

2. In the Class HindiTokenizer, what does the path - (path/"../tokenizer/hindi_lm_large.model") refer to? What language model is it referring to?

Any help would be appreciated. Thanks!

unable to open the pre-trained model for this project where is used curl to download it directly and also unzipped it there but then the file couldn't be accessed by the download link being specified. Is there any other model i can access or maybe access this from some other link. Any help please!

Hi Gaurav,

Great initiative on vernacular languages. I had a small doubt, and was hoping you could clarify it for me. While trying to replicate some of your codes in the repository, I noticed that in the Hindi_Classification_Model-BBC_Hindi_News_Data.ipynb notebook, you import a hindi-train.csv file (in cell [5]). Unfortunately, I wasn't able to locate that file. Could you guide me in the right direction?