Add descriptions how use main script to extract text and generate character level transliterations.

We should at least uniquify, which will downweight the boilerplate like headers and footers. But it would be even better to remove the most common lines.

Basically we should only preserve lines in a document that did not appear in any other document in the corpus.

We can clean very aggressively because lack of data is not a problem.

As has been added OpenSubClient.py need to update dependencies in requirements.txt
Also need Add in Readme.txt about OpenSubClient.py

Currently project downloads only monolingual corpus during each execution. We want to get extract multilingual corpuses as well.

Consistent style inside of one word and one row

Be smart about uppercase digraphs (eg Խ => KH vs Kh depending on context)

For example, a Wikipedia, but potentially others, like film subtitles, which will have more conversational data but still be free of translit.

Can basically extend generate.js in https://github.com/deeplanguageclass/fairseq-transliteration-data.

The most important part besides applying the mapping randomly is throwing out the lines which do not change.