bread-and-code / nltk-based-word-checking-dic-method Goto Github PK

1. find Various keywords that are primary concern of the consumers or seeking for information.
   These can be put under various categories such as brand, product line and sub-product line,
   usage, etc;
2. Examples of inputs (searched phrases) that the consumers may type in
3. Examples of formation of composite keywords (phrases) out of keywords and thus
   enhancing the list of keywords (‘shoe polish’ out of ‘shoe’ and ‘polish’)
4. Examples of Synonyms
   We need to create an optimal set of categories (buckets) in the range of 10 to 20 (e.g. brand,
   product line, price, region, etc.) and an optimal set of sub-categories under each category (e.g.
   wellness, skin care, makeup, etc under ‘product’ category; Lux, Hamam, Liril, Lakme, Lipton,
   etc sub-categories under ‘brand’ category; cheap, expensive under ‘price’ category; etc.) out of
   the Keywords; examples of the final attributes /keywords under a sub-category such as ‘loofah’,
   ‘soap’, ‘shower gel’ etc under ‘Lux’ sub-category. And you need to put all the key words in one
   or many categories and/ or sub-categories. The keywords may belong to different categories or
   sub-categories (e.g. polish can be under makeup ‘nail polish’ and also under accessories ‘shoe
   polish’).
   You may require applying more than one supervised and / or unsupervised learning
   techniques for the same.

If a word doesn't match with the dictionary , then is produce it's synonymy and check the words with positive and negative dictionary . And then separate them into two different txt file . In the "naturalprocessingwordnet.py" file you will get necessary function for synonyms .

we have positive word dictionary,negative word dictionary and a text file(with paragraphs). we have to write a program to tokenize key words from the text file . Then match those key words with positive and negative file. if the key word is matched with any of these file the store it in destinationposfile or destinationneg file. If the key word is not found then collect synonyms of that key word and again try to match those words with positive or negative file , if match found then store that key word in a destination file and if not found then store that key word in destinationnutral file.

With help of the "finder.py" script we can create log file . But in the files there are few duplicate words . or a same word few many time . Need help to remove the duplicates .