amartyaamp / codecomb Goto Github PK

Have separate sections for top C++ files , python files etc.

So the results would look like

Top C++ files 
============================

Top Python files
============================

I get the following warning during the initial configuration run.

C:\Users\kakalmad\AppData\Roaming\Python\Python37\site-packages\smart_open\smart_open_lib.py:398:
UserWarning: This function is deprecated, use smart_open.open instead. 
See the migration notes for details: https://github.com/RaRe-Technologies/smart_open/blob/master/README.rst#migrating-to-the-new-open-function

Not able to figure out where exactly this module is being called. I think one of the dependencies internally uses it.

Setup Info

• Windows 10
• Anaconda Python 3.7
• Anaconda Prompt

Idea is to make it object oriented.
Currently the code is fully with modularity imposed by functions only.

A class relationship model would enable the system to extend and scale.

• Notepad
• Atom
• Sublime Text
• Notepad++
• GVim
• Emacs

Our current representation uses averaged Word2Vec . There can be other word/ document embeddings which may improve the search results. For eg. -

• Fasttext (averaged)
• MOE (averaged)
• StarSpace (averaged)
• Glove (averaged)
• Universal Sentence encoder

Alongwith these, we need to look at different distance metrics that can be used

• Manhattan
• WMD-relax (pure WMD is too slow)

We need to set a threshold by comparing each or most of them with the current approach.
Accuracy, latency and speed of training is the main concern here.

Currently we have a hard-coded dictionary for this.

Allow users to exclude directories and files while indexing.

Currently our speed is bound by the dot product of query and the document vectors.
We can use NN algorithm to speed up the process. To find the closest vectors given to the query vectors.

word_tokenize is a much more sophisticated tokenizer and is recommended for NLP tasks

Currently only name and location is shown.
If we can show cosine similarity score with the search results users will have an intuition how much is the score difference in the first few items.

After a term is searched, when the list of files is shown to choose from, when I select a given file, the file opens in the editor of my choice but also opens a new cmd window. This behavior is a bit annoying.

I also ran into a problem where it suddenly began selecting a file on an infinite loop thereby continuously creating cmd.exe prompts and opening the file in my editor. Have not been able to reproduce this. But I believe this is related to the earlier behavior.

Setup Info

• Windows 10
• Anaconda Python 3.7
• Anaconda Prompt

• C#
• Java
• Ruby