u-alberta / prima Goto Github PK

I implemented the first function from this page which works well historically, didn't cause any collisions on my small sample, and produced hashes that gave good results for min_hash_sim.py. However, reading this page makes me wonder if we can work on finding a better one in the future.

shingles.db is created the first time min_hash.py is run then used for repeated calls to that function (it isn't recreated unless the user deletes it.)
inverted_index.db used to be created by tfidf.py in order to get my function k_means_clusterer.py to work but I changed that code so it doesn't need a database.
Right now I'm thinking keep shingles.db the way it is to save computation time on repeated calls to min_hash.py and just get rid of inverted_index.db altogether.

Should the user be able to specify the file type they want saved in the command line arguments?
In min_hash should they be able to specify k-shingles and number of hash functions?
In bm25 should they be able to specify number of documents to be returned or minimum score for documents returned?