bn-l / thsearch Goto Github PK

The second argument defaults to thsearch.txt and should allow for a relative (to the binary) or absolute path.

This will allow something like this:

thsearch something books

And an alternate books.txt configuration file next to the binary will search all books and look like this example:

+C:\ebooks\
>.epub

An error should be given if the file can't be found.

Parser checks for a new line type. E.g.:

~node_modules

and if this is in any part of a candidate file's path, the file is rejected.

• Benchmarking suite that sets up test files to be searched and then deletes them when finished.
• Warns tester how much space is about to be taken up (should also test if it's available).
• Is there a library that can help with this?

Instead of looking for candidate files on each run do the following:

• Build an index of candidate file paths, saving the modified time of all directories
• On each run check if a directory has been modified,
• If it has update the index
• Run threads using the index

A progress bar on the command line implemented in ascii. E.g.:

[======··············] 60%

Should be easy implement by watching the ConcurrentQueue

• Runs as a background task

• User can set maximum memory use. More recent files are given priority in the index.

• Embedded binaries (images etc) can be linked to (for a future GUI version)

• Index is stored as a reverse index:

  • First tokenisation (and removing stop words)
  • A forward index is then created of Documents and Tokens (with frequency--like tf-idf)
  • then a reverse index is created from the forward index of Tokens and the Documents the appear in (with frequency number)

Downsides:

• Introduces state: The program must rebuild the index on file changed. It can do this atomically but it now needs to track the files in its scope.

Reproduce:

• Add +C:\User\Documents or +C:\User\ to config file
• Run

Make sure folders that throw a read error are skipped

Using a custom string contains (e.g. expanding the one in Searcher) to do fuzzy matching of a query stem to a db stem or a string distance algorithm.