irvel / actionable-screenshots Goto Github PK

• Save OCR extractions on persistent storage and retrieve them back on application launch.
• Process only new screenshots on application launch or wake up.

We need a processing pipeline that:

• Loads existing Index
• Detects deleted and added screenshots
• Runs each screenshot through a classifier
• For images with text, apply OCR
• Updates index with new data
• Stores index to disk

Provide user with feedback of progress for processing new screenshots.

Photo thumbnails in the view all collection appears blurry even when the screenshot itself has enough pixels to fill the space.

Multiple CV tricks could help in improving OCR speed/accuracy:

• Only call OCR for regions that contain text
• Pre-process images to increase contrast and remove compression artifacts
• Some advanced thresholding based on different types of screenshots

In general we need to define metrics on general performance to identify which areas need to be worked on the most