Giter VIP home page Giter VIP logo

textinator's Introduction

Textinator

All Contributors

Simple macOS StatusBar / menu bar app to perform automatic text detection on screenshots.

Overview

Install the app per instructions below. Then, take a screenshot of a region of the screen using ⌘ + ⇧ + 4 (Cmd + Shift + 4). The app will automatically detect any text in the screenshot and copy it to your clipboard.

Watch the screencast

Installation

Download and open the latest installer DMG from the release page then drag the Textinator icon to Applications and follow instructions below to grant Desktop access and optionally grant Full Disk Access.

To launch Textinator the first time you'll need to right-click on the app icon and select "Open" otherwise you may get a warning about unknown developer as the app is not signed with an Apple Developer ID.

Installer DMG

Alternatively, to build from source:

  • clone the repo
  • cd into the repo directory
  • create a virtual environment and activate it
  • python3 -m pip install -r requirements.txt
  • python3 setup.py py2app
  • Copy dist/textinator.app to /Applications
  • Follow instructions below to grant Desktop and optionally Full Disk Access

See also Developer Notes below.

Grant Desktop access:

Textinator works by monitoring the file system for new screenshots. The macOS security model prevents apps from accessing files and folders without the user's explicit permission. The first time you launch Textinator, you will be prompted to grant it access to your Desktop.

Desktop access

The default location for new screenshots on your Mac is the Desktop folder so Desktop access should be sufficient in most cases. If you want Textinator to detect screenshots in other locations or if you have changed the default location for new screenshots, you will need to grant Full Disk Access.

Grant Full Disk Access:

  • Open System Settings...>Privacy & Security> Full Disk Access
  • Click the padlock if locked to unlock it and add Textinator to the list of allowed apps

System Preferences > Security & Privacy

Upgrading

To upgrade to the latest version, download the latest installer DMG from releases and drag the Textinator icon to Applications. If you have previously granted Textinator Full Disk Access, you will need to remove Textinator from Full Disk Access and re-add it per the instructions above. (This is a limitation of the macOS security model and not something Textinator can control.)

Usage

  • Launch Textinator from the Applications folder
  • Grant Desktop access if prompted
  • Click the menu bar icon to see preferences

Menu Bar Icon

  • Press ⌘ + ⇧ + 4 (Cmd + Shift + 4) to take a screenshot then paste the detected text wherever you'd like it to be.

  • Textinator can also monitor the clipboard for changes which means you can also copy an image from any app or press Control + ⌘ + ⇧ + 4 (Ctrl + Cmd + Shift + 4) to take a screenshot and copy it to the clipboard without creating a screenshot file. Textinator will then detect any text in the image and copy it to the clipboard, overwriting the copied image. This feature can be disabled by unchecking the "Detect text in images on clipboard" checkbox in the menu.

  • You can also use Textinator from the Services menu in Finder (and other apps). To use this feature, right click on an image file in Finder and select Services > Detect Text With Textinator from the context menu. Alternatively, you can select Finder > Services > Detect text with Textinator from the menu bar.

Settings

  • Text detection threshold confidence: The confidence threshold for text detection. The higher the value, the more accurate the text detection will be but a higher setting may result in some text not being detected (because the detected text was below the specified threshold). The default value is 'Low' which is equivalent to a VNRecognizeTextRequest confidence threshold of 0.3 (Medium = 0.5, High = 0.8).
  • Text recognition language: Select language for text recognition (languages listed by ISO code and are limited to those which your version of macOS supports).
  • Always detect English: If checked, always attempts to detect English text in addition to the primary language selected by Text recognition language setting.
  • Detect text in images on clipboard: If checked, Textinator will monitor the clipboard for changes and detect any text in any images copied to the clipboard. This feature can be disabled by unchecking the "Detect text in images on clipboard" checkbox in the menu.
  • Pause text detection: If checked, Textinator will not detect text in screenshots or images copied to the clipboard. If paused, the menu bar icon will change and the menu will show Resume text detection instead of Pause text detection.
  • Detect QR Codes: In addition to detecting text, also detect QR codes and copy the decoded payload text to the clipboard.
  • Notification: Whether or not to show a notification when text is detected.
  • Keep linebreaks: Whether or not to keep linebreaks in the detected text; if not set, linebreaks will be stripped.
  • Append to clipboard: Append to the clipboard instead of overwriting it.
  • Clear clipboard: Clear the clipboard.
  • Confirm clipboard changes: Show a confirmation dialog with detected text before copying to the clipboard.
  • Start Textinator on login: Add Textinator to the Login Items list so it will launch automatically when you login. This will cause Textinator to prompt for permission to send AppleScript events to the System Events app (see screnshot below).
  • About Textinator: Show the about dialog.
  • Quit Textinator: Quit Textinator.

When you first select Start Textinator on login, you will be prompted to allow Textinator to send AppleScript events to the System Events app. This is required to add Textinator to the Login Items list. The screenshot below shows the prompt you will see.

System Events permission

Inspiration

I heard mikeckennedy mention Text Sniper on Python Bytes podcast #284 and thought "That's neat! I bet I could make a clone in Python!" and here it is. You should listen to Python Bytes if you don't already and you should go buy Text Sniper!

This project took a few hours and the whole thing is a few hundred lines of Python. It was fun to show that you can build a really useful macOS native app in just a little bit of Python.

Textinator was featured on Talk Python to Me! Thanks Michael Kennedy for hosting me!

How Textinator Works

Textinator is built with rumps (Ridiculously Uncomplicated macOS Python Statusbar apps) which is a python package for creating simple macOS Statusbar apps.

At startup, Textinator starts a persistent NSMetadataQuery Spotlight query (using the pyobjc Python-to-Objective-C bridge) to detect when a new screenshot is created.

When the user creates screenshot, the NSMetadataQuery query is fired and Textinator performs text detection using a Vision VNRecognizeTextRequest call.

Textinator can also monitor the clipboard and detect text in images copied to the clipboard.

Notes

  • If building with pyenv installed python, you'll need to build the python with framework support:
    • env PYTHON_CONFIGURE_OPTS="--enable-framework" pyenv install -v 3.9.11
  • Requires a minimum of macOS Catalina (10.15). Tested on macOS Catalina (10.15.7), Big Sur (11.6.4), Ventura (13.5.1); should work on Catalina or newer.

License

MIT License

See Also

Text Sniper which inspired this project.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Bernhard Wagner
Bernhard Wagner

🤔 💻 ⚠️

This project follows the all-contributors specification. Contributions of any kind welcome!

Developer Notes

If you want to work on Textinator yourself or contribute changes, here are some notes:

Clone the repo and cd into the repo directory.

git clone [email protected]:RhetTbull/textinator.git cd textinator

If you want to contribute back to Textinator, fork the repo and clone your fork instead.

Install requirements and development requirements via pip:

python3 -m pip install -r requirements.txt -r dev_requirements.txt
pre-commit install

See also notes below about Testing.

Building the DMG for distribution requires create-dmg which can be installed with homebrew:

brew install create-dmg

To build Textinator, run the build.sh script:

./build.sh

This script cleans out old build files, builds the app with py2app, signs the app, and builds the DMG.

Textinator stores it's preferences in ~/Library/Application\ Support/Textinator/Textinator.plist. This is non-standard (by convention, apps store their preferences in ~/Library/Preferences/), but RUMPS doesn't provide a method to access the Preferences folder and it does provide a method to access the Application Support folder (rumps.App.open()), so I went with that.

The preferences can be read from the command line with:

defaults read ~/Library/Application\ Support/Textinator/Textinator.plist

For development and debugging it may be helpful to enable the debug log by setting debug=1 in Textinator.plist. You can do this from the command line with:

defaults write ~/Library/Application\ Support/Textinator/Textinator.plist debug -bool true

Similarly, you can disable the debug log with:

defaults write ~/Library/Application\ Support/Textinator/Textinator.plist debug -bool false

When debug is enabled, Textinator will log to ~/Library/Application\ Support/Textinator/Textinator.log. I find this more convenient than using the macOS Console app. Textinator will always log to the Console log as well so you can use Console if you prefer and filter on Textinator.

Most features of the app can be tested by simply running the textinator.py script: python3 src/textinator.py. The Services menu feature requires the app be built and installed because it needs runtime access to information in the app bundle's Info.plist which is built by py2app.

The version number is incremented by bump2version which is installed via python3 -m pip install -r dev_requirements.txt. To increment the version number, run bumpversion patch or bumpversion minor or bumpversion major as appropriate. See bumpversion --help for more information.

I've tried to document the code well so that you can use Textinator as a template for your own apps. Some of the features (such as creating a Services menu item) are not well documented (especially with respect to doing these things in python) and took me a lot of trial and error to figure out. I hope that this project will help others who want to build macOS native apps in python.

Testing

Textinator uses pytest to run unit tests. To run the tests, run pytest from the project root directory. Before running the tests, you'll need to install the development requirements via python3 -m pip install -r dev_requirements.txt. You will also need to enable your Terminal app to control your computer in System Preferences > Security & Privacy > Privacy > Accessibility. This is because the testing uses System Events scripting via applescript to simulate user actions such as clicking menu items. Your Terminal will also need to be granted Full Disk Access in System Preferences > Security & Privacy > Privacy > Full Disk Access.

The test suite requires the built app to be installed in /Applications/Textinator.app. Before running tests, uses ./build.sh to build the app then copy dist/Textinator.app to /Applications/Textinator.app.

The tests will modify the Textinator preferences but will backup your original preferences and restore them when testing is completed. The tests will also modify the clipboard and will create temporary files on the Desktop which will be cleaned up when testing is completed.

The test suite is slow due to required sleeps to allow the app to respond, Spotlight to index new files, etc. (Takes approximately 5 minutes to run on my MacBook Air). Because the test suite interacts with the user interface, it is best not to touch the keyboard or mouse while the tests are running.

The Services menu item is not tested by the test suite so this feature should be tested manually.

textinator's People

Contributors

allcontributors[bot] avatar bwagner avatar rhettbull avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

textinator's Issues

Add option to copy last detected text to clipboard

          > How about this, would it be possible to keep a clipboard cache that stores the text of the previous OCR job? I

That's not a bad idea. You could do this with a clipboard management app but it should be easy to add a "Copy last text detection to clipboard" feature and maybe an option to pop up a window with the last detection.

Originally posted by @RhetTbull in #31 (comment)

Feature: Pause app

Sometimes you don't want your clipboard affected when taking a screenshot containing text. It would be nice to be able to pause Textinator. Even better: Pause it for a period of time (or, if possible, for the next screenshot).
And/or add a configuration option whether to display a confirm dialog whenever Textinator is about to affect your clipboard.

Feature: Screenshot in clipboard

When you use Ctrl-Cmd-Shift-4 to copy a section of the screen, it is not saved as a screenshot, but is merely kept in the Clipboard. Often I don't need to have a screenshot as a file when capturing it just for the sake of text recognition. It would be useful to have Textinator also react to this constellation and perform text recognition of a picture in the clipboard.

Proposition: Make Textinator Into a Generalized Extensible Clipboard Manipulation Tool

I often wish for additional features to Textinator's existing functionality, like:

  • titlecase the contents of the clipboard, e.g. "the meaning of life" -> "The Meaning of Life"
  • when the clipboard contains a filename, replace invalid/unwanted characters, e.g.:
    "~/Documents/der frühling [Vol.2].pdf" -> "~/Documents/der_fruehling_Vol_2.pdf"
  • When copying text out of an Apple "Books" document, that app feels compelled to remind me that the text is copyrighted by appending a litany to what I had chosen to copy. In the case of program code that is particularly disruptive. Textinator would offer to revert the clipboard contents to the originally intended. Example: Highlight and copy some code out of Mark Lutz' "Programming Python, 4th edition", p. 972:
import time
gmt = time.gmtime(time.time())

What ends up in your clipboard instead:

import time
gmt = time.gmtime(time.time())”

Excerpt From
Programming Python
Mark Lutz
This material may be protected by copyright.

So, Textinator would become like a hub for clipboard manipulations of which its existing OCR-functionality is but a plugin.

Here's an example script I currently have in my bin-directory, which would become a plugin in this Textinator-hub:

#!/usr/bin/env python

from titlecase import titlecase
import pyperclip

pyperclip.copy(titlecase(pyperclip.paste()))

Some of Textinator's menu entries don't follow the official Apple Guideline

Some of Textinator's menu entries don't follow the official Apple Guideline: "Use title-style capitalization if you need to use more than one word in the menu title.":

  • Text detection confidence threshold -> - Text Detection Confidence Threshold
  • Text recognition language -> - Text Recognition Language
  • Always detect English -> - Always Detect English
  • Detect text in images on clipboard -> - Detect Text in Images on Clipboard
  • Pause text detection -> - Pause Text Detection
  • Detect QR codes -> - Detect QR Codes
  • Keep linebreaks -> - Keep Linebreaks
  • Append to clipboard -> - Append to Clipboard
  • Clear clipboard -> - Clear Clipboard
  • Confirm clipboard changes -> - Confirm Clipboard Changes
  • Start Textinator on login -> - Start Textinator on Login

Update rumps to 0.4.0

Rumps 0.4.0 now supports python 3.10
Update Textinator to use 0.4.0 and update docs

Unable to build from source

Following the instructions from Developer Notes with a few additions:

  • Python 3.10.13
  • OSX version 14.1.1 Sonoma on Apple M1
git clone https://github.com/RhetTbull/textinator.git
cd textinator
python -m venv ~/venv/textinator
source ~/venv/textinator/bin/activate
python3 -m pip install --no-cache-dir -r requirements.txt
python3 -m pip install --no-cache-dir -r dev_requirements.txt

The second pip install issues errors:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pyobjc-framework-coreml 10.0 requires pyobjc-core>=10.0, but you have pyobjc-core 8.5.1 which is incompatible.
pyobjc-framework-coreml 10.0 requires pyobjc-framework-Cocoa>=10.0, but you have pyobjc-framework-cocoa 8.5.1 which is incompatible.
pyobjc-framework-vision 10.0 requires pyobjc-core>=10.0, but you have pyobjc-core 8.5.1 which is incompatible.
pyobjc-framework-vision 10.0 requires pyobjc-framework-Cocoa>=10.0, but you have pyobjc-framework-cocoa 8.5.1 which is incompatible.
pyobjc-framework-vision 10.0 requires pyobjc-framework-Quartz>=10.0, but you have pyobjc-framework-quartz 8.5.1 which is incompatible.

I don't even understand where the pyobjc-core 8.5.1 comes from, since the requirements.txt asks for pyobjc-core>=9.2. I was deliberately using option --no-cache-dir for the pip install to make sure that no cached version of the packages would be installed.

Trying to solve it:

pip-compile requirements.txt dev_requirements.txt  -o -

gives me:

  ERROR: Cannot install -r dev_requirements.txt (line 2), -r requirements.txt (line 1) and pyobjc-core>=9.2 because these package versions have conflicting dependencies.

running build.sh in spite of this still generates a DMG file, but the contained application aborts with "Launch error".
Towards the end of the build-run there's:

...

Modules not found (unconditional imports):
 * _abcoll (rumps.packages.ordereddict)
 * _io._WindowsConsoleIO (importlib._bootstrap_external)
 * _overlapped (asyncio.windows_events)
 * com (com.sun.jna)
 * com.jna (com.sun)
 * com.sun (com.sun.jna.platform)
 * dummy_thread (rumps.packages.ordereddict)
 * jinja2 (pkg_resources._vendor.pyparsing.diagram)
 * pyparsing (pkg_resources._vendor.pyparsing.diagram)
 * railroad (pkg_resources._vendor.pyparsing.diagram)
 * thread (rumps.packages.ordereddict)
 * win32com (win32com)
 * win32com.shell (win32com.shell)
 * win32com.shellcon (win32com.shell)

Modules not found (conditional imports):
 * _manylinux (pkg_resources._vendor.packaging._manylinux)
 * com (pkg_resources._vendor.appdirs)
 * com.sun.jna (pkg_resources._vendor.appdirs)
 * com.sun.jna.platform (pkg_resources._vendor.appdirs)
 * pep517 (importlib.metadata)
 * win32com (pkg_resources._vendor.appdirs)
 * win32com.shell (pkg_resources._vendor.appdirs)
 
 ...

Running the generated application from the command line with /Applications/Textinator.app/Contents/MacOS/Textinator gives me:

ModuleNotFoundError: No module named 'rumps'

How do I set up my development environment for this to compile? I'd love to contribute to the project.

test suite currently has 8 passing and 10 failing tests

The testsuite currently seems to be failing:

================================================================= short test summary info ==================================================================
FAILED tests/test_textinator.py::test_screenshot_no_notification - AssertionError: assert 'notification:' not in '2023-11-17T... happened.\n'
FAILED tests/test_textinator.py::test_screenshot_qrcode - AssertionError: assert '' == 'https://gith...ll/textinator'
FAILED tests/test_textinator.py::test_screenshot_qrcode_with_text - AssertionError: assert 'https://github.com/RhetTbull/textinator' in 'SCAN ME'
FAILED tests/test_textinator.py::test_screenshot_qrcode_with_text_no_detect - AssertionError: assert 'https://git...l/textinator' not in 'SCAN ME\nht...l...
FAILED tests/test_textinator.py::test_pause - AssertionError: assert 'Paused\nHello World' == 'Paused'
FAILED tests/test_textinator.py::test_confidence - AssertionError: assert '' == 'Hello World'
FAILED tests/test_textinator.py::test_clipboard_basic - AssertionError: assert '' == 'Hello World'
FAILED tests/test_textinator.py::test_confirm_clipboard_changes_yes - assert False
FAILED tests/test_textinator.py::test_confirm_clipboard_changes_no - assert "'confirmation': 1" in "2023-11-17T16:11:19.032612 - search: an update happen...
FAILED tests/test_textinator.py::test_about - assert False
========================================================= 10 failed, 8 passed in 132.69s (0:02:12) =========================================================

Add help

Add a help menu that uses Help Viewer to show help page.

Add ability to override Dropbox's "Share screenshots and screen recordings" link sharing

Textinator doesn't OCR the text when Dropbox's "Share screenshots and screen recordings using Dropbox" setting is toggled on. Instead of getting a copy of the text found in the screenshot, I receive a copy of the Dropbox share url to the screenshot

Would it be possible to add the ability to override this feature?

Just curious because it takes many clicks to toggle Dropbox's feature on/off.

Thank you!

Screenshot 2024-04-28 at 8 08 05 AM

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.