wolfmanstout / screen-ocr Goto Github PK

View Code? Open in Web Editor NEW

37.0 37.0 7.0 216 KB

Easily perform OCR on portions of the screen, choosing from a selection of backends.

License: Apache License 2.0

Python 100.00%

ocr tesseract

screen-ocr's People

Contributors

Stargazers

Watchers

Forkers

mrob95 lexiconcode praneetshekhar sacho roman-k-tech mehdiordi

screen-ocr's Issues

winrt doesn't work well with a single digit?

While pytesseract.image_to_string(im, config="--psm 6") returns 6.

Newer library awailable: winsdk

Hello,

Seems 'winrt' is outdated and not supported. There is community substitute and it has exactly same api:
Is that possible to move to newer one?

https://stackoverflow.com/questions/69610231/why-cant-pip-find-winrt
https://pypi.org/project/winsdk/#history

ImportError: numpy.core.multiarray failed to import

On Windows 10:

(Testing) PS E:\> pip install screen-ocr[winrt]
Collecting screen-ocr[winrt]
  Downloading screen_ocr-0.3.0-py3-none-any.whl (15 kB)
Collecting rapidfuzz
  Downloading rapidfuzz-2.13.2-cp310-cp310-win_amd64.whl (1.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 4.1 MB/s eta 0:00:00
Requirement already satisfied: pillow in e:\venvs\testing\lib\site-packages (from screen-ocr[winrt]) (9.3.0)
Collecting screen-ocr[winrt]
  Downloading screen_ocr-0.2.0-py2.py3-none-any.whl (13 kB)
Collecting scikit-image
  Downloading scikit_image-0.19.3-cp310-cp310-win_amd64.whl (12.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.0/12.0 MB 7.5 MB/s eta 0:00:00
Requirement already satisfied: numpy in e:\venvs\testing\lib\site-packages (from screen-ocr[winrt]) (1.21.4)
Collecting screen-ocr[winrt]
  Downloading screen_ocr-0.1.1-py2.py3-none-any.whl (12 kB)
Requirement already satisfied: pandas in e:\venvs\testing\lib\site-packages (from screen-ocr[winrt]) (1.3.4)
  Downloading screen_ocr-0.1.0-py2.py3-none-any.whl (12 kB)
WARNING: screen-ocr 0.1.0 does not provide the extra 'winrt'
Collecting pytesseract
  Downloading pytesseract-0.3.10-py3-none-any.whl (14 kB)
Requirement already satisfied: pytz>=2017.3 in e:\venvs\testing\lib\site-packages (from pandas->screen-ocr[winrt]) (2021.3)
Requirement already satisfied: python-dateutil>=2.7.3 in e:\venvs\testing\lib\site-packages (from pandas->screen-ocr[winrt]) (2.8.2)
Collecting packaging>=21.3
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting scipy>=1.4.1
  Downloading scipy-1.9.3-cp310-cp310-win_amd64.whl (40.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.1/40.1 MB 20.4 MB/s eta 0:00:00
Collecting PyWavelets>=1.1.1
  Downloading PyWavelets-1.4.1-cp310-cp310-win_amd64.whl (4.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 29.5 MB/s eta 0:00:00
Collecting networkx>=2.2
  Downloading networkx-2.8.8-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 25.5 MB/s eta 0:00:00
Collecting imageio>=2.4.1
  Downloading imageio-2.22.4-py3-none-any.whl (3.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 30.8 MB/s eta 0:00:00
Collecting tifffile>=2019.7.26
  Downloading tifffile-2022.10.10-py3-none-any.whl (210 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 210.3/210.3 kB 6.5 MB/s eta 0:00:00
Collecting pyparsing!=3.0.5,>=2.0.2
  Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Requirement already satisfied: six>=1.5 in e:\venvs\testing\lib\site-packages (from python-dateutil>=2.7.3->pandas->screen-ocr[winrt]) (1.16.0)
Installing collected packages: tifffile, scipy, rapidfuzz, PyWavelets, pyparsing, networkx, imageio, packaging, scikit-image, pytesseract, screen-ocr
Successfully installed PyWavelets-1.4.1 imageio-2.22.4 networkx-2.8.8 packaging-21.3 pyparsing-3.0.9 pytesseract-0.3.10 rapidfuzz-2.13.2 scikit-image-0.19.3 scipy-1.9.3 screen-ocr-0.1.0 tifffile-2022.10.10

Quick test spin and immediate failure:

>>> import screen_ocr
RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "E:\venvs\Testing\lib\site-packages\screen_ocr\__init__.py", line 1, in <module>
    from ._screen_ocr import *
  File "E:\venvs\Testing\lib\site-packages\screen_ocr\_screen_ocr.py", line 7, in <module>
    from skimage import filters, morphology, transform
  File "E:\venvs\Testing\lib\site-packages\skimage\__init__.py", line 157, in <module>
    from .util.dtype import (img_as_float32,
  File "E:\venvs\Testing\lib\site-packages\skimage\util\__init__.py", line 16, in <module>
    from ._montage import montage
  File "E:\venvs\Testing\lib\site-packages\skimage\util\_montage.py", line 4, in <module>
    from .. import exposure
  File "E:\venvs\Testing\lib\site-packages\skimage\_shared\lazy.py", line 62, in __getattr__
    return importlib.import_module(f'{package_name}.{name}')
  File "E:\Python\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "E:\venvs\Testing\lib\site-packages\skimage\exposure\__init__.py", line 6, in <module>
    from ._adapthist import equalize_adapthist
  File "E:\venvs\Testing\lib\site-packages\skimage\exposure\_adapthist.py", line 21, in <module>
    from ..color.adapt_rgb import adapt_rgb, hsv_value
  File "E:\venvs\Testing\lib\site-packages\skimage\color\__init__.py", line 1, in <module>
    from .colorconv import (convert_colorspace,
  File "E:\venvs\Testing\lib\site-packages\skimage\color\colorconv.py", line 55, in <module>
    from scipy import linalg
  File "E:\venvs\Testing\lib\site-packages\scipy\__init__.py", line 225, in __getattr__
    return _importlib.import_module(f'scipy.{name}')
  File "E:\Python\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "E:\venvs\Testing\lib\site-packages\scipy\linalg\__init__.py", line 197, in <module>
    from ._misc import *
  File "E:\venvs\Testing\lib\site-packages\scipy\linalg\_misc.py", line 3, in <module>
    from .blas import get_blas_funcs
  File "E:\venvs\Testing\lib\site-packages\scipy\linalg\blas.py", line 213, in <module>
    from scipy.linalg import _fblas
ImportError: numpy.core.multiarray failed to import

For what it's worth if it helps diagnose:

(Testing) PS E:\> python --version
Python 3.10.4
(Testing) PS E:\> pip freeze
beautifulsoup4==4.11.1
cftime==1.5.1.1
chardet==5.0.0
click==8.1.3
colorama==0.4.5
comtypes==1.1.11
decorator==5.1.1
elevate==0.1.3
func-timeout==4.3.5
humanfriendly==10.0
humanize==4.1.0
imageio==2.22.4
lxml==4.9.1
netCDF4==1.5.8
networkx==2.8.8
nose==1.3.7
numpy==1.21.4
packaging==21.3
pandas==1.3.4
pathvalidate==2.5.0
Pillow==9.3.0
psutil==5.9.4
pynmea2==1.18.0
pyodbc==4.0.32
pyparsing==3.0.9
pypyodbc==1.3.6
pyreadline3==3.4.1
pytesseract==0.3.10
python-dateutil==2.8.2
python-magic-bin==0.4.14
pytz==2021.3
pyuac==0.0.3
PyWavelets==1.4.1
pywin32==304
pywinauto==0.6.8
rapidfuzz==2.13.2
scikit-image==0.19.3
scipy==1.9.3
screen-ocr==0.1.0
six==1.16.0
soupsieve==2.3.2.post1
svn==1.0.1
tee==0.0.3
tifffile==2022.10.10
winregistry==1.1.1

Two different versions of 'screen_ocr' are downloaded during pip install

Update
The underlying issue 'screen_ocr' downloaded twice with 2 different versions. screen-ocr-0.1.0 is installed not screen_ocr-0.1.1 despite 0.1.1 downloaded 1st. I suspect this is a PIP issue than the screen-ocr package.

See more details #3 (comment)

PS C:\Users\MainLaptop> pip install screen-ocr[winrt] --no-cache-dir
Collecting screen-ocr[winrt]
  Downloading screen_ocr-0.1.1-py2.py3-none-any.whl (12 kB)
....
Collecting screen-ocr[winrt]
  Downloading screen_ocr-0.1.0-py2.py3-none-any.whl (12 kB)
WARNING: screen-ocr 0.1.0 does not provide the extra 'winrt'

pip 21.1.2 from c:\users\mainlaptop\appdata\local\programs\python\python39\lib\site-packages\pip (python 3.9)

Installing from GitHub from zip pip install -e .[winrt] everything works as expected.

Original Issue

I originally tried from the readme python -m screen_ocr
No module named screen_ocr.__main__; 'screen_ocr' is a package and cannot be directly executed

I tried executing the contents of __main__ in an editor and received the following.

Traceback (most recent call last):
  File "c:\Users\Main\Desktop\screen_ocr.py", line 1, in <module>
    import screen_ocr
  File "c:\Users\Main\Desktop\screen_ocr.py", line 3, in <module>
    ocr_reader = screen_ocr.Reader.create_quality_reader(radius=10000)
AttributeError: partially initialized module 'screen_ocr' has no attribute 'Reader' (most likely due to a circular import)

import screen_ocr

ocr_reader = screen_ocr.Reader.create_quality_reader(radius=10000)

results = ocr_reader.read_nearby(screen_coordinates=(0,0))
print(results.as_string())

just select a rect rather than use radius

Nice code!

I get why you are using radius as a selection technique (as demo'd in your tobii video) - but is there a simple way to just select say a specific rect on the page/. If I use radius I'm going to get stuff I don't need

(I basically am trying to capture the message bar area seen here: https://youtu.be/VPfV0_kevLk?t=69 - rather than all the other squares of words)

Multi-monitor support

Unless I'm missing something I do not believe the package supports multi-monitors. This could be handled outside the package but it seems to make sense to have it integrated.

    def _screenshot_nearby(self, screen_coordinates):
        # TODO Consider cropping within grab() for performance. Requires knowledge
        # of screen bounds.
        screenshot = ImageGrab.grab()
        bounding_box = (max(0, screen_coordinates[0] - self.radius),
                        max(0, screen_coordinates[1] - self.radius),
                        min(screenshot.width, screen_coordinates[0] + self.radius),
                        min(screenshot.height, screen_coordinates[1] + self.radius))
        screenshot = screenshot.crop(bounding_box)
        return screenshot, bounding_box

Documentation missing.

Hey could you please at least tell the Usage clearly. It is very difficult for me to figure out the usage by myself or if there's any link to usage that would be helpful too.

AttributeError: 'NoneType' object has no attribute 'recognize_async'

I tried to install this package with winrt, but on import I got this error:

Traceback (most recent call last):
  File "C:\Users\Davron\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\Davron\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\projects\ocr\venv\lib\site-packages\screen_ocr\__main__.py", line 10, in <module>       
    results = ocr_reader.read_nearby(screen_coordinates=(0, 0))
  File "C:\projects\ocr\venv\lib\site-packages\screen_ocr\_screen_ocr.py", line 154, in read_nearby
    screenshot, offset=bounding_box[0:2], screen_coordinates=screen_coordinates
  File "C:\projects\ocr\venv\lib\site-packages\screen_ocr\_screen_ocr.py", line 160, in read_image
    result = self._backend.run_ocr(preprocessed_image)
  File "C:\projects\ocr\venv\lib\site-packages\screen_ocr\_winrt.py", line 68, in run_ocr
    lambda: asyncio.run(self._run_ocr_async(image))
  File "C:\Users\Davron\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 432, in result
    return self.__get_result()
  File "C:\Users\Davron\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "C:\Users\Davron\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\projects\ocr\venv\lib\site-packages\screen_ocr\_winrt.py", line 68, in <lambda>
    lambda: asyncio.run(self._run_ocr_async(image))
  File "C:\Users\Davron\AppData\Local\Programs\Python\Python37\lib\asyncio\runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "C:\Users\Davron\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 568, in run_until_complete
    return future.result()
  File "C:\projects\ocr\venv\lib\site-packages\screen_ocr\_winrt.py", line 46, in run_ocr_async
    result = await engine.recognize_async(bitmap)
AttributeError: 'NoneType' object has no attribute 'recognize_async'

Steps:

Install using pip install screen-ocr[winrt]
Download and install Tesseract
Try to run python -m screen_ocr

OC: Windows 10
Python: 3.7.0

"No module named 'winrt'" regardless of backend

Attempting to use this on Linux. I've tried installing it both in poetry 3.9.10 and 3.10.2 environments and my user system python (3.10.4) using either the tesseract or easyocr backends and whenever I try to run python -m screen_ocr to test if it is working, I get the No module named 'winrt' error. Apologies if this is a simple mistake on my part, but I'm at a loss right now.

Screen-OCR Documentation Need

Hello?
I've encountered screen-ocr[winrt] and I'm going to try it.
like.. lang=language, psm=6..etc.. used by tesseract.. How do I set the config parameter with screen-ocr??
I want to control parameters that psm, lang... ect.

If the block area is recognized using ocr_reader.read_screen(area),
the string is not output continuously.
Depending on the +/- number value or color of the letter, the order is not continuous and the order is mixed up.

I want to solve this problem.
Like using config parameter in tesseract..

What is the definition of kwargs in create_quality_reader(cls, **kwargs)?
I'd appreciate it if you could show me an example of how to use it.

I'm asking because I can't figure it out even if I look at the example.
thank U.

PS C:\GitHub> pip install screen-ocr[winrt] 
Collecting screen-ocr[winrt]
  Using cached screen_ocr-0.3.0-py3-none-any.whl (15 kB)
Requirement already satisfied: rapidfuzz in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from screen-ocr[winrt]) (2.11.1)
Requirement already satisfied: pillow in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from screen-ocr[winrt]) (9.2.0)
  Using cached screen_ocr-0.2.0-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: numpy in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from screen-ocr[winrt]) (1.23.0)
Requirement already satisfied: scikit-image in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from screen-ocr[winrt]) (0.19.3)
  Using cached screen_ocr-0.1.1-py2.py3-none-any.whl (12 kB)
Requirement already satisfied: pandas in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from screen-ocr[winrt]) (1.5.0)
  Using cached screen_ocr-0.1.0-py2.py3-none-any.whl (12 kB)
WARNING: screen-ocr 0.1.0 does not provide the extra 'winrt'
Requirement already satisfied: pytesseract in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from screen-ocr[winrt]) (0.3.10)
Requirement already satisfied: pytz>=2020.1 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from pandas->screen-ocr[winrt]) (2022.4)
Requirement already satisfied: python-dateutil>=2.8.1 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from pandas->screen-ocr[winrt]) (2.8.2)
Requirement already satisfied: packaging>=21.3 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from pytesseract->screen-ocr[winrt]) (21.3)
Requirement already satisfied: networkx>=2.2 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from scikit-image->screen-ocr[winrt]) (2.8.7)
Requirement already satisfied: scipy>=1.4.1 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from scikit-image->screen-ocr[winrt]) (1.9.2)
Requirement already satisfied: PyWavelets>=1.1.1 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from scikit-image->screen-ocr[winrt]) (1.4.1)
Requirement already satisfied: tifffile>=2019.7.26 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from scikit-image->screen-ocr[winrt]) (2022.10.10)
Requirement already satisfied: imageio>=2.4.1 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from scikit-image->screen-ocr[winrt]) (2.22.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from packaging>=21.3->pytesseract->screen-ocr[winrt]) (3.0.9)
Requirement already satisfied: six>=1.5 in c:\users\wwade\appdata\local\programs\python\python310\lib\site-packages (from python-dateutil>=2.8.1->pandas->screen-ocr[winrt]) (1.16.0)
Installing collected packages: screen-ocr
Successfully installed screen-ocr-0.1.0
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the 'C:\Users\wwade\AppData\Local\Programs\Python\Python310\python.exe -m pip install --upgrade pip' command.
PS C:\GitHub>

Its windows 10. Win 64. Is there anything else I should install?