staticf0x / unique-matcher Goto Github PK

View Code? Open in Web Editor NEW

4.0 2.0 2.0 8.85 MB

A tool for detecting unique items in PoE using Pillow, OpenCV and Tesseract

License: MIT License

Python 71.57% QML 16.64% Batchfile 0.09% Rust 7.38% Inno Setup 0.87% HTML 3.47%

pathofexile pathofexile-tools

unique-matcher's Introduction

Python projects

Unique Matcher – detect unique items in Path of Exile with OpenCV, Pillow and Tesseract
Daylio Stats – Django App for processing Daylio exports
Daylio Parser – parse Daylio CSV exports
GQL prompt – for browsing GQL schema
Schema Viewer – export yaml/jsonschema into HTML/Markdown
MR monitor – CLI tool for monitoring GitLab merge requests
GitLab Plumber – display info about GitLab pipelines
Python project template

Rust projects

mdbook-abbr – expand abbreviations in mdBook

System setup

~/bin files, ~/.config files and other
Sublime Text & Sublime Merge configs: sublime-config
Sublime Plugins & Snippets: sublime-plugins

Other

Rego syntax definition for Sublime Text: RegoSyntax

unique-matcher's People

Contributors

Stargazers

Watchers

Forkers

thirvondukr shonya3

unique-matcher's Issues

Bug: Clearing result table doesn't reset row numbers

Self explanatory

Eclipse Solaris incorrectly detected

Continuing from #19

Gladius miss-categorization

Seems as though it has miss-categorized the Scaeva with green sockets to the incorrect unique, "The Iron Mass"
I don't know a lot about how this works but I assume it has to do with the green sockets and that the iron mass is pretty green.

Attached are two screenshots.
The Scaeva without green sockets was properly categorized as Scaeva. The other screenshot was the bug.

Debug output

2023-09-17 23:35:50.103 | INFO    | Finding item in screenshot: Z:\Tesseract\UniqueMatcher\data\queue\2023-09-17-23-35-49.png
2023-09-17 23:35:50.184 | DEBUG   | Finding unique control start 1: min_val=0.07177774608135223
2023-09-17 23:35:50.185 | INFO    | Found unidentified item
2023-09-17 23:35:50.224 | DEBUG   | Finding unique control end 1: min_val=0.0661790519952774
2023-09-17 23:35:50.275 | DEBUG   | Unique item area has size: 104x208px
2023-09-17 23:35:50.394 | INFO    | Item base: Gladius
2023-09-17 23:35:50.396 | INFO    | Searching through 2 item base variants
2023-09-17 23:35:50.397 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.397 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.398 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.398 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.399 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.399 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.399 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.401 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.401 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.401 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.402 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.402 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.402 | INFO    | Item Scaeva has 12 variant(s)
2023-09-17 23:35:50.403 | DEBUG   | Cropping image based on item width and height
2023-09-17 23:35:50.403 | DEBUG   | Comparing histograms, hist_val=0.8016457031357794
2023-09-17 23:35:50.404 | DEBUG   | Sockets: 3, min_val: 0.873002827167511
2023-09-17 23:35:50.404 | DEBUG   | Comparing histograms, hist_val=0.8214023598614927
2023-09-17 23:35:50.405 | DEBUG   | Sockets: 2, min_val: 0.7245396971702576
2023-09-17 23:35:50.405 | DEBUG   | Comparing histograms, hist_val=0.8434407443700088
2023-09-17 23:35:50.406 | DEBUG   | Sockets: 1, min_val: 0.8466565608978271
2023-09-17 23:35:50.407 | DEBUG   | Comparing histograms, hist_val=0.8040844075602926
2023-09-17 23:35:50.407 | DEBUG   | Sockets: 3, min_val: 0.9075852632522583
2023-09-17 23:35:50.408 | DEBUG   | Comparing histograms, hist_val=0.8237651763618421
2023-09-17 23:35:50.409 | DEBUG   | Sockets: 2, min_val: 0.733775794506073
2023-09-17 23:35:50.409 | DEBUG   | Comparing histograms, hist_val=0.8442582564324173
2023-09-17 23:35:50.410 | DEBUG   | Sockets: 1, min_val: 0.8462110757827759
2023-09-17 23:35:50.410 | DEBUG   | Comparing histograms, hist_val=0.8071003945812215
2023-09-17 23:35:50.411 | DEBUG   | Sockets: 3, min_val: 0.896314263343811
2023-09-17 23:35:50.411 | DEBUG   | Comparing histograms, hist_val=0.8262279374378269
2023-09-17 23:35:50.412 | DEBUG   | Sockets: 2, min_val: 0.7291882038116455
2023-09-17 23:35:50.412 | DEBUG   | Comparing histograms, hist_val=0.8465699907700848
2023-09-17 23:35:50.413 | DEBUG   | Sockets: 1, min_val: 0.847061276435852
2023-09-17 23:35:50.413 | DEBUG   | Comparing histograms, hist_val=0.8276965579690819
2023-09-17 23:35:50.413 | DEBUG   | Sockets: 3, min_val: 0.9669584631919861
2023-09-17 23:35:50.414 | DEBUG   | Comparing histograms, hist_val=0.8417974335747536
2023-09-17 23:35:50.414 | DEBUG   | Sockets: 2, min_val: 0.7120041847229004
2023-09-17 23:35:50.415 | DEBUG   | Comparing histograms, hist_val=0.8580182881049224
2023-09-17 23:35:50.416 | DEBUG   | Sockets: 1, min_val: 0.8352833390235901
2023-09-17 23:35:50.416 | DEBUG   | Matching algorithm: MatchingAlgorithm.VARIANTS_ONLY
2023-09-17 23:35:50.417 | WARNING | min_val distance too low: 0.012535512447357178
2023-09-17 23:35:50.417 | DEBUG   | Histogram comparison seems to be more precise than template matching
2023-09-17 23:35:50.418 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.419 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.419 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.420 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.420 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.420 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.421 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.421 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.421 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.422 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.422 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.423 | DEBUG   | Changing item base image dimensions
2023-09-17 23:35:50.423 | INFO    | Item The Iron Mass has 12 variant(s)
2023-09-17 23:35:50.423 | DEBUG   | Cropping image based on item width and height
2023-09-17 23:35:50.424 | DEBUG   | Comparing histograms, hist_val=0.7850056564974949
2023-09-17 23:35:50.425 | DEBUG   | Sockets: 3, min_val: 1.0
2023-09-17 23:35:50.425 | DEBUG   | Comparing histograms, hist_val=0.8066186514033453
2023-09-17 23:35:50.426 | DEBUG   | Sockets: 2, min_val: 0.8600807189941406
2023-09-17 23:35:50.426 | DEBUG   | Comparing histograms, hist_val=0.8166058582770473
2023-09-17 23:35:50.427 | DEBUG   | Sockets: 1, min_val: 1.0
2023-09-17 23:35:50.427 | DEBUG   | Comparing histograms, hist_val=0.788355888132484
2023-09-17 23:35:50.428 | DEBUG   | Sockets: 3, min_val: 1.0
2023-09-17 23:35:50.428 | DEBUG   | Comparing histograms, hist_val=0.8089981626372713
2023-09-17 23:35:50.429 | DEBUG   | Sockets: 2, min_val: 0.7206321358680725
2023-09-17 23:35:50.429 | DEBUG   | Comparing histograms, hist_val=0.8167029753414062
2023-09-17 23:35:50.429 | DEBUG   | Sockets: 1, min_val: 1.0
2023-09-17 23:35:50.430 | DEBUG   | Comparing histograms, hist_val=0.7808457159536304
2023-09-17 23:35:50.430 | DEBUG   | Sockets: 3, min_val: 1.0
2023-09-17 23:35:50.431 | DEBUG   | Comparing histograms, hist_val=0.8036431782125755
2023-09-17 23:35:50.431 | DEBUG   | Sockets: 2, min_val: 0.7519840002059937
2023-09-17 23:35:50.431 | DEBUG   | Comparing histograms, hist_val=0.8132745348178227
2023-09-17 23:35:50.432 | DEBUG   | Sockets: 1, min_val: 1.0
2023-09-17 23:35:50.433 | DEBUG   | Comparing histograms, hist_val=0.8095208678625554
2023-09-17 23:35:50.433 | DEBUG   | Sockets: 3, min_val: 1.0
2023-09-17 23:35:50.434 | DEBUG   | Comparing histograms, hist_val=0.8266067089346673
2023-09-17 23:35:50.434 | DEBUG   | Sockets: 2, min_val: 0.6546422839164734
2023-09-17 23:35:50.434 | DEBUG   | Comparing histograms, hist_val=0.8281651068236053
2023-09-17 23:35:50.435 | DEBUG   | Sockets: 1, min_val: 1.0
2023-09-17 23:35:50.435 | DEBUG   | Matching algorithm: MatchingAlgorithm.VARIANTS_ONLY
2023-09-17 23:35:50.435 | WARNING | hist_val distance too low: 0.017608546297395944
2023-09-17 23:35:50.435 | DEBUG   | Template matching seems to be more precise than histogram comparison
2023-09-17 23:35:50.436 | DEBUG   | Matching algorithm: MatchingAlgorithm.DEFAULT
2023-09-17 23:35:50.436 | WARNING | hist_val distance too low: 0.015190724640086306
2023-09-17 23:35:50.436 | DEBUG   | Template matching seems to be more precise than histogram comparison
2023-09-17 23:35:50.437 | DEBUG   | Loading CSV
2023-09-17 23:35:50.443 | DEBUG   | Writing CSV

Feature: Background noise reduction

What are you trying to achieve?
Explore ways to reduce background noise like in case of Maligaro's Cruelty in the test data.

Describe the solution you'd like

What have you tried already?

Additional info

Add a desktop shortcut during installation

Create a single installer

This is required for the public release.

Ideally we need:

Package the app with PyInstaller so only one exe is needed to be run
Include all the steps from install.py
(If possible) install tesseract

Add tesseract to installer

Feature: Benchmark should log incorrectly processed images

Since we now have over 6000 screenshots, it would be great if it could just log the ones that fail so it's easy to find them and fix them.

Bug: pytests fails on PIL.UnidentifiedImageError but only in GH actions

FAILED tests/test_item_db.py::test_dimensions_from_image - PIL.UnidentifiedImageError: cannot identify image file '/home/runner/work/unique-matcher/unique-matcher/assets/items/Abberaths_Hooves.png'

This however works totally fine on my machine.

Find all non-global drops

There are still some non-global drops enabled, remove them.

Make screenshot shortcut configurable

Bug: exceptions besides UM base one not handled

Any other exceptions from used libraries or OSErrors will cause the app to hang.

Sort processed screenshots into item named folders

Upon successful detection, we could sort the screenshot into data/done/Item_Name and then use that for testing data.

After a new item is added to TableView, scroll down to it

Bug: If all results have min_val=1, no item should be identified

Describe the bug
Title

Expected behavior
This should raise an exception like if no item is found.

Add menu options to open various folders/files

Would be helpful to have some menu to open: folder with errors, done, results, maybe logs. A button to open the current CSV.

Feature: Add item coverage % to benchmark

What are you trying to achieve?
Now it says how many items there are in the data set, but not how many are in the DB, so we should add kind of "coverage" to improve the data output.

Describe the solution you'd like

What have you tried already?

Additional info

Feature: Verification tool

What are you trying to achieve?
Go through data/done, open the screenshot and the item image, ask the user if they match and if so, put the screenshot into tests/test_data/gathered/

Describe the solution you'd like

What have you tried already?
Now it takes too many steps to verify manually.

Additional info

Setup git LFS for the item images

Lifesprig detection

Lifesprig being detected as splinter of the moon (which is drop disabled)

2023-09-18 22:43:58.698 | INFO    | Finding item in screenshot: Z:\Tesseract\UniqueMatcher\data\queue\2023-09-18-22-43-58.png
2023-09-18 22:43:58.789 | DEBUG   | Finding unique control start 1: min_val=0.07177753746509552
2023-09-18 22:43:58.790 | INFO    | Found unidentified item
2023-09-18 22:43:58.837 | DEBUG   | Finding unique control end 1: min_val=0.0661790519952774
2023-09-18 22:43:58.898 | DEBUG   | Unique item area has size: 104x208px
2023-09-18 22:43:59.042 | INFO    | Item base: Driftwood Wand
2023-09-18 22:43:59.043 | INFO    | Searching through 2 item base variants
2023-09-18 22:43:59.044 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.045 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.045 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.046 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.046 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.046 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.047 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.047 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.048 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.048 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.049 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.049 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.050 | INFO    | Item Lifesprig has 12 variant(s)
2023-09-18 22:43:59.050 | DEBUG   | Cropping image based on item width and height
2023-09-18 22:43:59.051 | DEBUG   | Comparing histograms, hist_val=0.7783057155373589
2023-09-18 22:43:59.052 | DEBUG   | Sockets: 3, min_val: 0.5240201354026794
2023-09-18 22:43:59.053 | DEBUG   | Comparing histograms, hist_val=0.8003256025015042
2023-09-18 22:43:59.053 | DEBUG   | Sockets: 2, min_val: 1.0
2023-09-18 22:43:59.054 | DEBUG   | Comparing histograms, hist_val=0.8246409721706743
2023-09-18 22:43:59.054 | DEBUG   | Sockets: 1, min_val: 1.0
2023-09-18 22:43:59.055 | DEBUG   | Comparing histograms, hist_val=0.7780609895520049
2023-09-18 22:43:59.055 | DEBUG   | Sockets: 3, min_val: 0.42058509588241577
2023-09-18 22:43:59.056 | DEBUG   | Comparing histograms, hist_val=0.7982593199100291
2023-09-18 22:43:59.056 | DEBUG   | Sockets: 2, min_val: 1.0
2023-09-18 22:43:59.058 | DEBUG   | Comparing histograms, hist_val=0.8235379901464224
2023-09-18 22:43:59.058 | DEBUG   | Sockets: 1, min_val: 1.0
2023-09-18 22:43:59.058 | DEBUG   | Comparing histograms, hist_val=0.720513772884791
2023-09-18 22:43:59.059 | DEBUG   | Sockets: 3, min_val: 0.44054940342903137
2023-09-18 22:43:59.059 | DEBUG   | Comparing histograms, hist_val=0.7537652134119074
2023-09-18 22:43:59.062 | DEBUG   | Sockets: 2, min_val: 1.0
2023-09-18 22:43:59.062 | DEBUG   | Comparing histograms, hist_val=0.7910426834268216
2023-09-18 22:43:59.062 | DEBUG   | Sockets: 1, min_val: 1.0
2023-09-18 22:43:59.063 | DEBUG   | Comparing histograms, hist_val=0.813328753078876
2023-09-18 22:43:59.063 | DEBUG   | Sockets: 3, min_val: 0.45223498344421387
2023-09-18 22:43:59.064 | DEBUG   | Comparing histograms, hist_val=0.8237267441457768
2023-09-18 22:43:59.064 | DEBUG   | Sockets: 2, min_val: 1.0
2023-09-18 22:43:59.065 | DEBUG   | Comparing histograms, hist_val=0.8407226010907506
2023-09-18 22:43:59.065 | DEBUG   | Sockets: 1, min_val: 1.0
2023-09-18 22:43:59.066 | DEBUG   | Matching algorithm: MatchingAlgorithm.VARIANTS_ONLY
2023-09-18 22:43:59.066 | WARNING | min_val distance too low: 0.0199643075466156
2023-09-18 22:43:59.068 | DEBUG   | Histogram comparison seems to be more precise than template matching
2023-09-18 22:43:59.070 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.071 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.072 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.073 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.073 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.073 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.074 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.074 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.075 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.077 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.077 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.078 | DEBUG   | Changing item base image dimensions
2023-09-18 22:43:59.078 | INFO    | Item Splinter of the Moon has 12 variant(s)
2023-09-18 22:43:59.079 | DEBUG   | Cropping image based on item width and height
2023-09-18 22:43:59.079 | DEBUG   | Comparing histograms, hist_val=0.791768083447189
2023-09-18 22:43:59.080 | DEBUG   | Sockets: 3, min_val: 0.47686055302619934
2023-09-18 22:43:59.080 | DEBUG   | Comparing histograms, hist_val=0.8196966546366297
2023-09-18 22:43:59.080 | DEBUG   | Sockets: 2, min_val: 0.9686436653137207
2023-09-18 22:43:59.081 | DEBUG   | Comparing histograms, hist_val=0.8246386918758664
2023-09-18 22:43:59.081 | DEBUG   | Sockets: 1, min_val: 0.761078953742981
2023-09-18 22:43:59.082 | DEBUG   | Comparing histograms, hist_val=0.7918459995010625
2023-09-18 22:43:59.083 | DEBUG   | Sockets: 3, min_val: 0.40201422572135925
2023-09-18 22:43:59.083 | DEBUG   | Comparing histograms, hist_val=0.8204894007108667
2023-09-18 22:43:59.084 | DEBUG   | Sockets: 2, min_val: 0.9693483710289001
2023-09-18 22:43:59.085 | DEBUG   | Comparing histograms, hist_val=0.8250115295568501
2023-09-18 22:43:59.086 | DEBUG   | Sockets: 1, min_val: 0.712454080581665
2023-09-18 22:43:59.086 | DEBUG   | Comparing histograms, hist_val=0.7408209361541831
2023-09-18 22:43:59.087 | DEBUG   | Sockets: 3, min_val: 0.41681620478630066
2023-09-18 22:43:59.088 | DEBUG   | Comparing histograms, hist_val=0.7844299307264738
2023-09-18 22:43:59.088 | DEBUG   | Sockets: 2, min_val: 0.963662326335907
2023-09-18 22:43:59.089 | DEBUG   | Comparing histograms, hist_val=0.8006940506625019
2023-09-18 22:43:59.089 | DEBUG   | Sockets: 1, min_val: 0.7264325022697449
2023-09-18 22:43:59.089 | DEBUG   | Comparing histograms, hist_val=0.830934961346437
2023-09-18 22:43:59.093 | DEBUG   | Sockets: 3, min_val: 0.44344595074653625
2023-09-18 22:43:59.094 | DEBUG   | Comparing histograms, hist_val=0.8518463574666758
2023-09-18 22:43:59.094 | DEBUG   | Sockets: 2, min_val: 0.9495763778686523
2023-09-18 22:43:59.095 | DEBUG   | Comparing histograms, hist_val=0.8460945124117838
2023-09-18 22:43:59.095 | DEBUG   | Sockets: 1, min_val: 0.689351499080658
2023-09-18 22:43:59.096 | DEBUG   | Matching algorithm: MatchingAlgorithm.VARIANTS_ONLY
2023-09-18 22:43:59.096 | WARNING | min_val distance too low: 0.014801979064941406
2023-09-18 22:43:59.096 | DEBUG   | Histogram comparison seems to be more precise than template matching
2023-09-18 22:43:59.097 | DEBUG   | Matching algorithm: MatchingAlgorithm.DEFAULT
2023-09-18 22:43:59.097 | WARNING | min_val distance too low: 0.01857087016105652
2023-09-18 22:43:59.097 | WARNING | hist_val distance too low: 0.013785009949057625
2023-09-18 22:43:59.098 | DEBUG   | Template matching seems to be more precise than histogram comparison
2023-09-18 22:43:59.098 | DEBUG   | Loading CSV
2023-09-18 22:43:59.099 | DEBUG   | Writing CSV

Non ascii characters don't display in TableView

Disable flickable on the tableview

Make it configurable to choose the desired display for screenshots

Display errors in the results table

So they cannot be easily missed

Problem with detection: Flamesight, Galesight and Thundersight

These 3 items have the same image except for the color of the little gems: red, blue and yellow. However since it's a helm, it can have up to 4 sockets, which makes even histogram comparison impossible.

simple-term-menu is not supported on Windows

Which means the benchmark currently cannot be started on Windows. Also the helper script run_tests.py.

Feature: Better non-global item filtering

What are you trying to achieve?
We need to be able to identify non-global items but exclude them from the results. They must appear in the result table as an error (or rather something visually different).

Describe the solution you'd like
Instead of ignoring the items completely, we may enable:

Jewels
Malachai's Vision
Blood of Corruption

What have you tried already?

Additional info
This might decrease accuracy on existing items. For this reason, there should be some logic to prefer non-global drops.

Feature: combine result CSVs into one

What are you trying to achieve?
Combine two or more result CSVs into one.

Describe the solution you'd like
A UI screen to click on the ones I want to combine with preview of the clicked one and combined

What have you tried already?
Now it needs to be manual

Additional info

Feature: Icon in Qt

The icon is now only missing in the actual app window.

Detect screen with PoE automatically during runtime

Even when PoE is on screen 2, the production_Config.ini says adapter_name=NVIDIA GeForce RTX 3060 Ti(#0) suggesting PoE doesn't change this value when the window is moved to another screen. This might require querying the window manager to see where the window actually is.

Fix sockets and dimensions for the rest of the items

There are still some items that don't have the width,height or sockets,columns filled.

staticf0x / unique-matcher Goto Github PK

unique-matcher's Introduction

Python projects

Rust projects

System setup

Other

unique-matcher's People

Contributors

Stargazers

Watchers

Forkers

unique-matcher's Issues

Recommend Projects

Recommend Topics

Recommend Org