Comments (2)
If we're moving from Uri to Stream, we should also add a MimeType
or ContentType
property to help the client render in the correct format.
from strix-music.
Merging images
We're currently using the Uri
property to determine if an image should or shouldn't be merged with another image.
We need a solution to cover this once Uri is removed.
Generate a checksum
The easiest way to tell if 2 files are the same are to check them byte-per-byte. The calculation can be done ahead of time and a checksum can be created (MD5, SHA1, SHA-256, etc).
This is pretty close to how we avoid processing duplicate images in the SDK for scanned audio files.
However, with this new setup, the calculation must be done somewhere, and it isn't obvious where this should be.
- If implemented in CoreModels
- [con] All cores need to agree on the same hashing algorithm
- [con] Cores might choose not to implement this because of the extra effort.
- [pro] Existing hashes that come from server-side or a file system can be reused, calculation can potentially be sidestepped entirely
- [pro] If not provided by a core, we can do the calculation for them as it's needed when merging.
- If implemented in AppModels
- [con] No option for lazy calculations, potentially more resource intensive
- [con] To stay accurate, checksum must be generated at runtime, every time. There's no way to associate an IImage with a known checksum.
- [pro] Can be done on the fly during the collection merging process
- [pro] The SDK decides on the hashing algorithm
- [pro] Cores don't need to implement anything, just give us a data stream.
- [pro] Opens the door to very optimized image comparison, rather than comparing all the bytes.
Simple Heuristic image comparison
Since we know we're dealing with an image and have access to the raw bytes, some simple heuristic comparisons might be more than enough for now.
- Are the streams the same length? If not, they don't match.
- Checking the metadata, are the images the same resolution? If not, they don't match.
- Ignoring metadata, check the actual image bytes. If they stop matching before reaching the end of the stream, the images don't match.
- We can skip X number of bytes and make this number user-configurable - a speed/accuracy dial, of sorts.
Perceptual hashing algorithm (image fingerprinting)
See https://www.hackerfactor.com/blog/?%2Farchives%2F432-Looks-Like-It.html
This technique is similar to acoustic fingerprinting, but for images.
Rather than generating hashes that are a byte-per-byte representation like MD5 or SHA1, this technique aims to generate hashes that approximately represent what the image looks like.
By reducing size, reducing color, averaging the remaining colors, etc., you can compute a hash from the remaining bytes that is very close to what similar images would have generated.
From there, you can count how many bits don't match (a Hamming distance) and use that to determine how similar the image is.
Example
Pulling from our sources, let's create an image fingerprint for an image. This will be a minimum example to help you (and me) to understand the process, there are significant improvements you can make if you keep researching.
Reduce the image size
- The smaller this image, the more details it removes, the closer it can match other images
- Makes the original image size irrelevant
- Less pixels means less color complexity
Here's the same image, but 8x8 (64 pixels)
Blown up to the original size again:
Reduce color
- Convert to grayscale.
- Each pixel color is now somewhere between black and white.
- We now have 64 total colors in the image.
Average the colors
- Given the 64 colors we have now, find the average (mean) value of the 64 bytes in the image and record it somewhere.
Compute the bits
Using the average from before, go through each bytes and do the following:
- If the byte is above the mean, write a 1 (black)
- If the bytes is below the mean, write a 0 (white)
The result is a new image containing a rough outline of ONLY the most prominent features in the original:
Blown up to the original size again:
Lastly, create the hash.
- The image created in the previous step is an 8x8 image comprising of simple 1's and 0's, meaning it's a total of 64 bits.
- This can be converted to a 64-bit integer:
10319777633083913424
- Then, we can convert from base 10 to base 16 for a hex value:
8f373714acfcf4d0
. This is our hash. - Run this again with a different image and compute the Hamming distance of the hash to see how similar the two images are!
This is more than doable from scratch with just ImageSharp, but there are improved versions that would serve us much better if it's needed.
from strix-music.
Related Issues (20)
- Update timeline on website
- Update "Create a core" docs
- App stops playing when minimized HOT 3
- [Feature] Add a way for users to rehost IPFS content
- Crash when track collection changes on Zune Desktop
- Repeat button not working on Zune Desktop shell HOT 1
- App crashes when using local Kubo node without pubsub enabled HOT 1
- Strix Music Sdk Tests are failing
- [Feature] Log all thrown exceptions, even if caught
- Recreate WASM app head
- Back button does not restart song, always goes back
- Removing the media/audio source/core not working.
- [Feature] IPFS as a music source HOT 1
- Zune Track sorting debounce is too long
- [Feature] StrixDataRootPluginWrapper should expose the plugins provided in the constructor HOT 1
- Lag / delay when playing audio files
- Unable to select Album in Artist view
- Play/Pause button in Zune doesn't sync with System Media Transport controls
- Core disappears once the app is restarted.
- Crash when playing album
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from strix-music.