grdddj / similar-ordinals Goto Github PK

Website with a basic functionality:

• allow for user inputting a custom file or ordinal ID
• call backend with this data
• display results

In case of duplicates, we should make it clear which ordinal is the original one (first minted) and which is the duplicate. In case of more duplicates, sort them and probably give them ids - "duplicate 1, "duplicate 2", etc.

Because it is not so much fun on localhost.

After being public, we might send the website/API to some potentially interested parties, e.g.:

• webpages with statistics
• minting websites
• ordinals marketplaces

Having a CI running at least some basic static analysis is never a bad thing

Self-explanatory, will rather not mention the ordIDs here

It would be nice to add support for searching ordinals with similar text

All the code changes should be handled by Pull requests and then rebased on top of master

It should contain some useful description

Currently, the python API loads the ordinals fingerprints into memory on startup to start comparing them immediately on each request.

This is good enough and approximately matches the Rust implementation, which loads the JSON file from scratch each time.

The logical conclusion is to create a server in Rust - probably only for the purpose of getting results on the backend - and replicating the approach we have now in python API - load the data from JSON into memory at server startup, so it does not need to waste time for each request.

We can add a button to the popup with details about a similar ordinal that will trigger a similarity search with this ord ID

When some results are cool or important, there may be a need to save/share them.

Could be made possible with optional URL queries, e.g. ordsimilarity.com?ord_id=25 would automatically send a request for results with ord_id=25 and show them

They are currently throwing PIL error, because they are not "normal" image. Try to do it somehow, like taking the first frame and turning it into a picture

When there is a lot of duplicates, what is the case e.f. for ord_id 0, we might want to show just the duplicated ones (maybe sorted by their ID), or also some other ones with similarity 256 to make it interesting

For MVP, it should be fine to work with a static dataset, where we can compare all the ordinals with ord_id less than 1_044_219. Also, the similarity index is built from this dataset, so it does not account for all the newest ordinals above this ord_id.

However, later we might want to do the update process regularly, preferably in real-time, so we always have the latest data and do not miss some similarities.

The update logic could look something like this:

• get all the newly minted ordinals we do not have
• calculate and save hashes for them
• updating the similarity index (this is the tricky part)
  • calculate and save top_n similarities for all the new ones
  • for each ordinal from the old ones calculating similarity with all the new ones, and updating their top_n to possibly include similarities with some of the new ones

It is hard to troubleshoot errors that are not logged somewhere

It would be nice to show which ordinal are group of the same collection - and which are just "cheap copies"

It might be beneficial to use Hiro API, and not only because that way we would qualify for their price on the hackathon.

Actually, their API seems to be really nice - https://docs.hiro.so/ordinals

Some endpoints we could use:

• ordinals/v1/inscriptions
  • https://api.hiro.so/ordinals/v1/inscriptions?from_number=0&limit=60&to_number=59
    • that way we could check that our DB contains the same data, and potentially do some corrections
  • https://api.hiro.so/ordinals/v1/inscriptions?limit=60&from_number=1044000
    • we could use this when implementing the real-time DB update, to get all the ordinals that we do not yet have
• ordinals/v1/inscriptions/{id}/content
  • https://api.hiro.so/ordinals/v1/inscriptions/1213/content
    • we could link to this content when showing the small icon on the frontend

It would be pretty cool if we could have the option to charge some small money amounts (if we decide to) in BTC for using the service.

The API could be behind a "paywall" and users would need to send proof that they paid some satoshis to a lightning wallet.

On the frontend, this could be easily done by a QR code, how to solve it on the backend would require more thought...

Average hash is not the only one relatively-easy-to-use algorithm. There is a lot of other in imagehash library

We should add at least some basic test suite checking that the similarity algorithms (comparing the fingerprints) work as expected.

Some end-to-end tests of finding a similar picture in a directory of other pictures would be also nice.

API with basic functionality:

• take either ord_id or a file upload and return the most similar ordinal documents

Each file and function should preferably have at least a short description

It would be a valuable service to count with ordinals that are still in the mempool

(after a discussion with Robin Obermaier, Co-Founder & CEO at Liquidium)

We want to immediately know that the API is down, so we could troubleshoot it.

We might also have some self-healing capabilities so that the API restarts itself when in problems.