perama-v / min-know Goto Github PK

Problem

The data structures that min-know creates are distributable over IPFS by design (they are TODD-compliant). Currently the files are not shared over IPFS after a user obtains them.

Solution

Add the ability for a user to share over IPFS after obtaining the Chapters relevant to them.

Perhaps using: https://github.com/ferristseng/rust-ipfs-api to pin files to an existing local IPFS node.

Description

The library is generic and when initialising a database, the spec must be provided:

let db: Todd<AAISpec> = Todd::init(data_kind, DirNature::Sample)?;

However, you are allowed to pass in a mismatched data_kind:

// Should be this:
// let data_kind = DataKind::AddressAppearanceIndex(Network::default());

// But this is permitted:
let data_kind = DataKind::NameTags 

let db: Todd<AAISpec> = Todd::init(data_kind, DirNature::Sample)?;

Solution

Ideally, have the init() function derive the spec type from the data_kind enum to allow
omission of explicit typing.

let db = Todd::init(data_kind, DirNature::Sample)?;

Description

For a publisher to build/maintain the signatures database, min-know must ingest data from a raw source. The source must
have some format.

Some more background is previously explored here: https://perama-v.github.io/ethereum/protocol/poking/part_8

Resolution

Decide on a format by reviewing available raw data sources. For raw sources that do not already exist in this format, a
transformation may be applied prior to ingestion by min-know.

Existing databases

As previously explored: https://perama-v.github.io/ethereum/protocol/poking/part_8

Formats

The main consideration is file format and how collisions (two strings creating same 4byte signature) are represented.

Hence, flat files appears to be the best raw input format. The API-based databases could be transformed into this format if that
becomes relevant.

Parameter names

Some sources (topic0) separate out data in to with- and without-parameter name buckets. How should this be handled when ingesting raw data into min-know.

Solidity strips out the parameter names prior to signature computation. Thus a user cannot be sure if a given set of names
is correct. If they have the source code, they can (e.g., sourcify), but then they will not need the signature database (they can compute them from the ABI they have).

Leaning toward not distinguishing between them in the database, but undecided.

Open questions

• Are there more sources available?
• Of the sources listed that have an API but no database, can the format for this internal data be discovered to make min-know useful for them?
• Are any of the sources interested in transforming their data with min-know in order to publish with the in a distributable form (e.g., according to the signatures spec here: https://github.com/perama-v/TODD/blob/main/example_specs/signatures.md)

Problem

The prototype is an implementation of the address appearance index specs. The specs are a specific instantiation of the more general time ordered distributable database framework. Some parts of the prototype would be common to other instantiations and could be shared.

Solution

Generalise the core functions of the prototype so that it could be used for converting and managing other databases.
Make the core functions operate on more abstract terms, following RecordKey/RecordValue terminology as per:
perama-v/TODD#2