High-level sketch of the problem space about rep_lang HOT 3 CLOSED

The Reputation DSL

The Reputation DSL needs to exist as a flexible, expressive means of codifying reputation calculations so that they can be understood by others. Since reduce is a Turing-complete method, we believe a mapping from the DSL onto a set of MapReduce operations is likely to be an effective implementation path to investigate.

• The reputation calculation being made is a document that is ingested and parsed "online" by the zome (at runtime). This reputation calculation describes how to perform particular computations, but does not itself perform them.
• The computations driven by such a reputation calculation document are hard-coded and of a finite set. Input for the reputation calculations needs to validate that its syntax maps to valid available operations.

In other words— if someone wants to change the weights of a scoring computation (say we decide that being late is more acceptable and relax a threshold for the punctuality score), we do not have to re-deploy a new hApp/DNA. We can just modify something “lighter”, perhaps a datatype in the DHT, which will then be ingested, and the specified computation performed.

• We are likely to need formal typing of basic set operations such as COUNT, SUM, AVG etc in order to determine whether the syntax for a reputation calculation is valid. Looking at things like the BigQuery instruction set could be useful for this.
• Bonus points for the unit type of results to be derivable at execution time (but if syntax can be validated as per point 1, this is probably easy thereafter).
  • Also a need for labelling result types with new language. For example, defining COUNT(upvotes) - COUNT(downvotes) as post rating.
    • Does this become a unit type, or is it handled in some other way?

One idea @mhuesch & I have discussed is a DSL which works with some of Rust's built-in compiler libs, such as syn. It also looks like we aren't the first to consider such a thing— see syn-serde, rustc_ast::visit and Building Fast Interpreters with Rust. I'd kinda expect this to be easy-ish with Rust, given its lineage in ML; but honestly it's all a bit above my pay-grade.

@mhuesch & @pospi think the task is roughly to define a parser for a grammar, and an interpreter for the AST generated by the parser.

• curious about whether you can use Rust's built in parser & AST crawler crates to take a developer-defined language spec (as a parser impl) and parse it safely directly into a Rust-native AST that can be runtime-executed. Seems like that may save a lot of work, because then all we have to do is fill in the blanks in the common functions behind the layer of Rust that the AST parser parses into.
  • not sure about this approach, because it would require us to run the compiler inside of our executable. We would then produce WASM and run that?
  • thinking that the more straightforward parser / typechecker / interpreter (which is written in Rust and compiled to wasm) is the best way.
• concern that typechecking the AST is also non-trivial. That sorta fits into parsing - work that should happen before the AST is interpreted. We could skip typechecking but then things are a bit dangerous and reputation computations could fail at runtime in weird ways.

Other possible features

Chunking of reputation calculation code into hashed blocks. Consider the Reputation DSL as a LISP grammar-

• The AST for leaf nodes which bind to the "native" parts of the DSL ("the computations driven by such a reputation calculation document") or which contain only variable assignments are hashed first.
• Function block is stored as a Reputation Vault DNA entry, hash is now used as its identifier. This bubbles up to the top of the AST.
  • Bonus: creates a weave of interdependent reputation DSL statements between each reputation calculation, which helps to infer commonality and flow of assumptions.
  • Bonus: if newly authored functions match existing ones, this can be detected.
  • Bonus: different versions of reputation calculations can be intelligently diffed, even when the names of variables are changed.

from rep_lang.

Agree on the bit about the Rep Vault. Will share more details shortly, but this should suffice for now. In addition to the zomes mentioned we will have one to manage the 'consent mechanism' i.e. an agent authorising access to reputation data in another neighbourhood + a zome to store the list of 'trusted nodes' through which bridge calls occur + a zome to initiate the bridge call through RPCs.

from rep_lang.

this is useful historical context and parts of it can likely be lifted out (with appropriate credit ofc) into a wiki, when we are at the point of documenting the story of development. for now, it seems distracting as an open issue.

from rep_lang.