Implement API interface for DSL runtime about rep_lang HOT 9 CLOSED

I have made some decent progress on this over on this branch.

the type signatures are a little different:

pub fn parse_calculation(dsl_document: String) -> Result<syntax::Program, String>

pub fn get_calculation_type(program: syntax::Program) -> Result<types::Scheme, String>

pub fn reduce_calculation(
    prog: syntax::Program,
    input_data: &mut dyn Iterator<Item = eval::Value>,
) -> Result<ReputationCalculationOutput, ReputationCalculationError>


pub struct ReputationCalculationOutput {
    pub rcr_calculation: syntax::Expr,
    pub scheme: types::Scheme,
    pub value: eval::Value,
}

pub enum ReputationCalculationError {
    ArityMismatch(usize, usize),
    ProgramTypeInferenceError(TypeError),
    ProgramValuesUnificationError(TypeError),
}

// I opted to create a single function `get_calculation_type` above, onto whose output these functions can be applied.
// it returns a `Scheme`, which is `Type` along with any free `TVar`s in the type. so it's slightly different but these functions can be made to line up with each other.
pub fn type_arguments(ty: &Type) -> Vec<Type>
pub fn type_return(ty: &Type) -> Type

it seems to be going fairly well, and I have mostly filled in the body of reduce_calculation. it has some bugs which I've already identified / left as TODOs as I went along.

I have not seen any significant issues with the approach thus far - seems like it will work, and may be all wrapped up in a few more days of work.

I have not yet tackled poly_rs::Type::TRelated(op, Type, Type).

from rep_lang.

Looking great!

Clarification- do type_arguments and type_return deal with types::Scheme, rather than Type as indicated?

I have been thinking about ReputationCalculationOutput a bit more. What it really craves is some datum held as an indicator of the source information which yielded the given result.

Maybe a sum would be a good way to carry this information forward in aggregate? So we could include an additional source: eval::Value that can be used in combination with type_arguments to determine what kind of information was passed in, and how many records?

from rep_lang.

Looking great!

Clarification- do type_arguments and type_return deal with types::Scheme, rather than Type as indicated?

Right now they force the user to unpack the Scheme, get out the enclosed Type, and pass that in. I suppose I could make a trait method so that the functions work with both Scheme & Type.

I have been thinking about ReputationCalculationOutput a bit more. What it really craves is some datum held as an indicator of the source information which yielded the given result.

Maybe a sum would be a good way to carry this information forward in aggregate? So we could include an additional source: eval::Value that can be used in combination with type_arguments to determine what kind of information was passed in, and how many records?

Hmm, I think I lack context to understand this & the motivation. Would source be a single element or an iterator/vector of multiple, for the arguments provided to reduce_calculation?

Maybe it would help me to understand the constraints on the “caller” (which is the Vault?) - what it needs and is looking to do.

There’s a lot of data being plumbed around which I could easily imagine being useful “to the outside” and perhaps I can be more systematic in persisting it / making it available.

from rep_lang.

@pospi would it be a good use of time for you to run me through some of the vault so I can understand the constraints? then we can have two brains guiding the structuring of the API.

I see https://github.com/sacredcapital/ReputationVault as empty, maybe code is somewhere else.

from rep_lang.

I think the guts of this are mostly done on the branch. I have not implemented type inference for closures, which means they cannot be provided via the values iterator. I am pretty sure I know how to support them, but figure it is less likely that people will want to send in values?

So, I think we might be able to wire the top level API up to other things (the vault?) now and see what works / breaks.

from rep_lang.

Noting work to do following today's sync call:

• Currently all input_data are being evaluated to prog all at once; items from the iterator need to be incrementally evaluated for memory efficiency (like a foldl implementation). So whereas poly_rs is now handling the full fold operation over the complete set of input data, instead the language should be set up to execute individual fold steps of the current datum and return each accumulated result.
  • Probably the method for making this change would be to alter VList to wrap Iterator instead of Vec.
• Unit / integration testing for key features / aspects of the library so we can prove correctness when making any upgrades going forward
• Do some investigation into blowing the stack with unbounded iterators and have a think about how we can mitigate non-terminating computation
  • For non-agent-centric apps like Junto 1.0, how would they sandbox this? Would it execute on their servers or in a browser WASM sandbox that fetches data from Junto's web app over the network?
  • Could we potentially implement a total language to address this?

from rep_lang.

RE total langauges-

• poly_rs doesn't allow recursion yet
• poly_rs has no looping so far; only map & fold

So, unbounded computation is not an issue until we need such features.

from rep_lang.

• Another missing feature is needed, implementation still unknown but essentially the input_data needs to provide the Type of each datum as well as yielding each Value in turn. This is so that the Type expected as input to the Program can be validated against the Type found for each piece of data - it cannot be relied upon to match since such types are dynamically provided by Reputation Vault plugins.

from rep_lang.

I am favoring a more "organic" approach to developing this top-level API, starting with neighbour-hoods/social_sensemaker#8 as "first contact between Holochain and rep_lang".

there are good things in this issue to revisit when designing the successor "directed experiments" which will follow neighbour-hoods/social_sensemaker#8.

from rep_lang.