allenneuraldynamics / aind-behavior-curriculum Goto Github PK

no database stuff, use pd.DataFrame as your interaction layer.

We should consider refactoring the docs website to cross-reference with the API documentation of the package. For instance, Metrics should link to the respective technical reference instead of just highlighting the word.
Additionally, we should consider splitting the technical documentation by module instead of flattening the whole thing. As an example:

The callables in **Policy** and ...

should be replaced by:

The callables in :py:class:`~aind_behavior_curriculum.curriculum.Policy` and...

• add a basic Task schema, based on various examples
• add an example Task in ./examples/
• add a unit test that constructs the example Task

@bruno-f-cruz @hanhou please add links to your repos here.

This repository seems to have a complex class architecture. I went through many of the classes and I still could not understand all dependencies and the logic behind it. They are a few empty classes with empty layers.
I think this might warrant a review of the class architecture given that it is expected from any external users to subclass components to create a new task.

There are instructions for how to do this in the tag_and_publish tag_and_publish.yml.

Also need to add our service account to this repo so it has permission.

Describe the bug
curriculum, not cirriculum

I am going through the repo with an eye for simplicity.
why do we need this empty class?

See here :
https://github.com/AllenNeuralDynamics/aind-behavior-curriculum/blob/bb28161608b4544e5f71e23c2190a6682f17dcaf/src/aind_behavior_curriculum/task.py#L19C22-L19C44

I've begun reimplementing the dynamic routing task regimen here:
https://github.com/AllenNeuralDynamics/dynamic-routing-curriculum?tab=readme-ov-file

• I've made minimal usage of policies and hypergraphs because I want to find out where our problems are in full implementation
• I can see how hypergraphs and policies can reduce the number of redundant stages especially in the cases where I have circular stage transitions to just alternate task parametrization until a certain secondary condition is met.
• I'm in the process of getting trainer to work and if I understand the documentation correctly I may be able to get automagic transitions if I have policy transitions attached that are redundant with the stage transitions.
• This is just here to document my user experience?
• We currently have a part of the regimen that was always moved to via manual intervention and I'm not sure how this will fit in but it might become more clear once I start using Trainer.

It looks like the parameters are directly pass to the a stage object. Could we just have a Stage object? What is the usefulness of these 2 layers?

For some operating systems graphviz may not already be installed. It might be helpful for future users to have a note that they need that in the README as the error raised is informative but doesnt explicitly say "graphviz" anywhere in it and may confuse future users.

Use the same action as aind-data-schema.

• Add the generic Curriculum schema
• Add an example that uses the basic Task from #2

Worst case scenario this should be refactored to be a discriminated union (see more details here AllenNeuralDynamics/aind-data-schema#518).

Best case, see if there is a way to not have to pass the types.

It should mention Stage instead of Trainer I believe?

Policy and Stage remain one of the most subtle concepts in the present library. We should think about a better way to distinguish them. Here's I am currently seeing these two concepts/implementations, feel free to use this thread to discuss and hopefully spin a pull-request to the docs.

On the curriculum

As stage on the current docs:

A Curriculum is structured as a graph of training Stage. Each Stage is associated with a Task, which defines a set of configuration parameters via TaskParameters. Stages are connected by StageTransition, which are directed edges associated with a trigger condition.

In other words, a Curriculum, in its simplest form, can be seen as a container of Stages + any logic associated to the transition between them (StageTransitions).

Stage is, in turn, a "container" of a Task instance. The concept of an "instance" is critical here. While the Task object defines a class, Stage works by wrapping an instance. What this means is that two distinct Stages A and B, could actually implement the same Task type BUT different instances of the same Task (Think about two different training stages of the same task implementation).

Let's step back a bit here and think about what affordances we have at this point.

1 - In each Stage we can define a set of parameters for a specific Task
2 - We can express transition logic that transitions between Stages
3 - We can have Stages of different Tasks or the same Task

Lets look at two ways this could be used (I am going to use pseudo-code for brevity)

Curriculum within the same Task

Let's consider a Task named VrForaging. The experimenter has 2 distinct and discrete operation modes that the animal transitions through training. Let's call them: ForagingForApples and ForageForCheese. The animal starts training on ForagingForApples and after it fulfills a transition criteria (say collects 100 rewards in a behavior session), graduates to ForageForCheese.

Curriculum across Tasks

Let's consider a curriculum where the animal is first trained in the previous task VrForaging and Stage ForageApples. Now, instead of graduating to ForageForcheese, we want to teach the mouse a completely new task, say DynamicForaging. This new task also has stages, say BaitedStage.

As you can see, a Stage is always required. It is the materialization of a Task class/type.

But what about policies?

Consider the previous scenario where the animal is in Task VrForaging and Stage ForagingForApples. Additionally, consider a variable inside the task that depicts the amount of reward the animal receives. Say the experimenter wants to automatically update this value based on the amount of water the animal drank in the previous session.

We can express this logic using Stages by considering:

ForagingForApples(5ul) -> ForagingForApples(6ul) if WaterDrank > 1000 & WaterDrank< 1200;
ForagingForApples(5ul) -> ForagingForApples(7ul) if WaterDrank > 800  & WaterDrank < 1000;
ForagingForApples(5ul) -> ForagingForApples(8ul) if WaterDrank > 600  & WaterDrank < 800;
ForagingForApples(5ul) -> ForagingForApples(9ul) if WaterDrank > 400  & WaterDrank < 600;

While possible this is unnecessarily annoying as we would need to code a large number of stage transitions between all the stages.

To meet this need, we can instead use a Policy. A Policy can be thought of as a set of functions that run on top of the session outcome. For instance, the above scenario could be instead coded in the continuous domain by:

ForagingForApples(x), where x = WaterDrank / 2000

In this example, only one policy is active. However, multiple policies can be active simultaneously that are used to update different set of parameters.

In other words, a Stage, on top of being a simple container for a Task instance, also defines a set of policies. As a result, two stages can define the same Task, but differ solely on the underlying policies. (For instance, one Stage could update the WaterDrank by a factor of 1/2000 whereas the other may do it by a different factor of 1/4000)

Why Policy transitions?

The previous architecture already affords vast flexibility. However, one thing that becomes very difficult is to independently control active policies. Imagine that you have two concurrently active policies (UpdateWater and UpdateDistance). At some point, you want to change the UpdateWater to UpdateWaterByAlot. This could be done by coding an extra stage, as mentioned before. Unfortunately, we need to account for UpdateDistance too!
Couldn't we just add this Policy to the new stage? You could, but what happens if UpdateDistance has a corresponding UpdateDistanceByAlot too? Now we suddenly need to expand our number of stages to account for all possible pair-wise combinations of active policies.

To solve this, each Stage can instead also code its own set of PolicyTransitions that define how different policies transition into others. This allows different policies to be concurrently active within the Stage while simultaneously allow them to be updated to different ones independently.

• aind-behavior-cirriculum -> aind-behavior-curriculum
• fix setup.py/pyproject.toml/subdirectories, etc

Is your feature request related to a problem? Please describe.
From the point of view of the Task schema, no field is available to keep track of stage or experiment type. This would be interesting as the same Task could be used, via different parameters, to instantiate different experiments (or stages).

As I was thinking about this, it felt like violating a bit the separation of concerns principle but I think it may be warranted for a few reasons:

1. If a user wants to buy into the Task specification but does not want to use the full Curriculum (Curriculum actually deserializes the stage information but only with the full curriculum, not the suggestion)
2. For instances o Task kept in a remote database (e.g. slims) it would be nice to have a human readable field that indicated with stage or experiment it is being suggested by the Task json instance.

Describe the solution you'd like

Add stage_name to Task with the following signature:

`stage_name`: Optional[str] = Field(default=None)

The following should be added:

• When a Stage is instantiated if no name is passed to Stage (i.e. = None), a validator should attempt to use Task.stage_name automatically. If stage_name is null, it should throw, since name is a required field.

Describe alternatives you've considered

Leave it up to the user to add this field to TaskParameters. In which case it would not be the concern of the curriculum. But it would also be much harder to generate a common API that uses this field

I am a bit afraid of the possible duplication of metadata between Stage.name and this suggested property. If that is a problem, we may just want to change the name of this property for something else to prevent confusion. Perhaps task_descriptor?

Additional context
This field should only be used for generating human-readable metadata, I would not rely on it for any functional implication in the curriculum. Instead we should rely on Stage.name

• subclass the abstract Task schema
• save a file in examples/dynamic_foraging.py

Just going through this with an eye for simplicity.

I appreciate it is important to validate parameters values against a schema but could external user just pass a dictionary instead of subclassing?