and fMOST to modality.
For Lydia

Let's add some examples for:

• DataDescription
• Subject
• Procedures
• Processing

Create generic procedure with date, experimenter, animal weight

We have this in the subject schema for possible restrictions on publication/sharing of data derived from a specific mouse, but these restrictions could also stem from other resources used (eg. reagents). This should be a more generic property that is inherited at the data description level.
Not sure if there are other properties that come with it - will look into that.

Based on input from Bosiljka, including:

• injections that aren't virus
• injections of multiple virus/materials
• include genome copy, viral prep lot #, titer
• ICV/ICM injections

There are two Probe classes right now:

• aind_data_schema.procedures.Probe
• aind_data_schema.ephys_session.Probe

Recommendations:

• rename the ephys_session.Probe --> EphysProbe
• rename the procedures.Probe --> OphysProbe

for example, degree fields are [0,360], etc.

Right now institution and project are replicated in data_description and ephys_session / imaging acquisition. Let's just put them in one place.

User story

As a user, I want to retrieve the json schemas from an aws bucket, so I can avoid running python code and use the JSON files directly if I want.

Acceptance criteria

• When a GitHub Action to push updates to main is run, updated AindCoreModel schemas will be written out as JSON files and uploaded to an aws bucket.
• File names/prefixes contain version numbers (e.g. subject/0.0.1/subject.json).

Sprint Ready Checklist

• 1. Acceptance criteria defined
• 2. Team understands acceptance criteria
• 3. Team has defined solution / steps to satisfy acceptance criteria
• 4. Acceptance criteria is verifiable / testable
• 5. External / 3rd Party dependencies identified
• 6. Ticket is prioritized and sized

Notes

Add any helpful notes here.

Right now the Device base class is a common concept between the ephys and imaging modules. Let's move it up a level and share.

This will be true once ophys gets merged as well.

Motorized stage and DAQ are currently not exposed in a class. However they are a common item across SPIM systems and could be added as a dedicated class.

Suggested addition(s) to instrument.py

class MotorizedStage(Device):
    """Description of a motorized stage device"""

    manufacturer: str = Field(..., title="Manufacturer of device")  
    model: str = Field(..., title="Model of device")  
    axes: str = Field(..., title="Axes of device")
    travel: float = Field(..., title="Travel of device (mm)")

class DAQ(Device):
    """Description of a DAQ device"""

    manufacturer: str = Field(..., title="Manufacturer of device")  
    model: str = Field(..., title="Model of device")
    device_name: int = Field(..., title="PC device name")
    update_frequency: int = Field(..., title="Update frequency [hz]")

An alternative is to add them as an AdditionalImageDevice. Although since they are common to all systems, I think this is not the best solution?

Because genotype strings can change over time, add the MGI allele ID for the different driver/reporters (etc) to the schema

Describe the bug
When running sphinx, the following warning message is displayed:

WARNING: __all__ should be a list of strings, not [<class 'aind_data_schema.data_description.DataDescription'>, <class 'aind_data_schema.data_description.RawDataDescription'>, <class 'aind_data_schema.data_description.DerivedDataDescription'>, <class 'aind_data_schema.procedures.Procedures'>, <class 'aind_data_schema.ephys.ephys_rig.EphysRig'>, <class 'aind_data_schema.ephys.ephys_session.EphysSession'>, <class 'aind_data_schema.subject.Subject'>, <class 'aind_data_schema.subject.LightCycle'>, <class 'aind_data_schema.processing.DataProcess'>, <class 'aind_data_schema.processing.Processing'>, <class 'aind_data_schema.imaging.acquisition.Acquisition'>, <class 'aind_data_schema.imaging.instrument.Instrument'>, <class 'aind_data_schema.imaging.instrument.Microscope'>] (in module aind_data_schema) -- ignoring __all__

To Reproduce
Steps to reproduce the behavior:

1. Run sphinx-apidoc -o doc_template/source src
2. Run sphinx-build -b html doc_template/source/ doc_template/build/html
3. See warning

Expected behavior
We shouldn't be getting any warnings

Additional context
Add any other context about the problem here.

It's mostly a convenience for constructing names and such - it should not be exposed in the schema.

Describe the solution you'd like

Image orientation is often encoded in the community as code such as "RAS", "SLA" etc.
Consider providing a helper function to convert from these code to your Axis model and back.

Code snippet of my conversion

direction_lookup = {}
direction_lookup['L'] = "Left_to_right"
direction_lookup['R'] = "Right_to_left"
direction_lookup['A'] = "Anterior_to_posterior"
direction_lookup['P'] = "Posterior_to_anterior"
direction_lookup['I'] = "Inferior_to_superior"
direction_lookup['S'] = "Superior_to_inferior"

image_axes = {}
image_axes['x'] = {'pos':0, 'label':'X'}
image_axes['y'] = {'pos':1, 'label':'Y'}
image_axes['z'] = {'pos':2, 'label':'Z'}

    axes = [None,None,None]
    
    for a in image_axes :
        
        asize = hrow['%s_size' % a]
        aspacing = hrow['%s_resolution' % a]
        aextent = asize * aspacing
        
        axis = Axis(
            name = image_axes[a]['label'],
            dimension = asize, # this is typically called size
            direction = direction_lookup[hrow['orientation'][image_axes[a]['pos']]],
            voxel_size = aspacing, # this is typically called resolution, spacing, physical step size
            volume_size = aextent,  # this is typically calculate from size * spacing
        )
        axes[image_axes[a]['pos']] = axis

The examples are out of date, let's update them.

• (P0) can be queried via HTTP request (REST is fine, but whatever)
• (P0) query-able by schema name (retrieves latest) and also by version
• (P1) browsable in a web page

Describe the solution you'd like

I believe the target audience you want to capture is the scientist who has learnt how to program.
As such consider providing a little more installation instruction.

Re: pip install

• tell them what the command does

• tell them should be in the environment that they want this in

• tell them how to unsinstall

• tell them (and me!) what to do when someone has made changes in the repo and you want to make use of the changes

• • git pull or checkout of whatever

• do I uninstall first etc.

User story

As a user, I want to easily export the pydantic models into json, so I can use the json schemas.

Acceptance criteria

• There is a utility method that converts the pydantic models to json
• There is a main method that accepts an optional output dir argument. The main method will write the pydantic models as json files in the output directory (default to current working directory)

Sprint Ready Checklist

• 1. Acceptance criteria defined
• 2. Team understands acceptance criteria
• 3. Team has defined solution / steps to satisfy acceptance criteria
• 4. Acceptance criteria is verifiable / testable
• 5. External / 3rd Party dependencies identified
• 6. Ticket is prioritized and sized

Notes

Add any helpful notes here.

Update the definition of EphysSession coordinates to have *_ml *_ap *_dv in the name.

Acceptance Criteria:

• add a subclass that has the three named anatomical direction coordinate values in it
• do the same for FieldCoordinates3dMm
• update the tests
• patch version bump

Specimen is for keeping track of samples as they are created/subdivided by procedures.

Proposal:

• add a Specimen class to procedures.py
• and input_specimen and output_specimen to Procedure
• add a List[Specimen] to Procedures
• add specimen_id to EphysSession, Acquisition, and OphysSession.

Expand optical table dimension specification to tables with different width/lengths.

Change spec from optical_table_size to optical_table_length and optical_table_width.

Another option is to change optical_table_size to an array.

the JSON example is not rendering properly.

User story

As a user, I want a standardized name for my json file instances that validate against a schema, so I can ensure it is consistent across all projects.

Acceptance criteria

• Attach default json file name attribute to the Pydantic schema models
• (Stretch) Add a json writer method to the BaseModel class that a use can do something like:

p = Processing(...)
p.write("directory to write to", "optional file name if default not desired")

Sprint Ready Checklist

1. • Acceptance criteria defined
2. • Team understands acceptance criteria
3. • Team has defined solution / steps to satisfy acceptance criteria
4. • Acceptance criteria is verifiable / testable
5. • External / 3rd Party dependencies identified
6. • Ticket is prioritized and sized

Notes

• Let's make this a private attribute, so it doesn't show up in the JSONSchema
• Add the writer method and private attribute to base.AindSchema, now that it exists.

Right now the EphysRig has an array of Devices, whereas Instrument has all of the devices at the top level. Let's be consistent here.

Create a single manufacturer list - perhaps for specific modalities, perhaps for all modalities?

Acceptance criteria:

• as part of a PR, run all examples/*.py scripts and verify they finish without any errors
• write this as a test (e.g. tests/test_examples.py)

Describe the bug
The python statement:
from aind_data_schema import Axis

Generates the following error

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_15328/372763372.py in <module>
----> 1 from aind_data_schema import Axis

ImportError: cannot import name 'Axis' from 'aind_data_schema' (c:\users\lydian\onedrive - personal\onedrive\work\aind-data-schema\src\aind_data_schema\__init__.py)

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
I am able to import Axis like I do Acquisition, Subject, LightCycle etc.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

For purchased mice specifically.

The examples should always run, and if they don't we should update them. This feels like a test, so let's add a test that does this.

add animal weight, water consumed
start/end of head restraint?

Lightsource class currently specifies matching filter by #. But this field does not also map over into the Filter class. So the filter position # cannot be linked the the actual filter specs.

Suggest added the same filter_wheel_index into the Filter class.

class Filter(Device):
    """Description of a filter device"""

    type: FilterType = Field(..., title="Filter Type")
    diameter: float = Field(..., title="Size (mm)", ge=0)
    thickness: float = Field(..., title="Size (mm)", ge=0)
    filter_wheel_index: int = Field(..., title="Filter index")
    description: Optional[str] = Field(
        None, description="Where/how filter is being used", title="Description"
    )

Alternatives. Does it make sense to duplicate this same parameter into both classes? Or perhaps move the filter_wheel_index out of the Lightsource class and only into the Filter class.

Describe the bug
Docstrings in scripts (such as TODOs, Headframes) are being listed as description properties in the schemas.

Reproduce the problem

1. print(Procedures.schema_json(indent=3)) and see property Procedure underneath Headframe
2. print(Subject.schema_json(indent=3)) and see properties' (ex: Sex, Background Strain) have description: "TODO"

Screenshots
If applicable, add screenshots to help explain your problem.

Right now ephys.ephys_session.Laser and imaging.instrument.Laser have the same name. These should probably have different names, and maybe share a common base class.

Looks like it fits the mold, would just need to override the type field instead of using name.

This looks messy on pypi. Let's clean up so that it looks less bad.

The use of the term specimen is a holdover from a different system. Right now all use of the term specimen should actually be subject. specimen should be reserved for describing tissue samples.

This is blocked until all of the PRs adding schemas are merged.