In response to Yuri's comment on the gitlab MR 78

@corilo Move this issue to whichever issue queue you deem appropriate.

Once you have defined the necessary metadata / configuration data - there would be nmdc-schema issues to align those needs with changes to the metabolomics metadata specified in the schema

For metaMS:

After following standard install procedure, when I run:

metaMS dump_json_template MetamsFile.json

Error:

Traceback (most recent call last):
  File "/home/mbabinski/miniconda3/bin/metaMS", line 5, in <module>
    from metaMS.cli import cli
ModuleNotFoundError: No module named 'metaMS'

For miniwdl:

After following installation, when I run:

miniwdl run wdl/metaMS.wdl -i wdl/metams_input.json --verbose --no-cache --copy-input-files:

Error:

task runMetaMS (wdl/metaMS.wdl Ln 5 Col 1) failed :: dir: "/panfs/biopan04/scratch-mbabinski/metaMS/20210624_134633_gcmsMetabolomics/call-runMetaMS", error: "DockerException", message: "Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))"

With docker image there was no issue.

Thanks!
Michal

Metabolomics pipeline prototype work to be documented here per Yuri at sprint planning.

Current import/export functionality for LCMSBase class (in lcmsrefactor branch of CoreMS) includes import/export of parameters via json files. Expand this functionality and adjust defaults to use toml files instead.

New database interface has a better way of dealing with this and we should remove this parameter setting from the actual LCMS object.

Working subtasks:

• Create lipidomics workflow diagram ( eg https://github.com/microbiomedata/metaPro/blob/master/docs/detailed_workflow_diagram.png, but more simple like this one: https://nmdc-workflow-documentation.readthedocs.io/en/latest/chapters/8_Metabolomics_index.html#workflow-diagram )
• Create lipidomics page in NMDC workflows documentation site (- https://nmdc-workflow-documentation.readthedocs.io/)
• Add lipidomics workflow information to MetaMS docs (https://emsl-computing.github.io/MetaMS/)
• Add to CoreMS documentation re: LCMS data structure and processing
• Create detailed explainer for workflow tools/processes (eg https://github.com/microbiomedata/metaPro/blob/master/docs/index.rst)

Will change in lcmsrefactor branch.

Current peak picking parameters for LCMSBase class for minimum height and persistence are set as absolute values. In order to better parameterize these settings and follow conventions in CoreMS, we should change this to use a parameter like relative abundance for setting minimums.

As currently implemented, that is how we set our values, so functionally it will not change, just the parameter names will change.

• #42
• #43

LC-MS metabolomics WDL development

• Based on the CLI command for LC-MS data dependent data process, create a WDL workflow declaration and configuration json input file.

The workflow should be fully functional, able to process test datasets, and generate molecular table data products with putative metabolites annotation using mini wdl or cromwell based on the metaMS

• Add MassSpectra and LCMSBase classes to lcms_dev on CoreMS
• Modify mass_spectra/input modules to lcms_dev on CoreMS
• #37
• #38
• #46
• #40
• #58
• #39
• #51
• #41
• #52
• #53
• #54
• Release new version of CoreMS

This is mostly complete on the lcmsrefactor branch.

Below are minor issues that will be addressed on lcmsrefactor branch before merging onto the lcms_dev branch

• #45
• #44
• #47
• #48
• #49
• #50

create a large test data set with expected outcome if workflow run is successful

link test results to this issue

create a small test data set with expected outcome if workflow run is successful

link test results to this issue

• Identify important sample extraction details to capture / variations from cited protocols that are important (August)
• Generate sample prep metadata template (August)
• Complete example for MPLEx (September)
• Identify missing information/ other needed details (Contact other EMSL staff) (October)

See https://code.emsl.pnl.gov/mass-spectrometry/corems/-/merge_requests/78 @corilo

Develop LC-MS metabolomics CLI

• Finish LC-MS Data Dependent script, remove dependency of excel internal library, and release new CoreMS version
• Add LC-MS Data Dependent command to existing metaMS cli based on corems script: CoreMS
• Release new metaMS python package with new LC-MS data dependent workflow command
• Release new metaMS docker image with new LC-MS data dependent workflow command

CLI should be fully functional, able to process test datasets, and generate molecular table data products with putative metabolites annotation. New metaMS python package release, and new docker image published.

@smcolby has added corems.molecular_id.search.database_interfaces module to branch of lcmsrefactor (feature branch is metabrefapi.

Below is example code to work with GCMS workflow:

from corems.molecular_id.search.database_interfaces import MetabRefDatabaseInterface

# Initialize
metabref = MetabRefDatabaseInterface()

# Set token
# This isn't tied to the class, so as long as it's been set
# at some point and part of your environment, good to go.
metabref.set_token("metabref.token")

# Get some stuff
db = metabref.get_gcms_library(format='json')

The existing lipidomics workflow should access metabref using as much overlapping functionality as possible.

• #18
• #19
• #20
• #21
• #22

Following completion of #33, #34, #35, ensure GC-MS workflow is updated accordingly.

See https://code.emsl.pnl.gov/mass-spectrometry/corems/-/merge_requests/96; this should be merged in after https://code.emsl.pnl.gov/mass-spectrometry/corems/-/merge_requests/91

Establish LC-MS workflow for metabolomics data dependent analysis:

• Development of targeted data processing script
• Implementation of CoreMS objs (50%)
• WDL and metadata products

This is next in line for mergining into lcm_dev after #38 .

Metabolomics workflow is working at PNNL but need to figure out DB infrastructure that would allow it to be distributed outside of PNNL.

In response to Yuri's review on #37, I will use properties on LCMSMassFeature attributes that should be considered private.

Please add a directory named "doc" and put documentation there following the example format in "https://github.com/microbiomedata/mg_annotation/tree/master/docs". Thanks.