workflow4metabolomics / isa2w4m Goto Github PK

Makefile 52.72% Python 2.48% HTML 14.40% Shell 30.40%

isa2w4m's Introduction

Our project

The Workflow4Metabolomics, W4M in short, is a French infrastructure offering software tool processing, analyzing and annotating metabolomics data. It is based on the Galaxy platform.

In the context of collaboration between metabolomics (MetaboHUB French infrastructure) and bioinformatics platforms (IFB: Institut Français de Bioinformatique), we have developed full LC/MS, GC/MS and NMR pipelines using Galaxy framework for data analysis including preprocessing, normalization, quality control, statistical analysis and annotation steps. Those modular and extensible workflows are composed with existing components (XCMS and CAMERA packages, etc.) but also a whole suite of complementary homemade tools. This implementation is accessible through a web interface, which guarantees the parameters completeness. The advanced features of Galaxy have made possible the integration of components from different sources and of different types. Thus, an extensible Virtual Research Environment (VRE) is offered to metabolomics communities (platforms, end users, etc.), and enables preconfigured workflows sharing for new users, but also experts in the field.

Citation

Giacomoni F., Le Corguillé G., Monsoor M., Landi M., Pericard P., Pétéra M., Duperier C., Tremblay-Franco M., Martin J.-F., Jacob D., Goulitquer S., Thévenot E.A. and Caron C. (2014). Workflow4Metabolomics: A collaborative research infrastructure for computational metabolomics. Bioinformatics, http://dx.doi.org/10.1093/bioinformatics/btu813

Galaxy

Galaxy is an open, web-based platform for data intensive biomedical research. Whether on the free public server or your own instance, you can perform, reproduce, and share complete analyses.

Homepage: https://galaxyproject.org/

How to contribute

Get our tools

All our tools are publicly available in GitHub and freely installable through the Galaxy ToolShed

However, we will be glad to have [good] feedbacks on their usage in order to motivate us (and our funders).

It will also be great if you can cite our papers:

Franck Giacomoni, Gildas Le Corguillé, Misharl Monsoor, Marion Landi, Pierre Pericard, Mélanie Pétéra, Christophe Duperier, Marie Tremblay-Franco, Jean-François Martin, Daniel Jacob, Sophie Goulitquer, Etienne A. Thévenot and Christophe Caron (2014). Workflow4Metabolomics: A collaborative research infrastructure for computational metabolomics. Bioinformatics

doi:10.1093/bioinformatics/btu813

Push your tools / W4M as a Showcase

Your tools can be installed, integrated and hosted within the main W4M instance Tools.

Quality standards

However, the tools must stick to the IUC standards in order to be easily integrated:

Available in a GitHub repository
Conda dependencies
Functional tests using Planemo
Available in the Main ToolShed

In the first place, your tools will be displayed in the Contribution section of the tool panel. And eventually, it should be promoted among the other tools.

Advanced mode

In order to be fully integrated in our reference workflows, your tools must follow your exchange formats between tools (for more information, contact us).

A collaboration should be established if help is needed!

Support / HelpDesk

In all cases, the tools must be maintained by the developers themselves. A tool can be removed if this after sales service isn't done.

Guidelines

Writing a tool

isa2w4m's People

Contributors

Stargazers

Watchers

Forkers

sneumann

isa2w4m's Issues

Use isa-tab instead of isa as input type

Due to splitting of Galaxy ISA datatype into two types ISA-Tab and ISA-JSON, we need to change input from isa to isa-tab.

Reorder output datasets

data matrix
sample
variable

Move this tool into tools-metabolomics

Explore the possibility of moving this tool into tools-metabolomics

Make a bioconda package for the script.
Move XML and test-data into tools-metabolomics.

Error when converting MTBLS1 into Galaxy

Traceback (most recent call last):
  File "/Users/pierrick/Library/Python/3.6/lib/python/site-packages/pandas/core/indexes/base.py", line 2442, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5280)
  File "pandas/_libs/index.pyx", line 154, in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5126)
  File "pandas/_libs/hashtable_class_helper.pxi", line 1210, in pandas._libs.hashtable.PyObjectHashTable.get_item (pandas/_libs/hashtable.c:20523)
  File "pandas/_libs/hashtable_class_helper.pxi", line 1218, in pandas._libs.hashtable.PyObjectHashTable.get_item (pandas/_libs/hashtable.c:20477)
KeyError: 'mass_to_charge'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/pierrick/dev/galaxy-isa/tools/isa2w4m/isa2w4m.py", line 9, in <module>
    isatab2w4m.main()
  File "/Users/pierrick/dev/isa-api/isatools/convert/isatab2w4m.py", line 567, in main
    all_assays=args_dict['all_assays'])
  File "/Users/pierrick/dev/isa-api/isatools/convert/isatab2w4m.py", line 453, in convert2w4m
    variable_names = make_variable_names(measures_df)
  File "/Users/pierrick/dev/isa-api/isatools/convert/isatab2w4m.py", line 294, in make_variable_names
    for i, v in enumerate(assay_df[col].values):
  File "/Users/pierrick/Library/Python/3.6/lib/python/site-packages/pandas/core/frame.py", line 1964, in __getitem__
    return self._getitem_column(key)
  File "/Users/pierrick/Library/Python/3.6/lib/python/site-packages/pandas/core/frame.py", line 1971, in _getitem_column
    return self._get_item_cache(key)
  File "/Users/pierrick/Library/Python/3.6/lib/python/site-packages/pandas/core/generic.py", line 1645, in _get_item_cache
    values = self._data.get(item)
  File "/Users/pierrick/Library/Python/3.6/lib/python/site-packages/pandas/core/internals.py", line 3590, in get
    loc = self.items.get_loc(item)
  File "/Users/pierrick/Library/Python/3.6/lib/python/site-packages/pandas/core/indexes/base.py", line 2444, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5280)
  File "pandas/_libs/index.pyx", line 154, in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5126)
  File "pandas/_libs/hashtable_class_helper.pxi", line 1210, in pandas._libs.hashtable.PyObjectHashTable.get_item (pandas/_libs/hashtable.c:20523)
  File "pandas/_libs/hashtable_class_helper.pxi", line 1218, in pandas._libs.hashtable.PyObjectHashTable.get_item (pandas/_libs/hashtable.c:20477)
KeyError: 'mass_to_charge'

NA filtering

NA filtering does not to work.

Select study by study ID

Currently the study can be selected by filename only. Adds the possibility to submit to the study ID/name instead.

Failure when filtering on a column with only NA values

Trying to remove rows from variable metadata output, where an NA value is found in a certain column that contains only NA values fails:

Traceback (most recent call last):
  File "/home/pierrick/dev/galaxy/lib/galaxy/jobs/runners/local.py", line 130, in queue_job
    job_wrapper.finish( stdout, stderr, exit_code )
  File "/home/pierrick/dev/galaxy/lib/galaxy/jobs/__init__.py", line 1386, in finish
    'primary': self.tool.collect_primary_datasets(out_data, tool_working_directory, input_ext, input_dbkey)
  File "/home/pierrick/dev/galaxy/lib/galaxy/tools/__init__.py", line 1613, in collect_primary_datasets
    return output_collect.collect_primary_datasets( self, output, job_working_directory, input_ext, input_dbkey=input_dbkey )
  File "/home/pierrick/dev/galaxy/lib/galaxy/tools/parameters/output_collect.py", line 325, in collect_primary_datasets
    primary_data.set_meta()
  File "/home/pierrick/dev/galaxy/lib/galaxy/model/__init__.py", line 2038, in set_meta
    return self.datatype.set_meta( self, **kwd )
  File "/home/pierrick/dev/galaxy/lib/galaxy/datatypes/tabular.py", line 978, in set_meta
    data_row = next(reader)
StopIteration

Tried with MTBLS404 study and value database inside field "Variable metadata columns NA filtering".

Convert also ISA-JSON

Upgrade conversion so it can also handle ISA-JSON archives.
The work must be done inside isatools library.

Make planemo test work in Travis CI

W4M integration

Use conda for requirements.
Test with planemo.
Publish to testtoolshed.
Ask for installation on dev instance.

Test NA filtering on multiple columns

Rename output dataset

Set output names to:

"${isa.name} W4M var"
"${isa.name} W4M samp"
"${isa.name} W4M data"

Extract raw data

Hi, after a study is downloaded, I can get the three tables into W4M format. But where is the raw data ?
It would be great if the tool created a collection of raw data files that could then be passed
to xcms or alike. Or did I miss that somewhere ? Yours, Steffen