broadinstitute / imaging-platform-pipelines Goto Github PK

View Code? Open in Web Editor NEW

13.0 12.0 13.0 1.58 MB

Cell Painting and other pipelines from the Imaging Platform

carpenter-lab

imaging-platform-pipelines's Introduction

Broad Imaging Platform pipelines

Cell Painting and other pipelines from the Imaging Platform.

Guidelines for creating pipelines

Note:

Each pipeline has two versions – one with CreateBatchFiles module enabled (named e.g. analysis.cppipe), and the other with the moduled disabled (named e.g. analysis_without_batchfile.cppipe). When updating the pipeline, ensure that these two versions are always in sync. The former is required by a script for creating CellProfiler batch files, which in turn are used to create a list of groups of images to be processed together, as well as Distributed-CellProfiler configuration files. The latter is used to run the pipeline using Distributed-CellProfiler.
The ExportToSpreadsheet module should always be configured with these values in order that the files are saved in a standard format and location

    Select the column delimiter:Comma (",")
    Add image metadata columns to your object data file?:No
    Limit output to a size that is allowed in Excel?:No
    Select the measurements to export:No
    Output file location:Default Output Folder\x7C
    Select source of sample row name:Metadata
    Select the image to use as the identifier:None
    Select the metadata to use as the identifier:None
    Export all measurement types?:Yes
    :
    Representation of Nan/Inf:NaN
    Add a prefix to file names?:No
    Filename prefix:
    Overwrite existing files without warning?:Yes
    Data to export:Do not use
    Combine these object measurements with those of the previous object?:No
    File name:
    Use the object name for the file name?:Yes

The SaveImages module should always be configured with these values in order that the files are saved in a standard format and location. The values shown for Select image name for file prefix and Enter single file name are specific for this example and should be adapted appropriately.

    Select image name for file prefix:OrigDNA
    Enter single file name:\\\\g<Well>_s\\\\g<Site>--nuclei_outlines
    Output file location:Default Output Folder sub-folder\x7Coutlines
    Overwrite existing files without warning?:Yes

imaging-platform-pipelines's People

Contributors

Stargazers

Watchers

Forkers

histopathology kirby-mp ksulli01 alicezhang09 sofialotfi dlogan pamzgt gzzwjhzxh wangruobing2 kangsan422 mtegtmey gabcomolet relliott2023

imaging-platform-pipelines's Issues

Run JUMP_production

Dear all,

This job is quite helpful. However, I am reaching out for assistance with an issue I encountered while using the JUMP_analysis_v3.cppipe pipeline from the Broad Institute's imaging-platform-pipelines on GitHub (link: JUMP_production).

While processing our own images with this pipeline, I encountered an error at module number 13 (MeasureGranularity). Unfortunately, due to this error, the expected features related to Granularity are missing from the results.

I would greatly appreciate any guidance or support you could provide to help resolve this issue. Thank you very much for your time and assistance.

Looking forward to your response.

Best regards,

Update CP3 pipeline to avoid CreateBatchFile related issues

See broadinstitute/cmQTL#14 (comment)

Create new pipelines that include brightfield channel

Create pipelines for
cellpainting_a549_20x_with_bf_phenix_bin1 by adding brightfield to pipelines in
cellpainting_a549_20x_phenix_bin1

Create pipelines for
cellpainting_a549_20x_with_bf_with_qcflag_phenix_bin1 by adding FlagImage (appropriately configured to use CPA output) to cellpainting_a549_20x_with_bf_phenix_bin1

Update cellpainting_a549_20x_imagexpress

Changes were made to cellpainting_a549_20x_phenix_bin1 related to the output folders in the ExportToSpreadsheet and SaveImage modules that haven't been reflected in cellpainting_a549_20x_imagexpress

missing Top Line Per-Object csv

Hi,
JUMP_QC_Drag-and-Drop_v1.cppipe is not producing Top Line Per-Object.csv but only
and files without Top Line in the file name, which makes it impossible to run on the fly Knime QC.
Any advice is highly appreciated!
/Nikolai

Create bash script to automatically generate `without_batchfile` versions

Generating load_data.csv and load_data_with_illum.csv for JUMP_production

The entrypoint to the pipelines starting from raw images appears to be JUMP_illum_LoadData_v1.cpproj, which generates illumination correction fn files, and then JUMP_analysis_v3.cppipe for the feature generation, which takes the images and ICF npy files as input.

Both these pipelines require a csv file containing filepaths and metadata. Is the expected format and metadata for this file (columns) documented anywhere, and are there any vendor specific scripts to automatically generate them from default exports?

The youtube Headless CellProfiler/ DistributedCellProfiler Tutorial references loading the images into CellProfiler by drag-n-drop and then exporting the file, but this is not convenient for distributed / cloud use case.