cellprofiler / cellprofiler Goto Github PK

asdf

Attachments:
http://cellprofiler.org/issues/uploaded//web/wwwprod/sites/cellprofiler.org/cgi-bin/../issues/uploaded/plot_platemaps_2.png
http://cellprofiler.org/issues/uploaded//web/wwwprod/sites/cellprofiler.org/cgi-bin/../issues/uploaded/plot_platemaps_1.png

It is very difficult to judge the performance of the identify modules without being able to turn on/off the outlines in the figure panel where outlines are overlaid on the input image. Adding a toggle functionality would make users happier.

(OverlayOutlines is another module that would benefit from toggling; there might be others)

Original reporter: anne
Jira link: https://jira.broadinstitute.org/browse/IMG-1551
Assignee: leek

Error loading this pipeline with SVN 11602, at the RunImageJ module:
https://imageweb.broadinstitute.org/imaging/analysis/2007_05_14_Mitochondria_AnneCataldo/McPhie/Pipelines/PIPE_2011_10_14_1st_AND_2nd.cp

This will load on my Windows VM. Tubeness_2.0.jar is installed in the respective ImageJ preferences directories.

Here's the stacktrace:
***********
INFO:PipelineStatistics:Pipeline saved with CellProfiler SVN revision 11488
Pipeline saved with CellProfiler SVN revision 11488
ERROR:root:Failed to load pipeline
Traceback (most recent call last):
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/pipeline.py", line 760, in loadtxt
    module_name, from_matlab)
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/cpmodule.py", line 191, in set_settings_from_values
    v.value = value
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/settings.py", line 119, in __internal_set_value
    self.set_value(value)
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/settings.py", line 199, in set_value
    raise NotImplementedError("The count should be inferred, not set - actual: %d, set: %d"%(len(self.__sequence), count))
NotImplementedError: The count should be inferred, not set - actual: 0, set: 4
Failed to load pipeline
Traceback (most recent call last):
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/pipeline.py", line 760, in loadtxt
    module_name, from_matlab)
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/cpmodule.py", line 191, in set_settings_from_values
    v.value = value
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/settings.py", line 119, in __internal_set_value
    self.set_value(value)
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/settings.py", line 199, in set_value
    raise NotImplementedError("The count should be inferred, not set - actual: %d, set: %d"%(len(self.__sequence), count))
NotImplementedError: The count should be inferred, not set - actual: 0, set: 4

Original reporter: dlogan
Jira link: https://jira.broadinstitute.org/browse/IMG-1555
Fix version: next
Assignee: leek

Comment from leek on Wed, 19 Oct 2011 13:57:17 -0400:
Unfortunately the interprocess bridge that was done for the Mac is incomplete and doesn't support the parameters. This should be resolved when we have ImageJ 2.0 support because we will be able to run ImageJ 2.0 headless on the Mac.

Blah blah

Attachments:
http://cellprofiler.org/issues/uploaded/plot_platemaps_2.png
http://cellprofiler.org/issues/uploaded/plot_platemaps_1.png

Original reporter: klmadden
Jira link: https://jira.broadinstitute.org/browse/IMG-1577
Assignee: klmadden

This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&t=1463

I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?

The forum poster is running headless, so I'm not sure what recourse he has.

However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
(1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
(2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
(3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
----------------------------------------------
Traceback (most recent calllast):
  File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
    m = cpm.Measurements(filename = output_path)
  File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
    self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
  File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
    self.hdf5_file = h5py.File(self.filename,'w')
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
    fid = make_fid(name, mode, fapl)
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
  File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
IOError: unable to create file (File accessability: Unable to open file)

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1531
Fix version: current
Assignee: leek

Comment from leek on Fri, 16 Sep 2011 11:49:21 -0400:
I answered the post. The files are needed to keep the measurements from
consuming memory. They should be deleted at the end of the run.

On 9/16/2011 11:18 AM, Mark-Anthony Bray (JIRA) wrote:
> HDF5 file creation weirdness: File misnaming + locking
> ------------------------------------------------------
>
> Key:<a href="https://jira.broadinstitute.org/browse/IMG-1531"title="HDF5 file creation weirdness: File misnaming + locking">IMG-1531
> URL:https://jira.broadinstitute.org/browse/IMG-1531
> Project: Imaging
> Issue Type: Bug
> Components: CellProfiler
> Affects Versions: Current
> Reporter: Mark-Anthony Bray
> Assignee: Lee Kamentsky
> Priority: Major
> Fix For: Current
>
>
> This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463
>
> I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?
>
> The forum poster is running headless, so I'm not sure what recourse he has.
>
> However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
> (1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
> (2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
> (3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
> ----------------------------------------------
> Traceback (most recent call last):
> File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
> m = cpm.Measurements(filename = output_path)
> File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
> self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
> File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
> self.hdf5_file = h5py.File(self.filename,'w')
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
> fid = make_fid(name, mode, fapl)
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
> fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
> File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
> IOError: unable to create file (File accessability: Unable to open file)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see:http://www.atlassian.com/software/jira

undefined

Attachments:
http://cellprofiler.org/issues/uploaded/plot_platemaps_2.png

_allow user to save a training set (in the middle of a pipeline or at the end) and go back to it later to add more training examples
_incorporate worm toolbox training into the CPA UI

Original reporter: klmadden
Jira link: https://jira.broadinstitute.org/browse/IMG-1562
Fix version: future
Assignee: leek

path is necessary for creating subdirs.

Original reporter: thouis
Jira link: https://jira.broadinstitute.org/browse/IMG-1546
Assignee: leek

On 10/15/2011 6:32 AM, Thouis (Ray) Jones wrote:
> Do we allow even-side-length structuring elements in CP? If so, we
> should look at our chained operations (e.g., open) to make sure they
> behave correctly.
>
>
> ---------- Forwarded message ----------
> From: Tony Yu<[email protected]>
> Date: Sat, Oct 15, 2011 at 10:58
> Subject: Pull request: morphology fix
> To:[email protected]
>
>
> I just submitted a pull request for a somewhat-subtle bug with some
> morphology operations:
>
>https://github.com/scikits-image/scikits.image/pull/62
>
> Overly detailed explanation:
>
> Using a structuring element with one-or-both sides being even-numbered
> gives buggy output for morphological open/close and white/black tophat
> operations. These issues arise because even-numbered structuring
> elements (selems) aren't centered on a pixel. The dilate and erode
> operations are (correctly) defined so that they are symmetric
> (dilation on a white feature is exactly the opposite of erosion on an
> equivalent black feature). When chaining them (e.g. open() =
> dilate(erode())), however, this symmetry ends up shifting features. To
> fix this, one of the operations should be shifted. For example, if
> erosion is done with the selem shifted slightly to the lower-right,
> the dilation operation should shift theselem to the upper-left. This
> change prevents features from being shifted.
>
> The open/close operations aren't that bad: they just shift features.
> But, when calling white/black tophat operations, the shifting can lead
> to over/underflow of pixels. I've attached examples of the lena image
> after calling white tophat, both before and after the fix. The
> underflow should be apparent in the first image.
>
> Best,
> -Tony
>
>
> Example code:
>
> import os
> import numpy as np
> import matplotlib.pyplot as plt
>
> from scikits.image import data_dir
> from scikits.image.morphology import *
>
> lena = np.load(os.path.join(data_dir,'lena_GRAY_U8.npy'))
>
> plt.imshow(greyscale_white_top_hat(lena, square(4)))
> plt.show()
> _______________________________________________
> cellprofiler-dev mailing list
>[email protected]
>https://imap.broadinstitute.org/mailman/listinfo/cellprofiler-dev

Original reporter: leek
Jira link: https://jira.broadinstitute.org/browse/IMG-1553
Fix version: next
Assignee: leek

Brought to attention by forum post: http://cellprofiler.org/forum/viewtopic.php?f=16&t=1484

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1537
Fix version: next
Assignee: leek

Adding a fresh RunImageJ module, then mousing over the "Command" setting produces an ugly-huge-long popup of all the possible commands. It would be nice to truncate this.

Original reporter: dlogan
Jira link: https://jira.broadinstitute.org/browse/IMG-1558
(Original JIRA has attachments.)
Fix version: next
Assignee: leek

Comment from dlogan on Thu, 20 Oct 2011 15:35:02 -0400:
Lee said ImageJ 2.0 fixes this. But on my Mac, after I switch my Preferences to ImageJ 2.0, restart CP, and then add a RunImageJ module, I immediately get this error:

CellProfiler .app has a bunch of things that shouldn't be there:

Original reporter: thouis
Jira link: https://jira.broadinstitute.org/browse/IMG-1347
Assignee: thouis

ExportToDatabase should write the experiment measurements that are populated during prepare_run, especially the pipeline as stored in a measurement.

Original reporter: leek
Jira link: https://jira.broadinstitute.org/browse/IMG-1579
Fix version: current
Assignee: leek

We save the reworked pipeline in the .H5 file as an experiment measurement so that it can be loaded by BatchProfiler. We should save the original pipeline as well and that should be the one that's opened when you load an .H5 file.

undefined

Attachments:
http://cellprofiler.org/issues/uploaded/plot_platemaps_2.png
http://cellprofiler.org/issues/uploaded/plot_platemaps_1.png

Investigate everything in the following email trail:


At line 719 (in trunk, 705 in release), there's this:"blurred_image =
self.smooth_image(img,mask,1)" located after thresholding but prior to
segmentation.

However, if filter size = 0, smooth_image just returns the original
image; omitting this line makes no difference.
     -Mark

On 11/9/2011 1:06 AM, Thouis (Ray) Jones wrote:
> There's a hidden smooth in ID primary. It might be the culprit.
>
> We should expose watershed in one of the image processing modules.
>
> Ray
>
>
> On Tue, Nov 8, 2011 at 23:42, Carolina Wahlby
><[email protected]> wrote:
>> Hi!
>> A mini-pipeline and the result from CellProfiler attached (running on
>> the image from my previous email). It is the large connected regions I
>> want to chop up based on the local maxima.
>> Many thanks for any input!
>> Cheers
>> Carolina
>>
>>
>> On Tue, Nov 8, 2011 at 5:32 PM, Anne Carpenter<[email protected]> wrote:
>>> Can you send your pipeline, too?
>>>
>>> (and can someone'claim' the project so we don't have six people
>>> working on it once? I know you all want to make Carolina happy but
>>> let's not be redundant)
>>>
>>>
>>>
>>>
>>> On Tue, Nov 8, 2011 at 5:23 PM, Carolina Wahlby
>>><[email protected]> wrote:
>>>> Hi all,
>>>> I'm preparing a CellProfiler workshop focused on counting signals from
>>>> PLA- and padlock probes in cells and tissue.
>>>> For these images it is often important to separate clusters of signals
>>>> so that a correct count is achieved.
>>>> In theory, watershed segmentation should be able to find as many
>>>> individual objects as there are local intensity peaks. These local
>>>> peaks can also be found using the imextendedmax.m function in Matlab,
>>>> see attached result (desired_regs.pdf) running on attached image
>>>> (blob_details.png). However, I do not manage to get as good separation
>>>> using IdentyfyPrimaryObjects; objects are merged despite turning of
>>>> smoothing etc. Does anyone know a way to trick IdentifyPrimaryObjects
>>>> to get this right? Or is there some other approach? Enhance speckles
>>>> does not do it.
>>>> Thanks
>>>> Carolina
>>>>
>>>> PS
>>>> Matlab code to get desired result:
>>>> I=imread('blob_details.png');
>>>> BW = imextendedmax(I,1);
>>>> figure;subplot(121);imshow(I,[]);hold on; subplot(122);imshow(BW,[]);

Original reporter: leek
Jira link: https://jira.broadinstitute.org/browse/IMG-1571
(Original JIRA has attachments.)
Fix version: current
Assignee: leek

The attached patches should be reviewed and applied to CP 1.0

Original reporter: leek
Jira link: https://jira.broadinstitute.org/browse/IMG-1542
(Original JIRA has attachments.)
Assignee: leek

This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&t=1463

I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?

The forum poster is running headless, so I'm not sure what recourse he has.

However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
(1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
(2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
(3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
----------------------------------------------
Traceback (most recent calllast):
  File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
    m = cpm.Measurements(filename = output_path)
  File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
    self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
  File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
    self.hdf5_file = h5py.File(self.filename,'w')
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
    fid = make_fid(name, mode, fapl)
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
  File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
IOError: unable to create file (File accessability: Unable to open file)

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1531
Fix version: current
Assignee: leek

Comment from leek on Fri, 16 Sep 2011 11:49:21 -0400:
I answered the post. The files are needed to keep the measurements from
consuming memory. They should be deleted at the end of the run.

On 9/16/2011 11:18 AM, Mark-Anthony Bray (JIRA) wrote:
> HDF5 file creation weirdness: File misnaming + locking
> ------------------------------------------------------
>
> Key:<a href="https://jira.broadinstitute.org/browse/IMG-1531"title="HDF5 file creation weirdness: File misnaming + locking">IMG-1531
> URL:https://jira.broadinstitute.org/browse/IMG-1531
> Project: Imaging
> Issue Type: Bug
> Components: CellProfiler
> Affects Versions: Current
> Reporter: Mark-Anthony Bray
> Assignee: Lee Kamentsky
> Priority: Major
> Fix For: Current
>
>
> This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463
>
> I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?
>
> The forum poster is running headless, so I'm not sure what recourse he has.
>
> However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
> (1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
> (2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
> (3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
> ----------------------------------------------
> Traceback (most recent call last):
> File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
> m = cpm.Measurements(filename = output_path)
> File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
> self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
> File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
> self.hdf5_file = h5py.File(self.filename,'w')
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
> fid = make_fid(name, mode, fapl)
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
> fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
> File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
> IOError: unable to create file (File accessability: Unable to open file)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see:http://www.atlassian.com/software/jira

The settings for measuregranularity should be improved. I suggest:
Texture scale
Background removal radius
Subsampling factor (how much to scale down the image for speed)

 The first two should be in the unscaled pixel units. The code should automatically translate them to the relevant sizes after subsampling. Warnings can be generated if the subsampling causes them to become<1. Possibly, there should be an output"setting" that shows the scaled values that will be used (since they must be integers).

Original reporter: thouis
Jira link: https://jira.broadinstitute.org/browse/IMG-1536
Fix version: Current
Assignee: leek

This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463

I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?

The forum poster is running headless, so I'm not sure what recourse he has.

However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
(1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
(2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
(3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
----------------------------------------------
Traceback (most recent calllast):
  File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
    m = cpm.Measurements(filename = output_path)
  File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
    self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
  File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
    self.hdf5_file = h5py.File(self.filename,'w')
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
    fid = make_fid(name, mode, fapl)
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
  File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
IOError: unable to create file (File accessability: Unable to open file)

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1531
Fix version: current
Assignee: leek

Comment from leek on Fri, 16 Sep 2011 11:49:21 -0400:
I answered the post. The files are needed to keep the measurements from
consuming memory. They should be deleted at the end of the run.

On 9/16/2011 11:18 AM, Mark-Anthony Bray (JIRA) wrote:
> HDF5 file creation weirdness: File misnaming + locking
> ------------------------------------------------------
>
> Key:<a href="https://jira.broadinstitute.org/browse/IMG-1531"title="HDF5 file creation weirdness: File misnaming + locking">IMG-1531
> URL:https://jira.broadinstitute.org/browse/IMG-1531
> Project: Imaging
> Issue Type: Bug
> Components: CellProfiler
> Affects Versions: Current
> Reporter: Mark-Anthony Bray
> Assignee: Lee Kamentsky
> Priority: Major
> Fix For: Current
>
>
> This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463
>
> I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?
>
> The forum poster is running headless, so I'm not sure what recourse he has.
>
> However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
> (1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
> (2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
> (3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
> ----------------------------------------------
> Traceback (most recent call last):
> File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
> m = cpm.Measurements(filename = output_path)
> File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
> self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
> File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
> self.hdf5_file = h5py.File(self.filename,'w')
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
> fid = make_fid(name, mode, fapl)
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
> fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
> File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
> IOError: unable to create file (File accessability: Unable to open file)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see:http://www.atlassian.com/software/jira

undefined

Attachments:
{ "url": "http://cellprofiler.org/issues/uploaded/plot_platemaps_2.png" }

{ "url": "http://cellprofiler.org/issues/uploaded/plot_platemaps_1.png" }

This error often appears in test mode with pipelines that have a Crop module, but not in the analysis run:
--------------------------------------------------------
Traceback (most recent call last):
  File"cellprofiler\gui\pipelinecontroller.pyc", line 932, in do_step
  File"cellprofiler\modules\crop.pyc", line 294, in run
KeyError:'FirstImageSet'

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1569
Fix version: current
Assignee: leek

Use case:

- Threshold image A
- Threshold Image B
- Multiply threshA and threshB
- MeasureImageAreaOccupied on AtimesB
Result: error because product is not binary, according to MeasureImageAreaOccupied (which only tests dtype == bool).

Not sure the best approach, here:
- Add operations for binary images (AND, OR, XOR)
- Make the Multiply option DTRT if handed two binary images
- Add a"convert to binary" option in ImageMath
- Make cpimage check for (dtype == bool OR all values in {0, 1})

Personally, I like the last, I think.

Original reporter: thouis
Jira link: https://jira.broadinstitute.org/browse/IMG-1567
Assignee: leek

Adding a fresh RunImageJ module, then mousing over the "Command" setting produces an ugly-huge-long popup of all the possible commands. It would be nice to truncate this.

Original reporter: dlogan
Jira link: https://jira.broadinstitute.org/browse/IMG-1558
(Original JIRA has attachments.)
Fix version: next
Assignee: leek

This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463

I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?

The forum poster is running headless, so I'm not sure what recourse he has.

However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
(1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
(2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
(3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
----------------------------------------------
Traceback (most recent calllast):
  File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
    m = cpm.Measurements(filename = output_path)
  File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
    self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
  File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
    self.hdf5_file = h5py.File(self.filename,'w')
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
    fid = make_fid(name, mode, fapl)
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
  File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
IOError: unable to create file (File accessability: Unable to open file)

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1531
Fix version: current
Assignee: leek

Comment from leek on Fri, 16 Sep 2011 11:49:21 -0400:
I answered the post. The files are needed to keep the measurements from
consuming memory. They should be deleted at the end of the run.

On 9/16/2011 11:18 AM, Mark-Anthony Bray (JIRA) wrote:
> HDF5 file creation weirdness: File misnaming + locking
> ------------------------------------------------------
>
> Key:<a href="https://jira.broadinstitute.org/browse/IMG-1531"title="HDF5 file creation weirdness: File misnaming + locking">IMG-1531
> URL:https://jira.broadinstitute.org/browse/IMG-1531
> Project: Imaging
> Issue Type: Bug
> Components: CellProfiler
> Affects Versions: Current
> Reporter: Mark-Anthony Bray
> Assignee: Lee Kamentsky
> Priority: Major
> Fix For: Current
>
>
> This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463
>
> I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?
>
> The forum poster is running headless, so I'm not sure what recourse he has.
>
> However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
> (1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
> (2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
> (3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
> ----------------------------------------------
> Traceback (most recent call last):
> File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
> m = cpm.Measurements(filename = output_path)
> File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
> self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
> File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
> self.hdf5_file = h5py.File(self.filename,'w')
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
> fid = make_fid(name, mode, fapl)
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
> fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
> File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
> IOError: unable to create file (File accessability: Unable to open file)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see:http://www.atlassian.com/software/jira

Adding a fresh RunImageJ module, then mousing over the "Command" setting produces an ugly-huge-long popup of all the possible commands. It would be nice to truncate this.

Original reporter: dlogan
Jira link: https://jira.broadinstitute.org/browse/IMG-1558
(Original JIRA has attachments.)
Fix version: next
Assignee: leek

Comment from dlogan on Thu, 20 Oct 2011 15:35:02 -0400:
Lee said ImageJ 2.0 fixes this. But on my Mac, after I switch my Preferences to ImageJ 2.0, restart CP, and then add a RunImageJ module, I immediately get this error:

Traceback (most recent call last):

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/gui/moduleview.py", line 1609, in on_idle

    self.validate_module()

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/gui/moduleview.py", line 1620, in validate_module

    self.__module.test_valid(self.__pipeline)

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/cpmodule.py", line 311, in test_valid

    self.visible_settings()[0])

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/modules/run_imagej.py", line 488, in visible_settings

    self.command_settings_dictionary)

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/modules/run_imagej.py", line 326, in get_command_settings

    module_info = command.get_selected_leaf()[2]

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/settings.py", line 1546, in get_selected_leaf

    tree = self.get_tree()

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/settings.py", line 1562, in get_tree

    return self.__tree()

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/modules/run_imagej.py", line 284, in get_choice_tree

    ij2_module_service = IJ2.get_module_service(context)

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/imagej/imagej2.py", line 149, in get_module_service

    o = context.getService('imagej.ext.module.ModuleService')

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/imagej/imagej2.py", line 139, in getService

    '(Ljava/lang/Class;)Limagej/IService;', klass)

  File "/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/utilities/jutil.py", line 421, in call

    assert o is not None

AssertionError

Original reporter: thouis
Jira link: https://jira.broadinstitute.org/browse/IMG-1547
Assignee: leek

CalculateImageOverlap outputs"high-level" stats, however, naively(?) I expected it to output the number of pixels (and/or percent) co-occurring in the two binary images. Please make it so that the user isn't required to reverse-engineer this simple pixel count from the Recall and Precision.

Original reporter: dlogan
Jira link: https://jira.broadinstitute.org/browse/IMG-1560
Fix version: next
Assignee: dlogan

Would be great to:
(1) Store pipeline in database
(2) From ViewBatch in BatchProfiler, be able to click on a link and load (or at least download?) a pipeline
** Alternatively, be able to load a pipeline stored in the database directly from CP

Original reporter: dlogan
Jira link: https://jira.broadinstitute.org/browse/IMG-1578
Fix version: current
Assignee: leek

asdf

Attachments:
http://cellprofiler.org/issues/uploaded/attachmentlXdBuC/plot_platemaps_2.png
http://cellprofiler.org/issues/uploaded/attachmentr9w0Ut/plot_platemaps_1.png

As a new user to the test mode feature (which is AWESOME, by the way), it took me about ten runs before I realized what visual cues were available to tell me when the processing was complete (or, more precisely, paused). When making subtle changes to settings and watching for the output to update in the module's figure window, I could not always tell if processing had completed yet or not. The spinning beach ball was one cue since I guess my computer is slow, but eventually I realized that the Run button stays blue whilst processing and is notblue when it completes/pauses (at least, I think that's what happens). A simple minor improvement might be to change the text on the button to be"Running" during processing, and revert back to"Run" when complete.

Original reporter: anne
Jira link: https://jira.broadinstitute.org/browse/IMG-1552
Assignee: leek

See forum post:http://cellprofiler.org/forum/viewtopic.php?f=14&t=1548

All the Display* data tools show only the 1st cycles' worth of measurements from the output file as opposed to the full run's worth, as the help indicates.

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1574
Fix version: next
Assignee: leek

Blah blah

Blah

Attachments:
http://cellprofiler.org/issues/uploaded/plot_platemaps_2.png
http://cellprofiler.org/issues/uploaded/plot_platemaps_1.png

This issue was noticed when looking at this forum post: http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463

I have noticed the appearance of these files as well, even if I specify "Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if "Do not write..." is selected?

The forum poster is running headless, so I'm not sure what recourse he has.

However, if I specify "HDF5" as the measurement format, and allow overwriting, I notice the following:

(1) The output filename will be shown as "DefaultOut.h5" rather than the usual "DefaultOUT.h5" (note the case change)

(2) Running the pipeline with the output filename in (1) will create a "DefaultOUT" file (no extension but it's an HDF5 file)

(3) Changing the output filename to "DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:

Traceback (most recent call last):

  File "C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images

    m = cpm.Measurements(filename = output_path)

  File "C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init

    self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)

  File "C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init

    self.hdf5_file = h5py.File(self.filename, 'w')

  File "C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init

    fid = make_fid(name, mode, fapl)

  File "C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid

    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)

  File "h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)

IOError: unable to create file (File accessability: Unable to open file)

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1531
Fix version: current
Assignee: leek

Comment from leek on Fri, 16 Sep 2011 11:49:21 -0400:
I answered the post. The files are needed to keep the measurements from

consuming memory. They should be deleted at the end of the run.

On 9/16/2011 11:18 AM, Mark-Anthony Bray (JIRA) wrote:

> HDF5 file creation weirdness: File misnaming + locking

> ------------------------------------------------------

>

> Key: IMG-1531

> URL: https://jira.broadinstitute.org/browse/IMG-1531

> Project: Imaging

> Issue Type: Bug

> Components: CellProfiler

> Affects Versions: Current

> Reporter: Mark-Anthony Bray

> Assignee: Lee Kamentsky

> Priority: Major

> Fix For: Current

>

>

> This issue was noticed when looking at this forum post: http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463

>

> I have noticed the appearance of these files as well, even if I specify "Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if "Do not write..." is selected?

>

> The forum poster is running headless, so I'm not sure what recourse he has.

>

> However, if I specify "HDF5" as the measurement format, and allow overwriting, I notice the following:

> (1) The output filename will be shown as "DefaultOut.h5" rather than the usual "DefaultOUT.h5" (note the case change)

> (2) Running the pipeline with the output filename in (1) will create a "DefaultOUT" file (no extension but it's an HDF5 file)

> (3) Changing the output filename to "DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:

> ----------------------------------------------

> Traceback (most recent call last):

> File "C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images

> m = cpm.Measurements(filename = output_path)

> File "C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init

> self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)

> File "C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init

> self.hdf5_file = h5py.File(self.filename, 'w')

> File "C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init

> fid = make_fid(name, mode, fapl)

> File "C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid

> fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)

> File "h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)

> IOError: unable to create file (File accessability: Unable to open file)

>

> --

> This message is automatically generated by JIRA.

> For more information on JIRA, see: http://www.atlassian.com/software/jira

Several new measurements have been added (e.g. URL, image width and height to Image, Feret diameter to shape). These are breaking the nightly build for trivial reasons. The errors need to be reviewed and, if truly trivial, the reference data set needs to be updated.

Original reporter: leek
Jira link: https://jira.broadinstitute.org/browse/IMG-1533
Fix version: current
Assignee: leek

This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463

I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?

The forum poster is running headless, so I'm not sure what recourse he has.

However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
(1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
(2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
(3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
----------------------------------------------
Traceback (most recent calllast):
  File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
    m = cpm.Measurements(filename = output_path)
  File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
    self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
  File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
    self.hdf5_file = h5py.File(self.filename,'w')
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
    fid = make_fid(name, mode, fapl)
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
  File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
IOError: unable to create file (File accessability: Unable to open file)

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1531
Fix version: current
Assignee: leek

Comment from leek on Fri, 16 Sep 2011 11:49:21 -0400:
I answered the post. The files are needed to keep the measurements from
consuming memory. They should be deleted at the end of the run.

On 9/16/2011 11:18 AM, Mark-Anthony Bray (JIRA) wrote:
> HDF5 file creation weirdness: File misnaming + locking
> ------------------------------------------------------
>
> Key:<a href="https://jira.broadinstitute.org/browse/IMG-1531"title="HDF5 file creation weirdness: File misnaming + locking">IMG-1531
> URL:https://jira.broadinstitute.org/browse/IMG-1531
> Project: Imaging
> Issue Type: Bug
> Components: CellProfiler
> Affects Versions: Current
> Reporter: Mark-Anthony Bray
> Assignee: Lee Kamentsky
> Priority: Major
> Fix For: Current
>
>
> This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463
>
> I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?
>
> The forum poster is running headless, so I'm not sure what recourse he has.
>
> However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
> (1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
> (2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
> (3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
> ----------------------------------------------
> Traceback (most recent call last):
> File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
> m = cpm.Measurements(filename = output_path)
> File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
> self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
> File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
> self.hdf5_file = h5py.File(self.filename,'w')
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
> fid = make_fid(name, mode, fapl)
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
> fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
> File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
> IOError: unable to create file (File accessability: Unable to open file)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see:http://www.atlassian.com/software/jira

Adding a fresh RunImageJ module, then mousing over the"Command" setting produces an ugly-huge-long popup of all the possible commands. It would be nice to truncate this.

Original reporter: dlogan
Jira link: https://jira.broadinstitute.org/browse/IMG-1558
(Original JIRA has attachments.)
Fix version: next
Assignee: leek

Comment from dlogan on Thu, 20 Oct 2011 15:35:02 -0400:
Lee said ImageJ 2.0 fixes this. But on my Mac, after I switch my Preferences to ImageJ 2.0, restart CP, and then add a RunImageJ module, I immediately get this error:

Traceback (most recent call last):
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/gui/moduleview.py", line 1609, in on_idle
    self.validate_module()
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/gui/moduleview.py", line 1620, in validate_module
    self.__module.test_valid(self.__pipeline)
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/cpmodule.py", line 311, in test_valid
    self.visible_settings()[0])
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/modules/run_imagej.py", line 488, in visible_settings
    self.command_settings_dictionary)
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/modules/run_imagej.py", line 326, in get_command_settings
    module_info = command.get_selected_leaf()[2]
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/settings.py", line 1546, in get_selected_leaf
    tree = self.get_tree()
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/settings.py", line 1562, in get_tree
    return self.__tree()
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/modules/run_imagej.py", line 284, in get_choice_tree
    ij2_module_service = IJ2.get_module_service(context)
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/imagej/imagej2.py", line 149, in get_module_service
    o = context.getService('imagej.ext.module.ModuleService')
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/imagej/imagej2.py", line 139, in getService
    '(Ljava/lang/Class;)Limagej/IService;', klass)
  File"/Users/dlogan/svn/CellProfiler/trunk/CellProfiler/cellprofiler/utilities/jutil.py", line 421, in call
    assert o is not None
AssertionError
 

If a bad value exists in a setting (for example, a Text with default=[]), then the conversion to string in the new settings insert fails. Current SVN version (11531) puts the pipeline in a bad state (silently). add_module() should try to generate settings earlier, and if it fails, report to user.

Original reporter: thouis
Jira link: https://jira.broadinstitute.org/browse/IMG-1538
Fix version: Current
Assignee: leek

This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463

I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?

The forum poster is running headless, so I'm not sure what recourse he has.

However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
(1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
(2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
(3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
----------------------------------------------
Traceback (most recent calllast):
  File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
    m = cpm.Measurements(filename = output_path)
  File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
    self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
  File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
    self.hdf5_file = h5py.File(self.filename,'w')
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
    fid = make_fid(name, mode, fapl)
  File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
  File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
IOError: unable to create file (File accessability: Unable to open file)

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1531
Fix version: current
Assignee: leek

Comment from leek on Fri, 16 Sep 2011 11:49:21 -0400:
I answered the post. The files are needed to keep the measurements from
consuming memory. They should be deleted at the end of the run.

On 9/16/2011 11:18 AM, Mark-Anthony Bray (JIRA) wrote:
> HDF5 file creation weirdness: File misnaming + locking
> ------------------------------------------------------
>
> Key:<a href="https://jira.broadinstitute.org/browse/IMG-1531"title="HDF5 file creation weirdness: File misnaming + locking">IMG-1531
> URL:https://jira.broadinstitute.org/browse/IMG-1531
> Project: Imaging
> Issue Type: Bug
> Components: CellProfiler
> Affects Versions: Current
> Reporter: Mark-Anthony Bray
> Assignee: Lee Kamentsky
> Priority: Major
> Fix For: Current
>
>
> This issue was noticed when looking at this forum post:http://cellprofiler.org/forum/viewtopic.php?f=16&amp;t=1463
>
> I have noticed the appearance of these files as well, even if I specify"Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if"Do not write..." is selected?
>
> The forum poster is running headless, so I'm not sure what recourse he has.
>
> However, if I specify"HDF5" as the measurement format, and allow overwriting, I notice the following:
> (1) The output filename will be shown as"DefaultOut.h5" rather than the usual"DefaultOUT.h5" (note the case change)
> (2) Running the pipeline with the output filename in (1) will create a"DefaultOUT" file (no extension but it's an HDF5 file)
> (3) Changing the output filename to"DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:
> ----------------------------------------------
> Traceback (most recent call last):
> File"C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images
> m = cpm.Measurements(filename = output_path)
> File"C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in init
> self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)
> File"C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in init
> self.hdf5_file = h5py.File(self.filename,'w')
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 150, in init
> fid = make_fid(name, mode, fapl)
> File"C:\Python27\lib\site-packages\h5py_hl\files.py", line 51, in make_fid
> fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)
> File"h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)
> IOError: unable to create file (File accessability: Unable to open file)
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see:http://www.atlassian.com/software/jira