saalfeldlab / n5-ij Goto Github PK

View Code? Open in Web Editor NEW

14.0 11.0 8.0 2.72 MB

ImageJ convenience layer for N5

License: BSD 2-Clause "Simplified" License

Java 99.87% Groovy 0.13%

fiji fiji-plugin hdf5 n5 ome-zarr-converter zarr

n5-ij's Issues

Export of RGB types fails

Should RGB images be written as int32 images (rgba)? or as 3-channel uint8 images? or prompt user for a choice?

Exception in thread "main" java.lang.NullPointerException
	at org.janelia.saalfeldlab.n5.DatasetAttributes.asMap(DatasetAttributes.java:103)
	at org.janelia.saalfeldlab.n5.N5Writer.setDatasetAttributes(N5Writer.java:84)
	at org.janelia.saalfeldlab.n5.N5Writer.createDataset(N5Writer.java:133)
	at org.janelia.saalfeldlab.n5.imglib2.N5Utils.save(N5Utils.java:1273)
	at org.janelia.saalfeldlab.n5.ij.N5Exporter.writeSplitChannels(N5Exporter.java:248)
	at org.janelia.saalfeldlab.n5.ij.N5Exporter.process(N5Exporter.java:205)
	at org.janelia.saalfeldlab.n5.ij.N5Exporter.run(N5Exporter.java:280)
	at org.janelia.saalfeldlab.n5.demos.runImageJBogo.main(runImageJBogo.java:70)

Progress bar while file saving

When saving an active image window to the N5 format with File > Save as.. > Export N5 is there a way to look at the task progress? Thanks.

macro recording of crop broken

to reproduce

turn on macro recorder
open an n5 with crop options
observe crop parameters in the recorded macro
run recorded macro
observe that the result is not cropped, but the whole dataset

java.lang.runtime.exception when opening zarr

Hello.

Great work and many thanks for making this plugin!

I am trying to open a zarr file, but keep running into the following exception.

I tried it with "Open as virtual" checked and not. Do you know what went wrong?

Exception in thread "AWT-EventQueue-0" java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 1000000
	at net.imglib2.cache.util.CacheAsUncheckedCacheAdapter.get(CacheAsUncheckedCacheAdapter.java:78)
	at net.imglib2.img.cell.LazyCellImg$LazyCells.get(LazyCellImg.java:104)
	at net.imglib2.img.list.AbstractLongListImg$LongListRandomAccess.get(AbstractLongListImg.java:274)
	at net.imglib2.img.cell.CellRandomAccess.getCell(CellRandomAccess.java:137)
	at net.imglib2.img.cell.CellRandomAccess.updatePosition(CellRandomAccess.java:475)
	at net.imglib2.img.cell.CellRandomAccess.<init>(CellRandomAccess.java:131)
	at net.imglib2.img.cell.AbstractCellImg.randomAccess(AbstractCellImg.java:104)
	at net.imglib2.img.cell.AbstractCellImg.randomAccess(AbstractCellImg.java:51)
	at net.imglib2.util.Util.getTypeFromInterval(Util.java:806)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.read(N5Importer.java:390)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.process(N5Importer.java:510)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.process(N5Importer.java:482)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.process(N5Importer.java:530)
	at org.janelia.saalfeldlab.n5.ij.N5Importer$1.run(N5Importer.java:568)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.processThread(N5Importer.java:571)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.datasetSelectorCallBack(N5Importer.java:297)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.ok(DatasetSelectorDialog.java:542)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.lambda$run$5(DatasetSelectorDialog.java:270)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
	at java.awt.Component.processMouseEvent(Component.java:6535)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6300)
	at java.awt.Container.processEvent(Container.java:2236)
	at java.awt.Component.dispatchEventImpl(Component.java:4891)
	at java.awt.Container.dispatchEventImpl(Container.java:2294)
	at java.awt.Component.dispatchEvent(Component.java:4713)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4888)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4525)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4466)
	at java.awt.Container.dispatchEventImpl(Container.java:2280)
	at java.awt.Window.dispatchEventImpl(Window.java:2750)
	at java.awt.Component.dispatchEvent(Component.java:4713)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.awt.EventQueue$4.run(EventQueue.java:729)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 1000000
	at net.imglib2.cache.ref.SoftRefLoaderCache.get(SoftRefLoaderCache.java:145)
	at net.imglib2.cache.util.LoaderCacheAsCacheAdapter.get(LoaderCacheAsCacheAdapter.java:76)
	at net.imglib2.cache.util.CacheAsUncheckedCacheAdapter.get(CacheAsUncheckedCacheAdapter.java:74)
	... 53 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1000000
	at net.imglib2.img.basictypeaccess.array.AbstractByteArray.setValue(AbstractByteArray.java:67)
	at net.imglib2.type.numeric.integer.GenericByteType.setByte(GenericByteType.java:135)
	at org.janelia.saalfeldlab.n5.imglib2.N5CellLoader.lambda$createCopy$1(N5CellLoader.java:191)
	at org.janelia.saalfeldlab.n5.imglib2.N5CellLoader.load(N5CellLoader.java:144)
	at net.imglib2.cache.img.LoadedCellCacheLoader.get(LoadedCellCacheLoader.java:124)
	at net.imglib2.cache.img.LoadedCellCacheLoader.get(LoadedCellCacheLoader.java:84)
	at net.imglib2.cache.ref.SoftRefLoaderCache.get(SoftRefLoaderCache.java:135)
	... 55 more

Remember last container opened

Unify n5 reader and writer factories

Implementations:

NullPointerException after Browse

Found by Grace Park

Clicking Browse to choose a container from the plugin gave the error

Exception in thread “AWT-EventQueue-0” java.lang.NullPointerException
	at java.io.File.<init>(File.java:277)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openBrowseDialog(DatasetSelectorDialog.java:459)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openContainer(DatasetSelectorDialog.java:488)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openContainer(DatasetSelectorDialog.java:477)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.lambda$run$2(DatasetSelectorDialog.java:275)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
	at java.awt.Component.processMouseEvent(Component.java:6539)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6304)
	at java.awt.Container.processEvent(Container.java:2239)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2297)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
	at java.awt.Container.dispatchEventImpl(Container.java:2283)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
	at java.awt.EventQueue$4.run(EventQueue.java:733)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Write into existing n5 dataset

Specifying where the current image belongs into the larger volume

Warn users if an array exists, allow overwriting

If an array exists at the user specified path (or at a child of the user specified path),
prompt users with a dialog asking if they want to overwrite the array.

If the users decides to overwrite, delete the path and all children before writing the array.

Support for non-standard port numbers

When I try to open an N5 that lives on an S3 Proxy with a port number other than 80/443 (e.g. https://rokickik-dev.int.janelia.org:8000), it throws an exception (see below). When I run the proxy on port 443 (e.g. https://rokickik-dev.int.janelia.org), this error goes away. It looks like it's inserting brackets into the hostname before parsing.

(Fiji Is Just) ImageJ 2.14.0/1.54f; Java 1.8.0_322 [64-bit]; Mac OS X 10.16; 1677MB of 65435MB (2%)
 
org.janelia.saalfeldlab.n5.N5Exception: Could not create s3 client from uri: https://rokickik-dev.int.janelia.org:8000
	at org.janelia.saalfeldlab.n5.universe.N5Factory.createS3(N5Factory.java:208)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.newAmazonS3KeyValueAccess(N5Factory.java:109)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.access$200(N5Factory.java:80)
	at org.janelia.saalfeldlab.n5.universe.N5Factory$KeyValueAccessBackend.lambda$static$3(N5Factory.java:582)
	at org.janelia.saalfeldlab.n5.universe.N5Factory$KeyValueAccessBackend.apply(N5Factory.java:601)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.getKeyValueAccess(N5Factory.java:232)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.openN5Container(N5Factory.java:536)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.openN5Container(N5Factory.java:558)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.openReader(N5Factory.java:349)
	at org.janelia.saalfeldlab.n5.ij.N5Importer$N5ViewerReaderFun.apply(N5Importer.java:871)
	at org.janelia.saalfeldlab.n5.ij.N5Importer$N5ViewerReaderFun.apply(N5Importer.java:847)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openContainer(DatasetSelectorDialog.java:633)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.lambda$run$4(DatasetSelectorDialog.java:332)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
	at java.awt.Component.processMouseEvent(Component.java:6539)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6304)
	at java.awt.Container.processEvent(Container.java:2239)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2297)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
	at java.awt.Container.dispatchEventImpl(Container.java:2283)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
	at java.awt.EventQueue$4.run(EventQueue.java:733)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: org.janelia.saalfeldlab.n5.N5Exception: Could not create s3 client from uri: https://rokickik-dev.int.janelia.org:8000
	at org.janelia.saalfeldlab.n5.s3.AmazonS3Utils.createS3(AmazonS3Utils.java:132)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.createS3(N5Factory.java:206)
	... 48 more
Caused by: java.net.URISyntaxException: Malformed IPv6 address at index 9: https://[rokickik-dev.int.janelia.org:8000]
	at java.net.URI$Parser.fail(URI.java:2873)
	at java.net.URI$Parser.parseIPv6Reference(URI.java:3494)
	at java.net.URI$Parser.parseServer(URI.java:3244)
	at java.net.URI$Parser.parseAuthority(URI.java:3180)
	at java.net.URI$Parser.parseHierarchical(URI.java:3122)
	at java.net.URI$Parser.parse(URI.java:3078)
	at java.net.URI.<init>(URI.java:673)
	at java.net.URI.<init>(URI.java:774)
	at org.janelia.saalfeldlab.n5.s3.AmazonS3Utils.createS3(AmazonS3Utils.java:129)
	... 49 more

Feature: Image window name

It would be great if the Fiji image window carries the name of the dataset that is open. Currently all windows are named "(V)" which makes it difficult to navigate multiple open datasets without manually renaming the windows.

h5 parsing throws errors and fails

java.util.concurrent.ExecutionException: java.io.IOException: ncsa.hdf.hdf5lib.exceptions.HDF5SymbolTableException: Symbol table:Inappropriate type ["H5Gint.c line 327 in H5G__open_name(): not a group"]
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog$LoaderSorterAndCallback.run(DatasetSelectorDialog.java:609)
Caused by: java.io.IOException: ncsa.hdf.hdf5lib.exceptions.HDF5SymbolTableException: Symbol table:Inappropriate type ["H5Gint.c line 327 in H5G__open_name(): not a group"]
	at org.janelia.saalfeldlab.n5.hdf5.N5HDF5Reader.list(N5HDF5Reader.java:217)
	at org.janelia.saalfeldlab.n5.N5DatasetDiscoverer$1.call(N5DatasetDiscoverer.java:153)
	at org.janelia.saalfeldlab.n5.N5DatasetDiscoverer$1.call(N5DatasetDiscoverer.java:142)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: ncsa.hdf.hdf5lib.exceptions.HDF5SymbolTableException: Symbol table:Inappropriate type ["H5Gint.c line 327 in H5G__open_name(): not a group"]
	at ch.systemsx.cisd.hdf5.hdf5lib.H5.H5Gopen(Native Method)
	at ch.systemsx.cisd.hdf5.hdf5lib.H5GLO.H5Gopen(H5GLO.java:80)
	at ch.systemsx.cisd.hdf5.HDF5.openGroup(HDF5.java:322)
	at ch.systemsx.cisd.hdf5.HDF5$10.call(HDF5.java:423)
	at ch.systemsx.cisd.hdf5.HDF5$10.call(HDF5.java:1)
	at ch.systemsx.cisd.hdf5.cleanup.CleanUpCallable.call(CleanUpCallable.java:40)
	at ch.systemsx.cisd.hdf5.HDF5.getGroupMembers(HDF5.java:436)
	at ch.systemsx.cisd.hdf5.HDF5BaseReader.getAllGroupMembers(HDF5BaseReader.java:773)
	at ch.systemsx.cisd.hdf5.HDF5BaseReader.getGroupMembers(HDF5BaseReader.java:761)
	at ch.systemsx.cisd.hdf5.HDF5ObjectReadOnlyInfoProviderHandler.getGroupMembers(HDF5ObjectReadOnlyInfoProviderHandler.java:424)
	at org.janelia.saalfeldlab.n5.hdf5.N5HDF5Reader.list(N5HDF5Reader.java:214)

strange behavior using jq translation with hdf5 backends

point to an hdf5 file
use some metadata translation with jq
observed a dataset appeared in the tree view, but was not openable

Feature: Self-hosted S3 compatible object store

Hey Folks,

I'd like to request a new feature extension to the S3 i/o module -- support for named URLs to support self-hosted store implementations (like MinIO). A lot of file storage kit these days comes with S3 compatible stores so it would be a useful feature to be able to use with these.

Issue trying to open hdf5 images exported by BigStitcher

@StephanPreibisch found today

Feature: File browser

It would be great if the file browser starts in the last selected data path, rather than Documents (on Windows). This will make browsing through multiple datasets easier.

Add panel for inverse translation

Metadata translation panel should also have a field for users to provide an inverse translation.
Even though this should not be necessary most of the time; see also saalfeldlab/n5-imglib2#41

ImagePlus getOriginalFileInfo should be useful

ImagePlus.getOriginalFileInfo should hold information about the n5 store used to open a given image.
@cmhulbert

OutOfMemoryError during dataset discovery

Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:717)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378)
	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
	at org.janelia.saalfeldlab.n5.N5DatasetDiscoverer.discoverThreadsHelper(N5DatasetDiscoverer.java:141)
	at org.janelia.saalfeldlab.n5.N5DatasetDiscoverer.access$100(N5DatasetDiscoverer.java:51)
	at org.janelia.saalfeldlab.n5.N5DatasetDiscoverer$1$1.call(N5DatasetDiscoverer.java:167)
	at org.janelia.saalfeldlab.n5.N5DatasetDiscoverer$1$1.call(N5DatasetDiscoverer.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: unable to create new native thread
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog$LoaderSorterAndCallback.run(DatasetSelectorDialog.java:609)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:717)

NullPointerException after browse

I'm getting a new NullPointerException after browsing to an n5 and opening it:

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.lambda$null$10(DatasetSelectorDialog.java:620)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

The n5 I'm opening is an export output generated by https://github.com/saalfeldlab/stitching-spark

The n5 seems to load correctly in the Main tab but nothing is loaded in the other tabs. So I'm not sure if this is a real issue, because viewing seem to work okay afterwards, but the exception just seems worrisome.

Conversion of datatypes not-natively supported by ImageJ

Convert doubles to floats.
Convert integer types
Change display of datasets needing conversion in dialog

Converting integer types

Create a converter by:

iterating through the whole image, creating an array of unique values
sort
linearly map range of input to full range of uint16
Should I map signed 8-bit integers to uint8's?

Display of datasets needing conversion

Red text
append (remap to float), or (remap to 16-bit) to dataset name in as appropriate

Threadpool not being "joined" in n5 save()?

Hi!

I've noticed some odd behaviour in a Python script that uses pyimagej to generate and run a BeanScript that invokes N5IJUtils.save(); if I pass a threadpool object to N5IJUtils.save(), the enclosing Python process never seems to actually exit - despite control reaching the end of the Python script.

For example, this invocation of N5IJUtils.save() works fine, with the output correctly generated and the enclosing Python script exiting as expected:

try {
  N5IJUtils.save(
      image_object,
      new N5FSWriter(filepath),
      dataset,
      new int[] {128, 128, 64},
      new GzipCompression()
  );
} catch (Exception e) {
  print(e);
}

However, this invocation results in the process never ending, despite the output being written and control reaching the end of the enclosing Python script:

try {
  N5IJUtils.save(
      image_object,
      new N5FSWriter(filepath),
      dataset,
      new int[] {128, 128, 64},
      new GzipCompression(),
      Executors.newFixedThreadPool(nthreads)
  );
} catch (Exception e) {
  print(e);
}

It seems like the threadpool spawns some sort of rogue background threads or process that's never being join'd?

Thanks!

Metadata translation progress

Keeping track of progress on metadata translations here
https://github.com/saalfeldlab/n5-ij/tree/translation-metadata

Some of this work may move to n5-imglib2

Ideas for consideration

implement a dataset-wise translation that does not require operating on the entire tree

TODOs

Make CanonicalMetadataParser that removes translation functionality from TranslatedTreeMetadataParser (70ac4aa)
Make CanonicalMetadata an interface and simplify (d6603bc )
- Simplify deserializer to be more like transform deserializer in that it can return one of several subclasses (
  d6603bc )

Unit should be "micrometer" for compatbility with OME-NGFF

When exporting with OME-NGFF metadata, the current plugin produces something like this:

{"path":"image/s0","axes":[{"type":"space","name":"x","unit":"micron"},{"type":"space","name":"y","unit":"micron"},{"type":"space","name":"z","unit":"micron"}],"coordinateTransformations":[{"scale":[0.2,0.13000001170000106,0.13000001170000106],"type":"scale"},{"translation":[0.0,0.0,0.0],"type":"translation"}]}

The unit "micron" is not recommended by the spec which instead recommends "micrometer". If this isn't possible, perhaps OME-NGFF can be petitioned to add "micron", since it is a common term and valid in UDUNITS-2.

Opening N5 is slow

Hi, for me opening only uses a single thread more or less, which is not intended. Thus I was wondering how the LoopBuilder distributes tasks:

n5-ij/src/main/java/org/janelia/saalfeldlab/n5/ij/N5Importer.java

Lines 487 to 489 in ba707ba

 LoopBuilder.setImages( convImg, ipImg ) 

 .multiThreaded( new DefaultTaskExecutor( exec )) 

 .forEachPixel( (x,y) -> y.set( x ));

Are maybe all processed pixels close to each other, so it wouldn't really open different blocks in parallel since all threads wait for the same block to be loaded?

Also, for virtual loading one should use multi-threading as well. ImageJFunctions.show does take a ExecutorService optionally for that ... we wrote this for BigStitcher some time ago. Because even if you load one z-plane virtually, you still need to load the entire blocks that are usually big in z.

Happy to help,
Stephan

Exporter: add file browser to specify container

Should be able to create new container or choose:

.n5 directories
.zarr directories
.h5, .hdf, .hdf5 files

Support OME-NGFF metadata and group structure

Enhancement: Detect zarr hierarchical structure

Currently the button "Detect dataset" does not detect hierarchical structure of zarr datasets (https://zarr.readthedocs.io/en/stable/tutorial.html#groups). Zarr arrays contained in the datasets are correctly detected and opened if the user navigates to them through the hierarchical structure. It would be convenient if the user can open zarr arrays contained within the structure, in a similar manner to the N5 example in the README.

Exeception when saving as .zarr

I succeeded in saving a 704x1800x3814 unsigned 16-bit image to .n5 and .h5. When I try to save as .zarr I get the following error:

(Fiji Is Just) ImageJ 2.3.0/1.53q; Java 1.8.0_322 [64-bit]; Windows 10 10.0; 9717MB of 48813MB (19%)
 
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Module threw exception
	at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:307)
	at net.imagej.legacy.DefaultLegacyHooks.interceptRunPlugIn(DefaultLegacyHooks.java:166)
	at ij.IJ.runPlugIn(IJ.java)
	at ij.Executer.runCommand(Executer.java:152)
	at ij.Executer.run(Executer.java:70)
	at java.lang.Thread.run(Thread.java:750)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Module threw exception
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:303)
	... 5 more
Caused by: java.lang.RuntimeException: Module threw exception
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -1
	at java.lang.String.substring(String.java:1967)
	at org.janelia.saalfeldlab.n5.zarr.N5ZarrWriter.createDataset(N5ZarrWriter.java:260)
	at org.janelia.saalfeldlab.n5.imglib2.N5Utils.save(N5Utils.java:1315)
	at org.janelia.saalfeldlab.n5.ij.N5IJUtils.save(N5IJUtils.java:254)
	at org.janelia.saalfeldlab.n5.ij.N5IJUtils.save(N5IJUtils.java:219)
	at org.janelia.saalfeldlab.n5.ij.N5IJUtils.save(N5IJUtils.java:191)
	at org.janelia.saalfeldlab.n5.ij.N5Exporter.write(N5Exporter.java:332)
	at org.janelia.saalfeldlab.n5.ij.N5Exporter.process(N5Exporter.java:252)
	at org.janelia.saalfeldlab.n5.ij.N5Exporter.run(N5Exporter.java:420)
	at org.scijava.command.CommandModule.run(CommandModule.java:196)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:163)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	... 6 more

Enable users to manually specify axis order

Consistency between FIJI GUI and N5IJUtils.load()

Hi,

When I save an n5 dataset in the FIJI GUI, it asks me to specify a Dataset. Examining the resultant n5 directory shows a subdirectory with this name, under which all the data appears to be saved. So far so good.

However, trying to read the previously saved n5 data via e.g. a BeanScript using N5IJUtils.load() and passing in the same Dataset value used to generate the n5 output doesn't work; you need to append a bunch of stuff ("/c0/s0") to the Dataset value you previously specified in FIJI to get it to load.

E.g.: If I call my Dataset "default" when writing the n5 in FIJI, I have to use "default/c0/s0" as the Dataset value in N5IJUtils.load().

This is very confusing; why are these Dataset values different for writing and reading to/from the same n5 "file"? Can we rely on adding "/c0/s0" to our Dataset values being stable in the future?

Opening zarr files with .zattrs

Hello. I noticed that opening .zarr directories that contain .zattrs is not possible. This is the exception:

Exception in thread "AWT-EventQueue-0" java.lang.UnsupportedOperationException: JsonNull
	at com.google.gson.JsonElement.getAsString(JsonElement.java:179)
	at org.janelia.saalfeldlab.n5.zarr.N5ZarrReader.getZArraryAttributes(N5ZarrReader.java:258)
	at org.janelia.saalfeldlab.n5.zarr.N5ZarrReader.getDatasetAttributes(N5ZarrReader.java:267)
	at org.janelia.saalfeldlab.n5.zarr.N5ZarrReader.datasetExists(N5ZarrReader.java:275)
	at org.janelia.saalfeldlab.n5.zarr.N5ZarrReader.exists(N5ZarrReader.java:287)
	at org.janelia.saalfeldlab.n5.N5FSReader.<init>(N5FSReader.java:118)
	at org.janelia.saalfeldlab.n5.zarr.N5ZarrReader.<init>(N5ZarrReader.java:100)
	at org.janelia.saalfeldlab.n5.ij.N5Factory.openZarrReader(N5Factory.java:214)
	at org.janelia.saalfeldlab.n5.ij.N5Factory.openReader(N5Factory.java:383)
	at org.janelia.saalfeldlab.n5.ij.N5Importer$N5ViewerReaderFun.apply(N5Importer.java:691)
	at org.janelia.saalfeldlab.n5.ij.N5Importer$N5ViewerReaderFun.apply(N5Importer.java:672)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openContainer(DatasetSelectorDialog.java:548)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openContainer(DatasetSelectorDialog.java:527)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.lambda$run$2(DatasetSelectorDialog.java:309)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
	at java.awt.Component.processMouseEvent(Component.java:6539)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6304)
	at java.awt.Container.processEvent(Container.java:2239)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2297)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
	at java.awt.Container.dispatchEventImpl(Container.java:2283)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
	at java.awt.EventQueue$4.run(EventQueue.java:733)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Add progress monitoring

There should some indication of progress when loading / saving large n5's

Invalid default directories throws errors / breaks N5 dialog

Thanks to @krokicki who found this.

Need to check for invalid / faulty default directories (e.g. IJ.getDirectory("current") ) before trying to parse the container.

See:

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
	at java.io.File.<init>(File.java:277)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openBrowseDialog(DatasetSelectorDialog.java:450)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openContainer(DatasetSelectorDialog.java:480)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.openContainer(DatasetSelectorDialog.java:468)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.lambda$run$2(DatasetSelectorDialog.java:272)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
	at java.awt.Component.processMouseEvent(Component.java:6539)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6304)
	at java.awt.Container.processEvent(Container.java:2239)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2297)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
	at java.awt.Container.dispatchEventImpl(Container.java:2283)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
	at java.awt.EventQueue$4.run(EventQueue.java:733)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Can't open n5-viewer after fiji update_error relate to "net/imglib2/display/RealARGBColorConverter$Imp0"

The n5-viewer throws the following error after updating Fiji.

java.lang.NoClassDefFoundError: net/imglib2/display/RealARGBColorConverter$Imp0
at org.janelia.saalfeldlab.n5.bdv.N5Viewer.addSourceToListsRealType(N5Viewer.java:279)
at org.janelia.saalfeldlab.n5.bdv.N5Viewer.addSourceToListsGenericType(N5Viewer.java:242)
at org.janelia.saalfeldlab.n5.bdv.N5Viewer.exec(N5Viewer.java:133)
at org.janelia.saalfeldlab.n5.bdv.N5Viewer.run(N5Viewer.java:88)
at ij.IJ.runUserPlugIn(IJ.java:230)
at ij.IJ.runPlugIn(IJ.java:193)
at ij.Executer.runCommand(Executer.java:137)
at ij.Executer.run(Executer.java:66)
at java.lang.Thread.run(Thread.java:748)

Enhancement: Open Virtual by default

Since N5 datasets are optimal for storing large chunked n-dimensional arrays, it may be convenient to open dataset as "virtual" by default.

gson error for images having extra dimension of size 1

First of all, it is really great to have the N5 import and export tools now included in Fiji by default! Thanks very much for this effort! I have been meaning to work with N5 for a while and this makes it much easier.

I have been testing importing and exporting with different images and I keep getting an error from gson when it tries to write an NaN (see below). I seem to get this error from images that have an extra dimension of size 1. For example, an image that contains a Z axis but where all T were collected for Z = 0. Something like X, Y, Z, Channel, Time = 1024, 1024, 1, 2, 160. I think somehow the Z dimension in these videos triggers writing the NaN and gson is set to not allow writing of NaN values, perhaps because they would end up being strings and can cause a lot of problems when parsing json later on...

The videos still seem to be written and I can open them. Should I just ignore this error and continue on? Obviously, another solution would be to always work with the minimal dimensions required - to never have dimensions of size 1. But it seems that the SCIFIO reader we are using opens videos this way with some dimensions of size 1. I don't seem to have the issue when opening the videos with IJ1.

I guess the error below is actually coming from the N5Writer and not really the plugins here, but I thought reporting the issue here would be best since it involves the way videos are opened in Fiji. If you want I can report it over there...

Thanks again for this cool format and helpful plugins. If you don't see this as an issue, I get it, and feel free to close it. But please let me know if you have any tips on how I could deal with this.

Paintera label-multiset datasets cannot be opened

Datasets of type label-multiset should be open-able by imagej, it should convert to an integer type.

(Found by @jonesa3)

Metadata are parsed and the UI let's you select a label-multiset typed dataset, but whey opening, throws

this error trace

Exception in thread "AWT-EventQueue-1" java.lang.UnsupportedOperationException
	at net.imglib2.type.label.LabelMultisetType.getNativeTypeFactory(LabelMultisetType.java:170)
	at net.imglib2.img.imageplus.ImagePlusImgFactory.create(ImagePlusImgFactory.java:69)
	at net.imglib2.img.imageplus.ImagePlusImgFactory.create(ImagePlusImgFactory.java:76)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.read(N5Importer.java:489)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.process(N5Importer.java:624)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.process(N5Importer.java:596)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.process(N5Importer.java:651)
	at org.janelia.saalfeldlab.n5.ij.N5Importer$1.run(N5Importer.java:688)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.processThread(N5Importer.java:691)
	at org.janelia.saalfeldlab.n5.ij.N5Importer.datasetSelectorCallBack(N5Importer.java:372)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.ok(DatasetSelectorDialog.java:756)
	at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.lambda$6(DatasetSelectorDialog.java:320)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:262)
	at java.awt.Component.processMouseEvent(Component.java:6539)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6304)
	at java.awt.Container.processEvent(Container.java:2239)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2297)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
	at java.awt.Container.dispatchEventImpl(Container.java:2283)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
	at java.awt.EventQueue$4.run(EventQueue.java:733)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
	at org.GNOME.Accessibility.AtkWrapper$6.dispatchEvent(AtkWrapper.java:717)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

ImageJ Macro Recording fails to record virtual & crop options

Hi, something we should fix ... if recorded these options (virtual/crop) do not appear.

Thanks so much, Stephan

Fiji warning when pulling data from S3

Loading stuff from S3 through Fiji File/Import/N5 and receiving below warning. Plugin still works as expected.

Could not load AWS credentials, falling back to anonymous.
[pool-11-thread-1] ERROR org.janelia.saalfeldlab.n5.N5DatasetDiscoverer - Error encountered during metadata parsing
java.lang.InterruptedException
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
at java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.janelia.saalfeldlab.n5.N5DatasetDiscoverer.parseMetadataRecursive(N5DatasetDiscoverer.java:510)
at org.janelia.saalfeldlab.n5.ui.DatasetSelectorDialog.lambda$openContainer$16(DatasetSelectorDialog.java:657)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Could not load AWS credentials, falling back to anonymous.

Open n5's by drag+drop into Fiji

Possible behavior:

If there is one dataset in the container, open it immediately.
If there are several datasets
- Open all datasets immediately (?)
- Show dialog showing all datasets selected and prompt user for changes (?)

Intermittent test failure

When running the pom-scijava melting pot, the n5-ij component sometimes fails tests with the exception:

java.lang.AssertionError: /cosem_ms
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.junit.Assert.assertNotNull(Assert.java:713)
	at org.janelia.saalfeldlab.n5.metadata.MetadataTests.testCosemMetadataMultiscale(MetadataTests.java:58)

To work around this, I just rerun the CI build and it usually passes the next time. But it would be nice if this could be made more robust. Any ideas what might be going on here?

metadata parsing post n5 attribute cache

Use cases

n5-ij needs:

read metadata from an n5 group or dataset
write metadata into an n5 group or dataset
read metadata from an ImagePlus
write metadata into ImagePlus
Create a tree of all datasets in an n5 container down to the datasets
- n5.deepListDatasets creates a flat list from which we can build a tree

n5-viewer additionally needs:

identify and open multiscale groups of datasets

@cmhulbert
what does paintera need that's not the above

Invalid path in Import > HDF5/N5/Zarr/OME-NGFF ... crop dialog

Problem

When opening an N5 dataset with the Import > HDF5/N5/Zarr/OME-NGFF ... dialog, it is possible to specify a subfolder of an N5 container rather than the base path. E.g., in the following N5 container, specifying base.n5/grp instead of base.n5 to detect the dataset data.

base.n5/
├── attributes.json
└── grp
    └── data

This is convenient, since dataset detection from the root can take some time if the container contains a large hierarchy.

However, if the crop checkbox is checked, the string which is automatically inserted into the N5 path field of the crop dialog duplicates parts of the paths within the N5 container: instead of base.n5/grp/data it is base.n5/grp/grp/data when base.n5/grp is used in the import dialog. This leads to the following error:

not a dataset : base.n5/grp/grp/data

Workaround

Since the string in the crop dialog can be edited, one can simply fix it manually (which is a bit inconvenient since the field is too small to do this efficiently).

Suggested fix

It seems to me that getN5RootPath() in ui/DatasetSelectorDialog.java does not return the N5 root path but simply the string in the text field of the import dialog. Returning the actual N5 root there would probably fix the problem.

OME-Zarr containers do not parse when nested inside other paths

When using the "Open N5" dialog if I select an OME-Zarr it parses quickly and correctly:

But if I create an empty folder and put the OME-Zarr inside, and point "Open N5" to that containing folder, it gets stuck on the "Parsing..." step for several minutes and then eventually just presents the underlying directory structure:

There are no errors generated.

Issue opining up h5 file using N5 plugin - images are blank

Hi!
I'm trying to open some h5 file through import->N5 on image J, but the images turned up to be blank. The file is able to open up properly using the import->HDF5. I wonder what the issue is?

Context: My dataset is very large and exceeds the HDF5 plugin size limit, so I want to use the N5 plugin and use its virtual stack function. The N5 opens the file as blank no matter if the file size is big or small, or if I use virtual stack or not.

Thank you very much!

Feature request: option to generate multichannel/multiscale during export

When exporting N5 it would be nice to have an option to generate a multichannel/multiscale representation as supported by BDV. Meaning:

Etc. Along with the associated multiscale metadata.

I believe that BigStitcher already has this option when resaving to N5, so maybe code could be borrowed from there.

4D zarr datasets not loaded correctly in FIJI

I am using FIJI with ImageJ 1.53q. Looking in the installation direction jars subfolder I find n5-ij-3.2.2.jar and n5-zarr-0.0.7.jar

When I use File -> Import -> N5 to load a 4D zarr dataset, the resulting imagej hyperstack that is produced is not correct. This hyperstack has two sliders, "c" and "z". However, the data from the zarr is not put in the correct place. The data from the "c" and "z" dimensions in the zarr are mixed up in the imagej hyperstack.

A minimal example:
I create a zarr using the following python code

import zarr
import numpy as np

# zarr dimensions
nc = 5
nz = 30
nx = 256
ny = 256

# create zarr
fname = "test.zarr"
data = zarr.open(fname, "w")
data.create_dataset("test", shape=(nc, nz, ny, nx),
                    chunks=(1, nz, ny, nx), dtype='float32', compressor="none")

# zarr values are z-slice coordinates
for ii in range(nc):
    for jj in range(nz):
        data.test[ii, jj] = np.ones((ny, nx)) * jj

Next I open it in FIJI with File -> Import -> N5. Now I expect when I scroll through the "c" channel in the hyperstack that I will not see the value of the image change, since the image value only depends on the z-index. But I do see it change. For example, if I keep the z-index at value 1/30 and scroll the c index I see image values of 0, 1, 2, 3, and 4. If I increment the z-index to 2/30 and scoll c I see 5, 6, 7, 8, and 9.

New feature - add file size

In the 'Open N5' panel, it would be helpful to additonally see the file size (bytes). This will help determine what resolution to open based on your memory. Currently the image dimensions and image type are displayed.

	LoopBuilder.setImages( convImg, ipImg )
	.multiThreaded( new DefaultTaskExecutor( exec ))
	.forEachPixel( (x,y) -> y.set( x ));