seung-lab / tinybrain Goto Github PK

@nkemnitz tried to downsample a 64 GB image but was only attempting to generate a single mip level. This required 144 GB due to the extraneous bookkeeping required to prevent integer truncation at mip 2+. Why not have a very fast, low memory version for only mip 1?

It would be good compute several levels of "correct" segmentation hierarchy similarly to the averaging downsampler. This requires a Cython/C++ implementation (which is why we used Countless to begin with).

It would be cool if we could also do average upsampling really fast.

For some operations such as (more) precise meshing of downsample layers, a lossless kind of downsampling is needed.

This would take the form of recording the set of labels that occupy a voxel which hopefully will be many fewer than the number of voxels compressed into a voxel.

We have support for sparse segmentation downsampling, but sometimes it's useful to average images while ignoring background. It's pretty hacky to apply the segmentation downsampling to gray values.

The downside is that sparse seems a more complex computation than simple averaging so will be slightly slower and maybe more memory intensive.

The Precomputed format does not support sub-pixel precision, and so has issues when computing offsets that are not divisible by two. For instance, a dataset voxel offset of five would display correctly at mip 0, but at mip 1, would be required to be offset by either two or three, which introduces a shift of half a pixel relative to the original data.

@nkemnitz believes that tinybrain should be able to emulate sub-pixel precision. I think this is possible because (a) most datasets are ringed by black making the shift due to the boundary seem unreasonable as perturbing the boundary by a small amount should have no effect on the registration of the tissue (b) it would be possible to shift the location of tissue by half a pixel by shifting the downsample window by one pixel on source mip level.

This does present some challenges to the downsample pipeline however as it would require downloading an expanded region per task in order to provide the necessary material for the shift. Additionally, multiple shifts per a task may be in order depending on the numbers involved. Each integer truncation changes the prime factorization of the resulting number. Some of this would be solved in igneous and some would be solved in tinybrain.