Reading/Writing from a subset of a Dataset about hdf5.node HOT 8 CLOSED

Thanks for pointing to a use case and example. I'll experiment with adding subset access and am interested in making this project as useful and complete as possible

There is an additional library that does "region references, hyperslab selections, and bit-field"
https://www.hdfgroup.org/projects/jpss/documentation/HL/RM/index.html that I've been contemplating adding. It requires additional native compiling of a library( maybe could be made optional)

Whether it gets employed I would hope the javascript side to be the same for getting a subset/region either way.

The native side has went through a lot of refactoring and will continue for a while. Yet the api exposed on the javascript side should remain stable while gaining additional functionality. Subsets of datasets is a good one

Studying it...
.

from hdf5.node.

Decided to make it according to the part of the standard API and example you pointed to. Can build up in this nodejs interface what is needed for all region, hyperslabs and bitfields. Starting to come together; we'll see what the day brings...

from hdf5.node.

I put together a basic ability to write and read a subset for node Buffer io. Do you like node Buffer objects or are you wanting the other javascript arrays? I was running into the total dimension might be limited to int datatype which can't handle very very large datasets. Is there another datastructure better on the javascript/nodejs side? Eventually I want to add streaming.

After some more testing I'll publish but for a first look see http://rycole.com/hdf5.node/tut/subset_tutorial.html

When it writes it overwrites the subsets data. I want to add the other operations they have at some point (or and xor kind of operations)

from hdf5.node.

Wow, thanks for the quick response on this subset issue. If @jourdain were in radio contact, he would have liked to thank you himself 😄 I think he'll be back in a week or so, and at that point may have some input on the desired data structure on the node side.

from hdf5.node.

Hi @scottwittenburg,
It's an important feature purposed in hdf5 base library and needs to be available in this interface. I am also looking what to do for image interface; although compression algorithms do complicate.

That gives me another weekend nights to advance this some more:-)

I personally have application with h5 data stored at rank 3 for http://www.pointclouds.org/ data and algorithms so want to get it working and tested for higher ranks

from hdf5.node.

Thanks for looking into that but we move toward a design that does not require the need of subset read anymore. I know it is an important feature, and I would love to see it implemented, but I wanted to give you some heads up and relieve any pressure you could feel to implement that in the coming time.

from hdf5.node.

Ok, the git version is working for 2 rank datasets. I'll publish to npm along with the arrays of variable length chars when the next issue gets answered. Complete more datatypes

from hdf5.node.

Have this working for datasets and images

from hdf5.node.