segfault when reading string arrays about hdf5.node HOT 15 CLOSED

I'll try to set up a test case. You were working with variable length strings?
For char's (fixed length) I believe I returned a string or for node Buffer's, bytes/chars in the buffer.

I recently opened the hdf files under http://www.salome-platform.org/ app for cad and wow; they have a lot of interesting datatypes to examine. Lots of strings and arrays and varlen's. Let me look through these and see if there are cases like yours and add more handling of these.

from hdf5.node.

Yes, I would like to be able to handle arrays of variable length strings. I can provide a test file if that would be useful. And thanks for looking into it! Please let me know if I can be of any help.

from hdf5.node.

Hi @scottwittenburg,

Added dataset make and read for an Array of javascript strings. Now can't conclude this is the dataset typing your initial h5 file has so I may need a test file.
http://rycole.com/hdf5.node/tut/dataset-tutorial.html
shows how to use at "Array of variable length strings" section. HDFView shows the dataset typing but not the data itself. When read back I get the original strings and the h5 file has size.

The datatype is of the H5T_ARRAY class https://www.hdfgroup.org/HDF5/doc/RM/RM_H5T.html#Datatype-GetClass and the super type is made by their recommendation at https://www.hdfgroup.org/HDF5/doc/RM/RM_H5T.html#CreateVLString:

        hid_t type_id = H5Tcopy(H5T_C_S1);
        H5Tset_size(type_id, H5T_VARIABLE);

I'm thinking of writing some recursive methods on the native side to handle more types and construction of them.

This is currently the whole array in and out.

from hdf5.node.

Thanks for your efforts, it's really great how responsive you are to issues on the list :-) I tried out your changes, and while I can see that it will handle more types of data now, it still causes a segmentation fault on my particular datasets.

Can I share a test file with you? How would you prefer I do that?

For the data I'm trying to read, the datatype is H5T_STRING while the dataspace is H5S_SIMPLE, which is why in my fork of your repo I put the new code inside the H5T_STRING case rather than making a new case for H5T_ARRAY. However, I can see that the new case you have added will make reading and writing datasets more robust, and I appreciate your great work. So thanks again!

from hdf5.node.

Sure an example would be the next way to get in sync. It may be the 'array' is a set of elements on the dataspace instead of a defined datatype array. It is a case the interface will need to handle too.

from hdf5.node.

Share a file by a method you know works and I'll learn how to get it. If you make it small maybe even email

from hdf5.node.

I have not tried this before, but let's see if it works:

http://wikisend.com/download/167520/nba.h5

The file is about 149KB and the datasets which seem not to be readable are:

"player", "team", and "position"

Let me know if you are not able to get it, and I'll find another way to share.

from hdf5.node.

Thank you very much! HDFView is reading from it. We'll see what the evening brings

from hdf5.node.

Thank you very much 😄

from hdf5.node.

Hi, the read is working; have to think about writing since the javascript representation is currently the same for two dataset types; H5T_ARRAY with variable strings and H5T_STRING which can also be variable length and as many as the dataset's dimension.

try
{
    var file = new hdf5.File('./test/examples/nba.h5', Access.ACC_RDWR);
    var array=h5lt.readDataset(file.id, 'player');
    console.dir(array.length);
    if(array.constructor.name==='Array'){
        for(var mIndex=0;mIndex<array.length;mIndex++){
            console.dir(array[mIndex]);
        }
    }
    file.close();
}
catch(err) {
    console.dir(err.message);
}

It is also possible to achieve the same thing with H5T_COMPOUND compound datatypes so there is yet a third way. Think some more in the morning...

from hdf5.node.

A possibility is prototyping from Array's so that the constructor name is "VariableLengthArray" or a more appropriate name. Exposed from the native side so it knows about the type/constructor. The compound case would be handled under the current Table interface[add variable length capability in columns. The Table interface already has append and selection of rows, columns or regions.]

Do you need appending on Variable Length Array's? Any 'regex' search and retrieve?

from hdf5.node.

For our current use case, we do not need to append on variable length arrays. In fact, we don't need the ability to write at all, as we currently use C library interface to write our HDF5 files, and then they remain fixed and static forever.

I will ask around on my end and find out if any regex search and retrieve could be interesting down the road.

Thanks again for tackling this issue so quickly. I'll try out the new read capability today sometime and get back to you.

from hdf5.node.

So I finally got a chance to try it out, and it works great! Thanks so much for tackling this issue. Should I close this issue, or should I let you do it when you have finished work on writing n-dim arrays of varlen strings?

from hdf5.node.

This one can close; new ones can open. The refactor/rewrite for more datatypes will involve more than the varlen strings and possibly wide characters although I still need to learn what to do with wide characters in javascript.. The other part I want to complete more is the subset from datasets and then publish again.

Your example really helped! Any more you have are most welcome

from hdf5.node.

Sounds great, thanks again! I'll close this one now, and we'll see what examples I may come up with in the future :-)

from hdf5.node.