Comments (27)
rimmartin
commented on July 3, 2024
Ubunutu 12.04 is the same OS I'm using
The problem is I have not developed with this backward compatible with older versions of V8 and nodejs
node v0.11.13-pre and V8 3.25.30 are the latest I use. v0.10.26 is before the big change in the native interface to V8.
Are other libraries/dependencies requiring 0.10.26?
I'll be building and testing this week. A part I need to add is attributes and more metadata. What parts of h5 you need the most?
from hdf5.node.
iiSeymour
commented on July 3, 2024
The versions are those in standard Ubuntu repositories for 12.04. A version that builds with the current stable release v0.10.30 would be good. Both attributes and metadata would also be very useful to me!
from hdf5.node.
rmp
commented on July 3, 2024
Seconded! I have an application using node-webkit which could really benefit from hdf5 support but some current instability with streams in node 0.11 has left me stuck with 0.10.
from hdf5.node.
rimmartin
commented on July 3, 2024
Ok, not sure of the timeline of V8 and nodejs releases. So for a week I'll look into the attributes and metadata additions I'm making and possibly a branch that will work with nodejs v0.10.30.
And also publish so this can be pulled into other projects readily.
Now this is experimental; any feedback on what parts of h5 you want to use? For example the image interface? And what form would you like the data when it reaches javascript? I was only experimenting; haven't tied the data to any image library. What would be the most efficient for you? Shape of data? For IM or any other hdf5 module? So far the data is nearly direct to first memory it reaches(i.e. no copying needed to bring into js objects.)
from hdf5.node.
rmp
commented on July 3, 2024
Thanks for spending the time on this. For me I don't currently need images. I mostly use lists & lookups of groups/datasets/attrs and datasets with arrays of chars/ints/floats/doubles.
from hdf5.node.
rimmartin
commented on July 3, 2024
Ah ok, thank you. Will start with those. And what iiSeymour may need too.
from hdf5.node.
iiSeymour
commented on July 3, 2024
My needs are pretty much the same as @rmp
from hdf5.node.
rimmartin
commented on July 3, 2024
Well then we have a starting point :-)
from hdf5.node.
iiSeymour
commented on July 3, 2024
👍
from hdf5.node.
rimmartin
commented on July 3, 2024
Hi, I've got a good start. Am starting to write data sets.
Since V8-3.14.5.9 is incomplete wrt to Float64Array and other js array 'objects' I'm learning to make the v8:Local<v8Object> do the job. It will take a bit more this week to finish reading datasets and then on to the attributes and metadata.
http://izs.me/v8-docs/classv8_1_1Object.html
vs
https://thlorenz.github.io/v8-dox/build/v8-3.25.30/html/db/d85/classv8_1_1_object.html
It is doable; it just a little more than changing the v8 function calls and needs custom prototyping of some array objects.
I will have it under the same git branch in a legacy folder where I'll maintain a series for specific nodejs and v8 major changes
The mocha tests and js side will be exactly the same no matter the nodejs or v8 version
from hdf5.node.
iiSeymour
commented on July 3, 2024
Awesome, good progress.
from hdf5.node.
rimmartin
commented on July 3, 2024
I've committed and pushed current development. There is a ./legacy/node-v0.10.31 folder with equivalent code but runs with the same test code in ./test.
Should work with ./legacy/node-v0.10.x back a few versions.
Groups have attributes (mostly primitives and strings) mapped to javascript properties via refresh and flush. The main Readme.md has some examples and discussion. Maybe I need to build some wiki examples. I'll be adding attributes on the file object and on datasets, images and tables(yet to start tables where they will be mapped to array of arrays to accommodate columns of different types).
There are some shape limitations I want to experiment with going to the array objects of javascript. What is the largest size datasets you're currently working with?
Have any example I can test with?
from hdf5.node.
iiSeymour
commented on July 3, 2024
Hmm, I am still struggling to build the module 😦
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp info spawn python
gyp info spawn args [ '/usr/local/lib/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/home/cseymour/Code/node/node5/hdf5.node/legacy/node-v0.10.31/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/usr/local/lib/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/cseymour/.node-gyp/0.10.31/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/cseymour/.node-gyp/0.10.31',
gyp info spawn args '-Dmodule_root_dir=/home/cseymour/Code/node/node5/hdf5.node/legacy/node-v0.10.31',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory `/home/cseymour/Code/node/node5/hdf5.node/legacy/node-v0.10.31/build'
CXX(target) Release/obj.target/h5im/src/h5im.o
In file included from ../src/h5im.cc:2:0:
../src/hdf5.h:20:66: error: a brace-enclosed initializer is not allowed here before ‘{’ token
In file included from ../src/h5im.cc:2:0:
../src/hdf5.h:20:178: error: ISO C++ forbids initialisation of member ‘toAccessMap’ [-fpermissive]
../src/hdf5.h:20:178: error: making ‘toAccessMap’ static [-fpermissive]
../src/hdf5.h:20:178: error: invalid in-class initialisation of static data member of non-integral type ‘std::map<long unsigned int, long unsigned int>’
../src/hdf5.h:49:13: error: ‘shared_ptr’ in namespace ‘std’ does not name a type
In file included from ../src/Uint8Array.hpp:12:0,
from ../src/h5_im.hpp:11,
from ../src/h5im.cc:3:
../src/ArrayBuffer.hpp:24:9: error: ‘unique_ptr’ in namespace ‘std’ does not name a type
../src/ArrayBuffer.hpp: In constructor ‘NodeHDF5::ArrayBuffer::ArrayBuffer(long long unsigned int)’:
../src/ArrayBuffer.hpp:38:62: error: class ‘NodeHDF5::ArrayBuffer’ does not have any field named ‘buffer’
../src/ArrayBuffer.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::ArrayBuffer::NewInstance(const v8::Arguments&)’:
../src/ArrayBuffer.hpp:86:75: error: ‘class NodeHDF5::ArrayBuffer’ has no member named ‘buffer’
In file included from ../src/Uint8Array.hpp:12:0,
from ../src/h5_im.hpp:11,
from ../src/h5im.cc:3:
../src/ArrayBuffer.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::ArrayBuffer::slice(const v8::Arguments&)’:
../src/ArrayBuffer.hpp:107:72: error: ‘class NodeHDF5::ArrayBuffer’ has no member named ‘buffer’
../src/ArrayBuffer.hpp:108:77: error: ‘class NodeHDF5::ArrayBuffer’ has no member named ‘buffer’
../src/ArrayBuffer.hpp: In static member function ‘static v8::Local<v8::Object> NodeHDF5::ArrayBuffer::Instantiate(long long unsigned int)’:
../src/ArrayBuffer.hpp:126:75: error: ‘class NodeHDF5::ArrayBuffer’ has no member named ‘buffer’
In file included from ../src/h5_im.hpp:11:0,
from ../src/h5im.cc:3:
../src/Uint8Array.hpp: At global scope:
../src/Uint8Array.hpp:19:9: error: ‘unique_ptr’ in namespace ‘std’ does not name a type
../src/Uint8Array.hpp: In constructor ‘NodeHDF5::Uint8Array::Uint8Array(long long unsigned int)’:
../src/Uint8Array.hpp:43:61: error: class ‘NodeHDF5::Uint8Array’ does not have any field named ‘buffer’
../src/Uint8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Uint8Array::NewInstance(const v8::Arguments&)’:
../src/Uint8Array.hpp:100:75: error: ‘class NodeHDF5::Uint8Array’ has no member named ‘buffer’
In file included from ../src/h5_im.hpp:11:0,
from ../src/h5im.cc:3:
../src/Uint8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Uint8Array::get(const v8::Arguments&)’:
../src/Uint8Array.hpp:117:47: error: ‘class NodeHDF5::Uint8Array’ has no member named ‘buffer’
../src/Uint8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Uint8Array::set(const v8::Arguments&)’:
../src/Uint8Array.hpp:133:16: error: ‘class NodeHDF5::Uint8Array’ has no member named ‘buffer’
../src/Uint8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Uint8Array::slice(const v8::Arguments&)’:
../src/Uint8Array.hpp:153:72: error: ‘class NodeHDF5::Uint8Array’ has no member named ‘buffer’
../src/Uint8Array.hpp:154:77: error: ‘class NodeHDF5::Uint8Array’ has no member named ‘buffer’
../src/Uint8Array.hpp: In static member function ‘static v8::Local<v8::Object> NodeHDF5::Uint8Array::Instantiate(long long unsigned int)’:
../src/Uint8Array.hpp:182:75: error: ‘class NodeHDF5::Uint8Array’ has no member named ‘buffer’
../src/Uint8Array.hpp: In static member function ‘static v8::Local<v8::Object> NodeHDF5::Uint8Array::Instantiate(v8::Local<v8::Object>)’:
../src/Uint8Array.hpp:205:82: error: ‘class NodeHDF5::ArrayBuffer’ has no member named ‘buffer’
In file included from ../src/h5_im.hpp:12:0,
from ../src/h5im.cc:3:
../src/Int8Array.hpp: At global scope:
../src/Int8Array.hpp:19:9: error: ‘unique_ptr’ in namespace ‘std’ does not name a type
../src/Int8Array.hpp: In constructor ‘NodeHDF5::Int8Array::Int8Array(long long unsigned int)’:
../src/Int8Array.hpp:42:60: error: class ‘NodeHDF5::Int8Array’ does not have any field named ‘buffer’
../src/Int8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Int8Array::NewInstance(const v8::Arguments&)’:
../src/Int8Array.hpp:99:75: error: ‘class NodeHDF5::Int8Array’ has no member named ‘buffer’
In file included from ../src/h5_im.hpp:12:0,
from ../src/h5im.cc:3:
../src/Int8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Int8Array::get(const v8::Arguments&)’:
../src/Int8Array.hpp:116:47: error: ‘class NodeHDF5::Int8Array’ has no member named ‘buffer’
../src/Int8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Int8Array::set(const v8::Arguments&)’:
../src/Int8Array.hpp:132:16: error: ‘class NodeHDF5::Int8Array’ has no member named ‘buffer’
../src/Int8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Int8Array::slice(const v8::Arguments&)’:
../src/Int8Array.hpp:152:72: error: ‘class NodeHDF5::Int8Array’ has no member named ‘buffer’
../src/Int8Array.hpp:153:77: error: ‘class NodeHDF5::Int8Array’ has no member named ‘buffer’
../src/Int8Array.hpp: In static member function ‘static v8::Local<v8::Object> NodeHDF5::Int8Array::Instantiate(long long unsigned int)’:
../src/Int8Array.hpp:181:75: error: ‘class NodeHDF5::Int8Array’ has no member named ‘buffer’
../src/Int8Array.hpp: In static member function ‘static v8::Local<v8::Object> NodeHDF5::Int8Array::Instantiate(v8::Local<v8::Object>)’:
../src/Int8Array.hpp:203:82: error: ‘class NodeHDF5::ArrayBuffer’ has no member named ‘buffer’
../src/Uint8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Uint8Array::get(const v8::Arguments&)’:
../src/Uint8Array.hpp:118:5: warning: control reaches end of non-void function [-Wreturn-type]
../src/Int8Array.hpp: In static member function ‘static v8::Handle<v8::Value> NodeHDF5::Int8Array::get(const v8::Arguments&)’:
../src/Int8Array.hpp:117:5: warning: control reaches end of non-void function [-Wreturn-type]
make: *** [Release/obj.target/h5im/src/h5im.o] Error 1
make: Leaving directory `/home/cseymour/Code/node/node5/hdf5.node/legacy/node-v0.10.31/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/node-gyp/lib/build.js:267:23)
gyp ERR! stack at ChildProcess.emit (events.js:98:17)
gyp ERR! stack at Process.ChildProcess._handle.onexit (child_process.js:810:12)
gyp ERR! System Linux 3.2.0-58-generic
gyp ERR! command "node" "/usr/local/bin/node-gyp" "configure" "build"
gyp ERR! cwd /home/cseymour/Code/node/node5/hdf5.node/legacy/node-v0.10.31
gyp ERR! node -v v0.10.31
gyp ERR! node-gyp -v v1.0.1
gyp ERR! not ok from hdf5.node.
rimmartin
commented on July 3, 2024
Ah the next description I need to improve is the Compile section; and example of a build script. I'll put more together tonight.
I'm using c++11. My gcc flags look like
export FC=gfortran
export CC=gcc
export CXX=g++
export LDFLAGS="-fPIC -O4 -std=c++11 -L$HDF5_HOME/lib -I$HDF5_HOME/include/oce "
export CFLAGS="-fPIC -O4 -I/home/roger/Software/node-v0.10.31/src -I$V8_HOME/include -I$V8_HOME/src -I$HDF5_HOME/include -I$HDF5_HOME/include/oce "
export CXXFLAGS="-fPIC -O4 -std=c++11 -I/home/roger/Software/node-v0.10.31/src -I$V8_HOME/include -I$V8_HOME/src -I$HDF5_HOME/include -I$HDF5_HOME/include/oce "
export CPPFLAGS="-fPIC -O4 -std=c++11 -I/home/roger/Software/node-v0.10.31/src -I$V8_HOME/include -I$V8_HOME/src -I$HDF5_HOME/include -I$HDF5_HOME/include/oce "
You don't need oce. The build failure is most likely -std=c++11 turns on std::unique_ptr. Been experimenting with using c++11's memory management from
#include <memory> using all smart pointers(no new and deletes) while native to V8; seems to be compatible.
from hdf5.node.
iiSeymour
commented on July 3, 2024
Some detailed compilation instructions would be really useful. The steps I took were:
- Downloaded and built
hdf5with the--enable-cxxflag (installed in/usr/local/hdf5). - Updated path
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/hdf5/lib - Cloned
hdf5.nodeandcd hdf5.node/legacy/node-v0.10.31 - Modified
binding.gypto use/usr/local/hdf5 - Ran
node-gyp configure build
I am suppose to/how do I pass in compiler flags -std=c++11 using the node-gyp build step?
from hdf5.node.
rimmartin
commented on July 3, 2024
Yes, I'll write and post my build scripts after cleaning them up.
The compiler g++ picks it up from the exported CXXFLAGS flag
The node-gyp doesn't appear to pass such along(until I explore more node-gyp) but it leaves g++ pick up what it usually does from the environment
[ed.] perhaps this is the crude way; I'm checking if this can all be set up with the binding.gyp file. Then these settings will be there for next person...
from hdf5.node.
rimmartin
commented on July 3, 2024
The binding.gyp has more what it needs for the gcc compiler. Gyp has a cflags(but no cxxflags) which the g++ compiler picks up. I will be pulling and testing to the computers at work and begin applying it. Another app is using the image part.
After we get past the build please let me know what data structures you need
from hdf5.node.
iiSeymour
commented on July 3, 2024
Any development on this? I'd really like to get a build and start using it.
from hdf5.node.
rimmartin
commented on July 3, 2024
I cleaned up the build so the g++ compiler gets the -std=c++11 from the binding.gyp. The Readme.md is updated with more compile instructions. I've pulled it to my work machine for the first time, built and ran the tests. In the morning I'll pull a legacy node v0.10.31 on and test a legacy build on this fresh install as well.
Update and try a build and you should no longer get "error: ‘unique_ptr’ in namespace ‘std’ does not name a type" errors anymore do to the missing c++11 level of std.
from hdf5.node.
iiSeymour
commented on July 3, 2024
I have managed to complete the build step using your new build instructions (with a minor edit, my version of gcc wanted -std=c++0x instead of -std=c++11), giving:
├── binding.Makefile
├── config.gypi
├── h5im.target.mk
├── h5lt.target.mk
├── hdf5.target.mk
├── Makefile
└── Release
├── h5im.node
├── h5lt.node
├── hdf5.node
├── linker.lock
└── obj.target
├── h5im
│ └── src
│ └── h5im.o
├── h5im.node
├── h5lt
│ └── src
│ └── h5lt.o
├── h5lt.node
├── hdf5
│ └── src
│ ├── h5_file.o
│ ├── h5_group.o
│ └── hdf5.o
└── hdf5.node
So I've created a small test project:
├── app.js
├── node_modules
│ ├── h5im.node
│ ├── h5lt.node
│ └── hdf5.node
└── test.h5
Where app.js contains:
var hdf5 = require('hdf5');
var Access = {
ACC_RDONLY : 0, /*absence of rdwr => rd-only */
ACC_RDWR : 1, /*open for read and write */
ACC_TRUNC : 2, /*overwrite existing files */
ACC_EXCL : 3, /*fail if file already exists*/
ACC_DEBUG : 4, /*print debug info */
ACC_CREAT : 5 /*create non-existing files */
};
var filename = 'test.h5';
var file = new hdf5.File(filename, Access.ACC_RDONLY);Running node app.js gives the following error:
node: symbol lookup error: ./node_modules/hdf5.node: undefined symbol: _ZNK2H510H5Location11getFileNameEv
Any ideas?
from hdf5.node.
rimmartin
commented on July 3, 2024
Great! We're this far.
Check setting NODE_PATH. Search the readme.md for this. I'll try it after work tonight but I do recall seeing the same message when it doesn't know where the results of the build are
from hdf5.node.
iiSeymour
commented on July 3, 2024
Adding the obj.target directory to $NODE_PATH didn't help 😞
from hdf5.node.
rimmartin
commented on July 3, 2024
I have a friend who will help get this to be an installable module; be working with him on what it takes. Then we'll publish it to make it easier to add to projects
from hdf5.node.
iiSeymour
commented on July 3, 2024
Excellent, I really appreciate the effort.
from hdf5.node.
ryancole
commented on July 3, 2024
I'll take a stab at building it on OSX tonight.
Ryan
On Oct 8, 2014, at 3:12 PM, Chris Seymour [email protected] wrote:
Excellent, I really appreciate the effort.
—
Reply to this email directly or view it on GitHub.
from hdf5.node.
rimmartin
commented on July 3, 2024
That will be interesting, thanks Ryan
from hdf5.node.
rimmartin
commented on July 3, 2024
http://nodejs.org/ v0.12.0 is out!
testing with it and the harmony switch
from hdf5.node.
Related Issues (20)
- Support for Single Writer Multiple Reader (SWMR)? HOT 7
- Getting dataset attribute (getDatasetAttribute) on a 32 bit floating point in NODE/javascript returns a totally different value HOT 6
- hdf5_home_win does not get set HOT 2
- Read dataset with 2d array stored fails on reading chunks HOT 10
- issues reading 4 dimensional dataset HOT 2
- SyntaxError: unsupported data type on compound datasets HOT 5
- Node 12? HOT 32
- Delete attribute HOT 6
- Tutorial: "Writing & Reading subsets" is not working. HOT 3
- Segfault reading HOT 4
- issue with appending to tables on windows 10
- Error: The specified procedure could not be found. (process.dlopen) HOT 1
- `hdf5.File is not a constructor` when bundled with webpack HOT 2
- cannot install HOT 3
- Win10+Node v14.15.5 Compilation errors HOT 6
- Getting data from Buffer
- Compile Error HOT 12
- install with yarn? HOT 2
- windows-build-tools / vs2017? HOT 37
- Error with handling variable length data (H5T_VLEN) HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hdf5.node.