Giter VIP home page Giter VIP logo

Comments (13)

mgwalker avatar mgwalker commented on July 22, 2024

I decided to dust off my C++ and see if I could be helpful. I added Nan as a dependency (it abstracts Node and V8 APIs: https://www.npmjs.com/package/nan) and used that in the places where the build was breaking. I made some naive assumptions to get it to build, so it probably doesn't actually work, but building was one step!

I'll try tomorrow to see if I can get the tests to run and everything, and then if you're okay with adding Nan as a dependency, I can create a PR for you. In the meantime, here's the commit where I got everything building on Node 4.2 (and still builds on Node 0.12.7):

mgwalker@a8de73c

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

I'm not into nan. for the long haul. I have it building for v4; just didn't commit and publish. Am branching the code for v12 and v4.
I'll commit and publish tonight

from hdf5.node.

mgwalker avatar mgwalker commented on July 22, 2024

Cool, thanks for the update!

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

Nan limits experimenting with the latest change in v8 and there are some cool things they keep adding

The incompatible change is node::Buffer initialization on the native side going through a check

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

I pushed and published. I did this with nodejs v4.1.1 while building v4.2.1. Now this project builds with v4.2.1 yet during runtime with hdf5-1.8.15-patch1 I see a bug and am looking for it

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

Also the mac I have to use is at work and I'm getting the network admin to put nodejs v4.2.1 on it to test ad create prebuilt packages

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024
node::Buffer::HasInstance

is returning a true for an object that is not a node::Buffer. Investigating further...

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

HasInstance 1
constructor? Uint8Array
and Uint8Array is not a node::Buffer

Have to study this some more; definitely different between v4.1.1 and 4.2.1. Haven't tested 4.2.0

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

Ah!
https://nodesource.com/blog/cpp-addons-for-nodejs-v4
node::Buffer has been completely rewritten with Uint8Array as its base. [Hope the buffer size is not now limited to int.]

When I do

            var buffer=new Buffer(5*8, "binary");
            buffer.type=H5Type.H5T_NATIVE_DOUBLE;
            buffer.writeDoubleLE(1.0, 0);
            buffer.writeDoubleLE(2.0, 8);
            buffer.writeDoubleLE(3.0, 16);
            buffer.writeDoubleLE(4.0, 24);
            buffer.writeDoubleLE(5.0, 32);
            console.dir("Buffer name "+buffer.constructor.name);

it says it's a Buffer; but the same object on the native side will say Uint8Array and also satisfy the HasInstance regardless if constructed by Buffer or Uint8Array. v4.2.1 has the change; v4.1.1 doesn't; building and testing v4.2.0... v4.2.0 has the issue.

I don't know why they change the constructor name going from javascript to native side; going to try to join a discussion on this.

from hdf5.node.

mgwalker avatar mgwalker commented on July 22, 2024

Yeah 4.2.0 is when they upgraded the v8 engine. It was the thing that held up the LTS blessing.

Interesting find. I will follow the node discussion as well.

Thanks all for all your hard work!

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

ugh. unhappy with their take on it. Buffer=Uint8Array now

limits size to int and this is the size of all h5 dataset dimensions multiplied together.

It also is back to dealing with the Externalize which is a one time thing and breaks the multiple use of a buffer and passes memory cleanup to native. Have to think and study. Been wanting to look at streaming which will move multiple buffers along.

One of my design principles is avoid copying things and the Externalize just to get to the data causes a single use per buffer/object

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

in the case of binaryjs streaming all the way to a browser page, native streaming of chunks/sections was on my list to tackle. As a step I had implemented regions access of a dataset; foreseeing numerous applications won't want to send a terabyte to the browser anyway.

need to move on from 4.1.x because of other broken things such as Buffer.concat

from hdf5.node.

rimmartin avatar rimmartin commented on July 22, 2024

Currently making Uint8Array work on the native side for Buffer and be as interchangeable at this point as possible. Hope your dataset size is within the size limit on Uint8Array.

Then going to develop streams

from hdf5.node.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.