Problems with loading the Web Assembly module when using code bundlers (Vite, Webpack) about vizzu-lib HOT 30 CLOSED

Never mind, I have put in the link (got the description from the repo).

from vizzu-lib.

There seem to be two problems:

1. By default the webpack config of CRA does not load .wasm files
2. Even if it did (or one updated the config so it did), it wouldn't load cvizzu.wasm file, as it's not the target of any require or import statements

Potential solutions are:

1. change the import of the wasm in the js file, so webpack (or any other bundler) can pick up the reference to wasm. they cannot do this because of the reliance on import.meta.url to construct the require path of the wasm. I assume this is needed for non-npm uses, e.g. when served from a cdn, so this is not straightforward
2. referencing the wasm from the application itself without any real use just so webpack reaches it. In this scenario the output path would need to be fixed to the one the .js file looks for

from vizzu-lib.

I'm having a similar issue using Vite. The request to wasm returns 404 as it makes the request to http://localhost/cvizzu.wasm.

This is also not something wrong with the lib, more so needing some experimentation and a "how to use with x". I am still trying to figure it out and will write up a guide when I do.

from vizzu-lib.

This has left me quite frustrated. Apps created with create-react-app cannot seem to elegantly solve this problem, so the most straightforward solution is adding a step to one's build process which moves the .wasm file into the public folder (which is copied and served as-is after bundling).

Create-react-app now uses Webpack 5 which might solve this, as it has a built-in WASM loader (I've been trying to solve it using webpack 4). At some point I'll give that a try to see if it works, but for now I just admitted defeat and copied the wasm file to my static folder.

In general, I think it would be really valuable to set up a working example with at least one bundler and document it

from vizzu-lib.

I gave it a go on NextJS which uses webpack 5.
It gives me the following error:

Updating vizzu's package.json (adding the type: "module" line to it) solves this error but introduces another

In my understanding the core issue is the usage of BOTH import.meta and require(), which makes the npm package a weird mix of sorta-an-ES6-module-but-not-really.

I believe the issue can be solved in two ways:

1. Avoiding import.meta -> Adding the USE_ES6_IMPORT_META=0 flag to the emscripten build, which would avoid the use of import.meta. This should be the safer solution as it would work with older toolchains, e.g. webpack 4.
2. Avoiding 'require()' -> Adding the WASM_ASYNC_COMPILATION=0 MODULARIZE=1 EXPORT_ES6=1 flags to the build instead, which would export a "proper" ES6 module where import.meta can be used. (Caveat: I'm not sure that this is the right combination of flags, and what the potential side effects may be for other environments)

from vizzu-lib.

I've got a Vite example to go off. Not a bad starting place.

Have to copy the wasm to the public/, but eh.

from vizzu-lib.

yea for now, that's the best course of action. I'm 100% that we'll figure this out so future users won't have a problem

from vizzu-lib.

I've got a Vite example to go off. Not a bad starting place.

Thanks, this is great! I have added a markdown file to the project (and a link to it into the readme) to list 3rd party projects:
https://github.com/vizzuhq/vizzu-lib/blob/main/PROJECTS.md
If you could put a link to your repo along with a description line there, that would be great!

from vizzu-lib.

Unfortunately, USE_ES6_IMPORT_META=0 introduces another problem, this way it tries to load the wasm relative to the HTML document, not to the cvizzu.js.

from vizzu-lib.

Another solution could have been to create a single file (using SINGLE_FILE=1 emscripten flag), but that would increase the package size by ~25% due to the base64 encoding of the wasm file.

from vizzu-lib.

that's a shame. how about the modularization option?

Avoiding 'require()' -> Adding the WASM_ASYNC_COMPILATION=0 MODULARIZE=1 EXPORT_ES6=1 flags to the build instead, which would export a "proper" ES6 module where import.meta can be used. (Caveat: I'm not sure that this is the right combination of flags, and what the potential side effects may be for other environments)

I did some more digging and everyone on webpack 4 is doing the same thing Christopher and I did (explicitly copying the .wasm file to the right place) just with different methods.

from vizzu-lib.

We use the MODULARIZE=1 EXPORT_ES6=1 flags currently, I'm not sure that the WASM_ASYNC_COMPILATION=0 flag would make any difference, but we can give it a try.

from vizzu-lib.

WASM_ASYNC_COMPILATION=0 seems to have no effect on the problematic part of the generated code.
The file begins with the same code:

    var VizzuModule = (function() {
        var _scriptDir = import.meta.url;

There are two other problems with this flag. It won't work with the ASSERTIONS=0 flag which I would like to introduce in the next release, shaving off ~80kb of the generated JS code with it.
There is also a wasm size limit in Chrome which won't allow compilation of bigger wasm modules on the main thread.

from vizzu-lib.

Maybe we could modify the generated code in a post-build step? Would a conditional use of the import.meta object solve the problem? If so, could you create a patch to the generated file which works with CRA/Webpack?

from vizzu-lib.

I have some renewed motivation in trying to resolve this. is there way to get non-minified js out of the generation process?
I think one of the main problems webpack has is that there is no outright require or import statement in the js so "it doesn't know" that the .wasm file is used at all

from vizzu-lib.

I just found this repo, between that and Christopher's solution, I'll come up with something fairly easy to set up. the more I read about this the more I think it can be solved on the Emscripten side

from vizzu-lib.

@korompaiistvan
On this url you can find the build of the most recent development version:
https://vizzu-lib-main.storage.googleapis.com/lib/vizzu.js

from vizzu-lib.

But I'm afraid the emscripten wrapper is still not readable in this version:
https://vizzu-lib-main.storage.googleapis.com/lib/cvizzu.js

If you need a debug build (emscripten generates non-minified code for debug build), then I can produce that for you.

from vizzu-lib.

If you would like to experiment with the emscripten configuration, and build the project, then
https://github.com/vizzuhq/vizzu-lib/blob/main/project/build.md
This is the way! :)

from vizzu-lib.

@korompaiistvan
Thank you for providing the example and the tutorial on how to use Vizzu with React!
https://vizzuhq.github.io/vizzu-react-example/
https://github.com/vizzuhq/vizzu-react-example

from vizzu-lib.

@korompaiistvan
I have made the url of the wasm file configurable in Vizzu:

Vizzu.options({ wasmUrl: './whatever-folder/cvizzu.wasm' });

How much would this help with this bundler issue?

from vizzu-lib.

I have managed to reduce the problem. Now integrating Vizzu with Webpack needs only the steps listed here:
https://github.com/simzer/vizzu-webpack-demo/blob/main/README.md#adding-vizzu-to-a-project-which-using-webpack

from vizzu-lib.

i think that solution in the readme is a decent compromise. not as elegant as it would be if it worked automatically, but basically the next best thing

from vizzu-lib.

@korompaiistvan
Thanks, we will put this into our tutorial with some explanation then.
I will also try to update https://github.com/vizzuhq/vizzu-react-example and send a PR for you if you'd be willing to do a review.

from vizzu-lib.

I was unable to update the react example.
If I use the solution from the webpack-demo, it won't work with npm start.
However, I was able to build a correctly working version with npm run build.

from vizzu-lib.

that's probably got to do something with how CRA deals with webpack config (badly).
did you modify the config overrides js file or create a separate webpack config file?

also as a note: webpack devserver (npm start) and build works slightly differently so there might be a difference there as well.

I'm also wondering about postprocessing your npm package instead so it includes this explicit import but im not sure how exactly

from vizzu-lib.

I have modified the config-override.js file.

from vizzu-lib.

I have tried devserver (webpack serve) in the webpack-demo repo, and it works there.

from vizzu-lib.

Well, maybe the problem was that the cvizzu.wasm and vizzu.min.js were overwritten by a local build. Now that the necessary changes are released in 0.4.5, npm start is working.

from vizzu-lib.

PR sent to https://github.com/vizzuhq/vizzu-react-example. Closing this issue.

from vizzu-lib.