ggoodman / velcro Goto Github PK

@alexswan10k you were asking about the possibility of including files other than .js and .json in Velcro.

This is something I do want to support but having tried the webpack loader approach in the past, I'm not sure that I want to go that route again. Webpack's loader ecosystem is heavily invested in a world where access to the file-system is taken for granted, which conflicts with Velcro's philosophical direction.

I think an approach we can consider right now is providing a mechanism to inject different _Parser_s.

There is already a layer of indirection where we could support pluggable parser plugins:

Right now, the contract is that any transformations of the underlying files need to be passed back to Velcro as arrays of CodeChanges:

I'm not convinced that this is an approach that will continue to meet the different objectives of Velcro. I had decided to make the code changes raw json so that, for example, parsing could be delegated to a Worker. I think that's probably a useful constraint to keep in mind: the contract between a parser plugin and Velcro's bundler should be encodable in JSON.

My current thinking is along the lines below. There should be a conceptual difference between parsing and generating.

• parsers: something that takes raw content and produces dependencies and basic transforms to produce an intermediate format.
• generators (straw-man name): something that understands the intermediate format and is able to produce code in the desired target format (such as CommonJS, like we do now).

I don't know what the intermediate format should be. For example, Rollup uses ESM but that means introducing a CJS -> ESM transform which is quite a project in itself.

Me again, sorry!

So I have been experimenting with some non-trivial packages and am hitting a couple of issues:
When using "react-alice-carousel" for example, I get a stack overflow when running the bundle. I am pretty sure it is due to two files pointing to eachother, and the way the runner does not cache the runtime result of the reference.

https://cdn.jsdelivr.net/npm/[email protected]/lib/utils/animation.js
https://cdn.jsdelivr.net/npm/[email protected]/lib/utils/index.js

Unfortunately I still cannot get the tests running locally or i would be more than happy to dig right in and have a look myself. Is it perhaps worth noting on the readme.md the exact build steps required and any external dependencies required from a git clean -fdx?

I have had a go at putting together a test, but I am massively guessing without being able to run:

  it('will run more complex package using react-dom/server', async () => {
    const code = `
      const React = require('react');
      const ReactDOMServer = require('react-dom/server');
      const Carousel = require("react-alice-carousel").default;

      const comp = React.createElement(Carousel, null, 'hello world');

      module.exports = ReactDOMServer.renderToString(comp);
    `;
    const result = await execute(code, {
      dependencies: {
        react: '^16.13.1',
        'react-dom': '^16.13.1',
        'react-alice-carousel': '1.18.0'
      },
      readUrl,
      nodeEnv: 'production',
    });

    expect(result).toEqual('<h1 data-reactroot=""> something </h1>');
  });

There is also another problem here that probably warrants its own issue, but I wanted to discuss before making waves: non js or json files. Many react component packages import css files. I know previously it was mentioned you didn't want to commit too much to the webpack ecosystem, only loaders do solve the specific problem of integrating custom content files into a bundle. I understand if this interface is hard to make fit, but perhaps a plugin system to do this sort of thing might be useful? I actually need more control than a basic css loader anyway so would be happy to handroll something here myself if need be. At the moment these parsers are hard coded in graphBuilder:getParserForUri.

What are thoughts on ESModules and how they might fit into Velcro?

I appreciate that this has been the future for 10 years now, and because of the lack of bare import specifiers, they have been mostly useless.

I was chatting to one of the guys over at bit.dev, and he mentioned that their package CDN will be based around esmodules as they have somewhat more control over the output process. He also mentioned the notion of import maps which may well be the missing link required.

Hi,

Just trying to assemble the bundler for a project alongside the runtime (using the published npm packages), but I am hitting a few bumps with respect to their being inconsistencies between the expected resolverHost for @velcro/runtime and the actual resolverHost in @velcro/resolver-host-compound. (isCacheable seems to be missing in newer).

Looking at the versions, it does appear that @velcro/runtime is the only package that hasn't been updated, so perhaps the types are just out of sync and the core runtime needs releasing? If not perhaps I am assembling these packages incorrectly, in which case perhaps you could point me in the right direction?

While I am at it, I am also not 100% sure what the best approach is for substituting real packages with the runtime.

Currently I am doing something along the lines of:

`
const memoryHost = new Velcro.ResolverHostMemory(
initialFiles,
"compound_host_with_cache"
);
const resolverHostUnpkg = new VelcroResolverHostUnpkg.ResolverHostUnpkg();
const resolverHost = new VelcroResolverHostCompound.ResolverHostCompound({
"https://unpkg.com/": resolverHostUnpkg as any,
[memoryHost.urlFromPath("/").href]: memoryHost
});

const runtime = Velcro.createRuntime({
cache: _cache,
injectGlobal: Velcro.injectGlobalFromUnpkg,
resolveBareModule: async (a, b, name, d) => {
if (name === "react") return memoryHost.urlFromPath("/react").href;
if (name === "react-dom")
return memoryHost.urlFromPath("/react-dom").href;
if (name === "prop-types")
return memoryHost.urlFromPath("/prop-types").href;

  const res = await Velcro.resolveBareModuleToUnpkg(a, b, name, d);
  return res;
},
resolverHost, 
rules: [
   ...
    ]
  }

]

});
`

And then after assembling, I can do the following:

runtime.set(memoryHost.urlFromPath("/react").href, React); runtime.set(memoryHost.urlFromPath("/react-dom").href, ReactDom);

Although this might work, it doesn't seem to be consistent with the bundler api, and also it doesn't work for underlying webpack plugins like css-loader or style-loader. It would be nice to do this because stuff like sass-loader has tons of dependencies and is slow on first pass.

Any thoughts?

Error:

react-dom.development.js:11865 Uncaught GraphBuildError: Graph building failed with errors:
No such directory memory:/ at
 GraphBuilder.doAddUnresolvedUri:memory:/index.js
    at GraphBuilder.buildGraph (http://localhost:8080/public/148.js:1884:19)

Code sample as follows:

import * as Velcro from "@velcro/runner";
...
    const code = `
      console.log("I am a trivial sample");
    `;
    Velcro.execute(code, {
      readUrl: u => {
        console.log("fetching", u)
        return fetch(u).then(u => u.arrayBuffer())
      },
      external: (x) => {
        return false;
      },
      injectModules: {
      },
      dependencies: {},
      nodeEnv: "production"
    })

As discussed this also fails upon the (sandbox demo)[https://plnkr.co/edit/sDnzv3QEv6wudM9P].

Original findings:

I have had a go at trying to hook all this up, with little avail unfortunately. I keep hitting the same error as seen on the sandbox, after having tried various combinations of the new runner or composing these constructs myself.

I am pretty sure I have narrowed it down to the listEntries first line in memoryStrategy.ts

The offending line:

Uri.ensureTrailingSlash(uri).fsPath
where fsPath seems to be returning a backslash rather than a forward slash, which is then fed into getEntryAtPath, which then resolves a parent as undefined, as no segment/child can be found matching "" obviously!

I cant seem to locate fsPath on Uri or the builtin URL prototype, so perhaps you can help me work out where this is coming from?

I also noticed an inconsistency between Uri.parse('memory:///') and the generated Uri("memory:/directory/path"). Not sure if this is a problem though, i just cant rule it out yet either.

Thought I would just post a new issue for this to iron everything out, as I think there are still some steps missing.

Steps so far:
npm install lerna -g
(I tried npx but it just kept blowing up on "could not find lerna@latest)

goto root
git clean -fdx
npm install
lerna bootstrap
npm run test

Tests run, but all fail, following error:
ReferenceError: describe is not defined.

Have tried:
https://stackoverflow.com/questions/55807824/describe-is-not-defined-when-installing-jest
and installing:
https://www.npmjs.com/package/eslint-plugin-jest
but no avail.

Any thoughts?