lightpanda-io / zig-js-runtime Goto Github PK

Generate native APIs as Javascript objects and functions available in the context globals.

The default Javascript engine is v8 (Chromium). It's:

• fast
• easy to embed
• a defacto standard

But it's also fat (16 Mo added to the binary, at least) and complex.

Supporting other Javascript engine could allow integration in constrained environment (WASM, IoT, embed). And we could also make some performance vs consumption tests with v8.

Candidates are:

• QuickJS (Fabrice Bellard), update to date, small, easy to embed, not bad performance for an interpreter.
• SpiderMonkey (Firefox), other defacto standard. Seems to be quite small as it was successfully embed in WASM (1.4 Mo).
• JavascriptCore (Safari), other defacto standard. Was successfully embeded by bun.sh, who claims it has better performance than v8.

Let's say a JS function must accept a variadic arguments of Node as input, ...nodes.

We have 2 solutions to handle this on the native API:

1. use the built-in slice, ie. []Node
2. use a custom generated type, VariadicNodes generated through a provided helper function Variadic(comptime T: type) type

The slice solution is easier to use as you don't have to generate the Variadic<Type>. But it's also less explicit and can lead to some misunderstanding:

• it's not a JS Array as the semantics might make you think (see #52)
• the Native -> JS translation will not be possible, therefore returning []Node will be forbidden, which is less obvious than forbid to return a VariadicNode

JS -> Native
Native -> JS

Some complex types:

• Array
• Object
• Function

There is 2 types of JS Array:

• Array which is just an object list of heterogeneous types with some additional methods
• TypedArray subclasses for numerical values (Int<x>Array, Uint<x>Array, Float<x>Array, BigInt<x>Array and BigUint<x>Array)

It's not easy to translate those to zig native types (eg. slice, tuple) as:

• we don't know the shape and the length for Array
• we do know the type for TypedArray but not all native types have a corresponding JS typed array, eg. we will have to forbid []bool or []<UserType> as they don't have a TypedArray counterpart

As they are JS native types, Array and TypedArray exists in the JS engine native API (eg. v8.Array), with all their public properties and methods like Length().
So maybe the more logical solution is just to expose those types, ie. define a jsruntime.Array type that will use underneath v8.Array (or quickjs.Array).

https://bellard.org/quickjs/

Main advantages:

• small (binary size)
• light (ressources)
• easy to embed (in C)
• good coverage of JS standard (ES2020), including modules
• quite fast for a simple interpreter (2x times slower than v8 without JIT, QuickJS Benchmark)

It does not support ECMA402 (Internationalisation APIs).

The idea is to use QuickJS as the default JS engine, at least for the open source version.

We store right now on the reflect structures some fields which are comptime anyway and therefore could be either replace by the builtin function or simple methods, making reflect structures lighter without performance cost.

As tigerbeetle IO loop does not use zig async primitives (no suspend/resume or async/await) we sould be able to use the default self-hosted compiler (stage2).

We "just" need to adapt the lib.

iterable
read-only
etc...

see Object.getOwnPropertyDescriptors(obj)

JS -> Native
Native -> JS

The idea for primitive types to be as natural as possible, ie. translating native primitive types to JS primitive types.

JS primitive types:

• Boolean
• Null
• Undefined
• Number
• BigInt
• String
• Symbol

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#primitive_values

• f16 to f64
• i8 to i32
• u8 to u32
• i64 (Number)
• u64 (Number)
• i64 (BigInt)
• u64 (BigInt)
• bool
• string (list of bytes)
• undefined (void)
• null

Problem

When returning a user-defined object, a new JS object is always instantiated to hold the corresponding native object.
see. https://github.com/Browsercore/jsruntime-lib/blob/main/src/engines/v8/generate.zig#L291

How to reproduce

Add the following tests in types_native_test.zig:

car.brand === car.brand
car.brandPtr === car.brandPtr

Actual result

Both tests are false.

Expected result

In my opinion car.brand === car.brand could be false, as the getter returns a value and not a pointer. I'm open on this.
But car.brandPtr === car.brandPtr, who returns a pointer, should be true.

We need to handle JS functions prototype, knowing that Zig does not support inheritance and neither embedding struct.

The main goal of Browsercore is too build a small and efficient headless browser.

But, as the project develop, we can see we could built other stuff with it:

• a general JS runtime, like NodeJS, Deno or Bun, with native functions support (like NodeAPI)
• a WASM/JS serverless runtime, like Cloudflare workers or Fastly, with WASI support (and native function for JS)
• a server side rendering (SSR) library, with WebAPIs support
• a frontend test rendering library, with WebAPIs support
• an application JS runtime library, easily embedable, for any other kind of project

To do that we need at least the following:

1. Move browsercore-engine to a dedicated library
2. Allow a generic way to pass native functions to the API generator (tip: using AST)
3. Provide an external API to manipulate the engine in other languages
4. Support additional JS engines to make embeding easier (see #2)
5. Provide pre-built shared library including the JS engine (if it does not contradict with protect open sourcing the project)

Some Web APIs says “This parameter is not optional, but can be set to null”.

jsruntime does not support this for now:

• a nullable parameter (ie. ?<Type>) is translated as an optional parameter
• and if you remove the nullable modifier (ie. <Type>) a null value is not allowed

The easiest way to handle that would be to keep ?<Type> for optional parameter and create custom type for mandatory nullable parameter <NullableType>, eg. NullableNumber or NullableNode.

It’s a bit verbose but we can use a comptime function generator to help with that.

Maybe another solution is possible.

Source : Blog de Mitchell Hashimoto

The main goal of Browsercore is too build a small and efficient headless browser.

But, as the project develop, we can see we could built other stuff with it:

• a general JS runtime, like NodeJS, Deno or Bun, with native functions support (like NodeAPI)
• a WASM/JS serverless runtime, like Cloudflare workers or Fastly, with WASI support (and native function for JS)
• a server side rendering (SSR) library, with WebAPIs support
• a frontend test rendering library, with WebAPIs support
• an application JS runtime library, easily embedable, for any other kind of project

To do that we need at least the following:

1. Move browsercore-engine to a dedicated library
2. Allow a generic way to pass native functions to the API generator (tip: using AST)
3. Provide an external API to manipulate the engine in other languages
4. Support additional JS engines to make embeding easier (see Browsercore/browsercore-zig-jsengine: Issue Browsercore/jsruntime-lib#71)
5. Provide pre-built shared library including the JS engine (if it does not contradict with protect open sourcing the project)

https://github.com/mitchellh/libxev

libxev supporte WASI

Since zig-0.11 merge it seems we have a regression on re-allocations

zig-0.10 tag (2023-09-01_23-33_fba2064_main.txt)

  +--------------------------------------------------------------------------------------------+
  | Benchmark jsengine 🚀 (~= 100 iters)                                                       |
  +--------------------------------------------------------------------------------------------+
  | FUNCTION        | DURATION (per iter) | ALLOCATIONS (nb) | RE-ALLOCATIONS (nb) | HEAP SIZE |
  +--------------------------------------------------------------------------------------------+
  | With Isolate    |              619us  |               3  |                  2  |     72kb  |
  +--------------------------------------------------------------------------------------------+
  | Without Isolate |              259us  |               2  |                  1  |     24kb  |
  +--------------------------------------------------------------------------------------------+

zig-0.11 tag

  +--------------------------------------------------------------------------------------------+
  | Benchmark jsengine 🚀 (~= 100 iters)                                                       |
  +--------------------------------------------------------------------------------------------+
  | FUNCTION        | DURATION (per iter) | ALLOCATIONS (nb) | RE-ALLOCATIONS (nb) | HEAP SIZE |
  +--------------------------------------------------------------------------------------------+
  | With Isolate    |              624us  |               3  |               1039  |     72kb  |
  +--------------------------------------------------------------------------------------------+
  | Without Isolate |              262us  |               2  |                932  |     24kb  |
  +--------------------------------------------------------------------------------------------+

The default Javascript engine is v8 (Chromium). It's:

• fast
• easy to embed
• a defacto standard

But it's also fat (16 Mo added to the binary, at least) and complex.

Supporting other Javascript engine could allow integration in constrained environment (WASM, IoT, embed). And we could also make some performance vs consumption tests with v8.

Candidates are:

• QuickJS (Fabrice Bellard), update to date, small, easy to embed, not bad performance for an interpreter.
• SpiderMonkey (Firefox), other defacto standard. Seems to be quite small as it was successfully embed in WASM (1.4 Mo).
• JavascriptCore (Safari), other defacto standard. Was successfully embeded by bun.sh, who claims it has better performance than v8.

jsruntime-lib uses C lib : https://github.com/antirez/linenoise
which can be replaced by the zig port https://github.com/joachimschmidt557/linenoize

Only in public functions/structs

Relates to #56

Should we include i18n support (icu dependancy) ?

https://v8.dev/docs/build-gn#gn

Investigate if we can reuse the JS lib for completions from QuickJS

For example in document.createElement() we can return either any type of HTMLElement + HTMLUnknownElement.

Throw exceptions when the user input is invalid.

• TypeError: Illegal constructor
• TypeError: At least x argument required... (constructor)
• TypeError: At least x argument required... (method)

For now I close this ticket and create one to investigate custom exceptions types and wrong parameters types.
https://trello.com/c/gE6iL2hz

We can see custom exception types that are not part of v8 API (eg. Uncaught DOMException: instead of Uncaught xError).

How can we set thoses?

Right now I can not find any example of generic exception for wrong type arguments (TypeError?).
For example document.createElement(1) throws a custom DOMException with function implementation detail error.

Maybe there is no generic exception for wrong type arguments?

• Le native object a déjà été créé (par le parser, par une fonction d'un autre object, par tout autre moyen).
• Cela concerne a priori les fonctions method et getter (les autres ne peuvent pas renvoyer d'object natif)
• A priori pas besoin d'iterer sur l'object natif, puisque tous les attributs sont gérés par des fonctions (getter)

Il faut :

• Instancier un object JS correspondant via l'object template du Type du native object
• Bind l'object JS instancié au native object

Questions :

• Faut-il ajouter le native object au store ? Du coup se pose la question de la gestion de la mémoire (si on l'ajoute au store qui gère la mémoire, le store ou la fonction native initiale ?)
• Que faire si l'objet JS a déjà été créé (par exemple en cas d'appel multiple à la même method/getter) ? Faut-il créér un nouvel object JS ou renvoyer l'existant ? A vérifier dans Chrome
• Faut-il créér un permanent handle pour l'object JS ? La doc de v8 semble dire que c'est ce que Chrome fait pour le DOM, à vérifier. Si oui doit-il être weak ?

La 1ère étape est de s'assurer lors du reflect (comptime) que les types natifs en retour de fonction ont bien été eux-mêmes générés.
Puis à runtime il faut lors de nativeToJS récupérer le template associé au type généré pour pouvoir instancier l'objet en JS.

Il faut notamment vérifier les dépendances:

• v8
• lexbor
• Tigerbeetle IO event loop
• linenoise

2 cas à vérifier :

• Un object JS instancié côté JS est GC
• Un object JS instancié côté Natif est GC

Pour v8 il faut regarder le handle Permanent et son callback optionel SetWeak.

See https://trello.com/c/oESB3x7P

For example Uncaught DOMException:

https://trello.com/c/6mRMACUl/11-js-exceptions

Example: window.setTimeout

S’inspirer de ce qui est fait entre C et C++.