xtuc / webassemblyjs Goto Github PK

View Code? Open in Web Editor NEW

777.0 10.0 90.0 6.75 MB

Toolchain for WebAssembly

Home Page: https://webassembly.js.org

License: MIT License

JavaScript 99.37% Makefile 0.21% WebAssembly 0.21% CSS 0.01% Shell 0.16% C 0.04%

webassembly interpreter javascript ast compiler parser polyfill wasm wast

webassemblyjs's Introduction

webassemblyjs

Toolchain for WebAssembly

See WebAssembly.js.org for more information.

Packages

ast - AST utils for webassemblyjs
cli - Toolbox for WebAssembly
dce - Eliminate unused functions in your WASM binary
eslint-plugin-webassembly - ESLint plugin for WebAssembly.
floating-point-hex-parser - Parser function for floating point hexadecimals.
helper-buffer - Buffer manipulation helpers
helper-fsm - FSM implementation
helper-wasm-bytecode - Constants for the wasm format
helper-wasm-section - Section manipulation helpers
leb128 - LEB128 decoding
repl - WebAssembly REPL.
validation - Module AST validations
wasm-edit - Replace in-place an AST node in the binary.
wasm-gen - WebAssembly binary format printer
wasm-opt - WASM optimizer
wasm-parser - WebAssembly binary format parser
wasm-strip - Strips custom sections (deprecated, use wabt instead)
wast-loader - Webpack loader for WebAssembly text format
wast-parser - WebAssembly text format parser
wast-printer - WebAssembly text format printer
wast-refmt - WAST refmt
webassemblyjs - WebAssembly interpreter, implements the W3C WebAssembly API.

Node's Buffer

Some packages rely on Node's Buffer which isn't available in other environments. We recommend you to add https://github.com/feross/buffer in your building process.

Licence

MIT

webassemblyjs's People

Contributors

Stargazers

Watchers

Forkers

jimbog colineberhardt juanpicado johnsudaar ryyppy chicoxyzzy newtorob neilbryant sendilkumarn ohbarye zhouxingzu christiango bernalrs kustomzone nishant8bits lp249839965 papandreou onlywei menduz timhambourger chenrui2014 void4 eli-black-work turbolent kuba-kubula char0n daxianyu sokra cdetrio alilmq fitzgen sunny-kid aircom-inc cloned-repos jacarte unclebill sensational-code robertomalatesta momesana wangzhengbo openweb-studios-webland bonomali soersoft 0xf013 acrodriguez-livearea mtorp jerrymousez iskyler albertcabello martianboy techidaily hildjj ko-rei hjl19911127 msftenhanceprovenance huanghaoran555 thomasballinger zidingz mitschabaude 592767809 erosmlima kuustudio pyrolistical msduketown ailisp samancoding proxy-m d3lm bpowers renatoalencar phybrackets jemerald83 jeremyellingham bouk little-huang joshklop deongroenewald wwwehr skdziwak tnytown dig-doug codeyu001 lucianojean wikiwong pronebel pi0 clause davelak strogo nickstallman

webassemblyjs's Issues

Prettify .tjs files

Refac AST generation to use Number and String literals builders

Implement missing instructions

Block:

loop
br
br_if
brtable

Bounds checking for number literals in text

We need to ensure that only valid literals are used for each type by comparing the parsed value with the upper and lower bound of the ranges.

Handle i64 arguments/results in host function

According to the spec:

If the signature contains an i64 (as argument or result), the host function immediately throws a TypeError when called.

The reason is that you can't represent > 53 bits integers in JS where WebAssembly allows up to 64 bits.

Capture lost information of number values in JS floats

Since we are interpreting in JS it makes sense to use JS floats for all numbers and their operations wherever possible. However, I think there are a few problems with that:

NaN: There are many different NaN representations containing a payload. I don't think that there is a way to access the payload in JS (or that it is propagated through operations at all). But the WebAssembly spec says that the payloads should propagate through operations according to well-defined rules https://webassembly.github.io/spec/core/exec/numerics.html#aux-nans. I think this is important, because the bits of a NaN could be reinterpreted as an i64 integer for example. The same applies to the sign bit I think.
+-0: It is possible to distinguish +0 and -0 in JS but not with ===. That is why test cases with these values are currently useless, because the testing framework in JS treats them as the same. Again, I think we need to store some extra information about the sign.

I will try to come up with some simple examples and run them through the reference interpreter to make the issue more clear. We dont have enough implemented yet to show the NaN issues, but here is a sample test case with the +-0 issue https://github.com/xtuc/js-webassembly-interpreter/blob/master/test/interpreter/kernel/exec/numeric-instructions.js#L383-L398.

Implement floating point math instructions

See specification here: https://webassembly.github.io/spec/core/exec/numerics.html#op-fneg

Use WABT opcode

Use the op code definitions in https://github.com/WebAssembly/wabt/blob/master/src/opcode.def.

Currently there's no restriction for the number of arguments an instruction can take, we can use the opcode.def to define them.

@babel/code-frame in prod?

Following the discussion from #50 (comment).

Since watf is not available via the WebAssembly JS api, but it's very useful for testing.

Currently @babel/code-frame is in devDeps.

This is a blocker for the next release since our examples uses the watf parser, where the package is required.

I would like to provide multiple builds, but it might be not worth the overhead (bundle size) of the package.

Any thoughts?

Should the the different WATF formats have different AST representations

These two functions are equivalent -

operations written in a 'stack' form:

  (func $add (param f32) (param f32) (result f32)
   (get_local 0)
   (get_local 1)
   (f32.add)
  )

and in a 'function call' form:

  (func $add (param f32) (param f32) (result f32)
   (f32.add
     (get_local 0)
     (get_local 1)
   )
  )

However, our interpreter has to treat each differently. Here's an example that allows unary operations to use the 'function call' form:

ca3b3a3

I think it would be better if the interpreter didn't have to handle this. Should these two forms result in the same AST?

Support memarg load/store instructions

The syntax is described here: https://webassembly.github.io/spec/core/syntax/instructions.html#syntax-memarg.

WAST grammar (spec):

offset: offset=<nat>
align: align=(1|2|4|8|...)
<type>.load((8|16|32)_<sign>)? <offset>? <align>?
<type>.store(8|16|32)? <offset>? <align>?

Example:

(i64.load offset=0x000 align=1 (i32.const 0))

Question on programming style

Looking at the implementation of add, there is quite a bit of repetition:

case 'add': {

  switch (instruction.object) {

  case 'i32': {
    const [c1, c2] = pop2('i32', 'i32');

    pushResult(
      binopi32(c2, c1, '+')
    );

    break;
  }

  case 'i64': {
    const [c1, c2] = pop2('i64', 'i64');

    pushResult(
      binopi64(c2, c1, '+')
    );

    break;
  }

  case 'f32': {
    const [c1, c2] = pop2('f32', 'f32');

    pushResult(
      binopf32(c2, c1, '+')
    );

    break;
  }

  case 'f64': {
    const [c1, c2] = pop2('f64', 'f64');

    pushResult(
      binopf64(c2, c1, '+')
    );

    break;
  }

  default:
    throw new RuntimeError(
      'Unsupported operation ' + instruction.id + ' on ' + instruction.object
    );

  }

  break;
}

The following refactor gives the same functionality (ignoring the unsupported operation path for now):

case 'add': {

  const binopComputer = {
    'i32': binopi32,
    'i64': binopi64,
    'f32': binopf32,
    'f64': binopf64,
  };

  const [c1, c2] = pop2(instruction.object, instruction.object);

  pushResult(
    binopComputer[instruction.object](c2, c1, '+')
  );

  break;

}

Is a more 'long hand' approach being used for reasons of performance? Or is it worth exploring refactoring further?

Floating point hexadecimal format for number literals

https://github.com/WebAssembly/spec/blob/master/test/core/float_literals.wast#L95

Add and run prettier

Following #70 (comment)

Failed to install the repl in global

npm i -g webassembly-interpreter

$ wasm                                                                                                                                                                                     
module.js:471
    throw err;
    ^
Error: Cannot find module 'js-tokens'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/usr/lib/node_modules/webassembly-interpreter/node_modules/@babel/code-frame/lib/index.js:7:41)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)

Implement stack size limit

See discussions in the spec repo WebAssembly/design#1163

The reference interpreter throws call stack exhausted after 300 nested frames.

Remove link to implementation U32Literal

https://github.com/xtuc/js-webassembly-interpreter/blob/fb847512f0aa0e0a2f4e7ed26f7d14014023df76/src/types/AST.js#L5-L7

Make task to compile files on change like babel

Implement basic instructions linting

We can use static analysis to emit an early error when we are sure that it will lead to an invalid result at runtime.

Example:

(func (result i32)
  (i64.const 1)
)

We know that the result of i64.const is an i64 and the result of the func will be i64, so the result is not correct.

Implement condition instructions

See specification here: https://webassembly.github.io/spec/core/exec/numerics.html#op-ieq

incorrect implementation of i32.div_s

The current implementation treats div_s and div_u as the same operation. However, as per the specification, div_s uses a 'signed' interpretation of the values:

https://webassembly.github.io/spec/core/exec/numerics.html#aux-signed

this has an impact for numbers where the most significant bit is '1'.

To illustrate:

unsigned

(module
  (func $test (param i32) (param i32) (result i32)
   (get_local 0)
   (get_local 1)
   (i32.div_u)
  )
  (export "test" (func $test))
)

instance.exports.test(0xffffff00, 0x2);

2147483520

signed

(module
  (func $test (param i32) (param i32) (result i32)
   (get_local 0)
   (get_local 1)
   (i32.div_s)
  )
  (export "test" (func $test))
)

instance.exports.test(0xffffff00, 0x2);

-128

Decimal exponent number literal format

https://github.com/WebAssembly/spec/blob/master/test/core/float_literals.wast#L79.

I'm not sure how much of this already works due to parseFloat. The integer types need to allow it too so that definitely requires an additional case.

Parse error for nested instructions in br_if

Following the discussion in https://github.com/xtuc/js-webassembly-interpreter/pull/52/files/939a7c3cb91aa908cd4b8b683fd4d847424e3082#r159416366

Currently some code is duplicated, nested instructions in br_if should use the same code than for parsing blocks.

[WAST] support multiple params in function signature

Example: (func $name (param i32 i32 i32) (result i32))

Currently parses it as having only one i32 as param.

[exec] loopInstruction and ifInstruction runtime

Implement if and loop

[WAST] support func import instruction

AST node are already created https://github.com/xtuc/js-webassembly-interpreter/blob/ea4510c4fa1df116134aa868ec500ad79e3d336b/src/types/AST.js#L133-L138

i64 arithmetic on JS?

WebAssembly has a 64 bits float and integer type.

Even if we won't have interactions with >53bits integers between JavaScript code and the WebAssembly runtime (#21).

Our interpreter is still written in JavaScript and we need to handle it somehow, here's a few solutions:

Emulate 64 bits arithmetic over 32 bits (what asm.js does)
- We probably need a AST pass to tranform all i64 into i32.
Compile your program with the 32 bits mode? Which adds a restriction to use our project.

asm.js does it already, here's are a few notes that I found:

Unity in particular mentioned that emulation of 64-bit arithmetic in asm.js is really really slow

Implement `trunc` instructions

See specification here: https://webassembly.github.io/spec/core/exec/numerics.html#op-trunc-u

WASM code gen

In https://github.com/xtuc/inline-wast we use https://github.com/WebAssembly/wabt emscripten/JS version which has such a tool but there's also a lot of downside to it (size and slow).

It would be great to implement a WASM code generator (from our AST).

Fix flow type checking

br_if doesn't work for i64

br_if uses isZero, which itself depends on i32 specific logic and being able to use equality on the value property of StackLocal

Something to fix when adding a wrapper around numeric types for #87 and #85

Correct WATF grammar

Example:

  (func (
    (call 1)
  ))

Should be

  (func
    (call 1)
  )

WAST definition:

func:    ( func <name>? <func_sig> <local>* <instr>* )
instr:
  <expr>
  <op>                                                              ;; = (<op>)
  block <name>? <block_sig> <instr>* end <name>?                    ;; = (block <name>? <block_sig> <instr>*)
  loop <name>? <block_sig> <instr>* end <name>?                     ;; = (loop <name>? <block_sig> <instr>*)
  if <name>? <block_sig> <instr>* end <name>?                       ;; = (if <name>? <block_sig> (then <instr>*))
  if <name>? <block_sig> <instr>* else <name>? <instr>* end <name>? ;; = (if <name>? <block_sig> (then <instr>*) (else <instr>*))

op:
  unreachable
  nop
  br <var>
...

And WATF definitions is here

unit test issues

I've been digging into the code for this project, with a view to contributing. The first thing I did was clone and run the tests, unfortunately ~60 are failing for me - looks like all the tests that assert errors are thrown.

I've not quite got to the bottom of it, but have found a few issues:

From looking at the API docs, the second argument in assert.throws should be a regex, not a string, as currently used in the tests. If you provide a string as the second argument, it will assert that an error is thrown, but not validate the message.
If you extend Error, additional logic is required in order to propagate the message property as detailed in this StackOverflow question

It still doesn't explain why the tests fail for me but pass on Travis, however, I thought it was worth raising these.

One another note, do you have any objections to replacing your Makefile with npm run scripts? It is a more standard tooling for JS projects, and also allows you to do things like pass args to scripts, e.g.

npm test -- --inspect --debug-brk

This runs the tests with the debugger, allowing you to add breakpoints etc ...

Implement binary instructions

See specification here: https://webassembly.github.io/spec/core/exec/numerics.html#op-ictz

binary operations are the wrong way round?

From the tests:

https://github.com/xtuc/js-webassembly-interpreter/blob/master/test/interpreter/kernel/exec/numeric-instructions.js#L217

    {
      name: "f32.div",

      args: [{ value: 2.0, type: "f32" }, { value: 10.0, type: "f32" }],

      code: [
        t.instruction("get_local", [t.numberLiteral(0)]),
        t.instruction("get_local", [t.numberLiteral(1)]),
        t.objectInstruction("div", "f32")
      ],

      resEqual: 5.0
    },

My reading is that this is equivalent to:

(module
  (func $test (param f32) (param f32) (result f32)
   (get_local 0)
   (get_local 1)
   (f32.div)
  )
  (export "test" (func $test))
)

instance.exports.test(2, 10)

Which gives 0.2

i.e. it looks like the arguments to the binary operations are the wrong way round.

Caveat in call instruction for WAST

The call instruction seems different in the WAST format than the WASM format, it's unspecified tho.

In this example:

(module
  (func $b (export "main")
    (call $b)
  )
)

The ident b is not found. In the WASM specification, they refer to the moduleinst.funcaddrs, I would like to unify this behavior and allow a pointer lookup in WAST.

I'm thinking about an AST pass that replace the call $b by call index.

Remove the VM/sandbox in benchmarks

Following nodejs/node#17884

Running the benchmark in a sandbox decreases the performance and leads to incorrect results.

Implement memory operation instructions

Load:

Store:

See specification here https://webassembly.github.io/spec/core/exec/instructions.html#exec-load.

This depends on #29

Implement all instructions

Make type checking great again

Iteration of #84

Func result should be an array

Here https://github.com/xtuc/js-webassembly-interpreter/blob/c33f2e801002c9abfbca0a69d5e2820a726432b9/src/types/AST.js#L59 (and AST node builder)

Has impact in the WAST grammar and WASM decoder.

Implement `convert` instructions

Explicit conversion:

Implicit conversion:

f32.demote
f64.promote

See specification here: https://webassembly.github.io/spec/core/exec/numerics.html#op-convert-s

Fix our links to the specs

The whole doc has been moved in all our links are broken now.

Implement integer operations

https://webassembly.github.io/spec/core/exec/numerics.html#integer-operations

Pass our first spec test

https://github.com/xtuc/js-webassembly-interpreter/blob/160e8c8528fe7c7316b143f7520da9b5a6d5530b/test/spec.js#L8

Support NaN and Infinity text constants

The text format supports the following literals for floating point numbers

Spec is here https://webassembly.github.io/spec/core/text/values.html#text-float

Remove redundant tests when the spec tests are implemented

Starting with #100 we are using the official webassembly test suite as a way to integration test our code.

The eventual aim, via #98, is to run the spec tests directly agains this codebase. At this point, the integration tests, which mirror the spec tests, are redundant and can be deleted.

Implement comments

A comment can either be a line comment, started with a double semicolon ‘;;’‘;;’ and extending to the end of the line, or a block comment, enclosed in delimiters ‘(;’…‘;)’‘(;’…‘;)’. Block comments can be nested.

Spec here

Inf identifier node can collide

Has been introduced by #93.

Example

(f32.const inf)

and

(f32.const $inf)

produces the same AST. I decided to use an Identifier node because it's also done this way in JavaScript and given that it refers a global constant it make sense.

In JavaScript you can't redeclare the Infinity constant, I don't know how it should behave in WebAssembly.

Also note that Infinity is stringified in JSON as null, we can't use it in the actual Number type.

Implement `reinterpret` instructions

i32.reinterpret
i64.reinterpret
f32.reinterpret
f64.reinterpret

See specification here: https://webassembly.github.io/spec/core/exec/numerics.html#op-reinterpret

xtuc / webassemblyjs Goto Github PK

webassemblyjs's Introduction

webassemblyjs

Packages

Node's Buffer

Licence

webassemblyjs's People

Contributors

Stargazers

Watchers

Forkers

webassemblyjs's Issues

Recommend Projects

Recommend Topics

Recommend Org