Overhaul ufuzz.js for ES6 code about terser HOT 10 OPEN

By all means extend ufuzz to support ES6+. I'd be happy to critique it.

Adding things like const, let, classes, class expressions and arrow functions would be fairly straightforward. Destructuring - in all its various forms (declaration, default values, in function parameters and destructuring assignment) could be interesting. Correctly exercising generators would be brutal, for example. The trick is to always generate valid code that produces a result - not as easy as one might think. The other think to watch out for is the frequency of generation of a specific ES6 feature so that a good representative mix of functionality is produced.

The best way to start would be to run uglifyjs -h ast and knock off each node type one at a time.

I'd suggest to avoid generating async/await code until the node VM sandbox can handle it. We currently have no way to run asynchronous tests due to the timeout problem. All tests are synchronous for this reason.

OT - Would it be possible to put such non-trivial work in pull requests? It's easier to review that way.

from terser.

Expanding on the "representative mix of functionality" remark, we don't want to have destructuring in more than 5% of generated functions, for example. If a certain node type is too frequently generated it will cause various optimizations to be disabled such that it will not exercise the Compressor very well. I suppose a code coverage tool could gauge the effectiveness of ufuzz testing.

from terser.

BTW, reducing failing ufuzz test cases is somewhat of a black art. It takes me 30 minutes on average per test case to whittle it down to 5 - 10 lines to see whether it's a legitimate minify code generation failure or a false positive. That's why care has to be taken in ufuzz design to always try to generate valid code producing a valid result to reduce the false positive frequency.

from terser.

I'm having fun and found a parser error already :) looks like you can't have a class or function as the body of a for loop without a block.

from terser.

Would there be any merit to having an --ecma flag similar to acorn has to optionally restrict fuzzing to a certain ecma level?

from terser.

I think that flag might make the script a lot more complex, when it comes to choosing what to output next you'd have to have two lists of what's possible.

In other news, it looks like since classes make the code inside of them strict mode automatically, ufuzz is outputting a lot of errors now I've added them. This might be a while.

from terser.

There's a strict flag - maybe that can help.

      case '--use-strict':
        use_strict = true;

from terser.

Thanks!

from terser.

I made a fuzzer that's able to catch parsing errors in ES6 code, #198. The idea comes from https://github.com/Rich-Harris/butternut/blob/master/test/fuzz-test.js.

from terser.

I like it

from terser.