cdiggins / myna-parser Goto Github PK

View Code? Open in Web Editor NEW

79.0 5.0 16.0 1.68 MB

Myna Parsing Library

Home Page: https://cdiggins.github.io/myna-parser

License: MIT License

JavaScript 87.66% TypeScript 10.76% CSS 1.58%

parsing-library syntax-tree parser-combinators parser-api peg recursive-descent recursive-descent-parser

myna-parser's Introduction

Myna Parsing Library

Home Page | QUnit Tests | Source Code

Myna is an efficient and easy to use parsing library for JavaScript written using TypeScript 2.0 which targets ECMAScript 5.1.

Unlike several popular parsing libraries (e.g. Jison, PEG.js, nearley, and ANTLR) Myna is an API, not a code generation tool, which makes it easier for programmers to write, debug, and maintain their parsing algorithms. This makes Myna closest to Parsimmon and Chevrotain. Myna has no dependencies, you can just download myna.js and start using it immediately.

There are several example tools that demonstrate how to use Myna parsers and a number of sample grammars shipped with Myna that you can use or modify as needed.

Getting Started

You can either download the latest myna.js file via GitHub or via Unpkg and start using it in your project, or you can install Myna from npm.

Using Myna

Below is an example of how to use Myna from Node.JS in a single self-contained example:

    // Reference the Myna module
    var m = require('myna-parser');

    // Construct a grammar object 
    var g = new function() 
    {
        this.textdata   = m.notChar('\n\r"' + delimiter);    
        this.quoted     = m.doubleQuoted(m.notChar('"').or('""').zeroOrMore);
        this.field      = this.textdata.or(this.quoted).zeroOrMore.ast;
        this.record     = this.field.delimited(delimiter).ast;
        this.file       = this.record.delimited(m.newLine).ast;   
    }

    // Let consumers of the Myna module access 
    m.registerGrammar("csv", g, g.file);

    // Get the parser 
    var parser = m.parsers.csv; 
    
    // Parse some input and print the AST
    var input = 'a,1,"hello"\nb,2,"goodbye"';
    console.log(parse(input));

Only rules that are defined with the .ast property will create nodes in the output parse tree. This saves the work of having to convert from a Concrete Syntax Tree (CST) to an Abstract Syntax Tree (AST).

Myna Source Code and Dependencies

The Myna library is written in TypeScript 2.0 and is contained in one file myna.ts. The generated Myna JavaScript file that you would include in your project is myna.js.

Myna has no run-time dependencies. The Myna module, and the grammars are designed to be able to be run from the browser or from Node.JS.

Building Myna

The myna.js library is generated from the myna.ts source file using the TypeScript 2.0 compiler (tsc). I use npm as my build tool and task runner. I would welcome submissions for making my package cross platform. I use Visual Studio Code as my development environment.

The steps I use to making a patch and re-building/publishing Myna are:

npm run full - This will build the TypeScript files, run tests, update docs, create a minified version of the JS file.
git add . - Add the files to the git working state
git commit -m "message" - Create a git commit
npm version patch - Create a patch (will create a secondary Git commit)
git push -u Push the commits to the server
npm publish - Publish the Node package

Myna Tests

There are over a 1000 unit tests written using QUnit. There are also individual test files for each example tool, which you can run as a suite using node tests\test_all_tools.js.

Minification

For convenience I am providing a minified/obfuscated version dist/myna.min.js that is being generated with uglify.js.

Bugs and Issues

Please submit any issues, questions, or feature requests via the GitHub issue tracker.

Supporting Myna

You can show your support by reporting issues, making suggestions, contributing fixes, offering ideas, and providing feedback or critiques of any aspect of this project. Whether it is about code, development environment, documentation, processes, tests, philosophy, or general approach, it is all appreciated and helpful. I want this library to be as useful to you, as it is for me, and I want to continue to learn to be a better developer.

Letting me know how you use Myna, or why you decided against it would also be helpful, as would sharing your grammars with us!

Author

Christopher Diggins

License

Myna is licensed under the MIT License.

Acknowledgements

Thank you to my three gatitas Anneye, Anna, and Beatrice for their love, patience, and support. Also thank you to everyone who has ever written open-source code. We are doing this together!

Thanks to Eric Lindahl of Sciumo for being the first person to financially support the Myna project.

myna-parser's People

Contributors

Stargazers

Watchers

Forkers

bd82 elidoran sciumo hypercubed henry-luo nio3000 ninoseki hendrikbursian artbit liluxdev hhy5277 nirvanasupermind kpiwko saurabha-a luminarious

myna-parser's Issues

Possible performance degradation under Chrome Canary 61

Looks like Myna only runs much slower on Chrome Canary 61
(which includes the new TurboFan V8 compiler and Ignition interpreter).

I don't know if this is a bug in Chrome that will be fixed by the time an official version of Chrome is released or if it is a a real issue.

Just an heads up...

Also Note:

Most other parsing libraries get a speed boost in Chrome Canary 61.
Screen shot (I get about 500 ops/Second on Chrome 58)

Error recovery and continuation

The next big thing for Myna will be partial recovery from errors, so that multiple errors can be detected in the code. See: http://duriansoftware.com/joe/Constructing-human-grade-parsers.html.

Basically I see special rules to indicate where to advance to in the case of an error, and returning an array of errors from a parse (empty array indicating success).

Possible performance improvements

Class instances avoidance

I see much usage of the newkeyword.

I am uncertain but it appears as though it may also be used in hot spots( parser runtime) in the code and not only in one time initial definitions?

if that is indeed the case I suggest you try avoiding initializing class instances at parser runtime.
Plain objects ({}) if sufficient would be much faster.

Specific boolean conditions.

This is a minor thing but conditions such as:

            if (!this.createsAstNode)

Will take longer than a more precise condition:

if (this.createsAstNode === false)

If these kind of conditions appear in enough hot spots in the code base
you may be able to squeeze a a tiny bit of extra performance.

Myna module improperly exported

Running require("myna-parser") in Runkit failed. I discovered that the way I was exporting the module was incorrect.

In Myna.ts Instead of:

declare let module;
if (typeof module === "object" && module.exports) 
    module.exports.Myna = Myna;

I should have written:

declare let module;
if (typeof module === "object" && module.exports) 
    module.exports = Myna;

Create railroad diagrams

https://github.com/dundalek/GrammKit

2.4.0 no longer works in the browser.

Reproduce with this snippet:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
</head>
<body>

// will cause "Uncaught ReferenceError: exports is not defined"
<script src="https://unpkg.com/myna-parser/myna.js"></script>

</body>
</html>

Add Myna to JavaScript performance benchmarks

See https://sap.github.io/chevrotain/performance/

Add return types to standalone functions

Myna module functions don't have return types declared. This limits the effectiveness of TypeScript real-time type checking.

Question: how to check in sequence if there is a whitespace + symbol

Hello, I don't want to capture sequence, when there are no whitespaces: bla+something+bla, I only want to capture, when there are whitespaces: bla +something+ bla. I am having trouble getting this to work.

Catching everything between + is straight forward: m.seq('+', this.plainTextEmphasis, '+').ast, but I wan to parse it only when ther is a space/empty line before and after +.

I've tried just adding space char in the sequence before +, but that doesn't seem to work:
m.seq(' ', '+', this.plainTextEmphasis, '+', ' ').ast

I've also tried replacing whitespace with ws, atWs etc., but that didn't help either.

Does anyone know, how to do it properly?

Feature request: Karma and QUnit

It would be helpful to have an test harness and leverage qunit more effectively.
I propose using Karma and the QUnit karma plugin.

https://github.com/karma-runner/karma-qunit

How to use ?

Preamble: I like this library's API. I use both Chevrotain and Parsimmon for parsing. This is an interesting new library.

I'm having the same trouble I had with Chevrotain. Both projects don't say how to use them to parse in their README. I've looked thru the grammer/tools and I still can't quite figure out what it is you expect us to do.

What I'm looking for is how to do this:

// get the reusable parser created with a grammar
var parser = require('./getMyAwesomeMynaParser')

// export a parse function which uses it
module.exports = function parse(input) {
  return parser.parse(input)
  // or, use some top-level rule
  return parser.topRule(input)
}

Also, when I figure that out, I'm thinking it'll provide an AST. Or, is it possible to have the parser generate something as it parses similar to Parsimmon's map() and others to generate a value?

delay() rule ID number changes each time

I'm setting up an AST evaluator and instead of matching rules by their name I am using their id in the hopes it'll be faster to do number comparison than string comparison.

It's working except for the one delayed rule created via myna.delay(function() { return myna.choice(...) }).

When I run myna.parse(rule, input) the ID of the delayed node is brand new. The first one after the last of my rule's IDs.

And, if I run the parse() repeatedly then the ID changes each time. It's growing by 10. So, I'm wondering what's going on in parse() which creates 10 new rules. Perhaps that one rule is being used 10 times?

I'm thinking the original ID should remain the same. Maybe not for Delay.

Ah-hah. I looked in the JS source and found the M.choice() I'm using in the delayed rule creates a new Choice rule. So, it's doing this every time the delayed rule is used during a parse().

Do you think it's possible to reuse the ID from the "delayed rule" in the rule created when the deferred execution is resolved? In what I'm doing, I mean the myna.choice() would create the new Choice rule and use the ID from its wrapping Delay rule.

Wait, does it call that every time the rule is encountered or does it resolve the first time?

It looks like the delay is created with a function which is passed to the Delay constructor and then reused in parseImplementation() like this.fn().parse(p). So, it seems like it re-resolves it every time.

So, what do you think about making it capable of resolving only once and reusing the id assigned to the Delay rule which wrapped it?

explainjs listed as dependency

Explainjs is not a dependency. NPM must have updated the package.json when I was trying out explainjs.

Case insensitive keywords?

Should there be a way to declare a keyword (or char + seq) as case insensitive?

Something like:

this.exponent = m.seq(m.char("e").ignoreCase, this.plusOrMinus.opt, m.digits);

this.null = m.keyword("null").ignoreCase.ast;

ParserError messages don't seem to be propogated

A grammar I'm developing seems to be throwing a ParserError, although I'm having a lot of trouble finding out exactly where the error is happening. To make matters more complicated, I seem to be unable to see the error messages that are coming out of the ParserError. It seems like the message parameter is not being propogated to the error object or accessible in the console.

Here's a simple test case.

> try { throw new Error('123') } catch (e) { console.error(e.toString()) }
Error: 123
undefined
> try { throw new ParserError('123') } catch (e) { console.error(e.toString()) }
Error

Question on requirements for the programing language you are working on.

From the readme:

The JavaScript library that comes closest to what I was looking for in terms of API is probably Parsimmon. It did not satisfy my particular needs for the project I am working on

As another Parsing Library author (Chevrotain) I'm interested in what requirements Parsimmon did
not satisfy?

    var g = new function() 
    {
        var delimiter = ",";   // need to define delimiter first
        this.textdata   = m.notChar('\n\r"' + delimiter);    
        ...
    }
    console.log(parser(input));

the variable delimiter is not defined;
last line should be parser, not parse.