Giter VIP home page Giter VIP logo

antlr4-tool's Introduction

antlr4-tool

Overview

The purpose and intent of this project was to create and generate Typescript declaration files for the Antlr4 JavaScript target. Allowing for generated Antlr Parsers and Lexers that have full ES5 browser support. This tool also allows for pure JavaScript implementations, if needed.

Requirements

  • Java Runtime Environment 1.6+ (1.8+ recommended)

Getting Stated

  • Install antlr4-tool, adding it to your package.json
npm install --save-dev antlr4-tool
  • Install the Antlr4 library
npm install -S antlr4

#
# (TypeScript Only)
#
npm install -S @types/antlr4
  • Add a grammar to your project, e.g. path/to/Grammar.g4
"scripts": {
  "generate-parser": "antlr4-tool -o parser path/to/Grammar.g4"
}
  • Run the NPM script command
npm run generate-parser
  • Use your generated Parser

JavaScript

const antlr4 = require('antlr4')
const InputStream = antlr4.InputStream;
const CommonTokenStream = antlr4.CommonTokenStream;

const GrammarParser = require('./parser/GrammarParser').GrammarParser;
const GrammarLexer = require('./parser/GrammarLexer').GrammarLexer;

const inputStream = new InputStream('int x = 10;');
const lexer = new GrammarLexer(inputStream);
const tokenStream = new CommonTokenStream(lexer);
const parser = new GrammarParser(tokenStream);

// Parse the input, where `compilationUnit` is whatever entry point you defined
const tree = parser.compilationUnit();
console.log(tree);

TypeScript

import {InputStream, CommonTokenStream} from 'antlr4';
import {GrammarParser} from './parser/GrammarParser';
import {GrammarLexer} from './parser/GrammarLexer';

const inputStream = new InputStream('enter you grammar here');
const lexer = new GrammarLexer(inputStream);
const tokenStream = new CommonTokenStream(lexer);
const parser = new GrammarParser(tokenStream);

// Parse the input, where `compilationUnit` is whatever entry point you defined
const tree = parser.compilationUnit();
console.log(tree);

TypeScript Notes

Add to your tsconfig.json:

"typeRoots": ["types"],

For Command-Line Use

Installation For Command-Line Use

npm install -g antlr4-tool
Usage: antlr4-tool [options] <grammars...>

Options:

  -V, --version                 output the version number
  -o --output-dir [output_dir]  Output Directory (Default: Current Directory)
  -l --language [language]      Antlr Language Target: ts, typescript, js, javascript (Default: typescript)
  --listener                    Generate parse tree listener (Default)
  --no-listener                 Don't generate parse tree listener
  --visitor                     Generate parse tree visitor (Default)
  --no-visitor                  Don't generate parse tree visitor
  -h, --help                    output usage information

Finding the Version

antlr4-tool -V

Example Creating a C Parser for TypeScript

antlr4-tool -l ts -o c-parser samples/c/C.g4

Example Creating a C Parser for JavaScript

antlr4-tool -l js -o c-parser samples/c/C.g4

Example

Using antlr4-tool as a Library

const tool = require('antlr4-tool');

const opts = {
   language: 'TypeScript', // Only support for JavaScript & TypeScript
   grammarFiles:  ['samples/c/C.g4'],
   outputDirectory: 'c-parser'
};

const compiledResults = tool.compile(opts);

console.log(compiledResults);

antlr4-tool's People

Contributors

alwonder avatar andreysyagrovskiy avatar dependabot[bot] avatar jgellin-sf avatar johnholliday avatar jonfreedman avatar mcchatman8009 avatar patrick-remy avatar scorsi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

antlr4-tool's Issues

Windows support

Hi,

Which call break on Windows.
Replacing it with Where in antlr-compiler.js, line 98 fix. Nevertheless, it's not a clean solution, juste a temps one...

Refering to #21 , it could be fixed with a check on process.plateform.

Is it possible to integrate in next release?

Listener enter/exit wrong declared

The listener definition file declares the listener methods by the pattern "ruleEnter"/"ruleExit", but the js source shows that they are defined as "enterRule"/"exitRule".

literalNames and ruleNames properties are missing from Lexer.d.ts

e.g

TSELLexer.prototype.channelNames = [ "DEFAULT_TOKEN_CHANNEL", "HIDDEN",  "EXTRA" ];

TSELLexer.prototype.modeNames = [ "DEFAULT_MODE" ];

TSELLexer.prototype.literalNames = [ null, "'+'", "'-'", ... ];

TSELLexer.prototype.symbolicNames = [ null, "PLUS", "MINUS", ... ];

TSELLexer.prototype.ruleNames = [ "PLUS", "MINUS" ... ];

TSELLexer.prototype.grammarFileName = "TSELLexer.g4";

vs

export declare class TSELLexer extends Lexer {
    readonly channelNames: string[];
    readonly modeNames: string[];
    readonly symbolicNames: string[];
    readonly grammarFileName: string;

    constructor(input: InputStream);
}

antlr4-tool can't handle superClass

This is a super useful tool. I am trying to generate my grammar to multiple target languages (Typescript/Javascript and Java). Since I have some predicates, I have written a base parser class with the same name in each target language and referring to it with the superClass option. This problem/approach are similar to those described here: https://stackoverflow.com/questions/45196413/ways-of-keeping-antlr4-grammar-target-independent.

When I use antlr4-tool on this grammar I get the following:

Compiling src/json.g4...
evalmachine.<anonymous>:74
jsonParser.prototype = Object.create(BaseParser.prototype);
                                                ^

TypeError: Cannot read property 'prototype' of undefined
    at evalmachine.<anonymous>:74:49
    at Script.runInContext (vm.js:127:20)
    at Object.runInContext (vm.js:290:6)
    at Object.readParser (/issue/node_modules/antlr4-tool/dist/antlr-core/parser-util.js:34:8)
    at AntlrCompiler.compileTypeScript (/issue/node_modules/antlr4-tool/dist/antlr-core/antlr-compiler.js:66:37)
    at /issue/node_modules/antlr4-tool/dist/antlr-core/index.js:40:78
    at /issue/node_modules/antlr4-tool/dist/antlr-core/index.js:17:23
    at arrayEach (/issue/node_modules/lodash/lodash.js:516:11)
    at Function.forEach (/issue/node_modules/lodash/lodash.js:9342:14)
    at compileWithFunction (/issue/node_modules/antlr4-tool/dist/antlr-core/index.js:9:7)

The Javascript seems to generate OK, it's just the Typescript declarations that fail. I created the simplest base parser class I could in Typescript:

import { CommonTokenStream, Parser } from "antlr4";

export class BaseParser extends Parser {
  constructor(input: CommonTokenStream) {
    super(input);
  }
}

I compiled this code to ES5, then ran antlr4-tool and got the above error.

Fails with cannot find module 'anltr4/index' with antlr4 4.9

All is well with antlr4 4.8.0 but updating to 4.9 causes the following. I'm brand new in the javascript world so not very effective in debugging this. It seems there was an issue like this in 2018 #14. The only change I made was bumping antlr4 from 4.8 to 4.9.

yarn run v1.19.2
$ antlr4-tool --output-dir src/parser --language typescript grammar/DE6Parser.g4 grammar/DE6Lexer.g4 grammar/TagPath.g4
Compiling grammar/DE6Parser.g4, grammar/DE6Lexer.g4, grammar/TagPath.g4...
internal/modules/cjs/loader.js:883
throw err;
^

Error: Cannot find module 'antlr4/index'
Require stack:

  • /Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/parser-util.js
  • /Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/antlr-compiler.js
  • /Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/index.js
  • /Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/compile.js
  • /Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/cli.js
  • /Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/app.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)
    at Function.resolve (internal/modules/cjs/helpers.js:94:19)
    at vmRequire (/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/parser-util.js:24:32)
    at evalmachine.:3:14
    at Script.runInContext (vm.js:142:18)
    at Object.runInContext (vm.js:293:6)
    at Object.readParser (/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/parser-util.js:28:8)
    at AntlrCompiler.compileTypeScript (/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/antlr-compiler.js:66:37)
    at /Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/index.js:40:78
    at /Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/index.js:17:23 {
    code: 'MODULE_NOT_FOUND',
    requireStack: [
    '/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/parser-util.js',
    '/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/antlr-compiler.js',
    '/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/antlr-core/index.js',
    '/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/compile.js',
    '/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/cli.js',
    '/Users/drm/projects/nrg/dicomeditjs/de6js/node_modules/antlr4-tool/dist/app.js'
    ]
    }
    error Command failed with exit code 1.

Cannot read property 'join' of undefined

Hello again,

When we use the antlr4-tool cli without argument, it crashes. Generally, the help is shown by default.

MBP-de-Sylvain:ida scorsi$ antlr4-tool 
/Users/scorsi/.config/yarn/global/node_modules/antlr4-tool/dist/cli.js:32
    log(`Compiling ${antlrGrammars.join(', ')}...`);
                                   ^

TypeError: Cannot read property 'join' of undefined
    at Object.<anonymous> (/Users/scorsi/.config/yarn/global/node_modules/antlr4-tool/dist/cli.js:32:36)
    at Module._compile (internal/modules/cjs/loader.js:702:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:713:10)
    at Module.load (internal/modules/cjs/loader.js:612:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
    at Function.Module._load (internal/modules/cjs/loader.js:543:3)
    at Module.require (internal/modules/cjs/loader.js:650:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/Users/scorsi/.config/yarn/global/node_modules/antlr4-tool/dist/app.js:2:1)
    at Module._compile (internal/modules/cjs/loader.js:702:30)

Error: Command failed: which java

[email protected] generate-parser D:\test\antlr
antlr4-tool -o parser Expr.g4

Compiling Expr.g4...
'which' is not recognized as an internal or external command,
operable program or batch file.
child_process.js:651
throw err;
^

Error: Command failed: which java
'which' is not recognized as an internal or external command,
operable program or batch file.

at checkExecSyncError (child_process.js:611:11)
at Object.execSync (child_process.js:648:13)
at D:\test\antlr\node_modules\antlr4-tool\dist\antlr-core\antlr-compiler.js:98:19
at chdir (D:\test\antlr\node_modules\chdir\index.js:6:13)
at AntlrCompiler.compileJavaScript (D:\test\antlr\node_modules\antlr4-tool\dist\antlr-core\antlr-compiler.js:97:9)
at AntlrCompiler.compileTypeScript (D:\test\antlr\node_modules\antlr4-tool\dist\antlr-core\antlr-compiler.js:62:38)
at D:\test\antlr\node_modules\antlr4-tool\dist\antlr-core\index.js:40:78
at D:\test\antlr\node_modules\antlr4-tool\dist\antlr-core\index.js:17:23
at arrayEach (D:\test\antlr\node_modules\lodash\lodash.js:516:11)
at Function.forEach (D:\test\antlr\node_modules\lodash\lodash.js:9344:14)

package is a reserved word in strict mode

I get an error when trying to import in babel.

Error: SyntaxError: LanguageLexer.js: package is a reserved word in strict mode

// Generated from Language.g4 by ANTLR 4.7
// jshint ignore: start
var antlr4 = require('antlr4/index');
package antlr; // <---

Sub-rules are missing in Parser.d.ts

Hi! First of all, thank for this amazing package, it's very nice to have antlr available in browser side 😃

I just have some little mismatches between the PslParser.js and the PslParser.d.ts files.

Here my repro steps and bit more details:

Command

 antlr4-tool -l ts -o src/parser Grammar.g4

The grammar

grammar Grammar;

schema: 'schema' (schemaParams);
schemaParams: '(' (schemaParam (',' schemaParam)*)? ')';
schemaParam: ('a' | 'b' | 'c')*;

GrammarParser.js

SchemaParamsContext.prototype.schemaParam = function(i) {
    if(i===undefined) {
        i = null;
    }
    if(i===null) {
        return this.getTypedRuleContexts(SchemaParamContext);
    } else {
        return this.getTypedRuleContext(SchemaParamContext,i);
    }
};

GrammarParser.d.ts (actual)

export declare class SchemaParamsContext extends ParserRuleContext {
    
}

GrammarParser.d.ts (expected)

export declare class SchemaParamsContext extends ParserRuleContext {
+    schemaParam(): SchemaParamContext;
}

Cannot find module 'antlr4-tool/dist/cli'

If I try to run the cloned repo, I'll receive the following error:

$ antlr4-tool
internal/modules/cjs/loader.js:583
    throw err;
    ^

Error: Cannot find module 'antlr4-tool/dist/cli'

Steps to reproduce

$ git clone https://github.com/mcchatman8009/antlr4-tool
$ cd antlr4-tool
$ npm install
$ npm run build
$ ./dist/app.js # or sudo npm link && antlr4-tool

Automatic antlr4-grammar generation aborted because of process.exit

I need to generate the whole antlr4-grammar repo on demand and some are failing beecause of the javascript target. This is fine. However, the following line is aborting the whole generation.


Could you please change this to throw an exception and catch it in the cli, if someone wants to have the abortion?
It would be really helpful.
Anyway, thank you for that package :)

Cannot find module 'antlr4/index'

If I try to use the lib to compile a grammar outside of the source folder (e.g. on the Desktop), I'm receiving the following exception:

{ Error: Cannot find module 'antlr4/index'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:581:15)
    at Function.Module._resolveFilename (…/node_modules/tsconfig-paths/lib/register.js:73:40)
    at Function.Module._load (internal/modules/cjs/loader.js:507:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at sandbox.require.id (…/node_modules/node-eval/index.js:61:48)
    ***at Object.<anonymous> (…/Desktop/eval.js:3:14)*** 
    at _commonjsEval (…/node_modules/node-eval/index.js:80:21)
    at module.exports (…/node_modules/node-eval/index.js:36:19)
    at Object.readParser (…/node_modules/antlr4-tool/src/antlr-core/parser-util.ts:20:20)
    at AntlrCompiler.compileTypeScript (…/node_modules/antlr4-tool/src/antlr-core/antlr-compiler.ts:83:37)
    at …/node_modules/antlr4-tool/src/antlr-core/index.ts:48:63
    at …/node_modules/antlr4-tool/src/antlr-core/index.ts:20:25
    at arrayEach (…/node_modules/lodash/lodash.js:516:11)
    at Function.forEach (…/node_modules/lodash/lodash.js:9344:14)
    at compileWithFunction (…/node_modules/antlr4-tool/src/antlr-core/index.ts:10:7)
    at compileGrammarAsTypeScript (…/node_modules/antlr4-tool/src/antlr-core/index.ts:48:12)

Some problem with the CLI

Hello,

As I don't like to carry the antlr4 jar file into my repository, I was looking for a CLI to build my grammar.
I encountered some problem with the CLI.

No arguments passed

When we type npx antlr4-tool, we got the following output:

Cannot read property 'join' of undefined

We should got option -h by default I guess.

JavaScript language

It's a nice solution to give antlr4.js, maybe you should precise in the doc that it automatically output in JavaScript ? But it add complexity to the package. For me, it is unnecessary.
Maybe you could add some shortcut for the JavaScript language like javascript or js. I could imagine that for TypeScript, it is the same ?
I will create a pull request for this.

More options ?

I would like to specify -listener or -no-listener and -visitor or -no-visitor.

The "path" argument must be of type string. Received type object

Yeah, it is the really error. I typed the following command:
antlr4-tool -l JavaScript -o src ./grammar/IDA.g4
And I got the following error:

$ antlr4-tool -l JavaScript -o src ./grammar/IDA.g4
Compiling ./grammar/IDA.g4...
path.js:39
    throw new ERR_INVALID_ARG_TYPE('path', 'string', path);
    ^


TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type object
    at assertPath (path.js:39:11)
    at Object.resolve (path.js:1090:7)
    at _.each (/Users/scorsi/Code/ida/node_modules/antlr4-tool/dist/antlr-core/index.js:11:33)
    at arrayEach (/Users/scorsi/Code/ida/node_modules/lodash/lodash.js:516:11)
    at Function.forEach (/Users/scorsi/Code/ida/node_modules/lodash/lodash.js:9342:14)
    at compileWithFunction (/Users/scorsi/Code/ida/node_modules/antlr4-tool/dist/antlr-core/index.js:9:7)
    at compileGrammarAsJavaScript (/Users/scorsi/Code/ida/node_modules/antlr4-tool/dist/antlr-core/index.js:39:12)
    at Object.compile (/Users/scorsi/Code/ida/node_modules/antlr4-tool/dist/antlr-core/index.js:53:20)
    at module.exports (/Users/scorsi/Code/ida/node_modules/antlr4-tool/dist/compile.js:9:34)
    at Object.<anonymous> (/Users/scorsi/Code/ida/node_modules/antlr4-tool/dist/cli.js:34:28)

Hope you could help

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.