b3b00 / cslycli Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 1.0 495 KB

License: MIT License

C# 100.00%

cslycli's Introduction

cslycli's People

Contributors

Stargazers

Watchers

Forkers

tanzui

cslycli's Issues

token labels

Csly CLI should support lexeme labels as in

[Sugar("(")]
    [LexemeLabel("en","left paranthesis")]
    [LexemeLabel("fr","paranthèse ouvrante")]
    LeftPar,

This could be done using an attribute on token definition (using the same attributes mechanism as for production method names)

@label("en","left paranthesis");
@label("fr","paranthèse ouvrante");
[Sugar] LeftPar: "(";

extension named nodes

allow to name nodes in extension definition

name arriving node with

-> (<NAME>) 'x' ->

transition to node

-> 'y' @<NAME>

This will need to define transitions on many lines to allow caming back to a named node (looping or branching)

from c# to spec

generic lexer modes

Add support for generic lexer modes




[Push(submode)]
 [Sugar] OPEN : >>>

[Mode('subMode)]
[Sugar] ARROW : ->

[Pop]
 [Sugar] CLOSE : <<<

lexer extraction fails on CSLy sample JsonTokenGeneric

lexer extraction fails on https://github.com/b3b00/csly/blob/dev/src/samples/jsonparser/JsonTokenGeneric.cs
probably because tokens define a channel.
furthermore BOOLEAN token defines 2 possible values :

[Lexeme(GenericToken.KeyWord,channel:0, "true", "false")]
        BOOLEAN = 4,

generated lexer spec is

genericLexer JsonTokenGeneric;

[String] STRING;

[Double] DOUBLE;

[Int] INT;

[KeyWord] BOOLEAN : "0";

[Sugar] ACCG : "0";

[Sugar] ACCD : "0";

[Sugar] CROG : "0";

[Sugar] CROD : "0";

[Sugar] COMMA : "0";

[Sugar] COLON : "0";

checks : check if extension has an end

lexer extraction fails on csly sample MinimalXml

lexer extraction from https://github.com/b3b00/csly/blob/dev/src/samples/XML/MinimalXmlLexer.cs fails on COMMENT token.
token is not extracted leading to grammar error

rule comment : COMMENT references COMMENT that does not exist

provide an API

expose a C# API to allow use of spec files from any .net app.

IndentedWhileParser extraction causes compilation error

extracted IndentedWhileParserGeneric produces
grammar spec :


genericLexer IndentedWhileTokenGeneric;

[IndentationAware(true)]
[KeyWord] IF : "IF";
[KeyWord] IF : "if";

[KeyWord] THEN : "THEN";
[KeyWord] THEN : "then";

[KeyWord] ELSE : "ELSE";
[KeyWord] ELSE : "else";

[KeyWord] WHILE : "WHILE";
[KeyWord] WHILE : "while";

[KeyWord] DO : "DO";
[KeyWord] DO : "do";

[KeyWord] SKIP : "SKIP";
[KeyWord] SKIP : "skip";

[KeyWord] TRUE : "TRUE";
[KeyWord] TRUE : "true";

[KeyWord] FALSE : "FALSE";
[KeyWord] FALSE : "false";

[KeyWord] NOT : "NOT";
[KeyWord] NOT : "not";

[KeyWord] AND : "AND";
[KeyWord] AND : "and";

[KeyWord] OR : "OR";
[KeyWord] OR : "or";

[KeyWord] PRINT : "PRINT";
[KeyWord] PRINT : "print";

[KeyWord] RETURN : "RETURN";
[KeyWord] RETURN : "return";

[AlphaNumDashId] IDENTIFIER;

[String] STRING;

[Int] INT;

[Sugar] GREATER : ">";

[Sugar] LESSER : "<";

[Sugar] EQUALS : "==";

[Sugar] DIFFERENT : "!=";

[Sugar] CONCAT : ".";

[Sugar] ASSIGN : ":=";

[Sugar] PLUS : "+";

[Sugar] MINUS : "-";

[Sugar] TIMES : "*";

[Sugar] DIVIDE : "/";

[Sugar] SEMICOLON : ";";

[SingleLineComment] COMMENT : "#";



parser IndentedWhileParserGeneric;

[UseMemoization]
[Right 50] LESSER;
[Right 50] GREATER;
[Right 50] EQUALS;
[Right 50] DIFFERENT;
[Right 10] CONCAT;
program: sequence;
block : INDENT[d] sequence UINDENT[d];
statement : block;
sequence: statement*;
statement: IF[d] IndentedWhileParserGeneric_expressions THEN[d] block (ELSE[d] block)?;
statement: WHILE[d] IndentedWhileParserGeneric_expressions DO[d] block;
statement: IDENTIFIER ASSIGN[d] IndentedWhileParserGeneric_expressions;
statement: SKIP[d];
statement: RETURN[d] IndentedWhileParserGeneric_expressions;
statement: PRINT[d] IndentedWhileParserGeneric_expressions;
primary: INT;
primary: TRUE;
primary: FALSE;
primary: STRING;
primary: IDENTIFIER;
[Operand] operand: primary;
[Right 10] PLUS;
[Right 10] MINUS;
[Right 50] TIMES;
[Right 50] DIVIDE;
[Prefix 100] MINUS;
[Right 10] OR;
[Right 50] AND;
[Prefix 100] NOT;

When compiled an exception is thrown :

non terminal clause INDENT can not be discarded ! 84

generic lexer extension

extend dsl to help add generic lexer extensions.