johnedquinn / kanonic Goto Github PK

When parsing, we currently need to pass the grammar itself AND the ParserMetadata to the KanonicParser. Ideally, we only pass the ParserMetadata (generated). So, we need to either:

• Serialize the Grammar and generate the file
• Or, a much easier approach, generate the Grammar DSL to create the grammar.

Instead of using the Grammar directly, all generators should use the GrammarSpec.

So, right now, it's a dumb lever, and it needs to be smarter. It does most things well enough, but for some edge cases, it doesn't work. Formalize the Lexer and make it better.

kanonic-parser -> kanonic-syntax
kanonic-syntax -> kanonic-syntax-gen

Potentially do this. Could be fun.

Add pre-build task to generate some grammars into the tests project.

• From build/generated-src
• To build/generated-src/main

Right now, we install the partial-syntax command, and we run it manually. Ideally, this would be a build step after building runtime and gen but before tool.

Figure out how to create helper functions to look through children. AKA, given some node, generate functions to grab allowed children:

node.exprs()

Generate the rules necessary

Change the regexes for the following:

• Token: "[A-Z][A-Z_]*"
• Rule: "[a-z][a-z_]*"

Then, make the code generation pretty by splitting on the _.

In below, since the root is expr, and since one of expr's variants, symbol, is a group, it cause the root to be a generated node. Causing issues, since we can't visit a generated node. Needs research. Likely in the AstConverter.

kanonic: {
  name: ion;
  root: expr;
  package: "io.johnedquinn.ion.generated";
};

COLON: ":";
PAREN_LEFT: "\(";
PAREN_RIGHT: "\)";
SYMBOL_QUOTED: "'((\')|[^'])*'";
SYMBOL: "[a-zA-Z]+";
NUMBER: "[0-9]+";
LITERAL_STRING: "\"((\")|[^\"])*\"";

expr
	: annotation? expr --> annotated_expr
	| NUMBER --> number
	| PAREN_LEFT expr+ PAREN_RIGHT --> sexp
	| ( SYMBOL | SYMBOL_QUOTED ) --> symbol
	| LITERAL_STRING --> string
	;

Due to overriding some Gradle dependencies, there are intermittent build failures. Need to fix.

Don't implicitly add it.

For rule:

some_rule
    : other_rule PLUS other_rule --> rule_name
    ;

The generate rule looks like:

data class RuleName(...) : SomeRule(...) {
    public fun otherRule(): List<OtherRule> = {...}
    public fun PLUS(): List<TerminalNode> = {...}
}

But, we KNOW that only a single PLUS can be present, so, the function should be:

    public fun PLUS(): TerminalNode = {...}

We need to account for generated/hidden nodes while counting too.

Similar to ANTLR's generateGrammarSource we should create a Kanonic generate task.

• Print AST
• Print automaton
• Print grammar
• Print table
• Print tokens
• Print rules

We need to handle groups and repeating groups:

expr
    : expr (PLUS | MINUS | ASTERISK) atomic --> arithmeticExpr
    | expr ( PERIOD expr )+ --> indexingExpr
    ;

Probably would be best to address #3 first

Need to clean up the project and make all tests pass.

AKA:

expr
    : expr op=(PLUS | MINUS) atomic
    ;

ANTLR allows:

topRule:
    : key=IDENT EQUALS value=IDENT # IdentEqualsIdent
    | key=IDENT EQUALS ref=someOtherRule # IdentEqualsRef
    ;