gkappler / combinedparsers.jl Goto Github PK

View Code? Open in Web Editor NEW

77.0 1.0 10.0 2.65 MB

Compiled parser combinators and regular expressions in pure julia

License: MIT License

Julia 100.00%

parser-combinators julia-types

combinedparsers.jl's Introduction

CombinedParsers in pure Julia

A package for combining parsers and transforming strings into julia types.

Compose parsers parsimoneously within a functional parser combinator paradigm, utilize Julia's type inference for transformations, log conveniently for debugging, and let Julia compile your parser for performance.

The CombinedParsers design

is fast due to Julia parametric types, and compiler optimizations with generated functions,
parsing result transformations infer the domain data types,
is composable and optimizable with Julia method dispatch,
provides flexible public API for parsing, matching, iteration
can be defined with PCRE and EBNF syntax.

Getting started

CombinedParsers.jl is a registered package. Install with

] add CombinedParsers

Example: rational numbers arithmetics

This example demonstrates reading of arithmetical terms for rational numbers. Reflecting operator precedence, term are subterms, interleaved by */, and subterms are Either integer numbers

@syntax subterm = Either{Rational{Int}}([NumericParser(Int)]; convert=true)

or a subterm can also be an additive term in parentheses:

@syntax for parentheses in subterm
    mult = evaluate |> join(subterm, CharIn("*/"), infix=:prefix )
    @syntax term = evaluate |> join(mult,    CharIn("+-"), infix=:prefix )
    Sequence(2,'(',term,')')
end

This CombinedParser definition in 5,5 lines registers a @term_string macro for parsing and evaluating rational arithmetics:

julia> term"4*10+2"
42//1

Is every rational answer ultimately the inverse of a universal question in life?

Details in the full documentation example.

Acknowledgements

This package leverages Julia's compiler and superior type system to parsing.

I am thankful for contributions and inspiration from many great packages:

TextParse.jl

A bunch of fast text parsing tools, used in CSV.jl

CombinedParsers composes with fast TextParse.jl both ways because CombinedParser <: TextParse.AbstractToken and by providing a method for TextParse.tryparsenext, (leveraging the supreme Julia compiler, type and package architecture).

If you seek support with a CSV example, please contact me (e.g. address text field parsing).

Inspirations

The work was strongly inspired by the great Scala fastparse package, and also the elm parser.
Parsers.jl, a collection of parsers for date and primitive types, inspired the parse methods.
Automa.jl, a Julia package for text validation, parsing, and tokenizing based on state machine compiler. The package compiles deterministic finite automata. (Currently there is no inter-operation possible, because in Automa processing of parsed tokens is done with actions).
ParserCombinator.jl was a great inspiration. Yet I decided for a new design with a focus on transformations and type inference with parametric types, instead of basing this work off ParserCombinator, written before 2016 (and fixed for Julia 1.0 in 2018). CombinedParsers integrates into the Julia 1.0 Iteration API, small Union{Nothing,T} where T types instead of using Nullables, compiler optimizations and generated functions. I want to provide benchmarks comparisons with ParserCombinator.jl.

Next Steps

Syntax freeze -- your comments are appreciated!
decide for a error tracing strategy, backtracking. If you want to collaborate on stepping & debugging, please reach out to me.
Performance optimizations
streaming
test coverage underestimated (PCRE tests are not included in travis)

Contributing and Questions

Contributions and feedback are very welcome, especially regarding brief syntax and constructor dispatch. Please open an issue if you encounter any problems or would just like to ask a question, or contact me at [email protected].

combinedparsers.jl's People

Contributors

Stargazers

Watchers

Forkers

fingolfin zeta1999 rick2047 digital-carver ssghost playfloor pallharaldsson banhbio hiiroo jclugstor

combinedparsers.jl's Issues

Precompilation error for Julia 1.8

Similar to #35, change in 454c399 had no effect.

julia> using CombinedParsers
[ Info: Precompiling CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e]
Illegal inttoptr
          %magicptr = ptrtoint {} addrspace(10)* %132 to i64, !dbg !27

signal (6): Aborted
in expression starting at /home/aidanb/.julia/packages/CombinedParsers/kbHhW/src/re-parser.jl:42
unknown function (ip: 0x7f21fb93b4dc)
gsignal at /usr/bin/../lib/libc.so.6 (unknown line)
abort at /usr/bin/../lib/libc.so.6 (unknown line)
unknown function (ip: 0x7f21fad2f15f)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /usr/bin/../lib/libLLVM-14.so (unknown line)
unknown function (ip: 0x7f21fadb4153)
_ZN4llvm3orc14IRCompileLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm3orc31BasicIRLayerMaterializationUnit11materializeESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm3orc19MaterializationTask3runEv at /usr/bin/../lib/libLLVM-14.so (unknown line)
unknown function (ip: 0x7f21f69db27b)
_ZN4llvm3orc16ExecutionSession22dispatchOutstandingMUsEv at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm3orc16ExecutionSession17OL_completeLookupESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EESt10shared_ptrINS0_23AsynchronousSymbolQueryEESt8functionIFvRKNS_8DenseMapIPNS0_8JITDylibENS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISF_vEEEENSG_ISD_vEENS_6detail12DenseMapPairISD_SI_EEEEEE at /usr/bin/../lib/libLLVM-14.so (unknown line)
unknown function (ip: 0x7f21f69fbd6e)
_ZN4llvm3orc16ExecutionSession19OL_applyQueryPhase1ESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EENS_5ErrorE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS0_10LookupKindERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS8_EENS0_15SymbolLookupSetENS0_11SymbolStateENS_15unique_functionIFvNS_8ExpectedINS_8DenseMapINS0_15SymbolStringPtrENS_18JITEvaluatedSymbolENS_12DenseMapInfoISI_vEENS_6detail12DenseMapPairISI_SJ_EEEEEEEEESt8functionIFvRKNSH_IS6_NS_8DenseSetISI_SL_EENSK_IS6_vEENSN_IS6_SV_EEEEEE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EERKNS0_15SymbolLookupSetENS0_10LookupKindENS0_11SymbolStateESt8functionIFvRKNS_8DenseMapIS5_NS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISK_vEEEENSL_IS5_vEENS_6detail12DenseMapPairIS5_SN_EEEEEE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EENS0_15SymbolStringPtrENS0_11SymbolStateE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS0_15SymbolStringPtrENS0_11SymbolStateE at /usr/bin/../lib/libLLVM-14.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS_9StringRefENS0_11SymbolStateE at /usr/bin/../lib/libLLVM-14.so (unknown line)
unknown function (ip: 0x7f21fadb7601)
unknown function (ip: 0x7f21fadb798a)
unknown function (ip: 0x7f21fadba702)
jl_generate_fptr_impl at /usr/bin/../lib/julia/libjulia-codegen.so.1 (unknown line)
jl_compile_method_internal at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
ijl_apply_generic at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
#Sequence#47 at /home/aidanb/.julia/packages/CombinedParsers/kbHhW/src/CombinedParsers.jl:993
unknown function (ip: 0x7f21fb313879)
Sequence at /home/aidanb/.julia/packages/CombinedParsers/kbHhW/src/CombinedParsers.jl:987
unknown function (ip: 0x7f21fb3211cc)
unknown function (ip: 0x7f21fb320a87)
unknown function (ip: 0x7f21fb3218fb)
unknown function (ip: 0x7f21fb32276e)
unknown function (ip: 0x7f21fb340e63)
unknown function (ip: 0x7f21fb341976)
unknown function (ip: 0x7f21fb341976)
ijl_toplevel_eval_in at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
unknown function (ip: 0x7f21e1fc4a11)
_include at ./loading.jl:1488
include at ./Base.jl:419
unknown function (ip: 0x7f217f0757ea)
jl_f__call_latest at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
include at /home/aidanb/.julia/packages/CombinedParsers/kbHhW/src/re.jl:4
unknown function (ip: 0x7f217f05fc66)
unknown function (ip: 0x7f21fb3211cc)
unknown function (ip: 0x7f21fb320a87)
unknown function (ip: 0x7f21fb3218fb)
unknown function (ip: 0x7f21fb32276e)
unknown function (ip: 0x7f21fb340e63)
unknown function (ip: 0x7f21fb3415d9)
unknown function (ip: 0x7f21fb341976)
unknown function (ip: 0x7f21fb341976)
ijl_toplevel_eval_in at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
unknown function (ip: 0x7f21e1fc4a11)
unknown function (ip: 0x7f21e201ad1f)
unknown function (ip: 0x7f21e1fc4c5f)
unknown function (ip: 0x7f21e1fc4c7f)
jl_f__call_latest at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
include at /home/aidanb/.julia/packages/CombinedParsers/kbHhW/src/CombinedParsers.jl:13
unknown function (ip: 0x7f217eff5536)
unknown function (ip: 0x7f21fb3211cc)
unknown function (ip: 0x7f21fb320a87)
unknown function (ip: 0x7f21fb3218fb)
unknown function (ip: 0x7f21fb32276e)
unknown function (ip: 0x7f21fb340e63)
unknown function (ip: 0x7f21fb3415d9)
unknown function (ip: 0x7f21fb341976)
unknown function (ip: 0x7f21fb341976)
ijl_toplevel_eval_in at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
unknown function (ip: 0x7f21e1fc4a11)
unknown function (ip: 0x7f21e201ad1f)
unknown function (ip: 0x7f21e1fc5da6)
unknown function (ip: 0x7f21e1fc60e9)
unknown function (ip: 0x7f21fb3211cc)
unknown function (ip: 0x7f21fb320a87)
unknown function (ip: 0x7f21fb3218fb)
unknown function (ip: 0x7f21fb32276e)
unknown function (ip: 0x7f21fb340e63)
unknown function (ip: 0x7f21fb341976)
ijl_toplevel_eval_in at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
unknown function (ip: 0x7f21e1fc4a11)
unknown function (ip: 0x7f21e1f0ec6a)
unknown function (ip: 0x7f21e24b36f9)
unknown function (ip: 0x7f21e24b4478)
unknown function (ip: 0x7f21e24b45a8)
unknown function (ip: 0x7f21fb36b6ef)
jl_repl_entrypoint at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
main at /usr/bin/julia (unknown line)
unknown function (ip: 0x7f21fb8d62cf)
__libc_start_main at /usr/bin/../lib/libc.so.6 (unknown line)
_start at /usr/bin/julia (unknown line)
Allocations: 21346255 (Pool: 21340771; Big: 5484); GC: 25
ERROR: Failed to precompile CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e] to /home/aidanb/.julia/compiled/v1.8/CombinedParsers/jl_AieOMd.

Precompile fails on julia v1.6

Hi,
I just noticed that CombinedParsers no longer precompiles on v1.6.

julia> using CombinedParsers
[ Info: Precompiling CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e]
WARNING: could not import Printf.ini_hex into DoubleFloats
WARNING: could not import Printf.ini_HEX into DoubleFloats
ERROR: LoadError: LoadError: LoadError: Any <: CombinedParsers.CombinedParser. Fix with `push!(x|CombinedParsers.NamedParser{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#118#119", CombinedParsers.Sequence{Tuple{String, CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, SubString}, CombinedParsers.Transformation{CombinedParsers.Regexp.var"#112#114", CombinedParsers.Sequence{Tuple{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#104#105", CombinedParsers.Sequence{Tuple{CombinedParsers.NamedParser{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#102#103", CombinedParsers.Repeat{CombinedParsers.Transformation{CombinedParsers.var"#62#63"{2}, CombinedParsers.Sequence{Tuple{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#42#46", CombinedParsers.Repeat{CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}, Vector{Tuple{Int64, Any}}, Vector{CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}}, Vector{Tuple{Int64, Any}}, Vector{_A} where _A}, CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser}, CombinedParsers.Transformation{CombinedParsers.Regexp.var"#42#46", CombinedParsers.Repeat{CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}, Vector{Tuple{Int64, Any}}, Vector{CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}}, Vector{Tuple{Int64, Any}}, Vector{_A} where _A}}, Vector{Any}, Tuple{Vector{_A} where _A, CombinedParsers.CombinedParser, Vector{_A} where _A}}, Vector{Any}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, Vector{CombinedParsers.CombinedParser}}, Vector{Vector{Any}}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, CombinedParsers.CombinedParser}, CombinedParsers.Repeat{CombinedParsers.Transformation{CombinedParsers.var"#62#63"{2}, CombinedParsers.Sequence{Tuple{Char, CombinedParsers.NamedParser{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#102#103", CombinedParsers.Repeat{CombinedParsers.Transformation{CombinedParsers.var"#62#63"{2}, CombinedParsers.Sequence{Tuple{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#42#46", CombinedParsers.Repeat{CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}, Vector{Tuple{Int64, Any}}, Vector{CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}}, Vector{Tuple{Int64, Any}}, Vector{_A} where _A}, CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser}, CombinedParsers.Transformation{CombinedParsers.Regexp.var"#42#46", CombinedParsers.Repeat{CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}, Vector{Tuple{Int64, Any}}, Vector{CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}}, Vector{Tuple{Int64, Any}}, Vector{_A} where _A}}, Vector{Any}, Tuple{Vector{_A} where _A, CombinedParsers.CombinedParser, Vector{_A} where _A}}, Vector{Any}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, Vector{CombinedParsers.CombinedParser}}, Vector{Vector{Any}}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, CombinedParsers.CombinedParser}}, Vector{Any}, Tuple{Char, CombinedParsers.CombinedParser}}, Vector{Any}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, Vector{CombinedParsers.CombinedParser}}}, Vector{Any}, Tuple{CombinedParsers.CombinedParser, Vector{CombinedParsers.CombinedParser}}}, Vector{Any}, Any}, CombinedParsers.Repeat{CombinedParsers.FlatMap{CombinedParsers.Sequence{Tuple{CombinedParsers.NamedParser{CombinedParsers.Transformation{CombinedParsers.var"#62#63"{2}, CombinedParsers.Sequence{Tuple{String, CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, Tuple{UInt32, UInt32}}}, Vector{Any}, Tuple{String, Tuple{UInt32, UInt32}}}, Vector{Any}, Tuple{UInt32, UInt32}}, Vector{Any}, Tuple{UInt32, UInt32}}, CombinedParsers.Optional{CombinedParsers.ConstantParser{1, Char, Char}, Union{CombinedParsers.MatchState, CombinedParsers.None}, Union{Missing, Char}}}, Vector{Any}, Tuple{Tuple{UInt32, UInt32}, Union{Missing, Char}}}, Tuple{var"#s54", var"#s53", var"#s52"} where {var"#s54", var"#s53", var"#s52"}, CombinedParsers.Regexp.var"#110#111", Vector{CombinedParsers.CombinedParser}}, Vector{Tuple{var"#s54", var"#s53", var"#s52"} where {var"#s54", var"#s53", var"#s52"}}, Vector{Vector{CombinedParsers.CombinedParser}}}}, Vector{Any}, Tuple{Any, Vector{Vector{CombinedParsers.CombinedParser}}}}, Vector{Any}, CombinedParsers.CombinedParser}, String}, Vector{Any}, Tuple{String, SubString, CombinedParsers.CombinedParser, String}}, Vector{Any}, Any}, Vector{Any}, Any},y)`.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] push!(x::CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser}, y_::CombinedParsers.NamedParser{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#118#119", CombinedParsers.Sequence{Tuple{String, CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, SubString}, CombinedParsers.Transformation{CombinedParsers.Regexp.var"#112#114", CombinedParsers.Sequence{Tuple{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#104#105", CombinedParsers.Sequence{Tuple{CombinedParsers.NamedParser{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#102#103", CombinedParsers.Repeat{CombinedParsers.Transformation{CombinedParsers.var"#62#63"{2}, CombinedParsers.Sequence{Tuple{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#42#46", CombinedParsers.Repeat{CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}, Vector{Tuple{Int64, Any}}, Vector{CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}}, Vector{Tuple{Int64, Any}}, Vector{_A} where _A}, CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser}, CombinedParsers.Transformation{CombinedParsers.Regexp.var"#42#46", CombinedParsers.Repeat{CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}, Vector{Tuple{Int64, Any}}, Vector{CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}}, Vector{Tuple{Int64, Any}}, Vector{_A} where _A}}, Vector{Any}, Tuple{Vector{_A} where _A, CombinedParsers.CombinedParser, Vector{_A} where _A}}, Vector{Any}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, Vector{CombinedParsers.CombinedParser}}, Vector{Vector{Any}}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, CombinedParsers.CombinedParser}, CombinedParsers.Repeat{CombinedParsers.Transformation{CombinedParsers.var"#62#63"{2}, CombinedParsers.Sequence{Tuple{Char, CombinedParsers.NamedParser{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#102#103", CombinedParsers.Repeat{CombinedParsers.Transformation{CombinedParsers.var"#62#63"{2}, CombinedParsers.Sequence{Tuple{CombinedParsers.Transformation{CombinedParsers.Regexp.var"#42#46", CombinedParsers.Repeat{CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}, Vector{Tuple{Int64, Any}}, Vector{CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}}, Vector{Tuple{Int64, Any}}, Vector{_A} where _A}, CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser}, CombinedParsers.Transformation{CombinedParsers.Regexp.var"#42#46", CombinedParsers.Repeat{CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}, Vector{Tuple{Int64, Any}}, Vector{CombinedParsers.CombinedParser{CombinedParsers.MatchState, CombinedParsers.Always}}}, Vector{Tuple{Int64, Any}}, Vector{_A} where _A}}, Vector{Any}, Tuple{Vector{_A} where _A, CombinedParsers.CombinedParser, Vector{_A} where _A}}, Vector{Any}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, Vector{CombinedParsers.CombinedParser}}, Vector{Vector{Any}}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, CombinedParsers.CombinedParser}}, Vector{Any}, Tuple{Char, CombinedParsers.CombinedParser}}, Vector{Any}, CombinedParsers.CombinedParser}, Vector{Vector{Any}}, Vector{CombinedParsers.CombinedParser}}}, Vector{Any}, Tuple{CombinedParsers.CombinedParser, Vector{CombinedParsers.CombinedParser}}}, Vector{Any}, Any}, CombinedParsers.Repeat{CombinedParsers.FlatMap{CombinedParsers.Sequence{Tuple{CombinedParsers.NamedParser{CombinedParsers.Transformation{CombinedParsers.var"#62#63"{2}, CombinedParsers.Sequence{Tuple{String, CombinedParsers.Either{Vector{Any}, Tuple{Int64, Any}, Tuple{UInt32, UInt32}}}, Vector{Any}, Tuple{String, Tuple{UInt32, UInt32}}}, Vector{Any}, Tuple{UInt32, UInt32}}, Vector{Any}, Tuple{UInt32, UInt32}}, CombinedParsers.Optional{CombinedParsers.ConstantParser{1, Char, Char}, Union{CombinedParsers.MatchState, CombinedParsers.None}, Union{Missing, Char}}}, Vector{Any}, Tuple{Tuple{UInt32, UInt32}, Union{Missing, Char}}}, Tuple{var"#s54", var"#s53", var"#s52"} where {var"#s54", var"#s53", var"#s52"}, CombinedParsers.Regexp.var"#110#111", Vector{CombinedParsers.CombinedParser}}, Vector{Tuple{var"#s54", var"#s53", var"#s52"} where {var"#s54", var"#s53", var"#s52"}}, Vector{Vector{CombinedParsers.CombinedParser}}}}, Vector{Any}, Tuple{Any, Vector{Vector{CombinedParsers.CombinedParser}}}}, Vector{Any}, CombinedParsers.CombinedParser}, String}, Vector{Any}, Tuple{String, SubString, CombinedParsers.CombinedParser, String}}, Vector{Any}, Any}, Vector{Any}, Any})
    @ CombinedParsers ~/.julia/dev/CombinedParsers/src/CombinedParsers.jl:2547
  [3] top-level scope
    @ ~/.julia/dev/CombinedParsers/src/re-parser.jl:566
  [4] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [5] include(x::String)
    @ CombinedParsers.Regexp ~/.julia/dev/CombinedParsers/src/re.jl:4
  [6] top-level scope
    @ ~/.julia/dev/CombinedParsers/src/re.jl:553
  [7] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [8] include(x::String)
    @ CombinedParsers ~/.julia/dev/CombinedParsers/src/CombinedParsers.jl:13
  [9] top-level scope
    @ ~/.julia/dev/CombinedParsers/src/CombinedParsers.jl:2817
 [10] include
    @ ./Base.jl:386 [inlined]
 [11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1209
 [12] top-level scope
    @ none:1
 [13] eval
    @ ./boot.jl:360 [inlined]
 [14] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [15] top-level scope
    @ none:1
in expression starting at /home/jonas/.julia/dev/CombinedParsers/src/re-parser.jl:566
in expression starting at /home/jonas/.julia/dev/CombinedParsers/src/re.jl:1
in expression starting at /home/jonas/.julia/dev/CombinedParsers/src/CombinedParsers.jl:6
ERROR: Failed to precompile CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e] to /home/jonas/.julia/compiled/v1.6/CombinedParsers/jl_F6JdZu.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::Base.TTY, internal_stdout::Base.TTY)
   @ Base ./loading.jl:1356
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1302
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1017
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:910
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:897

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

Parse AST with CombinedParsers?

Hi, I'm trying to build a simple programming language. Is it possible to build the AST with CombinedParsers?

Interop with Automa.jl

Hello Gregor! I'm the current maintainer of Automa.jl (by accident, I didn't write it, but ended up maintaining it, for now).
You mentioned potential integration/collaboration between Automa and CombinedParsers (CP) and I think that sounds like an excellent idea!

The core limitation of Automa is that is uses FSMs, and there are many formats that just can't be parsed using FSMs - JSON, Newick, Julia code, and so on. There has been a longstanding desire among some users of Automa to create an Automa-like package for pushdown automata in order to parse context-free grammars. But it seems no-one who wants it has the skills and knowledge to implement it, so CP is probably the best bet for a good parsing package for that use case. I'll try out CP in the near future and see if it's a good fit for the BioJulia community :)

As I see it, CP and Automa are complementary: Automa is faster and more generic (in that it executes arbitrary code), but its limited to regular grammars, and its user experience is abysmal. I'm slowly pushing towards Automa v1.0, where I want to improve the interface: Make it simpler to use, have better debugging and so on.

What did you envision with integration? Perhaps CP can leverage Automa for some of its codegen? Automa could perhaps be automatically used to create smaller functions that CP can call when parsing. Like, CP could automatically find smaller regular patterns, and create mini-parsers using Automa, which it then calls itself.

While this should be possible in principle, and should make CP faster, I don't know if it's possible practically. Let's see.

If you need any information about Automa internals, let me know. For what it's worth, Automa does sort of support UTF-8, actually. It views non-ascii characters as a pattern of concatenated bytes.

Looking forward to hearing from you!

join does not work on Numeric / iterate not defined

I found this case where iterate(Numeric) is needed.
Maybe there is an easy way to fix this.

If it is hard to fix I would like to know if there is a good workaround.

Version

[5ae71ed2] CombinedParsers v0.1.4
[e0df1984] TextParse v1.0.1

Julia Version 1.5.2
Commit 539f3ce943 (2020-09-23 23:17 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

join(Numeric(Int),' ')

ERROR: MethodError: no method matching iterate(::TextParse.Numeric{Int64})
Closest candidates are:
  iterate(::ExponentialBackOff) at error.jl:252
  iterate(::ExponentialBackOff, ::Any) at error.jl:252
  iterate(::Base.AsyncGenerator, ::Base.AsyncGeneratorState) at asyncmap.jl:382
  ...
Stacktrace:
 [1] join(::Base.GenericIOBuffer{Array{UInt8,1}}, ::TextParse.Numeric{Int64}, ::Char) at .\strings\io.jl:294
 [2] sprint(::Function, ::TextParse.Numeric{Int64}, ::Vararg{Any,N} where N; context::Nothing, sizehint::Int64) at .\strings\io.jl:105
 [3] sprint(::Function, ::TextParse.Numeric{Int64}, ::Vararg{Any,N} where N) at .\strings\io.jl:101
 [4] join(::TextParse.Numeric{Int64}, ::Char) at .\strings\io.jl:301
 [5] top-level scope at REPL[13]:1

Confusing syntax

I'm confused by the exclamation mark syntax. I could find the documentation for the single exclamation mark (https://gkappler.github.io/CombinedParsers.jl/dev/lib/public/#Base.:!), which is a parser transformation to get the matched substring. At first, I thought it was a negation of the parser as that is the meaning of ! elsewhere in Julia. Maybe it would be better to use a longer name such as getmatch and possibly use a different shorthand as well. I also noticed the use of a double exclamation mark !!, but couldn't find documentation for it.

Publish the Org mode parser

I couldn't find a repo for the Org mode parser. Could you please publish it?

Recursion in EBNF

Hi, thanks for great library. However, I was trying EBNF grammar (I could not achieve recursion using the existing API).

p = ebnf"""
letter = "A" | "B" | "C" | "D" | "E" | "F" | "G"
       | "H" | "I" | "J" | "K" | "L" | "M" | "N"
       | "O" | "P" | "Q" | "R" | "S" | "T" | "U"
       | "V" | "W" | "X" | "Y" | "Z" | "a" | "b"
       | "c" | "d" | "e" | "f" | "g" | "h" | "i"
       | "j" | "k" | "l" | "m" | "n" | "o" | "p"
       | "q" | "r" | "s" | "t" | "u" | "v" | "w"
       | "x" | "y" | "z" ;
digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;
word = letter, {letter};
term = "(", word, " ", (word | term), ")";
"""

where I was attempting to parse the following;

match(p, "(NP (NNP Test))")

However, grammar does not compile and give StackOverflowError. Is this supposedly happen or am I missing something. Because, following EBNF grammar is working at https://mdkrajnak.github.io/ebnftest/;

Term = "(", Word, " ", (Word | Term), ")";
Word ::= Letter, {Letter}
Letter  ::= #'[A-Za-z]'
Number ::= Digits ( ( '.' | ',' ) Digits? )?
Digits ::= #'[0-9]+'

Any help or redirection would be helpful. Thanks in advance.

MethodError | is ambigious

Consider:

julia> using CombinedParsers

julia> using CombinedParsers.Regexp

julia> @with_names begin
           foo =  Either{Any}(Any["A", "B"])
           bar = re"b+"

           foobar = foo | bar
       end
ERROR: MethodError: |(::NamedParser{Either{Vector{Any}, Any, Any}, Any, Any}, ::NamedParser{Repeat{CombinedParsers.ConstantParser{Char, Char}, Int64, Vector{Char}}, Int64, Vector{Char}}) is ambiguous. Candidates:
  |(x::Union{Char, Regex, AbstractString, TextParse.AbstractToken}, y::Union{Char, Regex, AbstractString, TextParse.AbstractToken}) in CombinedParsers at /home/oxinabox/.julia/packages/CombinedParsers/T8bcG/src/operators.jl:123
  |(x, y::Union{Char, Regex, AbstractString, TextParse.AbstractToken}) in CombinedParsers at /home/oxinabox/.julia/packages/CombinedParsers/T8bcG/src/operators.jl:103
  |(x::TextParse.AbstractToken{T}, default::Union{Missing, T}) where T in CombinedParsers at /home/oxinabox/.julia/packages/CombinedParsers/T8bcG/src/operators.jl:144
Possible fix, define
  |(::TextParse.AbstractToken{T}, ::T) where T<:Union{Char, Regex, AbstractString, TextParse.AbstractToken}
Stacktrace:
 [1] top-level scope
   @ REPL[5]:5

It can be worked around by:

@with_names begin
    foo =  Either{Any}(Any["A", "B"])
    bar = re"b+"

    foobar = Either{Any}(Any[foo, bar])
end

dynamic `result_type` instead of type parameter

issue #30 pointed to possibly

refactor result_type to dynamic computation (not at construction time),
retain the option to assert a result_type with a WrappedParser.

TextParse.AbstractToken{T} compatibility required representing result_type as a type parameter.
This could be taken out of CombinedParsers though retaining compatibility with AbstractToken with a struct.

Maybe that also is a better option for state_type.

`join` `Repeat` parser result in no repetition.

The join function seems to remove the Repeat part. For example, the code in the docs parse(join(Repeat(AnyChar()),','),"a,b,c") cannot parse string like "abc,b,c"

Error vs sentinel

Sometimes people say that exceptions should only be raised for unexpected states in the program, but that a nonmatching parse is often an expected state so it should just return a sentinel value instead of raising an error. What do you think?

Need to fix a link in README.md

"the full documentation example" link in README.md is missing a "s" at the end -
https://github.com/gkappler/CombinedParsers.jl/blob/master/README.md

Details in [the full documentation example](https://gkappler.github.io/CombinedParsers.jl/dev/man/example-arithmetic).

Details in [the full documentation example](https://gkappler.github.io/CombinedParsers.jl/dev/man/example-arithmetics).

CombinedParsers >=0.1.2 broken for Julia 1.3.1 and 1.4.1 (works for 1.4.2, 1.5.0)

I had run some rudimentaly experiments with Julia 1.4.1 before but yesterday after updating CombinedParsers to 0.1.2 it crashes the REPL. Here is the error. (BTW, it works properly in 1.5)

(@v1.4) pkg> build CombinedParsers
   Building CodecZlib → `C:\Users\pmathur\.julia\packages\CodecZlib\5t9zO\deps\build.log`
   Building TimeZones → `C:\Users\pmathur\.julia\packages\TimeZones\v0mfN\deps\build.log`

(@v1.4) pkg> st CombinedParsers
Status `C:\Users\pmathur\.julia\environments\v1.4\Project.toml`
  [5ae71ed2] CombinedParsers v0.1.2
julia> using CombinedParsers
[ Info: Precompiling CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e]

signal (22): SIGABRT
in expression starting at REPL[4]:1
crt_sig_handler at /cygdrive/d/buildbot/worker/package_win64/build/src\signals-win.c:92
raise at C:\WINDOWS\System32\msvcrt.dll (unknown line)
abort at C:\WINDOWS\System32\msvcrt.dll (unknown line)
jl_intref at /cygdrive/d/buildbot/worker/package_win64/build/src\typemap.c:194 [inlined]
mtcache_hash_lookup at /cygdrive/d/buildbot/worker/package_win64/build/src\typemap.c:255 [inlined]
jl_typemap_intersection_visitor at /cygdrive/d/buildbot/worker/package_win64/build/src\typemap.c:545
jl_method_table_insert at /cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:1691
jl_insert_methods at /cygdrive/d/buildbot/worker/package_win64/build/src\dump.c:2267 [inlined]
_jl_restore_incremental at /cygdrive/d/buildbot/worker/package_win64/build/src\dump.c:3269
jl_restore_incremental at /cygdrive/d/buildbot/worker/package_win64/build/src\dump.c:3314
_include_from_serialized at .\loading.jl:681
_require_from_serialized at .\loading.jl:748
_require at .\loading.jl:1039
require at .\loading.jl:927
require at .\loading.jl:922
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
call_require at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:399 [inlined]
eval_import_path at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:436
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:656
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:764
jl_toplevel_eval at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:823 [inlined]
jl_toplevel_eval_in at /cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:843
eval at .\boot.jl:331
eval_user_input at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
macro expansion at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:118 [inlined]
#26 at .\task.jl:358
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
start_task at /cygdrive/d/buildbot/worker/package_win64/build/src\task.c:687
Allocations: 20912351 (Pool: 20908773; Big: 3578); GC: 20

jsoniq parser

Hi, your package is really cool!
I'm trying to implement the JSONiq core grammar. This grammar uses the EBNF except symbol, to exclude otherwise valid matches (the specification itself uses it once, but it references the XML specification in places where it uses the except operator). However, it appears that this package does not support excluding otherwise valid matches.
Would you be willing to add support for this? Or accept a PR (I don't know if I would manage it though 🙂, I'm new to Julia as well as parsing)?

`@enbf_str` macro to create a CombinedParser with EBNF syntax

Code works in REPL but errors with transformation type signature mismatch if in a package

The following code errors during precompile if I put it in a package and run Pkg.precompile() on it.
But works fine in the REPL.

using CombinedParsers
using CombinedParsers.Regexp
using CombinedParsers.TextParse: Numeric

"holds information on how to map an argument argment in a signature into a julia type"
struct Binder
    name::String
    type  # Intentionally left un-restricted for now. No restruction seems useful
    implicit::Bool
end

"Creates an `Array{T,N}``, if all elements of size are present then that array is initialized to hold `size`"
struct ArrayBuilder{T,N}
    size::Vector{Union{Symbol,Int}}
end
ArrayBuilder{T}(size) where T = ArrayBuilder{T,length(size)}(size)

    
@with_names begin
    name = map(Symbol, !re"arg\d+")
    scalar_type = Either{Any}(Any[
        parser("f32"=>Float32),
        parser("f64"=>Float64),
        parser("i32"=>Int32),
        parser("i64"=>Int64),
        parser("i8"=> Int8),
    ])
    
    size_ele = Numeric(Int) | name
    
    sizes = join(Repeat(size_ele),",")
    
    array_type = map(scalar_type * "[" * sizes * "]") do (T, _, sz, _)
        ArrayBuilder{T}(sz)
    end

    type = Either{Any}(Any[array_type, scalar_type])

    implicit = parser("?"=>true) | false

    arg_sig = map(implicit * name * ":" * type) do (i, n, _ , t)
        Binder(n, t, i)
    end
    arg_sigs = join(Repeat(arg_sig),",")
end

Error is:

[ Info: Precompiling ...]
ERROR: LoadError: LoadError: transformation type signature mismatch #10(Tuple{Bool, Symbol, SubString{String}, Any},)::Any[Union{}]<:Any
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] infer_result_type(::Function, ::Type, ::CombinedParsers.Sequence{Tuple{CombinedParsers.NamedParser{CombinedParsers.Optional{CombinedParsers.Transformation{CombinedParsers.Constant{Bool}, CombinedParsers.ConstantParser{String, SubString{String}}, CombinedParsers.MatchState, Bool}, Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Bool}, Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Bool}, CombinedParsers.NamedParser{CombinedParsers.Transformation{DataType, CombinedParsers.Transformation{CombinedParsers.MatchedSubSequence, CombinedParsers.Sequence{Tuple{CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.Repeat{CombinedParsers.ValueIn{Char, Set{Char}}, Int64, Vector{Char}}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Tuple{Char, Char, Char, Vector{Char}}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, SubString{String}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Symbol}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Symbol}, CombinedParsers.ConstantParser{String, SubString{String}}, CombinedParsers.NamedParser{CombinedParsers.Either{Vector{Any}, Any, Any}, Any, Any}}, Tuple{Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, CombinedParsers.MatchState, Any}, Tuple{Bool, Symbol, SubString{String}, Any}}, ::String; throw_empty_union::Bool)
    @ CombinedParsers ~/.julia/packages/CombinedParsers/vz0pA/src/transformation.jl:329
  [3] map(::Function, ::CombinedParsers.Sequence{Tuple{CombinedParsers.NamedParser{CombinedParsers.Optional{CombinedParsers.Transformation{CombinedParsers.Constant{Bool}, CombinedParsers.ConstantParser{String, SubString{String}}, CombinedParsers.MatchState, Bool}, Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Bool}, Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Bool}, CombinedParsers.NamedParser{CombinedParsers.Transformation{DataType, CombinedParsers.Transformation{CombinedParsers.MatchedSubSequence, CombinedParsers.Sequence{Tuple{CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.Repeat{CombinedParsers.ValueIn{Char, Set{Char}}, Int64, Vector{Char}}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Tuple{Char, Char, Char, Vector{Char}}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, SubString{String}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Symbol}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Symbol}, CombinedParsers.ConstantParser{String, SubString{String}}, CombinedParsers.NamedParser{CombinedParsers.Either{Vector{Any}, Any, Any}, Any, Any}}, Tuple{Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, CombinedParsers.MatchState, Any}, Tuple{Bool, Symbol, SubString{String}, Any}}; throw_empty_union::Bool)
    @ CombinedParsers ~/.julia/packages/CombinedParsers/vz0pA/src/transformation.jl:278
  [4] map(::Function, ::CombinedParsers.Sequence{Tuple{CombinedParsers.NamedParser{CombinedParsers.Optional{CombinedParsers.Transformation{CombinedParsers.Constant{Bool}, CombinedParsers.ConstantParser{String, SubString{String}}, CombinedParsers.MatchState, Bool}, Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Bool}, Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Bool}, CombinedParsers.NamedParser{CombinedParsers.Transformation{DataType, CombinedParsers.Transformation{CombinedParsers.MatchedSubSequence, CombinedParsers.Sequence{Tuple{CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.ConstantParser{Char, Char}, CombinedParsers.Repeat{CombinedParsers.ValueIn{Char, Set{Char}}, Int64, Vector{Char}}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Tuple{Char, Char, Char, Vector{Char}}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, SubString{String}}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Symbol}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, Symbol}, CombinedParsers.ConstantParser{String, SubString{String}}, CombinedParsers.NamedParser{CombinedParsers.Either{Vector{Any}, Any, Any}, Any, Any}}, Tuple{Union{CombinedParsers.MatchState, CombinedParsers.NoMatch}, Tuple{CombinedParsers.MatchState, CombinedParsers.MatchState, CombinedParsers.MatchState, Int64}, CombinedParsers.MatchState, Any}, Tuple{Bool, Symbol, SubString{String}, Any}})
    @ CombinedParsers ~/.julia/packages/CombinedParsers/vz0pA/src/transformation.jl:278

I am currently trying to find a MWE

Requiring code to infer is problematic
Base.return_type isn't part of julia's public API.
I think this is probably related to that.

`ParserCombinator.Delayed` equivalent?

There is a Delayed type in ParserCombinator.jl which allow us to define a loop in the grammar. What do we use in CombinedParser to achieve the same functionality?

Repeat_stop and Repeat_until are taking default Repeat_max values

When parsing data with Repeat_stop and Repeat_until, CombinedParsers gives up after 10^6 repeats. Parsing is aborted after that, even if more data is left. For example,

dat = [repeat([UInt8(0x61)],10^6+10)...,UInt8(0x62)];
p = Repeat_stop(byte, CharIn(UInt8(0x62)))
parse(p,dat)
1000000-element Array{UInt8,1}:
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
    ⋮
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61
 0x61

As you can see it only saw the first 10^6 values and then gave up.

To my understanding the problem is caused because of when declaring Repeat_stop(L1389) the arguments are just passed to Repeat() which causes it to take default max=Repeat_max which is 10^6. I propose we just expose the optional arguments to Repeat_stop and Repeat_until and that should just solve the problem. If you agree then I can make a pull request.

Difference between CharIn and Either

What is the difference between CharIn and Either? CharIn isn't restricted to Char types, so that might not be the best name, but maybe both CharIn and Either could be OneOf?

Parsing nonstring data

Sometimes I want to parse nonstring data, but I don't think that's allowed currently.

using CombinedParsers

parse(parser('a'), ['a']) # 'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)

parse(parser(1), [1])

# ERROR: LoadError: MethodError: no method matching _iterate(::Int64, ::Array{Int64,1}, ::Int64, ::Int64, ::Int64, ::Nothing, ::Int64)
# Closest candidates are:
#   _iterate(::CombinedParsers.Regexp.WithOptions{Char}, ::Any, ::Any, ::Any, ::Any, ::Nothing, ::Any)
#   _iterate(::Char, ::Any, ::Any, ::Any, ::Any, ::Nothing, ::Any)
#   _iterate(::AbstractString, ::Any, ::Any, ::Any, ::Any, ::Nothing, ::Any)
#   ...

ParserCombinator.jl says

It can parse any iterable type (not just strings)

but I heard that package is mostly unmaintained.

Bootstrapping parsers

Really nice work on the package! It looks like the best parser generator for Julia at the moment. Here's a few ideas I had about bootstrapping new parsers:

Import grammar.json files generated by treesitter
Parse Lark's grammar possibly using https://github.com/jamesrhester/Lerche.jl
Create a parser for the antlr4 grammar format. They have a huge collection of grammars in that repo.

How would you write an importer for those grammar files (i.e. generate CombinedParser.jl code based on the above grammar specifications to develop custom interpreters)?

Plans for the proprietary parsers

The proprietary parsers (TypeGraphs.jl, GraphQLAlchemy.jl, TypeDB.jl and FilingForest.jl) sound interesting. Could you expand on them a bit more? What are your plans for their future? I updated gRPC.jl to work with current Julia (not published yet). I'm interested in using it with the Dgraph database, so I'm particularly interested in GraphQLAlchemy.

Lazy parsing on a packet stream

I am trying to parse a binary file format which is made up of packets. I have figured out the packet grammar, and if I have a file with the stream saved in it I can parse all packets from it. But this is incredibly slow as each file can be several GBs long. To speed it up I was wondering if I can only parse specific packets selected by their ID which is right at the top of the structure. A minimum example of a packet would look like this

MAGIC NUM
ID: ID1
DATA1: VALUE 1
DATA2: VALUE 2

I want to parse till ID and then decide if I want to parse any further.

One approach I tried was to write a header parser. But the problem is since I don't know the size of the packets, I end up writing a negative look ahead for the magic number. This is equally as slow as just parsing the struct (this does feel like one of those good problems doesn't it :P).

Type inference/checking for builtins

The internal logic doesn't accept Core.Builtin types, for which return_types fails:

julia> map(tuple, NumericParser(Int))
ERROR: ArgumentError: argument is not a generic function
Stacktrace:
 [1] return_types(f::Any, types::Any, interp::Core.Compiler.NativeInterpreter)
   @ Base ./reflection.jl:1238
 [2] return_types
   @ ./reflection.jl:1236 [inlined]
 [3] infer_result_type(::Function, ::Type, ::NumericParser{TextParse.Numeric{Int64}, Int64}, ::String; throw_empty_union::Bool)
   @ CombinedParsers ~/.julia/packages/CombinedParsers/kbHhW/src/transformation.jl:330
 [4] map(::Function, ::NumericParser{TextParse.Numeric{Int64}, Int64}; throw_empty_union::Bool)
   @ CombinedParsers ~/.julia/packages/CombinedParsers/kbHhW/src/transformation.jl:284
 [5] map(::Function, ::NumericParser{TextParse.Numeric{Int64}, Int64})
   @ CombinedParsers ~/.julia/packages/CombinedParsers/kbHhW/src/transformation.jl:284
 [6] top-level scope
   @ REPL[6]:1

Expected result: a parser with return type Tuple{Int}.

(I know that map(x -> tuple(x), NumericParser(Int)) works instead, but that's rather redundant...)

`join` doesn't work with repeat minimum of 0

I want to parse zero or more things, separated by commas. However, join always includes at least one repetition:

julia> Repeat("a", max=2)("")
SubString{String}[]

julia> Repeat("a", max=2)("a")
1-element Vector{SubString{String}}:
 "a"

julia> join(Repeat("a", max=2), ",")
🗄 Sequence |> map(#90)
├─ a 
└─ ,a{,1} Sequence |> map(#68) |> Repeat
::Vector{SubString{String}}


julia> join(Repeat("a", max=2), ",")("a")
1-element Vector{SubString{String}}:
 "a"

julia> join(Repeat("a", max=2), ",")("")
ERROR: ArgumentError: no successfull parsing.
...

Precompilation failure on Julia 1.10.x

Subject says it all.

Precompiling project...
  ✗ CombinedParsers
  0 dependencies successfully precompiled in 26 seconds. 142 already precompiled.
  1 dependency errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

Checking the error gives back this.

PkgPrecompileError: The following 1 direct dependency failed to precompile:

CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e]

Failed to precompile CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e] to "C:\\Users\\marco\\.julia\\compiled\\v1.10\\CombinedParsers\\jl_75B0.tmp".
ERROR: LoadError: Nothing <: CombinedParsers.CombinedParser. Fix with `push!(x|CombinedParsers.ConstantParser{Nothing, Nothing},y)`.
re""
Stacktrace:
  [1] error(s::String)
    @ Base .\error.jl:35
  [2] pushfirst!(x::CombinedParsers.Either{Vector{Any}, Any, CombinedParsers.CombinedParser}, y_::Nothing)
    @ CombinedParsers C:\Users\marco\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:2003
  [3] pushfirst!(x::CombinedParsers.NamedParser{CombinedParsers.Either{Vector{Any}, Any, CombinedParsers.CombinedParser}, Any, CombinedParsers.CombinedParser}, y::Nothing)
    @ CombinedParsers C:\Users\marco\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:2025
  [4] top-level scope
    @ C:\Users\marco\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:638
  [5] include(mod::Module, _path::String)
    @ Base .\Base.jl:495
  [6] include(x::String)
    @ CombinedParsers C:\Users\marco\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:13
  [7] top-level scope
    @ C:\Users\marco\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:2325
  [8] include
    @ .\Base.jl:495 [inlined]
  [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base .\loading.jl:2222
 [10] top-level scope
    @ stdin:3
in expression starting at C:\Users\marco\.julia\packages\CombinedParsers\kbHhW\src\bnf.jl:1
in expression starting at C:\Users\marco\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:6
in expression starting at stdin:

This is on Julia 1.0.x (running on Windows, same on Mac OS).

The situation is even worse if you have several SciMl.ai packages installed. Many get downgraded, but the installation still fails.

All the best

Marco

Parsing binary formats

Is there a way to use this to parse binary based formats (similar to BSON or MsgPack?). From what I understand it seems like this library works only on purely text format, can't find a way to make a parser which takes in bytes/uint8

Help with parsing files with separators

I don't even know how to describe this issue. But I will try my best. I have reduced my problem to as small as possible, but if you need details feel free to ask.

Problem: Lets say I have to parse a binary file in which each dataframe is separated by 0x0203. Its that separator and then a bunch of binary values which are not relevant to this discussion. I wrote code to mock this,

byte = Bytes(1,UInt8)
@syntax t = Sequence("\\x", integer_base(16,2,2)) do v
    CharIn(convert(UInt8,v[2]))
end

@syntax sep = Sequence(t"\x02",t"\x03")
@syntax data = Repeat_stop(byte,sep)

p = Repeat(Sequence(sep, data))
dat = Array{UInt8}([2,3,1,1,1,1,1,1,2,3,11,1,1,])

parse(p,dat)
ERROR: BoundsError: attempt to access 13-element Array{UInt8,1} at index [14:14]
Stacktrace:
 [1] throw_boundserror(::Array{UInt8,1}, ::Tuple{UnitRange{Int64}}) at .\abstractarray.jl:541
 [2] checkbounds at .\abstractarray.jl:506 [inlined]
 [3] getindex at .\array.jl:815 [inlined]
 [4] get(::Bytes{UInt8}, ::Array{UInt8,1}, ::Int64, ::Int64, ::Int64, ::MatchState) at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:31
 [5] macro expansion at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:229 [inlined]
 [6] get at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:199 [inlined]
 [7] get at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:378 [inlined]
 [8] get(::Repeat{Transformation{IndexAt{Int64},Sequence{Tuple{NegativeLookahead{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}}},Bytes{UInt8}},MatchState,Tuple{NegativeLookahead{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}}},UInt8}},MatchState,UInt8},Int64,Array{UInt8,1}}, ::Array{UInt8,1}, ::Int64, ::Int64, ::Int64, ::Int64) at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:179
 [9] get at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:9 [inlined]
 [10] macro expansion at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:229 [inlined]
 [11] get(::Sequence{Tuple{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}},NamedParser{Repeat{Transformation{IndexAt{Int64},Sequence{Tuple{NegativeLookahead{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}}},Bytes{UInt8}},MatchState,Tuple{NegativeLookahead{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}}},UInt8}},MatchState,UInt8},Int64,Array{UInt8,1}},Int64,Array{UInt8,1}}},Array{Any,1},Tuple{Tuple{Char,Char},Array{UInt8,1}}}, ::Array{UInt8,1}, ::Int64, ::Int64, ::Int64, 
::Array{Any,1}) at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:199
 [12] get(::Repeat{Sequence{Tuple{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}},NamedParser{Repeat{Transformation{IndexAt{Int64},Sequence{Tuple{NegativeLookahead{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}}},Bytes{UInt8}},MatchState,Tuple{NegativeLookahead{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}}},UInt8}},MatchState,UInt8},Int64,Array{UInt8,1}},Int64,Array{UInt8,1}}},Array{Any,1},Tuple{Tuple{Char,Char},Array{UInt8,1}}},Array{Array{Any,1},1},Array{Tuple{Tuple{Char,Char},Array{UInt8,1}},1}}, ::Array{UInt8,1}, ::Int64, ::Int64, ::Int64, ::Array{Array{Any,1},1}) at C:\Users\pmathur\.julia\dev\CombinedParsers\src\get.jl:165
 [13] get at C:\Users\pmathur\.julia\dev\CombinedParsers\src\match.jl:286 [inlined]
 [14] get at C:\Users\pmathur\.julia\dev\CombinedParsers\src\match.jl:283 [inlined]
 [15] tryparse at C:\Users\pmathur\.julia\dev\CombinedParsers\src\match.jl:237 [inlined]
 [16] tryparse at C:\Users\pmathur\.julia\dev\CombinedParsers\src\match.jl:235 [inlined]
 [17] #parse#133 at C:\Users\pmathur\.julia\dev\CombinedParsers\src\match.jl:224 [inlined]
 [18] parse(::Repeat{Sequence{Tuple{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}},NamedParser{Repeat{Transformation{IndexAt{Int64},Sequence{Tuple{NegativeLookahead{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}}},Bytes{UInt8}},MatchState,Tuple{NegativeLookahead{NamedParser{Sequence{Tuple{CharIn{UInt8},CharIn{UInt8}},MatchState,Tuple{Char,Char}},MatchState,Tuple{Char,Char}}},UInt8}},MatchState,UInt8},Int64,Array{UInt8,1}},Int64,Array{UInt8,1}}},Array{Any,1},Tuple{Tuple{Char,Char},Array{UInt8,1}}},Array{Array{Any,1},1},Array{Tuple{Tuple{Char,Char},Array{UInt8,1}},1}}, ::Array{UInt8,1}) at C:\Users\pmathur\.julia\dev\CombinedParsers\src\match.jl:224
 [19] top-level scope at REPL[88]:1

It seems to be a binary problem, because I can get it to work if working with chars

@syntax sep = parser("ab");
@syntax data = Repeat_stop(AnyChar(),sep)
p = Repeat(Sequence(sep, data))
parse(p,"abccccccccabccccccc")
2-element Array{Tuple{SubString,Array{Char,1}},1}:
 ("ab", ['c', 'c', 'c', 'c', 'c', 'c', 'c', 'c'])
 ("ab", ['c', 'c', 'c', 'c', 'c', 'c', 'c'])

On "Example: rational numbers arithmetics"

I just learned about this package in the list of Juliacon talks, and wanted to have a look, as it sounds really cool. Unfortunately, I didn't quite understand the example in the README named in the title of this issue.; so I am somewhat relieved to see that Julia apparently also doesn't understand it ;-).

Indeed https://gkappler.github.io/CombinedParsers.jl/dev/ contains this right now:

julia> result_type(term)
ERROR: UndefVarError: term not defined

julia> term"(1+2)/5"
ERROR: LoadError: UndefVarError: @term_str not defined
in expression starting at none:1

julia> # The defined `CombinedParser` `term` function
       # provides optional logging of the parsing process.
       term("1/((1+2)*4+3*(5*2))",log = [:parenthesis])
ERROR: UndefVarError: term not defined

(Personally, I never use Documenter's @repl anymore, instead only jldoctest, which IMHO has two advantages: 1. it makes it easy to read the examples in the raw Markdown; 2. it serves as "free" extra test; if something changes in your output, you normally want to know it (even though that of course adds some extra work if the change is intentional).

I also cannot get it to work locally in Julia 1.4.2:

julia> using CombinedParsers
[ Info: Precompiling CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e]

julia> using TextParse

julia> @syntax subterm = Either{Rational{Int}}(TextParse.Numeric(Int));
ERROR: TypeError: in Tuple, in parameter, expected Type, got Tuple{DataType}
Stacktrace:
 [1] either_state_type at /Users/mhorn/.julia/packages/CombinedParsers/ZtE2s/src/CombinedParsers.jl:2352 [inlined]
 [2] Either{Rational{Int64},S,T} where T where S(::TextParse.Numeric{Int64}) at /Users/mhorn/.julia/packages/CombinedParsers/ZtE2s/src/CombinedParsers.jl:2293
 [3] top-level scope at /Users/mhorn/.julia/packages/CombinedParsers/ZtE2s/src/CombinedParsers.jl:1073

Parsing algorithms for ambiguities

There are various parsing algorithms that have different properties. Earley parsers can see if a string matches the grammar in multiple different ways and return all of the options. That's mentioned briefly in the ParserCombinator.jl docs and is supported by Python's Lark.

tried it with Julia 1.9-DEV

julia> using CombinedParsers
[ Info: Precompiling CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e]
Illegal inttoptr
          %magicptr = ptrtoint {} addrspace(10)* %131 to i64, !dbg !27

signal (22): SIGABRT
in expression starting at C:\Users\MrJSa\.julia\packages\CombinedParsers\kbHhW\src\re-parser.jl:42
crt_sig_handler at /cygdrive/c/buildbot/worker/package_win64/build/src\signals-win.c:93
raise at C:\WINDOWS\System32\msvcrt.dll (unknown line)
abort at C:\WINDOWS\System32\msvcrt.dll (unknown line)
runOnFunction at /cygdrive/c/buildbot/worker/package_win64/build/src\llvm-gc-invariant-verifier.cpp:209
.text$_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
operator() at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:947 [inlined]
withModuleDo<(anonymous namespace)::OptimizerT::operator()(llvm::orc::ThreadSafeModule, llvm::orc::MaterializationResponsibility&)::<lambda(llvm::Module&)> > at /cygdrive/c/buildbot/worker/package_win64/build/usr/include/llvm/ExecutionEngine/Orc\ThreadSafeModule.h:136 [inlined]
operator() at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:919 [inlined]
CallImpl<(anonymous namespace)::OptimizerT> at /cygdrive/c/buildbot/worker/package_win64/build/usr/include/llvm/ADT\FunctionExtras.h:216
.text$_ZN4llvm3orc16IRTransformLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
emit at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:522
.text$_ZN4llvm3orc31BasicIRLayerMaterializationUnit11materializeESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc19MaterializationTask3runEv at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
_ZN4llvm6detail18UniqueFunctionBaseIvJSt10unique_ptrINS_3orc4TaskESt14default_deleteIS4_EEEE8CallImplIPFvS7_EEEvPvRS7_ at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc16ExecutionSession22dispatchOutstandingMUsEv at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc16ExecutionSession17OL_completeLookupESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EESt10shared_ptrINS0_23AsynchronousSymbolQueryEESt8functionIFvRKNS_8DenseMapIPNS0_8JITDylibENS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISF_EEEENSG_ISD_EENS_6detail12DenseMapPairISD_SI_EEEEEE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
_ZN4llvm3orc25InProgressFullLookupState8completeESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc16ExecutionSession19OL_applyQueryPhase1ESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EENS_5ErrorE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc16ExecutionSession6lookupENS0_10LookupKindERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS8_EENS0_15SymbolLookupSetENS0_11SymbolStateENS_15unique_functionIFvNS_8ExpectedINS_8DenseMapINS0_15SymbolStringPtrENS_18JITEvaluatedSymbolENS_12DenseMapInfoISI_EENS_6detail12DenseMapPairISI_SJ_EEEEEEEEESt8functionIFvRKNSH_IS6_NS_8DenseSetISI_SL_EENSK_IS6_EENSN_IS6_SV_EEEEEE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EERKNS0_15SymbolLookupSetENS0_10LookupKindENS0_11SymbolStateESt8functionIFvRKNS_8DenseMapIS5_NS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISK_EEEENSL_IS5_EENS_6detail12DenseMapPairIS5_SN_EEEEEE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EENS0_15SymbolStringPtrENS0_11SymbolStateE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS0_15SymbolStringPtrENS0_11SymbolStateE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
.text$_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS_9StringRefENS0_11SymbolStateE at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\bin\libLLVM-13jl.dll (unknown line)
addModule at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:1148
jl_add_to_ee at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:1519
jl_add_to_ee at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:1541 [inlined]
_jl_compile_codeinst at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:162
jl_generate_fptr_impl at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:357
jl_compile_method_internal at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2106
jl_compile_method_internal at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2050 [inlined]
_jl_invoke at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2384 [inlined]
ijl_apply_generic at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2574
#Sequence#47 at C:\Users\MrJSa\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:993
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1841 [inlined]
do_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:730
Sequence at C:\Users\MrJSa\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:987
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1841 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:126
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:215
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:166 [inlined]
eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:594
jl_interpret_toplevel_thunk at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:750
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:912
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:856
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:856
ijl_toplevel_eval at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1349
_include at .\loading.jl:1409
include at .\Base.jl:428
jfptr_include_39935.clone_1 at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1841 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:774
include at C:\Users\MrJSa\.julia\packages\CombinedParsers\kbHhW\src\re.jl:4
unknown function (ip: 000000005f3913d6)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1841 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:126
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:215
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:166 [inlined]
eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:594
jl_interpret_toplevel_thunk at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:750
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:912
jl_eval_module_expr at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:203 [inlined]
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:715
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:856
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:856
ijl_toplevel_eval at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1349
_include at .\loading.jl:1409
include at .\Base.jl:428
jfptr_include_39935.clone_1 at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1841 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:774
include at C:\Users\MrJSa\.julia\packages\CombinedParsers\kbHhW\src\CombinedParsers.jl:13
unknown function (ip: 000000005f2f1e56)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1841 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:126
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:215
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:166 [inlined]
eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:594
jl_interpret_toplevel_thunk at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:750
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:912
jl_eval_module_expr at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:203 [inlined]
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:715
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:856
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:856
ijl_toplevel_eval at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1349
_include at .\loading.jl:1409
include at .\Base.jl:428 [inlined]
include_package_for_output at .\loading.jl:1475
jfptr_include_package_for_output_26996.clone_1 at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1841 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:126
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:215
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:166 [inlined]
eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:594
jl_interpret_toplevel_thunk at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:750
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:912
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:856
ijl_toplevel_eval at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1349
include_string at .\loading.jl:1359
exec_options at .\client.jl:297
_start at .\client.jl:516
jfptr__start_30067.clone_1 at C:\Users\MrJSa\AppData\Local\Programs\Julia-1.9\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1841 [inlined]
true_main at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:567
jl_repl_entrypoint at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:711
mainCRTStartup at /cygdrive/c/buildbot/worker/package_win64/build/cli\loader_exe.c:59
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 17691130 (Pool: 17684746; Big: 6384); GC: 22
ERROR: Failed to precompile CombinedParsers [5ae71ed2-6f8a-4ed1-b94f-e14e8158f19e] to C:\Users\MrJSa\.julia\compiled\v1.9\CombinedParsers\jl_25DA.tmp.
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base .\loading.jl:1626
 [3] compilecache
   @ .\loading.jl:1570 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base .\loading.jl:1271
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base .\loading.jl:1144
 [6] macro expansion
   @ .\loading.jl:1124 [inlined]
 [7] macro expansion
   @ .\lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base .\loading.jl:1088

Extreme precompilation time

I am getting some very long precompilation time.
and AFAICT it is just my 1 parser

  1 dependency successfully precompiled in 46 seconds (39 already precompiled)

CombinedParsers.jl is my only direct dependency,
The parser in question is:

struct Binder
    name::Symbol
    type
    implicit::Bool
end

struct ArrayBuilder{T,N}
    size::Vector{Union{Symbol,Int}}
end
ArrayBuilder{T}(size) where T = ArrayBuilder{T,length(size)}(size)

@with_names begin
    name = map(Symbol, !re"arg\d+")
    scalar_type = Either{Any}(Any[
        parser("f32"=>Float32),
        parser("f64"=>Float64),
        parser("i32"=>Int32),
        parser("i64"=>Int64),
        parser("i8"=>Int8),
    ])
    size_ele = Numeric(Int) | name
    sizes = join(Repeat(size_ele),",")
    array_type = map(scalar_type * "[" * sizes * "]") do (T, _, sz, _)
        ArrayBuilder{T}(sz)
    end
    type = Either{Any}(Any[array_type, scalar_type])
    implicit = parser("?"=>true) | false
    arg_sig = map(implicit * name * ":" * type) do (i, n, _ , t)
        Binder(n, t, i)
    end
    arg_sigs = join(Repeat(arg_sig),",")
end