add field error_type (enum) to struct hs_compile_error about hyperscan HOT 3 OPEN

This would be a fairly significant change. The Hyperscan compiler can return a lot of different error messages -- consider everything that can be produced by the parser, for example!

It may also require an ABI change, and we try to limit those as much as possible to allow for easy library upgrades.

We view pretty much any error result from compilation as fatal, and generally the only code we see that needs to retry compilation under any circumstances tends to be prefiltering code that tries for full Hyperscan support first, then switches on HS_FLAG_PREFILTER for a second attempt if that fails.

Can you describe your application and what you are trying to do as a result of "Pattern too large"? If you're able to share the patterns that you are seeing issues with, we would love to see them with a view to improving pattern support in general.

from hyperscan.

We view pretty much any error result from compilation as fatal, and generally the only code we see that needs to retry compilation under any circumstances tends to be prefiltering code that tries for full Hyperscan support first, then switches on HS_FLAG_PREFILTER for a second attempt if that fails.

I want to distinguish two cases:

• pattern is incorrect. Print error message and abort the program.
• Pattern is correct, but too large for Hyperscan. Pass it to other regex library which is slower but accepts the pattern.

Can you describe your application and what you are trying to do as a result of "Pattern too large"? If you're able to share the patterns that you are seeing issues with, we would love to see them with a view to improving pattern support in general.

I uploaded the pattern and the testing tool to gist. The pattern matches post address in Russia. Original pattern (written for other library) is stored in file original-pattern.txt. Then I applied pattern preprocessor which takes patterns written for that library and produces patterns for hyperscan. I saved the result to processed-pattern.txt. Hyperscan accepts first pattern, but it fails with "Pattern is too large" on preprocessed pattern. I have to preprocess regexps to achieve compatibility with other regexp library. Syntax and meaning of some character sets in that library differ from ones of Hyperscan, so the only reliable way to use hyperscan in place of that library is to preprocess a pattern. Results of preprocessing reproduce original behaviour precisely, but are sometimes too large for Hyperscan.

from hyperscan.

Thanks for the extra detail. (We will also take a look at this pattern and see what we can do to improve support for it.)

Hyperscan is able to generate a very large number of compile errors, and we would prefer not to lose this expressiveness by hardening them into enums.

Assuming the semantics of both libraries are the same (as you say), are you sure you need to distinguish these cases -- can't you just always fall back to your alternative library if a pattern fails compilation with Hyperscan?

One possibility is that you could call hs_expression_info() first, which parses the expression and does some preliminary analysis -- this will generate errors for expressions with invalid syntax without going through the full compile process.

from hyperscan.