marcomaggi / cre2 Goto Github PK
View Code? Open in Web Editor NEWC language wrapper for RE2 the regular expressions library from Google
License: Other
C language wrapper for RE2 the regular expressions library from Google
License: Other
./configure: line 1873: MMUX_INIT: command not found
./configure: line 1874: syntax error near unexpected token cre2,0,0,0' ./configure: line 1874:
MMUX_LIBTOOL_LIBRARY_VERSIONS(cre2,0,0,0)'
Hi,
I found a crash when fuzzing the cre2 by a synthetical harness. The sample code is listed bellow.
The internal implementation of cre2_set_compile()
returns false if prog_
is nullptr, true for success.
However, the wrapper cre2_set_compile
cast the bool
return type to int
, which changes the original
meanings of its internal implementation and may be confusing to developers.
The result is taht the after use of cre2_set_match() will dereference the null prog_
and cause crash, if without checking
the returned confusing int type variable v28
is true.
As the raw meanings fo cre2_set_compile()
is returning ture for success and false for failure, the return type of cre2_set_compile
should be bool
not int
.
#include "/home/loydlv/vbd/hopper_bench/re2/cre2.h"
#include <stdlib.h>
#include <string.h>
typedef int64_t i64;
int main() {
void *v0 = cre2_opt_new(); // opt
if (v0 == NULL) return 0;
void *v2 = v0; // opt
cre2_anchor_t v3 = { 288053034, }; // anchor
i64 v4 = 640LL; // m
cre2_opt_set_max_mem(v2, v4); // $relative
cre2_set *v6 = cre2_set_new(v2, v3); // set
if (v6 == NULL) return 0;
cre2_set *v8 = v6; // set
char v9_tmp[] = {77, 92, 80, 123, 77, 99, 125, 43, 50, 71, 58, 73, 8, 90, 102, 82, 108, 62, 50, 0, 0, 0, 0, 0, }; // pattern
char *v9 = malloc(sizeof v9_tmp);
memcpy(v9, v9_tmp, sizeof v9_tmp);
char *v10 = v9; // pattern
int v27 = cre2_set_add_simple(v8, v10); // $relative
int v28 = cre2_set_compile(v8); // $relative
int v31_tmp[] = {-801400760, -1308314430, 1402451838, -960879144, 1974520777, -552581394, 1195814910, -595774769, 994109953, -1768353690, -500381521, -1947593372, -1051774523, -907454282, 1014428481, -2110187081, 633553943, 449537480, 1814070959, 1886959844, 1448186699, 0, }; // match_
int *v31 = malloc(sizeof v31_tmp);
memcpy(v31, v31_tmp, sizeof v31_tmp);
int *v32 = v31; // match_
int v34 = cre2_set_match(v8, NULL, 0, v32, 22); // $relative
cre2_set_delete(v8); // $target
}
Hi,
we found a crash in cre2_find_named_capturing_groups
by using fuzzing.
When analyzing the crash dump, the program crashed at
const std::map<std::string, int>& m = TO_CONST_RE2(re)->NamedCapturingGroups();
.
Here is the trigger program to help you understand and fix this crash,
looking forward to your early reply or fix, thanks.
#include "cre2.h"
#include <stdbool.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <math.h>
typedef uint8_t u8;
typedef uint16_t u16;
typedef uint32_t u32;
typedef uint64_t u64;
typedef int8_t i8;
typedef int16_t i16;
typedef int32_t i32;
typedef int64_t i64;
typedef float f32;
typedef double f64;
int main() {
i8 *v0 = NULL; // pattern
i32 v1 = 0; // pattern_len
void *v2 = cre2_opt_new(); // opt
if (v2 == NULL) return 0;
void *v4 = v2; // opt
void *v5 = cre2_new(v0, v1, v4); // re
if (v5 == NULL) return 0;
void *v7 = v5; // re
i8 v8_tmp[] = {6, 6, -21, -118, -20, -37, -114, -120, -30, -56, -3, 3, 69, -46, 114, 118, 99, 76, 117, -83, -76, 119, 11, 25, -42, 127, 0, }; // name
i8 *v8 = malloc(sizeof v8_tmp);
memcpy(v8, v8_tmp, sizeof v8_tmp);
i8 *v9 = v8; // name
i8 v10_tmp[] = {74, 21, 46, -6, -50, -114, 0, 109, 109, 22, -41, -57, 37, -125, -2, -92, 120, 110, 55, 42, -110, 43, -73, -28, -93, 28, 79, 0, }; // pattern
i8 *v10 = malloc(sizeof v10_tmp);
memcpy(v10, v10_tmp, sizeof v10_tmp);
i8 *v11 = v10; // pattern
i32 v12 = 28; // pattern_len
void *v13 = cre2_new(v11, v12, v4); // re
if (v13 == NULL) return 0;
void *v15 = v13; // re
cre2_named_groups_iter_t *v16 = cre2_named_groups_iter_new(v15); // iter
if (v16 == NULL) return 0;
cre2_named_groups_iter_t *v18 = v16; // iter
i32 v19_tmp[] = {511718782, 19157193, 12820848, -1277658995, -1778605620, -374704804, 545222913, 276302240, 2028354637, 1332063271, 1973481400, 1532375303, 69417297, -465807661, -1391787334, -1573236937, -1162225574, -1519906975, -857038784, 0, }; // index
i32 *v19 = malloc(sizeof v19_tmp);
memcpy(v19, v19_tmp, sizeof v19_tmp);
i32 *v20 = v19; // index
i8 **v21 = &v9; // name
bool v22 = cre2_named_groups_iter_next(v18, v21, v20); // $relative
i32 v23 = cre2_find_named_capturing_groups(v7, v9); // $target
}
hi marcomaggi,
configure.sh tries to call configure (without the .sh). This is fixed by removing the .sh extension from configure.sh
hi, here's the error i'm getting:
/Library/Developer/CommandLineTools/usr/bin/make all-am
depbase=`echo src/cre2.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ./libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -I../src -I/Users/xxx/cre2/build/../../re2 -g -O2 -MT src/cre2.lo -MD -MP -MF $depbase.Tpo -c -o src/cre2.lo ../src/cre2.cpp &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -I../src -I/Users/xxx/cre2/build/../../re2 -g -O2 -MT src/cre2.lo -MD -MP -MF src/.deps/cre2.Tpo -c ../src/cre2.cpp -fno-common -DPIC -o src/.libs/cre2.o
../src/cre2.cpp:204:29: error: variable length array of non-POD element type 're2::StringPiece'
re2::StringPiece match_re2[nmatch];
^
../src/cre2.cpp:293:1: error: variable length array of non-POD element type 're2::StringPiece'
DEFINE_MATCH_ZSTRING_FUN(cre2_full_match,FullMatchN)
^
../src/cre2.cpp:275:26: note: expanded from macro 'DEFINE_MATCH_ZSTRING_FUN'
re2::StringPiece strv[nmatch]; \
^
../src/cre2.cpp:293:1: error: variable length array of non-POD element type 'RE2::Arg'
../src/cre2.cpp:276:19: note: expanded from macro 'DEFINE_MATCH_ZSTRING_FUN'
RE2::Arg argv[nmatch]; \
^
../src/cre2.cpp:294:1: error: variable length array of non-POD element type 're2::StringPiece'
DEFINE_MATCH_ZSTRING_FUN(cre2_partial_match,PartialMatchN)
^
../src/cre2.cpp:275:26: note: expanded from macro 'DEFINE_MATCH_ZSTRING_FUN'
re2::StringPiece strv[nmatch]; \
^
../src/cre2.cpp:294:1: error: variable length array of non-POD element type 'RE2::Arg'
../src/cre2.cpp:276:19: note: expanded from macro 'DEFINE_MATCH_ZSTRING_FUN'
RE2::Arg argv[nmatch]; \
^
../src/cre2.cpp:324:1: error: variable length array of non-POD element type 're2::StringPiece'
DEFINE_MATCH_ZSTRING_FUN2(cre2_consume,ConsumeN)
^
../src/cre2.cpp:304:26: note: expanded from macro 'DEFINE_MATCH_ZSTRING_FUN2'
re2::StringPiece strv[nmatch]; \
^
../src/cre2.cpp:324:1: error: variable length array of non-POD element type 'RE2::Arg'
../src/cre2.cpp:305:19: note: expanded from macro 'DEFINE_MATCH_ZSTRING_FUN2'
RE2::Arg argv[nmatch]; \
^
../src/cre2.cpp:325:1: error: variable length array of non-POD element type 're2::StringPiece'
DEFINE_MATCH_ZSTRING_FUN2(cre2_find_and_consume,FindAndConsumeN)
^
../src/cre2.cpp:304:26: note: expanded from macro 'DEFINE_MATCH_ZSTRING_FUN2'
re2::StringPiece strv[nmatch]; \
^
../src/cre2.cpp:325:1: error: variable length array of non-POD element type 'RE2::Arg'
../src/cre2.cpp:305:19: note: expanded from macro 'DEFINE_MATCH_ZSTRING_FUN2'
RE2::Arg argv[nmatch]; \
^
../src/cre2.cpp:356:1: error: variable length array of non-POD element type 're2::StringPiece'
DEFINE_MATCH_REX_FUN(cre2_full_match_re,FullMatchN)
^
../src/cre2.cpp:338:26: note: expanded from macro 'DEFINE_MATCH_REX_FUN'
re2::StringPiece strv[nmatch]; \
^
../src/cre2.cpp:356:1: error: variable length array of non-POD element type 'RE2::Arg'
../src/cre2.cpp:339:19: note: expanded from macro 'DEFINE_MATCH_REX_FUN'
RE2::Arg argv[nmatch]; \
^
../src/cre2.cpp:357:1: error: variable length array of non-POD element type 're2::StringPiece'
DEFINE_MATCH_REX_FUN(cre2_partial_match_re,PartialMatchN)
^
../src/cre2.cpp:338:26: note: expanded from macro 'DEFINE_MATCH_REX_FUN'
re2::StringPiece strv[nmatch]; \
^
../src/cre2.cpp:357:1: error: variable length array of non-POD element type 'RE2::Arg'
../src/cre2.cpp:339:19: note: expanded from macro 'DEFINE_MATCH_REX_FUN'
RE2::Arg argv[nmatch]; \
^
../src/cre2.cpp:387:1: error: variable length array of non-POD element type 're2::StringPiece'
DEFINE_MATCH_REX_FUN2(cre2_consume_re,ConsumeN)
^
../src/cre2.cpp:367:26: note: expanded from macro 'DEFINE_MATCH_REX_FUN2'
re2::StringPiece strv[nmatch]; \
^
../src/cre2.cpp:387:1: error: variable length array of non-POD element type 'RE2::Arg'
../src/cre2.cpp:368:19: note: expanded from macro 'DEFINE_MATCH_REX_FUN2'
RE2::Arg argv[nmatch]; \
^
../src/cre2.cpp:388:1: error: variable length array of non-POD element type 're2::StringPiece'
DEFINE_MATCH_REX_FUN2(cre2_find_and_consume_re,FindAndConsumeN)
^
../src/cre2.cpp:367:26: note: expanded from macro 'DEFINE_MATCH_REX_FUN2'
re2::StringPiece strv[nmatch]; \
^
../src/cre2.cpp:388:1: error: variable length array of non-POD element type 'RE2::Arg'
../src/cre2.cpp:368:19: note: expanded from macro 'DEFINE_MATCH_REX_FUN2'
RE2::Arg argv[nmatch]; \
^
17 errors generated.
make[1]: *** [src/cre2.lo] Error 1
make: *** [all] Error 2
any ideas what is going wrong here?
this is the configure command i'm using: ../configure LDFLAGS="-L``pwd``/../../re2/obj" CPPFLAGS="-I``pwd``/../../re2"
, it passes alright and detects re2 properly
The cre2_find_named_capturing_groups
API allows converting a given capture group name to an index. Sometimes however this isn't enough and you don't know the names of all the capture groups ahead of time.
I propose an API for iterating over the dictionary of capture groups:
cre2_named_groups_iter* cre2_named_groups_iter_new(cre2_regexp_t* reg);
bool cre2_named_groups_iter_next(cre2_named_groups_iter* iter, char** name, int* index);
void cre2_named_groups_iter_delete(cre2_named_groups_iter* reg);
Is this something you'd be interested in a PR to support?
In file included from /usr/include/c++/4.8/mutex:35:0,
from /usr/local/include/re2/re2.h:187,
from conftest.cpp:57:
/usr/include/c++/4.8/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
#error This file requires compiler and library support
./doc/cre2.texi:7: @include: could not find version.texi
./doc/cre2.texi:65: warning: undefined flag: VERSION
./doc/cre2.texi:109: warning: undefined flag: VERSION
Makefile:1440: recipe for target 'doc/cre2.info' failed
make[1]: *** [doc/cre2.info] Error 1
make[1]: Leaving directory '/home/jack/code/cre2'
Makefile:963: recipe for target 'all' failed
make: *** [all] Error 2
This error:
cre2.texi:7: @include `version.texi': No such file or directory.
cre2.texi:467: warning: undefined flag: VERSION.
cre2.texi:497: warning: undefined flag: VERSION.
makeinfo: Removing output file `cre2.info' due to errors; use --force to preserve.
make[2]: *** [cre2.info] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
from trying to build from source using this shell script in zsh:
git clone git://github.com/marcomaggi/cre2.git
cd cre2
sh autogen.sh
./configure && \
make && \
make install
Hi marcomaggi, I wrote a test program that uses static libraries(libcre2.a) directly at compile time. The program terminates during execution. How do I use static libraries(libcre2.a) to compile programs?
this is my compile command:
g++ test.cc libcre2.a libre2.a -lpthread
thank you for your help!
Hey! I tried to build this on alpine linux but got this error:
../src/cre2.h:65:60: error: unknown type name 'int64_t'
and a few more of these. I applied a simple patch: https://gist.github.com/chmanie/e0d695b148b252723257eb204e53ace3
While this works for me, I don't know how this could affect other platforms, so I'm hoping to start a conversation on how to fix this for good.
Hi marcomaggi, Can you provide a cre2.a file to use like libre2.a?
i try cre2 in my centos computer, i spent a lot of time to "make", but it failed in different ways. So, could you please provide a "*.a" file link libre2.a?
Making install in src
../../install-sh -c -d '/usr/local/lib'
/bin/sh: ../../install-sh: Permission denied
make[2]: *** [install-libLTLIBRARIES] Error 1
make[1]: *** [install-am] Error 2
make: *** [install-recursive] Error 1
From running in a shell script:
curl -O cre2.googlecode.com/files/cre2-0.1b3.tar.gz && \
tar zxf cre2-0.1b3.tar.gz && \
rm cre2-0.1b3.tar.gz && \
cd cre2-0.1b3 && \
mkdir build && \
cd build && \
../configure && \
make && \
make install
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.