Comments (13)
You're right - I think gobbling any trailing stars fixes the regex
"/*"([^*]|"*"[^/])*"*"+"/"
or to make it easier to read with whitespace ignored
(?x: "/*" ( [^*] | "*"[^/] )* "*"+"/" )
I'll make the change if you think this is right
from flex.
from flex.
Sorry - that had a typo - I corrected it above - it should have been
(?x: "/*" ( [^*] | "*"[^/] )* "*"+"/" )
This accepts
/* text ** more text */
But it has another problem - stemming from the original regex - where it accepts invalid comments like
/* comment 1 **/ invalid missing comment start */
This is because for the intermediate **/
input, "*"[^/]
consumes **
then [^*]
consumes /
and the lexer merrily continues to a successful match at the very last '*/`
from flex.
I feel this cannot be done with Flex regex - it seems to require some lookahead to avoid prematurely consuming the star from a comment end delimiter */
I'm actually surprised that every single basic regex solution I found online is wrong! Some of these posts are decades old
The Flex doc also has a FAQ on matching C comments - there it only has couple example patterns that are clearly labelled wrong and doesn't purport to offer a working regex - that's probably what needs to be done for this section too
The trailing context "*"/[^/]
you tried can't be used inside group parentheses - that's why you got the error - I've always avoided using it for this and other limitations
from flex.
from flex.
from flex.
from flex.
from flex.
Yep - this works - now the middle part of the regex “*”+[^*/]
only works for runs of stars that don't end at an end delimiter - this means an end delimiter can only be matched by the last part of the regex - and matching won't continue past the first end delimiter
Can test with grep
grep -Po '(?x: \/\* ([^*] | \*+[^*/])* \*+\/)' <<EOF
/* text ** more text */
/* some text **/
/* comment 1 *//* comment 2 */
/* comment 3 */ /* comment 4 */
/* comment 5 */
/* comment 6 */ invalid comment missing start after comment 6 */
/* comment 7 **/ invalid comment missing start after comment 7 */
/* comment 8 // **/ invalid comment missing start after comment 8 */
EOF
/* text ** more text */
/* some text **/
/* comment 1 */
/* comment 2 */
/* comment 3 */
/* comment 4 */
/* comment 5 */
/* comment 6 */
/* comment 7 **/
/* comment 8 // **/
and multiline comments
grep -z -Po '(?x: \/\* ([^*] | \*+[^*/])* \*+\/)' <<EOF
/* multiline
comment 9 */
EOF
/* multiline
comment 9 */
Incidentally the dotall flag (?s:)
is not needed since there's no dot in the regex
from flex.
The C++ comment regex doesn't account for newline escapes in the comment body either - it fails for
grep -z -Po '(?x: \/ (\\ \n)* \/ [^\n]* )' <<EOF
not comment before /\\
/ multiline split delimiter \\
> comment 1
> not comment after
> EOF
/\
/ multiline split delimiter \
Adding another match for escaped newlines after comment start fixes it
grep -z -Po '(?x: \/ (\\ \n)* \/ ( (\\ \n) | [^\n] )* )' <<EOF
not comment before /\\
/ multiline split delimiter \\
comment 1
not comment after
EOF
/\
/ multiline split delimiter \
comment 1
from flex.
So the full correct regex for C and C++ comments is
("/*"([^*]|"*"[^*/])*"*"+"/")|("/"(\\\n)*"/"((\\\n)|[^\n])*)
or
(?x: ( "/*" ( [^*] | "*"+ [^*/] )* "*"+ "/" ) | ( "/" (\\ \n)* "/" ( (\\ \n) | [^\n] )* ) )
from flex.
Fix in PR 614, #614
from flex.
fixed by #614
from flex.
Related Issues (20)
- Broken yywrap()?
- Some situation where flex generated cpp code generates an access out-of-bounds HOT 2
- FYI: New warning for C++ lexer with gcc 13 -Wall
- BSD licensing while containing GPL-2.0 header in some files HOT 2
- Docs typo: C++ header file is FlexLexer.h not yyFlexLexer.h HOT 1
- aarch64-linux-gnu-gcc build flex2.6.4 will error HOT 2
- Inconsistent handling of in memory buffers compared to file buffers
- question HOT 1
- can't enlarge buffer, but no REJECT HOT 11
- Build without info/man/examples HOT 2
- Potential out-of-bounds access of `yy_nxt` HOT 7
- Buffer overflow with `-CF -8` options and 0x00 0xFF input HOT 23
- Fix sed incompatibilities on MacOS X HOT 10
- Cannot match non-ASCII characters HOT 1
- Flex segfaults after reading EOF in `input()` HOT 2
- Noob question: Where's the makefile? HOT 5
- crash when yylex HOT 1
- Where is the publishing GPG / PGP key? HOT 2
- yy_scan_string() function string size using int HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flex.