Comments (6)
I've been puzzling over this for a couple of hours.
I have three questions for you. I beg your pardon if they are very silly.
- Do you already have some lex that works on z/OS? Doesn't need to be Flex, just a lex that works in EBCDIC.
- Do you have a yacc or bison that works on z/OS?
- Did you define the {characters} pattern in the definitions section of your scanner? (If so, what is the definition?)
I have a guess: Your {characters} pattern isn't getting defined the way you expect. I see why you may need it. I'd try having Flex dump your scanner tables and see if your character classes look right. I suspect whichever class includes uppercase alphabetic characters also includes '}' and a bunch of undefined points between I and J.
I think the fix will be in src/parse.y. Near the beginning you'll find the CCL_EXPR macro that assumes isascii() returns true, which it won't. Near the end you'll find the ccl_expr rule that determines what the [:alpha:], etc. classes match. They depend on the CCL_EXPR macro, so they might not be working correctly. The range class definition is just above those and it's almost certainly wrong for EBCDIC, too.
from flex.
Wow, i did not expected any reaction to this ticket, while you have possibly already found problem :)
this is great.
- depends from company, everyone has own setup, as ZOS comes only with shell and few other unix utils. RocketSoftware has flex v2.5.4 ported by someone some years ago, nobody knows history where it came from :)
- yes, we have bison v3.0.4, came from same person who did flex. As well i have ported bison v3.3.2.
- yes, it is defined as
characters [a-zA-Z0-9]+[a-zA-Z0-9_]*
i have tested with \w+ - not working at all, as well with {[a-zA-Z0-9]+[a-zA-Z0-9_]*} instead of {characters}- same behaviour
To get flex v2.6.4 compiled, i have used flex v2.5.4 and bison v3.0.4.
flex v2.5.4 is used to compile same scanner.l, everything is working fine.
I do have access to sources from where flex v2.5.4 was built, but i have not found any suspicious changes or something worth attention.
I can try to compare official src/parse.y from flex v2.5.4 with what we have.
from flex.
Neat! I've seen mailing list chatter about a patch for EBCDIC during the 2.3 era but I couldn't find the sources.
The ranges like [a-z] are what's causing the problem. They are contiguous in ascii but broken up into 9 character blocks in EBCDIC. If you break them up further into contiguous sequences they should work.
Escape sequences like '\w' are defined similarly, but I forget which file they're in.
from flex.
okay, we are getting closer:
replacing the [a-zA-Z0-9]+[a-zA-Z0-9_]* by:
[abcdefghijklmnopqrstuvwxyABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789]+[abcdefghijklmnopqrstuvwxyABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_]*
solves issue
I have reviewed CCL_EXPR macro in 2.5.4 version, it is different, but copypaste this macro from our v2.5.4 into 2.6.4 did nothing.
from flex.
Did the warning about rules that can't be matched also go away?
Reading the z/OS 2.5 docs tonight. You probably don't need to worry about the CCL_EXPR macro or the use of functions like isalpha(). Looks the z/OS XL C/C++ library defines them in terms of the current locale (e.g. IBM-1047, ISO8859-1). Even isascii() is available with BSD semantics if you define the _XOPEN_SOURCE macro before including ctype.h.
I don't see isascii() in the z/OS Metal C library reference, but it would just test whether the argument fits in 7 bits. Something like:
int isascii(int c) {
return ((c & 0xFFFFFF80) == 0);
}
Adjust for sizeof(int), inline, etc.
from flex.
yes, warning went away.
docs for zos is here https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R5Library?OpenDocument
Particulalry you are interested in z/OS XL C/C++ Runtime Library Reference https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R5sc147314?OpenDocument
I do compile with :
-Wc,xplink -D_XPLATFROM_SOURCE=1 -DI370 -D_UNIX03_SOURCE -D_UNIX03_THREADS -D_POSIX_THREADS
Also config.h has:
#define _ALL_SOURCE 1
#define _XOPEN_SOURCE 600
This means, isascii() should behave like you normally expect.
metalC is only C, there is no library from IBM, you have to create your own functions, even malloc, etc. There is something called Callable Services, but it is out of this issue scope.
from flex.
Related Issues (20)
- Add Scorecard Action to monitor project's security posture HOT 2
- Broken yywrap()?
- Some situation where flex generated cpp code generates an access out-of-bounds HOT 2
- FYI: New warning for C++ lexer with gcc 13 -Wall
- BSD licensing while containing GPL-2.0 header in some files HOT 2
- Docs typo: C++ header file is FlexLexer.h not yyFlexLexer.h HOT 1
- aarch64-linux-gnu-gcc build flex2.6.4 will error HOT 2
- C-99 comment rule flawed HOT 13
- Inconsistent handling of in memory buffers compared to file buffers
- question HOT 1
- can't enlarge buffer, but no REJECT HOT 11
- Build without info/man/examples HOT 2
- Potential out-of-bounds access of `yy_nxt` HOT 7
- Buffer overflow with `-CF -8` options and 0x00 0xFF input HOT 23
- Fix sed incompatibilities on MacOS X HOT 10
- Cannot match non-ASCII characters HOT 1
- Flex segfaults after reading EOF in `input()` HOT 2
- Noob question: Where's the makefile? HOT 5
- crash when yylex HOT 1
- Where is the publishing GPG / PGP key? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flex.