Processing this chunk with lcpp:

#define foo \
   /* Comment here  \
        which spans lines */ \
   this should never appear in the output \
   but it does

causes string "this should never appear in the output but it does" to appear in the final output - whereas with cpp this does not happen.

processing this input:

return 0xff;

results in the following output:

return255;

Conversion to decimal is probably OK (though nominally preprocessor should not do it), but eating up the space before is a problem.

Testcase:

print(__FILE__)

The file name gets substituted without the double quotes.

The ## is the pre-processor token pasting operator.The left token is appended with the right token.

example:

define DECLARE_HANDLE(name) struct name##__ { int unused; }; typedef struct name##__ *name

Should expand to:
struct HINSTANCE__ { int unused; }; typedef struct HINSTANCE__ *HINSTANCE;

Processing this file:

#define MACRO(a) a

MACRO(((int)0))
MACRO((int)1)
MACRO(2)

results in output:

((int)0)
(int)
2

i.e. if a macro argument is preceded by a parenthesized expression, this expression itself is taken as an argument.

The following diff takes care of the problem:

diff --git a/lcpp.lua b/lcpp.lua
index 99daf75..4450dcd 100755
--- a/lcpp.lua
+++ b/lcpp.lua
@@ -1238,20 +1238,25 @@ local function replaceArgs(argsstr, repl)
        argsstr = argsstr:sub(2,-2)
        -- print('argsstr:'..argsstr)
        local comma
+       local arg_acc = ''^M
        for k, v, start, end_ in tokenizer(argsstr, LCPP_TOKENIZE_MACRO_ARGS) do
                -- print("replaceArgs:" .. k .. "|" .. v)
                if k == "ARGS" or k == "PARENTHESE" or k == "STRING_LITERAL" or
                        k == "FUNCTIONAL" or k == "SINGLE_CHARACTER_ARGS" then
-                       table.insert(args, v)
+                       arg_acc = arg_acc..v^M
                        comma = false
                elseif k == "COMMA" then
                        if comma then
                                -- continued comma means empty parameter
                                table.insert(args, "")
+                       else^M
+                               table.insert(args, arg_acc)^M
+                               arg_acc = ''^M
                        end
                        comma = true
                end
        end
+       table.insert(args, arg_acc)^M
        local v = repl:gsub("%$(%d+)", function (m) return args[tonumber(m)] or "" end)
        -- print("replaceArgs:" .. repl .. "|" .. tostring(#args) .. "|" .. v)
        return v

Compiling this code, reduced from a real world include file:

#define _(t,n) typedef union n##t

_ (u8, 16);

#undef _

Results in output "typedef uniou16 16u8", the correct result should be "typedef union 16u8".

Hi

The code below causes an error

local lcpp = require("lcpp")
ffi.cdef[[
#define numberOne 1
]]
print(ffi.C.numberOne)

how to get the #define from cdef ?

Thanks

Hello,

when trying to parse a simple file like:

define FOOBAR

undef FOOBAR

lcpp complains about FOOBAR being redefined. I think this is because of "undefine()" calling "state:define(key, nil)". The latter checks:
"if not override and state:defined(key) then error() ..."
It might be instead:

"if value and not override and state:defined(key) then error() ..."

Cheers

hello,

i just noticed that a single line like the following:

define SOME_VARIABLE 0xFF

is properly parsed, but:

define SOME_VARIABLE (0xFF)

fails with:
lcpp ERR [0000] already defined: SOME_VARIABLE

any clue about it before i try to debug it?

cheers,
ronan

think of:

deinfe FOO BAR

define FOO 123

... this must cause an "already defined excepction", but does not. instead you get:
BAR => 123

"foo" "bar" should give "foobar"

Hello,

great project! I think there is an issue with the way you handle configuration though.

You set the configuration variables (lcpp.LCCP_LUA, lcpp.LCCP_FFI, lcpp.LCPP_TEST) in the source and you use them when the module is required. That means you cannot actually load lcpp without running the tests or overloading the FFI.

There are several solutions to this, the simplest being to replace those options by functions of the lcpp module that would have to be called explicitly (e.g. lcpp.use_ffi()). That would mean everything is off by default.

Another possibility would be to make the module return a function that takes a table of options and returns the actual lcpp table, so you would use it (for instance) like that:

local lcpp = (require "lcpp"){test = false, lua = true}

Anything not set explicitly (nil) could take a default value of your choice.

The order of application of the macros must be the opposite (I suppose). An example:

#define log(x) puts(x)
#define TOSTR(x) _TOSTR(x)
#define _TOSTR(x)  #x
int main(int argc, char const *argv[])
{
    printf("hi from line %u\n", __LINE__);
    log("hi from line " TOSTR(__LINE__));
    puts("hi from line " TOSTR(__LINE__));
    return 0;
}

If I precompile this code with Clang (also with gcc and cl.exe) I get:

#1 "test_short.c"
#1 "<built-in>" 1
#1 "<built-in>" 3
#170 "<built-in>" 3
#1 "<command line>" 1
#1 "<built-in>" 2
#1 "test_short.c" 2
int main(int argc, char const *argv[])
{
    printf("hi from line %u\n", 11);
    puts("hi from line " "12");
    puts("hi from line " "13");
    return 0;
}

In the puts line, Clan use this order to evaluate the macros:

1. TOSTR
2. _TOSTR
3. #x
   4.__LINE__
   BUT applies the macros in the opposite order:
   1.__LINE__
   2.#x

But if I use lcpp:

../lua/src/lua -e 'lcpp = require("lcpp"); local out = lcpp.compileFile("test_short.c"); print(out);'

I get

int main(int argc, char const *argv[])
{
printf("hi from line %u\n", 10);
puts("hi from line ");
puts("hi from line __LINE__");
return 0;
}

Because the application goes in the same order of the evaluation:

1. TOSTR
2. _TOSTR
3. #x
   So, the stringification is applied before the __LINE__ substitution and then __LINE__ is a string, not an identifier

Sorry for my english and thank you for your work.

Processing the below .h file (synthesized from real-world sequence of includes from the fd.io) with lcpp results in an infinite loop.

Using anything other than a single underscore for the macro definition in the last part, e.g. two underscores, results in successful processing.

/*
 * Copyright (c) 2015 Cisco and/or its affiliates.
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at:
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
/*
  Copyright (c) 2001, 2002, 2003 Eliot Dresselhaus

  Permission is hereby granted, free of charge, to any person obtaining
  a copy of this software and associated documentation files (the
  "Software"), to deal in the Software without restriction, including
  without limitation the rights to use, copy, modify, merge, publish,
  distribute, sublicense, and/or sell copies of the Software, and to
  permit persons to whom the Software is furnished to do so, subject to
  the following conditions:

  The above copyright notice and this permission notice shall be
  included in all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
  EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
  MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
  NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
  LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
  WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/

#ifndef included_clib_vec_bootstrap_h
#define included_clib_vec_bootstrap_h

#define _vec_find(v)    ((vec_header_t *) (v) - 1)

#define _vec_len(v)     (_vec_find(v)->len)

#define vec_len(v)      ((v) ? _vec_len(v) : 0)


/* ALU function definition macro for functions taking two bitmaps. */
#define _(name, body, check_zero)                               \
always_inline uword *                                           \
clib_bitmap_##name (uword * ai, uword * bi)                     \
{                                                               \
  bi_len = vec_len (bi);                                        \
  for (i = 0; i < vec_len (ai); i++)                            \
    {                                                           \
      do { body; } while (0);                                   \
    }                                                           \
  if (check_zero)                                               \
    _vec_len (ai) -= n_trailing_zeros;                          \
  return ai;                                                    \
}
/* ALU functions: */
_ (and, a = a & b, 1)
_ (andnot, a = a &~ b, 1)
_ (or,  a = a | b, 0)
_ (xor, a = a ^ b, 1)
#undef _

#endif /* included_clib_vec_bootstrap_h */

If I try to load some libs, I've got an error:

/home/v/.luarocks/share/lua/5.2/lcpp.lua:1952: unknown type namespace on line _linenum_

What about it?

P.S.: I load it with ffi.cdef().

Judging by the lack of license I'm guessing MIT is gonna be your choice but it is still better for clarity, incidentally are you going for GCC compatibility at all? Would make an interesting drop in replacement for it when compiling the linux kernel. btw I also made my own replacement to make/nmake/gnumake (hated their poor syntax), I've yet to successfully compile kernel code linux mint which I'm on but it works fine for user land code (just look via my profile if you're interested), just not got a solid api just yet.

lcpp does not seem to look in /usr/local/include which is problematic if I would like to include external libraries.