graphitemaster / incbin Goto Github PK

I just tried to compile incbin.c on my 32bit Win7 maschine and it went berserk about missing semicolons and such. Weirdly, renaming the file to incbin.cpp (tricking cl into C++ mode) fixed the issue. Would be nice if you could look into this.

Instead of doing the big data array approach when you don't have the .incbin directive, why not use a custom resource type and generate a resource file as well as a source file that loads the data from the resource?

You use a user-defined resource type to point the resource to the binary blob. Each embedded binary gets its own resource id in the generated resource script and the generated C++ source file loads the resources and obtains pointers to their data. The code would call FindResourceA to get the HRSRC for the blob and then LoadResource followed by LockResource to get the pointer to the data and SizeofResource to get the data length.

For large binary payloads, this avoids all the compiler processing work of C++ source files of large data arrays.

Thanks for INCBIN! Before I found your tool I was trying to do this method myself, and was struggling to get it to link while cross-compiling. I am hopefull that this tool will also make it easier for me to make mac builds when the time comes.

When I used the latest incbin.h, I got this error when cross-compiling using the MXE cross compile on a linux host:

$ /home/jasonwoof/software/mxe/usr/bin/i686-w64-mingw32.static-gcc -c -o test.o test.c
/tmp/cca2SpPv.s: Assembler messages:
/tmp/cca2SpPv.s:5: Warning: .type pseudo-op used outside of .def/.endef ignored.
/tmp/cca2SpPv.s:5: Error: junk at end of line, first unrecognized character is `b'
/tmp/cca2SpPv.s:10: Warning: .type pseudo-op used outside of .def/.endef ignored.
/tmp/cca2SpPv.s:10: Error: junk at end of line, first unrecognized character is `b'
/tmp/cca2SpPv.s:15: Warning: .type pseudo-op used outside of .def/.endef ignored.
/tmp/cca2SpPv.s:15: Error: junk at end of line, first unrecognized character is `b'
zsh: exit 1     /home/jasonwoof/software/mxe/usr/bin/i686-w64-mingw32.static-gcc -c -o test.o

$ cat test.c
#include <stdio.h>
#define INCBIN_STYLE INCBIN_STYLE_SNAKE
#define INCBIN_PREFIX binary_
#include "incbin.h"
INCBIN(game_js, "game.js");

int
main(int argc, char **argv) {
    printf("%s, %0d\n",
        binary_game_js_data,
        binary_game_js_size
    );
    return 0;
}

As a workaround, I commented out the .type asm directive like this:

/* It's safe to use `@' on other architectures */
#    define INCBIN_TYPE(NAME)    "#.type " INCBIN_STRINGIZE(INCBIN_PREFIX) #NAME ", @object\n"

Then my test compiles and runs perfectly (built on linux, tested with wine) and also compiles and runs fine an/for native linux.

Since MSVC does not support the macro method, I thought about just typing together a little ASM file to pull in the binaries. Thing is, I don't know assembly...

Could you provide me a basic example of an ASM file with which i can include binary files into the same object, just as if i had used the INCBIN macro?

Kind regards, Ingwie

Pretty simple. If I compile incbin.c using Visual Studio 2017, it complains about a missing MAX_PATH.

The following invocation gives me what I need. It's a little overkill, but it's probably the safest way to ensure MAX_PATH exists.

#ifdef _WIN32
#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#endif

OK, now that I've looked down into the tests, which take the address of the end symbol, I know how to use it:

I didn't find anywhere in the docs that show an example usage. What they do say implies that the symbol is already a const unsigned char*, when it's actually a const unsigned char immediately after the data, of which you must take the address.

I notice there is a .travis.tml file, but the project isn't enabled on TravisCI. So, why not?

When I test the incbin in arm compiler 5 with the following code:

#define INCBIN_PREFIX
#define INCBIN_STYLE INCBIN_STYLE_SNAKE
#include "incbin.h"


INCBIN(HOTEL_MP3_ROM, "hotel.mp3");

I get following error message:

error:  #412: an "asm" declaration is not allowed here
  INCBIN(HOTEL_MP3_ROM, "hotel.mp3");

The arm compiler 5 version is V5.06 update 7 (build 960).

Please help.

By the way, the official method to detect arm compiler 5 and arm compiler 6 is as following:

// for arm compiler 5
#undef __IS_COMPILER_ARM_COMPILER_5__
#if ((__ARMCC_VERSION >= 5000000) && (__ARMCC_VERSION < 6000000))
#   define __IS_COMPILER_ARM_COMPILER_5__       1
#endif


//for arm compiler 6

#undef __IS_COMPILER_ARM_COMPILER_6__
#if defined(__ARMCC_VERSION) && (__ARMCC_VERSION >= 6010050)
#   define __IS_COMPILER_ARM_COMPILER_6__       1
#endif

I hope this helps.

Thanks for making this. Spent so long trying to find a way to accomplish this task easily and all failed as the ld on Mac OS does not have an option for embedded binaries. I was wondering what the incbin.d file does. Thanks in advance.

Would you be open to travis-ci.org testing of incbin? I'd be happy to contribute it.

Hey.

I was writing a script to automate use of Incbin on Windows - and also to test what I can do with IncBin from a script - and came across this:

[email protected] ~/W/IceTea $ ./out/incbin -o test.c ./out/icetea-scripts.incbin.h
included `./out/icetea-scripts.incbin.h'
generated `test.c'
[email protected] ~/W/IceTea $ ./out/incbin -o test.c (pwd)/out/icetea-scripts.incbin.h
included `/Users/Ingwie/Work/IceTea/out/icetea-scripts.incbin.h'
generated `test.c'
[email protected] ~/W/IceTea $ ./out/incbin -I. -o test.c (pwd)/out/icetea-scripts.incbin.h
failed to open `/Users/Ingwie/Work/IceTea/out/icetea-scripts.incbin.h' for reading
[email protected] ~/W/IceTea $ ./out/incbin -I . -o test.c (pwd)/out/icetea-scripts.incbin.h
included `.'
failed to include data `lib/std.os'
included `/Users/Ingwie/Work/IceTea/out/icetea-scripts.incbin.h'
generated `test.c'

-I. causes an error, whilst -I . does not let me see my files.

I am not depending on -I for now, but it might be good to look into this. :)

The readme says it's known to work on TinyCC, but I couldn't get it to work.
TCC's assembler doesn't support the .incbin directive:
https://bellard.org/tcc/tcc-doc.html#Directives
I get the following error:
test.c:3: error: bad expression syntax ["test.txt"]

Hi, I develop code that is built on windows with MSVC and on linux with GCC. I use CMake.

I use the "tool" on windows to generate data.c but my problem is I can't find a way to express relative path that works with MSVC and GCC:

• GCC: INCBIN(myData, "../../../resources/config-schema.json");
• MSVC: INCBIN(myData, "resources/config-schema.json");

It seems that with GCC the path has to be relative to the build directory and with MSVC the path has to be relative to the current working directory when the build command is executed.

when i tried to build incbin.c under msvc2017 - i've find error

incbin.exe secret.cpp
debug assert due uninitiazed variable "ret" - possible fix

int main(int argc, char **argv) {
    int ret = 0, i, paths, files = 0, style = kCamel;
................

I compiled the incbin.c file and made an executable if I run it with help I get:

 ./a.exe phong_lighting.glsl  -I../../Src/Engine/EmbeddedShaders/ -o ../../buildvs/data.c  -help
C:\Users\Makogan\Documents\neverengine\libraries\incbin\a.exe [-help] [-Ipath...] | <sourcefiles> | [-o output] | [-p prefix]
   -o         - output file [default is "data.c"]
   -p         - specify a prefix for symbol names (default is "g")
   -S<style>  - specify a style for symbol generation (default is "camelcase")
   -I<path>   - specify an include path for the tool to use
   -help      - this
example:
   C:\Users\Makogan\Documents\neverengine\libraries\incbin\a.exe source.c other_source.cpp -o file.c
styles (for -S):
   camelcase
   snakecase

If I try to run it normally (without the help flag) this is the data.c file that gets generated:

/* File automatically generated by incbin */
#include "incbin.h"

#ifdef __cplusplus
extern "C" {
#endif


#ifdef __cplusplus
}
#endif

No errors or warnings are thrown by the tool.

I am on windows 10

I just noticed that the new INCBIN_OUTPUT_DATA_SECTION, INCBIN_OUTPUT_SIZE_SECTION actually don't work because they're unused in the code...

The one flaw in this otherwise perfect-for-the-job tool is that an INCBIN()ed binary doesn't get included in the gcc auto-dependency-file-generation of " -MMD -MP -MF"". I wonder if there is some kind of way to fake out gcc/g++ to emit a dependency on the binary file?

It seems Cygwin support could be easily added by adding a defined(CYGWIN) where elif defined(MINGW32) || defined(MINGW64) are checked.

I got the following compile error if building with arm gcc version 6.5.0.

Assembler messages:
Error: unaligned opcodes detected in executable segment

However it only happens if I add the -Os optimisation flag.

If I make the following change it fixes the issue:

Index: incbin.h
===================================================================
--- incbin.h	(revision 175ae785f27956a0ed5b08b84881a227f2bec0d4)
+++ incbin.h	(date 1571921457000)
@@ -350,6 +350,8 @@
             INCBIN_MANGLE INCBIN_STRINGIZE(INCBIN_PREFIX) #NAME INCBIN_STYLE_STRING(SIZE) ":\n" \
                 INCBIN_INT INCBIN_MANGLE INCBIN_STRINGIZE(INCBIN_PREFIX) #NAME INCBIN_STYLE_STRING(END) " - " \
                            INCBIN_MANGLE INCBIN_STRINGIZE(INCBIN_PREFIX) #NAME INCBIN_STYLE_STRING(DATA) "\n" \
+            INCBIN_ALIGN_HOST \
+            ".text\n" \
     ); \
     INCBIN_EXTERN(NAME)

I got the idea from the comment in README.md file of the https://github.com/RyogaMasaki/megadev project.

I don't know if this is a bug in the gcc assembler as I couldn't see anything wrong with the assembler being generated.

So all I did was:

$ gcc incbin/incbin.c -o incbin
$ ./incbin -o scripts.cpp src/scripts.rc

and got an error:

[email protected] ~/W/IceTea $ file src/scripts.rc 
src/scripts.rc: ASCII c program text
[email protected] ~/W/IceTea $ ./out/incbin -o scripts.cpp ./src/scripts.rc 
failed to open `./src/scripts.rc' for reading

What exactly could be the issue for this?

Hi, I've checked if this would work using the emscripten compiler.
I try to compile the test program.

<inline asm>:1:17: error: Expected ,, instead got: 
.section .rodata

<inline asm>:16:6: error: symbol 'gLoremData': unsupported subtraction expression used in relocation.
.int gLoremEnd - gLoremData

I've no doubt the section statement can be fixed, by examing some -S output from the compiler.
Regarding the second, I don't have any ideas.

The end pointer value is not correct.

I get such output from printing it, using GCC/Clang on x86_64.
Value is 0xe0001, and expectation is 0x5574b697d01e.

Basic Example
Size: 14
Start: 0x5574b697d010
End: 0xe0001
Hello, World!

Hello.c.gz
Hello.dat.gz

Bitcode is an Apple technology that basically allows Apple to recompile apps submitted to the App Store at will for optimization reasons.

https://developer.apple.com/documentation/xcode/reducing_your_app_s_size/doing_basic_optimization_to_reduce_your_app_s_size

One of the limitations of this technology is that the app must be 100% buildable from the bitcode embedded into the binary, which I assume is in fact the LLVM IR. This apparently disallows any sort of architecture-specific assembly in the source code.

As a result, when trying to upload an app with bitcode enabled, ITMS returns this error:

ITMS-90562: Invalid Bundle - Bitcode failed to compile for your watchOS binary because it includes assembly source code or inline assembly. To resolve this issue, rewrite the corresponding code in a higher-level language such as Objective-C or Swift and redeliver your app.

While looking for a workaround, I found that Xcode allows one to add LLVM assembly sources to a project. I wasn't able to find a directive equivalent to .incbin in the LLVM assembly manual, though, but maybe I simply missed it.

Anyway, the workaround, of course, is to recode any binaries into C-array literal syntax and compile them as C code.

I suggest that this issue be mentioned in the README. It would also be helpful if the library emits a warning if targeting the iOS platform.

When giving incbin a file to include, it usually seems to operate from the current working directory. How can the search path be influenced? -I on Clang does not seem to yield any results.

Compiling on Mingw-W64 x86_64-ucrt-posix-seh 11.2.0 causes the program to do undefined things such as crash when creating a const struct and corrupts data that is imported.

I am trying to include image and audio resources via Incbin and then placing them into a struct so I get the file size and the pointer to the data into one varible. This is done as follows

#define RES_INC(name, path) INCBIN(name, path); const struct file_struct s_ ## name = {g ## name ## Data, g ## name ## Size};

Looking at the program in gdb, it crashes when making the struct rater than the INC bin. It crashes at

__static_initialization_and_destruction0

This happens after the first included file was added successfully. The file_struct struct is as follows

struct file_struct { const unsigned char * data; size_t size; };

if I add a bunch of padding ints after each struct the program runs but other varibles that are in other .cpp files are being altered

The build error: https://travis-ci.org/andyli/incbin/builds/367773037

Undefined symbols for architecture x86_64:
  "_gLoremData", referenced from:
      _main in asserts.c.o
  "_gLoremEnd", referenced from:
      _main in asserts.c.o
  "_gLoremSize", referenced from:
      _main in asserts.c.o
  "_gOnebyteData", referenced from:
      _main in asserts.c.o
  "_gOnebyteEnd", referenced from:
      _main in asserts.c.o
  "_gOnebyteSize", referenced from:
      _main in asserts.c.o
  "_gSevenbytesData", referenced from:
      _main in asserts.c.o
  "_gSevenbytesEnd", referenced from:
      _main in asserts.c.o
  "_gSevenbytesSize", referenced from:
      _main in asserts.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [asserts] Error 1
make[1]: *** [CMakeFiles/asserts.dir/all] Error 2
make: *** [all] Error 2

My test code can be seen at andyli@47c3b76

Notice that the test passed in the Linux builds.

Hello,

in my Python compiler Nuitka, I am generating extension modules, and will now start using incbin after previously working with linker options to include stuff directly, but LTO came in the way of that. And then I found your brilliant tool to solve the issue in a way more portable fashion. And it's working very nicely.

However, what I discovered is that the symbols are exported, and what happens is when I load two extension modules, is that they get confused with their read only symbols matching, and then only one gets loaded and the other used wrongly. I have added CRC32 checking, and it fails if the second module is loaded, where otherwise in all of Nuitka this doesn't happen.

I am aware that I can make the symbol names not match, by prefixing randomly, but since the users don't cooperate, I would e.g. have to use the hash. Alternatively I think a linker script might prevent the export of the symbols.

In my use in Nuitka, I only need the data locally, I do not need it to be visible to outside code, i.e. I would be fine if the symbol was static in the C sense, and I was wondering if there was a way to just make it local through incbin. Checking out the assembler man page revealed that there is no such thing as a local symbol for it (of course?). So I tried to empty the declaration of INCBIN_GLOBAL and this results in the symbols not being exported and that seems to work, so I am now having a INCBIN_LOCAL that guards this, and it seems to work, at least the binary extension modules doesn't export the symbols anymore, yet the data seems accessible by the code in the same unit.

I guess, that might make sense to add it as a feature to incbin? I would make a PR with the basic change if you agree.

So it turns out.. .align 4 on ARM means 'align on 16 bytes'.

.align 1 means 'align on 2 bytes'. (so 1<<1).

.align 0, which could conceivably do the right thing, means align on 2 bytes, for $reasons.

.balign 1 on GNU AS does the right thing on all platforms, but may be GNU AS specific. Meanwhile, I have found no portable way of saying ".align 0" for reals on ARM. A cursory web search appears to say that you should simply not have an .align statement then.

Everything is terrible. Thoughts welcome on what the best thing to do would be! We need the 'End' token to be aligned flush after the binary object.

VSCode, PlatformIO
ESP8266 nodeMCU v3.0
PIO`s INCBIN lib https://github.com/AlexIII/incbin-arduino?utm_source=platformio&utm_medium=piohome (just a small add-on to this lib)

I can't take any datum from incbin`d array. It throws exception.

#include "Arduino.h"
#include "incbin.h"

INCBIN( my, "path/to/file.extension");

void setup(){
  Serial.begin(9600);
  Serial.println();
  
  Serial.println("assigning");
  uint8_t datum = *gmyData;
  Serial.println("assigned");
  
  Serial.println(datum);
}

void loop() {}

monitor output

assigning

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Exception (3):
epc1=0x402010f1 epc2=0x00000000 epc3=0x00000000 excvaddr=0x40238b74 depc=0x00000000

LoadStoreError: Processor internal physical address or data error during load or store
  epc1=0x402010f1 in setup at ??:?
  excvaddr=0x40238b74 in ethernet_input at ??:?

>>>stack>>>

ctx: cont
sp: 3ffffe00 end: 3fffffc0 offset: 0190
3fffff90:  feefeffe 00000000 3ffee564 402010eb  
3fffffa0:  feefeffe 00000000 3ffee69c 40201e60  
3fffffb0:  feefeffe feefeffe 3ffe85dc 40100cc1  
<<<stack<<<

0x402010eb in setup at ??:?
0x40201e60 in loop_wrapper() at core_esp8266_main.cpp:?
0x40100cc1 in cont_wrapper at ??:?

File is a "primary" certificate gtsltsr.crt from point 1.b from https://cloud.google.com/iot/docs/how-tos/mqtt-bridge#using_a_long-term_mqtt_domain

Also didn't work gmyData[0],
ButSerial.write( gmyData, gmySize) works and prints correct result and doesn't throw exception. WTF???

I have almost completely ported my project to use incbin. However, I am struggling with C++.

I created a file called scripts.rc which simply contains:

#include "incbin.h"

INCBIN(Configurable, "lib/configurable.os");
INCBIN(libIceTea, "lib/IceTea.os");
INCBIN(main, "lib/main.os");

And within my main code:

// Embed
#include "incbin.h"
#undef INCBIN
#define INCBIN(NAME, PATH) INCBIN_EXTERN(NAME)
#include "scripts.rc"

That, on it's own, works. Its a cheaty hack so that I can use the three step build on all platforms. That is:

gcc incbin/incbin.c -o out/incbin
out/incbin src/scripts.rc -o src/scripts.cpp
g++ src/*.cpp -o icetea

However, the const keyword is giving me issues. The ...Data[] symbols get all mangled with the name, and it turns out that this is why.

For now, I have to find a way to work this out...

Just after you fixed the incbin tool problem, I updated and re-compiled my source using the script - the tool itself works again, thanks for that :).

But now I have a more awkward problem:

src/IceTea.cpp:154:29: error: use of undeclared identifier 'gSTDData'
        {"std.os",          INCBIN_DATA(STD),           INCBIN_LEN(STD)},
                            ^
src/IceTea.cpp:20:27: note: expanded from macro 'INCBIN_DATA'
#define INCBIN_DATA(name) g ## name ## Data
                          ^
<scratch space>:356:1: note: expanded from here
gSTDData
^

I was so confused that it took me a while untill I actually realized how I could best test this. I took the file containing the INCBIN statements and wrote a teensy tiny C++ program:

#include "scripts.rc"
int main() {
  return 0;
}

Nothing special going on - except script.rc, which is the header in which I have my incbin statements. the .rc is probably misleading, but it makes me think of resource container. So that works for me - besides, it has worked for a logn time now :).

After compiling this tiny program, I inspected the symbol table...and:

[email protected] ~/W/IceTea $ nm t
0000000100000000 T __mh_execute_header
00000001000014f0 s _gConfigurable0_0_DATA
00000001000015a8 s _gConfigurable0_0_END
00000001000015b0 s _gConfigurable0_0_SIZE
000000010000a860 s _gInternalBootstrapIt0_0_DATA
000000010000d0ee s _gInternalBootstrapIt0_0_END
000000010000d0f0 s _gInternalBootstrapIt0_0_SIZE
0000000100001000 s _gSTD0_0_DATA
00000001000014de s _gSTD0_0_END
00000001000014e0 s _gSTD0_0_SIZE
00000001000015c0 s _glibIceTea0_0_DATA
000000010000a84b s _glibIceTea0_0_END
000000010000a850 s _glibIceTea0_0_SIZE
0000000100000f90 T _main
                 U dyld_stub_binder

0_0_DATA and such! Now, that does explain why my tiny helper macro did no longer work. :)

Looking into incbin.h, this also does not seem to be the intended behavior either. Could you look into this please? This basically broke my app for now. ^^"

Hi, quick question, is the data loaded into memory at launch?

I'm looking for a solution that allows me to embed a couple very large (GB-scale) zip files, and would allow me to "mmap" into that section of the memory, without having to load the entire binary's data at load, any suggestions?

Very tiny change. Therefore just a diff. :)

[email protected] ~/W/I/incbin $ git diff
diff --git a/incbin.h b/incbin.h
index e8cb45a..709e4e0 100644
--- a/incbin.h
+++ b/incbin.h
@@ -22,7 +22,11 @@
 # endif
 #endif

+#ifndef _MSC_VER
 #define INCBIN_ALIGN __attribute__((aligned(INCBIN_ALIGNMENT)))
+#else
+#define INCBIN_ALIGN __declspec(align(INCBIN_ALIGNMENT))
+#endif

 #ifdef __cplusplus
 #  define INCBIN_EXTERNAL extern "C"

Sorry for the flood of messages, by the way. o.o

I have a link error if I use INCBIN

I have a data.cpp file with this:

#include "incbin.h"

INCBIN(Icon, "Icon.png");

const unsigned char* an_icon = gIconData;

and in my main.cpp I try to access

extern const unsigned char* an_icon;

int main()
{
    const unsigned char* icon = an_icon;
    return 0;
}

I get link error that gIconData is not defined.

When I change incbin.h to the following it works:

diff --git a/incbin.h b/incbin.h
index 85be21a..9c7330e 100644
--- a/incbin.h
+++ b/incbin.h
@@ -95,16 +95,16 @@
  */
 #define INCBIN(NAME, FILENAME) \
     __asm__(INCBIN_SECTION \
-            INCBIN_GLOBAL(g ## NAME) \
-            INCBIN_TYPE(g ## NAME) \
+            INCBIN_GLOBAL(g ## NAME ## Data) \
+            INCBIN_TYPE(g ## NAME ## Data) \
             ".align " INCBIN_STRINGIZE(INCBIN_ALIGNMENT) "\n" \
-            INCBIN_MANGLE "g" #NAME ":\n" \
+            INCBIN_MANGLE "g" #NAME "Data:\n" \
                 ".incbin \"" FILENAME "\"\n" \
             INCBIN_GLOBAL(g ## NAME ## Size) \
             INCBIN_TYPE(g ## NAME ## Size) \
             ".align " INCBIN_STRINGIZE(INCBIN_ALIGNMENT) "\n" \
             INCBIN_MANGLE "g" #NAME "Size:\n" \
-                INCBIN_INT INCBIN_MANGLE "g" #NAME "Size - " INCBIN_MANGLE "g" #NAME "\n" \
+                INCBIN_INT INCBIN_MANGLE "g" #NAME "Size - " INCBIN_MANGLE "g" #NAME "Data\n" \
     ); \
     INCBIN_EXTERN(NAME)

With this I still get the same error if I use INCBIN_EXTERN(Icon) instead of my own extern const unsigned char* an_icon;

The size is not calculated correctly when I use incbin on Mac. I get for every included file the value 65536

git clone [email protected]:graphitemaster/incbin.git
cd incbin

Create a source file:

$ cat test.cpp
#include <incbin.h>
#include <unistd.h>

INCBIN(test_txt, "test.txt");

int main(int, char **)
{
    write(1, gtest_txtData, gtest_txtSize);
    return 0;
}

Create a data file to include in a separate directory:

$ mkdir data
$ cat data/test.txt
Hello, world!

This works perfectly:

$ clang++ --version
clang version 17.0.0 ([email protected]:llvm/llvm-project.git 91ae1f5d08d98daae2122b0ab3779b8e1603a045)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin

$ clang++ -I. -Idata test.cpp && ./a.out 
Hello, world!

But this does not:

$ clang++ -I. -Idata -flto=thin -fuse-ld=lld test.cpp && ./a.out 
error: Could not find incbin file 'test.txt'
.incbin "test.txt"
        ^
error: Could not find incbin file 'test.txt'
.incbin "test.txt"
        ^
ld.lld: error: /tmp/test-e5faa6.o <inline asm>:6:9: Could not find incbin file 'test.txt'
.incbin "test.txt"
        ^

clang++: error: linker command failed with exit code 1 (use -v to see invocation)

The culprit is LTO, and the include path.
If I remove LTO or specify the absolute path inside the INCBIN macro, it works.

The issue found by ClickHouse in ClickHouse/ClickHouse#52489

We include binary data into the executable, but recently we did it with llvm-objcopy and used it by iterating over symbols of the binary at runtime. Changing it to INCBIN is a nice improvement!

I am trying to make a resource system where given files are archived into a tarball, which is linked into the executable (where incbin comes into play).
On paper this is not a problem, but it seems that with certain filetypes incbin is not including the entirety of the file.
Example:

#define INCBIN_PREFIX r_
#include "incbin.h"
#include <fstream>
INCBIN(grc, "./test.tar");

int main(int argc, char *argv[]) {
	std::ofstream out("./test2.tar");
	out << r_grcData;
	out.close();
}

C version, to ensure it's not a C++ thing;

#define INCBIN_PREFIX r_
#include "incbin.h"
#include <stdio.h>
INCBIN(grc, "./test.tar");

int main(int argc, char *argv[]) {
	const char *data = (const char *)r_grcData;
	FILE *fp = fopen("./test2.tar", "wb");
	fputs(data, fp);
	fclose(fp);
}

test.tar is 1536 bytes.
test2.tar is 8 bytes.

A hex editor tells me that the first null character in test.tar is on the ninth byte (the cutoff of test2.tar).

This happens with a PNG file, where the cutoff happens at its first null byte.

I'm not entirely sure if this is user error on my part or incbin not knowing when to stop including file. It's pretty rare to find a file with no null bytes in it, maybe it is misinterpreting the null character as the end of the file.

This happens both with gcc/g++ 11.1.0 and clang/clang++ 12.0.1

Although one could simply add incbin to the version control of an application as it is quite small, this has the caveat that you cannot tell automatically if a new version is available. Being able to use conan to download and install the project would facilitate integrating the tool in projects.

I have set up my IceTea tool to also build through XCode. This is merely an excercise to find my way around in the IDE itself by using code that I already wrote. On pure command line, everything works exactly as expected. But with XCode, incbin no longer emits files. I even copied the compiled binary and compared them, and saw that they were pretty different.

Here is an extract of what XCode does:

CompileC build/IceTea.build/Release/incbin.build/Objects-normal/x86_64/incbin.o /Users/Ingwie/Work/IceTea/deps/incbin/incbin.c normal x86_64 c com.apple.compilers.llvm.clang.1_0.compiler
    cd /Users/Ingwie/Work/IceTea/ide
    export LANG=en_US.US-ASCII
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c -arch x86_64 -fmessage-length=204 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit=0 -fcolor-diagnostics -std=gnu99 -fmodules -gmodules -fmodules-prune-interval=86400 -fmodules-prune-after=345600 -fbuild-session-file=/var/folders/kz/l3s2fpd51ln9_p9ymvqbyshm0000gn/C/org.llvm.clang/ModuleCache/Session.modulevalidation -fmodules-validate-once-per-build-session -Wnon-modular-include-in-framework-module -Werror=non-modular-include-in-framework-module -Wno-trigraphs -fpascal-strings -Os -fno-common -Wno-missing-field-initializers -Wno-missing-prototypes -Wunreachable-code -Werror=deprecated-objc-isa-usage -Werror=objc-root-class -Wno-missing-braces -Wparentheses -Wswitch -Wunused-function -Wno-unused-label -Wno-unused-parameter -Wunused-variable -Wunused-value -Wempty-body -Wconditional-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wconstant-conversion -Wint-conversion -Wbool-conversion -Wenum-conversion -Wshorten-64-to-32 -Wpointer-sign -Wno-newline-eof -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -fasm-blocks -fstrict-aliasing -Wdeprecated-declarations -mmacosx-version-min=10.11 -g -fvisibility=hidden -Wno-sign-conversion -iquote /Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/incbin-generated-files.hmap -I/Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/incbin-own-target-headers.hmap -I/Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/incbin-all-target-headers.hmap -iquote /Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/incbin-project-headers.hmap -I/Users/Ingwie/Work/IceTea/ide/build/Release/include -I/Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/DerivedSources/x86_64 -I/Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/DerivedSources -F/Users/Ingwie/Work/IceTea/ide/build/Release -MMD -MT dependencies -MF /Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/Objects-normal/x86_64/incbin.d --serialize-diagnostics /Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/Objects-normal/x86_64/incbin.dia -c /Users/Ingwie/Work/IceTea/deps/incbin/incbin.c -o /Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/Objects-normal/x86_64/incbin.o
/Users/Ingwie/Work/IceTea/deps/incbin/incbin.c:26:11: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
    chr = *n;
        ~ ^~
/Users/Ingwie/Work/IceTea/deps/incbin/incbin.c:32:30: warning: implicit conversion loses integer precision: 'long' to 'int' [-Wshorten-64-to-32]
            chr = *n + *line - pos;
                ~ ~~~~~~~~~~~^~~~~
/Users/Ingwie/Work/IceTea/deps/incbin/incbin.c:51:16: warning: implicit conversion loses integer precision: 'long' to 'int' [-Wshorten-64-to-32]
    return pos - *line;
    ~~~~~~ ~~~~^~~~~~~
/Users/Ingwie/Work/IceTea/deps/incbin/incbin.c:103:1: warning: control may reach end of non-void function [-Wreturn-type]
}
^
/Users/Ingwie/Work/IceTea/deps/incbin/incbin.c:277:9: warning: variable 'ret' may be uninitialized when used here [-Wconditional-uninitialized]
    if (ret == 0) {
        ^~~
/Users/Ingwie/Work/IceTea/deps/incbin/incbin.c:106:12: note: initialize the variable 'ret' to silence this warning
    int ret, i, paths, files = 0, style = kCamel;
           ^
            = 0
/Users/Ingwie/Work/IceTea/deps/incbin/incbin.c:257:25: warning: variable 'count' may be uninitialized when used here [-Wconditional-uninitialized]
                    if (count == 12) {
                        ^~~~~
/Users/Ingwie/Work/IceTea/deps/incbin/incbin.c:239:43: note: initialize the variable 'count' to silence this warning
                unsigned char *data, count;
                                          ^
                                           = '\0'
6 warnings generated.

Ld build/Release/incbin normal x86_64
    cd /Users/Ingwie/Work/IceTea/ide
    export MACOSX_DEPLOYMENT_TARGET=10.11
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -L/Users/Ingwie/Work/IceTea/ide/build/Release -F/Users/Ingwie/Work/IceTea/ide/build/Release -filelist /Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/Objects-normal/x86_64/incbin.LinkFileList -mmacosx-version-min=10.11 -Xlinker -dependency_info -Xlinker /Users/Ingwie/Work/IceTea/ide/build/IceTea.build/Release/incbin.build/Objects-normal/x86_64/incbin_dependency_info.dat -o /Users/Ingwie/Work/IceTea/ide/build/Release/incbin

GenerateDSYMFile build/Release/incbin.dSYM build/Release/incbin
    cd /Users/Ingwie/Work/IceTea/ide
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/dsymutil /Users/Ingwie/Work/IceTea/ide/build/Release/incbin -o /Users/Ingwie/Work/IceTea/ide/build/Release/incbin.dSYM

CodeSign build/Release/incbin
    cd /Users/Ingwie/Work/IceTea/ide
    export CODESIGN_ALLOCATE=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate

Signing Identity:     "-"

    /usr/bin/codesign --force --sign - --timestamp=none /Users/Ingwie/Work/IceTea/ide/build/Release/incbin

Any idea why incbin may have stopped emitting files? It reads my input as expected, but thats it. There is no emitting being done. Maybe it is using a flag that I've overseen that might be causing issues.

I want to use the utility to modify an executable with resource changes. I have written this code:

#include <stdio.h>
#include "incbin.h"

int main(int argc, char** argv) {
    
    INCBIN(Code, "test.exe");
    
    unsigned char blip = gCodeData[0];
    printf("length %u", gCodeSize);
}

I have tried unsuccessfully to compile this code with Clang, Cygwin and MSys.

With Clang it throws these errors:

inc.o : error LNK2019: unresolved external symbol gCodeData referenced in function main
inc.o : error LNK2019: unresolved external symbol gCodeSize referenced in function main

With both Cygwin and MSys it throws these errors:

Warning: .type pseudo-op used outside of .def/.endef: ignored.
Error: junk at end of line, first unrecognized character is `g'
Error: file not found: test.exe
Warning: .type pseudo-op used outside of .def/.endef: ignored.
Error: junk at end of line, first unrecognized character is `g'
Warning: .type pseudo-op used outside of .def/.endef: ignored.
Error: junk at end of line, first unrecognized character is `g'

I get similar "junk at end of line" errors when I try to Make the "asserts.c" program packed with the repo.

Edit: I cannot withdraw an issue once submitted.

The program will not compile from within my NetBeans environment and this might be because of settings.

I have since successfully compiled and run this minimal working example from within MSys:

#include <stdio.h>
#include "incbin.h"

INCBIN(Code, "test.exe");

int main(int argc, char** argv) {
    printf("Length %u\n", gCodeSize);
}

Today I tried to use incbin with the Embarcadero compiler, and it doesn't work.

Has nobody tried it on Embarcadero?

I love incbin, we use it in PowerDNS. Sadly, it does not work on ARM, specifically on our Raspberry PIs. And this is of course a very important target audience. Has anyone managed to find the #ifdefs to make it work?

It spews out:

/tmp/ccHhbOO1.s: Assembler messages:
...
/tmp/ccaQN9ku.s:140: Error: unrecognized symbol type ""
/tmp/ccaQN9ku.s:145: Error: unrecognized symbol type ""
/tmp/ccaQN9ku.s:150: Error: unrecognized symbol type ""
/tmp/ccaQN9ku.s:157: Error: unrecognized symbol type ""

This is line 140:

.type gjs_rickshaw_min_jsData, @object

145:

.type gjs_rickshaw_min_jsEnd, @object

With some context:

 .section .rodata
.global gjs_rickshaw_min_jsData
.type gjs_rickshaw_min_jsData, @object
.align 4
gjs_rickshaw_min_jsData:
.incbin "./dnsdistdist/html/js/rickshaw.min.js"
.global gjs_rickshaw_min_jsEnd
.type gjs_rickshaw_min_jsEnd, @object
.align 1
gjs_rickshaw_min_jsEnd:
.int 1
.global gjs_rickshaw_min_jsSize
.type gjs_rickshaw_min_jsSize, @object
.align 4
gjs_rickshaw_min_jsSize:
.int gjs_rickshaw_min_jsEnd - gjs_rickshaw_min_jsData

This looks exactly like what gets emitted on x86-64, except for the .aligns etc. Perhaps on ARM the '@object' syntax does not work?

Help!

(full background, with some overhead, at PowerDNS/pdns#9835 (comment))

We have this piece of code:

INCBIN(api_swagger_yaml, "api-swagger.yaml");
static const string g_api_swagger_yaml{(const char*)gapi_swagger_yamlData, gapi_swagger_yamlSize};
INCBIN(api_swagger_json, "api-swagger.json");
static const string g_api_swagger_json{(const char*)gapi_swagger_jsonData, gapi_swagger_jsonSize};

When we compile this with gcc's or clang's Address Sanitizer, it crashes in a string constructor.

Reordering the code like this fixes the problem:

INCBIN(api_swagger_yaml, "api-swagger.yaml");
INCBIN(api_swagger_json, "api-swagger.json");
static const string g_api_swagger_yaml{(const char*)gapi_swagger_yamlData, gapi_swagger_yamlSize};
static const string g_api_swagger_json{(const char*)gapi_swagger_jsonData, gapi_swagger_jsonSize};

A copy/paste of my understanding of the reason, from the linked ticket:

• INCBIN sets .section rodata
• the compiler sets .bss for storing the const string
• the next INCBIN call sets .section rodata again
• the compiler omits .bss for the second const string - presumably because it assumes it is already in BSS

Now the question is - can we add something to the macro to 'restore' the previously active section, or inform the compiler that it needs to repeat its section choice?

Please fix incbin.exe help.

Now it says that input files are binaries, but in fact utility wants source file with INCBIN macro.

In a situation where the file is included properly and no compile-time errors are encountered, trying to refer to the Data variable will yield nothing.

...

#define INCBIN_PREFIX r_
#include "lib/incbin/incbin.h"
INCBIN(grc, "./data.tar");

...

std::vector<Resource::fileContents> Resource::getFileList() {

    char *tmp = reinterpret_cast<char *>(tarData);
    tmp[r_grcSize + 1] = '\0';

    mtar_mem_stream_t e;
    e.data = tmp;
    e.size = r_grcSize;
    e.pos = 0;

    mtar_open_mem(&tarball, &e);
//    mtar_open(&tarball, "./data.tar", "r"); // this works
    std::vector<Resource::fileContents> list;
    mtar_header_t h;
    while ((mtar_read_header(&tarball, &h)) != MTAR_ENULLRECORD) {
        list.push_back(Resource::fileContents{h.name, h.size});
        mtar_next(&tarball);
    }
    return list;
}

Resource::Resource() {
    tarData = calloc(1, r_grcSize);
    memcpy(tarData, r_grcData, r_grcSize);
    fileList = getFileList();
}

This code crashes because r_grcData is empty. I am using GCC 11 and glibc.
The tar processing code works perfectly and does everything I want it to do when using it's normal filename-based open function. The memory stream counterpart is given a direct copy of the r_grcData variable, which should contain the tar file all the same, but doesn't.