flatbuffers.h yields several compile warnings under VS2013 Update 2 with warning level 4, both when used to compile flatc, as well as use a generated header file. Ignoring them when building flatc is less of an issue, but it'd be nice if including a generated header (which then includes flatbuffers.h) didn't force us to back off or ignore warnings.

For example, when just including a generated header (haven't actually used code from it yet):
...flatbuffers.h(410) : error C2220: warning treated as error - no 'object' file generated
...flatbuffers.h(410) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers.h(419) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data

And when compiling flatc (also some come from CPP files):
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : error C2220: warning treated as error - no 'object' file generated (...flatbuffers\src\idl_gen_text.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\idl_gen_text.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : error C2220: warning treated as error - no 'object' file generated (...flatbuffers\src\flatc.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\flatc.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(419) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\idl_gen_text.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(419) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\flatc.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : error C2220: warning treated as error - no 'object' file generated (...flatbuffers\src\idl_gen_cpp.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\idl_gen_cpp.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(640) : warning C4100: 'other' : unreferenced formal parameter (...flatbuffers\src\idl_gen_text.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(419) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\idl_gen_cpp.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(640) : warning C4100: 'other' : unreferenced formal parameter (...flatbuffers\src\flatc.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(652) : warning C4245: 'return' : conversion from 'int' to 'size_t', signed/unsigned mismatch (...flatbuffers\src\idl_gen_text.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(652) : warning C4245: 'return' : conversion from 'int' to 'size_t', signed/unsigned mismatch (...flatbuffers\src\flatc.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(640) : warning C4100: 'other' : unreferenced formal parameter (...flatbuffers\src\idl_gen_cpp.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(652) : warning C4245: 'return' : conversion from 'int' to 'size_t', signed/unsigned mismatch (...flatbuffers\src\idl_gen_cpp.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : error C2220: warning treated as error - no 'object' file generated (...flatbuffers\src\idl_gen_java.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\idl_gen_java.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(419) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\idl_gen_java.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(640) : warning C4100: 'other' : unreferenced formal parameter (...flatbuffers\src\idl_gen_java.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : error C2220: warning treated as error - no 'object' file generated (...flatbuffers\src\idl_parser.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(652) : warning C4245: 'return' : conversion from 'int' to 'size_t', signed/unsigned mismatch (...flatbuffers\src\idl_gen_java.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(410) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\idl_parser.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(419) : warning C4244: 'argument' : conversion from 'flatbuffers::uoffset_t' to 'flatbuffers::voffset_t', possible loss of data (...flatbuffers\src\idl_parser.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(640) : warning C4100: 'other' : unreferenced formal parameter (...flatbuffers\src\idl_parser.cpp)
...flatbuffers\include\flatbuffers/flatbuffers.h(652) : warning C4245: 'return' : conversion from 'int' to 'size_t', signed/unsigned mismatch (...flatbuffers\src\idl_parser.cpp)
...flatbuffers\src\idl_gen_text.cpp(157) : warning C4244: 'argument' : conversion from 'const int' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers\src\idl_gen_text.cpp(158) : warning C4244: 'argument' : conversion from 'const int' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers\src\idl_gen_text.cpp(176) : warning C4244: 'argument' : conversion from 'int' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers\src\idl_gen_text.cpp(204) : warning C4244: 'argument' : conversion from 'int' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers\src\idl_parser.cpp(105) : warning C4244: 'argument' : conversion from 'int' to 'char', possible loss of data
...flatbuffers\src\idl_gen_java.cpp(81) : warning C4244: 'argument' : conversion from 'int' to 'char', possible loss of data
...flatbuffers\src\idl_gen_java.cpp(82) : warning C4244: 'argument' : conversion from 'int' to 'char', possible loss of data
...flatbuffers\src\idl_parser.cpp(371) : warning C4244: 'argument' : conversion from 'const int' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers\src\idl_parser.cpp(417) : warning C4244: 'argument' : conversion from 'int' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers\src\idl_parser.cpp(429) : warning C4244: 'argument' : conversion from 'int' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers\src\idl_gen_java.cpp(376) : warning C4100: 'file_name' : unreferenced formal parameter
...flatbuffers\src\idl_gen_text.cpp(141) : warning C4244: 'argument' : conversion from 'const int' to 'flatbuffers::voffset_t', possible loss of data
...flatbuffers\src\idl_gen_text.cpp(191) : see reference to function template instantiation 'void flatbuffers::GenField<uint8_t>(const flatbuffers::FieldDef &,const flatbuffers::Table *,bool,int,int,std::string *)' being compiled
...flatbuffers\include\flatbuffers/util.h(35) : warning C4127: conditional expression is constant (...flatbuffers\src\idl_gen_cpp.cpp)
...flatbuffers\src\idl_gen_cpp.cpp(96) : see reference to function template instantiation 'std::string flatbuffers::NumToString(T)' being compiled
with
[
T=int
]
...flatbuffers\include\flatbuffers/util.h(35) : warning C4127: conditional expression is constant (...flatbuffers\src\idl_gen_java.cpp)
...flatbuffers\src\idl_gen_java.cpp(106) : see reference to function template instantiation 'std::string flatbuffers::NumToString(T)' being compiled
with
[
T=int
]
...flatbuffers\src\idl_gen_text.cpp(37) : warning C4100: 'indent_step' : unreferenced formal parameter
...flatbuffers\src\idl_gen_text.cpp(60) : see reference to function template instantiation 'void flatbuffers::Print(T,flatbuffers::Type,int,int,flatbuffers::StructDef *,std::string *)' being compiled
with
[
T=unsigned char
]
...flatbuffers\src\idl_gen_text.cpp(126) : see reference to function template instantiation 'void flatbuffers::PrintVector<uint8_t>(const flatbuffers::Vector<uint8_t> &,flatbuffers::Type,int,int,std::string *)' being compiled
...flatbuffers\src\idl_gen_text.cpp(37) : warning C4100: 'indent' : unreferenced formal parameter
...flatbuffers\src\idl_gen_text.cpp(37) : warning C4100: 'type' : unreferenced formal parameter
...flatbuffers\include\flatbuffers/util.h(35) : warning C4127: conditional expression is constant (...flatbuffers\src\idl_parser.cpp)
...flatbuffers\src\idl_parser.cpp(48) : see reference to function template instantiation 'std::string flatbuffers::NumToString<size_t>(T)' being compiled
with
[
T=size_t
]
...flatbuffers\include\flatbuffers/util.h(35) : warning C4127: conditional expression is constant (...flatbuffers\src\idl_gen_text.cpp)
...flatbuffers\src\idl_gen_text.cpp(40) : see reference to function template instantiation 'std::string flatbuffers::NumToString(T)' being compiled
with
[
T=unsigned char
]
...flatbuffers\src\idl_gen_text.cpp(60) : see reference to function template instantiation 'void flatbuffers::Print(T,flatbuffers::Type,int,int,flatbuffers::StructDef *,std::string *)' being compiled
with
[
T=unsigned char
]

Currently, GeneratorOptions.strict_json allows serializing fields with quotes.
Parser.Parse fails when the json source contains fields with quotes.

I guess that putShort should be replaced by addShort?

java.lang.IndexOutOfBoundsException: null
    at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_05]
    at java.nio.HeapByteBuffer.putShort(HeapByteBuffer.java:330) ~[na:1.8.0_05]
    at flatbuffers.FlatBufferBuilder.putShort(FlatBufferBuilder.java:102) ~[classes/:na]
    at flatbuffers.FlatBufferBuilder.endObject(FlatBufferBuilder.java:195) ~[classes/:na]

GenStruct does not wrap names of object members with quotation marks. This violation of the standard precludes the use of strict JSON parsers "on the other end".

The help message from flatc may lead one to expect that the "-t" option will parse a binary flatbuffer file and produce text (the inverse of -b). This is not the case - it seems to parse text and produce text as output. Please provide an option to parse binary and product text, so that it is easy to decode binary blobs produced in flatbuffer format.

To geht the length of for example std::vector one has to call the size() function.
The template class Vector should have a function with the same name, calling Length() looks strange.

In lines 227 and 229 of tests/test.cpp, constants are declared that overflow their types. The int32 code is:

const int32_t  int_val    = 0x83333333;

and the int64 code is:

const int64_t  long_val   = 0x8444444444444444;

The two values overflow. For reference, see these calculations:

int32_t: (2**31 - 1) - 0x83333333 = -53687092
int64_t: (2**63 - 1) - 0x8444444444444444 = -307445734561825861

Currently, when a json field appears more than once, we get an assertion failure in FlatBufferBuilder.EndTable (line 501).

Here's my workaround:
dyu@01a0bc5

Not sending it as a PR as it is not a complete solution (limits the table to 64 fields for it to work).

If I use flatc -c /.fbs to generate code for multiple schemas, each of which begins with "namespace foo.bar;", then each generated files gains another level of namespace nesting depending on where it fell in the command line.

The first file contains definitions in namespace foo::bar, the second in foo::bar::foo::bar, and so on.

It'd be nice if accessing and creating byte vectors was as easy as it is for Strings. AFAICT, if you want to create a vector of bytes you have to iterate and add one at a time.

If the API below seems like a reasonable approach I will try to create a PR this weekend.

Creation:

Under the covers this would just delegate to a new method on FlatBufferBuilder that would use System.arraycopy to add the bytes.

byte[] data = getData();
Event.createDataVector(fbb, data);

Event e = Event.getRootAsEvent(bb);
byte[] data = e.dataVector();

Beacuse our game develop with lua,But I don't how to binding it with lua.Is there any paln for lua?

What is the reason of using a root type. If i have Request and Response tables to be sent between two parties, so why we could not create them separately. A buffer could contain either a Request or a Response. In fact any message(Table) should be able to a root in a buffer.

I opend build/VS2010/FlatBuffers.sln in Visual Studio.
A release build fails because warnings are treated as error.
Visual Studio reports 2 warnings in samples/sample_binary.cpp because "inv" and "pos" are initialized but not referenced.

Version is 1.0.1

std::string is used as a accumulator for the output string in GenerateText(...) in idl_gen_text.cpp. Each time one byte is added, the entire string is copied in a new container. At least internally a better way of handling this is probably to use std::ostream &_text as argument for GenStruct (in idl_gen_text.cpp), and wrap in GenerateText the call to GenStruct with

std::stringstream stringaccumulator;
GenStruct(..., stringaccumulator);
_text = stringaccumulator.str();

Scientiffic notation is not handled in json.
Here is an example:

flatc.exe -b monster.fbs monsterdata.json

monsterdata.json:

{
  pos: {
    x: 0.001,
    y: 2,
    z: 3
  },
  hp: 80,
  name: "MyMonster"
}

flatc.exe -b monster.fbs monsterdata.json
flatc.exe: line 3: expecting: , instead got: ♦

{
  pos: {
    x: 1e-3,
    y: 2,
    z: 3
  },
  hp: 80,
  name: "MyMonster"
}

You mention memory efficiency as the main selling point of flatbuffers. However, capnp already exists, with zero copy "serialization," and a range of compelling features (RPC, etc). Still you saw the need to build flatbuffers, which means this library either (a) does something capnp doesn't, or (b) beats capnp in some respect. Could you please describe somewhere what this rationale is? Why should someone pick flatbuffers instead of capnp?

Sometimes we don't know the length of a vector before processing it, so it will be nice to move the num_elems from the startVector to the endVector function, like that we add the length after finishing the vector because at that moment the length should be known.
This is useful for example if we are reading vector data from a stream, or copying only positive elements of an array etc.

I am using a vanilla version of gcc 4.8.3 and recent commits to flatbuffers have caused builds to fail:

First off, commit ebac1e1 caused this build failure:

[  7%] Building CXX object CMakeFiles/flatc.dir/src/idl_parser.cpp.o
In file included from /home/amehta/workspace/external/flatbuffers/src/idl_parser.cpp:21:0:
/home/amehta/workspace/external/flatbuffers/include/flatbuffers/util.h: In function ‘int flatbuffers::ToUTF8(uint32_t, std::string*)’:
/home/amehta/workspace/external/flatbuffers/include/flatbuffers/util.h:157:29: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
     if (ucc < (1 << max_bits)) {  // does it fit?
                             ^
cc1plus: all warnings being treated as errors

  if (ucc < static_cast<uint32_t>(1 << max_bits)) {  // does it fit?

[amehta@amehta-ld1 build]$ make
Scanning dependencies of target flatc
[  7%] Building CXX object CMakeFiles/flatc.dir/src/idl_parser.cpp.o
[ 15%] Building CXX object CMakeFiles/flatc.dir/src/idl_gen_cpp.cpp.o
[ 23%] Building CXX object CMakeFiles/flatc.dir/src/idl_gen_java.cpp.o
[ 30%] Building CXX object CMakeFiles/flatc.dir/src/idl_gen_go.cpp.o
[ 38%] Building CXX object CMakeFiles/flatc.dir/src/idl_gen_text.cpp.o
[ 46%] Building CXX object CMakeFiles/flatc.dir/src/flatc.cpp.o
Linking CXX executable flatc
[ 46%] Built target flatc
Scanning dependencies of target flatsamplebinary
[ 53%] Building CXX object CMakeFiles/flatsamplebinary.dir/samples/sample_binary.cpp.o
In file included from /home/amehta/workspace/external/flatbuffers/samples/sample_binary.cpp:19:0:
/home/amehta/workspace/external/flatbuffers/samples/monster_generated.h: In member function ‘bool MyGame::Sample::Monster::Verify(const flatbuffers::Verifier&) const’:
/home/amehta/workspace/external/flatbuffers/samples/monster_generated.h:65:32: error: ‘VerifyTable’ was not declared in this scope
     return VerifyTable(verifier) &&
                                ^
/home/amehta/workspace/external/flatbuffers/samples/monster_generated.h: In function ‘bool MyGame::Sample::VerifyAny(const flatbuffers::Verifier&, const void*, uint8_t)’:
/home/amehta/workspace/external/flatbuffers/samples/monster_generated.h:111:95: error: no matching function for call to ‘flatbuffers::Verifier::VerifyTable(const MyGame::Sample::Monster*) const’
     case Any_Monster: return verifier.VerifyTable(reinterpret_cast<const Monster *>(union_obj));
                                                                                               ^
/home/amehta/workspace/external/flatbuffers/samples/monster_generated.h:111:95: note: candidate is:
In file included from /home/amehta/workspace/external/flatbuffers/samples/sample_binary.cpp:17:0:
/home/amehta/workspace/external/flatbuffers/include/flatbuffers/flatbuffers.h:693:29: note: bool flatbuffers::Verifier::VerifyTable(const T*) [with T = MyGame::Sample::Monster] <near match>
   template<typename T> bool VerifyTable(const T *table) {
                             ^
/home/amehta/workspace/external/flatbuffers/include/flatbuffers/flatbuffers.h:693:29: note:   no known conversion for implicit ‘this’ parameter from ‘const flatbuffers::Verifier*’ to ‘flatbuffers::Verifier*’
In file included from /home/amehta/workspace/external/flatbuffers/samples/sample_binary.cpp:19:0:
/home/amehta/workspace/external/flatbuffers/samples/monster_generated.h: In function ‘bool MyGame::Sample::VerifyMonsterBuffer(const flatbuffers::Verifier&)’:
/home/amehta/workspace/external/flatbuffers/samples/monster_generated.h:118:112: error: no matching function for call to ‘flatbuffers::Verifier::VerifyBuffer() const’
 inline bool VerifyMonsterBuffer(const flatbuffers::Verifier &verifier) { return verifier.VerifyBuffer<Monster>(); }
                                                                                                                ^
/home/amehta/workspace/external/flatbuffers/samples/monster_generated.h:118:112: note: candidate is:
In file included from /home/amehta/workspace/external/flatbuffers/samples/sample_binary.cpp:17:0:
/home/amehta/workspace/external/flatbuffers/include/flatbuffers/flatbuffers.h:748:29: note: bool flatbuffers::Verifier::VerifyBuffer() [with T = MyGame::Sample::Monster] <near match>
   template<typename T> bool VerifyBuffer() {
                             ^
/home/amehta/workspace/external/flatbuffers/include/flatbuffers/flatbuffers.h:748:29: note:   no known conversion for implicit ‘this’ parameter from ‘const flatbuffers::Verifier*’ to ‘flatbuffers::Verifier*’
make[2]: *** [CMakeFiles/flatsamplebinary.dir/samples/sample_binary.cpp.o] Error 1
make[1]: *** [CMakeFiles/flatsamplebinary.dir/all] Error 2
make: *** [all] Error 2

Feature request: fixed length arrays, especially allowed within structs.

If I want to store a 256-bit hash in a struct, I'd either need to convert to a table in order to use [ubyte], or else store 4 ulongs (which is ugly). Would be convenient if there was a way to store a fixed-length array in a struct.

if there a struct Vec3 in flatbuffer idl file monstor:
struct Vec3 (force_align : 8 ) {
x : float;
y : float;
z : float;
}
after run the command "flatc -c monster.fbs", i got the following codes:
MANUALLY_ALIGNED_STRUCT(8) Vec3 {
private:
float x_;
float y_;
float z_;

public:
Vec3(float x, float y, float z)
: x_(flatbuffers::EndianScalar(x)), y_(flatbuffers::EndianScalar(y)), z_(flatbuffers::EndianScalar(z)) {}

float x() const { return flatbuffers::EndianScalar(x_); }
float y() const { return flatbuffers::EndianScalar(y_); }
float z() const { return flatbuffers::EndianScalar(z_); }
};
STRUCT_END(Vec3, 12);

apparently, the size "12" is not right which should be "16". if i define a object "Vec3 v3", the gcc gives the following prompting “error: static assertion failed: compiler breaks packing rules”

waiting for the author checks this bug.

anyone thinking about go language support or know where one is baking ?

this seems like an excellent way to pass data on a grid of servers.

http://google.github.io/flatbuffers/ scrolling doesn't work on Android (Chrome, Chrome Beta, Dolphin, Firefox, etc...). All touchmove events are getting preventDefault'ed by Javascript, but the page doesn't appear to be doing scrolling itself, so navigation/scrolling is effectively broken.

Various functions in idl_gen_text.cpp always print line feeds. If there is no need for pretty-print, this behaviour should be configurable similar to the amount of indentation.

I think you should document what do you do with unsigned types in java.

New to this amazing lib, I'm very curious about the generated header of monster.fbs
add_name calls AddOffset(10, name);
add_inventory calls AddOffset(14, inventory);

I manually changed 14 to 12, and next color field from 16 to 12, everything works jsut fine!
and from the source code I read, AddOffset just PushElement of length sizeof(voffset_t), so there shouldn't be any problem with that.
And I print all the voffset_t in vtable offsets, field 12 is 0.
Forgive me if I missing something really obviously...

flatbuffers.h is missing #include <algorithm>

In the Java growByteBuffer, you blindly double the array size. This is a bug waiting to happen for some poor soul who overflows int.
One option is to investigate the internals of java.util.HashMap to see a nice way to grow arrays without overflowing. If nothing else, at least try to fail gracefully in this case.

I don't think there is a relation between the Table and the Constants classes, and it should not extend it, If it is just to use the constants then use:
import static flatbuffers.Constants.*;
However there is strong relation between Table and Struct. Table should extend Struct.
Struct has bb, bb_pos and Table has them also with additional function, and in fact a Table is a variable length Struct, so there is a relation between them.
If Table extends Struct then we will be able to add Struct in union also.

When calling the CreateVectorOfStructures function it then passes your std vector through the internal CreateVector function rather than the internal CreateVectorOfStructures one.

Original Code:

template Offset<Vector<const T *>> CreateVectorOfStructs(
const std::vector &v) {
return CreateVector(&v[0], v.size());
}

Changed to:

template Offset<Vector<const T *>> CreateVectorOfStructs(
const std::vector &v) {
return CreateVectorOfStructs(&v[0], v.size());
}

OK, this one is really for Version 2 or something in the distant future, but anyway...

The idea came while writing about FlatBuffers, so I just copy my post here:
My favorite feature of FlatBuffers is it's random access to parts of the data. If you have a big data file, it should be relatively efficient to access just some small parts of it. Of course, you need to have the complete byte buffer in memory, but besides that, there's only overhead for the data you actually look at. This makes it an interesting database format for read-only data. Hmm, if it also had indices for non-linear look-ups...

CMakeLists.txt file is trying to set -Werror and -Wextra compiler flags. These flags do not exist in MSVC. These flags should be set in case of MSVC. The workaround I did was to replace the following line:

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -pedantic -Werror -Wextra")

with:

if(CMAKE_COMPILER_IS_GNUCXX)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -pedantic -Werror -Wextra")
endif(CMAKE_COMPILER_IS_GNUCXX)

It started compiling after that on windows, I have not tried compiling it on Linux though with these changes.

Thanks,
Zarian.

I'm trying to compile flatc.

From the root folder 'flatbuffers-master' I use the command "cmake ." which works.

Then I use "make".

Here I get an error message about multiple definitions of flatbuffer_version_string, being first defined in idl_gen_cpp.cpp

I'm new to this so any help would be appreciated. Is there not a flatc.exe I can just download instead of having to compile?

Regards.

With non-trivial schemas, it becomes cumbersome to create a flat buffer. The main reasons are:

• Objects may not be nested
• Offsets need to be stored for later reference

For deeper data structures this process is not straight forward, because you cannot just traverse your object tree just once. You have to do it multiple times and add data objects to store the offset information.

It would be cool to have this (boiler plate) code generated. Managing offsets will become an internal implementation detail, the developer does not need to worry about. By having some intermediate data structures it should be possible to allow nested creation of object hierarchies. The actual creation of the low level flat buffer might be deferred until all data is complete.

This higher level API could built up on and coexist with the existing API.

I want to use flatbuffers with a ByteBuffer pooler or reuse an existing ByteBuffer, so it will be nice if add a constructor to the FlatBufferBuilder that accepts a ByteBuffer as parameter.

Lets say I have two Java peers that exchange message like these :
table Challenge{x:int;y:int;token:int;}
struct Point{x:int;y:int;}
If a message arrives I have to know its type first, so I want to be able to warp them in:
table Message{type:int;content:nested;}
(the compiler should generate fieldNameType and fieldNameOffset for nested fields and TYPE constant for tables and structs)

and use the following code to write them:

        FlatBufferBuilder fbb = new FlatBufferBuilder(100);
        Challenge.startChallenge(fbb);
        Challenge.addX(fbb, x);
        Challenge.addY(fbb, y);
        Challenge.addToken(fbb, t);
        int c = Challenge.endChallenge(fbb);
        Message.startMessage(fbb);
        Message.addContentType(fbb, Challeng.type); 
        Message.addContentOffset(fbb, c);
        int m = Message.endMessage(fbb);
        fbb.finish(im);

        Message msg=Message.getRootAsMessage(bb, 0);
        int c=msg.contentOffset();
        int t=msg.contentType();
        switch(t){
           case Point.TYPE:
                Point p=Point.getRootAsPoint(bb, c);
                processPoint(p);
                break;
           case Challenge.TYPE:
                Challenge ch = Challenge.getRootAsChallenge(bb, c)
                processChallenge(ch);
                break;
        }

The C++-generated and Java-generated serializations in the tests/ directory differ. Is this explained anywhere? If not, I don't see how to tell whether it's a bug or expected behavior.

More details:

$ diff tests/monsterdata_test_wire.bin tests/monsterdata_java_wire.bin
Binary files tests/monsterdata_test_wire.bin and tests/monsterdata_java_wire.bin differ

$ xxd tests/monsterdata_test_wire.bin
0000000: 2800 0000 0000 0000 0000 0000 1c00 3c00  (.............<.
0000010: 0800 0000 0600 2c00 0000 3000 0000 0500  ......,...0.....
0000020: 3400 3800 0000 0000 1c00 0000 0001 5000  4.8...........P.
0000030: 0000 803f 0000 0040 0000 4040 0000 0000  ...?...@..@@....
0000040: 0000 0000 0000 0840 0400 0500 0600 0000  .......@........
0000050: 0000 0000 4c00 0000 3c00 0000 3000 0000  ....L...<...0...
0000060: 0400 0000 0200 0000 0a00 1400 1e00 2800  ..............(.
0000070: 1c00 0800 0000 0000 0600 0000 0000 0000  ................
0000080: 0000 0000 0000 0000 0000 0000 1c00 0000  ................
0000090: 0000 1400 0500 0000 0001 0203 0400 0000  ................
00000a0: 0900 0000 4d79 4d6f 6e73 7465 7200 0000  ....MyMonster...

$ xxd tests/monsterdata_java_wire.bin
0000000: 2400 0000 0000 0000 1c00 4400 1c00 0000  $.........D.....
0000010: 1a00 1400 0000 1000 0000 0f00 0800 0400  ................
0000020: 0000 0000 1c00 0000 4000 0000 6400 0000  [email protected]...
0000030: 0000 0001 6300 0000 6800 0000 0000 5000  ....c...h.....P.
0000040: 0000 803f 0000 0040 0000 4040 0000 0000  ...?...@..@@....
0000050: 0000 0000 0000 0840 0400 0500 0600 0000  .......@........
0000060: 0000 0000 0000 0000 0200 0000 1e00 2800  ..............(.
0000070: 0a00 1400 1c00 0700 0000 0000 0400 0000  ................
0000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000090: 1c00 0000 1400 0005 0000 0000 0102 0304  ................
00000a0: 0900 0000 4d79 4d6f 6e73 7465 7200 0000  ....MyMonster...

Include guards do not strip directory seperators. Running flatc test/foo.fbs will generate the include guard FLATBUFFERS_GENERATED_TEST/FOO which doesn't compile.

I could just run the command in the directory test but that would make my build script a lot more complicated.

Got "unknown escape code in string constant" on Parser.Parse

Under node repl, it works as expected:

JSON.parse('{"a":"\u0061"}')
{ a: 'a' }

i am writing lots of webgl code

i need to transport lots of data to visualise point clouds and objects.

with web gl. Collada and gltf are used to represent the data.
Collada is xml and so highly verbose
GLTF is much more compact, and the current king of 3d file format compression, without loosing ability to manipulation the data on the client.

Obviously i would need to write a data mapper from collada to FB.

Would it make sense to have flat buffers work i javascript ?

this is a bit of a crazy idea perhaps :)

thinking about how to use flat buffers as a simple database.
As a database, you can ONLY search against pre indexed fields, so you woudl be reading the actual indexes, and then scooping up an object from the FB database which is basically just a file system with locks.
On saving a index would need to be updated. that is orthagonal to the main fB code.
On searching a map reduce style hadoop or the apache storm, and any other map reduce type system.
locking can easily be handled in various ways.

I am wondering though is the properties of flat buffers pay off for read and write to file systems ?

A vector (of byte,int,short,char) in a Table should be returned as a (Byte,Int,Short,Char)Buffer in addition of returning it element by element. The new ByteBuffer will share the same backing array with the original one but with different start and limit positions, like that we can process it separately or send it to the networking without additional bytes. The code could be something like:

public ByteBuffer inventoryAsBB(){int o = __offset(14); bb.position(__vector(o));return o != 0 ? (ByteBuffer)bb.slice().limit(__vector_len(o)) : null;}

This method might be also useful to use in getRootAs if the vectore data is a nested table

ByteBuffer bb=rootMonster.enemyAsBB();
Monter enemy=Monster.getRootAsMonster(bb,0);

Currently, there are only impressive numbers available for the C++ implementation. The Java API is different in a couple of ways, so it would be very interesting how it performs on popular VMs, e.g. Java 7 and Android 4.4.

Apart from:

Parser::Parse(const char *source, const char *filepath)

Consider:

Parser::ParseJson(const char *source, const char *end) // or size_t len

The former code waits for kTokenEof.
The latter would allow callers to provide json inside a buffer.
If the buffer is the same exact size as the json input, there would be no room to insert the null char.
Would be inefficient to allocate a new buffer + 1 just for the null delim.
This is in the context of taking json input from the network/web.

Given a schema like this:

namespace Foo;

table A {
  BLength:int;
  B:[int];
} 

root_type A;

public int BLength() { int o = __offset(4); return o != 0 ? bb.getInt(o + bb_pos) : 0; }
public int BLength() { int o = __offset(6); return o != 0 ? __vector_len(o) : 0; }

namespace Foo;

table T {
}

union U { T }

table A {
  BType:int;
  B:U;
} 

root_type A;

public int BType() { int o = __offset(4); return o != 0 ? bb.getInt(o + bb_pos) : 0; }
public byte BType() { int o = __offset(6); return o != 0 ? bb.get(o + bb_pos) : 0; }

I have built the master I downloaded today and everything built fine with mingw-gcc 4.9-x64 under Windows 8.1-64. But when I run the flattests.exe it fails with a runtime crash and the messages below. Also I do not know how to run and use the other Exes: "flatsamplebinary.exe" and "flatsampletext.exe".

Fail Message:
TEST FAILED: C:\Users\xxx\Downloads\Serializer-Libs\FlatBuffers\flatbuffers-master\tests\test.cpp:155, flatbuffers::LoadFile( "tests/monster_test.fbs", false, &schemafile) (0) != 1
Assertion failed!

Program: C:\Users\xxx\Downloads\Serializer-Libs\FlatBuffers\flatbuffers-master\tests\flattests.exe
File: C:\Users\xxx\Downloads\Serializer-Libs\FlatBuffers\flatbuffers-master\tests\test.cpp, Line 45

Expression: 0

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

how to reproduction:

Currently, flatc does not seem to understand relative paths. So, I've copied monster.fbs from sample to source root. Then, I executed flatc as follows:

hyunsik@Hyunsiks-MacBook-Pro:flatbuffers$ ./flatc -j monster.fbs 
Segmentation fault: 11

(lldb) run -j monster.fbs
Process 85655 launched: './flatc' (x86_64)
Process 85655 stopped
* thread #1: tid = 0x101ba30, 0x00000001000730f2 flatc`flatbuffers::java::GenStructArgs(struct_def=0x0000000100204380, code_ptr=0x00007fff5fbfe5b8, nameprefix=0x00007fff5f4006f1) + 530 at idl_gen_java.cpp:127, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x7fff5f3fffa8)
    frame #0: 0x00000001000730f2 flatc`flatbuffers::java::GenStructArgs(struct_def=0x0000000100204380, code_ptr=0x00007fff5fbfe5b8, nameprefix=0x00007fff5f4006f1) + 530 at idl_gen_java.cpp:127
   124         it != struct_def.fields.vec.end();
   125         ++it) {
   126      auto &field = **it;
-> 127      if (IsStruct(field.value.type)) {
   128        // Generate arguments for a struct inside a struct. To ensure names
   129        // don't clash, and to make it obvious these arguments are constructing
   130        // a nested struct, prefix the name with the struct name.
(lldb) p field.value.type
(flatbuffers::Type) $0 = {
  base_type = BASE_TYPE_FLOAT
  element = BASE_TYPE_NONE
  struct_def = 0x0000000000000000
  enum_def = 0x0000000000000000
}

When you create an empty vector, the eventual call of CreateVector(const T *v, size_t len) leads to a segmentation fault. The error is caused by a do while loop that decrements i = len = 0 and tries to access the element at that position, although its condition is already false.

The do while loop should be changed to a while loop, or a comment should indicate a non-empty vector as a precondition for the CreateVector function family.