C flatcc builder (put) optimization about objectbox-generator HOT 6 OPEN

I can't answer that, but the mentioned list of chained pages will be recycled efficiently if you do cache the builder, and so will several internal stacks. You can control how much memory the builder retains between builds in case you have a large spike. See custom version of the reset call.

from objectbox-generator.

Note: I'm not sure the copy could be avoided, considering how flatcc_builder works - it doesn't produce the final output on the first call, only when calling "finalize". Maybe there's another mode similar to other flatbuffers implementations producing the final buffer right away without the in-between "emitter"?

Maybe @mikkelfj has some suggestions?

from objectbox-generator.

The copy can be avoided, to a degree:

You can call a different version of finalize that accepts a fixed size buffer, and you can call yet another function to detect the size of the buffer required.

However, the default backend emitter of the builder is collecting data in chained pages, not in contiguous memory, so this memory must be copied out in order to read the data as a buffer. And this data generally needs to be aligned to the largest alignment unit used in the buffer. Usually 8 bytes suffice, but char mybuf[4096]; will not be aligned. While this works on most CPU's today, the verifier will still complain due to semantics.

If you want to further optimize, you can provide you own emitter backend. It is essentially a simple callback that receives an iovec array of small chunks of data to copy. There are a few emitter examples in the test directory.

from objectbox-generator.

Another topic: are we able to reuse flatcc_builder_t? I don't recall if we can cache it.

PS.: flatcc related optimizations can be done after 1.0

from objectbox-generator.

Also see the discussion about an overload for generated _put() and _get() functions #11 (review)

from objectbox-generator.

Also to keep in mind: there's an internal discussion/feature request for providing flatbuffers serialization as part of the C-API. It's mainly for other language bindings but it would also solve this issue (data copy), simplify the code generator (we could have the same code for C++ and C) and drop the C flatc dependency for end-users altogether. Internal issue number: 363

from objectbox-generator.