unity-technologies / flatsharp Goto Github PK

FlatSharp is Google's FlatBuffers serialization format implemented in C#, for C#. FlatBuffers is a zero-copy binary serialization format intended for high-performance scenarios. FlatSharp leverages the latest and greatest from .NET in the form of Memory<T> and Span<T>. As such, FlatSharp's safe-code implementations are often faster than other implementations using unsafe code. FlatSharp aims to provide 3 core priorities:

• Usability
• Safety & Speed
• Compatibility with other C#-focused projects like Unity, Blazor, and Xamarin.

All FlatSharp packages are published on nuget.org:

• FlatSharp.Runtime: The runtime library. You always need this.
• FlatSharp.Compiler: Build time compiler for generating C# from an FBS schema.

FlatSharp is a mature library and has been shipped to production at Microsoft, Unity3D, and others. Full status can be found at ProjectStatus.md.

If you're completely new to FlatBuffers, take a minute to look over the FlatBuffer overview. Additionally, it's worth the time to understand the different elements of FlatBuffer schemas.

FlatSharp, like other FlatBuffers implementations, uses FBS files to define schemas. Because FlatSharp runs with your build, all code is generated at build time, making FlatSharp compatible with .NET AOT, Blazor, and Unity.

// all FlatSharp FBS attributes start with the 'fs_' prefix.
attribute "fs_serializer";

namespace MyNamespace;

enum Color : ubyte { Red = 1, Green, Blue }

table Person (fs_serializer) {
    Id:int;
    Name:string;
    Parent:Person (deprecated);
    Children:[Person];
    FavoriteColor:Color = Blue;
    Position:Location;
}

struct Location {
    Latitude:float;
    Longitude:float;
}

rpc_service PersonService {
    GetParent(Person):Person;
}

Serialization is easy!

Person person = new Person(...);
int maxBytesNeeded = Person.Serializer.GetMaxSize(person);
byte[] buffer = new byte[maxBytesNeeded];
int bytesWritten = Person.Serializer.Serialize(buffer, person);

Deserializing is easier!

Person p = Person.Serializer.Parse(data);

FlatSharp supports some interesting features not covered here. Detailed documentation is in the wiki. The samples solution is a good tutorial and has full examples of:

• Basic Usage
• Deserialization Modes (Lazy, Greedy, and everything in between)
• Vectors (Lists)
• IO Options
• gRPC
• Copy Constructors
• FBS Includes
• Sorted Vectors
• Indexed Vectors
• Unions
• String deduplication
• Fixed-Length Struct Vectors
• Write-Through (update buffers in place)
• Value Type Structs
• Unsafe Options

FlatSharp works by generating subclasses of your data contracts based on the schema that you define. That is, when you attempt to deserialize a MonsterTable object, you actually get back a subclass of MonsterTable, which has properties defined in such a way as to index into the buffer, according to the deserialization mode specified (greedy, lazy, etc).

Serializers are a common vector for security issues. FlatSharp takes the following approach to security:

• All operations are overflow-checked
• No unsafe code; all operations bounds-checked
• No IL generation
• Use standard .NET libraries for reading and writing from memory
• Depth tracking enabled for recursive schemas to prevent stack overflows.

FlatSharp does use some techniques such as MemoryMarshal.Read on certain hot paths, but these usages are narrowly scoped and well tested.

FlatSharp is really, really fast. The FlatSharp benchmarks were run on .NET 7.0, using a C# approximation of Google's FlatBuffer benchmark, which can be found here.

The benchmarks test 4 different serialization frameworks, all using default settings:

• FlatSharp -- 7.0.0
• Protobuf.NET -- 3.1.22
• Google's C# Flatbuffers -- 22.10.26
• Message Pack C# -- 2.4.35

The full results for each version of FlatSharp can be viewed in the benchmarks folder. Additionally, the benchmark data contains performance data for many different configurations of FlatSharp and other features, such as sorted vectors and shared strings.

Serialization benchmarks are not reflective of "real-world" performance, because processes rarely do serialization-only workflows. In reality, your serializer is going to be competing for L1 cache and other resources along with everything else in your program (and everything else on the machine). So while these benchmarks show that FlatSharp is faster by a wide margin, these benefits may not translate to any practical effect in your environment, depending completely upon your own workflows and data structures. Your choice of serialization format and library should be informed by your needs: Do you need lazy access? Do you care about compact message size? Is serialization on your hot path? Don't make your choice based on the results of a benchmark that shows best-case results for all serializers by virtue of that being the only thing running on the machine at that point in time.

This data shows the mean time it takes to serialize a typical message containing a 30-item vector containing a variety of data types:

How much time does it take to parse and then fully enumerate the message from the serialization benchmark?

FlatSharp is licensed under Apache 2.0.