This repository contains the public API definitions for Liftbridge. It is primarily intended for Liftbridge client developers.

The client-facing gRPC API is defined in api.proto.

It is also possible for a client to publish messages to Liftbridge via NATS directly. Liftbridge accepts plain NATS messages, allowing it to make existing subjects durable without any publisher changes. However, these messages will not have some features such as acks.

In order to opt into Liftbridge-specific features, the message must be prefixed with the following header and be encoded as a Message (defined in api.proto).

All Liftbridge messages and RPCs sent over NATS are prefixed with an envelope header. This includes client-facing messages, such as publishes and acks, as well as internal RPCs like replication.

0               8               16              24              32
├───────────────┴───────────────┴───────────────┴───────────────┤
│                          Magic Number                         │
├───────────────┬───────────────┬───────────────┬───────────────┤
│    Version    │   HeaderLen   │     Flags     │    MsgType    │
├───────────────┴───────────────┴───────────────┴───────────────┤
│                       CRC-32C (optional)                      │
└───────────────────────────────────────────────────────────────┘

The Liftbridge magic number is B9 0E 43 B4. This was chosen by random but deliberately restricted to invalid UTF-8 to reduce the chance of a collision. This was also verified to not match known file signatures.

The version byte allows for future protocol upgrades. This should only be bumped if the envelope format changes or if the message encoding changes in a non-backwards-compatible way. Adding fields to the messages should not require a version bump.

Currently, the only supported protocol version is v0, i.e. 0x00.

The header length is the offset of the payload. This is included primarily for safety.

The flag bits are defined as follows:

This is the Liftbridge-specific message type the envelope contains:

The CRC-32C (Castagnoli) is the checksum of the payload (i.e. from HeaderLen to the end). This is optional but should significantly reduce the chance that a random NATS message is interpreted as a Liftbridge message.