Yrs synchronization and persistence using Kafka and an ephemeral RocksDB instance fed by a compacted topic.
Two topics are required to use yrs-kafka, a compacted topic and a changelog topic. To configure these two topics using Redpanda, run the following commands:
rpk topic create y-compacted -c cleanup.policy=compact
rpk topic create y-changelog use yrs::{updates::decoder::Decode, Text, Transact, Update};
async fn run() {
let yrs_kafka = yrs_kafka::start(yrs_kafka::config::Config {
db_path: "/tmp/yrs-kafka".into(),
kafka: yrs_kafka::config::KafkaConfig {
brokers: vec!["localhost:9092".to_string()],
group_id: "yrs-kafka".to_string(),
changelog_topic: "y-changelog".to_string(),
compacted_topic: "y-compacted".to_string(),
},
})
.unwrap();
let doc = yrs::Doc::new();
let update = yrs_kafka.load_document("my-document-id").await.unwrap();
let update = update.get().as_deref().unwrap();
doc.transact_mut().apply_update(Update::decode_v1(update).unwrap());
let update = {
let text = doc.get_or_insert_text("article");
let mut txn = doc.transact_mut();
text.insert(&mut txn, 0, "hello");
text.insert(&mut txn, 5, " world");
txn.encode_update_v1()
};
yrs_kafka.update(b"my-document-id", update).await.unwrap();
}Documents are assigned a de facto "owner" via partition ownership of the changelog topic. All changes to a given document are handled by the owner and merged into the compacted topic, which is consumed by other instances of the service.
Changes published to the compacted topic are persisted into the local instance's ephemeral RocksDB, except in the case of document owners which merge into their RocksDB instance upon read from the changelog.
Reading document state is performed entirely using the instance's local ephemeral RocksDB.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent