- Avoid vendor lock-in, your data stays with you.
- Built for operational simplicity
- no infra to setup and manage apart from the storage bucket.
- Designed for correctness
- sync protocol is causally consistent under concurrent writes.
- Web optimized, 10x smaller than the AWS S3 browser client.
- Offline-first, fast page loads and no lost writes.
Tested with S3, Backblaze, R2 and self-hosted solutions like Minio. Interactive demo available on Observable
MPS3 is a key-value document store. A manifest lists all keys in the DB as references to files hosted on s3. Setting a key first writes the content to storage, then updates the manifest. To enable subscriptions, the client polls the manifest for changes. To enable causally consistent concurrent writes, the manifest is represented as a time indexed log of patches and checkpoints which is resolved on read.
MPS3 is built on strong theoretical foundations. Technical articles are written in /docs, some highlights:-
- Randomized, Efficient, Causal consistency checking
- JSON Merge Patch: Algebra and Applications
- The sync protocol for a client-side, causally consistent, multiplayer DB over the S3 API.md)
To use this library you construct an MP3S class.
Quick start (Codepen)
import {MPS3} from 'https://cdn.skypack.dev/[email protected]?min';
const mps3 = new MPS3({
defaultBucket: "<BUCKET>",
s3Config: {
region: "<REGION>",
credentials: {
accessKeyId: "<ACCESS_KEY>",
secretAccessKey: "<SECRET_KEY>"
}
}
});
mps3.put("key", "myValue"); // can await for confirmation
mps3.subscribe("key", (val) => console.log(val)); // causally consist listeners
const value = await mps3.get("key"); // read-after-write consistFor the client to work properly some CORS configuration is required on the bucket so the Javascript environment can observe relevant metadata.
[{
"AllowedHeaders": ["*"],
"AllowedMethods": ["GET", "PUT", "DELETE"],
"AllowedOrigins": ["*"],
"ExposeHeaders": ["X-Amz-Version-Id", "ETag", "Date"]
}]There is no in-built authorization. Every use-case needs different authorization. A malicious user could sabotage the manifest file if they have unrestricted write permissions to the manifest file, but not all use-cases have malicious users. There are a few options:-
- Share access key only to trusted personal.
- If using S3 and IAM, issue STS tokens that grant access to a subpath of a bucket per user/team
- For public use, use a third-party auth solution and a authenticating proxy. Verify manifest changes are valid during passthrough, there is an example of an proxy configuration here that hides credentials from the browser using a CloudFlare worker.
Consult the API Documentation for advanced usage.
- atomic batch operations
- multiple manifests
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent