Giter VIP home page Giter VIP logo

toeverything / octobase Goto Github PK

View Code? Open in Web Editor NEW
1.1K 15.0 77.0 12.22 MB

๐Ÿ™ OctoBase is the open-source database behind AFFiNE, local-first, yet collaborative. A light-weight, scalable, data engine written in Rust.

Home Page: https://octobase.dev

License: GNU Affero General Public License v3.0

Rust 74.68% TypeScript 9.82% HTML 0.03% C 4.30% Kotlin 1.15% Java 1.40% Shell 0.02% Swift 6.02% JavaScript 1.75% CSS 0.49% MDX 0.33%
collaborative self-contained decentralized-storage database p2p rust serverless crdt storage

octobase's Introduction

Octobase
๐Ÿ™ Local-first, yet collaborative. A light-weight, scalable, data engine written in Rust.

Rust-version-icon codecov stars Issues Closed Join Telegram Follow Twitter

OctoBase is an offline-available, scalable, self-contained collaborative database, which was originally designed for AFFiNE. AFFiNE is a local-first open source knowledge base that provides full functionality in any network environment.

Based on OctoBase, you can not only implement a rich text editor for offline writing, but also implement richer offline collaboration functions based on OctoBase's data abstraction, such as: multidimensional tables, drawing boards, etc.

Features

As an offline collaborative data database, OctoBase has the following characteristics:

  • ๐Ÿ“š Multi-platform available offline collaboration, Schemaless, structured/unstructured/rich text data storage.

  • ๐Ÿ—ƒ๏ธ Binary storage that supports data deduplication and rich media editing.

  • ๐Ÿ” High-performance real-time full-text indexing with high-quality multilingual word segmentation support.

  • ๐ŸŒ CRDT-driven P2P synchronization with rich multi-platform native support.

  • ๐Ÿ”’ Fine-grained permission control with advanced permission management.

OctoBase provides native support for offline collaboration, full-text indexing, and binary storage, making it easy for developers to build secure and high-performance local-first collaborative applications that work seamlessly across multiple platforms.

With OctoBase, you will have access to same data abstractions across platform that enable you to maintain consistency and coherence across all your applications, regardless of the devices or platforms used.

Additionally, OctoBase can function as a standalone server database, or it can be integrated directly into your application as an embedded database while remaining fully functional.

Project status

The OctoBase project is currently under heavy development, most components are not yet production ready. Major changes may occur at any time before the version reaches 1.0.

OctoBase has been used in AFFiNE Cloud. We are currently working on the following components:

  • Core
    • โœ… CRDT operation (based on y-octo)
    • โœ… CRDT-based data structure compatible with blocksuite
    • ๐Ÿšง Full-text indexing
  • Sync
  • Storage
    • โœ… Storage agnostic CRDT data storage
      • โœ… SQLite adapter
      • โœ… Postgres adapter
      • ๐Ÿšง S3 adapter
    • โœ… Storage agnostic Blob storage
      • โœ… SQLite adapter
      • โœ… Postgres adapter
      • ๐Ÿšง S3 adapter
  • Binding

Contributions

A very good place to ask questions and discuss development work is our Telegram Group.

We gladly accept contributions via GitHub pull requests, you can go to contributing to see more information.

Before you start contributing, please make sure you have read and accepted our Contributor License Agreement. To indicate your agreement, simply edit this file and submit a pull request.

Goals

OctoBase aims to make it easy for developers to build local-first applications on all common platforms. In order to achieve this goal, we will strive to do these things:

  • Make it easy to build on all supported platforms.
  • CRDT-driven peer-to-peer synchronization model.
  • Self-contained library distribution.
  • Minimize external dependencies.
  • Out-of-the-box permission control.

You can see a summary of our recent updates here: What's New.

Project Overview

โ”œโ”€โ”€ apps
โ”‚   โ”œโ”€โ”€ android ##  Android scaffolding project
โ”‚   โ”œโ”€โ”€ homepage ## OctoBase homepage & docs
โ”‚   โ”œโ”€โ”€ cloud ## affine-cloud backend
โ”‚   โ””โ”€โ”€ keck ## collaboration backend
โ”‚   โ””โ”€โ”€ swift ## iOS scaffolding project
โ””โ”€โ”€ libs ##
    โ”œโ”€โ”€ jwst  ## OctoBase core library
    โ”œโ”€โ”€ jwst-binding ## Multilingual FFI bindings for OctoBase
    โ”œโ”€โ”€ jwst-codec ## CRDT implementation for OctoBase
    โ”œโ”€โ”€ jwst-logger ## logger plugins for OctoBase
    โ”œโ”€โ”€ jwst-rpc ## sync plugins for OctoBase
    โ”œโ”€โ”€ jwst-storage ## storage plugins for OctoBase

In the process of project development, there are many software development concepts that have influenced us. Thank you very much for these excellent software:

  • Fossil -- Source code management tool made with CRDTs which inspired our design on block data structure.
  • SQLite -- "Small. Fast. Reliable. Choose any three." We like this idea very much.

Hiring

Some amazing companies including OctoBase are looking for developers! Are you interested in helping build with OctoBase and/or its partners? Check out some of the latest jobs available.

Building

See BUILDING.md for instructions on how to build Octobase from source code.

License

Currently, this repository is under active development and most components are not yet production ready, so all code is under AGPL-3.0. We will switch to MPL 2.0 or a more looser license release after the corresponding components are ready for production.

octobase's People

Contributors

a-wing avatar anchpop avatar austaras avatar brooooooklyn avatar colelawrence avatar darkskygit avatar diamondthree avatar doodlewind avatar forehalo avatar fundon avatar himself65 avatar imsingee avatar jimmfly avatar lawvs avatar linonetwo avatar miracleyin avatar thearas avatar thorseraq avatar tzhangchi avatar xuanwo avatar zuoxiaodong0815 avatar zxch3n avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

octobase's Issues

Support single file workspace import & export

One basic design decision in our native client is to persistent the workspace content, including YDoc and blobs, into a single SQLite-based binary file (e.g., my-workspace.db). This allows for intuitive file management on user side and better data ownership.

Meanwhile, based on how CRDT works, when user opens this workspace file, we will automatically duplicate it into the native client data directory, using this implicitly-duplicated workspace file as the single source of truth (on top of our provider-based synchronization), and automatically writes to the original user-selected workspace file. This will make sure the user could always see and keep the latest version of their selected workspace.

@joebeijing will help with more product-side related requirement.

/cc @tzhangchi @HeJiachen-PM

๐Ÿ”ฅ Roadmap for OctoBase pre-alpha

JWSTBase

  • Final tasks has been move to hidden comment

workspace

  • block index @darkskygit
  • cross-docs block or docs relink @darkskygit
    • we can link another docs as a block children, but did we need to hidden this detial as subdoc abstract?
    • when we link doc as a subdoc, how we build the cross-doc block graph?
  • sync block to another workspace @darkskygit

block

Cross Platform

Storage

  • SQLite / MySQL feature
  • stand-alone crate

Keck

Document

  • How to create a basic project with jwst
  • How to organizing data using the Block abstraction

Low Priority

plugin

  • dynamic plugin research: how we create the plugin solution with rust? @darkskygit

Support single page sharing

Stage1

  1. feat mod:base
    darkskygit

Stage2

  1. feat mod:base mod:libs
    darkskygit
  2. feat mod:base
    darkskygit
  3. feat mod:libs
    darkskygit
  4. feat mod:base mod:libs
    darkskygit

Stage3

  1. feat mod:base
  2. feat mod:storage mod:sync
  3. feat mod:base mod:sync

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.