Giter VIP home page Giter VIP logo

rosedb's Introduction

rosedb_ico.png

 Go Report Card GitHub top language GitHub stars codecov CodeFactor Go Reference Mentioned in Awesome Go

English | 简体中文

rosedb is not production-ready, we are doing more test now.

rosedb is a fast, stable and embedded key-value (k-v) storage engine based on bitcask. Its on-disk files are organized as WAL(Write Ahead Log) in LSM trees, optimizing for write throughput.

rosedb supports a variety of data structures such as string, list, hash, set and sorted set.

rosedb is written in pure Go, easy to understand and use.

Our vision is to build an efficient k-v storage engine for Go-based applications. Feel free to give us any advices, and also please give us a star ✨,that helps a lot, thanks!

Stargazers over time

Features

  • Supports all common data structures : string, list, hash, set, zset.
  • Easy to embed (import "github.com/roseduan/rosedb").
  • Low latency and high throughput (see Benchmark).
  • Has built-in parallel execution of data modification on many provided data structures.
  • Comes with rosedb-cli for command line access, that is compatible with redis-cli.
  • Supports TTL-based key eviction.
  • Supports prefix scan and range scan for string keys.
  • Support simple transaction, ACID features.
  • Merge operation can be stopped manually.

Usage

Cli example

Navigate to rosedb/cmd/server and run main.go

Xnip2021-04-14_14-33-11.png

Open a new terminal, navigate to rosedb/cmd/cli, and run main.go

Xnip2021-04-14_14-35-50.png

Or you can just use redis-cli or any other redis client:

2021-05-14 上午11.19.24.png

Embedding example

Import rosedb in the application:

import "github.com/roseduan/rosedb"

Open a connection to the database:

package main

import (
	"github.com/roseduan/rosedb"
	"log"
)

func main() {
	config := rosedb.DefaultConfig()
	db, err := rosedb.Open(config)
	
	if err != nil {
		log.Fatal(err)
	}
	
  // don`t forget to close!
	defer db.Close()
	
	//...
}

Deploy and run on Docker

docker build -t="rosedb:v1.2.9" .
docker run --name=rosedb -itd -p 5200:5200 rosedb:v1.2.9
docker exec -it rosedb sh

$ rosedb-cli
127.0.0.1:5200>set hello rosedb
OK
127.0.0.1:5200>get hello
rosedb

Command

String

  • Set
  • SetNx
  • Get
  • GetSet
  • Append
  • StrLen
  • StrExists
  • StrRem
  • PrefixScan
  • RangeScan
  • Expire
  • Persist
  • TTL

List

  • LPush
  • RPush
  • LPop
  • RPop
  • LIndex
  • LRem
  • LInsert
  • LSet
  • LTrim
  • LRange
  • LLen
  • LKeyExists
  • LValExists

Hash

  • HSet
  • HSetNx
  • HGet
  • HGetAll
  • HDel
  • HExists
  • HLen
  • HKeys
  • HVals

Set

  • SAdd
  • SPop
  • SIsMember
  • SRandMember
  • SRem
  • SMove
  • SCard
  • SMembers
  • SUnion
  • SDiff

Zset

  • ZAdd
  • ZScore
  • ZCard
  • ZRank
  • ZRevRank
  • ZIncrBy
  • ZRange
  • ZRangeWithScores
  • ZRevRange
  • ZRevRangeWithScores
  • ZRem
  • ZGetByRank
  • ZRevGetByRank
  • ZScoreRange
  • ZRevScoreRange

TODO

see here.

Benchmark

Environment

  • Go version:1.14.4
  • OS: macOS Catalina 10.15.7
  • CPU: 2.6GHz 6-Core Intel Core i7
  • Memory: 16 GB 2667 MHz DDR4
  • Code: rosedb-bench
  • Other databases for comparison:
    • Badger
    • GoLevelDB
    • Pudge

Result

1,000,000 iterations

go test -bench=. -benchtime=1000000x
badger 2021/05/16 21:59:53 INFO: All 0 tables opened in 0s
badger 2021/05/16 21:59:53 INFO: Discard stats nextEmptySlot: 0
badger 2021/05/16 21:59:53 INFO: Set nextTxnTs to 0
goos: darwin
goarch: amd64
pkg: rosedb-bench
BenchmarkPutValue_BadgerDB-12                    1000000             11518 ns/op            2110 B/op         46 allocs/op
BenchmarkGetValue_BadgerDB-12                    1000000              3547 ns/op            1172 B/op         20 allocs/op
BenchmarkPutValue_GoLevelDB-12                   1000000              4659 ns/op             352 B/op          9 allocs/op
BenchmarkGetValue_GoLevelDB-12                   1000000              2838 ns/op             814 B/op         13 allocs/op
BenchmarkPutValue_Pudge-12                       1000000              8512 ns/op             791 B/op         22 allocs/op
BenchmarkGetValue_Pudge-12                       1000000              1253 ns/op             200 B/op          6 allocs/op
BenchmarkPutValue_RoseDB_KeyValRam-12            1000000              4371 ns/op             566 B/op         11 allocs/op
BenchmarkGetValue_RoseDB_KeyValRam-12            1000000               481 ns/op              56 B/op          3 allocs/op
BenchmarkPutValue_RoseDB_KeyOnlyRam-12           1000000              4255 ns/op             566 B/op         11 allocs/op
BenchmarkGetValue_RoseDB_KeyOnlyRam-12           1000000              2986 ns/op             312 B/op          8 allocs/op
PASS
ok      rosedb-bench    46.388s

2,500,000 iterations

go test -bench=. -benchtime=2500000x
badger 2021/05/16 22:06:08 INFO: All 0 tables opened in 0s
badger 2021/05/16 22:06:08 INFO: Discard stats nextEmptySlot: 0
badger 2021/05/16 22:06:08 INFO: Set nextTxnTs to 0
goos: darwin
goarch: amd64
pkg: rosedb-bench
BenchmarkPutValue_BadgerDB-12                    2500000             11660 ns/op            2150 B/op         46 allocs/op
BenchmarkGetValue_BadgerDB-12                    2500000              4180 ns/op            1222 B/op         21 allocs/op
BenchmarkPutValue_GoLevelDB-12                   2500000              4637 ns/op             336 B/op          9 allocs/op
BenchmarkGetValue_GoLevelDB-12                   2500000              2942 ns/op             817 B/op         14 allocs/op
BenchmarkPutValue_Pudge-12                       2500000              9238 ns/op             763 B/op         22 allocs/op
BenchmarkGetValue_Pudge-12                       2500000              1275 ns/op             200 B/op          6 allocs/op
BenchmarkPutValue_RoseDB_KeyValRam-12            2500000              4474 ns/op             566 B/op         11 allocs/op
BenchmarkGetValue_RoseDB_KeyValRam-12            2500000               525 ns/op              56 B/op          3 allocs/op
BenchmarkPutValue_RoseDB_KeyOnlyRam-12           2500000              4294 ns/op             566 B/op         11 allocs/op
BenchmarkGetValue_RoseDB_KeyOnlyRam-12           2500000              3038 ns/op             312 B/op          8 allocs/op
PASS
ok      rosedb-bench    119.529s

Conclusion

Badger

Stable read and write performance. Write: 11000+ ns/op. Read: 4000+ ns/op.

GoLevelDB

Write performance is almost 2.5x that of Badger. Read is almost 3000 ns/op, a little faster than that of Badger.

Pudge

Write performance is between that of GoLevelDB and Badger, almost 8500 ns/op. Read is very fast and stable, almost twice as fast as that of GoLevelDB.

RoseDB

Write performance is stable, almost the same as that of GoLevelDB, 2.5x that of Badger.

In KeyValueRamMode, since the values are all in memory, it is the fastest of all.

In KeyOnlyRamMode, it is almost the same as GoLevelDB.

Contributing

If you are intrested in contributing to rosedb, please see here: CONTRIBUTING

Contact me

If you have any questions, you can contact me by email: [email protected]

License

rosedb is licensed under the term of the MIT License

rosedb's People

Contributors

dxyinme avatar guoyuanchao1202 avatar hdt3213 avatar impersonality avatar jaylzhou avatar jimersylee avatar nanfeng1999 avatar nc-77 avatar qraffa avatar qshine avatar ricardochou avatar roseduan avatar taiyang-li avatar wsyingang avatar wziww avatar zour-ma avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.