Could you share the design rationale? Thanks!

go.test.sh run successful now.
when start vasto sever or store，I encounter below errors：

E1013 00:06:29.768819 8411 store_in_cluster.go:30] read file /tmp/go-build169079065/cluster.config: open /tmp/go-build169079065/cluster.config: no such file or directory

does it need certrain cluster.config files？

I tried to use the vasto bench to test it. first open one terminal for vasto master or server，then open the other for vasto bench，but vasto bench always showed as below and won't make any progress again：

benchmarking on cluster with master localhost:8278
put : --- [--------------------------------------------------------------------] 0% NaN ops/sec

The vasto master or server side just printed out the message such as：
I1013 10:05:21.397870 8502 master_server.go:41] Vasto master starts on :8278
I1013 10:05:30.134602 8502 master_grpc_server_for_client.go:30] + client 127.0.0.1:54859
E1013 10:05:30.135152 8502 master_grpc_server_for_client.go:96] master add client benchmarker: client key is already in use: benchmark:dc1:127.0.0.1:54859
I1013 10:05:30.135119 8502 master_grpc_server_for_client.go:30] + client 127.0.0.1:54861
E1013 10:05:30.152217 8502 master_grpc_server_for_client.go:96] master add client benchmarker: client key is already in use: benchmark:dc1:127.0.0.1:54861
I1013 10:06:31.644583 8502 master_grpc_server_for_client.go:41] - client 127.0.0.1:54859
I1013 10:06:31.645002 8502 master_grpc_server_for_client.go:41] - client 127.0.0.1:54861
I1013 10:06:39.134168 8502 master_grpc_server_for_client.go:30] + client 127.0.0.1:54864
E1013 10:06:39.134886 8502 master_grpc_server_for_client.go:96] master add client benchmarker: client key is already in use: benchmark:dc1:127.0.0.1:54864
I1013 10:06:53.839102 8502 master_grpc_server_for_client.go:41] - client 127.0.0.1:54864

I tried to use vasto bench -c1 parameter， but it did not change anything for the bench result.
Is there any settings I miss？ In my knowledge, vasto server include both vasto store and master, so it should be fine to open a vasto server for vasto bench to test.
I just want to make a benchmark test about the vasto performance，could you provide detail instructions to benchmark vasto ？

This is a question for understanding data movement during adding or removing a node to cluster.

In my understanding, jump consistent hashing does not hash node on the ring. It seems like their is no ring concept. The output of hash is a number in [0, nodes) range which maps to a node.

If a new node is added to cluster, how will we know which all keys need to move to this node ?
Will we have to go to all nodes, read all keys and see if they belong to this new node ?

Can you point me to code which does this ?

In ring based consistent hashing, new node will be in ring and will copy keys from next node to itself. So, it may require going over all keys on only one node. However, in real world, one node maps to many buckets on the ring, so we may have to go to multiple nodes in ring based consistent hashing as well.

// start one vasto shell
vasto shell
create cluster ks1 dc1 3 1
use keyspace ks1
desc ks1 dc1

delete cluster ks1 dc1

the last line should be delete

I get my bechmark on a i7 7550/16G laptop
time ./vasto bench --keyspace=ks1 -n 1024000 -c 16
benchmarking on cluster with master localhost:8278
put : 35s [===================================================================>] 100% 29145.12 ops/sec
put,get : 29058.5 op/s
Microseconds per op:
Count: 1024000 Average: 527.6445 StdDev: 371.17
Min: 0.0000 Median: 464.7338 Max: 26443.0000
put : 35s [====================================================================] 100% 29142.38 ops/sec
get : 31s [===================================================================>] 100% 32221.78 ops/sec
put,get : 32110.0 op/s
Microseconds per op:
Count: 1024000 Average: 470.7078 StdDev: 201.14
Min: 0.0000 Median: 464.2912 Max: 15578.0000
put : 35s [====================================================================] 100% 29142.38 ops/sec
get : 31s [====================================================================] 100% 32218.41 ops/sec
real 1m7.219s
user 0m43.063s
sys 1m17.438s

By the way，is it possible to replace the embed rocksdb with other KV engine？
For most of other KY engines，there is no equivalent Merge operation as Roscksdb‘s. Only some simple put/get/delete/iterate/batch write operations

I think about distributed kv, but i want to use ceph crushmap to ave ability to specify how much replicas needed and provide failure domains. what do you think?
(i'm port some parts of ceph crushmap parsing code to go - https://github.com/unistack-org/go-crush )

I tried to test vasto primary/replica failure restore latency with create cluster ks1 dc1 2 2.
However，the created primary/replica store is both on the same store/process even I had added more than one store to the cluster，both primary/replica shard will shutdown in machine failure situation.
Is it intentional setting？
In that case，how can I test the primary/replica failure restore latency？

I manage to read the code，it seems that vasto client is only send its rocksDB requests to primary shard by default.
In case of primary hitch，vasto will update primary/replica using the PeriodicTask to synchronize the new rocksDB requests and replica router information to the client.
So the minimum failure restore latency will be time for primary/replica sync plus the PeriodicTask gap？
Could you explain the primary/replica sync mechanism if I miss some thing？

OS:ubuntu; gcc version 7.3.0; go 1.10.4; and rocksdb master branch with shared lib: librocksdb.so.5.17
after export LD_LIBRARY_PATH=/usr/local/lib, the go.test.sh is running now,
but still fail for the below message:

ok github.com/chrislusf/vasto/storage/binlog 0.097s coverage: 82.4% of statements
ok github.com/chrislusf/vasto/storage/codec 0.027s coverage: 94.4% of statements
free(): invalid pointer
FAIL github.com/chrislusf/vasto/storage/rocks 0.083s

github.com/chrislusf/gorocksdb

exec: "gcc": executable file not found in %PATH%

github.com/chrislusf/vasto/cmd/master

cmd\master\master_server.go:56: cannot use ms (type *masterServer) as type pb.VastoMasterServer in argument to pb.RegisterVastoMasterServer:
*masterServer does not implement pb.VastoMasterServer (wrong type for CompactCluster method)
have CompactCluster("context".Context, *pb.CompactClusterRequest) (*pb.CompactClusterResponse, error)
want CompactCluster("golang.org/x/net/context".Context, *pb.CompactClusterRequest) (*pb.CompactClusterResponse, error)

Recently I have been thinking about what to do with my Master's degree graduation project. Today, I came across this idea as mentioned in the title. Is it valuable? If so, is it feasible?

Hi, any plans for replacing RocksDb with pure Go version like Badger, Bolt, Bitcask,... ?

my enviroment is ubuntu, go 10.4.
vasto use the current master branch.
go build shows the below message:

google.golang.org/genproto/googleapis/rpc/status

../../../google.golang.org/genproto/googleapis/rpc/status/status.pb.go:111:28: undefined: proto.InternalMessageInfo

Do you have any plans for java driver?

Using consistent hash for sharding will take more time to migrate between shards，and range key queries would be hard to implement.
Would you consider key range sharding for another sharding strategy？

@chrislusf
I would be obliged if you could compare between vasto and cassandra
-- why Vasto will be needed when we can run Cassandra with RocksDB (as engine)

OS：ubuntu ，go version 1.10.4
go get github.com/chrislusf/vasto

github.com/chrislusf/gorocksdb

gopath/src/github.com/chrislusf/gorocksdb/options.go:1045:2: could not determine kind of name for C.rocksdb_options_set_allow_ingest_behind

I have successful go get&build tecbot/gorocksdb，
however I fail to go get chrislusf/gorocksdb and chrislusf/vasto