Giter VIP home page Giter VIP logo

tencent / paxosstore Goto Github PK

View Code? Open in Web Editor NEW
1.7K 114.0 335.0 1.51 MB

PaxosStore has been deployed in WeChat production for more than two years, providing storage services for the core businesses of WeChat backend. Now PaxosStore is running on thousands of machines, and is able to afford billions of peak TPS.

License: Other

C++ 98.28% Makefile 0.52% C 0.28% CMake 0.32% Shell 0.11% Starlark 0.48%
paxos database distributed-database consensus

paxosstore's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

paxosstore's Issues

本地单机模拟测试会失败

在ConnWorker.cpp文件, clsConnWorker::HandleNego函数,
会检查是否为当前paxos内部的IP, 并会在361行会重置ServerId的信息。
如果像 README 例子中一样直接使用本地来启动三个card_srv会引发各种问题。
测试时,可以将ConnWorker.cpp 361行 注释掉。
// poNegoCtx->SetServerID(i);

线上系统注释掉的问题也不大。作为paxos应该信任内部结点的信息, 不存在拜占庭将军问题,如果不信任应该用其他加密方式来初始化链接,而不是暴力重置server_id。

Documentation Link

Hi,

Great work on the project!
I wanted to check if you have any link to the documentation available?

Thanks,
Mark

paxosstore和phxpaxos

我想问问两个项目的区别,根据文档,这里两个项目都声称在wechat里用过,请问有没有性能比较呢?所以wechat里面是两个系统同时在用吗?

编译完后没找到可执行的server程序

Hi,你好,我编译后只看到build/example目录下的membase_paxoskv,这貌似只是用来跑示例程序的,不是server程序,你们是不是还没完全整理完代码呢?

paxoskv如何使用

您好,感觉paxosstore很强大啊,可是不知道是不是我漏看了什么,没有找到暴露给客户端直接能用的put,get接口,请问paxoskv有快速上手的教程么

还有一个问题哈,有paxoskv + rpc的服务接口么

Paxosstore数据模型

看了源码。目前开源的paxoskv是Log-as-value设计方式,功能只支持kv的set,get,为什么没有del?
还有Key也只能使uint64_t的logid?不明白这个设计为什么?
查看了你们发表的论文,是不是说你们存储的都是消息和图片,都是用唯一ID标识记录的?
如果要支持字符串的KEY,做一个KV存储,好像没法直接用paxoskv。
我的上述理解可能不对,请多多指教,给与解释说明。

无法执行certain中的card-srv及其他工具

我已经按照步骤成功编译出了可执行文件,且没有任何错误,但是执行时,却没有任何反应,即使在里面写一句helloworld亦无法打印,请问是编译出了什么问题还是缺少其他库引起该问题,有人遇到过同样的问题吗

Certain组件的性能问题

请问目前PaxosStore的组件开源完了吗,经测试,Certain的入库性能很差,能否提供官方的性能指标

关于一致性读的疑问

简单的说, 读接口即读当前的plog, 并判定是否是最新数据(分布式角度). 更具体的说: 
-  读接口LocalChosenIndex + 1 == OtherMaxIndex  当且仅当OtherMaxIndex均
    为Pending时, 本地chosen数据最新

这里我有一个疑问, OtherMaxIndex均能确定处于Pending时,是否是即使处于这么一个中间状态:paxos acceptors已经形成多数派但是设置的proposer并没有到达chosen,就当作这种状态还没有真的chosen.

另外源码中,

int can_read_3svr(..) {
 ...
 if (is_peer_chosen(peer_status)) {               
        return PAXOS_GET_LOCAL_OUT; 
 }
return PAXOS_GET_MAY_LOCAL_OUT;
}

跟note文档描述并不是一致的。在远端都不是chosen的情况,选择了重做写并等待或返回失败。

我的第二个疑问是, 源码是不是选择了更严格的模式,因为存在上面说的中间的状态的情况,选择了重做完等待或者返回失败

PaxosKV在跨机房环境下响应耗时

Hi,挺感谢你们能够开源paxosstore,想问下,"微信PaxosStore内存篇:十亿Paxos/分钟的挑战" 这篇文章结尾处提供了PaxosKV的压测性能,平均耗时是7.28ms,这是在6台机器都部署在同机房的环境下测的吗?在跨机房部署的环境下,相应耗时会怎么样?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.