Giter VIP home page Giter VIP logo

tencent / msec Goto Github PK

View Code? Open in Web Editor NEW
2.7K 340.0 746.0 39.19 MB

Mass Service Engine in Cluster(MSEC) is opened source by QQ team from Tencent. It is a backend DEV &OPS engine, including RPC,name finding,load balance,monitoring,release and capacity management.

License: Other

Shell 0.37% CMake 0.02% Java 56.78% C++ 24.80% C 8.13% Makefile 0.90% Python 1.01% CSS 2.58% JavaScript 4.76% HTML 0.01% Assembly 0.02% PHP 0.35% Objective-C 0.01% M4 0.09% Roff 0.01% TSQL 0.16%

msec's People

Contributors

alex1007 avatar joshle avatar msecowner avatar senorsen avatar tencent-adm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

msec's Issues

PHP的拓展库添加后不能正常运行怎么办?

原本装的是PHP 7.1,不能用,所以重新装一次,装回了5.6.29,但是还是报错的!路径确认过并没有问题,`[root@mail ~]# php -m

PHP Warning: PHP Startup: nlb_php: Unable to initialize module
Module compiled with build ID=API20131226,TS,debug
PHP compiled with build ID=API20131226,NTS
These options need to match
in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/local/php/lib/php/extensions/no-debug-non-zts-20131226/srpc_comm_php.so' - libsrpc_proto_php_c.so: cannot open shared object file: No such file or directory in Unknown on line 0
`

es扩容失败

es包发送已经成功了,但是部署时显示“ES fails to start, please check if the directory or the port is occupied”。还有一个疑问,单独部署es的时候提示需要用非root用户,部署agent的时候又需要是root用户,这里是不是矛盾

shm_statistics & server_statistics

nlb/api/nlbapi_test.c中,line 115~117用到了上面这两个结构体,但是找不到他们定义,请把这两个结构体补充上,谢谢。

fatal bug: 除零错误

在测试nlb/src/api/nlbapi_test.c的时候发现:
nlb/src/api/nlbapi.c: calc_success_ratio最后一步req_total结果为0!!!

修改为:

float calc_success_ratio(struct shm_servers *shm_servers, struct server_info *server)
{
    uint32_t req_total;

    req_total = server->failed + server->success;

    if (req_total < shm_servers->shaping_request_min) {
        return 100.0;
    }

    return ((float)server->success) / (req_total == 0 ? 1e-6f : (float)(req_total));
}

nlb/src/comm/nlbfile.c:check_and_mkdir权限错误

在测试nlb/agent的时候,发现用户vag启动进程agent,agent进程创建的/var/nlb,vag用户却进不去,最后发现是nlb/src/comm/nlbfile.c:check_and_mkdir代码写的有瑕疵,现改动如下:

mode_t getumask()
{
    mode_t mask = umask(0);
    return mask;
}

/**
* @brief 检查并创建目录
*/
bool check_and_mkdir(const char *path)
{
    if (!path) {
        return false;
    }
 
    if (!check_dir_exist(path)) {
        mode_t mask = getumask();
        if (mkdir(path, ~mask & 0777) < 0)  {
         return false;
        }
    }
 
    return true;
}

get_third_party.sh下载失败

download zlib for monitor_server:

--2017-01-24 13:54:48-- http://zlib.net/zlib-1.2.8.tar.gz
正在解析主机 zlib.net (zlib.net)... 69.73.132.10
正在连接 zlib.net (zlib.net)|69.73.132.10|:80... 已连接。
已发出 HTTP 请求,正在等待回应... 404 Not Found
2017-01-24 13:54:49 错误 404:Not Found。

Description和Readme.md语言不统一

Description里是英文,点进项目里后发现Readme.md是中文。在Github Trending里只能看到项目名称和Description,可能造成非中文用户的误解。

服务状态的说明

服务的状态没有一个完整的说明,什么时候enable,什么时候disable,一直中间的切换机制.

性能问题

您好,拿到了源码以后,第一时间看了下源码中的文档,发现测试报告中,java和c++ 处理差距很大啊,长连接相差了几百倍,这个框架是对java支持不友好吗???还是因为别的问题???麻烦告知下,万分感谢。

msec/nlb下的loadreport

最近研究nlb,nlb/src/agent/zkloadreport用于server mode下定时向zk节点/loadreport/10.0.0.1设置整个server的load信息:{timestamp: ms, cpu: cpu_percent, mem_total: mem_total, mem_free: mem_free}。

我的问题是:这些上报的值在哪个模块被使用?我穷尽洪荒之力去查找也没有找到,恕我资质愚钝,请bisonliao之后的维护者出于严谨的态度指导下,这个值是怎么用的呢?

monitor/agent/monitor_api/opt_time.h:Frequent_Attr_API_Ext多余

上面这个宏你虽然定义了,但是没见到调用的地方。

另外,这个宏里面使用的Attr_API这个接口,如果记忆不错的话,这是腾讯内部monitor的上报接口吧?这个内部的东西的实现你没有暴露出来,你调用他干嘛呢?

题外话:腾讯内部上报Attr_API这个借口是个整数id,一个整数挺简洁的,无非是后台多一个table,记录整数对应的服务+属性名称,你们却用了service+attr替代之,搞了这么长的字符串,实在想不明白有嘛好处?

求解惑?

1、该引擎能否集成(简易配置)添加之已有系统?

2、各个语言模块是否最优选择(可选一种代替)?

3、模块是否能相互独立成一个Project?

4、是否有合理的竞争力?(相同产品之间的差异及优势)

5、能傻瓜式部署(一键式)?

6、能否提供集成改产品的整套(简易)案例来参考?

7、TBD。

异构服务添加失败

异构的外部组件增加IP:PORT提示添加成功,然后再也找不到添加的内容,数据库表t_second_level_service_ipinfo中无内容,且无日志可查询;

制定发布计划时提示send file to *** failed:java.net.ConnectException: Connection refused

发布时提示send file to **** failed:java.net.ConnectException: Connection refused,这个是什么原因呢?

业务机满足机器的配置要求、业务运营机agent命令执行结果如下:
starting nlb...
Nlb version: 0.1.5
Zookeeper c libary version: 3.4.8
mode : 3 (1:SERVER_MODE 2:CLIENT_MODE 3:MIX_MODE)
timeout : 10000 (zookeeper timeout config)
port : 2841 (nlb agent listen port)
local addr : ***** (local interface address)
log level : 1 (1: ERROR 2: WARN 3: INFO 4:DEBUG)
zk host : :8972 (zookeeper server host)
2017-05-02 15:38:49,421:9608:ZOO_INFO@log_env@726: Client environment:zookeeper.version=zookeeper C client 3.4.8
2017-05-02 15:38:49,421:9608:ZOO_INFO@log_env@730: Client environment:host.name=localhost.localdomain
2017-05-02 15:38:49,421:9608:ZOO_INFO@log_env@737: Client environment:os.name=Linux
2017-05-02 15:38:49,421:9608:ZOO_INFO@log_env@738: Client environment:os.arch=3.10.0-327.el7.x86_64
2017-05-02 15:38:49,421:9608:ZOO_INFO@log_env@739: Client environment:os.version=#1 SMP Thu Nov 19 22:10:57 UTC 2015
2017-05-02 15:38:49,421:9608:ZOO_INFO@log_env@747: Client environment:user.name=user
2017-05-02 15:38:49,421:9608:ZOO_INFO@log_env@755: Client environment:user.home=/root
2017-05-02 15:38:49,421:9608:ZOO_INFO@log_env@767: Client environment:user.dir=/msec/agent/nlb/bin
2017-05-02 15:38:49,421:9608:ZOO_INFO@zookeeper_init@800: Initiating client connection, host=
:8972 sessionTimeout=10000 watcher=0x404538 sessionId=0 sessionPasswd= context=(nil) flags=0
Agent start success!!
2017-05-02 15:38:49,425:9608:ZOO_INFO@check_events@1728: initiated connection to server [
:8972]
2017-05-02 15:38:49,430:9608:ZOO_INFO@check_events@1775: session establishment complete on server [
*:8972], sessionId=0x15bbf4b06f50007, negotiated timeout=10000
nlb agent started
root 9608 0.0 0.0 15736 952 ? Ss 15:38 0:00 ./nlbagent ****:8972 -m mix -i ens32
./start.sh:行2: killall: 未找到命令
Monitor agent started
no crontab for root
no crontab for root
Logsys Flume start ok.

added system monitor report

          • /msec/agent/monitor/sysmon.py >>/msec/agent/monitor/sysmon.log 2>&1 &

added for crontab logsys

*/1 * * * * /msec/agent/logsys/tools/auto_reflash_conf.sh > /dev/null 2>&1 &
*/1 * * * * /msec/agent/logsys/tools/check_process.sh > /dev/null 2>&1 &
starting remote_shell agent...
load rsa public key from pub.txt
remote_shell_agent started

msec qq群684241802

给官方提issue,响应太慢,只能依靠大伙自己的力量了,感兴趣的加入。为了msec继续发展,请鹅厂的老朋友不要close这个issue,多谢。

能否提供架构图

感谢鹅厂开源!

能否提供架构图以及部署方法

或者提供一个QuickStart用来体验一下。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.