Giter VIP home page Giter VIP logo

slowquery's Introduction

slowquery

Slowquery图形化显示MySQL慢日志工具 https://www.oschina.net/p/slowquery

视频演示:https://www.douyin.com/video/7301328824501914890

背景:由于天兔Lepus慢查询工具是运行在PHP CI框架里,而不是作为一个独立的web页面接口,所以想直接接入自动化运维平台里,移植代码比较困难,固考虑重构。

参考了开源工具Anemometer图形展示思路,并且把小米Soar工具集成进去,开发在页面上点击慢SQL,就会自动反馈优化建议,从而降低DBA人肉成本,同时也支持自动发送邮件报警功能。

agent客户端慢日志采集分析是结合Percona pt-query-digest工具来实现。

需要安装的步骤如下:

1、percona-toolkit工具的安装

2、php web mysql环境的搭建

# yum install httpd mysql php php-mysqlnd -y

3、安装Slowquery并配置

4、导入慢查询日志

5、访问界面,查看慢查询

6、配置邮件报警

image image

点击+号,点下面的SQL,会调用Soar反馈优化建议:

image image

工具搭建配置

1、移动到web目录

mv  slowquery  /var/www/html/

2、进入到slowquery/slowquery_table_schema目录下

导入dbinfo_table_schema.sql和slowquery_table_schema.sql表结构文件到你的运维管理机MySQL里。

(注:dbinfo表是保存生产MySQL主库的配置信息。)

例:

mysql -uroot -p123456 sql_db < ./dbinfo_table_schema.sql

mysql -uroot -p123456 sql_db < ./slowquery_table_schema.sql 

录入你要监控的MySQL主库配置信息

例:

mysql> INSERT INTO sql_db.dbinfo VALUES (1,'192.168.148.101','test','admin','123456',3306);

3、修改配置文件config.php,将里面的配置改成你的运维管理机MySQL的地址(用户权限最好是管理员)

4、修改配置文件soar_con.php,将里面的配置改成你的运维管理机MySQL的地址(用户权限最好是管理员)

5、进入到slowquery/client_agent_script目录下,把slowquery_analysis.sh脚本拷贝到生产MySQL主库上做慢日志分析推送,并修改里面的配置信息

定时任务(10分钟一次)

*/10 * * * * /bin/bash /usr/local/bin/slowquery_analysis.sh > /dev/null 2>&1

6、别的就没啥配置的了,直接打开浏览器访问slowquery.php就OK了。

http://yourIP/slowquery/slowquery.php

加一个超链接,可方便地接入你们的自动化运维平台里。

7、慢查询邮件推送报警配置。进入到slowquery/alarm_mail/目录里,修改sendmail.php配置信息

定时任务(每隔3小时慢查询报警推送一次)

0 */3 * * * cd /var/www/html/slowquery/alarm_mail;/usr/bin/php  /var/www/html/slowquery/alarm_mail/sendmail.php > /dev/null 2>&1

image


2023年9月13日更新: 用自研的sql_helper替换掉soar,无需部署,直接拉取docker pull slowquery即可。

服务端

拉取镜像

shell> docker pull docker.io/hcymysql/slowquery:2023-09-13

启动

shell> docker run -itd -e "TERM=xterm-256color" --privileged --name slowquery -p 80:80 -p 3306:3306 <IMAGE ID> /usr/sbin/init

进入docker里,启动httpd服务

shell> docker exec -it slowquery /bin/bash
shell> systemctl start httpd.service 

录入你要监控的MySQL主库配置信息

mysql> INSERT INTO slowquery.dbinfo VALUES (1,'192.168.148.101','test','admin','123456',3306);

客户端部署

进入到slowquery/client_agent_script目录下,把slowquery_analysis.sh脚本拷贝到生产MySQL主库上做慢日志分析推送,并修改里面的配置信息

定时任务(10分钟一次)

*/10 * * * * /bin/bash /usr/local/bin/slowquery_analysis.sh > /dev/null 2>&1

慢查询邮件推送报警配置

进入到slowquery/alarm_mail/目录里,修改sendmail.php配置信息

定时任务(每隔3小时慢查询报警推送一次)

0 */3 * * * cd /var/www/html/slowquery/alarm_mail;/usr/bin/php /var/www/html/slowquery/alarm_mail/sendmail.php

slowquery's People

Contributors

hcymysql avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

slowquery's Issues

页面显示不了

部署后,页面也能打开 选择数据库后 页面一直是空白的。是哪个环节出问题了吗? 我看数据库里面的室友数据的

slow log 抽象

很多慢查询可能只是where id=xxx不一样 怎么把这类的sql抽象成一类呢?谢谢

部署起来太多坑了

难以部署起来。太多坑了。
以下error如何解决?
2019/05/05 15:47:03 [error] 6539#0: *1509 FastCGI sent in stderr: "PHP message: PHP Notice: Undefined index: action in /usr/share/slowquery/slowquery.php on line 4
PHP message: PHP Notice: A session had already been started - ignoring session_start() in /usr/share/slowquery/slowquery.php on line 73" while reading response header from upstream, client: 192.168.20.118, server: localhost, request: "POST /slowquery.php?dbname=zabbix&page=1 HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-fpm5.6.sock:", host: "192.168.20.119", referrer: "http://192.168.20.119/slowquery.php?dbname=zabbix&page=1"
2019/05/05 15:47:03 [error] 6539#0: *1509 FastCGI sent in stderr: "PHP message: PHP Notice: A session had already been started - ignoring session_start() in /usr/share/slowquery/slowquery.php on line 103
PHP message: PHP Warning: mysqli_fetch_array() expects parameter 1 to be mysqli_result, boolean given in /usr/share/slowquery/slowquery.php on line 146" while reading upstream, client: 192.168.20.118, server: localhost, request: "POST /slowquery.php?dbname=zabbix&page=1 HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-fpm5.6.sock:", host: "192.168.20.119", referrer: "http://192.168.20.119/slowquery.php?dbname=zabbix&page=1"
2019/05/05 15:47:03 [error] 6539#0: *1509 FastCGI sent in stderr: "PHP message: PHP Warning: mysqli_fetch_array() expects parameter 1 to be mysqli_result, boolean given in /usr/share/slowquery/get_graph_data.php on line 24" while reading response header from upstream, client: 192.168.20.118, server: localhost, request: "POST /get_graph_data.php HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-fpm5.6.sock:", host: "192.168.20.119", referrer: "http://192.168.20.119/slowquery.php?dbname=zabbix&page=1"

问题回报

小弟测试版本如下,遇到些问题
MySQL Version: 5.7.25
pt-query-digest Version: 3.0.13

pt-query-digest 分析 mysql5.7 生成 checksum 包含数字和字母
故 mysql_slow_query_review 和 mysql_slow_query_review_history 需修改字段 checksum 为 varchar(32) 或 char(32)

另外以下字段 default 不应该为 '0000-00-00 00:00:00',建议直接 not null

  1. ts_min
  2. ts_max

感谢大佬维护,给个赞先 !

老板,我打开slowquery.php页面报了这几个错,您知道咋回事儿不

页面中可以选择db,但是在页面中的不同位置,可以看到这几个报错:
Notice: Undefined index: action in /usr/local/apache2/htdocs/slowquery.php on line 4
Notice: session_start(): A session had already been started - ignoring in /usr/local/apache2/htdocs/slowquery.php on line 79
Notice: session_start(): A session had already been started - ignoring in /usr/local/apache2/htdocs/slowquery.php on line 109
Notice: Undefined index: page in /usr/local/apache2/htdocs/slowquery.php on line 113
Warning: mysqli_fetch_array() expects parameter 1 to be mysqli_result, bool given in /usr/local/apache2/htdocs/slowquery.php on line 152

页面无法显示查询数据

大佬,按文档部署遇到以下问题。
运维监控主机环境:centos7.8 mysql5.7.40 php5.4.16
被监控mysql主机环境:centos7.8 mysql5.7.25 percona-toolkit-3.0.12 pt-query-digest 3.0.12

目前,运维监控主机的Mysql sql_db库中mysql_slow_query_review_history,mysql_slow_query_review 表,显示已采集到被监控主机mysql-slow相关记录,但在直接 select sample,db_max from mysql_slow_query_review_history where checksum=${checksum} limit 1 查询报错 ,参考以前issues,检查对于表字段checksum已是varchar类型。 最头疼是,前端页面也查询,展示不到记录。
看http 日志报以下错误。
[Tue Feb 21 09:49:48.523692 2023] [:error] [pid 10716] [client X.X.X.X:52279] PHP Notice: Undefined index: action in /var/www/html/slowquery/slowquery.php on line 4, referer: http://X.X.X.X:/slowquery/slowquery.php
[Tue Feb 21 09:49:48.525082 2023] [:error] [pid 10716] [client X.X.X.X:52279] PHP Notice: Undefined index: dbname in /var/www/html/slowquery/slowquery.php on line 61, referer: http://X.X.X.X:/slowquery/slowquery.php
[Tue Feb 21 09:49:48.525151 2023] [:error] [pid 10716] [client X.X.X.X:52279] PHP Notice: A session had already been started - ignoring session_start() in /var/www/html/slowquery/slowquery.php on line 79, referer: http://10.38.1.200/slowquery/slowquery.php
[Tue Feb 21 09:49:48.525344 2023] [:error] [pid 10716] [client X.X.X.X::52279] PHP Notice: A session had already been started - ignoring session_start() in /var/www/html/slowquery/slowquery.php on line 109, referer: http://10.38.1.200/slowquery/slowquery.php
[Tue Feb 21 09:49:48.526400 2023] [:error] [pid 10716] [client X.X.X.X::52279] PHP Notice: Undefined index: page in /var/www/html/slowquery/slowquery.php on line 113, referer: http://X.X.X.X:/slowquery/slowquery.php
[Tue Feb 21 09:49:48.530640 2023] [:error] [pid 10716] [client X.X.X.X::52279] PHP Warning: mysqli_fetch_array() expects parameter 1 to be mysqli_result, boolean given in /var/www/html/slowquery/slowquery.php on line 152, referer: http://10.38.1.200/slowquery/slowquery.php

explain数据不展示

您好,大佬explain过程中出现一下问题,麻烦帮指导下,谢谢

image
通过慢查询explain过程中出现调用dbinfo数据链接报错。
尝试手动把$con_explain = mysqli_connect(修改成数据库连接信息,不在报数据库链接失败)
但是,explain没有任何显示
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.