Giter VIP home page Giter VIP logo

metrics-sql's Introduction

Metrics SQL

This a Yammer|Codahale|Dropwizard Metrics extension to instrument JDBC resources and measure SQL execution times.

CircleCI Coverage Status Maven Central

Supported metrics

Description Default metric name Metric type
Connection opening (getConnection()) java.sql.Connection Timer
Connection life (between getConnection() and close()) java.sql.Connection Timer
Statement life (between createStatement() and close()) java.sql.Statement Timer
Statement execution (execute(), executeQuery()...) java.sql.Statement.[select * from my_table].exec Timer
PreparedStatement life (between prepareStatement() and close()) java.sql.PreparedStatement.[select * from my_table] Timer
PreparedStatement execution (execute(), executeQuery()...) java.sql.PreparedStatement.[select * from my_table].exec Timer
CallableStatement life (between prepareCall() and close()) java.sql.CallableStatement.[call_something()] Timer
CallableStatement execution (execute(), executeQuery()...) java.sql.CallableStatement.[call_something()].exec Timer
ResultSet life (between executeQuery.() and close()..) java.sql.ResultSet.[select * from my_table] Timer
ResultSet rows (next()) java.sql.ResultSet.[select * from my_table].rows Meter

Metric naming is tunable, to be more Graphite or InfluxDB compliant, see MetricNamingStrategy. Metering can be disabled per metric, you can select which metrics you (don't) want.

Setup

DataSource level

Wrap your existing DataSource using JdbcProxyFactory or MetricsSql builder class:

    metricRegistry = new MetricRegistry();
    dataSource = MetricsSql.forRegistry(metricRegistry)
                    .wrap(mysqlDataSource);

The String mysql is a datasource Id used in metric names.

Connection level

Same as DataSource

    metricRegistry = new MetricRegistry();
    connection = MetricsSql.forRegistry(metricRegistry)
                    .wrap(mysqlConnection);

Driver level

  1. Register Metrics SQL JDBC Driver: replace the original JDBC driver by com.github.gquintana.metrics.sql.Driver
  2. Change JDBC URL prefix: jdbc:xxx becomes jdbc:metrics:xxx

Examples:

jdbc:metrics:mysql://localhost:3306/sakila?profileSQL=true
jdbc:metrics:postgresql://localhost/demo?metrics_driver=org.postgresql.Driver&ssl=true
jdbc:metrics:h2:~/test;AUTO_SERVER=TRUE;;AUTO_RECONNECT=TRUE;metrics_driver=org.h2.Driver;metrics_proxy_factory=caching

The driver supports several options:

  • metrics_driver: the real driver class to wrap
  • metrics_registry: the name of the shared metric registry to use (see SharedMetricRegistries)
  • metrics_naming_strategy: the strategy used to generate what should be metered and the timer names: class name implementing MetricNamingStrategy
  • metrics_proxy_factory: the strategy used to create proxies: either reflect (the default), cglib or caching,

Configuration

Naming strategy

The Naming strategy implements MetricNamingStrategy and can configure: * Which operation should be timed (return null means not timed) * How the metric is named

The DefaultMetricNamingStrategy generate metric names like:

java.sql.Statement.[select * from my_table].exec

When the database is set,

    dataSource = MetricsSql.forRegistry(metricRegistry)
                    .withDefaultNamingStrategy("my_database")
                    .wrap(mysqlDataSource);

It will produce:

java.sql.Statement.my_database.[select * from my_table].exec

This is useful when there are multiple datasources.

There is also the StrictMetricNamingStrategy which removes also special chars from the SQL query:

java.sql.Statement.select_from_my_table.exec
java.sql.Statement.my_database.select_from_my_table.exec

These settings are also available as URL properties:

jdbc:metrics:h2;metrics_naming_strategy=default;metrics_database=my_database

Proxy factory

The Proxy factory implements ProxyFactory, can configure how JDBC elements are wrapped

  • ReflectProxyFactory uses reflection and simple java.lang.reflect.Proxy
  • CGLibProxyFactory, requires the CGLib library on the classpath and uses CGLib based proxies.

SharedMetricRegistries

The Driver uses the SharedMetricRegistries singleton to lookup (and register) the MetricRegistry:

connection = DriverManager.getConnection("jdbc:metrics:h2;metrics_registry=my_registry", "sa", "");
metryRegistry = SharedMetricRegistries.getOrCreate("my_registry");

Integration

Unprepared statement with unbound parameters

Beware of using unprepared statements and unbound parameters, it will generate a lot of metrics. For instance ...:

Statement statement = connection.createStatement();
statement.execute("insert into METRICS(ID, NAME) values(1, 'One')");
statement.execute("insert into METRICS(ID, NAME) values(2, 'Two')");

... will generate 2 metrics!

There are several options:

  • Use prepared statements and bound parameters
  • Tune the naming strategy to filter unprepared statements
  • Tune the naming strategy to make both SQL statements generate the same metric name

JMX

The JmxReporter doesn't play well with DefaultMetricNamingStrategy, you'll have to change either the naming strategy or the object name factory. A SqlObjectNameFactory is provided:

JmxReporter.forRegistry(metricRegistry)
    .registerWith(mBeanServer)
    .createsObjectNamesWith(new SqlObjectNameFactory())
    .build();

metrics-sql's People

Contributors

gquintana avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

metrics-sql's Issues

use micrometer metrics

I have worked on a fork of metrics-sql as a proof-of-concept that uses micrometer instead of dropwizard metrics. The advantage for me is that micrometer can be used as an abstraction for any underlying metrics framework.
https://github.com/pjfanning/metrics-sql
I can continue with my fork as a standalone project but if you think it's a good idea, maybe it can be merged into your project instead (after my tests are beefed up, etc).

StatsD reporter

Is there any statsDReporter for the metrics collected by the framework?

sqls shoud have sanatized names

sql contains . (tablename.colname) - this does not play well with metrics/graphite where . is used to separate/name metrics - hence the namingstrategy should replace dots with something else like dot

See dropwizard/metrics#637 for related issue.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.