Giter VIP home page Giter VIP logo

noderig's Introduction

Noderig - Export OS stats as Sensision or Prometheus Metrics

Build Status

Noderig collect OS metrics and expose them through a Sensision HTTP endpoint. Each collector is easily configurable, thanks to a simple level cursor

Noderig metrics:

  • CPU
  • Memory
  • Load
  • Disk
  • Net
  • External collectors

Status

Noderig is currently under development. Feel free to comment or contribute!

Building

Noderig is pretty easy to build.

  • Clone the repository
  • Install glide, follow instructions here https://glide.sh/
  • Download dependencies glide install
  • Build and run go run noderig.go

Usage

noderig [flags]

Flags:
      --config string     config file to use
  -l  --listen string     listen address (default "127.0.0.1:9100")
  -v  --verbose           verbose output
      --period uint       default collection period (default 1000)
      --cpu uint8         cpu metrics level (default 1)
      --disk uint8        disk metrics level (default 1)
      --mem uint8         memory metrics level (default 1)
      --net uint8         network metrics level (default 1)
      --load uint8        load metrics level (default 1)
  -c  --collectors string external collectors directory (default "./collectors")
  -k  --keep-for uint     keep collectors data for the given number of fetch (default 3)

Collectors

Noderig have some built-in collectors.

CPU

LevelMetricDescriptionModule
0disabled metrics
1os.cpu{}combined percentage of cpu usage
2os.cpu.iowait{}combined percentage of cpu iowait
os.cpu.user{}combined percentage of cpu user
os.cpu.systems{}combined percentage of cpu systems
os.cpu.nice{}combined percentage of cpu nice
os.cpu.irq{}combined percentage of cpu irq
os.cpu.temperature{id=n}temperature of cpu ntemperature
3os.cpu.iowait{chore=n}chore percentage of cpu iowait
os.cpu.user{chore=n}chore percentage of cpu user
os.cpu.systems{chore=n}chore percentage of cpu systems
os.cpu.nice{chore=n}chore percentage of cpu nice
os.cpu.irq{chore=n}chore percentage of cpu irq
os.cpu.temperature{core=n}temperature of cpu core ntemperature

Memory

0disabled metrics
1os.mem{}percentage of memory used
os.swap{}percentage of swap used
2os.mem.used{}used memory (bytes)
os.mem.total{}total memory (bytes)
os.swap.used{}used swap (bytes)
os.swap.total{}total swap (bytes)

Load

0disabled metrics
1os.load1{}load 1
2os.load5{}load 5
os.load15{}load 15

Disk

0disabled metrics
1os.disk.fs{disk=/dev/sda1}disk used percent
2os.disk.fs.used{disk=/dev/sda1, mount=/}disk used capacity (bytes)
os.disk.fs.total{disk=/dev/sda1,mount=/}disk total capacity (bytes)
os.disk.fs.inodes.used{disk=/dev/sda1,mount=/}disk used inodes
os.disk.fs.inodes.total{disk=/dev/sda1,mount=/}disk total inodes
3os.disk.fs.bytes.read{name=sda1}disk read count (bytes)
os.disk.fs.bytes.write{name=sda1}disk write count (bytes)
4os.disk.fs.io.read{name=sda1}disk io read count (bytes)
os.disk.fs.io.write{disk=/sda1}disk io write count (bytes)
5os.disk.fs.io.read.ms{name=sda1}disk io read time (ms)
os.disk.fs.io.write.ms{name=sda1}disk io write time (ms)
os.disk.fs.io{name=sda1}disk io in progress (count)
os.disk.fs.io.ms{name=sda1}disk io time (ms)
os.disk.fs.io.weighted.ms{name=sda1}disk io weighted time (ms)

Net

0disabled metrics
1os.net.bytes{direction=in}in bytes count (bytes)
os.net.bytes{direction=out}out bytes count (bytes)
2os.net.bytes{direction=in,iface=eth0}iface in bytes count (bytes)
os.net.bytes{direction=out,iface=eth0}iface out bytes count (bytes)
3os.net.packets{direction=in,iface=eth0}iface in packet count (packets)
os.net.packets{direction=out,iface=eth0}iface out packet count (packets)
os.net.errs{direction=in,iface=eth0}iface in error count (errors)
os.net.errs{direction=out,iface=eth0}iface out error count (errors)
os.net.dropped{direction=in,iface=eth0}iface in drop count (drops)
os.net.dropped{direction=out,iface=eth0}iface out drop count (drops)

Custom

With Noderig you can define set-up custom collectors as defined in http://bosun.org/scollector/external-collectors. To be enable you need to define a collectors folder using the noderig parameter "collectors". This fold need to have a strict arborescence: a number folder and then the exectutable collectors.

For example to define a script shell collectors reach the noderig collectors file:

cd ~/collectors
mkdir 10
``

Then inside the 10 folder write the following executable `test.sh` shell script.

```sh
#!/bin/sh

now=$(date +%s)

echo my.metric $now 42

And execute noderig:

./build/noderig --collectors ~/collectors

To conclude you can tun noderig custom collectors with the following configuration parameters:

keep-metrics: true # To always keep in Noderig the last metrics values
keep-for: 3 # Keep-for returned the number values to keep

The keep-for parameter with keep-metrics at true keep the last N values otherwise it keep each values for n calls to the noderig metrics endpoint.

Configuration

Noderig can read a simple default config file.

Configuration is load and override in the following order:

  • /etc/noderig/config.yaml
  • ~/noderig/config.yaml
  • ./config.yaml
  • config filepath from command line

Definitions

Config is composed of three main parts and some config fields:

Collectors

Noderig have some built-in collectors. They could be configured by a log level. You can also defined custom collectors, in an scollector way. (see: http://bosun.org/scollector/external-collectors) To configure a custom collectors in noderig reach custom collectors.

cpu: 1  # CPU collector level     (Optional, default: 1)
mem: 1  # Memory collector level  (Optional, default: 1)
load: 1 # Load collector level    (Optional, default: 1)
disk: 1 # Disk collector level    (Optional, default: 1)
net: 1  # Network collector level (Optional, default: 1)

Collectors Modules

Some collectors have additionals modules. Add module to <collector>-mods list to enable them.

cpu-mods:
  - temperature

Collectors Options

Some collectors can accept optional parameters.

net-opts:
  interfaces:            # Give a filtering list of interfaces for which you want metrics
    - eth0
    - eth1

Net-opts, interfaces field support now regular expression to white-list interface based on golang MatchString implementation. However to use a regular expression you need to prefix the string value by a ~. To whitelist all eth interfaces, you can set:

net-opts:
  interfaces:            # Give a filtering list of interfaces for which you want metrics
    - ~eth*
disk-opts:
  names:            # Give a filtering list of disks for which you want metrics
    - sda1
    - sda3

Parameters

Noderig can be customized through some parameters.

period: 1000             # Duration within all the sources should be scraped in ms (Optional, default: 1000)
listen: none             # Listen address, set to none to disable http endpoint    (Optional, default: 127.0.0.1:9100)
collectors: /opt/noderig # Custom collectors directory                             (Optional, default: none)

To force default labels to each metrics in Noderig, you can set up a configuration key called labels. It expects a label string map as defined below:

labels: { 
  host: "srv001", 
  dc: "uk1", 
  type: "web_server", 
}

Sample metrics

1484828198557102// os.cpu{} 2.5202020226869237
1484828198560976// os.mem{} 24.328345730457112
1484828198560976// os.swap{} 0
1484828198557435// os.load1{} 0.63
1484828198561366// os.net.bytes{direction=in} 858
1484828198561366// os.net.bytes{direction=out} 778
1484828197570759// os.disk.fs{disk=/dev/sda1} 4.967614357908193

Prometheus output format

To use Noderig and expose a Prometheus native format, just set the following two configuration lines in the config file:

format: "prometheus"  # Expose a Prometheus format in Noderig as: https://prometheus.io/docs/instrumenting/exposition_formats/
separator: "_"        # Metrics classnames separator, '_' is the default one for Prom, but you can use any other supported by your storage backend

Contributing

Instructions on how to contribute to Noderig are available on the Contributing page.

Get in touch

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.