The monitoringtoolkit from manojkumar542

Monitoringtoolkit using jmeter,influxDB & Grafana

It automates jmeter executions and shows live results in grafana dashboard using docker compose easily
Create a folder under which docker-compose file resides and add grafana,influxdb and jmeter folder to store the dockerfile and concerned config and data files
Tweak the jmeter script and data files according to your need and i'm using sample jpetstore demo jmeter script file for execution.
Update the jmeter dockerfile accordingly based on your requirement
Update the docker compose file according to your need.
Use docker container name to establish connection between containers.

Setup as below

run docker-compose build
docker-compose up -d
docker-compose down

With a single docker-compose up -d command we can setup our jmeter execution and monitor the live results in grafana and saves the raw and html report of the jmeter execution in your host machine

Create a folder under which docker-compose file resides and add grafana,influxdb and jmeter folder to store the dockerfile and concerned config and data files accordingly on the github repo

For setting up only jmeter docker image please refer the below links https://performanceengineeringsite.wordpress.com/2020/02/01/setup-jmeter-in-linux-docker-container-with-all-plugins/

https://github.com/manojkumar542/Jmeter-docker

Basic of docker-compose file

<strong>Docker compose file</strong>
version: '3.8'
services:
  influxdb:
    image: 'influxdb:1.8.9-alpine'
    container_name: influxdb
    restart: always
    ports: 
      - '8086:8086'
    networks:
      - monitoring
    environment:
      - INFLUXDB_DB=jmeter
    volumes:
      - influxdb-storage:/var/lib/influxdb
    
  grafana:
    build:
      context: ./grafana
    image: 'grafana:8.1.0'
    container_name: grafana
    ports:
      - '3000:3000'
    restart: always
    networks: 
      - monitoring
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=grafana
      - GF_INSTALL_PLUGINS=grafana-clock-panel,briangann-gauge-panel,natel-plotly-panel,grafana-simple-json-datasource
    volumes:
      - grafana-volume:/etc/grafana/provisioning
      - grafana-data:/var/lib/grafana
    depends_on: 
      - influxdb

  jmeter:
    build:
      context: ./jmeter
    image: 'jmeter-docker:latest'
    container_name: jmeter
    restart: always
    networks:
      - monitoring
    volumes:
      - ./jmeter:/results
    depends_on: 
      - influxdb
      - grafana

networks:
  monitoring:
volumes:
  influxdb-storage:
  grafana-data:
  grafana-volume:

Steps to be followed to create Docker Compose Yaml file

1. Create customized docker files for jmeter & grafana and place in each respective folders and for influxdb can be directly pull it from dockerhub.

2. Create a Network to connect multiple containers to establish connection between them and default network is bridge and just define it in the yaml no need to explicitly create it.

3. Create volumes such as named volumes to persist the data whenever there are any unseen container exists and we can use volumes to get the data back just define it in Yaml docker will create for you

4. Create environment variables to pass it to the runnning containers such as ex: creating TSDB with name Jmeter in influxdb before running jmeter executions. INFLUXDB_DB:jmeter

5. Use docker-build to build the docker image from the specific build context folder where dockerfile resides to get the latest changes into docker image.

6. Finally just run docker-compose up -d to run all the services defined in docker-compose.yaml file in detached mode.

Yaml files grafana and automatic creation of Datasource and Dashboard

FROM grafana/grafana:8.1.0

ENV GF_SECURITY_ADMIN_USER=admin
ENV GF_SECURITY_ADMIN_PASSWORD=grafana
ENV GF_INSTALL_PLUGINS=grafana-clock-panel,briangann-gauge-panel,natel-plotly-panel,grafana-simple-json-datasource

# automatically provisions datasources and dashboards in grafana
ADD ./provisioning /etc/grafana/provisioning

# Add configuration file
#ADD ./grafana.ini /etc/grafana/grafana.ini

Grafana Datasource Yaml format
apiVersion: 1
datasources:
  - name: 'Influx-Jmeter'
    type: influxdb
    access: proxy
    database: jmeter
    url: http://influxdb:8086
    isDefault: true
    editable: true

Grafana Dashboard format download here

Now go to docker-compose file folder and run docker-compose up -d

Use docker exec -it containerid /bin/sh to connect to containers and check the influxdb to check the database with name jmeter is created or not same way you can do it for grafana to check the dashboards and datasources are referring correctly from the copied files.

Since i have enabled port mapping to expose container port to localhost port i can able to access both grafana and influxdb using httP://localhost:portnumber

But in containers they use internal ips to communicate and internally container name DNS is mapped to ip that's the reason i have used container names to configure datasource connection in influxdb url in grafana

Using docker-compose build  by providing build context we can build the image from the specific docker file folder using docker-compose yaml itself as below
  jmeter:
    build:
      context: ./jmeter
    image: 'jmeter-docker:latest'
    container_name: jmeter
    restart: always
    networks:
      - monitoring
    volumes:
      - ./jmeter:/results
    depends_on: 
      - influxdb
      - grafana

Context - where exactly the dockerfile resides and build and tag with the image name specified in our case it's <strong>'jmeter-docker:latest'</strong>

depends_on - make sure services in the section specified will be starting and running which are dependent on the current service.