Giter VIP home page Giter VIP logo

terraform-aws-eks-grafana-prometheus's Introduction

terraform-aws-eks-grafana-prometheus

Lint Status LICENSE

Terraform module for deploying Kubernetes Prometheus and Grafana inside a pre-existing EKS cluster.

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open source project and maintained independently of any company. Prometheus joined the Cloud Native Computing Foundation in 2016 as the second hosted project, after Kubernetes.

Grafana is open source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations.

Usage

module "grafana_prometheus_monitoring" {
  source = "git::https://github.com/DNXLabs/terraform-aws-eks-grafana-prometheus.git"

  enabled = true
}

Check if Prometheus and Grafana components deployed as expected:

kubectl get all -n prometheus
kubectl get all -n grafana

In order to access the Prometheus and Grafana server URL, we are going to use the kubectl port-forward command to access the application.

# Prometheus
kubectl port-forward --address 0.0.0.0 -n prometheus deploy/prometheus-server 8001:9090
# Grafana
kubectl port-forward --address 0.0.0.0 -n grafana deploy/grafana 8001:3000

When logging in, use the username and password admin.

Dashboards

Log in to Grafana dashboard using credentials supplied during configuration.

Cluster Monitoring Dashboard

For creating a dashboard to monitor the cluster:

  • Click "+" button on left panel and select "Import".
  • Enter 3119 dashboard id under Grafana.com Dashboard.
  • Click "Load".
  • Enter Kubernetes Cluster Monitoring as the Dashboard name.
  • Select "Prometheus" as the endpoint under prometheus data sources drop down.
  • Click "Import".

This will show monitoring dashboard for all cluster nodes

grafana-all-nodes

Pods Monitoring Dashboard

For creating a dashboard to monitor all the pods:

  • Click "+" button on left panel and select "Import".
  • Enter 6417 dashboard id under Grafana.com Dashboard.
  • Click "Load".
  • Enter Kubernetes Pods Monitoring as the Dashboard name.
  • Click change to set the Unique identifier (uid).
  • Select "Prometheus" as the endpoint under prometheus data sources drop down.s
  • Click "Import".

grafana-all-pods

Requirements

Name Version
terraform >= 0.13
aws >= 3.13, < 4.0
helm >= 1.0, < 3.0
kubernetes >= 1.10.0, < 3.0.0

Providers

Name Version
helm >= 1.0, < 3.0
kubernetes >= 1.10.0, < 3.0.0

Inputs

Name Description Type Default Required
create_namespace_grafana Whether to create Grafana Kubernetes namespace with name defined by namespace. bool true no
create_namespace_prometheus Whether to create Prometheus Kubernetes namespace with name defined by namespace. bool true no
enabled Variable indicating whether deployment is enabled. bool true no
helm_chart_grafana_name Grafana Helm chart name to be installed string "grafana" no
helm_chart_grafana_release_name Grafana Helm release name string "grafana" no
helm_chart_grafana_repo Grafana repository name. string "https://grafana.github.io/helm-charts" no
helm_chart_grafana_version Grafana Helm chart version. string "6.15.0" no
helm_chart_prometheus_name Prometheus Helm chart name to be installed string "prometheus" no
helm_chart_prometheus_release_name Prometheus Helm release name string "prometheus" no
helm_chart_prometheus_repo Prometheus repository name. string "https://prometheus-community.github.io/helm-charts" no
helm_chart_prometheus_version Prometheus Helm chart version. string "14.5.0" no
mod_dependency Dependence variable binds all AWS resources allocated by this module, dependent modules reference this variable. any null no
namespace_grafana Kubernetes namespace to deploy Grafana stack Helm charts. string "grafana" no
namespace_prometheus Kubernetes namespace to deploy Prometheus stack Helm charts. string "prometheus" no
settings_grafana Additional settings which will be passed to Grafana Helm chart values. map 
{
  "adminPassword": "admin",
  "persistence": {
    "enabled": true,
    "storageClassName": "gp2"
  }
}
 no
settings_prometheus Additional settings which will be passed to Prometheus Helm chart values. map 
{
  "alertmanager": {
    "persistentVolume": {
      "storageClass": "gp2"
    }
  },
  "server": {
    "persistentVolume": {
      "storageClass": "gp2"
    }
  }
}
 no

Outputs

No output.

Authors

Module managed by DNX Solutions.

License

Apache 2 Licensed. See LICENSE for full details.

terraform-aws-eks-grafana-prometheus's People

Contributors

arthurbdiniz avatar caiovfernandes avatar mosampaio avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

alexandre1984rj arun9theja ahikars syllogy apostatedevops byunjuneseok 12abneesh ryanoatz99 joaopdelboni chuongdlb inanibharat bridgecrew-perf4 danieloliveira97 dhmittal22 dev-sptg kyungwon3 dilfuza97 sobhareddy philipnwachukwu jwcastillo oulai87 mojojo4ever solankipintu0099 lingeshkm lemin-ou caetanoandre vincent-tang-123

terraform-aws-eks-grafana-prometheus's Issues

use variable instead of hardcoded namespace

Contact Details (optional)

[email protected]

Description. What happened?

It seems that I can't use a different namespace in the grafana.yaml file.
I've fixed it by changing the file to a template and using an existing variable.

from:
url: http://prometheus-server.prometheus.svc.cluster.local
to:
url: http://prometheus-server.${namespace_prometheus}.svc.cluster.local

in helm-grafana.tf I've changed to the following:
from:
file("${path.module}/grafana.yaml"),
to:
templatefile("${path.module}/grafana.yaml.tpl", {namespace_prometheus = var.namespace_prometheus}),

Basically, I can modify the URL in the grafana dashboard but there is a bug that no matter what value I enter it keeps trying with the hardcoded value.

Steps to reproduce

run with a different namespace for prometheus.

Expected behavior

to accept the correct namespace

Actual behavior

hardcoded namespace

Occurrence

Frequently

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct

Getting error with namespace resource block

Contact Details (optional)

[email protected]

Description. What happened?

Hi,
I have connected to the ec2 linux machine and set kube config to my cluster and clone this repo code to ec2 and ran apply command.

I am facing with below error. Could you please help me what is missing.

kubernetes_namespace.prometheus[0]: Creating...
kubernetes_namespace.grafana[0]: Creating...
╷
│ Error: Post "http://localhost/api/v1/namespaces": dial tcp 127.0.0.1:80: connect: connection refused
│
│ with kubernetes_namespace.grafana[0],
│ on namespace.tf line 1, in resource "kubernetes_namespace" "grafana":
│ 1: resource "kubernetes_namespace" "grafana" {
│
╵
╷
│ Error: Post "http://localhost/api/v1/namespaces": dial tcp 127.0.0.1:80: connect: connection refused
│
│ with kubernetes_namespace.prometheus[0],
│ on namespace.tf line 10, in resource "kubernetes_namespace" "prometheus":
│ 10: resource "kubernetes_namespace" "prometheus" {

Steps to reproduce

  1. connect to bastion box

  2. update-kubeconfig to the cluster

  3. create folder and install terraform

  4. copy all below tf and yaml files to the newly created folder
    [root@ip-10-85-0-145 grafana]# pwd
    /root/grafana
    [root@ip-10-85-0-145 grafana]# ls
    grafana.yaml helm-grafana.tf helm-prometheus.tf namespace.tf variables.tf
    [root@ip-10-85-0-145 grafana]#

  5. Terraform init

  6. Terraform apply

Output : Error as shown below:

kubernetes_namespace.grafana[0]: Creating...
╷
│ Error: Post "http://localhost/api/v1/namespaces": dial tcp 127.0.0.1:80: connect: connection refused
│
│ with kubernetes_namespace.grafana[0],
│ on namespace.tf line 1, in resource "kubernetes_namespace" "grafana":
│ 1: resource "kubernetes_namespace" "grafana" {
│
╵
╷
│ Error: Post "http://localhost/api/v1/namespaces": dial tcp 127.0.0.1:80: connect: connection refused
│
│ with kubernetes_namespace.prometheus[0],
│ on namespace.tf line 10, in resource "kubernetes_namespace" "prometheus":
│ 10: resource "kubernetes_namespace" "prometheus" {
│
╵

Expected behavior

No response

Actual behavior

it should be able to create both grafana and prometheus namespaces and pods with the other required resources .

Occurrence

Frequently

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.