influxdata / terraform-aws-influx Goto Github PK

View Code? Open in Web Editor NEW

50.0 10.0 43.0 973 KB

Reusable infrastructure modules for running TICK stack on AWS

Home Page: https://gruntwork.io/

License: Apache License 2.0

Shell 22.15% HCL 63.68% Go 14.16%

terraform-aws-influx's Introduction

TICK Stack AWS Module

This repo contains the official module for deploying the TICK stack on AWS using Terraform and Packer.

Quick start

If you want to quickly spin up an InfluxDB cluster, you can run the simple example that is in the root of this repo. Check out influxdb-cluster-simple example documentation for instructions.

What's in this repo

This repo has the following folder structure:

root: The root folder contains an example of how to deploy InfluxDB as a single-cluster. See influxdb-cluster-simple for the documentation.
modules: This folder contains the main implementation code for this Module, broken down into multiple standalone submodules.
examples: This folder contains examples of how to use the submodules.
test: Automated tests for the submodules and examples.

How to use this repo

The general idea is to:

Telegraf
1. Use the scripts in the install-telegraf modules to create an AMI with Telegraf installed, this AMI will generally be for the Application server.
2. Configure each application server to execute the run-telegraf script during boot.
InfluxDB
1. Use the scripts in the install-influxdb modules to create an AMI with InfluxDB Enterprise installed.
2. Deploy the AMI across one or more Auto Scaling Groups (ASG) using the influxdb-cluster module.
3. Configure each server in the ASGs to execute the run-influxdb script during boot.
4. Deploy a load balancer in front of the data node ASG.
Chronograf
1. Use the scripts in the install-chronograf modules to create an AMI with Chronograf installed.
2. Deploy the AMI in a single Auto Scaling Group (ASG) using the chronograf-server module.
3. Configure the server to execute the run-chronograf script during boot.
4. Deploy a load balancer in front of the ASG.
Kapacitor
1. Use the scripts in the install-kapacitor modules to create an AMI with Kapacitor installed.
2. Deploy the AMI in a single Auto Scaling Group (ASG) using the kapacitor-server module.
3. Configure the server to execute the run-kapacitor script during boot.
4. Deploy a load balancer in front of the ASG.

See the examples folder for working sample code.

What's a Module?

A Module is a canonical, reusable, best-practices definition for how to run a single piece of infrastructure, such as a database or server cluster. Each Module is written using a combination of Terraform and scripts (mostly bash) and include automated tests, documentation, and examples. It is maintained both by the open source community and companies that provide commercial support.

Instead of figuring out the details of how to run a piece of infrastructure from scratch, you can reuse existing code that has been proven in production. And instead of maintaining all that infrastructure code yourself, you can leverage the work of the Module community to pick up infrastructure improvements through a version number bump.

Who maintains this Module?

This Module is maintained by Gruntwork. If you're looking for help or commercial support, send an email to [email protected]. Gruntwork can help with:

Setup, customization, and support for this Module.
Modules for other types of infrastructure, such as VPCs, Docker clusters, databases, and continuous integration.
Modules that meet compliance requirements, such as HIPAA.
Consulting & Training on AWS, Terraform, and DevOps.

How do I contribute to this Module?

Contributions are very welcome! Check out the Contribution Guidelines for instructions.

How is this Module versioned?

This Module follows the principles of Semantic Versioning. You can find each new release, along with the changelog, in the Releases Page.

During initial development, the major version will be 0 (e.g., 0.x.y), which indicates the code does not yet have a stable API. Once we hit 1.0.0, we will make every effort to maintain a backwards compatible API and use the MAJOR, MINOR, and PATCH versions on each release to indicate any incompatibilities.

License

This code is released under the Apache 2.0 License. Please see LICENSE and NOTICE for more details.

terraform-aws-influx's People

Contributors

Stargazers

Watchers

terraform-aws-influx's Issues

Investigate test failures

Nightly builds are failing, investigate the root cause.

Chronograf server ASG target groups are getting removed

Problem

Hi, I'm using the terraform modules to run a chronograf and a kapacitor server on AWS. I encountered the problem, that terraform apply would remove the target group from the auto scaling group, even tough there were no real changes to the resources. (This could lead to down time on production systems.) When running terraform apply a second time, terraform would reattach the target group to the ASG.
This problem occurs in the AWS terraform provider and it's rerlated to the lifecycle of the ASG. There is this GitHub issue discussing this: hashicorp/terraform-provider-aws#14540

Possible solution

To solve the issue for my project, I forked the this repo and added a lifecycle block to the ASG resource, as suggested in here: hashicorp/terraform-provider-aws#14540 (comment) After running terraform apply the problem was solved.

resource "aws_autoscaling_group" "autoscaling_group" {
  name = var.cluster_name

...

  lifecycle {
    ignore_changes = [load_balancers, target_group_arns]
  }

...

}

The terraform apply did not cause any down time. I'm not sure about backwards compatibility. I'm still running v0.1.2 of the TICK stack modules and had no problems.

I hope you can use this information to create a fix release and thanks for creating and maintaining these module.

Removal of node from cluster in case ec2 is terminated

If an ec2 node is terminated due to an unhealthy state, ec2 cluster will automatically add a new node to the cluster to balance it. However, the addition of the new node to influxdb cluster fails as the previous node is still not removed from the cluster.

Do you support the removal of a node from influxdb cluster while the machine is getting terminated.

install-telegraf Permission Denied

When following the examples/tick-ami example, the build finishes with:

==> tick-ami-ubuntu: Uploading C:\Users\maxschommer\Documents\Github\terraform-aws-influx\examples\tick-ami/../../modules/ => /tmp/terraform-aws-influx/modules
==> tick-ami-ubuntu: Provisioning with shell script: C:\Users\MAXSCH~1\AppData\Local\Temp\packer-shell705441023
==> tick-ami-ubuntu: /tmp/script_9674.sh: 2: /tmp/script_9674.sh: /tmp/terraform-aws-influx/modules/install-telegraf/install-telegraf: Permission denied
==> tick-ami-ubuntu: Provisioning step had errors: Running the cleanup provisioner, if present...
==> tick-ami-ubuntu: Terminating the source AWS instance...
==> tick-ami-ubuntu: Cleaning up any extra volumes...
==> tick-ami-ubuntu: No volumes to clean up, skipping
==> tick-ami-ubuntu: Deleting temporary security group...
==> tick-ami-ubuntu: Deleting temporary keypair...
Build 'tick-ami-ubuntu' errored after 3 minutes 19 seconds: Script exited with non-zero exit status: 126.Allowed exit codes are: [0]       

==> Wait completed after 3 minutes 19 seconds

==> Some builds didn't complete successfully and had errors:
--> tick-ami-ubuntu: Script exited with non-zero exit status: 126.Allowed exit codes are: [0]

==> Builds finished but no artifacts were created.

Running from Windows 10. I tried changing the inline shell scripts from
"/tmp/terraform-aws-influx/modules/install-telegraf/install-telegraf --version {{user `telegraf_version`}}"
to
"sudo /tmp/terraform-aws-influx/modules/install-telegraf/install-telegraf --version {{user `telegraf_version`}}"
and received a command not found error:

==> tick-ami-ubuntu: Uploading C:\Users\maxschommer\Documents\Github\terraform-aws-influx\examples\tick-ami/../../modules/ => /tmp/terraform-aws-influx/modules
==> tick-ami-ubuntu: Provisioning with shell script: C:\Users\MAXSCH~1\AppData\Local\Temp\packer-shell056613131
==> tick-ami-ubuntu: sudo: /tmp/terraform-aws-influx/modules/install-telegraf/install-telegraf: command not found
==> tick-ami-ubuntu: Provisioning step had errors: Running the cleanup provisioner, if present...
==> tick-ami-ubuntu: Terminating the source AWS instance...
==> tick-ami-ubuntu: Cleaning up any extra volumes...
==> tick-ami-ubuntu: No volumes to clean up, skipping
==> tick-ami-ubuntu: Deleting temporary security group...
==> tick-ami-ubuntu: Deleting temporary keypair...
Build 'tick-ami-ubuntu' errored after 2 minutes 58 seconds: Script exited with non-zero exit status: 1.Allowed exit codes are: [0]

==> Wait completed after 2 minutes 58 seconds

==> Some builds didn't complete successfully and had errors:
--> tick-ami-ubuntu: Script exited with non-zero exit status: 1.Allowed exit codes are: [0]

==> Builds finished but no artifacts were created.

Common TCK scripts

Install and run -scripts for Kapacitor, Chronograf, and Telegraf are not platform dependent, so they should be broken into a separate repository to enable reusing them in different cloud implementations (AWS / GCP / Azure / ...)

influxdata / terraform-aws-influx Goto Github PK

terraform-aws-influx's Introduction

TICK Stack AWS Module

Quick start

What's in this repo

How to use this repo

Telegraf

InfluxDB

Chronograf

Kapacitor