Terragrunt Reference Architecture (upd: May 2020)

License: MIT License

HCL 97.85% Go 0.92% Python 1.22%

terragrunt terragrunt-managed terraform reference-architecture

terragrunt-reference-architecture's Introduction

Hi there, I'm Anton 👋

I am AWS Community Hero and a founder of Betajob - a company that helps build solutions using AWS and specializes in infrastructure as code, DevOps, and reusable infrastructure components.

I like

Travel to share knowledge on conferences, meetups, podcasts, workshops, mentorship, etc.
Figure out best&worst practices (by asking "Why?"), and publishing them as Terraform best practices and Terragrunt Reference Architecture
Open-source, Terraform and AWS
Making Terraform live streams and sending weekly newsletters is fun, too.

Some of my open-source projects

serverless.tf - Doing serverless on AWS with Terraform.
Terraform cost estimation - anonymized, secure, free.
Terraform AWS modules which were provisioned more than 200 million times.
pre-commit-terraform to keep Terraform projects clean and nice.
Hide posts about certifications in your LinkedIn newsfeed - Chrome extension.
Remember to give a star to terraform-aws-devops repository for a complete list of my projects and links.

Connect with me

terragrunt-reference-architecture's People

Contributors

Stargazers

Watchers

Forkers

tkh iphpdonthitme andrewlubenets satyaelipe khairajpamnani weitzj pramine dimagv vaibhav-walke sadok-f versus vitalidn codeshane fuez dev-resources jimsmith khushil daemondreams mrwlad piksel-oliverslater james-gonzalez gaboesquivel safarcik hblabvn full360 sungchun12 savadev umairedu mkostrikin emctl joshrwolf fhchina valery-barysok ssk77 itcloud-services yahav876 ae-exact adriell abhinaba-chakraborty-by edtepes-sof waldt asterix201 teknofilenet lacrazyboy justin-watkinson-sp louiekwan richardsonlima vguerrapdel ravi2rao charanitha raaji1981 huamuling kingslincharles22 boost-entropy-aws lightyear0 eddeev prathimatech wxzjq naradinc seprtk latifshaikh7 crazytechie1990 dcode0 granite-cloud standardgalactic onlykumarabhishek mcrivar vitorsavickirentscars andreymatrosov dferns11 davidpesticcio davesys911 svalevka chethansiddagiri terraformlearning2021 trushitpandya subhransu-1 gitniran ramchandercp rodvaldes rajucp ssprasad100 tobernguyen victornico15 m-tok dannykruitbosch lostar01 houssoli edgarsandi sathishphcl princebirring assassin010 asasar tymik yuta0805 shubham-traqmatix tidakebapu96 ankitcharles lapnd niranx

terragrunt-reference-architecture's Issues

Reference Workflow

Do you have any suggestions for what a work flow would look like for the developers interacting with a repository following the same structure as this repository?

Things that would be interesting to understand your opinion on:

What would an automated pipeline look like for this repository?
Would a PR be able to sufficiently serve as the 'manual approval's step for each PR made to this infrastructure?
What steps would be necessary to deploy a change to one of the base modules to all of the environments (how many PRs, how would you validate the change as it is deployed to each environment)?

Clarify the VPC portion in acme-prod

In acme-prod/eu-central-1 is a folder to create the VPC. If I have two regions and in this example there's a us-east-1 region, would I need to create a vpc folder w/ the same code in the us-east-1 folder? if so, I thought terragrunt is supposed to allow you to have DRY code?

Strange output with the Terragrunt alias

I was using this

terragrunt () {
	local action=$1
	shift 1
	command terragrunt $action "$@" 2>&1 | sed -E "s|$(dirname $(pwd))/||g;s|^\[terragrunt\]( [0-9]{4}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2})* ||g;s|(\[.*\]) [0-9]{4}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}|\1|g"
}

to get nice output, but what I have noticed you can see below

terragrunt apply

... 

Plan: 1 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

yes
  Enter a value:
google_project_iam_member.cluster_service_account-resourceMetadata-writer[0]: Creating...
google_container_cluster.primary: Modifying... [id=gke-cluster]

As you can see I have to type the answer and only after the answer you can see the expected field Enter a value:

Problem loading configurations from a parent directory

This file uses commands to load common.tfvars and regional.tfvars

However, it seems to have a problem loading this common.tfvars in when it runs, saying it can't find common.tfvars.

Call to function "find_in_parent_folders" failed

I just tried navigating into the region level directory under the "parent" terragrunt.hcl file.
It errored out and no detail on error, just this

time=2021-04-16T12:39:14-05:00 level=error msg=Encountered the following errors:
Hit multiple errors:
Hit multiple errors:
exit status 127

It also generated a backend file in the "parent" directory that has the common variables, but is not the individual region's section.

When I copied the debug command output and run the command below it plans fine.

terraform plan -var-file="/Users/sheldonhull/git/repo/terragrunt/staging/eu-central-1/vpc/terragrunt-debug.tfvars.json" "/Users/sheldonhull/git/repo/terragrunt/staging/eu-central-1/vpc/.terragrunt-cache/QaAhZIo4_imktYigJ0DeKyQKkDk/ky_VpYadOztK_rvSkFtkAnFs6aY/modules/vpc"

This is a bit confusing as new terragrunt user.
I'm not seeing the native terraform commands exposed easily to know what happened, it's just outputting the terraform help content.

The error doesn't even show the actual terraform init/plan command, just this:


DEBU[0001] Run this command to replicate how terraform was invoked:  prefix=[/Users/sheldonhull/git/repo/terragrunt/staging/eu-central-1/vpc]
DEBU[0001] 	terraform  -var-file="/Users/sheldonhull/git/git/repo/terragrunt/staging/eu-central-1/vpc/terragrunt-debug.tfvars.json" "/Users/sheldonhull/git/terragrunt/staging/eu-central-1/vpc/.terragrunt-cache/QaAhZIo4_imktYigJ0DeKyQKkDk/ky_VpYadOztK_rvSkFtkAnFs6aY/modules/vpc"  prefix=[/Users/sheldonhull/git/repo/terragrunt/staging/eu-central-1/vpc]
panic: runtime error: index out of range [0] with length 0

goroutine 19 [running]:
github.com/gruntwork-io/terragrunt/options.(*TerragruntOptions).InsertTerraformCliArgs(0xc000365dc0, 0x2848b78, 0x0, 0x0)
	github.com/gruntwork-io/terragrunt/options/options.go:296 +0x37c
github.com/gruntwork-io/terragrunt/cli.runTerragruntWithConfig(0xc0003b4700, 0xc000365dc0, 0xc000894000, 0x0, 0xf, 0x0)
	github.com/gruntwork-io/terragrunt/cli/cli_app.go:620 +0xae
github.com/gruntwork-io/terragrunt/cli.RunTerragrunt(0xc0003b4700, 0x5, 0x1f21e28)
	github.com/gruntwork-io/terragrunt/cli/cli_app.go:474 +0x85c
github.com/gruntwork-io/terragrunt/configstack.(*runningModule).runNow(0xc0002e8820, 0xc000149f5f, 0x0)
	github.com/gruntwork-io/terragrunt/configstack/running_module.go:243 +0x185
github.com/gruntwork-io/terragrunt/configstack.(*runningModule).runModuleWhenReady(0xc0002e8820, 0xc0000a83c0)
	github.com/gruntwork-io/terragrunt/configstack/running_module.go:206 +0xd5
github.com/gruntwork-io/terragrunt/configstack.runModules.func1(0xc00088a410, 0xc0000a83c0, 0xc0002e8820)
	github.com/gruntwork-io/terragrunt/configstack/running_module.go:172 +0x5d
created by github.com/gruntwork-io/terragrunt/configstack.runModules
	github.com/gruntwork-io/terragrunt/configstack/running_module.go:170 +0x11b

Directory Structure

.                                                                                                   
├── README.md                                                                                       
├── Taskfile.yml                                                                                    
├── modules
│   ├── aws-data
│   │   ├── README.md
│   │   ├── main.tf
│   │   ├── outputs.tf
│   │   └── versions.tf
│   └── vpc
│       ├── README.md
│       ├── label.tf
│       ├── outputs.subnets.tf
│       ├── outputs.vpc.tf
│       ├── variables.common.tf
│       ├── variables.stack.tf
│       ├── versions.tf
│       └── vpc.tf
├── terragrunt
│   └── staging     <<<<<<<<<<< THIS IS MY WORKING DIRECTORY, running terragrunt from this context
│       ├── Taskfile.yml
│       ├── common.tfvars  <---- NEVER SEES TO FIND THIS
│       ├── eu-central-1   <---- navigating into this subdirectory and running doesn't change it's ability to find the common.tfvars
│       │   ├── regional.tfvars
│       │   └── vpc
│       │       ├── terragrunt-debug.tfvars.json
│       │       └── terragrunt.hcl
│       ├── global  <---- not doing anything with it, they are all commented out for now
│       │   ├── iam-account
│       │   │   └── terragrunt.hcl
│       │   ├── iam-assumable-roles
│       │   │   └── terragrunt.hcl
│       │   └── regional.tfvars
│       └── terragrunt.hcl  <---- what i'm calling "ROOT" so I can "run-all" against all the pieces in this stack

Does this actually work?

I noticed the terragrunt.hcl file is missing. Not sure how to properly use this repo as a reference.

The terragrunt function wrapper is not compatible with the later versions of Terragrunt

terragrunt () {
	local action=$1
	shift 1
	command terragrunt $action "$@" 2>&1 | sed -E "s|$(dirname $(pwd))/||g;s|^\[terragrunt\]( [0-9]{4}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2})* ||g;s|(\[.*\]) [0-9]{4}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}|\1|g"
}

The problem is that with any command which requires input, it never comes up.

Example:

terragrunt run-all apply
time=2021-12-04T14:55:46Z level=info msg=The stack at uat will be processed in the following order for command apply:
Group 1
- Module uat/network

Group 2
- Module uat/ecs_service

I'm not actually sure what the fix would be... but basically, it's now showing Are you sure you want to run 'terragrunt apply' in each folder of the stack described above? (y/n) until you actually enter y or n

[question] Can you share some thoughts on Terragrunt in the context of upcoming 0.12 and Terraform SaaS?

Benefits of Terragrunt?

Terragrunt vs. Terraform SaaS?

Sharing some of your thoughts would be helpful

antonbabenko / terragrunt-reference-architecture Goto Github PK