I'm just starting to have a go at using this package and I've come across an issue with how AWS authentication is handled.

In the AWS module there are a lot of auth methods such as NewAuthenticatedSession, CreateAwsSessionWithMfa and NewAuthenticatedSessionFromRole. The issue is that none of these apart from NewAuthenticatedSession are implemented anywhere which makes them unusable.

Am I missing something or should I have a look in to how to get these properly implemented?

Terratest has an aws package with a number of utilities useful for AWS. It would be great to add similar utilities for testing code in GCP.

Validation of creation of DB instances
Validation of creation of DB parameter groups
Validation of creation of DB option groups
Validation of association of the above

Hello,

First of all, let me start by saying that I recently discovered this library, and that I truly appreciate it, as it is very useful!

With that being said, I was trying to test a terraform module that created dynamoDb tables.
During the tutorials, I noticed it was possible to test RDS databases, but now, upon viewing the documentation of the aws package, I noticed that there was no function available for testing DynamoDb.

Is this really the case, or did I missed the function completely?

Question:
The Terraform output can be defined as a list, e.g.

output "CPPublicIPs" {
  value = ["${oci_core_instance.CPServer.*.public_ip}"]
}

will provide a following output:

CPPublicIPs = [
    129.146.109.157,
    129.146.111.161,
    129.146.101.69
]

However, Terratest function Output provides just a string. Is there a recommendation how to deal with output list? Does it have to be parsed manually? E.g.

out := terraform.Output(t, terraformOptions, "CPPublicIPs")
ips := strings.Split(out, "\n")

Wouldn't be convenient to have a function like OutputList(t *testing.T, options *Options, key string) []string?

I'm trying to pass IAM policy JSON through terraformOptions.Vars, but without success (raw JSON results in syntax error). Any help?

Hello all, I use the tfvars file just about everywhere for my terraform (try to avoid using the -var arg), is there any chance that -var-file arg support will be coming soon?

I started planning a skeleton for a series of guides/blog posts related to Terratest. The end goal here would be to launch a resource site/write a series of blog posts to foster further Terratest adoption and promote best practices.

Skeleton

1. Requirements
2. Installation
3. Testing HTTP Services
4. Testing Packer templates
5. Testing Terraform AWS resources
6. Testing Terraform GCP resources
7. Testing Docker
8. Files
9. Git
10. Misc
    1. Random Functions
    2. Retry

N.B: This is a work in progress.

I've been using Terratest in anger recently and have spotted a few things about the library.

Part of a pipeline that is building an AMI, I've found it quite common to add a tag to an AMI once it has passed testing. Adding a method to add tags to an AMI, along with other AWS resources, would be useful.

Terratest has an aws package with a number of utilities useful for AWS. It would be great to add similar utilities for testing code in Azure.

After I tried SSH example plenty of times, which is pretty popular and really interesting, I found it quite difficult to use key pair. Sometimes I would like to use my local private and public key, which means that I do not generate a new key pair and utilize it as "terraform_ssh_example_test.go" does. If I use AWS, it is not a big issue because I can upload my local key pair to AWS and then import it, using AWS specific functions such as "ImportEC2KeyPair".
However, if I would like to try other kinds of cloud platform, say, Azure, I cannot find a easy method to utilize given functions in terratest/modules/ssh, because SSH host requires AWS specific input aws.Ec2Keypair. As a result, I have to write my own test code, which is quite similar to given functions in Terratest but instead not platform specific, to test SSH to virtual machines on other platforms.
So does anyone meet the same issue? I will feel quite glad if someone knows the solution to my question. Thanks a lot!

In the past few days, I used given http example and it worked pretty well. However, I think that existing functions do not meet all of my requirements.

First, sometimes I do not need validation of response body. For instance, I would like to deploy a template web application, but I am not aware of the whole body of this web app. As a result, I only care about whether I can get an http response successfully, instead of checking contents.

Second, in some cases, I would like to validate specific parts of response body, not the whole content. For example, I want to launch a Ubuntu Linux server and run commands to install nginx. This process can be done using checkSshCommandE. Next, after I open port 80 to allow http request, I would like to send an http request to see whether the response body contains the word "nginx". In this circumstance, the body is really long and even varies among different versions. By contrast, I can make sure that the content contains "nginx". So my requirement here is just to check a substring.

In above cases, I have to write my own test code using retry.DoWithRetry, HttpGet, etc. I wonder whether these enhancement suggestions are really worthwhile. If needed, I can add some features and make pull requests. Thanks a lot!

I've been using Terratest in anger recently and have spotted a few things about the library.

Adding support for SCPing content onto the instances is something that I feel would be handy to include as an easier way to get files onto instances.

Support for SCPing content off of instances while may be nice isn't as critical, due to the fact you can always just cat filename.

The use case I have in mind is creating an AMI using Packer, including baking in relevant binaries, and then testing the AMI by spinning up a simple test instance and copying configuration over to the test instance.

I've been using Terratest in anger recently and have spotted a few things about the library.

A useful feature to add would be to share an AMI to other AWS accounts.

The use case I have in mind is once an AMI has been built and tested, to then share it from an AMI-only account to other AWS accounts (dev, prod, etc).

Since Go 1.10, test results get cached. This leads to the tests not being run sometimes because Go has cached the results and just assumes it will get the same results because the Go code hasn't changed. We should add a point to the documentation "Testing best practises" that asks users to set the GOCACHE environment variable to off for terratests:

$ GOCACHE=off go test -v -timeout 45m ./...

Similar to how you can check an S3 bucket exists and confirm that it has the content you expect I'd like to be able to check that an IAM policy or S3 bucket policy exists.

I'd also like to check that it has the document contents I expect based on terraform outputs.

Testing this quickly I was able to use json.Compact on terraform's output and assert.JSONeq from testify on the document return from these to test equality.

I don't know how to add these to the automated tests without adding complexity to them but if this isn't a worry I'd be happy to.

Issue: Test timing out after AMI is created and not going to defer step.

func TestPackerBuild(t *testing.T) {
  t.Parallel()

  test_structure.RunTestStage(t, "build_ami", func() {
    buildBaseAMI(t, awsRegion, workingDir, baseTemplate)
  })


  defer test_structure.RunTestStage(t, "cleanup_ami", func() {
    deleteAMI(t, awsRegion, workingDir)
  })
}

func buildBaseAMI(t *testing.T, awsRegion string, workingDir string, fileName string) {
  packerOptions := &packer.Options{
    Template: workingDir + fileName,
    Env: map[string]string{
    },
  }

  test_structure.SavePackerOptions(t, workingDir, packerOptions)

  amiID := packer.BuildAmi(t, packerOptions)

  test_structure.SaveAmiId(t, workingDir, amiID)
}

Right after a log line that claims AMI is created AMIs were created:\nus-east-1: ami-sdf8v8w\n terratest_1 | panic: test timed out after 30m0s Nothing happens and it times after after configured timeout. The AMI was created just fine in the AWS console.

Error

terratest_1  | panic: test timed out after 30m0s
terratest_1  |
terratest_1  | goroutine 22 [running]:
terratest_1  | testing.(*M).startAlarm.func1()
terratest_1  |  /usr/local/go/src/testing/testing.go:1240 +0xfc
terratest_1  | created by time.goFunc
terratest_1  |  /usr/local/go/src/time/sleep.go:172 +0x44
terratest_1  |
terratest_1  | goroutine 1 [chan receive]:
terratest_1  | testing.tRunner.func1(0xc4201e6000)
terratest_1  |  /usr/local/go/src/testing/testing.go:753 +0x11d
terratest_1  | testing.tRunner(0xc4201e6000, 0xc4201e5df8)
terratest_1  |  /usr/local/go/src/testing/testing.go:783 +0xf2
terratest_1  | testing.runTests(0xc4201195a0, 0xa2ea10, 0x1, 0x1, 0x411a99)
terratest_1  |  /usr/local/go/src/testing/testing.go:1061 +0x2c4
terratest_1  | testing.(*M).Run(0xc420138080, 0x0)
terratest_1  |  /usr/local/go/src/testing/testing.go:978 +0x171
terratest_1  | main.main()
terratest_1  |  _testmain.go:42 +0x151
terratest_1  |
terratest_1  | goroutine 20 [IO wait]:
terratest_1  | internal/poll.runtime_pollWait(0x7fec55acaf00, 0x72, 0xc420057770)
terratest_1  |  /usr/local/go/src/runtime/netpoll.go:173 +0x57
terratest_1  | internal/poll.(*pollDesc).wait(0xc4200986a8, 0x72, 0xffffffffffffff01, 0x88d9c0, 0xa01468)
terratest_1  |  /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b
terratest_1  | internal/poll.(*pollDesc).waitRead(0xc4200986a8, 0xc4201f6001, 0xf52, 0xf52)
terratest_1  |  /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
terratest_1  | internal/poll.(*FD).Read(0xc420098690, 0xc4201f60ae, 0xf52, 0xf52, 0x0, 0x0, 0x0)
terratest_1  |  /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d
terratest_1  | os.(*File).read(0xc420096158, 0xc4201f60ae, 0xf52, 0xf52, 0x0, 0x88, 0xc42008a0c0)
terratest_1  |  /usr/local/go/src/os/file_unix.go:226 +0x4e
terratest_1  | os.(*File).Read(0xc420096158, 0xc4201f60ae, 0xf52, 0xf52, 0x0, 0x2, 0x2)
terratest_1  |  /usr/local/go/src/os/file.go:107 +0x6a
terratest_1  | bufio.(*Scanner).Scan(0xc420057ad8, 0xc420057a48)
terratest_1  |  /usr/local/go/src/bufio/scan.go:208 +0xbf
terratest_1  | infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/shell.readStdoutAndStderr(0xc4201e60f0, 0x88f820, 0xc420096158, 0x88f820, 0xc420096168, 0x0,
0x0, 0xc4201c0270, 0xc420192300)
terratest_1  |  /go/src/infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/shell/command.go:94 +0x15e
terratest_1  | infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/shell.RunCommandAndGetOutputE(0xc4201e60f0, 0x80a0d3, 0x6, 0xc42009aec0, 0x3, 0x4, 0x0, 0x0,
0xc4201c0270, 0xc420057c98, ...)
terratest_1  |  /go/src/infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/shell/command.go:74 +0x2b6
terratest_1  | infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/packer.BuildAmiE(0xc4201e60f0, 0xc4201c0240, 0x33, 0xc420057d40, 0x73983d, 0xc4201e60f0)
terratest_1  |  /go/src/infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/packer/packer.go:41 +0x15e
terratest_1  | infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/packer.BuildAmi(0xc4201e60f0, 0xc4201c0240, 0x16, 0xc4201c0240)
terratest_1  |  /go/src/infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/packer/packer.go:24 +0x39
terratest_1  | infrastructure/tests/packer.buildBaseAMI(0xc4201e60f0, 0x80b184, 0x9, 0x81090c, 0x16, 0x8120d5, 0x19)
terratest_1  |  /go/src/infrastructure/tests/packer/wm-base_test.go:67 +0x6a3
terratest_1  | infrastructure/tests/packer.TestPackerBuild.func1()
terratest_1  |  /go/src/infrastructure/tests/packer/wm-base_test.go:26 +0x60
terratest_1  | infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/test-structure.RunTestStage(0xc4201e60f0, 0x80ae48, 0x9, 0xc420057f58)
terratest_1  |  /go/src/infrastructure/tests/vendor/github.com/gruntwork-io/terratest/modules/test-structure/test_structure.go:24 +0x20a
terratest_1  | infrastructure/tests/packer.TestPackerBuild(0xc4201e60f0)
terratest_1  |  /go/src/infrastructure/tests/packer/wm-base_test.go:25 +0x115
terratest_1  | testing.tRunner(0xc4201e60f0, 0x8232e0)
terratest_1  |  /usr/local/go/src/testing/testing.go:777 +0xd0
terratest_1  | created by testing.(*T).Run
terratest_1  |  /usr/local/go/src/testing/testing.go:824 +0x2e0
terratest_1  |
terratest_1  | goroutine 21 [chan receive]:
terratest_1  | testing.runTests.func1.1(0xc4201e6000)
terratest_1  |  /usr/local/go/src/testing/testing.go:1068 +0x3b
terratest_1  | created by testing.runTests.func1
terratest_1  |  /usr/local/go/src/testing/testing.go:1068 +0xa2
terratest_1  | FAIL     infrastructure/tests/packer     1800.013s
tests_terratest_1 exited with code 1
 ~/go/src/wm/infrastructure/tests  TEC

I would like to propose a new feature of running a terraform plan with the -detailed-exitcode flag and the ability to assert that:

1. there are no deviations (exit 2)
2. if there are any errors (exit 1) e.g. when there is a resource ends up in a locked state due to an aborted terraform apply

If conceptually agreed upon, I can work on it.

It should be possible to use the -no-color option when running terraform. This is because in certain situations, such as when running on a CI server, the colour escape codes get displayed rather than changing the colour of the output text.

I've been using Terratest in anger recently and have had an idea about potential further functionality for the library.

As part of building an AMI, it's common to spin up the AMI once created in a test instance to test that the AMI contains all of the relevant changes, such as confirming that it contains the enhanced networking driver or that a piece of software that was installed can be started by systemd. Other solutions in this area would be something like serverspec.

My idea is about adding similar functionality to serverspec to Terratest to make it easier to verify the state of an instance, such as simplifying the querying of the group, encrypted password, etc of a user or querying the state of a service in systemd.

In some of the documentation, the deferral of the destruction of resources comes after the InitAndApply function is executed. This can lead to problems if InitAndApply aborts the test because then the defer statement is never called. I think we should change all the examples to have the following order:

defer terraform.Destroy(t, terraformOptions)
terraform.InitAndApply(t, terraformOptions)

It's already implemented like that in all the example tests but not in the documentation. This would solve the problem of new users having to clean up resources manually if InitAndApply fails and aborts the test.

Hey @brikis98 and company,

I'd like to start a discussion here about assertion patterns in terratest.

I've seen a few helper functions in the modules directory such as GetPublicIpsOfEc2Instances and FindS3BucketWithTag. There are super useful.

We're interested in going a bit further and creating assertion functions that test cases can use to ensure that the infrastructure was created as expected. As a starting point, we're interested in ones that assert properties of AWS resources; more can be added later.

Some questions I have -- feel free to add more thoughts into the mix:

1. What should the function signatures look like? Here's an idea for a pattern: Assert<Resource>Has<Property>. For example, AssertEC2InstanceHasAddr(t *testing.T, instanceID string, addr string).

2. Should assertions live in a separate repository, like terratest-assertions? (I'm thinking that the scope of this repository is already getting reasonably wide; it might improve velocity if the assertions live elsewhere.)

3. Should assertions be struct methods? In order not to be excessively verbose, it might be convenient to wrap a *testing.T and client.ConfigProvider[1] into a struct and then call the assertion methods on it.

[1] client.ConfigProvider is the argument for all service constructor methods in the AWS Go SDK, see https://docs.aws.amazon.com/sdk-for-go/api/aws/client/#ConfigProvider

Terraform Project Structure:

-- terraform-root
---- main.tf
---- variables.tf
---- outputs.tf
---- test/
-------- basic_test.go
---- examples/
-------- elasticsearch-auth-iam-user/main.tf

my basic_test.go looks like the following

package test

import (
    "fmt"
    "testing"
    "github.com/gruntwork-io/terratest/modules/terraform"
)

func TestTerraformBasicExample(t *testing.T) {

	terraformOptions := &terraform.Options{
		TerraformDir: "../ examples/elasticsearch-auth-iam-user",

		Vars: map[string]interface{}{
                        "domain_name": "test",
            	        "stack": "elasticsearch",
                        "workspace": "stg",
		},
	}

	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)
}

cd into test
run "go test -timeout 5m".

I get "terraform init" succeeded but "terraform apply" hanged without any outputs until timing out.

Log looks like

[ec2-user@ip-10-104-88-81 test]$ go test -timeout 5m
init...................TestTerraformBasicExample 2018-08-20T10:37:20Z retry.go:69: Running terraform [init -upgrade=false]
TestTerraformBasicExample 2018-08-20T10:37:20Z command.go:52: Running command terraform with args [init -upgrade=false]
TestTerraformBasicExample 2018-08-20T10:37:20Z command.go:96: Initializing modules...
TestTerraformBasicExample 2018-08-20T10:37:20Z command.go:96: - module.elasticsearch_security_group
TestTerraformBasicExample 2018-08-20T10:37:20Z command.go:96:   Getting source "git::ssh://[email protected]/iot-infrastructure/tf-module-aws-security-group.git?ref=master"
TestTerraformBasicExample 2018-08-20T10:37:21Z command.go:96:
TestTerraformBasicExample 2018-08-20T10:37:21Z command.go:96: Initializing provider plugins...
TestTerraformBasicExample 2018-08-20T10:37:21Z command.go:96: - Checking for available provider plugins on https://releases.hashicorp.com...
TestTerraformBasicExample 2018-08-20T10:37:21Z command.go:96: - Downloading plugin for provider "aws" (1.32.0)...
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96:
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: The following providers do not have any version constraints in configuration,
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: so the latest version was installed.
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96:
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: To prevent automatic upgrades to new major versions that may contain breaking
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: changes, it is recommended to add version = "..." constraints to the
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: corresponding provider blocks in configuration, with the constraint strings
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: suggested below.
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96:
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: * provider.aws: version = "~> 1.32"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96:
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: Terraform has been successfully initialized!
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96:
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: You may now begin working with Terraform. Try running "terraform plan" to see
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: any changes that are required for your infrastructure. All Terraform commands
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: should now work.
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96:
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: If you ever set or change modules or backend configuration for Terraform,
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: rerun this command to reinitialize your working directory. If you forget, other
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:96: commands will detect it and remind you to do so if necessary.
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:20 [INFO] Terraform version: 0.11.7  41e50bd32a8825a84535e353c3674af8ce799161
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:20 [INFO] Go runtime version: go1.10.1
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:20 [INFO] CLI args: []string{"/usr/local/bin/terraform", "init", "-upgrade=false"}
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:20 [DEBUG] Attempting to open CLI config file: /home/ec2-user/.terraformrc
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:20 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:20 [INFO] CLI command args: []string{"init", "-upgrade=false"}
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:20 [DEBUG] command: loading backend config file: /home/ec2-user/go/src/tf-module-aws-elasticsearch
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:20 [ERR] Checkpoint error: Get https://checkpoint-api.hashicorp.com/v1/check/terraform?arch=386&os=linux&signature=1876a657-05db-8573-ee03-ee17a78287f6&version=0.11.7: Forbidden
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] found "git::ssh://[email protected]/iot-infrastructure/tf-module-aws-security-group.git?ref=master" in ".terraform/modules/30207133dde280c8832b06615f77d133": true
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [TRACE] "git::ssh://[email protected]/iot-infrastructure/tf-module-aws-security-group.git?ref=master" stored in ".terraform/modules/30207133dde280c8832b06615f77d133"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [INFO] command: empty terraform config, returning nil
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] command: no data state file found for backend config
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] New state was assigned lineage "33329760-d197-8ad8-7d93-841dc70d9b45"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [INFO] command: backend initialized: <nil>
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] checking for provider in "."
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] checking for provider in "/usr/local/bin"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] checking for provisioner in "."
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] checking for provisioner in "/usr/local/bin"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [INFO] Failed to read plugin lock file .terraform/plugins/linux_386/lock.json: open .terraform/plugins/linux_386/lock.json: no such file or directory
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [INFO] command: backend <nil> is not enhanced, wrapping in local
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] checking for provider in "."
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] checking for provider in "/usr/local/bin"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] plugin requirements: "aws"=""
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] fetching provider info for aws version 1.32.0
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] getting provider "aws" version "1.32.0"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:21 [DEBUG] plugin cache is disabled, so downloading aws 1.32.0 from https://releases.hashicorp.com/terraform-provider-aws/1.32.0/terraform-provider-aws_1.32.0_linux_386.zip?checksum=sha256:b9b35d1040c7d764e7dc0a1083826f8d9b63f49f46a53ee521c6c6c138130d64
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] looking for the aws 1.32.0 plugin we just installed
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] checking for provider in ".terraform/plugins/linux_386"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] found provider "terraform-provider-aws_v1.32.0_x4"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] all plugins found discovery.PluginMetaSet{discovery.PluginMeta{Name:"aws", Version:"1.32.0", Path:"/home/ec2-user/go/src/tf-module-aws-elasticsearch/.terraform/plugins/linux_386/terraform-provider-aws_v1.32.0_x4"}:struct {}{}}
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] filtered plugins discovery.PluginMetaSet{discovery.PluginMeta{Name:"aws", Version:"1.32.0", Path:"/home/ec2-user/go/src/tf-module-aws-elasticsearch/.terraform/plugins/linux_386/terraform-provider-aws_v1.32.0_x4"}:struct {}{}}
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] checking for provider in "."
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] checking for provider in "/usr/local/bin"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] checking for provider in ".terraform/plugins/linux_386"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] found provider "terraform-provider-aws_v1.32.0_x4"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:22 [DEBUG] found valid plugin: "aws", "1.32.0", "/home/ec2-user/go/src/tf-module-aws-elasticsearch/.terraform/plugins/linux_386/terraform-provider-aws_v1.32.0_x4"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:23 [DEBUG] checking for provider in ".terraform/plugins/linux_386"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:23 [DEBUG] found provider "terraform-provider-aws_v1.32.0_x4"
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:100: 2018/08/20 10:37:23 [DEBUG] plugin: waiting for all plugin processes to complete...
apply...................TestTerraformBasicExample 2018-08-20T10:37:23Z retry.go:69: Running terraform [apply -input=false -lock=false -auto-approve -var access_policies="a" -var domain_name="test" -var stack="elasticsearch" -var workspace="stg"]
TestTerraformBasicExample 2018-08-20T10:37:23Z command.go:52: Running command terraform with args [apply -input=false -lock=false -auto-approve -var access_policies="a" -var domain_name="test" -var stack="elasticsearch" -var workspace="stg"]

main.tf looks like

# If comment out this module usage, terratest works
/*
module "elasticsearch_security_group" {
  source = "git::ssh://[email protected]/iot-infrastructure/tf-module-aws-security-group.git?ref=master"
  name = "elasticsearch_security_group"
  vpc_id = "${var.vpc_id}"
  ingress_cidr_blocks = ["0.0.0.0/0"]
}
*/
resource "aws_elasticsearch_domain" "es_test" {
  domain_name           = "test"

  cluster_config {
    instance_type = "r3.large.elasticsearch"
    instance_count = "1"
    dedicated_master_enabled = "false"
    zone_awareness_enabled = "false"
  }
}

# meta parameters for module

variable "vpc_id" {
  type = "string"
  default = "vpc-42f2792a"
}

# Provider
provider "aws" {
  region = "eu-central-1"
}

If I comment module "elasticsearch_security_group" out. It works. Can it possible terratest has problem with module ?

It would be nice to have supports for Digitalocean.

The http_helper package currently supports the HTTP GET method only. Additionally, it doesn't allow to send any HTTP Header.

Questions:

1. Is it planned to support other HTTP methods, e.g. POST?

2. Is it planned to support HTTP Headers? This could be important if a service expects some mandatory headers, e.g. a token etc.

I have used Terratest in several cases on AWS and Azure. I found your work was extremely fascinating and really useful in plenty of circumstances. However, I met some problems and would like to report an issue here. I feel curious about whether it's my fault or there actually occurs some defects.

Firstly, when I am using ssh example for the first time, the program cannot find the directory named /.test-data, unless I create such directory and files like TerraformOptions.json manually. But for the second time, this error no longer occurs.

Secondly, when I try to use some modules, the program seems cannot find the output unless I explicitly write output in tf file. For instance, I have to add such lines to guarantee that the program runs correctly even if this module actually has some outputs.
output "resource_group_name" {
value = "${module.xxx.resource_group_name}"
}
As a result, I wonder whether Terratest really supports usage of modules.

Last but not the least, when the test fails, I found it really difficult to locate the error, because the log is too long and I have to dive into the log deeply and find what the error actually is. I am just wondering how I can make this process easier. Thanks a lot!

After I read most of code in modules, I found that there are no particular functions related to database. Therefore, if I would like to deploy a database (i.e. Microsoft SQL Server) using terraform, I have to write my own test code to check whether the database runs correctly.

As a result, I wonder whether it is necessary to add methods to support database operations, such as connection, executing SQL commands, etc. I have written several functions to help my own validation, but I really want to know whether adding such kind of features to terratest is important.

Question:
In our Terraform scripts, we are using a concept of workspaces. For example, it's part of naming convention of resources:

resource "oci_core_instance" "CPServer" {
  display_name = "cpServer${count.index}-${terraform.workspace}"
  # other configuration
}

Is there a way to work with workspaces in the Terratest test? E.g. switch to the existing, or create the new one.

Is it even a valid use case?

I was looking on how to start contributing to this project. I saw this issue here:
"Contributing
Contributions are very welcome! Check out the Contribution Guidelines for instructions."
The hyperlink that is embedded to "Contribution Guidelines" is broken. As i click it, its leading me to "404" error.

When running terratests locally, it doesn't automatically upgrade to the latest version of providers and modules. Terraform solves that by having an -upgrade=true flag. It would be great if this flag was part of terraform.Options.

type Options struct {
        ...
        Upgrade    bool    // Whether the -upgrade flag of the terraform init command should be set to true or not
}

This will make it easier to see what modules/functions are available.

I am considering adding a set of functions to return the Terraform output for a map, similar to the recently added OutputList and OutputListE functions.

That was actually suggested in the PR and I have a need for it: I have a Terraform module that generates standardized AWS tags and outputs them as a map.

The function signatures would be:

OutputMap(t *testing.T, options *Options, key string) [string]string
OutputMapE(t *testing.T, options *Options, key string) ([string]string, error)

If you think it would be valuable to the community, I can submit a PR for that feature.

After reading the contributions guideline, i decided to ask here itself. Noticed that there is no open issues at this repo. Could i still get to participate and contribute in anyway?

I've been using Terratest in anger recently and have spotted a few things about the library.

Adding a log message to the runSSHCommand method would mean that when CheckSshCommand fails, you aren't just left with a message of how the exit status for some command wasn't 0. An example can be found in RunCommandAndGetOutputE which makes it easier to track what went wrong when a test fails.

Use case: It's difficult to track down what failed when executing numerous CheckSshCommand when testing an AMI

In a similar vein to having the aws package with a number of utilities useful for AWS, Terratest should provide functionalities for interacting with Kubernetes and Helm for verification.

I will be working on the following functionalities as needed:

• Open proxy to service in background, with ability to close it
• Get helm releases
• Apply helm chart
• Upgrade helm release
• Delete helm release
• Create a namespace
• Delete YAML resource
• Apply YAML resource to a namespace
• Delete config cluster, user, and context
• Get nodes in cluster
• Get ready nodes in cluster
• Wait for all nodes in cluster to be ready

As discussed in #128 connectivity tests to remaining RDS databases needs to be implemented:

• SQL Server
• PostgreSQL
• Oracle
• Aurora - might already be covered with existing MySQL & above mentioned PostgreSQL.

The previous method also should be renamed from GetWhetherSchemaExistsInRdsMySqlInstance to GetWhetherSchemaExistsInMySqlInstance since the method is not RDS specific. will keep the new method names accordingly.

We often use Terratest to:

1. Spin up one or more EC2 Instances by running terraform apply.
2. Wait for those Instances to boot and do something (e.g., form a cluster), usually by executing some sort of code as part of User Data.
3. Test that the Instances are working correctly.
4. Tear down the servers.

If the process of spinning up the EC2 Instances fails (step 1), we get an error from Terraform that makes it clear what went wrong. But if step 2 fails, then from the console where we ran terraform apply, we cannot see what wrong. That's because the error happened in the EC2 Instances, and Terraform has no visibility into what's happening with those. This makes it tougher to diagnose a test that failed in CI, as the error log you need is on an EC2 Instance that was already torn down per step 4.

Our best workaround so far is to use the AWS syslog APIs to fetch the most recent data from syslog. However, not all apps write to syslog, and the APIs only give us the last X KB, so important data is often missing.

This issue is to explore the question of, what's the simplest solution we could build to be able to retrieve the log data from EC2 Instances and display it in stdout of the test so the user can diagnose it without having to SSH to the EC2 Instances?

Add support for configuring the Terraform backend when running terraform init

From terraform init --help:

Usage: terraform init [options] [DIR]
...
Options:

  -backend=true        Configure the backend for this configuration.

  -backend-config=path This can be either a path to an HCL file with key/value
                       assignments (same format as terraform.tfvars) or a
                       'key=value' format. This is merged with what is in the
                       configuration file. This can be specified multiple
                       times. The backend type must be in the configuration
                       itself.

Terratest has an aws package with a number of utilities useful for AWS. It would be great to add similar utilities for testing code in OCI.

A module that will check specific set of keys like companyTeam, companyCost and such. This will help to make sure that terraform code must have mandatory tags.

Question:
I went through the examples and ssh package source code, but I didn't find a function for loading of existing SSH keys/key-pair. All the scenarios are about:

1. generating of a key-pair,
2. importing of the key-pair to AWS,
3. temporary saving of the key-pair for later use,
4. loading of the saved key-pair,
5. some tests,
6. deleting of the key-pair.

I'm missing a function which just performs key-pair loading, but not from the transient test data, but from the "real" keys stored somewhere on the filesystem.

The GetAccountId method currently calls iam.GetUser and extracts the account ID from the user's ARN:

https://github.com/gruntwork-io/terratest/blob/master/modules/aws/account.go#L27

This means the method only works when authenticated as an IAM User, but will not work if you use, say, IAM Roles for auth. A better approach would be to use sts.GetCallerIdentity, which works with all auth mechanisms, and returns the account ID as a field, requiring no extra extraction.

Initially, we supported only AWS. We now support GCP and thanks to #158, #159, and #160, we have limited support for Oracle Cloud Infrastructure (OCI). We should update our README to reflect this support.

Hello,
I'd like to test that S3 replication is setup correctly. I can use GetS3ObjectContents to test the replica destination but I have no easy way to put an object. This looks pretty easy given that the get test already exists, open to this idea before I start work?

Hi, I'm currently trying to install terratest via dep and I'm running into some roadblocks. I installed dep with brew install dep, and successfully ran dep init and dep ensure in my terraform-tests directory, but when I try to run dep ensure -add github.com/gruntwork-io/terratest, I get this error:

Solving failure: No versions of github.com/gruntwork-io/terratest met constraints:
	v0.9.3: Could not introduce github.com/gruntwork-io/[email protected], as its subpackage github.com/gruntwork-io/terratest does not contain usable Go code (*build.NoGoError).. (Package is required by (root).)

Are there some other steps that are not present in the docs? For reference, here are the outputs of go version:

go version go1.10.2 darwin/amd64

and dep version:

dep:
    version     : v0.4.1
    build date  : 2018-01-27
    git hash    : 37d9ea0
    go version  : go1.9.3
    go compiler : gc
    platform    : darwin/amd64

Thanks!

I have 2 modules in my terraform file and would like to invoke only one module from the terraform file via my Terratest suite. Unfortunately, there is no support in Terratest for this.

Ideally in my test suite, I would want to call the first module via -target and then call the second module via -target, thereby I can test upgrading to v1.4.6 does not cause any issues.

Terraform does not support using 'count' on modules yet and hence this feature to use -target in terraformOptions of Terratest will be of great help.

Terraform Module

provider "aws" {
region = "${var.region}"
}

module "aws_s3_previous" {
#Testing source from this module
//count = "${var.current == false ? 1: 0}"
source = "git::ssh://xxxx-xxxxxx-xxxxxxxx-xxxxxx-bucket.git?ref=v1.4.5"
region = "${var.region}"
app_name = "${var.app_name}"
owner = "${var.owner}"
allow_force_destroy = "${var.allow_force_destroy}"
}

module "aws_s3_current" {
#Testing source from this module
//count = "${var.current == true ? 1: 0}"
source = "git::ssh://xxxx-xxxxxx-xxxxxxxx-xxxxxx-bucket.git?ref=v1.4.6"
region = "${var.region}"
app_name = "${var.app_name}"
owner = "${var.owner}"
allow_force_destroy = "${var.allow_force_destroy}"
}

terraform {
backend "local" {
path = "../stateFiles/terraform.tfstate"
}
}

Use case

Ability to do "terraform apply -target=${module.aws_s3_previous}" in Terratest.

In #159, we added support for OCI. We now need to configure Terratest's CI environment with OCI credentials so the tests work. For the time being, I will configure the tests to skip during CI so the build will pass.

$ cd terraform-test/test
$ go test ./...
vendor/github.com/gruntwork-io/terratest/url_checker.go:7:2: cannot find package "github.com/gruntwork-io/terratest/http" in any of:
    /usr/local/go/src/github.com/gruntwork-io/terratest/http (from $GOROOT)
    /Users/josh/go/src/github.com/gruntwork-io/terratest/http (from $GOPATH)

Should this package be named http?

'cc @brikis98

I've been using Terratest in anger recently and have spotted a few things about the library.

As part of building a pipeline for some Packer/Terraform infrastructure, it's common that AMIs will be created that are instantly destined for the bin - either because a branch/PR is being tested or because the AMI failed testing. It'd be useful to add a method that is capable of both deregistering the AMI as well as deleting the snapshot behind the AMI.

Is it possible to swap executable that is used from terraform to terragrunt?