Giter VIP home page Giter VIP logo

azure-pipelines-tasks-terraform's Introduction

NOTICE: PROJECT TRANSITIONED TO NEW OWNER

@jason-johnson has taken over ownership of this extension. The previous publisher will no longer be creating updates for this extension so, going forward, please use the new publisher as described in this updated documentation.

Azure Pipelines Extension for Terraform

Build Status Visual Studio Marketplace Installs - Azure DevOps Extension

This contains tasks for installing and executing Terraform commands from Azure Pipelines. These extensions are intended to work on any build agent. They are also intended to provide a guided abstraction to deploying infrastructure with Terraform from Azure Pipelines.

The tasks contained within this extension are:

This extension also contains views for the pipeline summary to help inspect actions performed by terraform.

The views contained within this extension are:

Telemetry Collection

The software may collect information about you and your use of the software and send to the repository owner. The repository owner may use this information to provide services and improve our products and services. You may turn off the telemetry as described below.

Disabling Telemetry Collection

Telemetry collection can be disabled by setting the allowTelemetryCollection property to false.

From classic pipeline editor, uncheck the Allow Telemetry Collection checkbox to disable telemetry collection.

Preferred Languages

We prefer all communications to be in English.

azure-pipelines-tasks-terraform's People

Contributors

aleks-ivanov avatar benjamincburns avatar cedricfortin avatar charleszipp avatar denwin avatar dependabot[bot] avatar dvasdekis avatar fsocietyinc avatar jakobsack avatar jaredfholgate avatar jason-johnson avatar loispostula avatar maddingo avatar martenvd avatar nickdrouin avatar owlleg6 avatar piizei avatar rubensprt avatar simonalling avatar someoneelsescloud avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

azure-pipelines-tasks-terraform's Issues

Azure DevOps Releace Pipeline Service connection credentials may not be used properly within the Terraform task.

To whom this may concern,
I am contacting you as I received a strange error during a Terraform task, which Microsoft could not resolve.
The details of the error are as follows:

  • The Terraform task fails with the error "You must specify either the access key / SAS token or the resource group for your storage account" only in the specific release pipeline (the "dev" pipeline).

  • In another release pipeline (the "common" pipeline) that uses the same service connection, Terraform tasks run successfully.
    If you run the same Terraform command manually, it will succeed.

  • Only in the "dev pipeline", Terraform tasks may not be able to properly use service connection credentials.

The "dev pipeline" and the "common pipeline" each use the same yaml.

So why does only the "dev pipeline" fail with an authentication error?
A log file of the "dev pipeline" that failed due to an error is attached, so please use it for investigation.

Thank you for your cooperation in advance.

DevOps_dev.zip

Add GitHub Codespaces configuration

  • GitHub Codespaces is a configurable cloud development environment available in your browser on GitHub or through Visual Studio Code.

Pipeline Foundation is a non-profit initiative with the sole purpose of giving back to the IT community by assisting OSS projects with DevOps implementations and best practices.

Built with ❤ by Pipeline Foundation.

New Feature Request: Support TFE TF Cloud tokens

Describe the feature
To access remote backends and a tfe module registry you must provide credentials. If we could populate this token as an option it would mean not having to create a secured file

credentials "app.terraform.io" {
token = "xxxxxx.atlasv1.zzzzzzzzzzzzz"
}

https://www.terraform.io/docs/cli/config/config-file.html#credentials-1

Similar to this
https://github.com/hashicorp/setup-terraform/blob/9771d0c9cdc9b0d88429a5ed16417f3a0362fbde/lib/setup-terraform.js#L80

Ability to set azure cloud environment in terraform init

I would need the possibilty to set the azure cloud environment for example to azure china cloud, so the task can create the storage account for the tf state files there.

az cloud set --name AzureChinaCloud

would be the necessary step before the az login that happens before the resources for the remote backend are created

Make backend configuration optional when using backend type `azurerm`

Given I have the following terraform template

terraform {
  backend "azurerm" {
    resource_group_name  = "rg-name"
    storage_account_name = "stwhatever"
    container_name       = "terraformstate"
    key                  = "some.terraform.tfstate"
  }
}

I would like to be able to execute terraform init with a service connection but, no backend configuration since its already provided in the template

- task: TerraformCLI@0
  inputs:
    command: 'init'
    backendType: 'azurerm'
    backendServiceArm: 'my-service-connection'

** Note; In this case, ensureBackend may not be possible given it needs the storage sku and location to create properly. The user would need to provide the SKU and Location for ensureBackend to work.

Replaces charleszipp/azure-pipelines-tasks-terraform-old#253

Invalid OSSKU

Describe the bug
Cannot create a WINDOWS node pool using Terraform. Can create using Portal GUI or az-cli using same parameters.

To Reproduce
Steps to reproduce the behavior:

  1. Setup pipeline as (include yaml configuration or screenshots of classic ui editor)

resource "azurerm_kubernetes_cluster_node_pool" "winxxx" {
availability_zones = [1, 2, 3]
enable_auto_scaling = true
kubernetes_cluster_id = azurerm_kubernetes_cluster.aks_cluster.id
max_count = 3
min_count = 1
mode = "User"
name = "win01"
orchestrator_version = data.azurerm_kubernetes_service_versions.current.latest_version
os_disk_size_gb = 128
os_type = "Windows" # Default is Linux, we can change to Windows
vm_size = "Standard_D2s_v3"
priority = "Regular" # Default is Regular, we can change to Spot with additional settings like eviction_policy, spot_max_price, node_labels and node_taints
node_labels = {
"nodepool-type" = "user"
"environment" = var.environment
"nodepoolos" = "windows"
"app" = "dotnet-apps"
}
tags = {
"nodepool-type" = "user"
"environment" = var.environment
"nodepoolos" = "windows"
"app" = "dotnet-apps"
}
}

  1. Execute pipeline

  2. See error

/usr/local/bin/terraform apply -auto-approve /home/vsts/work/1/terraform-manifests-out/test-27.out
azurerm_kubernetes_cluster_node_pool.winxxx: Creating...
Error: creating/updating Managed Kubernetes Cluster Node Pool "win01" (Resource Group "terraform-aks-test"): containerservice.AgentPoolsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="InvalidOSSKU" Message="OSSKU='Ubuntu' is invalid, details: Windows does not allow OSSKU selection"
│ with azurerm_kubernetes_cluster_node_pool.winxxx,
│ on 10-aks-cluster-windows-user-nodepools.tf line 3, in resource "azurerm_kubernetes_cluster_node_pool" "winxxx":
│ 3: resource "azurerm_kubernetes_cluster_node_pool" "winxxx" {
##[error]Terraform command 'apply' failed with exit code '1'.
##[error]╷
│ Error: creating/updating Managed Kubernetes Cluster Node Pool "win01" (Resource Group "terraform-aks-test"): containerservice.AgentPoolsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="InvalidOSSKU" Message="OSSKU='Ubuntu' is invalid, details: Windows does not allow OSSKU selection"

│ with azurerm_kubernetes_cluster_node_pool.winxxx,
│ on 10-aks-cluster-windows-user-nodepools.tf line 3, in resource "azurerm_kubernetes_cluster_node_pool" "winxxx":
│ 3: resource "azurerm_kubernetes_cluster_node_pool" "winxxx" {

Finishing: Terraform Apply

Expected behavior
A clear and concise description of what you expected to happen.
should have created a windows node pool with 1 node

Screenshots
If applicable, add screenshots to help explain your problem.

Pipeline Logs
Include logs that help demonstrate the problem. Please make sure to redact any sensitive info such as secrets.

Agent Configuration

  • OS: ubuntu
  • Hosted in Azure DevOps
  • Terraform version used (Default for hosted agent is acceptable). 1.0.6
  • AzureCLI version used (Default for hosted agent is acceptable or N/A)

Additional context
Add any other context about the problem here.
this is a class example: from https://www.udemy.com/course/azure-kubernetes-service-with-azure-devops-and-terraform/learn/lecture/23644074 at 4:28

the class was record some time after 11/4/20 so this appears to have been functional in an
earlier version of the plugin OR an earlier version of Terraform

azurerm.log

Variables created by Terraform Output not working in subsequent tasks.

Describe the bug
It seems that as of this morning the pipelines using variables created from the Terraform Output task are not being passed the value assigned to them into subsequent steps. Our last successful deployment was approx. 9 hours ago which was prior to the 0.6.24 release.

To Reproduce
Steps to reproduce the behavior:

  1. Setup pipeline as (include yaml configuration or screenshots of classic ui editor)
    The first template has the Terraform Output task and the second template takes in a variable created by the Terraform Output task, in this case TF_OUT_BEDROCK_KEY_VAULT_NAME
    - template: templates/deploy-infrastructure.yaml@commonPipeline
    parameters:
    Action: deploy
    AzureServiceConnection: ${{parameters.AzureServiceConnection}}
    AzureRegion: ${{parameters.AzureRegion}}
    SubscriptionId: ${{parameters.SubscriptionId}}
    Environment: ${{parameters.Environment}}
    EnvironmentTier: ${{parameters.EnvironmentTier}}
    OrganizationalPrefix: ${{parameters.OrganizationalPrefix}}
    TerraformBackingStoreResourceGroupName: rg-terraform-${{parameters.Environment}}
    TerraformBackingStoreResourceGroupLocation: $(bootstrapAzureRegion)
    TerraformBackingStoreStorageAccountName: $(ArmOutputs.tfStateSAName)
    TerraformBackingStoreContainerName: ${{parameters.TerraformBackingStoreContainerName}}
    TerraformBackingStoreStateFileName: 'vantage-insight/terraform.tfstate'
    WorkingPath: $(Agent.BuildDirectory)/self
    InfrastructureRelativePath: iac/insight
    - template: templates/retrieve-powerplatformpipelinecontext.yaml@commonPipeline
    parameters:
    AzureServiceConnection: ${{parameters.AzureServiceConnection}}
    KeyVaultName: $(TF_OUT_BEDROCK_KEY_VAULT_NAME)
    ClientIdSecretName: 'PowerPlatform--ClientId'
    ClientSecretSecretName: 'PowerPlatform--ClientSecret'
    ClientIdPipelineVariableName: 'POWER_PLATFORM_AZDO_CLIENT_ID'
    ClientSecretPipelineVariableName: 'POWER_PLATFORM_AZDO_CLIENT_SECRET'
    ScriptWorkingPath: $(Agent.BuildDirectory)/commonPipeline
    The output variable is passed into another
  2. Execute pipeline
  3. See error

Expected behavior
The expected behavior would have been that the pipelines would have continued to work after the update.

Screenshots
If applicable, add screenshots to help explain your problem.
After Terraform Output task:
image

Terraform Output variable passed into powershell script:
image

Last successful deployment was 9 hours ago, right before the 0.6.24 release:
image

Pipeline Logs
Include logs that help demonstrate the problem. Please make sure to redact any sensitive info such as secrets.

2021-05-25T19:34:08.4185095Z ##[command]Clear-AzContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue
2021-05-25T19:34:09.4957143Z ##[command]Clear-AzContext -Scope Process
2021-05-25T19:34:10.3671957Z ##[command]Connect-AzAccount -ServicePrincipal -Tenant *** -Credential System.Management.Automation.PSCredential -Environment AzureCloud @processScope
2021-05-25T19:34:12.7576259Z ##[command] Set-AzContext -SubscriptionId ************ -TenantId ***
2021-05-25T19:34:13.8896322Z VERBOSE: Leaving Initialize-AzModule.
2021-05-25T19:34:16.6017284Z ##[debug]Error record:
2021-05-25T19:34:16.6623742Z ##[debug]TF_OUT_BEDROCK_KEY_VAULT_NAME : The term 'TF_OUT_BEDROCK_KEY_VAULT_NAME' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
2021-05-25T19:34:16.6637643Z ##[debug]At D:\a_temp\e531ed91-0558-4dd1-8c95-52d2dccbf8b8.ps1:37 char:91
2021-05-25T19:34:16.6652120Z ##[debug]+ ... eVariable.ps1' -KeyVaultName "$(TF_OUT_BEDROCK_KEY_VAULT_NAME)" -KeyV ...
2021-05-25T19:34:16.6666142Z ##[debug]+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-05-25T19:34:16.6679787Z ##[debug] + CategoryInfo : ObjectNotFound: (TF_OUT_BEDROCK_KEY_VAULT_NAME:String) [], ParentContainsErrorRecordExce ption
2021-05-25T19:34:16.6696055Z ##[debug] + FullyQualifiedErrorId : CommandNotFoundException
2021-05-25T19:34:16.6710445Z ##[debug]
2021-05-25T19:34:16.6732933Z ##[debug]Script stack trace:
2021-05-25T19:34:16.6782038Z ##[debug]at , D:\a_temp\e531ed91-0558-4dd1-8c95-52d2dccbf8b8.ps1: line 37
2021-05-25T19:34:16.6795742Z ##[debug]at , : line 1
2021-05-25T19:34:16.6817052Z ##[debug]Exception:
2021-05-25T19:34:16.6863659Z ##[debug]System.Management.Automation.ParentContainsErrorRecordException: The term 'TF_OUT_BEDROCK_KEY_VAULT_NAME' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
2021-05-25T19:34:16.7055356Z ##[error]The term 'TF_OUT_BEDROCK_KEY_VAULT_NAME' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
2021-05-25T19:34:16.7057151Z ##[debug]Processed: ##vso[task.logissue type=error]The term 'TF_OUT_BEDROCK_KEY_VAULT_NAME' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
2021-05-25T19:34:16.7737768Z ##[debug]Exit code: 1
2021-05-25T19:34:16.7779954Z ##[debug]Leaving Invoke-VstsTool.

Agent Configuration
Agent name: 'Hosted Agent'
Current agent version: '2.186.1'
Operating System Microsoft Windows Server 2019
Terraform v0.13.3
AzureCLI 2.1.0

Additional context
It seems the problem started occurring after the 0.6.24 release of pipeline tasks

`terraform import` is running `terraform validate` instead

Hi guys, I'm really struggling to get this thing to import my existing resources.

The step I'm trying to run in my azure-pipelines.yml file:

  - task: TerraformCLI@0
    displayName: 'terraform import dev'
    inputs:
        workingDirectory: $(Build.SourcesDirectory)/infra
        environmentServiceName: 'MyServiceName'
        command: import azurerm_resource_group.myrg "/subscriptions/000-...-0000/resourceGroups/MyRG"

What I expect (i.e. same as what I get on my desktop):

azurerm_resource_group.myrg: Importing from ID "/subscriptions/000-...-0000/resourceGroups/MyRG"...
azurerm_resource_group.wqi-rg: Import prepared!
  Prepared azurerm_resource_group for import
azurerm_resource_group.wqi-rg: Refreshing state... [id=/subscriptions/000-...-0000/resourceGroups/MyRG]
(import process continues)

I get instead:

2021-06-03T03:36:21.2532013Z ##[section]Starting: terraform import dev
2021-06-03T03:36:21.2918807Z ==============================================================================
2021-06-03T03:36:21.2919232Z Task         : Terraform CLI
2021-06-03T03:36:21.2920516Z Description  : Execute terraform cli commands
2021-06-03T03:36:21.2920986Z Version      : 0.6.25
2021-06-03T03:36:21.2921271Z Author       : Charles Zipp
2021-06-03T03:36:21.2921548Z Help         : 
2021-06-03T03:36:21.2921846Z ==============================================================================
2021-06-03T03:36:22.8179145Z [command]C:\hostedtoolcache\windows\terraform\0.15.5\x64\terraform.exe version
2021-06-03T03:36:23.2690240Z Terraform v0.15.5
2021-06-03T03:36:23.2691134Z on windows_amd64
2021-06-03T03:36:23.2692038Z + provider registry.terraform.io/hashicorp/azurerm v2.61.0
2021-06-03T03:36:23.2694151Z + provider registry.terraform.io/hashicorp/random v3.1.0
2021-06-03T03:36:23.2936349Z 
2021-06-03T03:36:23.3642853Z [command]C:\hostedtoolcache\windows\terraform\0.15.5\x64\terraform.exe validate
2021-06-03T03:36:28.9785074Z �[32m�[1mSuccess!�[0m The configuration is valid.
2021-06-03T03:36:28.9786073Z �[0m
2021-06-03T03:36:29.0407425Z 
2021-06-03T03:36:29.5076381Z ##[section]Finishing: terraform import dev

It's my terraform import step, but it's running terraform validate instead!

Edit: Am running on public cloud Azure, standard current version of everything. Complete pipeline below:

trigger:
- master

pool:
  vmImage: windows-latest

parameters:
- name: 'environments'
  type: object
  default: 
  - dev
  - tst
  - prd

steps:
- ${{ each env in parameters.environments }}:
  - task: TerraformInstaller@0
    displayName: install terraform
  - task: TerraformCLI@0
    displayName: 'Terraform set workspace to ${{ env }}'
    inputs:
      workingDirectory: $(Build.SourcesDirectory)/infra
      command: workspace
      workspaceSubCommand: new
      workspaceName: ${{ env }}
  - task: TerraformCLI@0
    displayName: 'terraform init ${{ env }}'
    inputs:
        workingDirectory: $(Build.SourcesDirectory)/infra
        command: init
        backendType: local
  - task: TerraformCLI@0
    displayName: 'terraform import ${{ env }}'
    inputs:
        workingDirectory: $(Build.SourcesDirectory)/infra
        ${{ if eq(parameters['env'], 'prd') }}:
          environmentServiceName: 'ServiceConnection (Prod)'
        ${{ if ne(parameters['env'], 'prd') }}:
          environmentServiceName: 'ServiceConnection (Non Prod)'
        ${{ if eq(parameters['env'], 'prd') }}:
          command: import azurerm_resource_group.myrg "redacted"
        ${{ if eq(parameters['env'], 'tst') }}:
          command: import azurerm_resource_group.myrg "redacted"
        ${{ if eq(parameters['env'], 'dev') }}:
          command: import azurerm_resource_group.myrg "redacted"

Thanks :)

Add support for azure resource manager service connections using managed identity

Hello,

I currently have a pipeline running fine with a SPN on a self hosted agent.

I would like to switch to a managed identity (VM)
I setup a new Service Connection in azure Devops, and got a simple powershell pipeline to work to validate the managed identity is fine.

Now with TerraformCLI@0, terraform init fails with this error :
##[error]Terraform backend initialization for AzureRM only support service principal authorization
(nothing more useful when I activate TF_LOG TRACE)

Does TerraformCLI@0 support managed identity ?

For info : I have also tried using MS's TerraformTaskV1@0
I get an error too, but different, where I think it is "badly" trying to authenticate will a null identifier instead of using MSI
(there is "-backend-config=arm_client_id=null -backend-config=arm_client_secret=null" in the command line....)

Failed to get existing workspaces: Error retrieving keys for Storage Account "xxx": azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.Storage/storageAccounts/xxx/listKeys?api-version=2016-01-01: StatusCode=400 -- Original Error: adal: Refresh request failed. Status Code = '400'. Response body: {"error":"unauthorized_client","error_description":"AADSTS700016: Application with identifier 'null' was not found in the directory '***'. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You may have sent your authentication request to the wrong tenant.\r\nTrace ID: 6274265d-18cf-4f75-b5c4-3fa164bc1b00\r\nCorrelation ID: d27e928c-76b4-4ad0-a4cc-be07b3480d6a\r\nTimestamp: 2021-05-19 23:09:07Z","error_codes":[700016],"timestamp":"2021-05-19 23:09:07Z","trace_id":"6274265d-18cf-4f75-b5c4-3fa164bc1b00","correlation_id":"d27e928c-76b4-4ad0-a4cc-be07b3480d6a","error_uri":"https://login.microsoftonline.com/error?code=700016"}�[0m

Any help or guidance would be appreciated, maybe I'm missing something obvious ?...

Regards,

Olivier Beau

`terraform import` doesn't support `runAzLogin: true`

Describe the bug
Running import without runAzLogin: true I get:

 Error: Error building AzureRM Client: obtain subscription(*azure sub id*) from Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1: ERROR: Please run 'az login' to setup account.

But running import with runAzLogin: true I get the same error.

To Reproduce
Try the below pipeline:

trigger:
- master
pool:
  vmImage: windows-latest
parameters:
- name: 'environments'
  type: object
  default: 
  - dev
  - tst
  - prd

steps:
- ${{ each env in parameters.environments }}:
  - task: TerraformInstaller@0
    displayName: install terraform
  - task: TerraformCLI@0
    displayName: 'Terraform set workspace to ${{ env }}'
    inputs:
      workingDirectory: $(Build.SourcesDirectory)/infra
      command: workspace
      workspaceSubCommand: new
      workspaceName: ${{ env }}
  - task: TerraformCLI@0
    displayName: 'terraform init ${{ env }}'
    inputs:
      workingDirectory: $(Build.SourcesDirectory)/infra
      command: init
      backendType: local
  - task: TerraformCLI@0
    displayName: 'terraform import ${{ env }}'
    inputs:
      command: import
      workingDirectory: $(Build.SourcesDirectory)/infra
      resourceAddress: azurerm_resource_group.myrg
      resourceId: "/subscriptions/*redacted*/resourceGroups/myrg"
      runAzLogin: true

Expected behavior
I want to be able to import my resource groups, like I can on desktop.

Pipeline Logs

C:\hostedtoolcache\windows\terraform\0.15.5\x64\terraform.exe version
Terraform v0.15.5
on windows_amd64
+ provider registry.terraform.io/hashicorp/azurerm v2.61.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
C:\hostedtoolcache\windows\terraform\0.15.5\x64\terraform.exe import azurerm_resource_group.myrg /subscriptions/*redacted*/resourceGroups/myrg

│ Error: Error building AzureRM Client: obtain subscription(*redacted*) from Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1: ERROR: Please run 'az login' to setup account.
│   with provider["registry.terraform.io/hashicorp/azurerm"],
│   on D:\a\1\s\infra\main.tf line 67, in provider "azurerm":
│   67: provider "azurerm" {

Finishing: terraform import dev

Agent Configuration
All default versions, Azure hosted

Improvement - more decisive warnings about added, updated or deleted resources

Due to the reconstruction of the repository I cannot see whether a reason existed as to why always all 3 lines are been displayed rather than only those >0.

https://github.com/charleszipp/azure-pipelines-tasks-terraform/blob/05b63e3915bacfe4f0ed39781bf6a68bc2ea43d9/tasks/terraform-cli/src/commands/tf-plan.ts#L82-L86

I suggest these 2 Improvements:

  1. only output those >0:
private planSummaryReport(toAdd: string, toUpdate: string, toDestroy: string, planName: string) { 
     if (toAdd >0 )      this.logger.warning(`Plan '${planName}' is going to create ${toAdd} resources.`) 
     if (toUpdate>0 )  this.logger.warning(`Plan '${planName}' is going to update ${toUpdate} resources.`) 
     if (toDestroy>0 ) this.logger.warning(`Plan '${planName}' is going to destroy ${toDestroy} resources.`) 
 } 
  1. Also it is only checked how many resources change, I would like to also see, whether No changes do exist.
private planSummaryReport(toAdd: string, toUpdate: string, toDestroy: string, planName: string, noChanges: boolean) { 
     ...
     if (noChanges) this.logger.warning(`Plan '${planName}' shows the infrastructure is up to date and has no changes.`) 
}

TerraformCLI does not support 'fmt' command as documented

Describe the bug
TerraformCLI task does not support 'fmt' command as documented. I manually installed the extension for Azure DevOps server but can confirm on the Microsoft hosted version also does not have the 'fmt' option either.

To Reproduce
Steps to reproduce the behavior:

  1. Setup pipeline as YAML
  2. Add a Task (via the assistant)
  3. Select TerraformCLI
  4. 'fmt' is not in the list (also manually writing the task in YAML, fmt is highlighted as an unsupported value)

Expected behavior
To be able to use the 'fmt' command to lint Terraform configuration files.

Screenshots
If applicable, add screenshots to help explain your problem.

Pipeline Logs
Include logs that help demonstrate the problem. Please make sure to redact any sensitive info such as secrets.

Agent Configuration

  • OS: [e.g. ubuntu debian]
  • Hosted/Self Hosted
  • Terraform version used (Default for hosted agent is acceptable)
  • AzureCLI version used (Default for hosted agent is acceptable or N/A)

Additional context
Add any other context about the problem here.

Simplify using different service connections for azurerm backend and providers

Description

When the azure storage account backend and azurerm provider use different service connections, users have to do one of the following:

  1. Ensure the identity (i.e. service principal) backing the service connection has access to both the backend storage and the subscription targeted by the azurerm subscription.
  2. Set ARM_ACCESS_KEY environment variable when executing commands that use the azurerm provider and backend such as plan, apply, etc.

This is reported to occur when the azurerm backend and azurerm provider target two different subscriptions.

Ask

Execute init (azurerm backend) and azurerm provider commands (plan/apply/destroy etc) using different service connections WITHOUT having to

  1. grant the service connection identity used by the azurerm provider access to the azurerm backend storage account
  2. provide azurerm backend storage account access key (when executing azurerm provider commands such as plan/apply/destroy)

Approaches Considered

Store Access Key in State

If the access key is set via -backend-config=access-key=<my-key> at the time terraform init is run, it will be stored in the local terraform.tfstate file on the agent. When subsequent azurerm provider commands are run, terraform will use this key to access the backend storage account (as opposed to service principal or managed identity).

Since the storage account could be new, the key would most likely need to be looked up at the time the TerraformCLI task executes init. Alternative would be to allow the user to configure the access key.

Order Plans by name in ADO UI

Any chance the plans dropdown could be ordered?

We can have multiple plans of the same resource with a stage number at the end and this would make it much easier too find the correct one.

EG

Plan-StageA-1
Plan-StageA-2
Plan-StageB-1

I think it could be done here but not worked with pipeline UI's before so not 100%!

https://github.com/charleszipp/azure-pipelines-tasks-terraform/blob/05b63e3915bacfe4f0ed39781bf6a68bc2ea43d9/views/terraform-plan/src/plan-summary-tab/plan-summary-tab.tsx#L43

Terraform Output needs a few additional features

Current state

  • terraform output only outputs string, number and bool
  • it uses -json by default
  • it create TF_OUT_* variables
  • even though -json is been used, the outputed variables are in raw format
  • no logging to the console

Issues:

  • lists, objects, etc. will not be returned
  • no warning if a certain object is not been returned
  • if -raw is been used in the commandOptions this will cause an error
  • as state above, at first glance terraform output does not return anything - as it is only silently put into the variables
  • the actual -json cannot be returned

Tasks:

  • add a warning if something is not outputted
  • add logging to the console of the pipeline, what was outputted (somthing like TF_OUT_SOMETHING=something )
  • include a specific outputtype argument [ raw | json ] and outputkey if a specific key is to be returned - at first only raw
  • extend the functionality to actually output json-format just as terraform does it and extend the outputtype to allow json
  • extend the functionality to actually return all kind of output

Don't know whether this should be logged in separate issues - thought this one could serve as a parent issue.
Also this is a rather extensive change, hence the priority above, in which I would do the changes

Edit: "give a warning if commandOptions is been used and deprecate commandOptions" this is still required for no-color or to choose a specific state file

Pull shared organizations from variable group

The list of organizations to share after publishing new pre-release versions should be retrieved from a variable group, so that those organizations are not viewable by the public.

Currently, these are hard-coded within the pipeline yaml. This should come from a pipeline variable instead.

Microsoft upstream?

How is this repo different from the microsoft/azure-pipelines-extensions version of the TF extension?

Can't seem to find any docs on this.

Thank you!

Var-file causes errors for terraform validate after 0.15.0

According to the 0.15.0 release notes -var-file is no longer valid for terraform validate. After 0.12 this option was ignored but did not cause an error. In 0.15, providing this causes the following error.

2021-04-15T16:26:21.3455555Z ##[section]Starting: terraform validate
2021-04-15T16:26:21.3466179Z ==============================================================================
2021-04-15T16:26:21.3466829Z Task         : Terraform CLI
2021-04-15T16:26:21.3467243Z Description  : Execute terraform cli commands
2021-04-15T16:26:21.3467535Z Version      : 0.6.20
2021-04-15T16:26:21.3467906Z Author       : Charles Zipp
2021-04-15T16:26:21.3468291Z Help         : 
2021-04-15T16:26:21.3468760Z ==============================================================================
2021-04-15T16:26:21.8462145Z [command]/opt/hostedtoolcache/terraform/0.15.0/x64/terraform version
2021-04-15T16:26:21.9253003Z Terraform v0.15.0
2021-04-15T16:26:21.9253759Z on linux_amd64
2021-04-15T16:26:21.9254313Z + provider registry.terraform.io/hashicorp/azurerm v2.55.0
2021-04-15T16:26:21.9314643Z 
2021-04-15T16:26:22.5203959Z [command]/opt/hostedtoolcache/terraform/0.15.0/x64/terraform validate -var-file=/home/vsts/work/_temp/default.vars
2021-04-15T16:26:22.5772067Z �[31m╷�[0m�[0m
2021-04-15T16:26:22.5773901Z �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mFailed to parse command-line flags�[0m
2021-04-15T16:26:22.5775355Z �[31m│�[0m �[0m
2021-04-15T16:26:22.5776511Z �[31m│�[0m �[0m�[0mflag provided but not defined: -var-file
2021-04-15T16:26:22.5777777Z �[31m╵�[0m�[0m
2021-04-15T16:26:22.5778238Z 
2021-04-15T16:26:22.5779043Z For more help on using this command, run:
2021-04-15T16:26:22.5779999Z   terraform validate -help
2021-04-15T16:26:22.5875811Z 
2021-04-15T16:26:22.5917453Z ##[error]Terraform command 'validate' failed with exit code '1'.
2021-04-15T16:26:22.5935703Z ##[error]�[31m╷�[0m�[0m
�[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mFailed to parse command-line flags�[0m
�[31m│�[0m �[0m
�[31m│�[0m �[0m�[0mflag provided but not defined: -var-file
�[31m╵�[0m�[0m

For more help on using this command, run:
  terraform validate -help

2021-04-15T16:26:23.2944148Z ##[section]Finishing: terraform validate

Workaround is to remove the secureVarsFile input from the terraform validate command. Also ensure -var-file is not provided in the commandOptions input.

Support Workspace Commands

As a release engineer, I would like to be able to create and select a terraform workspace, so that I can test features in isolated infrastructure.

Supported Sub-Commands

Workspace commands will be added in the following order

  • select
  • new
  • new succeeds when workspace exists with toggle to fail if exists (backwards compatibility)
  • show
  • delete
  • list

Select is being used first so those with existing workspaces can use this immediately after that work is completed.

Expected YAML Syntax

The following is the yaml expected to use workspaces. Final syntax will be added to the Overview doc paired with examples in pipelines/test directory.

Workspace Select

- task: TerraformCLI@0
  displayName: 'terraform workspace select'
  inputs:
    command: workspace
    workspaceName: foo
    workspaceSubCommand: select
    workingDirectory: $(terraform_templates_dir)

selecting the latest version of this terraform task

Hi everyone,

I am trying to use the 0.6.23 version for Terraform Init (0.15 upgrade), but somehow I can't find where to specify the task version.
The changelog shows 0.6.23 as the latest, as well as the MS Extension Marketplace, but my agent is using 0.6.8.
I am using the current syntax:

  • task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-cli.TerraformCLI@0
    displayName: 'terraform init'
    inputs:
    ...

I have also tried with:

  • charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-cli.TerraformCLI@0.6.23
  • [email protected]
  • TerraformCLI@0

Can someone please point out how to use the 0.6.23 version?
image

ailed to load "./terraform.tfplan" as a plan file

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Setup pipeline as (include yaml configuration or screenshots of classic ui editor)
  2. Execute pipeline
  3. See error

Expected behavior
Run Terraform plan

variables: 
- name: WORKDIR
  value: $(System.DefaultWorkingDirectory)/infra/unit_env
stages:
- stage: TERRAFORM 
  jobs: 
  - job: Init_and_plan
    steps:
    - task: TerraformInstaller@0
      inputs:
        terraformVersion: 'latest'

    - task: TerraformCLI@0
      inputs:
        command: 'init'
        workingDirectory: '$(WORKDIR)'
        backendType: 'azurerm'
        backendServiceArm: 'PLACEHOLDER'
        ensureBackend: true
        backendAzureRmResourceGroupName: 'PLACEHOLDER'
        backendAzureRmResourceGroupLocation: 'westeurope'
        backendAzureRmStorageAccountName: 'PLACEHOLDER'
        backendAzureRmContainerName: 'terraform'
        backendAzureRmKey: 'PLACEHOLDER'

    - task: TerraformCLI@0
      inputs:
        command: 'plan'
        workingDirectory: '$(WORKDIR)'
        environmentServiceName: 'PLACEHOLDER'
        commandOptions: '--var-file ./variables.tfvars --out ./terraform.tfplan -detailed-exitcode'
        publishPlanResults: 'terraform.tfplan'
    
    - bash: |
        echo $TERRAFORM_PLAN_HAS_CHANGES
        echo "##vso[task.setvariable variable=TERRAFORM_PLAN_HAS_CHANGES;isOutput=true;]$TERRAFORM_PLAN_HAS_CHANGES"
        echo "variable value is $(TERRAFORM_PLAN_HAS_CHANGES)"
      name: setVar

    - publish: $(WORKDIR) # Publish complete working directory. https://learn.hashicorp.com/tutorials/terraform/automate-terraform?in=terraform/automation&utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#plan-and-apply-on-different-machines
      artifact: terraform


    - task: TerraformCLI@0
      inputs:
        command: 'apply'
        workingDirectory: $(WORKDIR)
        environmentServiceName: 'PLACEHOLDER'
        commandOptions: '"./terraform.tfplan"'
        allowTelemetryCollection: false

Screenshots

image

Pipeline Logs
Include logs that help demonstrate the problem. Please make sure to redact any sensitive info such as secrets.

Agent Configuration

  • OS: [e.g. ubuntu debian]
  • Hosted/Self Hosted
  • Terraform version used (Default for hosted agent is acceptable)
  • AzureCLI version used (Default for hosted agent is acceptable or N/A)

Additional context
Add any other context about the problem here.

TerraformCLI - "show" does not work for Azure ApiManagement service

Describe the bug
TerraformCLI task with command "show" for a tfPlan file generates an error parsing JSON contents.

To Reproduce
Steps to reproduce the behavior:

  1. Setup pipeline as (include yaml configuration or screenshots of classic ui editor)
- task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-cli.TerraformCLI@0
        displayName: Terraform plan
        inputs:
          command: plan
          commandOptions: '-detailed-exitcode -out=$(Pipeline.Workspace)/tfplan -input=false ${{ parameters.tfSecrets }}'
          workingDirectory: ${{ parameters.workingDirectory }}
          environmentServiceName: ${{ parameters.backendServicePrincipal }}
          publishPlanResults: ${{ parameters.artifactTfPlanName }}

- task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-cli.TerraformCLI@0
        displayName: Terraform show
        inputs:
          command: show
          environmentServiceName: ${{ parameters.backendServicePrincipal }}
          inputTargetPlanOrStateFilePath: $(Pipeline.Workspace)/tfplan
  1. Execute pipeline
  2. See error
    ##[error]SyntaxError: Unexpected end of JSON input

Expected behavior
TF show finishes and TERRAFORM_PLAN_HAS_DESTROY_CHANGES is set to "true"

Agent Configuration

  • OS: ubuntu-20.04
  • Hosted
  • Terraform version used: 1.0.5
  • AzureCLI version used: default
  • Azurerm provider version used: 2.73.0

Additional context
Azure ApiManagement contains policies, that are xml contents. When "show" command is used, it creates blocks like this:
`"xml_content": "\u003cpolicies\u003e\r\n\t\u003cinbound\u003e\r\n\t\t\u003cquota-by-key calls="40" renewal-period="3600" counter-key="@(context.Request.IpAddress)" ...",``
The regex used in tfShow task does not manage tabs and escaped double quotes, so JSON.parse throws an error.

A PR is linked with a test that reproduces the issue and a change at show task that fixes this problem #155

Expose path or url to publishPlanResult files

The content of the publishPlanResult file is useful in determining what resources are actually going to be changed. In some cases, users would like access to this file to parse the content and emit warnings for changes to critical infrastructure. Using this file for this purpose over the JSON output via terraform show is favored due to the fact it redacts sensitive values. The JSON output from show does not.

Important: This should also consider that multiple publishPlanResult files could be produced within a single pipeline. The solution should provide access to all published plans.

cc @DenWin Replacing #34 with this issue to focus on providing access to publishPlanResult files

Double quotes escaping problem on Windows agents

Describe the bug
Problem occurs when I want to call apply command with -replace <address> command options.
Terraform running on Windows need special escaping to be handled (https://www.terraform.io/docs/cli/commands/state/rm.html#example-remove-a-particular-instance-of-a-resource-using-for_each).

To Reproduce
Steps to reproduce the behavior:

  1. Setup pipeline as (include yaml configuration or screenshots of classic ui editor)
- task: TerraformCLI@0
  inputs:
    command: apply
    commandOptions: -auto-approve -replace module.frontdoor.null_resource.https_configuration[\"demo\"]
    workingDirectory: ${{ parameters.directory }}
    environmentServiceName: ${{ parameters.subscription }}
    runAzLogin: true
  1. Execute pipeline
  2. See error

Agent Configuration

  • OS: windows2019
  • Hosted
  • Terraform version: default
  • AzureCLI version: default

Warning issued on 'TerraformCLI@0' task running 'plan' that has no changes

Describe the bug
Since updating to extension version 0.6.24, I am seeing a warning issued after an otherwise successful terraform plan task that does not have any changes. This leads to following tasks failing (picking up the warning).

To Reproduce
When on extension version 0.6.24:

  • run a TerraformCLI@0 task to plan that does not produce any changes (infrastructure is up-to-date)
  • Example task (using detailed-exitcode):
- task: TerraformCLI@0
  continueOnError: true
  inputs:
    command: 'plan'
    commandOptions: '-out=tfplan.bin -input=false -detailed-exitcode'
    workingDirectory: '$(tfModulePath)'
    environmentServiceName: '${{ parameters.devopsServiceConnection }}'
    publishPlanResults: '${{ parameters.devopsArtifact }}'
  env:
    TF_IN_AUTOMATION: 'True'
    TF_CLI_ARGS_plan: '$(TF_CLI_ARGS_plan)'
    tenant_id: '$(tenant_id)'
    subscription_id: '$(subscription_id)'
    client_id: '$(client_id)'
    client_secret: '$(client_secret)'
  name: 'tfPlanTargeted'
  displayName: 'tf - plan'

Expected behavior
Task should not return with a warning if infrastructure is up-to-date

Screenshots
image

Agent Configuration

  • ubuntu-latest
  • hosted
  • 0.14.11 (we are not currently able to use 0.15,x for reasons not related this extension)
  • default

Unable to retrieve latest version in a Self Hosted Agent

I am bumping into a FetchError while installing terraform in my self hosted agent in corporate network.
I'm pretty sure this is related to proxy not being used, even though proxy variables are there.

Any ideas ? or workarounds ?

2021-03-29T10:25:30.6058008Z ==============================================================================
2021-03-29T10:25:30.6058388Z Task : Terraform Installer
2021-03-29T10:25:30.6058699Z Description : Installs a specific version of terraform
2021-03-29T10:25:30.6058987Z Version : 0.6.15
2021-03-29T10:25:30.6059222Z Author : Charles Zipp
2021-03-29T10:25:30.6059481Z Help :
2021-03-29T10:25:30.6059860Z ==============================================================================
2021-03-29T10:25:30.6146911Z ##[debug]Using node path: C:\agent\externals\node\bin\node.exe
2021-03-29T10:25:30.9211053Z ##[debug]agent.TempDirectory=C:\agent_work_temp
2021-03-29T10:25:30.9238318Z ##[debug]loading inputs and endpoints
2021-03-29T10:25:30.9243179Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2021-03-29T10:25:30.9254471Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2021-03-29T10:25:30.9261340Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2021-03-29T10:25:30.9266731Z ##[debug]loading INPUT_TERRAFORMVERSION
2021-03-29T10:25:30.9273647Z ##[debug]loading SECRET_AD_JOIN_PASSWD
2021-03-29T10:25:30.9278281Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
2021-03-29T10:25:30.9285190Z ##[debug]loaded 6
2021-03-29T10:25:30.9291757Z ##[debug]Agent.ProxyUrl=http://my.proxy:8080
2021-03-29T10:25:30.9298195Z ##[debug]Agent.ProxyUsername=undefined
2021-03-29T10:25:30.9298919Z ##[debug]Agent.ProxyPassword=undefined
2021-03-29T10:25:30.9299532Z ##[debug]Agent.ProxyBypassList=undefined
2021-03-29T10:25:30.9300238Z ##[debug]expose agent proxy configuration.
2021-03-29T10:25:30.9301211Z ##[debug]Agent.CAInfo=undefined
2021-03-29T10:25:30.9301868Z ##[debug]Agent.ClientCert=undefined
2021-03-29T10:25:30.9302484Z ##[debug]Agent.SkipCertValidation=undefined
2021-03-29T10:25:31.0114785Z ##[debug]Agent.ProxyUrl=http://my.proxy:8080
2021-03-29T10:25:31.0115980Z ##[debug]Agent.ProxyUsername=undefined
2021-03-29T10:25:31.0116703Z ##[debug]Agent.ProxyPassword=undefined
2021-03-29T10:25:31.0117370Z ##[debug]Agent.ProxyBypassList=undefined
2021-03-29T10:25:31.0118012Z ##[debug]Agent.CAInfo=undefined
2021-03-29T10:25:31.0118644Z ##[debug]Agent.ClientCert=undefined
2021-03-29T10:25:31.0119507Z ##[debug]check path : C:\agent_work_tasks\TerraformInstaller_11645770-d18e-11e8-8f5b-1b8b62612b3b\0.6.15\node_modules\azure-pipelines-tool-lib\lib.json
2021-03-29T10:25:31.0120707Z ##[debug]adding resource file: C:\agent_work_tasks\TerraformInstaller_11645770-d18e-11e8-8f5b-1b8b62612b3b\0.6.15\node_modules\azure-pipelines-tool-lib\lib.json
2021-03-29T10:25:31.0121576Z ##[debug]system.culture=en-US
2021-03-29T10:25:31.0232500Z ##[debug]terraformVersion=latest
2021-03-29T10:25:31.0233433Z ##[debug]downloadUrl=undefined
2021-03-29T10:25:31.0252069Z Getting latest version
2021-03-29T10:25:52.0833612Z ##[debug]task result: Failed
2021-03-29T10:25:52.0865171Z ##[error]Error: Unable to retrieve latest version: FetchError: request to https://checkpoint-api.hashicorp.com/v1/check/terraform failed, reason: connect ETIMEDOUT 172.64.204.26:443

Improve docs: publishPlanResults should not be the name of a subfolder of workingDirectory

Describe the bug
When publishPlanResults is the name of a subfolder of workingDirectory, say 'SomeName', the task will fail with error:

##[error]Error: EISDIR: illegal operation on a directory, open '/home/vsts/work/1/s/iac/terraform/SomeName'

To Reproduce
Steps to reproduce the behavior:

  1. Set publishPlanResults as the same name of some subfolder of where you will run the terraform task.
  2. Execute pipeline
  3. See error

Expected behavior
At a minimum, docs contain a note about not using a subfolder name as the publishPlanResults name.
Ideally, the error is caught and proper guidance is provided to the user, or the task can work around this limitation.

Agent Configuration

  • Hosted
  • Terraform v0.15.3 on linux_amd64
  • Task version used 0.6.23

Additional context

Missing Terraform init -reconfigure implementation

Describe the bug
As a Terraform user coming from v. 0.14.5, I want to upgrade to the latest official release which is v. 1.0.0.
Because I am coming from a version before v. 0.15.0, I need to run the Terraform init command with the "-reconfigure" flag as specified by the Terraform migration page (last item on the page).
When I update my Azure DevOps pipeline to contain this command, I get the following error:

Task         : Terraform CLI
Description  : Execute terraform cli commands
Version      : 0.6.8
Author       : Charles Zipp
Help         : 
==============================================================================
##[error]Error: Support for command "init -reconfigure" is not implemented

To Reproduce
Steps to reproduce the behavior:

  1. Setup Terraform Init DevOps task
- task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-cli.TerraformCLI@0
      displayName: 'terraform init'
      inputs:
        command: init -reconfigure
        workingDirectory: '$(System.DefaultWorkingDirectory)/${{ parameters.artifact_name }}/${{ parameters.tf_folder }}'
        backendType: azurerm
        backendServiceArm: '${{ parameters.service_connection }}'
        ensureBackend: true
        backendAzureRmResourceGroupName: '$(TF_BACKEND_RG)'
        backendAzureRmResourceGroupLocation: '$(TF_BACKEND_LOCATION)'
        backendAzureRmStorageAccountName: '$(TF_BACKEND_STORAGEACCOUNT)'
        backendAzureRmContainerName: '${{ parameters.tf_container }}'
        backendAzureRmKey: '$(TF_BACKEND_KEY)'
  1. Execute pipeline
  2. See error
Task         : Terraform CLI
Description  : Execute terraform cli commands
Version      : 0.6.8
Author       : Charles Zipp
Help         : 
==============================================================================
##[error]Error: Support for command "init -reconfigure" is not implemented

Expected behavior
Terraform runs the init command and does the reconfiguration as the documentation mentions and task succeeds.

Agent Configuration

  • OS: Ubuntu-latest
  • Terraform version used v. 1.0.0
  • AzureCLI version used (Default for hosted agent is acceptable or N/A)

Question: Is there a way to publish a custom plan to the Terraform Plan View (TFC)?

If using Terraform Cloud or Scalr with remote execution, terraform cli does not support exporting of the plan using the -out parameter. Unfortunately, this means that the Terraform plan task cannot publish the plans and be viewed in the Terraform Plan View (tab).

Is there a way to upload/attach this manually after the Terraform plan task?

Sample:

  1. Terraform init
  2. Terraform plan (without publishing of plan: "publishPlanResults")
  3. Use TFC or Scalr REST API to download the json plan
  4. Upload/Attach the plan

Terraform environment variable TERRAFORM_PLAN_HAS_CHANGES is not created when terraform plan is executed

Describe the bug
Starting with version 0.6.23 environment variable TERRAFORM_PLAN_HAS_CHANGES is not created and populated with false or true.

This block will succeed:

    - task: [email protected]
      displayName: "Terraform Plan"
      inputs:
        command: 'plan'
        workingDirectory: '$(System.DefaultWorkingDirectory)/${{parameters.configuration_folder}}'
        environmentServiceName: '${{variables.subscription_name}}'
        commandOptions: '--var-file=vars_env_${{parameters.subscription_code}}.tfvars -out=$(System.DefaultWorkingDirectory)/${{parameters.configuration_folder}}/terraform.tfplan'
        publishPlanResults: 'terraform_plan'
    - task: PowerShell@2
      displayName: 'Set variable output_terraform_plan_has_changes'
      inputs:
        targetType: 'inline'
        script: |
          Write-Host "Terraform changes detected: $(TERRAFORM_PLAN_HAS_CHANGES)"

This block will fail:

    - task: [email protected]
      displayName: "Terraform Plan"
      inputs:
        command: 'plan'
        workingDirectory: '$(System.DefaultWorkingDirectory)/${{parameters.configuration_folder}}'
        environmentServiceName: '${{variables.subscription_name}}'
        commandOptions: '--var-file=vars_env_${{parameters.subscription_code}}.tfvars -out=$(System.DefaultWorkingDirectory)/${{parameters.configuration_folder}}/terraform.tfplan'
        publishPlanResults: 'terraform_plan'
    - task: PowerShell@2
      displayName: 'Set variable output_terraform_plan_has_changes'
      inputs:
        targetType: 'inline'
        script: |
          Write-Host "Terraform changes detected: $(TERRAFORM_PLAN_HAS_CHANGES)"

Exception message:

TERRAFORM_PLAN_HAS_CHANGES : The term 'TERRAFORM_PLAN_HAS_CHANGES' is not recognized as the name of a cmdlet, 
function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the 
path is correct and try again.
At D:\a\_temp\cf205a36-eb72-43a3-99bd-1e47c15f25f7.ps1:3 char:43
+ ... rite-Host "Terraform changes detected: $(TERRAFORM_PLAN_HAS_CHANGES)"
+                                              ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (TERRAFORM_PLAN_HAS_CHANGES:String) [], ParentContainsErrorRecordExcepti 
   on
    + FullyQualifiedErrorId : CommandNotFoundException

Add option for `show` output

There's no way to save the output from the show command.
image

terraform show -json > $(Build.ArtifactStagingDirectory)/tfstate.json is what I run as a bash command instead.
I guess it would be nice to have this command so there's possibility to save output (binary or json).

No terraform plans have been published for this pipeline run. The terraform cli task must run plan with publishPlanResults: string (where string represents the plan name) to view plans.

Describe the bug
no plan view

To Reproduce
Steps to reproduce the behavior:

          - task: TerraformCLI@0
            displayName: "terraform plan"
            inputs:
              command: plan
              environmentServiceName: "${{ parameters.AZURERMCONNECTION }}"
              commandOptions: "\
                -compact-warnings \
                -detailed-exitcode \
                -parallelism=300 \
                -out=terraform.tfplan.zip \
                "
              workingDirectory: self
              publishPlanResults: "plan"
              runAzLogin: false
            env:
              AZDO_PERSONAL_ACCESS_TOKEN: $(System.AccessToken)
2021-04-16T16:49:59.4663199Z ##[section]Starting: terraform plan
2021-04-16T16:49:59.4673141Z ==============================================================================
2021-04-16T16:49:59.4673451Z Task         : Terraform CLI
2021-04-16T16:49:59.4673714Z Description  : Execute terraform cli commands
2021-04-16T16:49:59.4673941Z Version      : 0.6.20
2021-04-16T16:49:59.4674168Z Author       : Charles Zipp
2021-04-16T16:49:59.4674376Z Help         : 
2021-04-16T16:49:59.4674640Z ==============================================================================
2021-04-16T16:49:59.9063555Z [command]/opt/hostedtoolcache/terraform/0.15.0/x64/terraform version
2021-04-16T16:49:59.9660409Z Terraform v0.15.0
2021-04-16T16:49:59.9661319Z on linux_amd64
2021-04-16T16:49:59.9662158Z + provider registry.terraform.io/microsoft/azuredevops v0.1.3
2021-04-16T16:49:59.9731234Z 
2021-04-16T16:49:59.9889385Z [command]/opt/hostedtoolcache/terraform/0.15.0/x64/terraform plan -compact-warnings -detailed-exitcode -parallelism=300 -out=terraform.tfplan.zip
XXXX
2021-04-16T16:50:03.3411889Z 
2021-04-16T16:50:03.3413711Z Terraform used the selected providers to generate the following execution
2021-04-16T16:50:03.3414473Z plan. Resource actions are indicated with the following symbols:
2021-04-16T16:50:03.3416239Z   �[32m+�[0m create
2021-04-16T16:50:03.3417169Z �[0m
2021-04-16T16:50:03.3418681Z Terraform will perform the following actions:
2021-04-16T16:50:03.3419993Z 
2021-04-16T16:50:03.3457849Z 
2021-04-16T16:50:03.3458695Z �[0m�[1mPlan:�[0m 2 to add, 0 to change, 0 to destroy.
2021-04-16T16:50:03.3605550Z ─────────────────────────────────────────────────────────────────────────────�[0m
2021-04-16T16:50:03.3605779Z 
2021-04-16T16:50:03.3606073Z Saved the plan to: terraform.tfplan.zip
2021-04-16T16:50:03.3606246Z 
2021-04-16T16:50:03.3606602Z To perform exactly these actions, run the following command to apply:
2021-04-16T16:50:03.3607085Z     terraform apply "terraform.tfplan.zip"
2021-04-16T16:50:03.6492016Z 
2021-04-16T16:50:03.6560649Z ##[section]Finishing: terraform plan

Expected behavior
to see plan view

Screenshots
image

Agent Configuration

  • Hosted
2021-04-16T16:49:39.2191447Z ##[section]Starting: Initialize job
2021-04-16T16:49:39.2192957Z Agent name: 'Hosted Agent'
2021-04-16T16:49:39.2193353Z Agent machine name: 'fv-az131-6'
2021-04-16T16:49:39.2193626Z Current agent version: '2.184.2'
2021-04-16T16:49:39.2235508Z ##[group]Operating System
2021-04-16T16:49:39.2235790Z Ubuntu
2021-04-16T16:49:39.2235948Z 20.04.2
2021-04-16T16:49:39.2236113Z LTS
2021-04-16T16:49:39.2236268Z ##[endgroup]
2021-04-16T16:49:39.2236468Z ##[group]Virtual Environment
2021-04-16T16:49:39.2236685Z Environment: ubuntu-20.04
2021-04-16T16:49:39.2236897Z Version: 20210412.1
2021-04-16T16:49:39.2237218Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu20/20210412.1/images/linux/Ubuntu2004-README.md
2021-04-16T16:49:39.2237661Z Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu20%2F20210412.1
2021-04-16T16:49:39.2237935Z ##[endgroup]
2021-04-16T16:49:39.2238950Z Current image version: '20210412.1'
2021-04-16T16:49:39.2241416Z Agent running as: 'vsts'
2021-04-16T16:49:39.2301748Z Prepare build directory.
2021-04-16T16:49:39.2568132Z Set build variables.
2021-04-16T16:49:39.2602894Z Download all required tasks.
2021-04-16T16:49:39.2724819Z Downloading task: TerraformInstaller (0.6.20)
2021-04-16T16:49:41.3034668Z Downloading task: AzureCLI (2.1.0)
2021-04-16T16:49:41.5047811Z Downloading task: TerraformCLI (0.6.20)
2021-04-16T16:49:42.2830794Z Downloading task: PublishPipelineArtifact (1.2.3)
2021-04-16T16:49:42.3487071Z Checking job knob settings.
2021-04-16T16:49:42.3501975Z    Knob: AgentToolsDirectory = /opt/hostedtoolcache Source: ${AGENT_TOOLSDIRECTORY} 
2021-04-16T16:49:42.3504501Z    Knob: AgentPerflog = /home/vsts/perflog Source: ${VSTS_AGENT_PERFLOG} 
2021-04-16T16:49:42.3506442Z    Knob: TaskRestrictionsEnforcementMode = WarningOnly Source: $(agent.taskRestrictionsEnforcementMode) 
2021-04-16T16:49:42.3507198Z    Knob: EnableTaskRestrictionsTelemetry = true Source: $(agent.enableTaskRestrictionsTelemetry) 
2021-04-16T16:49:42.3507885Z Finished checking job knob settings.
2021-04-16T16:49:42.3915954Z Start tracking orphan processes.
2021-04-16T16:49:42.4123777Z ##[section]Finishing: Initialize job

Additional context
Add any other context about the problem here.

How to upgrade the task version ?

Describe the bug
I am unable to upgrade to > 0.15 and due to _arm and as per #44 it should be automatic, but I think it is not working because it is not taking the latest terraform CLI version.

This is how my starting init looks like:
Starting: init

Task : Terraform CLI
Description : Execute terraform cli commands
Version : 0.6.8
Author : Charles Zipp
Help :

and I see the latest version is 0.6.27.

so is this why it is not working ? if so, how can I configure my pipeline to use the latest version and why is it not picking up the latest one ?

To Reproduce
This is the part of the pipeline that uses terraform init
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
displayName: 'terraform install'
inputs:
terraformVersion: ${{ parameters.terraformVersion }}
- task: TerraformCLI@0
displayName: init
inputs:
command: 'init'

Expected behavior
terraform init should work with version > 0.15

Set terraform output variables as multi-job output pipeline variables

Set the pipeline variables generated from terraform output to be accessible across multiple jobs within a stage as documented here.

This would require setting the isOutput=true flag when setting the variable.

Currently the azure-pipelines-task-lib does not provide a means to do this. See this issue. However, if can write the task.setvariable command manually to stdout then this should still be possible.

This should include the TERRAFORM_PLAN_HAS_CHANGES variable produced when running terraform plan with detailed exit code.

Support for AzureRM Service Connections that have Management Group Scope

As an Azure Administrator, I have many subscriptions grouped together under management groups. I have created a Service Connection in Azure DevOps has a scope defined at a particular management group scope (instead of subscription). This allows me to avoid creating a separate service principal and service connection for every subscription I create. Least privilige is acheived by only grouping subscriptions together in a management group that I am comfortable having a single service principal with 'Contributor' rights over.

However in the task, when I select my service connection that has management group contributor permissions I do not have the ability to select the subscription which contains the storage account for my Terraform State storage.

Request: Add an additional field to select the subscription if an AzureRM service connection is selected that has Management group-level permissions. If you add this feature, yours will be the only Terraform extension on the marketplace that has it.

image
image

Support for GCS and AWS

Hi,

Please can we have support for backend types GCS and AWS? Is this a lot of work or should it be rather trivial to support?

Terraform show doesn't show

Describe the bug
When running Terraform show via a Devops release pipeline, no output is produced, past generic warnings.

As an example, the below output is received - I can see a destroy will occur, but where is the actual output from terraform show?

To Reproduce
Steps to reproduce the behavior:

  1. Setup pipeline as image
  2. Execute pipeline
  3. See logs below

Expected behavior
A human-readable output in the logs for the terraform show step

Pipeline Logs

2021-06-23T10:17:49.8546136Z ##[debug]Evaluating condition for step: 'terraform show'
2021-06-23T10:17:49.8548074Z ##[debug]Evaluating: succeeded()
2021-06-23T10:17:49.8548578Z ##[debug]Evaluating succeeded:
2021-06-23T10:17:49.8549696Z ##[debug]=> True
2021-06-23T10:17:49.8550348Z ##[debug]Result: True
2021-06-23T10:17:49.8551093Z ##[section]Starting: terraform show
2021-06-23T10:17:49.8664654Z ==============================================================================
2021-06-23T10:17:49.8664964Z Task         : Terraform CLI
2021-06-23T10:17:49.8665228Z Description  : Execute terraform cli commands
2021-06-23T10:17:49.8665454Z Version      : 0.6.25
2021-06-23T10:17:49.8665662Z Author       : Charles Zipp
2021-06-23T10:17:49.8665862Z Help         : 
2021-06-23T10:17:49.8666138Z ==============================================================================
2021-06-23T10:17:49.8720848Z ##[debug]Using node path: C:\agents\2.187.2\externals\node\bin\node.exe
2021-06-23T10:17:50.0733662Z ##[debug]agent.TempDirectory=D:\a\_temp
2021-06-23T10:17:50.1088397Z ##[debug]loading inputs and endpoints
2021-06-23T10:17:50.1095139Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2021-06-23T10:17:50.1095814Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2021-06-23T10:17:50.1128665Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2021-06-23T10:17:50.1129331Z ##[debug]loading INPUT_AIINSTRUMENTATIONKEY
2021-06-23T10:17:50.1137317Z ##[debug]loading INPUT_ALLOWTELEMETRYCOLLECTION
2021-06-23T10:17:50.1138430Z ##[debug]loading INPUT_BACKENDAZURERMSTORAGEACCOUNTSKU
2021-06-23T10:17:50.1139449Z ##[debug]loading INPUT_BACKENDTYPE
2021-06-23T10:17:50.1139956Z ##[debug]loading INPUT_COMMAND
2021-06-23T10:17:50.1150959Z ##[debug]loading INPUT_COMMANDOPTIONS
2021-06-23T10:17:50.1151534Z ##[debug]loading INPUT_ENSUREBACKEND
2021-06-23T10:17:50.1152546Z ##[debug]loading INPUT_INPUTTARGETPLANORSTATEFILEPATH
2021-06-23T10:17:50.1153188Z ##[debug]loading INPUT_RUNAZLOGIN
2021-06-23T10:17:50.1154988Z ##[debug]loading INPUT_WORKINGDIRECTORY
2021-06-23T10:17:50.1155680Z ##[debug]loading INPUT_WORKSPACESUBCOMMAND
2021-06-23T10:17:50.1156561Z ##[debug]loaded 14
2021-06-23T10:17:50.1157033Z ##[debug]Agent.ProxyUrl=undefined
2021-06-23T10:17:50.1157927Z ##[debug]Agent.CAInfo=undefined
2021-06-23T10:17:50.1158550Z ##[debug]Agent.ClientCert=undefined
2021-06-23T10:17:50.1168185Z ##[debug]Agent.SkipCertValidation=undefined
2021-06-23T10:17:50.3865447Z ##[debug]allowTelemetryCollection=false
2021-06-23T10:17:50.3866945Z ##[debug]System.TeamFoundationCollectionUri=https://dev.azure.com/medicalprotection/
2021-06-23T10:17:50.3872770Z ##[debug]SYSTEMVSSCONNECTION auth param ACCESSTOKEN = ***
2021-06-23T10:17:50.3873511Z ##[debug]Agent.ProxyUrl=undefined
2021-06-23T10:17:50.3909472Z ##[debug]command=show
2021-06-23T10:17:50.3911466Z ##[debug]command=show
2021-06-23T10:17:50.3917463Z ##[debug]workingDirectory=D:\a\r1\a\Terraform-Recovery-Services-Composition\drop
2021-06-23T10:17:50.3924036Z ##[debug]which 'terraform'
2021-06-23T10:17:50.3932065Z ##[debug]found: 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:50.3933757Z ##[debug]which 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:50.3936420Z ##[debug]found: 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:50.3938875Z ##[debug]C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe arg: version
2021-06-23T10:17:50.3943689Z ##[debug]exec tool: C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe
2021-06-23T10:17:50.3944373Z ##[debug]arguments:
2021-06-23T10:17:50.3944921Z ##[debug]   version
2021-06-23T10:17:50.3951189Z [command]C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe version
2021-06-23T10:17:50.5199712Z Terraform v1.0.0
2021-06-23T10:17:50.5208403Z on windows_amd64
2021-06-23T10:17:50.5250496Z + provider registry.terraform.io/hashicorp/azurerm v2.64.0
2021-06-23T10:17:50.5251977Z + provider registry.terraform.io/hashicorp/random v3.1.0
2021-06-23T10:17:50.5253397Z 
2021-06-23T10:17:50.5254132Z ##[debug]Exit code 0 received from tool 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:50.5407482Z ##[debug]STDIO streams have closed for tool 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:50.5408999Z ##[debug]command=show
2021-06-23T10:17:50.5540842Z ##[debug]secureVarsFile=undefined
2021-06-23T10:17:50.5543211Z ##[debug]secureVarsFile=undefined
2021-06-23T10:17:50.5544161Z ##[debug]secure file name for id undefined = undefined
2021-06-23T10:17:50.5573128Z ##[debug]commandOptions=-no-color
2021-06-23T10:17:50.5574198Z ##[debug]commandOptions=-no-color
2021-06-23T10:17:50.5576157Z ##[debug]inputTargetPlanOrStateFilePath=D:\a\r1\a/Terraform-Recovery-Services-Composition/drop/plan.tfplan
2021-06-23T10:17:50.5577157Z ##[debug]command=show
2021-06-23T10:17:50.5577921Z ##[debug]workingDirectory=D:\a\r1\a\Terraform-Recovery-Services-Composition\drop
2021-06-23T10:17:50.5578751Z ##[debug]which 'terraform'
2021-06-23T10:17:50.5579545Z ##[debug]found: 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:50.5580365Z ##[debug]which 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:50.5581140Z ##[debug]found: 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:50.5582031Z ##[debug]C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe arg: show
2021-06-23T10:17:50.5584633Z ##[debug]C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe arg: -json
2021-06-23T10:17:50.5587280Z ##[debug]C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe arg: -no-color
2021-06-23T10:17:50.5588284Z ##[debug]C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe arg: D:\a\r1\a/Terraform-Recovery-Services-Composition/drop/plan.tfplan
2021-06-23T10:17:50.5589216Z ##[debug]exec tool: C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe
2021-06-23T10:17:50.5589949Z ##[debug]arguments:
2021-06-23T10:17:50.5590613Z ##[debug]   show
2021-06-23T10:17:50.5591249Z ##[debug]   -json
2021-06-23T10:17:50.5592202Z ##[debug]   -no-color
2021-06-23T10:17:50.5593007Z ##[debug]   D:\a\r1\a/Terraform-Recovery-Services-Composition/drop/plan.tfplan
2021-06-23T10:17:52.4319311Z ##[debug]Exit code 0 received from tool 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:52.4324808Z ##[debug]STDIO streams have closed for tool 'C:\hostedtoolcache\windows\terraform\1.0.0\x64\terraform.exe'
2021-06-23T10:17:52.4332854Z ##[debug]inputTargetPlanOrStateFilePath=D:\a\r1\a/Terraform-Recovery-Services-Composition/drop/plan.tfplan
2021-06-23T10:17:52.4334402Z ##[debug]inputTargetPlanOrStateFilePath=D:\a\r1\a/Terraform-Recovery-Services-Composition/drop/plan.tfplan
2021-06-23T10:17:52.4370694Z ##[debug]set TERRAFORM_PLAN_HAS_DESTROY_CHANGES=true
2021-06-23T10:17:52.4373275Z ##[debug]Processed: ##vso[task.setvariable variable=TERRAFORM_PLAN_HAS_DESTROY_CHANGES;isOutput=false;issecret=false;]true
2021-06-23T10:17:52.4374224Z ##[debug]set vso[task.setvariable variable=TERRAFORM_PLAN_HAS_DESTROY_CHANGES] to true
2021-06-23T10:17:52.4402396Z ##[warning]Destroy detected!
2021-06-23T10:17:52.4411444Z ##[debug]Processed: ##vso[task.issue type=warning;]Destroy detected!
2021-06-23T10:17:52.4413970Z ##[debug]allowTelemetryCollection=false
2021-06-23T10:17:52.4415111Z ##[debug]command=show
2021-06-23T10:17:52.4415680Z ##[debug]command=show
2021-06-23T10:17:52.4416238Z ##[debug]executed command 'show'
2021-06-23T10:17:52.4417018Z ##[debug]set TERRAFORM_LAST_EXITCODE=0
2021-06-23T10:17:52.4420641Z ##[debug]Processed: ##vso[task.setvariable variable=TERRAFORM_LAST_EXITCODE;isOutput=false;issecret=false;]0
2021-06-23T10:17:52.4453426Z ##[section]Finishing: terraform show

Agent Configuration

  • OS: vs2017-win2016
  • Hosted
  • Terraform version used: 1.0.0
  • AzureCLI version used: default

Additional context
Our plan files are being produced at build phase - the file is then contained in an artifact and released via a release pipeline - I want a manual intervention in the release pipeline to check the output of terraform show.

Failure in 'terraform plan' stage when azure storage backend is chosen

Describe the bug
When initializing 'terraform init' step with azure storage account info as the backend, 'terraform plan' stage fails.

To Reproduce
Run the following script:

task: TerraformInstaller@0
  displayName: Download terraform
  inputs:
    terraformVersion: '0.15.5'
    downloadUrl: 'https://releases.hashicorp.com/terraform/0.15.5/terraform_0.15.5_linux_amd64.zip'
- task: TerraformCLI@0
  displayName: 'Terraform init'
  inputs:
    command: 'init'
    backendType: 'azurerm'
    backendServiceArm: 'Development and test (MASKED)'
    backendAzureRmResourceGroupName: 'myresourcegroup'
    backendAzureRmStorageAccountName: 'myaccountname'
    backendAzureRmContainerName: 'multitstate'
    backendAzureRmKey: 'terraform.state'
    allowTelemetryCollection: false

- task: TerraformCLI@0
  displayName: 'Terraform plan'
  inputs:
    command: 'plan'
    environmentServiceName: 'Development and test (MASKED)'
    allowTelemetryCollection: false

- task: TerraformCLI@0
  displayName: 'Terraform apply'
  name: terraformApply
  inputs:
    command: 'apply'
    environmentServiceName: 'Development and test (MASKED)'
    allowTelemetryCollection: false

- task: TerraformCLI@0
  displayName: 'Terraform output'
  inputs:
    command: 'output'
    allowTelemetryCollection: false

Expected behavior
The pipeline should not fail.

Screenshots
n/a

Pipeline Logs

Starting: Terraform plan

Task : Terraform CLI
Description : Execute terraform cli commands
Version : 0.6.25
Author : Charles Zipp
Help :

/opt/hostedtoolcache/terraform/0.15.5/x64/terraform version
Terraform v0.15.5
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.0.0. You can update by downloading from https://www.terraform.io/downloads.html

/opt/hostedtoolcache/terraform/0.15.5/x64/terraform plan

│ Error: No configuration files

│ Plan requires configuration to be present. Planning without a configuration
│ would mark everything for destruction, which is normally not what is
│ desired. If you would like to destroy everything, run plan with the
│ -destroy option. Otherwise, create a Terraform configuration file (.tf
│ file) and try again.

##[error]Terraform command 'plan' failed with exit code '1'.
##[error]╷
│ Error: No configuration files

│ Plan requires configuration to be present. Planning without a configuration
│ would mark everything for destruction, which is normally not what is
│ desired. If you would like to destroy everything, run plan with the
│ -destroy option. Otherwise, create a Terraform configuration file (.tf
│ file) and try again.

Finishing: Terraform plan

Agent Configuration

  • OS: [e.g. ubuntu debian]
  • Hosted/Self Hosted
  • Terraform version used (Default for hosted agent is acceptable)
  • AzureCLI version used (Default for hosted agent is acceptable or N/A)

Additional context
n/a

`SyntaxError: Unexpected token M in JSON at position 82780` when doing a `terraform show ./tfplan`

Describe the bug

Azure DevOps is reporting an error when the pipeline runs a terraform show -json ./tfplan task. The error says:

2021-07-01T06:22:45.4636143Z ##[error]SyntaxError: Unexpected token M in JSON at position 82780

Full debug output from the task is available at https://gist.github.com/ausfestivus/9eca2d6d03aa81ff7a2c3251335b5eb9

To Reproduce

The error occurs with this terraform code. When I run the same pipeline YAML with a different set of terraform files the problem does not occur.

Expected behavior

The error should not occur and the terraform command should succeed.

Screenshots

n/a

Pipeline Logs

See gist https://gist.github.com/ausfestivus/9eca2d6d03aa81ff7a2c3251335b5eb9

Agent Configuration

  • OS: ubuntu 20 LTS
  • Self hosted
  • 0.13.7
  • AzureCLI version used (Default for hosted agent is acceptable or N/A)

Additional context

I suspect this is an AzDO problem but I am stuck on how to troubleshoot further. A google search turned up a link to this repo at #171 but that URL 404s. Its probably an issue in the old repo.

EDIT

  • corrected the command that is being run that produces the error

support for terraform 0.15

Terraform has removed some arguments in their 0.15 release.

Can we add these new arguments to support 0.15?

Old Name New Name
arm_client_id client_id
arm_client_secret client_secret
arm_subscription_id subscription_id
arm_tenant_id tenant_id

When using Terraform CLI with command show we get a new error: TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined

Describe the bug
We are using the Terraform CLI DevOps extension version 0.6.8.
This always worked and now it has started to fail with minimal error information.

To Reproduce
Steps to reproduce the behavior:

  1. Using the extension run a terraform plan that outputs the plan to a file
- task: TerraformCLI@0
      displayName: 'Terraform Plan'
      condition: and(
          always(),
          succeeded()
        )
      inputs:
        command: 'plan'
        commandOptions: '-lock-timeout="${{ parameters.terraform_lock_timeout }}" -out plan.tfplan -detailed-exitcode -input=false'
        environmentServiceName: '${{ parameters.deploymentServiceConnection }}'
        workingDirectory: '$(Build.SourcesDirectory)/automation/terraform/${{ parameters.tf_module }}'
      env:
        TF_VAR_config_root: $(Build.SourcesDirectory)/${{ parameters.configurationPath }}
        TF_VAR_config_directory: ${{ parameters.config }}
        ${{ each tf_var in parameters.tf_vars }}:
          TF_VAR_${{ tf_var.Key }}: ${{ tf_var.Value }}
        ${{ each env_var in parameters.env_vars }}:
          ${{ env_var.Key }}: ${{ env_var.Value }}
  1. Using the extension run a terraform show with the plan file from step 1
    - task: TerraformCLI@0
      displayName: 'Detect Destroy Actions'
      condition: and(
          always(),
          succeeded(),
          eq(variables['TERRAFORM_PLAN_HAS_CHANGES'], 'true')
        )
      inputs:
        command: 'show'
        inputTargetPlanOrStateFilePath: 'plan.tfplan'
        workingDirectory: '$(Build.SourcesDirectory)/automation/terraform/${{ parameters.tf_module }}'
  1. See error:
##[error]TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined

Expected behavior
We have been using this for a while and we use it to detect if anything in the plan is going to be detroyed

Screenshots
image

Pipeline Logs
2021-05-12T03:12:53.9955373Z ##[debug]Evaluating condition for step: 'Detect Destroy Actions'
2021-05-12T03:12:53.9960260Z ##[debug]Evaluating: and(always(), succeeded(), eq(variables['TERRAFORM_PLAN_HAS_CHANGES'], 'true'))
2021-05-12T03:12:53.9961993Z ##[debug]Evaluating and:
2021-05-12T03:12:53.9963407Z ##[debug]..Evaluating always:
2021-05-12T03:12:53.9965115Z ##[debug]..=> True
2021-05-12T03:12:53.9966112Z ##[debug]..Evaluating succeeded:
2021-05-12T03:12:53.9967676Z ##[debug]..=> True
2021-05-12T03:12:53.9968638Z ##[debug]..Evaluating eq:
2021-05-12T03:12:53.9969631Z ##[debug]....Evaluating indexer:
2021-05-12T03:12:53.9970778Z ##[debug]......Evaluating variables:
2021-05-12T03:12:53.9972229Z ##[debug]......=> Object
2021-05-12T03:12:53.9973855Z ##[debug]......Evaluating String:
2021-05-12T03:12:53.9976407Z ##[debug]......=> 'TERRAFORM_PLAN_HAS_CHANGES'
2021-05-12T03:12:53.9978350Z ##[debug]....=> 'true'
2021-05-12T03:12:53.9980048Z ##[debug]....Evaluating String:
2021-05-12T03:12:53.9981986Z ##[debug]....=> 'true'
2021-05-12T03:12:53.9983188Z ##[debug]..=> True
2021-05-12T03:12:53.9984393Z ##[debug]=> True
2021-05-12T03:12:53.9986328Z ##[debug]Expanded: and(True, True, eq('true', 'true'))
2021-05-12T03:12:53.9987588Z ##[debug]Result: True
2021-05-12T03:12:53.9988998Z ##[section]Starting: Detect Destroy Actions
2021-05-12T03:12:54.0006679Z ==============================================================================
2021-05-12T03:12:54.0007671Z Task : Terraform CLI
2021-05-12T03:12:54.0008417Z Description : Execute terraform cli commands
2021-05-12T03:12:54.0009757Z Version : 0.6.8
2021-05-12T03:12:54.0010847Z Author : Charles Zipp
2021-05-12T03:12:54.0011391Z Help :
2021-05-12T03:12:54.0011991Z ==============================================================================
2021-05-12T03:12:54.0301461Z ##[debug]Using node path: /apps/agent/externals/node/bin/node
2021-05-12T03:12:54.2904532Z ##[debug]agent.TempDirectory=/apps/agent/_work/_temp
2021-05-12T03:12:54.3125178Z ##[debug]loading inputs and endpoints
2021-05-12T03:12:54.3142834Z ##[debug]loading INPUT_COMMAND
2021-05-12T03:12:54.3215268Z ##[debug]loading INPUT_WORKINGDIRECTORY
2021-05-12T03:12:54.3222509Z ##[debug]loading INPUT_RUNAZLOGIN
2021-05-12T03:12:54.3229575Z ##[debug]loading INPUT_BACKENDTYPE
2021-05-12T03:12:54.3231393Z ##[debug]loading INPUT_ENSUREBACKEND
2021-05-12T03:12:54.3233031Z ##[debug]loading INPUT_BACKENDAZURERMSTORAGEACCOUNTSKU
2021-05-12T03:12:54.3234915Z ##[debug]loading INPUT_AIINSTRUMENTATIONKEY
2021-05-12T03:12:54.3236539Z ##[debug]loading INPUT_ALLOWTELEMETRYCOLLECTION
2021-05-12T03:12:54.3238664Z ##[debug]loading INPUT_INPUTTARGETPLANORSTATEFILEPATH
2021-05-12T03:12:54.3240287Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2021-05-12T03:12:54.3241920Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2021-05-12T03:12:54.3244221Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2021-05-12T03:12:54.3246917Z ##[debug]loading SECRET_xxxxxxxxx-AZURERM-SPN-CLIENT-ID
2021-05-12T03:12:54.3249137Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
2021-05-12T03:12:54.3251620Z ##[debug]loading SECRET_xxxxxxxxx-AZUREAD-SPN-CLIENT-ID
2021-05-12T03:12:54.3254298Z ##[debug]loading SECRET_xxxxxxxxx-AZURERM-SPN-SECRET
2021-05-12T03:12:54.3256126Z ##[debug]loading SECRET_xxxxxxxxxAZDOAUTOMATION-PAT-MODULE-DOWNLOAD
2021-05-12T03:12:54.3257776Z ##[debug]loading SECRET_AZDO_PERSONAL_ACCESS_TOKEN
2021-05-12T03:12:54.3259719Z ##[debug]loading SECRET_xxxxxxxxx-AZUREAD-SPN-SECRET
2021-05-12T03:12:54.3262164Z ##[debug]loading SECRET_xxxxxxxxxAZDOAUTOMATION-PAT
2021-05-12T03:12:54.3265743Z ##[debug]loading SECRET_xxxxxxxxx-SONARQUBE-TOKEN
2021-05-12T03:12:54.3268467Z ##[debug]loading SECRET_xxxxxxxxx-PIMINTERFACE-SECRET
2021-05-12T03:12:54.3270490Z ##[debug]loading SECRET_xxxxxxxxx-NETIQ-IDM-INTEGRATION-PARTNER-USERNAME
2021-05-12T03:12:54.3272233Z ##[debug]loading SECRET_ARTIFACTORY_PROVIDER_API_KEY
2021-05-12T03:12:54.3274462Z ##[debug]loading SECRET_xxxxxxxxx-ARTIFACTORY-ACCESS-TOKEN
2021-05-12T03:12:54.3276819Z ##[debug]loading SECRET_xxxxxxxxx-ARTIFACTORY-TOKEN-NAME
2021-05-12T03:12:54.3279244Z ##[debug]loading SECRET_xxxxxxxxx-NETIQ-IDM-RESTDRIVER-USERNAME
2021-05-12T03:12:54.3281284Z ##[debug]loaded 27
2021-05-12T03:12:54.3283021Z ##[debug]Agent.ProxyUrl=undefined
2021-05-12T03:12:54.3284808Z ##[debug]Agent.CAInfo=undefined
2021-05-12T03:12:54.3286550Z ##[debug]Agent.ClientCert=undefined
2021-05-12T03:12:54.3288459Z ##[debug]Agent.SkipCertValidation=undefined
2021-05-12T03:12:54.7754827Z ##[debug]allowTelemetryCollection=true
2021-05-12T03:12:54.7758749Z ##[debug]aiInstrumentationKey=ec11a02f-c97a-4b55-8005-acf129682668
2021-05-12T03:12:54.8100004Z ##[debug]System.TeamFoundationCollectionUri=https://dev.azure.com/xxxxgroup/
2021-05-12T03:12:54.8106019Z ##[debug]System.TeamProject=xxxxxxxxx
2021-05-12T03:12:54.8108534Z ##[debug]System.HostType=build
2021-05-12T03:12:54.8110301Z ##[debug]Agent.OS=Linux
2021-05-12T03:12:54.8111975Z ##[debug]Agent.OSArchitecture=X64
2021-05-12T03:12:54.8116591Z ##[debug]Agent.JobStatus=Succeeded
2021-05-12T03:12:54.8122326Z ##[debug]System.TeamFoundationCollectionUri=https://dev.azure.com/xxxxgroup/
2021-05-12T03:12:54.8129911Z ##[debug]SYSTEMVSSCONNECTION auth param ACCESSTOKEN = ***
2021-05-12T03:12:54.8131693Z ##[debug]Agent.ProxyUrl=undefined
2021-05-12T03:12:54.8171722Z ##[debug]command=show
2021-05-12T03:12:54.8176400Z ##[debug]command=show
2021-05-12T03:12:54.8184136Z ##[debug]workingDirectory=/apps/agent/_work/8/s/automation/terraform/deploy_devops_tools
2021-05-12T03:12:54.8195031Z ##[debug]which 'terraform'
2021-05-12T03:12:54.8201930Z ##[debug]found: '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:54.8204857Z ##[debug]which '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:54.8207908Z ##[debug]found: '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:54.8212803Z ##[debug]/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform arg: version
2021-05-12T03:12:54.8223976Z ##[debug]exec tool: /apps/agent/_work/_tool/terraform/0.13.4/x64/terraform
2021-05-12T03:12:54.8226859Z ##[debug]arguments:
2021-05-12T03:12:54.8228865Z ##[debug] version
2021-05-12T03:12:54.8236041Z [command]/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform version
2021-05-12T03:12:54.8889183Z (node:17787) Warning: Ignoring extra certs from /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem, load failed: error:0B07C065:x509 certificate routines:X509_STORE_add_cert:cert already in hash table
2021-05-12T03:12:54.8891135Z
2021-05-12T03:12:55.4954243Z Terraform v0.13.4
2021-05-12T03:12:55.4955835Z + provider registry.terraform.io/hashicorp/azuread v1.4.0
2021-05-12T03:12:55.4957099Z + provider registry.terraform.io/hashicorp/null v3.1.0
2021-05-12T03:12:55.4958790Z + provider registry.terraform.io/hashicorp/random v3.0.0
2021-05-12T03:12:55.4959899Z + provider registry.terraform.io/hashicorp/time v0.7.1
2021-05-12T03:12:55.4960943Z + provider registry.terraform.io/microsoft/azuredevops v0.1.2
2021-05-12T03:12:55.4962285Z + provider tfe.cmltd.net.au/xxxx/artifactory v0.0.14+27
2021-05-12T03:12:55.4963363Z + provider tfe.cmltd.net.au/xxxx/sonarqube v0.0.38+53
2021-05-12T03:12:55.4964263Z
2021-05-12T03:12:55.4965172Z Your version of Terraform is out of date! The latest version
2021-05-12T03:12:55.4966797Z is 0.15.3. You can update by downloading from https://www.terraform.io/downloads.html
2021-05-12T03:12:55.5002522Z
2021-05-12T03:12:55.5012477Z ##[debug]Exit code 0 received from tool '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:55.5016198Z ##[debug]STDIO streams have closed for tool '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:55.5049960Z ##[debug]secureVarsFile=undefined
2021-05-12T03:12:55.5051841Z ##[debug]secureVarsFile=undefined
2021-05-12T03:12:55.5053904Z ##[debug]secure file name for id undefined = undefined
2021-05-12T03:12:55.5056060Z ##[debug]commandOptions=undefined
2021-05-12T03:12:55.5057738Z ##[debug]commandOptions=undefined
2021-05-12T03:12:55.5060609Z ##[debug]inputTargetPlanOrStateFilePath=plan.tfplan
2021-05-12T03:12:55.5075898Z ##[debug]command=show
2021-05-12T03:12:55.5077972Z ##[debug]workingDirectory=/apps/agent/_work/8/s/automation/terraform/deploy_devops_tools
2021-05-12T03:12:55.5095492Z ##[debug]which 'terraform'
2021-05-12T03:12:55.5098293Z ##[debug]found: '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:55.5101737Z ##[debug]which '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:55.5104901Z ##[debug]found: '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:55.5107100Z ##[debug]/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform arg: show
2021-05-12T03:12:55.5109608Z ##[debug]/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform arg: -json
2021-05-12T03:12:55.5112279Z ##[debug]/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform arg: plan.tfplan
2021-05-12T03:12:55.5114390Z ##[debug]exec tool: /apps/agent/_work/_tool/terraform/0.13.4/x64/terraform
2021-05-12T03:12:55.5116263Z ##[debug]arguments:
2021-05-12T03:12:55.5117713Z ##[debug] show
2021-05-12T03:12:55.5119277Z ##[debug] -json
2021-05-12T03:12:55.5120772Z ##[debug] plan.tfplan
2021-05-12T03:12:59.7406299Z ##[debug]Exit code 0 received from tool '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:59.7570013Z ##[debug]STDIO streams have closed for tool '/apps/agent/_work/_tool/terraform/0.13.4/x64/terraform'
2021-05-12T03:12:59.7655288Z ##[debug]inputTargetPlanOrStateFilePath=plan.tfplan
2021-05-12T03:12:59.7765094Z ##[debug]inputTargetPlanOrStateFilePath=plan.tfplan
2021-05-12T03:12:59.7767086Z ##[debug]allowTelemetryCollection=true
2021-05-12T03:12:59.7825239Z ##[error]TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined
2021-05-12T03:12:59.7839370Z ##[debug]Processed: ##vso[task.issue type=error;]TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined
2021-05-12T03:12:59.7841067Z ##[debug]allowTelemetryCollection=true
2021-05-12T03:12:59.7848498Z ##[debug]command=show
2021-05-12T03:12:59.7849889Z ##[debug]command=show
2021-05-12T03:12:59.7851343Z ##[debug]command=show
2021-05-12T03:12:59.7853119Z ##[debug]executed command 'show'
2021-05-12T03:12:59.7855961Z ##[debug]set TERRAFORM_LAST_EXITCODE=1
2021-05-12T03:12:59.7861338Z ##[debug]Processed: ##vso[task.setvariable variable=TERRAFORM_LAST_EXITCODE;isOutput=false;issecret=false;]1
2021-05-12T03:12:59.7862815Z ##[debug]task result: Failed
2021-05-12T03:12:59.7865628Z ##[error]TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined
2021-05-12T03:12:59.7868440Z ##[debug]Processed: ##vso[task.issue type=error;]TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined
2021-05-12T03:12:59.7872780Z ##[debug]Processed: ##vso[task.complete result=Failed;]TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined
2021-05-12T03:13:00.2336049Z ##[section]Finishing: Detect Destroy Actions

Agent Configuration

  • OS: linux - ubuntu
  • Self Hosted
  • 0.13.4
  • AzureCLI version used (Default for hosted agent is acceptable or N/A)

Additional context

Multiline sensitive output causes cli task to fail when running terraform output

First of all thanks for creating this awesome extension!

Terraform 0.15 makes it mandatory to mark output as sensitive, if it includes data that a provider marked as sensitive. However, using TerraformCLI@0 with the 'output' command blocks multiline output: 'Secrets cannot contain multiple lines'

This is a problem, as some multiline output is flagged sensitive by the provider e.g.
azurerm_kubernetes_cluster.kube_admin_config_raw
tls_private_key.private_key_pem

For now I hold off on upgrading to terraform 0.15, but I think it is a good practice to flag output as sensitive when appropriate. I'm hoping the extension will address this scenario.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.