Consumers of the module may want to control the name of the created resources, for example if they have a naming convention to follow.

Does this module work with docker-in-docker for a self-hosted agent running in Azure Container Instances?

Now that the Azure DevOps provider for Terraform is released, we can add the support in this module for adding/removing the agent pool, instead of requiring these pools to be created before the module is used.

I'm new in azure devops especially related to the aci devops agent.
May you have the video for how to run create the aci devops agent ? abit confusing from the article.

I'm trying to replace some build agents that another company was hosting for us. As they are no longer hosting said build agents for us I'm working on replacing them. I was able to get up to the point of building on the agent when I got this error message:

##[error]Unhandled: Unable to locate executable file: 'docker'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.

In researching it appears to be that I need to run some form of docker.sock. But I'm not seeing how to do that with an ACI. Is there any way to do this? Either with this module or otherwise?

The test TestDeployAzureDevOpsLinuxAgentsInVirtualNetwork is failing because it times out after 30 min. This is not related to go test framework, but seems like a timeout in Terraform / Azure. To investigate.

Hi

Thanks for this module, even though I would not use it as it is currently, I do appreciate your article and the code provided here. It helped me get started with my own implementation (all though I would prefer to use a module).

Consider using for_each instead of count when creating resources. Reasoning for this is that if I configure 10 linux or windows images and at some point, decide to remove the first then all agents are destroyed and the remaining 9 are re-created.

Just an improvement note.

Kind Regards
Jakub

I am using the provided Dockerfile and am getting this error when I try and run the image both locally and in the container.

exec ./start.sh: no such file or directory

Any idea what's going on here? And how to fix it?

I would like to see support added to the example docker file for the run one flag. https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops#run-once

I am willing to submit a pr adding this support.

While passing external resources such as resource group and other services, the create resource flag should be set to false

"create_resource_group = false"

It'd be nice if the module supported tags

ERROR: Unsupported argument in module "aci-devops-agent"
tags = {

An argument named "tags" is not expected here

It should be possible to specify Docker images hosted in a private registry. For that, the module should accept credentials and connection information as input variables.

The container_group resource in Terraform has a image_registry_credential that allows to pass this information.

#37

I have 2 terraform pipelines in azure devops:

1- provisions vnet and azure container instance and registers it as an agent pool node.
2- uses the self hosted agent pool which uses the aci from the first pipeline to provision other stuff.

The second pipeline fails when it reaches init with the following message:

##[error]Terraform command 'init' failed with exit code '1'.: Failed to get existing workspaces: containers.Client#ListBlobs: Failure sending request: StatusCode=0 -- Original Error: Get "https://xxx.blob.core.windows.net/terraform?comp=list&prefix=xxx-infra-dev.tfstateenv%253A&restype=container": dial tcp xx.xxx.xx.x:443: connect: connection timed out

this how I provision the ACI (I removed some parts from)

terraform {
  required_version = "~> 0.13"
  backend "azurerm" {}
}
provider "azurerm" {
  version                    = "~> 2.8.0"
  skip_provider_registration = true
  features {}
}


module "aci-devops-agent" {
  source                   = "Azure/aci-devops-agent/azurerm"
  resource_group_name      = var.resource_group_name
  location                 = var.location
  enable_vnet_integration  = true
  create_resource_group    = false
  vnet_resource_group_name = var.resource_group_name
  vnet_name                = local.virtual_network_name
  subnet_name              = data.azurerm_subnet.subnet["mgmt"].name

  linux_agents_configuration = {
    agent_name_prefix = "aci-${var.environment}-${var.app_name}"
    agent_pool_name   = var.agent_pool_name
    count             = 1,
    docker_image      = "jcorioland/aci-devops-agent"
    docker_tag        = "0.2-linux"
    cpu               = 1
    memory            = 4
  }

  azure_devops_org_name              = "xxx"
  azure_devops_personal_access_token = var.pat

}

and the agent is successfully detected in azure devops

Hello,

Thanks for your article which is very useful.

I saw that both, Linux or Windows docker images are more than 1,5 Go, is that an issue when running in term on top of ACI ?

Is there a way to reduce those images size ?

Azure Container Instance (container group) supports managed identities (both system and user assigned) for Linux container groups. It would be great to support this in this module.

Proposed implementation:

resource "azurerm_user_assigned_identity" "example" {
  resource_group_name = azurerm_resource_group.example.name
  location            = azurerm_resource_group.example.location

  name = "identity1"
}
resource "azurerm_user_assigned_identity" "example2" {
  resource_group_name = azurerm_resource_group.example.name
  location            = azurerm_resource_group.example.location

  name = "identity2"
}
module "aci-devops-agent" {
  source                  = "../../../"
  enable_vnet_integration = false
  create_resource_group   = true
  linux_agents_configuration = {
    agent_name_prefix = "linuxagent-${var.random_suffix}"
    count             = var.agents_count
    docker_image      = var.agent_docker_image
    docker_tag        = var.agent_docker_tag
    agent_pool_name   = var.azure_devops_pool_name
    cpu               = 1
    memory            = 4
    # array to specify existing (or created by current template) user assigned identities
    user_assigned_identity_ids = [azurerm_user_assigned_identity.example.id, data.azurerm_identity.example2.id]

    # flag to enable system assigned identity (default = false for retrocompatibility)
    use_system_assigned_identity = true
  }
  image_registry_credential = {
    username = var.docker_registry_username
    password = var.docker_registry_password
    server   = var.docker_registry_url
  }
  resource_group_name                = "rg-terraform-azure-devops-agents-e2e-tests-${var.random_suffix}"
  location                           = var.location
  azure_devops_org_name              = var.azure_devops_org_name
  azure_devops_personal_access_token = var.azure_devops_personal_access_token
}

Since we do not pass the subscription ID as an environment variable, we are getting the below error.

Error building AzureRM Client: 1 error occurred:
	* A Subscription ID must be configured when authenticating as a Service Principal using a Client Secret.
  on .terraform/modules/agent.aci-devops-agent/provider.tf line 1, in provider "azurerm":
   1: provider "azurerm" {

Could you please make the subscription ID as an optional one?

provider "azurerm" {
  subscription_id= var.ARM_SUBSCRIPTION_ID ? var.ARM_SUBSCRIPTION_ID : <ENV variable for subscriptionid>
  features {}
}

Terraform breaks for os_type variable, expect "Linux and Windows" instead of "linux and windows".
Ip address type should "Private & Public" instead of "private & public"

54. ip_address_type = var.enable_vnet_integration ? "private" : "public"
55. os_type = "linux"
56. ip_address_type = var.enable_vnet_integration ? "private" : "public"
57. os_type = "windows"

AZP_TOKEN variable is not protected and can be accessed in clear in the portal for example.
An improvement on this can be to use secure_environment_variables to create the AZP_TOKEN environment variable.

This applies for windows and linux containers instances :

When using your module I get this error

Error: Error building AzureRM Client: 1 error occurred:
* A Subscription ID must be configured when authenticating as a Service Principal using a Client Secret.
on .terraform/modules/aci-devops-agent/terraform-azurerm-aci-devops-agent-0.9/provider.tf line 1, in provider "azurerm":
1: provider "azurerm" {

Also my devops project is https://%projectname%.visualstudio.com/ wouldn't this cause an issue for your

So I tried to create the container based off your main.tf but I'm getting this error when trying to pull the image, I'm not sure if I'm configuring it wrong.

_t C:\azp\start.ps1:32 char:14

• ... $package = Invoke-RestMethod -Headers @{Authorization=("Basic $base6 ...

•             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  

  • CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:Htt
    pWebRequest) [Invoke-RestMethod], WebException
  • FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe
    ll.Commands.InvokeRestMethodCommand
    Cannot index into a null array.
    At C:\azp\start.ps1:33 char:3
• $packageUrl = $package[0].Value.downloadUrl

• + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
  + FullyQualifiedErrorId : NullArray_
  
  
  

2. Downloading and installing Azure Pipelines agent...
   Exception calling "DownloadFile" with "2" argument(s): "Value cannot be null.
   Parameter name: address"
   At C:\azp\start.ps1:40 char:36

• $wc.DownloadFile($packageUrl, "$(Get-Location)\agent.zip")

•                                ~~~~~~~~~~~~
  

  • CategoryInfo : NotSpecified: (:) [], MethodInvocationException
  • FullyQualifiedErrorId : ArgumentNullException

Expand-Archive : The path 'agent.zip' either does not exist or is not a valid
file system path.
At C:\azp\start.ps1:42 char:3

• Expand-Archive -Path "agent.zip" -DestinationPath "\azp\agent"

• + CategoryInfo          : InvalidArgument: (agent.zip:String) [Expand-Arch 
  

ive], InvalidOperationException
+ FullyQualifiedErrorId : ArchiveCmdletPathNotFound,Expand-Archive

3. Configuring Azure Pipelines agent...
   Cleanup. Removing Azure Pipelines agent...
   .\config.cmd : The term '.\config.cmd' is not recognized as the name of a
   cmdlet, function, script file, or operable program. Check the spelling of the
   name, or if a path was included, verify that the path is correct and try again.
   At C:\azp\start.ps1:68 char:5

• .\config.cmd remove --unattended `
  

• ~~~~~~~~~~~~
  

  • CategoryInfo : ObjectNotFound: (.\config.cmd:String) [], Comman
    dNotFoundException
  • FullyQualifiedErrorId : CommandNotFoundException

.\config.cmd : The term '.\config.cmd' is not recognized as the name of a
cmdlet, function, script file, or operable program. Check the spelling of the
name, or if a path was included, verify that the path is correct and try again.
At C:\azp\start.ps1:48 char:5

• .\config.cmd --unattended `
  

• ~~~~~~~~~~~~
  

  • CategoryInfo : ObjectNotFound: (.\config.cmd:String) [], Comman
    dNotFoundException
  • FullyQualifiedErrorId : CommandNotFoundException

Hi,
How can I hide confidential information from the Azure DevOps token that is displayed when executing the terraform plan?

  + resource "azurerm_container_group" "linux-container-group" {
      + fqdn                = (known after apply)
      + id                  = (known after apply)
      + ip_address          = (known after apply)
[...]

      + container {
          + commands              = (known after apply)
          + cpu                   = 1
          + environment_variables = {
              + "AZP_AGENT_NAME" = "..."
              + "AZP_POOL"       = "..."
              + "AZP_TOKEN"      = "TOKEN as plain text" # sensitive data

Currently, if you call the terraform destroy commands, all the containers instances are removed, but the agents are still present in the Azure DevOps agents pool. They are marked as offline, so no job can be scheduled on it, but still there and have to be removed manually.

We should find a way to clean up automatically when terraform destroy is called. Maybe using a local executer or something like that.

Hi there,

I have build Windows containers and pushed to ACR ok
The Terraform files build and the ACI's are running.
But the agents are not appearing in the Azure DevOps Agent pool

My Terraform Code:

module "aci-devops-agent" {
source = "Azure/aci-devops-agent/azurerm"
resource_group_name = "rg-aci-devops"
location = "uksouth"
create_resource_group = true

enable_vnet_integration = false

windows_agents_configuration = {
agent_name_prefix = "windows-agent"
agent_pool_name   = "Azure_Docker_Pool_TestLabs"
count             = 2,
docker_image      = "acrnewasm.azurecr.io/servercore"
docker_tag        = "v1"
cpu               = 1
memory            = 4

}

azure_devops_org_name = "ORGNAME"
azure_devops_personal_access_token = "PATTOKEN"
}

Windows containers cannot be added to container groups within vNets at present (https://docs.microsoft.com/en-us/azure/container-instances/container-instances-virtual-network-concepts#other-limitations).

It would be useful to include this in the README, as it cost me some time to implement the module before receiving the error message that Windows containers arent supported yet