Giter VIP home page Giter VIP logo

azure-devtestlab's Introduction

Azure Lab Services Community

This is a public, community-contributed repository that contains:

  • Artifacts for Azure DevTest Labs: These artifacts can be deployed onto virtual machines in Azure DevTest Labs via the Azure Portal or through scripts.

  • Environments for Azure DevTest Labs: An environment is a collection of resources defined as ARM templates, which represent the Public Environment repository in Azure DevTest Labs and can be accessed via the Azure Portal or through scripts.

  • Samples for both Azure DevTest Labs and Classroom Labs: These samples vary from QuickStart Templates, to SDK samples and scripts that can be used to facilitate outside-the-box scenarios. They are used to demonstrate how Azure Lab Services can be integrated in automation scenarios.

  • Azure Pipelines Tasks for Azure DevTest Labs: This is the source code used to author Azure Pipelines tasks specifically geared to demonstrate use of Azure DevTest Labs in builds/releases. Refer to Azure DevTest Labs Tasks for more information and to install.

Azure DevTest Labs PowerShell Module

Az.DevTestLabs is a PowerShell module to simplify the management of Azure DevTest Labs. It provides composable functions to create, query, update and delete labs, VMs, Custom Images and Environments. The source can be located under /samples/DevTestLabs/Modules/Library/.

Azure Lab Services PowerShell Module

Az.LabServices is a PowerShell module to simplify the management of Azure Lab Services. It provides composable functions to create, query, update and delete lab accounts, labs, VMs and Images. The source can be located under /samples/ClassroomLabs/Modules/Library/.

Azure Lab Services April 2022 update (preview)

If you are using the update (with lab plans) the Lab Services module is available in the PowerShell gallery. There is a dedicated Lab Services repository with samples for the updated modules and the improved service.

Contributions

Contributions are encouraged and welcome. Please refer to the respective sections (e.g. Artifacts, Environments, Tasks) for more details on the respective process to follow.

Open Source Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azure-devtestlab's People

Contributors

andremarques023 avatar azabbasi avatar d3r3kk avatar dependabot[bot] avatar drwill-ms avatar emaher avatar ericame avatar francedot avatar grwann avatar gxy001 avatar iouri-s avatar jaymarf avatar joerage avatar leovms avatar lucabol avatar markusheiliger avatar michaelwstark avatar mithunshanbhag avatar nicolehaugen avatar petehauge avatar ramankumarlive avatar rchaganti avatar rogerbestmsft avatar ryanvog avatar rzander avatar shichaoh avatar togores-msft avatar varshagangu9 avatar vinagesh avatar yvand avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

azure-devtestlab's Issues

Adding GitHub based Repository only uses master branch

Hello,

I've been trying to add a custom repo, using a branch of my fork of this repo. The "branchRef" property seems to be ignored and master is always shown. It also validates whatever branch I provide, via ARM Template or the portal.

Can anybody else replicate this?

Marcus

Constraint of number of VMs per dev test lab user getting bypassed when using template from template repository

Hi, I am facing the following issue :

I created Devtest Lab with a template repository following the steps mentioned in https://github.com/maniSbindra/univ-devtest-lab . I modified the parameter file to allow only one vm per devtest lab user. Currently only one environment template exists in the template repository which creates a single Ubuntu VM once provisioned.

After the devtest lab was created I gave access to an user with role devtestlabuser.

The issue is that using the portal from dev test lab the user is able to add multiple environments using the template from the template repository , and a VM is created each time, and this bypasses the constraint of only one VM per devtestlab user.

Please let me know if further clarification is needed regarding the issue.

AzureDevTestLabsCreateVM - Deployment not found

Getting this error every time I deploy now, worked fine yesterday, used to happen sporadically but is now consistent

Any ideas?

2017-06-28T08:51:10.5434745Z Executing the powershell script: F:\Agents\RELAZTEST\1\tasks\AzureDevTestLabsCreateVM\1.0.13\task.ps1
 
2017-06-28T08:51:10.8354957Z Looking for Azure PowerShell module at C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Azure.psd1
 
2017-06-28T08:51:18.3660246Z AzurePSCmdletsVersion= 1.5.0
 
2017-06-28T08:51:18.5740378Z Get-ServiceEndpoint -Name 5b2c6382-7b8b-4585-a242-b607cc2155e7 -Context Microsoft.TeamFoundation.DistributedTask.Agent.Worker.Common.TaskContext
 
2017-06-28T08:51:18.6000427Z tenantId= ********
 
2017-06-28T08:51:18.6000427Z azureSubscriptionId= 75a7f9a8-a64b-445b-9faf-ad081418ffa2
 
2017-06-28T08:51:18.6010410Z azureSubscriptionName= Development
 
2017-06-28T08:51:19.2580854Z Add-AzureRMAccount -ServicePrincipal -Tenant ******** -Credential System.Management.Automation.PSCredential
 
2017-06-28T08:51:20.8232299Z Select-AzureRMSubscription -SubscriptionId 75a7f9a8-a64b-445b-9faf-ad081418ffa2 -tenantId ********
 
2017-06-28T08:51:21.1782227Z Starting Azure DevTest Labs Create VM Task
 
2017-06-28T08:51:21.1802214Z Task called with the following parameters:
 
2017-06-28T08:51:21.1802214Z   ConnectedServiceName = 5b2c6382-7b8b-4585-a242-b607cc2155e7
 
2017-06-28T08:51:21.1812218Z   LabId = /subscriptions/75a7f9a8-a64b-445b-9faf-ad081418ffa2/resourceGroups/AZ-SMOKETESTRG192155/providers/Microsoft.DevTestLab/labs/AZ-SMOKETEST
 
2017-06-28T08:51:21.1812218Z   TemplateName = F:\BuildTools\ARMTemplates\win10vpn.json
 
2017-06-28T08:51:21.1822208Z   TemplateParameters = -newVMName 'SmokeA-2590' -userName '********' -password (ConvertTo-SecureString -String '********' -AsPlainText -Force)
 
2017-06-28T08:51:21.1822208Z   OutputResourceId = labVMId
 
2017-06-28T08:51:21.1842206Z Validating input parameters
 
2017-06-28T08:51:21.2932319Z Fetching lab 'AZ-SMOKETEST'
 
2017-06-28T08:51:26.1295675Z Preparing deployment parameters
 
2017-06-28T08:51:26.1445684Z Invoking deployment with the following parameters:
 
2017-06-28T08:51:26.1455714Z   DeploymentName = Dtl441559d88c4942bda80938008eb00da5
 
2017-06-28T08:51:26.1455714Z   ResourceGroupName = az-smoketestrg192155
 
2017-06-28T08:51:26.1465724Z   TemplateFile = F:\BuildTools\ARMTemplates\win10vpn.json
 
2017-06-28T08:51:26.1465724Z   TemplateParameters = -labName 'AZ-SMOKETEST' -newVMName 'SmokeA-2590' -userName '********' -password (ConvertTo-SecureString -String '********' -AsPlainText -Force)
 
2017-06-28T08:52:00.9600589Z ##[error]The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: 
 
2017-06-28T08:52:00.9610197Z ##[error]Deployment 'Dtl441559d88c4942bda80938008eb00da5' could not be found.

Artifact windows-ConfigurationManagerCurrentBranch(EVAL) is broken

it seems that the ConfigMgr Artifact is not working anymore...
Artifact returns with the following error:
Unable to generate valid template for the artifact - Configuration Manager Current Branch with error(s) - Deployment template validation failed: 'The template variable '_commandToExecute' is not valid: Unable to parse template language expression 'concat('powershell.exe -ExecutionPolicy bypass -File SetupCMCB.ps1', ' -SiteCode ', parameters('SiteCode'), ' -SiteName ', , '", parameters('SiteName')), '"'': expected token 'Identifier' and actual 'Comma'. Please see https://aka.ms/arm-template-expressions for usage details..'.

Template Parameters parsing doesn't work

I am trying to make use of the expiration date feature in Dev Test labs. However, rather than setting a fixed date, I wanted to set a sliding date (14 days from now).

I wanted to pass this into the Create DevTest VM task via the "Template Parameters". I added the parameter into my json ARM template and expected that i could populate it from the Template Parameters option within the create task, like so:

-newVMName '$(Release.ReleaseName)' -userName '$(vmUsername)' -password (ConvertTo-SecureString -String '$(vmPassword)' -AsPlainText -Force) -expirationDate ((Get-Date).AddDays(14).ToString("o"))

Is this supported?

windows-selenium Artifact is failing

WinRM is already set up to receive requests on this computer.
WinRM is already set up for remote management on this computer.
[06/06/2017 07:13:08] ========== Configuration ==========
[06/06/2017 07:13:08] RawPackagesList : firefox,googlechrome,selenium-chrome-driver,selenium-ie-driver
[06/06/2017 07:13:08] ========== Configuration ==========
[06/06/2017 07:13:08] Installing Chocolatey ...
WARNING: It's very likely you will need to close and reopen your shell
before you can use choco.
WARNING: You can safely ignore errors related to missing log files when
upgrading from a version of Chocolatey less than 0.9.9.
'Batch file could not be found' is also safe to ignore.
'The system cannot find the file specified' - also safe.
[06/06/2017 07:13:14] Success.
[06/06/2017 07:13:14] Installing package: firefox ...
[06/06/2017 07:13:51] Success.
[06/06/2017 07:13:51] Installing package: googlechrome ...
ERROR: Installation failed. Please see the chocolatey logs in %ALLUSERSPROFILE%\chocolatey\logs folder for details.

Mostly something to do with the googlechrome package.

PFA the chocolately log file:
chocolatey.txt

Azure DTL - The selected virtual machine does not support creating a custom image.

I am using a custom template to provision a VM with data disk in dev test lab. When i browse to the machine in DTL and click Create Custom Image from the blade, it shows me the following information message...

The selected virtual machine does not support creating a custom image.

image

Do machines creatd from custom images not support custom image functionality?

Issue with creating image when other artifacts have been used

I am deploying a VM with a number of artifacts. I am then trying to create an image of the VM. When I do that, another artifact is attempted to run which syspreps the VM. That artifact fails with the error:

Multiple VMExtensions per handler not supported for OS type 'Windows'. VMExtension 'customScriptArtifact-183362431' with handler 'Microsoft.Compute.CustomScriptExtension' already added or specified in input.

I have read up on this and found the following workaround:

https://blogs.technet.microsoft.com/meamcs/2016/01/30/run-two-powershell-scripts-on-a-same-vm-through-custom-script-extension-at-different-stage-of-deployment-in-arm/

This article suggests that I run the following command:

Remove-AzurermVMCustomScriptExtension -ResourceGroupName $ResourceGroupName -VMName $CustVMname –Name $customscriptname -Force

This is fine and I can work this into my workflow, but my question is:

How do I know what the $customscriptname parameter should be to pass into the command above?

I am using this in the context of the ImageFactory, so it's not just a case of manually looking up the extension that was used and then removing it. It needs to be automatable. In this case, the only extension applied to the VM is named: WinRMCustomScriptExtension

Logs from the Artifact Extension are truncated

When I go to the Artifact status blade in the portal, the Extension Message seems to be truncated.

It would be really useful to see the full logs here.

I am executing a lengthy powershell script and am trying to output to here, but I only get the last 79 lines of the logs.

Error when attempting to create Azure DevTestLab Create VM function: Get-AzureDtlLab

Validating input parameters
2017-03-02T17:23:09.3715367Z
2017-03-02T17:23:09.3715367Z
2017-03-02T17:23:09.5287467Z Fetching lab 'XXXXXXXXX'
2017-03-02T17:23:09.5287467Z
2017-03-02T17:23:09.5287467Z
2017-03-02T17:23:10.0121846Z ##[error]System.Management.Automation.ActionPreferenceStopException: The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop:
2017-03-02T17:23:10.0121846Z The term 'Get-AzureRmResource' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
2017-03-02T17:23:10.0121846Z at System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable input)
2017-03-02T17:23:10.0121846Z at System.Management.Automation.PowerShell.Worker.ConstructPipelineAndDoWork(Runspace rs, Boolean performSyncInvoke)
2017-03-02T17:23:10.0121846Z at System.Management.Automation.PowerShell.Worker.CreateRunspaceIfNeededAndDoWork(Runspace rsToUse, Boolean isSync)
2017-03-02T17:23:10.0121846Z at System.Management.Automation.PowerShell.CoreInvokeHelper[TInput,TOutput](PSDataCollection1 input, PSDataCollection1 output, PSInvocationSettings settings)
2017-03-02T17:23:10.0121846Z at System.Management.Automation.PowerShell.CoreInvoke[TInput,TOutput](PSDataCollection1 input, PSDataCollection1 output, PSInvocationSettings settings)
2017-03-02T17:23:10.0121846Z at Microsoft.TeamFoundation.DistributedTask.Handlers.LegacyVSTSPowerShellHost.VSTSPowerShellHost.Main(String[] args)
2017-03-02T17:23:10.0434371Z ##[error]LegacyVSTSPowerShellHost.exe completed with return code: -1.
2017-03-02T17:23:10.0434371Z

Any ideas?
The error is occuring in Get-AzureDtlLab when it calls Get-AzureRmResource.

Is this a dataissue - we are passing the wrong LabId?

Thanks for any help with this....?

Artifact installation fails when ARM Template parameter type is bool

For example, the Artifactfile.json below will not work because of the parameter of type bool

{
  "$schema": "https://raw.githubusercontent.com/Azure/azure-devtestlab/master/schemas/2015-01-01/dtlArtifacts.json",
  "title": "Bool test",
  "description": "Uses the bool parameter type",
  "tags": [
    "Windows"
  ],
  "parameters": { 
      "succeed": {
          "type": "bool",
          "displayName": "True or false?",
          "description": "True or false?"
      }
   },
  "targetOsType": "Windows",
  "runCommand": {
    "commandToExecute": "[concat('powershell.exe -ExecutionPolicy bypass \"& ./Succeed.ps1 ', parameters('succeed'), '\"')]"
  }
}

Artifact installation fails with this error message:

Unable to generate valid template for the artifact - Bool test with error(s) - Deployment template validation failed: 'The provided value for the template parameter 'succeed' at line '13' and column '21' is not valid.'.

And this call stack:

Error when applying artifacts: Microsoft.DevTestLab.Common.Validation.ResourceException 
at Microsoft.DevTestLab.VirtualMachine.Provider.Utils.ArtifactsInstaller.<GenerateArtifactsArmTemplates>d__18.MoveNext() 
in V:\DTL-CICD-1-1\_work\5\s\main\src\Dtc\VirtualMachine\VirtualMachine.Provider\Utils\ArtifactsInstaller.cs:line 349  
at Microsoft.DevTestLab.VirtualMachine.Provider.Actions.ApplyArtifacts.ApplyArtifactsExecutor.<ApplyArtifactsToVirtualMachine>d__7.MoveNext() 
in V:\DTL-CICD-1-1\_work\5\s\main\src\Dtc\VirtualMachine\VirtualMachine.Provider\Actions\ApplyArtifacts\ApplyArtifactsExecutor.cs:line 79  

Artifact for installing VSTS Build Agent doesn't work

There appears to be 2 issues with this artifact

  1. There doesn't appear to be any parameter checking, it's difficult to get the parameters correct. IE: we put in the full URL instead of just the account name the first time and took some time to figure it out. Would be better if the artifact could handle the parameters a bit better
  2. Even with the correct parameters, get an error: "An error occurred while sending the request" and "Agent configuration failed with exit code: 1". No real information on why it failed.

DownloadFiles: Encountered an error while downloading files

Hello guys,
I got this error:

DownloadFiles: Encountered an error while downloading files. Cancelling download tasks. Exception: Microsoft.WindowsAzure.Storage.StorageException: The remote server returned an error: (404) Not Found.

What I did, step by step:

  1. I have three Storage Accounts in my DevTest lab. In one of them I put my PowerShell script (DevTest Labs > dtl name > storage account name > Blob service > artifacts)
  2. Create new VM with artifact "Run PowerShell" (Use this to run a script using PowerShell )

Result:
artifact - microsoft azure

where I made mistakes?

Google Chrome on Windows 2016

Chrome installation fails on Windows Server 2016. It also references the googlechrome-allusers package in Chocolatey that is starting to get very old, and not approved.

After an hour and more it times out, with log saying:

[02/27/2017 09:17:06] ========== Configuration ==========
[02/27/2017 09:17:07] RawPackagesList : googlechrome-allusers
[02/27/2017 09:17:07] ========== Configuration ==========
[02/27/2017 09:17:07] Installing Chocolatey ...
WARNING: It's very likely you will need to close and reopen your shell 
  before you can use choco.
WARNING: You can safely ignore errors related to missing log files when 
  upgrading from a version of Chocolatey less than 0.9.9. 
  'Batch file could not be found' is also safe to ignore. 
  'The system cannot find the file specified' - also safe.
PATH environment variable does not have C:\ProgramData\chocolatey\bin in it. Adding...
[02/27/2017 09:17:15] Success.
[02/27/2017 09:17:15] Installing package: googlechrome-allusers ...

windows-download-sas-blob only supports anonymous access

Unless I'm missing something, the windows-download-sas-blob artifact only supports anonymous access to blob storage. It would be great if it supported SAS or account key access.

Context

I'm trying to follow this guide http://www.visualstudiogeeks.com/blog/DevOps/Use-VSTS-ReleaseManagement-to-Deploy-and-Test-in-AzureDevTestLabs but it seems to me the windows-download-sas-blob will only work if the container has public access enabled. I'm not sure if I'm missing something though!

Error - Run Powershell artifact commandToExecute is present in both protected and public sections

I just cannot figure out how to run a script using the Run PowerShell artifact.

I have a simple PowerShell script to unzip a file and it works successfully when I run the following command locally on the VM
powershell.exe -file unzipPS40.ps1 d:\sonarqube.zip c:\sonarqube

The unzipPS40.ps1 is hosted on my GitHub account and I've tried every combination I can think of to get the Run Powershell artifact to run a script by using UNC and relative paths, with and without parameters for the script, but all I get in the Azure portal is an unhelpful 'Error while installing artifacts' 'Failed to apply artifacts to VM'

Could you improve the hints with the syntax required?

The VM is running Windows Server 2012 R2 with PowerShell 4.0

After much digging I found this log which is hopefully helpful for you:

[2668+00000001] [05/28/2016 10:29:41.27] [INFO] HandlerSettings = ProtectedSettingsCertThumbprint: 36F9******FCB8, ProtectedSettings: {nnkL****F2g3xY=}, PublicSettings: {FileUris: [https://raw.githubusercontent.com/*******/powershell/master/unzipPS40.ps1], CommandToExecute: powershell.exe -ExecutionPolicy bypass -File unzipPS40.ps1 d:\sonarqube.zip c:\artifactdownload}
[2668+00000001] [05/28/2016 10:29:41.51] [FATAL] Invalid Configuration - CommandToExecute is present in both protected and public configuration sections; it must only be specified in one section.

Error when deploying nested template using cross resource group deployment

I'm attempting to deploy a template containing a variety of nested templates, one of which is using the new cross resource group deployment option to update resources as part of the deployment. When I deploy manually using PS it completes fine but when attempting to deploy from DevTest Labs I get the following error:

Deployment template validation failed: 'The template resource 'UpdateVNetDNS' at line '175' and column '10' is invalid. 
The api-version '2016-09-01' used to deploy the template does not support 'ResourceGroup' property. Please use api-version '2017-05-10' or later to deploy the template. 
Please see https://aka.ms/arm-template/#resources for usage details.'.

Is there any way around this?

Windows TFS Test Agent like artifact (vstf_testagent)

Hello
Would be great if you add vstf_testagent like artifact

In my workflow I use DevTest Labs tasks namely your task "AzureDtlCreateVM" and see that creating a new VM takes on average 9 minutes, from my custom VHD. It is OK

What is not OK, is how long TFS's Test Agent installation takes. From what I got it is around 13 minutes on a good day. and up to 19 minutes on a bad one. This pushes overall provisioning time to at least 21 minutes!

I think if I re-build my custom VHD with TestAgent pre-installed. It will help me to reduce the time for download and install Test Agent Of course I can do it myself,
but I think if you add such artifact in your repository it will be much better and useful

VSTS Deployment Group - Fails to apply artifact on DTL VM creation

When attempting to create a VM within DevTest Labs - pre-conifgured to apply the VSTS Deployment group artifact, we're getting a failure message. However, if I was to create the VM and apply the artifact afterwards manually, it would be installed correctly. Do you know why this would be?

Also, if the first issue above is able to be resolved, then how would we go about supplying the correct access token to the artifact during the release process (VM creation) as I believe the access token to refresh after a certain period of time. Does this make sense?

Thanks.

Artifact stuck on "installing"

I am trying to use the windows-download-sas-blob artifact to download a file from a URL, but it is stuck on "Installing". The logs are all completely empty too.

I have checked the following:

  • Portal: Artifact blade just says "installing" and has no messages. When it does eventually fail, there are no errors displayed.
  • Filesystem: C:\WindowsAzure\Logs\Plugins\Microsoft.Compute.CustomScriptExtension\1.8\commandexecution_1.txt shows the following:
[11/11/2016 01:58:17.65] Executing: C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.8\enable.cmd 
[11/11/2016 01:58:19.29] Execution Complete.
######
Execution Output:
C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.8>if not exist RuntimeSettings\*.settings exit /b -2 
C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.8>start /B cmd /C bin\CustomScriptHandler.exe "enable" 


Execution Error:

######
Command C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.8\enable.cmd of Microsoft.Compute.CustomScriptExtension has exited with Exit code: 0
Plugin (name: Microsoft.Compute.CustomScriptExtension, version: 1.8) completed successfully.

Neither of which are very helpful.

I have tested the URL works and is accessible.

ConfigMgr - Deployment Issues

Hello,

I need to simulate applications deployments with devtestlabs COnfigMgr Scenario to validate my package/Applications.
But the content can't be deployed to the client computer (on the same Vnet with NSG (Allow All Any=Any)

Do you have the same issue?

Thanks,

Yoann

Disconnect between DevTest Lab and the Azure File Copy VSTS task

I'm not sure whether this should be in this repo or in the https://github.com/Microsoft/vsts-tasks.

Scenario

  • I am using DevTest lab to create 5 machines.
  • I am using the Advanced Settings -> Number of instances feature.
  • This creates each VM in a separate Resource Group.
  • I am then trying to deploy files to all of the VMs using the VSTS task Azure File Copy. This task requires a resource group that contains all the machines. DevTest lab doesn't give me one, so I can't use this task.

Is there any way around this other than using the File Copy task multiple times (once for each machine)?

I also tried moving all the VMs into a single resource group, but I get an error saying that managed disks can't be moved.

Updating a system during Artifact execution can 'hang' Azure DevTest agent

When working with system package management tools, a bug with the azure tools (or rather, a bug with the process being reset and not recovering from that reset) was encountered.

When updating the system using yum update --assumeyes --quiet command from within an Azure DevTest artifact (specifically the yum artifact - see my pull request #74 ), the Windows Azure Linux Agent gets restarted in specific instances (probably a package that the Windows Azure Linux Agent service depends on gets updated, and therefore a restart of the agent is performed during the command).

The bug is that the agent after being restarted doesn't pick up where it left off and finish the rest of the artifact's process. The symptom is on the Azure DevTest Labs blade, the VM with the artifact applied to it never finishes installing all of the artifacts it was given, and there doesn't seem to be a way to reset the status of the VM.

The log messages from /var/log/messages that point to the reset of the Azure Linux Agent Service:
Apr 29 06:12:39 localhost systemd: Stopping Windows Azure Linux Agent...
Apr 29 06:12:39 localhost systemd: Started Windows Azure Linux Agent.
Apr 29 06:12:39 localhost systemd: Starting Windows Azure Linux Agent...

Procedure:

  1. Log into the Azure portal and navigate to Azure DevTest Labs, and enter an existing lab instance (create one if one does not exist).
  2. Click on '+ Lab VM' button at the top of the Azure DevTest Labs blade.
  3. Choose the Linux image named 'CentOS-based 7.2'.
  4. Fill in the settings for the new Lab VM, username, password, etc...
  5. Click on the 'Artifacts >' button in the Lab VM blade, opening the Add Artifacts blade.
  6. Choose the Yum Artifact
  7. Enter 'git vim' (no quotes) in the 'Packages' text field.
  8. Ensure 'true' (no quotes) is set in the 'Run Update First' text field.
  9. Leave the 'Additional Options' field blank.
  10. Click Add, the Add Artifact blade will close
  11. In the Add Artifacts blade, click OK
  12. In the Lab VM blade, click Create
  13. The lab Vm will begin being created.

Expected Result:

The lab VM will be created, and the packages updated on the machine with the desired extra packages installed.

Observed Result:

The lab VM will never progress past the status 'Installing 1/1 Artifacts'

Further Observations:

Log into the machine once it gets to the status 'Installing 1/1 Artifacts'

Get root priviledges on the machine

sudo su

List the latest messages in the system log and pull out the Azure DevTest Yum Artifact messages:

cat /var/log/messages | grep AZDEVTST
...
AZDEVTST_YUMPKG[n]: Updating the system using yum update. Command line being used is: 'yum --assumeyes --quiet update'.
...this means that the last message in the log thus far is that yum update was being executed on the machine.

List the latest messages in the system log and pull out Azure Service messages

cat /var/log/messages | grep "Azure Linux Agent"
Apr 29 06:44:06 localhost systemd: Started Windows Azure Linux Agent.
Apr 29 06:44:06 localhost systemd: Starting Windows Azure Linux Agent...
Apr 29 06:47:35 localhost systemd: Stopping Windows Azure Linux Agent...
Apr 29 06:47:35 localhost systemd: Started Windows Azure Linux Agent.
Apr 29 06:47:35 localhost systemd: Starting Windows Azure Linux Agent...
...note that the Azure Linux Agent was started (at system startup presumably), stopped, then started again!

List all the messages and note where the stop/start cycle of the Azure Linux Agent occurs in relation to the yum update task run by the Yum Artifact.

linux-swapfile fails on CentOS-based 7.2

I think it's failing because fallocate is creating a file with holes:
https://bugzilla.redhat.com/show_bug.cgi?id=1129205

However, the error message from swapon -fv is not consistent with that bug report. The error I get from swapon is below.

$ sudo swapon -fv /mnt/ZWQ1NGUzM2ZmMmE3OWVmOWI1OTYyOTNi
swapon /mnt/ZWQ1NGUzM2ZmMmE3OWVmOWI1OTYyOTNi
swapon: /mnt/ZWQ1NGUzM2ZmMmE3OWVmOWI1OTYyOTNi: found swap signature: version 1, page-size 4, same byte order
swapon: /mnt/ZWQ1NGUzM2ZmMmE3OWVmOWI1OTYyOTNi: pagesize=4096, swapsize=2147483648, devsize=2147483648
swapon: /mnt/ZWQ1NGUzM2ZmMmE3OWVmOWI1OTYyOTNi: swapon failed: Invalid argument

problems with AD Join

I have a Windows 2012 R2 machine I'm trying to get to join a domain using the AD Join artifact. I've used this artifact in the past with Win10 successfully, but no matter what I'm using for my username/password combo, I'm getting the following:

"The resource operation completed with terminal provisioning state 'Failed'. VM has reported a failure when processing extension 'customScriptArtifact-1278997910'. Error message: "Failed to load settings from runtime settings file."."

Add ability to transfer VHD as part of ARM Template deployment

Would it be possible to add the ability to transfer a VHD to the uploads folder as part of the ARM template. Ie. Specify a URI to a VHD in another storage account to transfer over.

Otherwise I nee to deploy 1 ARM template to create the lab, execute some powershell to transfer a VHD, then deploy another ARM template to create a custom image from the uploaded VHD.

linux-vsts-build-agent Artifact?

Any plans to expand on the windows-vsts-build-agent Artifact to also include a Linux variation?

Was using Ubuntu machines from a DTL to host VSTS Build Agents for Docker builds, and was sad to see that there wasn't a Linux option to automatically configure the VSTS agent. It wasn't too bad to setup by hand, but would have been lovely to use a DTL Artifact.

Could probably usethe VSTS section from the Azure Quickstart Template's vsts-fullbuild-ubuntu-vm as a starting point.

Thanks!

Security Model is broken!

And yet again the security model in Dev Test Labs is broken. It seems a recent roll out has affected users that have admin permissions to the lab through custom RBAC roles. When the user tries to start a VM that has been auto shut down by policy the following error message shows up...

Failed to start virtual machine 'AZSU-D-DTL1-005'. Error: The client 'Tarun.Arora2@xxxcom' with object id '4f95944a-e0e0-4749-b14d-ea759988c730' does not have authorization to perform action 'Microsoft.Compute/virtualMachines/start/action' over scope '/subscriptions/xxx/resourceGroups/azsu-dtlab-devtest-xxxx-001-Dev03-941230/providers/Microsoft.Compute/virtualMachines/AZSU-D-DTL1-005'.

This is really annoying as this sort of issue has happened in the past and has happened again.

Improve artifact output when choco install fails

The output received when an artifact fails to install a choco package does not contain enough details for the owner to take action. For example, a user applied the windows-ssms artifact from the public repo. When it failed the console output looked like this:

WinRM is already set up to receive requests on this computer.
WinRM is already set up for remote management on this computer.
[05/10/2017 13:19:03] ========== Configuration ==========
[05/10/2017 13:19:03] RawPackagesList : sql-server-management-studio
[05/10/2017 13:19:03] ========== Configuration ==========
[05/10/2017 13:19:03] Installing Chocolatey ...
WARNING: It's very likely you will need to close and reopen your shell
  before you can use choco.
WARNING: You can safely ignore errors related to missing log files when
  upgrading from a version of Chocolatey less than 0.9.9.
  'Batch file could not be found' is also safe to ignore.
  'The system cannot find the file specified' - also safe.
[05/10/2017 13:19:08] Success.
[05/10/2017 13:19:08] Installing package: sql-server-management-studio ...
ERROR: Installation failed. Please see the chocolatey logs in %ALLUSERSPROFILE%\chocolatey\logs folder for details.

The user then has to log into the VM and search through the logs folder to determine why the artifact failed. It would be nice if the artifact output were more actionable and said something like "Install is not supported on Win2008" or "Install failed because SSMS is already installed".

The choco package logs contain the details users need to know. It would be nice if the artifacts surfaced this to users more conveniently.

The artifact authoring experience is terrible.

I've spent quite a bit of time authoring Azure RM templates, VSO build tasks and DevTest Lab artifacts. When I compare and contrast the experiences, the DTL artifact authoring is the worst of the three. Here is some (hopefully actionable) feedback:

Artifact manifest

Support more parameter types: Looks like we only support string and securestring types for parameters. Can we please add support for other types (int, bool, object, array etc)?

Support static dropdown lists for parameters: Self explanatory.

Support dynamic dropdown lists for parameters: Similar to VSO tasks, we should provide dynamic dropdown lists for parameters. VSO dynamic dropdowns allow you to call REST endpoints and bind the result to the underlying list. This functionality would be tremendously useful for DevTest Lab artifacts (E.g. You can query the Chocolatey/WebPI catalog and populate your dropdown with list of available packages, versions, skus etc). See example of a VSO dynamic dropdown here.

Support regex or custom validators for string parameters: Currently, when using string parameters we can only avail of "allowEmpty" and "defaultValue" fields. Would be great if we can specify regex and other custom validators (e.g. string length must meet some conditions etc) in the artifactfile.json.

Built-in helper functions: Similar to Azure RM template functions (here) we should provide some built-in helper functions that could be used in artifactfile.json (e.g. function to url-encode a string, find-replace a substring etc).

Artifact scripting

Powershell named parameters: It is not possible to call named parameters in a powershell script, if the parameter values (usually supplied by the lab user) contains a quote or a whitespace.

Running as elevated admin: Artifact authors should not have to write code to run their artifacts in the context of an elevated admin (this is what most chocolatey based artifacts are doing. See example here). Would be great if the artifactfile.json exposed some field (say 'RunElevated'), which when used, causes the artifact to automatically run elevated.

Artifact execution

Indicating "critical" errors: As of today, artifact execution errors are treated as non-critical. If one artifact fails, it does not stop execution of other artifacts in the list. Artifact authors (and/or lab users) need a mechanism to mark an artifact failure "Critical". E.g. If installation of VS or SQL fails, then don't bother installing the other artifacts in the list (abort execution of all subsequent artifacts). This is tremendously useful for long-running machine prep scenarios (e.g. dev machine prep involves installation of VS, cloning repos, installing editors & other tools etc).

Reboot after execution: Some installers exit with 3010 code, which indicates a successful installation, but a reboot is required. There should be a field (say 'RebootIfSuccessful'), which when used, forces the machine to reboot after the artifact executes.

Checkin process(es)

Bot/Auto validation on checkins and submitted PRs: We should auto-validate all PRs and checkin to the artifact repo master (license agreement check, schema checks for artifactfile.json, running unit tests etc). This is standard practice for many azure repos (E.g. The Azure quickstart gallery runs a travis/jenkins bot on every PR and checkin. See example here). I accidentally checked in a regression (wrong casing for artifactfile.json). Issues like these are hard to diagnose and should be caught by the auto-validation process.

Need mandatory readme.md for artifacts: All artifacts should have a mandatory readme.md (see example here). On the portal's 'Add artifact' blade, it would be useful to have a hyperlink to the artifact’s readme file. E.g. http://imgur.com/lt3f5pX

Miscellaneous

Documentation: Where does the most up to date documentation on artifact authoring reside? On MSDN? Or on github? Both these set of docs seem to be a bit out of sync.

Custom log files: What is the "recommended" location for generating custom log files (via the artifact's powershell script)? I currently log to this location: %SystemDrive%\Packages\Plugins\Microsoft.Compute.CustomScriptExtension[version]\Downloads.

Custom VM with 2 NIC

Hi,
I'm testing a custom VM over DevTest Labs. I need a way to add a secondary NIC to my VM. Is that even possible ?

Thanks!

Custom Image Name Validation

Has something changed with the custom image name validation when creating a custom image?

I am using the ImageFactory and I am getting an error when running "SnapImageFromVM.ps1". I now get an error:

The entity name 'name' is invalid according to its validation rule: ^[^_\W][\w-._]{0,79}(?<![-.])$.

I am also no longer able to create custom images through the portal with spaces, commas and brackets.

Have the validation rules changed?

Accessing secrets from ARM templates doesn't work for me

I am trying to access secrets in the ARM template by following the guide here:

https://azure.microsoft.com/en-us/updates/azure-devtest-labs-keep-your-secrets-safe-and-easy-to-use-with-the-new-personal-secret-store/

It says:

In addition to creating VMs and using artifacts, you can save your secret name with formulas and Azure Resource Manager templates. Here's an example of how it's used in a Resource Manager template to provision a new lab VM

However it does not seem to work for me.

I am trying to use the windows-vsts-download-and-run-script artifact. I am trying to pull the Personal Access Token from the store as follows:

"Download_VSTS_Drop_and_Run_Script.PersonalAccessToken": {
      "type": "securestring",
      "defaultValue": "[[[VSTS-PAT]]"
    },

NB. The portal's ARM template outputs: "[[[VSTS-PAT]]", but the article above shows "[[VSTS-PAT]]". I have tried both and neither work.

Unable to use Artefacts from Custom Repository

When using Artefacts from a custom Repository (copied exact the same files as hosted here) will generate an error: "Resource Id provided is not in correct format for property ArtifactId."

Have I missed something in the documentation ?

Access denied when installing chocolately package

I'm trying to use the chocolately artifact to install the powershell chocolately package.

The following logs are found in the event log:

Windows update could not be installed because of error 2147942405 "Access is denied." (Command line: ""C:\Windows\System32\wusa.exe" "C:\Users\artifactInstaller\AppData\Local\Temp\chocolatey\PowerShell\5.1.14409.20170510\Win8.1AndW2K12R2-KB3191564-x64.msu" /quiet /norestart /log:"C:\Users\artifactInstaller\AppData\Local\Temp\chocolatey\PowerShell-Install-201705230841.evtx" ")

Looks like a permissions issue. Are there known issues?

BYOT - How to install Artifacts?

I can't figure out how I can use artifacts when I am using my own ARM template. Is it possible?

I am currently doing domain join, installing BGInfo and running a custom powershell script as a custom extension. I want to now install the VSTS Agent on this host, how can i do this in my ARM template?

Tarun

image factory breaking change to Devtest labs custom images

I've been using the Image Factory scripts to run my own custom image factory.
As of 14th June. DevTest labs custom images no longer create a vhd in the lab storage account. They seem to have been switched over entirely to the managed vm image.

As a result the distribute images script in the image factory example fails to get a sourceVHDLocation for copying. (.Properties.Vhd.ImageName)

As far as I understand support for copying managed images across subscriptions is on the roadmap but not available. Any ideas how to fix this in the mean time?

Windows-clone-git artifact doesn't work, but reports success

Output shows:
Could not find a part of the path 'C:\Packages\Plugins\Microsoft.Compute.Custom
ScriptExtension\1.8\Downloads\3\iourisnew.131238248337269027\windows-clone-g587
395211\scripts\GitEnlister-2016-11-17-03-01-30\Logs\GitEnlister.log'.

This is because log folder is created after we log.

Script returns success because we try to log from "catch" which throws again since the log folder is not created.

When these two issues are fixed, we get a new error:

Cannot find drive. A drive with the name ''C' does not exist.

That is because we use apostrophes on windows command line, instead of quotes.

Can you use standard Virtual Machine ARM Template options?

When I create a new SQL IAAS VM, I am able to choose all sorts of SQL options, such as enabling SQL Authentication. I want to use these in my DevTest VMs too.

I have looked at the ARM templates produced by the standard Virtual Machines blade. Is this transferable into the DevTest VM ARM template?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.