The first time I wanted to use templates and I immediately ran into that issue. What a poor design. Users should not need to care about when and how variables are expanded. That is just an implementation detail. It should be as easy as this:

# pipeline.yml
steps:
- template: myTemplate.yml
  condition: $(someVariableInMyCurrentFileAndIdontCareWhetherItIsDefinedByAMatrixOrWhatNot)

The pipeline should then make 2 evaluation passes:

• one on DOM level as explained to load all templates unconditionally
• one after job orchestration to invalidate all unneeded steps

How hard can it be?

from azure-pipelines-yaml.

@vtbassmatt , the template expression stuff is mostly hacky to me. I would argue that the goal should be to be able to copy/paste as much existing logic from a YAML file to a template and it still work. As such I would expect to be able to reference parameters inside a condition just like I can reference variables and env vars. I don't believe templates currently support variable declarations but if they did I would expect to be able to use them just like any other non-template YAML file.

As a user of YAML I shouldn't have to be concerned with how the parser is implemented just to use it for "normal" things. My focus should be on getting my standalone scripts running inside the engine. That is hard enough as it is without the added complexity of figuring out the rules of the YAML parser.

from azure-pipelines-yaml.

parameters:
- name: publishNow
  type: string
    
steps: 
- task: Bash@3
  displayName: 'PublishNow'
  inputs:
    targetType: 'inline'
    script: |
        echo publishNow in templates: ${{ parameters.publishNow }}  

- ${{ if eq(parameters.publishNow, 'true') }}:
  - template: create-resource-group.yml
  - template: prepare-azure-packer-cfg.yml 
  - template: build-image.yml
  - template: delete-resource-group.yml

@vtbassmatt Sorry for posting on a closed issue. But this looks very relevant.

In the above yaml, PublishNow task prints the value of parameters.publishNow correctly as true but the condition eq(parameters.publishNow, 'true') never evaluates to true. What is wrong here?

from azure-pipelines-yaml.

You have a single paren on the left and 2 parens on the right so it is invalid

This. Sorry for the noise, I was blind.

from azure-pipelines-yaml.

The VS Code extension doesn't understand template syntax. Your YAML looks correct to me.

from azure-pipelines-yaml.

Was working through this example based on the value of
- ${{ if eq(variables['Agent.JobStatus'], 'Failed') }}:

• Could not achieve my desired result due to the evauluation of the if statement above seems to be done prior to all steps and not during runtime

I wanted to send a different slack message based on a failure of a previous step or not as per below examples:

You will need to add two steps to the pipe and provide a "condition":

- template: ../SlackMessageTasks/send-slack-message.yml@templates
    parameters:
      slack_message: 'Deploy to `${{ parameters.test_env_num }}` Completed for ClientLogin, Build `$(Build.BuildNumber)`, Branch `$(Build.SourceBranch)` for `$(Build.RequestedFor)` success! :check:'
      SlackChannelConfig: 'test-environments-$(EnvNum)'
      slack_step_tite: 'Success ${{ parameters.test_env_num }}'
      run_if: ne(variables['Agent.JobStatus'], 'Failed')

And then inside your template:

step: PrintToSlack
   condition: ${{ parameters.run_if }}

I know it's not elegant and it will always show one step as skipped, but it's the only thing I could get working.

By the way, you can also use run_if: failed() and run_if: succeeded()

from azure-pipelines-yaml.

Either the template should be taught the right condition (most common case?) or it should be parameterized if the user of the template needs to be in control.

from azure-pipelines-yaml.

@ericsciple, Are you saying that a -template is effectively a #include of the underlying template into the existing YAML file? If so then why, on the original issue, did you say the syntax (I'm trying to use below) is supported?

@vtbassmatt, Why was this issue closed when the original issue this was based upon (before the migration) is still valid and still not working? What do you mean by "taught the right condition"? Based upon your second statement of "parameterized" it seems like you're trying to suggest that a template should have some sort of "enabled" parameter that should be set by the code. That doesn't make sense to me because none of the existing tasks do this because they support the condition clause. The ask on this issue is that we can use condition on a template. Why is this different than an existing task? Even if it currently doesn't support it, the recommended solution of using a template expression isn't working for people, myself included. When I try this (from Eric's original solution under the original issue).

variables:
   someVar: true
steps:
- ${{ if ne(variables['someVar'], 'true') }}:
   - template: mytemplate.yml

I get the error:
The directive 'if' is not allowed in this context. Directives are not supported for expressions that are embedded within a string. Directives are only supported when the entire value is an expression.

from azure-pipelines-yaml.

@carct variables aren't visible at template expansion time. I'm working on a doc to make this more clear. As a user of the YAML build system, there are 3 execution times to consider:

• template expansion
• job orchestration
• agent-side runtime

Template expansion is just a DOM-manipulation step. Think of text templating like Jinja or C preprocessor macros, slightly enlightened about YAML structures so that you don't have to worry as much about spacing. It's not aware of most of the other features of the YAML syntax, including [edit] user-provided variables. It only considers parameters, which are explicitly designed for the template expansion step, [edit] and system-provided variables.

from azure-pipelines-yaml.

@vtbassmatt it is a bit confusing: "variables aren't visible at template expansion time"
but above, @ericsciple used it basically the same way, or the important difference is that his template is seen as a step of a phase and not as a whole job, as in my case ?

also, your line is again confusing as I successfully manage to pass parameters values to the template with the help of variables

another thing that i did notice while working with these YAML Templates is that inside a template i used in the condition of a step the following line "condition: eq(variables['myVar'], 'true')", nothing weird until now, just that 'myVar' is actually a parameter of the template and does not exist as a Var, so again.. a bit confusing :)

coming back to my issue, what would your suggestion be regarding it ?
i confirmed that the only issue is that the variable is not expanded at that level

thanks for your time !

LATER EDIT:
so i changed a bit my parent-YAML to be more similar to the above approach with the template being a step of a job and now i get this error: "Expected a scalar value" and the coordinates to the first character of my template condition expression line

LATER EDIT x2:
:) finally i managed to make it work and now my structure is something like this for the parent-YAML:

jobs
 - job
  pool
  condition: in(variables['myVar'], 'opt1', 'opt2')
  steps:
    -template: .\path-to-my.yaml
     parameters:

and of course that i had to convert my template YAML to be a steps template and not a jobs one as it was initially

in the end it's not so bad or impossible to do, but proper documentation is needed as i had to do a lot of tryouts to finally make it do my bidding :) and i don't consider my use-case so niched, i just wanted to execute a template based on a truly simple condition that uses a variable in it...

from azure-pipelines-yaml.

Thanks. This is something we'll consider for a future enhancement. I don't know if there's a good reason why variables aren't available in that context, or if it's just an implementation quirk.

from azure-pipelines-yaml.

how about just a 'condition:' line under "- template: xxxxx' ??
Other tasks can do this, why not templates? The template would be expanded at compile time and show up as a step, then the steps within get skipped.
this one is killing my builds

from azure-pipelines-yaml.

Hi Just a update from my experience which is working smoothly for me now.

• I am using the run_if option suggested above.
• The key for me was to ensure that the value run_if is passed as a string into the template parameter type.
• The end goal for me was to pass values into the template conditions based on previous step outputs My Example below

=========== Pipeline Definition =============

steps:
- powershell: |
       Write-Host("##vso[task.setvariable variable=azureDevopsTestRunFailedResultCount;isOutput=true]0")
   name: RetrieveResults

- template: ./doSomething.yml
  parameters:
    run_if: and(eq(variables['RetrieveResults.azureDevopsTestRunFailedResultCount'], 0), eq(variables['RetrieveResults.azureDevopsTestRunIgnoredResultCount'], 0))

=========== YML Template =============

parameters:
  - name: run_if
    default: true

steps:
- task: PowerShell@2
  condition: |
    and
    (
      ${{ parameters.run_if }},
      succeeded()
    )
  inputs:
    targetType: 'inline'
    script: |
      Write-Host "DoSomething"

from azure-pipelines-yaml.

guys, i seem to encounter a bit of an issue with a condition, which i tried setting in multiple ways :)

my current approach on setting it on a template yaml reference is as you said above:

jobs:
- ${{ if in(variables['TestRun'], 'SmokeTests', 'AllTests') }}:
  - template: ".\\my-template.yml"

indentation seems to be fine, the template line is one tab more to the right that the condition line and the condition lines starts at the same indentation as the 'jobs' line

the thing is that i don't get any error, but it doesn't execute either, although i'm sure that i do have a variable 'TestRun' (settable at queue time, but it has a default value on it) // does it influence that this variable is declared in the UI side of the Build Definition and not in the YAML (i know that some time ago you couldn't declare in the yaml that the variable should be settable at queue time) ?

i tried in a few other ways with basically the same main expression 'in(variables['TestRun'], 'SmokeTests', 'AllTests')', but the above seemed to get me most far :) only that it is always false :)

btw, is it normal behavior that if the condition set a on a template reference evaluates to false to not even see that phase in the Build Execution ? (as opposed to the message 'skipped' that i can see when a condition is not met on a specific step)

appreciate any kind of help, thank you!

from azure-pipelines-yaml.

For the completeness; the if ne(variables['someVar'], '') condition (from above) works.

I am a bit puzzled; the example for the conditional templates works does not cause parser errors, but it never runs. I have the following configuration files:

azure-pipelines.yml:

pool: hosted ubuntu 1604
variables:
   someVar: true
steps:
- script: echo someVar is $(someVar)
  displayName: 'Check someVar Script'

- script: 'echo someVar is $(someVar) in a conditional script'
  condition: ne(variables['someVar'], '')
  displayName: 'Conditional Script (Should run)'

- script: 'echo someVar is $(someVar) in a conditional script'
  condition: eq(variables['someVar'], '')
  displayName: 'Conditional Script (Should NOT run)'

- ${{ if ne(variables['someVar'], '') }}:
   - template: mytemplate.yml

- script: echo hi
  displayName: 'Always Run Script'

mytemplate.yml:

steps:
- script: echo from mytemplate.yml
  displayName: 'Template Script'

And the build steps from the template are not executed:

Can someone please help me figure out what's wrong with my build? Thank you!

from azure-pipelines-yaml.

There is another problem, perhaps this could go to a separate GH issue.

Let assume, I would like to run different jobs when the build is manually triggered or scheduled (CRON) than when it is a rolling (individual commits/pushes plus the batched ones).

Formally:

• run the release build when Build.Reason is Manual or Scheduled.
• run a fast build when Build.Reason is NOT Manual AND NOT Scheduled.

the thing is that i don't get any error, but it doesn't execute either

Since the conditions ({{ }}) do not work for the template, I declared the condition for the job instead. (See the LATER EDIT x2 here.)

I ended up having the following configuration:

azuere-pipelines.yml:

jobs:
- job: rolling_macOS
  displayName: 'Rolling Build (macOS)'
  pool:
    vmImage: 'macOS-10.13'
  condition: notIn(variables['Build.Reason'], 'Manual', 'Schedule'))
  steps:
    - template: rolling-build-steps.yml

- job: release_macOS
  displayName: 'Release Build (macOS)'
  pool:
    vmImage: 'macOS-10.13'
  condition: in(variables['Build.Reason'], 'Manual', 'Schedule'))
  steps:
    - template: release-build-steps.yml

- job: release_windows
  displayName: 'Release Build (Windows)'
  pool:
    vmImage: 'vs2017-win2016'
  condition: in(variables['Build.Reason'], 'Manual', 'Schedule'))
  steps:
    - template: release-build-steps.yml

release-build-steps.yml:

steps:
- script: echo release build
  displayName: 'Release Build Script'

rolling-build-steps.yml:

steps:
- script: echo rolling build
  displayName: 'Rolling Build Script'

Error:

An error occurred while loading the YAML build pipeline. Unexpected symbol: ')'. Located at position 55 within expression: notIn(variables['Build.Reason'], 'Manual', 'Schedule')). For more help, refer to https://go.microsoft.com/fwlink/?linkid=842996

A working solution 🎉:

Transform the notIn(variables['Build.Reason'], 'Manual', 'Schedule')) and in(variables['Build.Reason'], 'Manual', 'Schedule')) expressions into another format. It seems, it cannot parse notIn.

azure-pipelines.yml:

jobs:
- job: rolling_macOS
  displayName: 'Rolling Build (macOS)'
  pool:
    vmImage: 'macOS-10.13'
  condition: and(ne(variables['Build.Reason'], 'Manual'), ne(variables['Build.Reason'], 'Schedule'))
  steps:
    - template: rolling-build-steps.yml

- job: release_macOS
  displayName: 'Release Build (macOS)'
  pool:
    vmImage: 'macOS-10.13'
  condition: or(eq(variables['Build.Reason'], 'Manual'), eq(variables['Build.Reason'], 'Schedule'))
  steps:
    - template: release-build-steps.yml

- job: release_windows
  displayName: 'Release Build (Windows)'
  pool:
    vmImage: 'vs2017-win2016'
  condition: or(eq(variables['Build.Reason'], 'Manual'), eq(variables['Build.Reason'], 'Schedule'))
  steps:
    - template: release-build-steps.yml

When the build is triggered by a push:

When the build is triggered manually:

from azure-pipelines-yaml.

@kittaakos the error about the ) is correct. You have a single paren on the left and 2 parens on the right so it is invalid. Try this.

steps:
- ${{ if in(variables['Build.Reason'], 'Manual', 'Schedule') }}:

And in last post you have the same issue.

condition: notIn(variables['Build.Reason'], 'Manual', 'Schedule')

The parens are for the notIn function. You have an extra paren that isn't needed.

For the template expression around your template the syntax looks correct. I have a similar expression and it is working fine. I have noticed that YAML is picky about spacing though. Indentation is important and the docs make it clear that you should avoid tabs. So I would recommend that you remove all the whitespace from the template line and then add enough spaces until the - lines up with the $ above it. I'm not sure it is the cause of the issue here but I've had problems with indentation in my YAML.

from azure-pipelines-yaml.

@sl-NZ

Strange, I use that very setup myself:

////////////////
// pipeline.yml
- template: build-sdk.yml
   parameters:
   condition: eq(variables['buildSdk'], 'true')

////////////////
// build-sdk.yml
parameters:
   condition: True

- bash: |
    # Build if needed
   condition: and(succeeded(), ${{ parameters.condition }})

from azure-pipelines-yaml.

@spygi the doc was published a while ago here: https://docs.microsoft.com/azure/devops/pipelines/process/runs. Sorry that wasn't clear!

from azure-pipelines-yaml.

I'm no longer working on Azure DevOps.

from azure-pipelines-yaml.

conditions are evaluated at runtime, not during plan compilation. I dont think we should mix the two

from azure-pipelines-yaml.

what does your template look like?

from azure-pipelines-yaml.

When I run this:

pool: hosted ubuntu 1604
variables:
   someVar: true
steps:
- ${{ if ne(variables['someVar'], '') }}:
   - template: mytemplate.yml
- script: echo hi

I don't get any error. I think the error about the directive not allowed must be coming from your template file.

I would expect the error to include the file name and line and column. Does it not?

from azure-pipelines-yaml.

@ericsciple I think you're right. I have been updating the templates to use the inline template expression stuff and I finished updating them all. I had temporarily commented out that expression to get past it. I added it back and I don't get the error anymore. I cannot confirm it is working as expected yet because the build is failing in an earlier step but it queues now. Thanks.

from azure-pipelines-yaml.

Think of template expansion like a Jinja or Handlebars template (though it doesn't operate on pure text - it operates on the DOM of the YAML file). The conditional-inclusion stuff there is ${{ if ... }} as you and Eric have been talking about. The condition directive is a runtime concept. That's what I don't want to mix.

Both of them support roughly the same capabilities in the expression language. If we're missing something there, let us know so we can schedule it.

from azure-pipelines-yaml.

Hey, sorry, I was wrong about what's in scope in the various steps. Can you paste in your whole YAML (or if not comfortable sharing here, you can email me: mattc at xbox dot com)?

from azure-pipelines-yaml.

well, i did manage to make it work as per my latest edit to the post

the only thing is that it was a bit confusing and to be frank with you, i would like to still be able to set such a condition on Jobs Template
i don't find it too useful to have this kind of structure available:

jobs:
 - ${{ if in('something', 'opt1', 'opt2') }}:
   - template: ".\my-jobs-template.yml"
     parameters:
...........................

but unable to actually use some variables in that if execution condition :)

in my current case, i can be ok with the conversion to a Steps Template, but in other future cases i might not be so keen on that :))

if you think that we can get around the discussed limitations, we can take this to private, but from what i understood from you this is kinda' the way to do it...

from azure-pipelines-yaml.

variables aren't visible at template expansion time

@vtbassmatt, do predefined build variables available here? I would like to do something like this.

pool: hosted ubuntu 1604
variables:
   someVar: true
steps:
- ${{ if in(variables['Build.Reason'], 'Manual', 'Schedule')) }}:
   - template: mytemplate.yml
- script: echo hi

I am getting the following error:

azure-pipelines.yml (Line: 5, Col: 3): Unexpected symbol: ')'. Located at position 52 within expression: in(variables['Build.Reason'], 'Manual', 'Schedule')). For more help, refer to https://go.microsoft.com/fwlink/?linkid=842996

For the completeness; the if ne(variables['someVar'], '') condition (from above) works.

Thank you!

Edit: I have updated the error message.

Edit2: Someone else has already reported this (or similar) but the original GH issue has been closed.

from azure-pipelines-yaml.

So, according to this comment system variables work. That means (as I understand it) that only the strict subset of system variables can be used.

I created another issue to request support for agent variables too, which would be quite helpful IMHO.

For reference, my non-working template:

template:

# Installs PowerShell Core
parameters:
  version: '6.2.2'

steps:
- ${{ if eq(variables['Agent.OS'], 'Darwin') }}:
  - template: /provision.yml
    parameters:
      displayName: Provision PowerShell Core v${{ parameters.version }} for Mac
      item: 'https://github.com/PowerShell/PowerShell/releases/download/v${{ parameters.version }}/PowerShell-${{ parameters.version }}-osx-$(Agent.OSArchitecture).pkg"'

- ${{ if eq(variables['Agent.OS'], 'Windows_NT') }}:
  - template: /provision.yml
    parameters:
      displayName: Provision PowerShell Core v${{ parameters.version }} for Windows
      item: 'https://github.com/PowerShell/PowerShell/releases/download/v${{ parameters.version }}/PowerShell-${{ parameters.version }}-win-$(Agent.OSArchitecture).msi'

- ${{ if eq(variables['Agent.OS'], 'Windows_NT') }}:
  - powershell: |
      Write-Host "##vso[task.setvariable variable=PATH;]$($env:PATH + ';' + $env:ProgramFiles + '\PowerShell\' + '${{ parameters.version }}'.Substring(0, 1))"
    displayName: Add PowerShell Core to PATH

pipeline:

- stage: Windows
  dependsOn: []
  jobs:
  - job: Windows
    pool: 
      vmImage: windows-2019
    steps:
    - checkout: none
    - template: install-pwsh/v1.yml
    - pwsh: write-host PowerShell $PSEdition $PSVersionTable.PSVersion

Results in:

Not working (there's no install/provision step there, and the powershell step renders 6.2.1 which is not the version I wanted instaled) since it tries to expand based on agent variables which does not work.

from azure-pipelines-yaml.

I'm sorry, but I still can't get this to work.

parameters:
  clean: false

steps:
  - ${{ if eq(parameters['clean'], 'true') }}:
    - template: clean.yml

Gives me in vscode:

Unexpected property ${{ if eq(parameters['clean'], 'true') }}:
The first property must be a task

from azure-pipelines-yaml.

@ata18 that's unexpected. Could you please file a ticket on Developer Community?

from azure-pipelines-yaml.

i have the same as ATA18

from azure-pipelines-yaml.

@ata18, @hvandenborn

Try removing the apostrophes arount true. I have found somewhere, that this can cause troubles in YAML, because one value is treated as boolean, while 'true' is treated as string and it is evaluated to boolean false.

from azure-pipelines-yaml.

Was working through this example based on the value of
- ${{ if eq(variables['Agent.JobStatus'], 'Failed') }}:

• Could not achieve my desired result due to the evauluation of the if statement above seems to be done prior to all steps and not during runtime

I wanted to send a different slack message based on a failure of a previous step or not as per below examples:

Happy State, send happy message when all previous steps are successful

  - template: ../SlackMessageTasks/send-slack-message.yml@templates
    parameters:
      slack_message: 'Deploy to `${{ parameters.test_env_num }}` Completed for ClientLogin, Build `$(Build.BuildNumber)`, Branch `$(Build.SourceBranch)` for `$(Build.RequestedFor)` success! :check:'
      SlackChannelConfig: 'test-environments-$(EnvNum)'
      slack_step_tite: 'Success ${{ parameters.test_env_num }}'

UnHappy State, send message if one of the previous steps have failed

  - ${{ if eq(variables['Agent.JobStatus'], 'Failed') }}:
    - template: ../SlackMessageTasks/send-slack-message.yml@templates
      parameters:
        slack_message: ':x-cross: Deploy to `${{ parameters.test_env_num }}` failed for ClientLogin, Build `$(Build.BuildNumber)`, Branch `$(Build.SourceBranch)` for `$(Build.RequestedFor)`'
        SlackChannelConfig: 'test-environments-$(EnvNum)'
        slack_step_tite: 'Failed ${{ parameters.test_env_num }}'

from azure-pipelines-yaml.

Hey @denravonska, thanks for your suggestion but unfortunately no luck on this
Got below errors when passing the boolean, the pipelines try's to expand the value yaml of AgentStatus as soon as you try to run the pipeline and does not recognise the values

Pipeline run start errors:

/Templates/DeployTasks/deploy-clientlogin-test.yml@templates (Line: 82, Col: 15): The 'run_if' parameter value 'eq(variables['Agent.JobStatus'], 'Succeeded')' is not a valid Boolean.

/Templates/DeployTasks/deploy-clientlogin-test.yml@templates (Line: 82, Col: 15): The 'run_if' parameter value 'succeeded()' is not a valid Boolean.

This would essentially explain why:
https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#context

Thanks for your help regardless, I will place the task step for failure directly into my pipeline for now.

from azure-pipelines-yaml.

@denravonska A possibility could be that the template I am consuming is in a different repository from the Primary pipeline?

from azure-pipelines-yaml.

@sl-NZ I don't think it matters. If I understand it correctly the template parameter expansion just pastes whatever is passed to it into where it's used. So the final, expanded results should be

condition: succeeded()

Did you put quotes around the condition?

condition: '${{ parameters.run_if }}'

instead of:

condition: ${{ parameters.run_if }}

? Not sure if it matters though.

from azure-pipelines-yaml.

@sl-NZ template expressions are evaluated at "compilation" time and not while the pipeline is running. To me they are glorified #if statements. This diagram shows the process.

If you want to change the parameters passed to a task based upon parameters at compile time then you can do that with a single step and use template expressions to conditionally include/exclude values. Something like this (not tested).

- template: ../SlackMessageTasks/send-slack-message.yml@templates
   parameters:
      slack_message: ':x-cross: Deploy to `${{ parameters.test_env_num }}` failed for ClientLogin, Build `$(Build.BuildNumber)`, Branch `$(Build.SourceBranch)` for `$(Build.RequestedFor)`'
      SlackChannelConfig: 'test-environments-$(EnvNum)'
      - ${{ if ne(variables['Agent.JobStatus'], 'Failed') }}:
        slack_step_tite: 'Success${{ parameters.test_env_num }}'
      - ${{ if eq(variables['Agent.JobStatus'], 'Failed') }}:
        slack_step_tite: 'Failed ${{ parameters.test_env_num }}'

In my experience if you have only a single value to swap then it isn't bad. If the task itself changes you need multiple steps. Alternatively I tend to prefer variables when possible (templates make this hard). So instead of using template expressions everywhere I'll define a variable (if possible) or step(s) that set env vars that later steps just use directly. So, in your example, I'd probably use a variable for the messages and then have a single template expression above this to set the values I want to use in the task. Again, this isn't always possible. Templates in particular don't currently allow variables to be declared so I tend to use env variables (ugly but it works). If you only need the values once though then inline template expressions are easier.

from azure-pipelines-yaml.

@CoolDadTx That won't work as the job status is not available at template compile time. He needs to pass an evaluation expression to the template, as in my example, so the task is able to evaluate and conditionally run or skip itself at runtime.

from azure-pipelines-yaml.

@denravonska I wasn't focusing on the expression itself. I was addressing the posters comment about when template expressions are evaluated and the need for multiple steps. For the very specific case of checking the status of the agent then condition is fine.

from azure-pipelines-yaml.

@CoolDadTx No, your example is faulty :) You cannot do a static expression on a dynamic condition.

from azure-pipelines-yaml.

@denravonska I don't know what you mean by dynamic condition but template expressions can reference any values that can be determine at the point the template is "compiled". That includes build variables and template parameters. We do this heavily in our highly customized YAML builds and we have no problems whatsoever. Of course getting all this debugged and working was entirely different story but you can use some variables like build variables in template expressions. Variables whose values change during the build won't work correctly however so you have to switch to env variables (or perhaps YAML variables).

Nevertheless your experience may be different than mine and what works for our builds might not work for yours. This is one of the currently annoying features of YAML in that it is very finicky. Even in our own templates we have found that certain expressions or syntax works fine in one template but not in another. Although most of our issues are related to how the PS tasks are generated. Still I've shared my feedback on this issue and I'm not going to argue with you about what does and doesn't work. This is a closed topic anyway. The OP should open a new one if they are having issues with template expressions.

from azure-pipelines-yaml.

Just want to point out to folks reaching this point, this comment here was very helpful for me.

@vtbassmatt is your doc available somewhere? User-defined variables are part of the runtime execution right?

from azure-pipelines-yaml.

@denravonska I don't know what you mean by dynamic condition but template expressions can reference any values that can be determine at the point the template is "compiled". That includes build variables and template parameters. We do this heavily in our highly customized YAML builds and we have no problems whatsoever. Of course getting all this debugged and working was entirely different story but you can use some variables like build variables in template expressions. Variables whose values change during the build won't work correctly however so you have to switch to env variables (or perhaps YAML variables).

Nevertheless your experience may be different than mine and what works for our builds might not work for yours. This is one of the currently annoying features of YAML in that it is very finicky. Even in our own templates we have found that certain expressions or syntax works fine in one template but not in another. Although most of our issues are related to how the PS tasks are generated. Still I've shared my feedback on this issue and I'm not going to argue with you about what does and doesn't work. This is a closed topic anyway. The OP should open a new one if they are having issues with template expressions.

Hi @CoolDadTx,
Can you please provide an example of working condition in template parameters?

I'm trying to use sth like that:

  - template: /templates/template.yaml
    parameters:
      build: ${{ parameters.param1}}
      repo: ${{ variables.var1 }}
      ${{ if eq(variables.var2, 'True') }}:
        extraparam: 0

I would like to be able to pass extraparam to template as parameter when var2==True.

Thanks,
Andrzej

from azure-pipelines-yaml.

@andrzej-jedrzejewski You're trying to apply a condition on a parameter to a template. We don't do that so I cannot confirm it'll work. Personally I don't think it makes sense to conditionally include a parameter or not because the template defines the parameters it needs anyway. So making it conditional when calling it doesn't change the template itself. What we do is just set the parameter to a value. If the parameter cannot have a reasonable default value then we include an extra parameter indicating whether to use it or not. For example in our master YAML file we have a template that initializes our build process but must support .NET Framework and Core apps, publishing, etc so we make these parameters to the template. Since we are at the master YAML level we can set the values directly but you could use template expressions if you wanted I suspect.

- template: yaml/initialize-build.yml@templates
  parameters:
      packageBuild: true
      publishBuild: true

We do have a couple of templates we only want to call if certain conditions are set.

- ${{ if eq(parameters.metadataEnabled, 'true') }}:
  - template: add-props-metadata.yml
    parameters:      
       assemblyTypeName: ${{ parameters.metadataTypeName }} 
       isPublic: ${{ parameters.metadataIsPublic }}

I suspect your issue is that you're trying to use var2 as the condition but your template is defining and possibly changing that value. IIRC template condition expressions are evaluated at the point the template is called and so any adjustments to the variable wouldn't be seen in the condition because it is too early in the template process. Of course if the variable is an external variable then it will work just fine as when the template gets "loaded" the variable is already set. Switching to just using the current variable value and optionally including a "use this parameter" indicator to the template would probably resolve the issue as I mentioned earlier.

- template: /templates/template.yaml
  parameters:
     extraparam: 0
     useextraparam: ${{ eq(variables.var2, 'true') }}

from azure-pipelines-yaml.

@andrzej-jedrzejewski You're trying to apply a condition on a parameter to a template. We don't do that so I cannot confirm it'll work. Personally I don't think it makes sense to conditionally include a parameter or not because the template defines the parameters it needs anyway.

Thank you for your response. We have a template with a parameter that has a default value. Default value is valid for 99% cases but there is one case that requires us to override it with variable. We also use very similar code to your examples. I just don't understand why if statement is not able evaluate properly very simple condition like here:

  - template: /templates/template.yaml
    parameters:
      build: ${{ parameters.param1}}
      repo: ${{ variables.var1 }}
      ${{ if eq(variables.var2, 'True') }}:
        extraparam: 0

from azure-pipelines-yaml.

As I mentioned the conditional template expression is evaluated when the template is loaded and therefore it'll use the value of the variable at the point the template is loaded into memory, not whatever value you eventually assign it. Parameters work because parameters are set when the template is loaded but variables not so much. That is one reason why our templates tend to include an "enable" parameter to allow callers to determine whether to enable functionality or not rather than the template itself.

We also sometimes use multi-step templates which can sometimes help to work around issues, but not with conditions. As a last resort we call to Powershell to get the exact behavior we want.

from azure-pipelines-yaml.

parameters:
- name: publishNow
  type: string
    
steps: 
- task: Bash@3
  displayName: 'PublishNow'
  inputs:
    targetType: 'inline'
    script: |
        echo publishNow in templates: ${{ parameters.publishNow }}  

- ${{ if eq(parameters.publishNow, 'true') }}:
  - template: create-resource-group.yml
  - template: prepare-azure-packer-cfg.yml 
  - template: build-image.yml
  - template: delete-resource-group.yml

@vtbassmatt Sorry for posting on a closed issue. But this looks very relevant.

In the above yaml, PublishNow task prints the value of parameters.publishNow correctly as true but the condition eq(parameters.publishNow, 'true') never evaluates to true. What is wrong here?

I am having the same experience.
eq doesnt seem to work as expected with parameter values in template expressions?
@vtbassmatt any ideas?

from azure-pipelines-yaml.

@spygi the doc was published a while ago here: https://docs.microsoft.com/azure/devops/pipelines/process/runs. Sorry that wasn't clear!

I think the docs are good but there is something which needs to be spelled out with examples better IMHO. It says over and over again that 'variables arent available at template expansion time', but there are circumstances which make it APPEAR as though they are, but they arent.

(excuse the syntax)
Its perfectly legal to have:
pipeline.yml

variables:
publishNowVariable: true

steps:

• template: template.yml
  inputs:
  - publishNow: $(publishNowVariable)

template.yml

parameters:

• name: publishNow
  type: string

steps:

• task: Bash@3
  displayName: 'PublishNow'
  inputs:
  targetType: 'inline'
  script: |
  echo publishNow in templates: ${{ parameters.publishNow }}

• ${{ if eq(parameters.publishNow, 'true') }}:

  • template: create-resource-group.yml
  • template: prepare-azure-packer-cfg.yml
  • template: build-image.yml
  • template: delete-resource-group.yml

But during the template expansion step, all instances of parameters.publishNow will be replaced by a string literal "$(publishNowVariable)" for the purposes of template expression evaluation.
So:

• ${{ if eq(parameters.publishNow, 'true') }}:
  becomes
  if( eq('$(publishNowVariable)','true')
  which is evaulated by the template expression process as false

BUT!
echo publishNow in templates: ${{ parameters.publishNow }}

gets expanded to
echo publishNow in templates: $(publishNowVariable)

at template expansion time but not evaluated as part of a template expression.

When we get to runtime, the varaible is dereferenced as we get:
echo publishNow in templates: true

So it kind of appears as though somehow variables are available to templates, but its an illusion created by no properly understanding steps of the two phases.

It took me 6 hours of googling until I eventually stumbled upon the initializeLog.txt file in the 'download logs' zip in the Portal (this could be better documented!) and then the penny dropped.

from azure-pipelines-yaml.

Hi @RDavis3000 I would suggest to open a ticket on https://developercommunity.visualstudio.com/search?space=21 for this question - this repo is mostly for questions regarding templates located here, but devcom portal would be a better place for generic ADO questions.

from azure-pipelines-yaml.

Please add a 'condition' property to a template. It solves soooo many problems.
How hard can it be, you just take over the expression when expending the template so the expression is evaluated run-time and not compile-time.

Some use cases which are now impossible:

• have a condition to only run when previous stage was successful. The succeeded() cannot be used currently since this is a runtime condition.
• have a condition which uses a property set by another stage. E.g. a stage to verify which files have change. Only when sources have changed, the build and deploy stages should run.
  This does not work:

- ${{ if (eq(stageDependencies.checkStage.checkJob.outputs['checkStep.output'], 'true' }}:
  - template: my-template

Error unrecognised value 'stageDependencies'

from azure-pipelines-yaml.

Hi Team I am running this pipeline and getting the same error directive are not supported for the expression that are embedded within the string Directive are only supported when the entire value is an expression. Below is the pipeline script
name: $(Build.BuildNumber).$(Version.Revision)$(Version.Suffix)

trigger:

• none

pool:
name: Portal21

parameters:

• name: environment
  displayName: CWP environment
  type: string
  default: Please select environment
  values:

  • Please select environment
  • dev
  • ts1
  • ts2
  • ts3
  • stg
  • prf
  • tst
  • cte

• name: region
  displayName: CWP Region
  type: string
  default: Please select environment Central or East
  values:

  • central
  • east
  • both

variables:
Build.BuildNumber: rel23.3.2
Version.Revision: $[counter(variables['Build.BuildNumber'], 0)]

Set the suffix of the version number depending on whether this is master, pr, or other branch

${{ if eq(variables['Build.SourceBranch'], 'refs/heads/master') }}:
Version.Suffix: "" # master
${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
Version.Suffix: "-pr" # pull request
${{ if and(ne(variables['Build.SourceBranch'], 'refs/heads/master'), ne(variables['Build.Reason'], 'PullRequest')) }}:
Version.Suffix: "Branch"

steps:

• checkout: self

• task: UseNode@1
  displayName: "Use Node 16.x"
  inputs:
  version: "16.19.1"

• script: "npm install"

• task: Npm@1
  inputs:
  command: "custom"
  customCommand: run build validate $(Build.Repository.LocalPath)/igniset /subscriptions/MemberPortalNonProdServicePrincipalConnection/resourceGroups/mbr-portal-dev-us-c-rg/providers/Microsoft.DataFactory/factories/configservicedf-dev
  displayName: Validate

• task: Npm@1
  inputs:
  command: custom
  workingDir: $(Build.Repository.LocalPath)
  customCommand: run build export $(Build.Repository.LocalPath)/igniset /subscriptions/MemberPortalNonProdServicePrincipalConnection/resourceGroups/mbr-portal-dev-us-c-rg/providers/Microsoft.DataFactory/factories/configservicedf-dev 'ArmTemplate'
  displayName: Validate and Generate ARM template

• task: PublishBuildArtifacts@1
  inputs:
  targetPath: $(Build.Repository.LocalPath)/ArmTemplate
  PathtoPublish: $(Build.Repository.LocalPath)/ArmTemplate
  ArtifactName: ArmTemplate

• task: AzureKeyVault@1
  inputs:
  azureSubscription: 'MemberPortalNonProdServicePrincipalConnection'
  KeyVaultName: 'mbr-portal-common-kv'
  SecretsFilter: |
  ${{ if eq(parameters.environment, 'dev') }}
  dev-apim-subscription-key
  ${{ elseif eq(parameters.environment, 'prf') }}
  prf-apim-subscription-key
  ${{ elseif eq(parameters.environment, 'stg') }}
  stg-apim-subscription-key
  ${{ elseif eq(parameters.environment, 'tst') }}
  tst-apim-subscription-key
  ${{ else }}
  
  ${{ endif }}
  RunAsPreJob: true

• bash: |
  if [[ "$(environment)" == "dev" ]]; then
  echo "##vso[task.setvariable variable=subscriptionKey;issecret=true]$(dev-apim-subscription-key)"
  elif [[ "$(environment)" == "prf" ]]; then
  echo "##vso[task.setvariable variable=subscriptionKey;issecret=true]$(prf-apim-subscription-key)"
  elif [[ "$(environment)" == "stg" ]]; then
  echo "##vso[task.setvariable variable=subscriptionKey;issecret=true]$(stg-apim-subscription-key)"
  elif [[ "$(environment)" == "tst" ]]; then
  echo "##vso[task.setvariable variable=subscriptionKey;issecret=true]$(tst-apim-subscription-key)"
  fi
  displayName: 'Assign Secret Value'

• task: AzureResourceManagerTemplateDeployment@3
  inputs:
  ConnectedServiceName: 'Azure Non Prod'
  azureSubscription: 'MemberPortalNonProdServicePrincipalConnection'
  ResourceGroupName: 'mbr-portal-${{ parameters.environment }}-us-c-rg'
  Location: 'Central US'
  templateLocation: 'Linked artifact'
  csmFile: '$(Build.Repository.LocalPath)/ArmTemplate/ARMTemplateForFactory.json'
  csmParametersFile: '$(Build.Repository.LocalPath)/ArmTemplate/ARMTemplateParametersForFactory.json'
  overrideParameters:
  - 'factoryName mbrconfigservicedf-${{ parameters.environment }}'
  - 'triggerswhenexceluploaded_properties_typeProperties_scope /subscriptions/83ff40db-93fd-4781-8d9a-20c8e6eb9b55/resourceGroups/mbr-portal-${{ parameters.environment }}-us-c-rg/providers/Microsoft.Storage/storageAccounts/mbrconfigdf${{ parameters.environment }}uscst'
  - 'default_properties_ENV_value ${{ parameters.environment }}-'
  - 'default_properties_SUBSCRIPTION_KEY_value $(subscriptionKey)'
  - 'mbrconfigservicedfkeyvaultlink_properties_typeProperties_baseUrl https://mbr-portal-${{ parameters.environment }}-us-c-kv.vault.azure.net'
  - 'mbrconfigservicedfbloblink_properties_typeProperties_connectionString_secretName DATAFACTORY-CONNECTION-STRING'

  deploymentMode: 'Incremental'
  StopStartTriggers: true
  DeployGlobalParams: true

from azure-pipelines-yaml.

Thanks. This is something we'll consider for a future enhancement. I don't know if there's a good reason why variables aren't available in that context, or if it's just an implementation quirk.

Fast forward 5 years later same old error, Nice!

from azure-pipelines-yaml.

If any one had the same issue as I did which is Conditions in Variables Template by Parameters ( ChatGPT Gave me wrong answer that it's not possible sadly ..)

It's possible like explained above simply under variables call the template and pass parameters to it:

variables:
  - template: variables/TestingVars.yml
    parameters:
      param1: '${{ parameters.param1 }}'
      param2: '${{ parameters.param2}}'

and inside the Template

parameters:
  - name: param1
    type: string
  - name: param2
    type: string

variables:
- name: var1
  ${{ if eq(parameters.param1, 'test') }}:
      value: 'true'
  ${{ else }}:
    value: 'false'

for some reason such simple action took few hours to find answer to, and chatGPT wasn't helping.

from azure-pipelines-yaml.