PowerShell scripts limited to 2000 characters about azure-pipelines-yaml HOT 6 CLOSED

Just encountered the exact same issue. Was tweaking my template repository when it occurred.

Really wish there was a simple way to use scripts from repository resources. Perhaps store the contents of repository resources in a separate directory and persist them through the build. Being able to separate scripts from the yaml would make it much easier to write tests.

from azure-pipelines-yaml.

2. values calculated in one step would need to be recalculated in the next

Why would they need to be recalculated? Can't you mark the values as output variables to share them between steps?

from azure-pipelines-yaml.

Promoting local variables (storing calculated values) to output parameters is equivalent to making a local variable a global in your favorite programming language just so you don't exceed an arbitrary limit in a function. It makes no sense when the script limit itself is arbitrary. The only reason they need to be shared between steps is because of the limit. The variables themselves are not for use outside the (preferably single) script they are needed in.

from azure-pipelines-yaml.

Honestly, I wouldn't be writing a 2000 line function - That screams "refactor".

It's probably not an arbitrary limit, it's probably in place to prevent long-running scripts that lock up build pipelines.

Output variables could be restricted/scoped to a "Stage", perhaps that's a way to control the scope of access?

from azure-pipelines-yaml.

I appreciate your trying to help but you clearly down't understand the problems that are occurrng here or have simply never run into them. This is in no way solvable by refactoring. In fact refactoring makes the problem worse.

Firstly you said a 2000 line function is bad and I'd agree however we are not talking about a 2000 line function we are talking about a 2000 character file. Completely different concepts. You probably don't pay too much attention to how large your simplest source files are but count the characters (including spaces) in one of them to see my point. The entire script is put into a script file that is limited to 2K. That means you have to go "embedded code" route with minimal spacing, very short variable names and no comments. Every single one of those adds to the character limit.

Secondly it is pretty arbitrary. So much so that in the initial release it was limited to 500 or so. They then upped the limit to 2K (probably by changing a constant) and then said use scripts for anything bigger. In general this is correct however scripts cannot be used in step templates. This is a long standing limitation of templates. When you have lots of repos that want to build things similarly you use script templates. To ensure you don't replicate code these are generally stored in a template repo that is easily referenced in any other repo. The problem is that the existing logic for templates copies the scripts from the template repo into the build repo. Any references you have to external scripts won't work unless they are already in the build repo. Hence you cannot rely on the standard workaround of using script files inside step templates.

Refactoring can be used to break up a script but creating PS functions inside a script requires that you compare the total characters in the function (including the function declaration portion) against how many times that function is called. If the # of chars in the function is less than the chars it takes to call the function * the # of calls then a function makes sense. Of course this will change over the life of the script so it has to be weighed carefully.

You also cannot use output variables in many cases. For example suppose you want to generate a metadata source file that contains build information such as versioning, build date, source paths, etc. Every line you put into that file is part of the character count so on top of the actual contents (variable references, quotes, etc) you also have to include the string logic to inject it into an array, string or whatever. Then you need to add in the actual call to set the file contents. Of course if it is a really large file then you can break it up into multiple steps however each step would need to start and end with a get-content/set-content call. These eat up part of your character counts. It is made even harder if you actually need to inject lines into an existing file instead of writing a clean file each build.

On top of all this the total length of the script has little impact on the run length. A 1000 line script will run faster than breaking it up into separate scripts because each script requires the build to start a new PS session. Additionally a single line PS script can easily lock up a build by simply looping for a long time. The argument that script length negatively impacting performance is really outdated and has been since modern languages were developed. The only impact a larger script has is the amount of space it takes up in the build and the size of the generated file that is passed to PS ultimately. But breaking up the script into several smaller scripts doesn't change this overall.

from azure-pipelines-yaml.

In order to consolidate to fewer feedback channels, we've moved suggestions and issue reporting to Developer Community. Sorry for any confusion resulting from this move.

from azure-pipelines-yaml.