Passing complex parameters from Azure Devops YAML
For a while now it has been possible to create both build and release pipelines in Azure Devops using YAML code. This enables you to have your pipelines saved in your repository, with the rest of your code. Another advantage is that it enables you to alter your pipeline for new features or functions you are introducing, without breaking the pipeline for other branches or older releases. After all the pipeline YAML is taken from the specific branch you are building or releasing.
I am starting to get some experience using these YAML pipelines, though I am far from being an expert on it. So far I have only really scratched the surface. But one of the things that has been annoying me a little from the beginning, is the apparent inability to pass complex parameters from your YAML code to, for instance, a powershell task.
So what would a complex parameter look like in YAML? Lets say we define an object describing a virtual network with some subnets. In Yaml we coud do something like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
parameters:
- name: hubConfig
type: object
default:
vnetRsg: bicep
vnetRsgLocation: westeurope
vnetName: Hub
vnetAddressSpace: 10.10.0.0/16
snetConfig:
- name: frontend
range: 10.10.0.0/24
- name: backend
range: 10.10.1.0/24
But how can we pass this along? I have been searching for this occaissionally and, while people have found creative ways of working around this, directly passing the parameter did not appear to be possible:
- Passing arrays as parameter to devops pipelines
- How to pass complex devops parameters
- Azure devops yaml template passing hashset
While these solutions are creative and could possibly be used in some scenarios, it feels cumbersome, errorprone and not very universally applicable.
But then I came about this post: Allow type casting or expression function from YAML
I am not sure when this was introduced and I have not been able to track down this expression in the official Microsoft Devops documentation here. But it exists and it works!
The post listed above shows how it is possible to convert a complex YAML object (or array of objects) into a json string, simply by using:
1
${{ convertToJson(parameters.hubConfig) }}
A string can of course easily be passed along and PowerShell (and likely most other languages you would like to use) can convert to or from JSON without any problems.
For a full example of a pipeline that uses a YAML object and passes it into PowerShell, look at this (test) pipeline that have been using trying to get this to work. Converting to a PS object and then converting it back is probably not necessary… However if I don’t, I get Az Cli errors that I have not yet been able to reproduce using a local PowerShell terminal. In any case it should work to illustrate the usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
parameters:
- name: hubConfig
type: object
default:
vnetSubscriptionId: <snip>
vnetRsg: bicep
vnetRsgLocation: westeurope
vnetName: Hub
vnetAddressSpace: 10.10.0.0/16
snetConfig:
- name: frontend
range: 10.10.0.0/24
- name: backend
range: 10.10.1.0/24
- name: spokeConfig
type: object
default:
- vnetSubscriptionId: <snip>
vnetRsg: bicep2
vnetRsgLocation: westeurope
vnetName: Spoke1
vnetAddressSpace: 10.11.0.0/16
snetConfig:
- name: frontend
range: 10.11.0.0/24
- name: backend
range: 10.11.1.0/24
- vnetSubscriptionId: <snip>
vnetRsg: bicep3
vnetRsgLocation: westeurope
vnetName: Spoke2
vnetAddressSpace: 10.12.0.0/16
snetConfig:
- name: frontend
range: 10.12.0.0/24
- name: backend
range: 10.12.1.0/24
jobs:
- deployment: hub_spoke_deployment
displayName: 'Create hub/spoke configuration'
pool:
vmImage: windows-latest
environment: t
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: AzureCLI@2
inputs:
azureSubscription: spn-bicep-devops
scriptType: 'pscore'
scriptLocation: inlineScript
inlineScript: |
"Setting input Variables"
$hubconfig = '${{ convertToJson(parameters.hubConfig) }}' | ConvertFrom-Json -Depth 10
$spokeconfig = '${{ convertToJson(parameters.spokeConfig) }}' | ConvertFrom-Json -Depth 10
"Setting Az Deployment Variables"
$hubconfigRaw = ($hubconfig | convertto-json -Depth 10 -Compress).replace('"','\"')
$spokeconfigRaw = ($spokeconfig | convertto-json -Depth 10 -Compress).replace('"','\"')
"Starting az deployment"
az deployment sub create `
--template-file $(Pipeline.Workspace)\s\bicep-tests\main.bicep `
--location westeurope `
--parameters hubConfig=$hubconfigRaw `
--parameters spokeConfig=$spokeconfigRaw
With regards to the test example: I am thinking of using a PowerShell script make the YAML task part a bit more clean. Again, the converting to and from JSON may not be needed, but I like the part that I can use ‘-Compress’ to remove unneeded tabs or linebreaks and maybe prevent some additional future headaches.
I have tested doing it this way and I will include it in case anyone has use for it. I will post two final pieces of code below. The first one (YAML) is the replacement for the ‘inlineScript’ part above and the second is the PowerShell script to transform the input into something that can be be passed to Az Cli. I have done some tests with it and it appears to work fine. However, if you decided to copy (parts) of it, make sure to do you own due diligence!
The ‘inlineScript’ replacement for the YAML pipeline above:
1
2
3
4
5
6
7
8
9
10
11
12
inlineScript: |
$arguments = @{
'name' = 'MyDeploymentName'
'template-file' = '$(Pipeline.Workspace)\s\bicep-tests\main.bicep'
'location' = 'westeurope'
'hubConfig' = '${{ convertToJson(parameters.hubConfig) }}'
'spokeConfig' = '${{ convertToJson(parameters.spokeConfig) }}'
}
$(Pipeline.Workspace)\s\bicep-tests\Create-AzDeploymentYaml.ps1 `
-subscription `
-deploymentArguments $arguments
Powershell script to transform the arguments and run ‘az deployment’:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
[CmdletBinding()]
param (
[Parameter(ParameterSetName=’Resourcegroup’)]
[switch]
$group,
[Parameter(ParameterSetName=’Subscription’)]
[switch]
$subscription,
[Parameter(ParameterSetName=’Managementgroup’)]
[switch]
$managementgroup,
[Parameter(Mandatory=$True)]
[HashTable]$deploymentArguments,
[int]$jsonDepth = 10
)
If ($deploymentArguments) {
foreach ($key in $deploymentArguments.Keys){
if ($key -in ('name','location', 'template-file', 'query-string', 'subscription', 'template-uri', 'management-group-id', 'resource-group')){
[string[]]$arguments += ("--{0}" -f $key)
[string[]]$arguments += $deploymentArguments[$key]
} elseif ($deploymentArguments[$key] | Out-String | ConvertFrom-Json -ErrorAction SilentlyContinue){
# Test if json otherwise expect singular item
# Annoyingly Test-Json does not work with Json arrays...
("Adding argument {0} as a json argument" -f $key)
$json = $deploymentArguments[$key] | ConvertFrom-Json -Depth $jsonDepth
[string[]]$arguments += "--parameters"
[string[]]$arguments += ("{0}=`"{1}`"" -f $key, ($json | convertto-json -Depth $jsonDepth -Compress).replace('"','\"'))
} else {
("Adding argument {0} as a NON json argument" -f $key)
[string[]]$arguments += "--parameters"
[string[]]$arguments += ("{0}={1}" -f $key, $deploymentArguments[$key])
}
}
} else {
Throw "No deployment arguments found"
}
"Arguments found:"
$arguments
If ($group) {
$scope = 'group'
} elseif ($subscription){
$scope = 'sub'
} else {
$scope = "mg"
}
"Starting az deployment"
az deployment $scope create `
@arguments