Dynamic temporary variables are the icing on the cake for Azure DevOps Pipeline variable customization

Hello everyone, I'm Gan Qian, a researcher at Microsoft MVP Labs in this issue. Azure Pipeline itself already provides built-in variables. Different from the above methods, today I will bring how to create and use dynamic temporary variables in the runtime of Azure DevOps Pipeline to realize dynamic customization of variables. Next, let's find out in the experiment!

Analysis of ideas

In the Azure Terraform series we shared, we introduced the issue of remote storage of Terraform state files. We added azure_cli_script to the Task Job of the Azure DevOps Pipeline to execute the inline script (this script helps us create the Azure required for Terraform state file storage. Resource Group, Azure Storage Account, Azure KeyVault, etc.). It should be noted that there is a dynamic variable used in the inline script, which temporarily stores the Account Key of the Azure Storage Account, as shown in the following figure:

In this article, I continue to lead you to analyze how to create and use dynamic temporary variables in Azure DevOps Pipeline running, and use dynamic temporary variables to replace Azure Pipeline pipeline variables.

Project overall structure diagram

Pipeline variable definition, output

At this stage, we need to use the azure_cli_script task to create dynamic temporary variables and output parameters, the most important of which is to output dynamic temporary variables. The variables output by Task yaml as shown below are used in the same stage, different jobs

 - stage: script
  jobs:
   - job: azure_cli_script
     steps: 
      - task: AzureCLI@2
        displayName: 'Azure CLI :Create Storage Account，Key Vault And Set KeyVault Secret'
        name: 'output_variable'
        inputs:
          azureSubscription: 'Microsoft Azure Subscription(xxxxxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)'
          scriptType: 'bash'
          addSpnToEnvironment: true
          scriptLocation: 'inlineScript'
          inlineScript: |
              # create azure resource group
              az group create --location eastasia --name $(terraform_rg)
​
              # create azure storage account
              az storage account create --name $(storage_account) --resource-group $(terraform_rg) --location eastasia --sku Standard_LRS
​
              # create storage account container for tf state 
              az storage container create --name $(storage_account_container) --account-name $(storage_account)
​
              # query storage key and set variable
              ACCOUNT_KEY=$(az storage account keys list --resource-group $(terraform_rg) --account-name $(storage_account) --query "[?keyName == 'key1'][value]" --output tsv)
​
              # create azure keyvault
              az keyvault create --name $(keyvault) --resource-group $(terraform_rg) --location eastasia --enable-soft-delete false
​
              # set keyvault secret,secret value is ACCOUNT_KEY
              az keyvault secret set --name $(keyvault_sc) --vault-name $(keyvault)  --value $ACCOUNT_KEY
​
              # set secret varivale and add to environment
              echo "##vso[task.setvariable variable=ACCOUNT_KEY;isOutput=true]$ACCOUNT_KEY"
              #echo "##vso[task.setvariable variable=ACCOUNT_KEY;issecret=true;isOutput=true]$ACCOUNT_KEY"
​
   - job: same_stage_echo
     dependsOn: azure_cli_script
     variables:
       ACCOUNT_KEY: $[dependencies.azure_cli_script.outputs['output_variable.ACCOUNT_KEY']]
     steps:
       - task: Bash@3
         displayName: 'Bash :output temporary variables in different jobs on the same stage'
         inputs:
           targetType: 'inline'
           script: |
             # echo ACCOUNT_KEY
             echo "ACCOUNT_KEY is $ACCOUNT_KEY"

Output variables for different stages

 - stage: echo_varibale
  dependsOn: script
  jobs:
    - job: different_stage_echo
      variables:
        ACCOUNT_KEY: $[stageDependencies.script.azure_cli_script.outputs['output_variable.ACCOUNT_KEY']]
      steps:
        - task: Bash@3
          displayName: 'Bash :output temporary variables in same jobs on the same stage'
          inputs:
            targetType: 'inline'
            script: |
              # echo ACCOUNT_KEY
              echo "ACCOUNT_KEY is $ACCOUNT_KEY"

The following is the complete azure-pipelines-1.yaml

 # Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
​
trigger:
- remote_stats
​
pool:
  vmImage: ubuntu-latest
​
stages:
- stage: script
  jobs:
   - job: azure_cli_script
     steps: 
      - task: AzureCLI@2
        displayName: 'Azure CLI :Create Storage Account，Key Vault And Set KeyVault Secret'
        name: 'output_variable'
        inputs:
          azureSubscription: 'Microsoft Azure Subscription(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx)'
          scriptType: 'bash'
          addSpnToEnvironment: true
          scriptLocation: 'inlineScript'
          inlineScript: |
              # create azure resource group
              az group create --location eastasia --name $(terraform_rg)
​
              # create azure storage account
              az storage account create --name $(storage_account) --resource-group $(terraform_rg) --location eastasia --sku Standard_LRS
​
              # create storage account container for tf state 
              az storage container create --name $(storage_account_container) --account-name $(storage_account)
​
              # query storage key and set variable
              ACCOUNT_KEY=$(az storage account keys list --resource-group $(terraform_rg) --account-name $(storage_account) --query "[?keyName == 'key1'][value]" --output tsv)
​
              # create azure keyvault
              az keyvault create --name $(keyvault) --resource-group $(terraform_rg) --location eastasia --enable-soft-delete false
​
              # set keyvault secret,secret value is ACCOUNT_KEY
              az keyvault secret set --name $(keyvault_sc) --vault-name $(keyvault)  --value $ACCOUNT_KEY
​
              # set secret varivale and add to environment
              echo "##vso[task.setvariable variable=ACCOUNT_KEY;isOutput=true]$ACCOUNT_KEY"
              #echo "##vso[task.setvariable variable=ACCOUNT_KEY;issecret=true;isOutput=true]$ACCOUNT_KEY"
​
   - job: same_stage_echo
     dependsOn: azure_cli_script
     variables:
       ACCOUNT_KEY: $[dependencies.azure_cli_script.outputs['output_variable.ACCOUNT_KEY']]
     steps:
       - task: Bash@3
         displayName: 'Bash :output temporary variables in different jobs on the same stage'
         inputs:
           targetType: 'inline'
           script: |
             # echo ACCOUNT_KEY
             echo "ACCOUNT_KEY is $ACCOUNT_KEY"
​
- stage: echo_varibale
  dependsOn: script
  jobs:
    - job: different_stage_echo
      variables:
        ACCOUNT_KEY: $[stageDependencies.script.azure_cli_script.outputs['output_variable.ACCOUNT_KEY']]
      steps:
        - task: Bash@3
          displayName: 'Bash :output temporary variables in same jobs on the same stage'
          inputs:
            targetType: 'inline'
            script: |
              # echo ACCOUNT_KEY
              echo "ACCOUNT_KEY is $ACCOUNT_KEY"

• Key point: The difference between the use of variables in the pipeline and the use of dynamic temporary variables
• Use in Pipeline: $(variable name)
• How to use dynamic temporary variables: $variable name

Configuring Pipeline Pipeline Variables

Use the inline script to create an Azure Storage Account and Azure Key Vault using the Azure CLI to control parameters using managed variables

Run Pipeline to see configuration output

Since we have specified the work branch "remote_stats" in the azure-pipelines-1.yaml file, whenever we trigger the "push" or "pull_request" action of the "remote_stats" branch, it will trigger the running of the Azure DevOps Pipeline.
Job output within the same stage:

Job output of different stages:

Summarize

In this experiment, we learned how to create dynamic temporary variables and the output of variables during the running of Azure DevOps Pipeline, which makes us more flexible to declare custom dynamic temporary variables in any job, and apply dynamic temporary variables to any In the job, this method is different from the variables in the Pipeline pipeline, especially in the definition stage and usage syntax, please refer to the official documentation for details.

Related Links:

• Set variables in script
  https://docs.microsoft.com/en-us/azure/devops/pipelines/process/set-variables-scripts?WT.mc_id=AZ-MVP-5004191&view=azure-devops&tabs=bash
• github code address
  https://github.com/yunqian44/Terraform_Cnbate_Traffic_Manager
• The use of Terraform in Azure DevOps series https://www.cnblogs.com/AllenMaster/category/1876925.html

---

Microsoft Most Valuable Professional (MVP)

The Microsoft Most Valuable Professional is a global award given to third-party technology professionals by Microsoft Corporation. For 29 years, technology community leaders around the world have received this award for sharing their expertise and experience in technology communities both online and offline.

MVPs are a carefully selected team of experts who represent the most skilled and intelligent minds, passionate and helpful experts who are deeply invested in the community. MVPs are committed to helping others and maximizing the use of Microsoft technologies by Microsoft technical community users by speaking, forum Q&A, creating websites, writing blogs, sharing videos, open source projects, organizing conferences, and more.
For more details, please visit the official website: https://mvp.microsoft.com/zh-cn

Long press to identify the QR code

Follow Microsoft China MSDN