Jade Engineering Blog: Enforce DevOps policies with Azure Pipelines

Many organizations have created or are working towards creating a central team responsible for supporting teams and pushing out high level initiatives (e.g. Terrafom modules that abstract complex cloud networking) with the hopes of disbanding this central team and putting the members into existing development teams.

But, as the saying goes, “with great power comes great security risks” (or something like that). The industry now recognizes that frequent, immutable deployments are table stakes. Now, more teams are turning their attention to the security of their deployment pipelines.

Ensuring compliance at scale

Code-based pipelines offer flexibility and allow teams to easily scale their practices and allow organisations to consolidate CI/CD tools and processes into a single place.

Essentially, they allow engineers to configure their deployment pipelines using the same tooling they use for their application. They can also apply the same checks and balances, such as pull requests. Having one centralized location also makes it easier for developers to share tools and practices.

But with all of this power, there is an increased need to to ensure all developers working within our organization are following some form of generic security guidelines, but we don't want to stop them from being able to move fast - we need templates!

Azure pipeline templates to the rescue!

We use a tool called Azure DevOps to manage some of our project and automated test and deployment processes.

Our plan was for each project a that team would create in Azure DevOps to get a barebones pipeline that would be based on our base pipeline. This core pipeline would perform activities like static code analysis and check package dependencies.

resources:
  repositories:
    - repository: Templates
      type: git
      name: drosenbrock/My.Security.Templates
      ref: release/0.0.3                            ## Allows versioning of templates
extends:
  template: base/static-security-test.yml@Templates
  parameters:
    environment: $      ## If your template expects inputs

Teams would then create build/test/deploy pipelines specific to their project needs. Their pipelines are then passed to another template we created, this pipeline would scan the tasks developers are using and block any potential outdated or vulnerable ones.

extends:
  template: base/integration.yml@Templates
# Pass the developer defined build files in the 'steps' directory to the integration pipeline
    projectBuildSteps:
      - template: steps\project-build.yml
    unitTestSteps:
      - template: steps\project-unit-tests.yml
    projectDeploySteps:
      - template: steps\project-deploy.yml
    postDeploymentTests:
      - template: steps\post-deploy-tests.yml

Using a template to run these developer-created pipelines, our template can control things like the build environment, for example on-prem agents vs self-hosted agents or using containers to build projects. We can also restrict what tasks are run.

The below example is a template that a developer would extend, this template expects some ‘steps’ to execute, we use a pipeline loop to iterate over the steps passed in and check the tasks contained within the steps passed in are allowed (as defined in "allowedTasks").

This setup is very strict and contains an opt-in workflow where teams will raise a request use a task that we don’t already allow, this is ideal for highly regulated software.

allowedTasks:'FileTransform@1, AzureFileCopy@4,AzureFileCopy@2'
strategy:
  runOnce:
    deploy:
      steps:
        - $:
          - $:
            - powershell: Write-Error "The task $ is not allowed"
              displayName: Check Build Task
              failOnStderr: true
              errorActionPreference: stop
            - $

With this concept in mind, we can create more templates to perform other tasks at any phase of the continuous integration and delivery process.

But wait, how do you ensure teams actually extend these templates?
Azure DevOps has a built in feature for this called Required Templates Check, although it only applies at a service connection level, this is good enough for most projects.

Next time, we'll look at how to extend this even further with an API and a database to track changes.

Things to take away:

• Azure Templates can be used to run tasks on top of the normal developer tasks
• We can change the build environment and restrict what tasks we allow
• Developers are only responsible for knowing how to build and deploy their own project
• Templates can be used in any part of the pipeline
• YAML pipelines allow us to use GIT workflows on them
• You can use Azure Required Template Check feature to enforce use of your templates