Running Terraform in an Azure DevOps Pipeline: A Comprehensive Guide

In today’s world of cloud infrastructure and agile development practices, Infrastructure as Code (IaC) is crucial for simplifying the provisioning and management of resources. Terraform is a popular IaC tool that allows you to define and automate your cloud infrastructure. In this blog post, we will walk you through integrating Terraform with Azure DevOps pipelines, enabling you to create, update, and manage your infrastructure automatically during your development lifecycle.

Prerequisites:

• Basic understanding of Terraform and its concepts
• An Azure DevOps account and project
• An Azure subscription

Table of Contents:

1. Setting up Terraform in Azure DevOps
2. Creating a Terraform configuration file
3. Configuring the Azure DevOps pipeline
4. Running the pipeline
5. Managing Terraform state in Azure
6. Best practices for Terraform in Azure DevOps pipelines

Setting up Terraform in Azure DevOps:

You’ll need to install the Terraform extension from the Visual Studio Marketplace to get started. This extension provides tasks that can be used in Azure DevOps pipelines to run Terraform commands. To install the extension, follow these steps:

• Navigate to the Visual Studio Marketplace
• Search for “Terraform”
• Click on the “Terraform” extension by Microsoft DevLabs.
• Click “Get it free” and follow the prompts to install the extension in your Azure DevOps organization.

Creating a Terraform configuration file:

Before configuring the pipeline, you must create a Terraform configuration file (main.tf) describing the infrastructure resources you want to provision. This file will be stored in your source control repository.

Here’s an example configuration file for creating an Azure resource group and a virtual network:

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "West Europe"
}

resource "azurerm_virtual_network" "example" {
  name                = "example-network"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
}

Configuring the Azure DevOps pipeline:

1. Now it’s time to create an Azure DevOps pipeline to automate the deployment of the Terraform configuration. Create a new pipeline using the Classic Editor or YAML in your project. Add the following tasks to your pipeline:

• Terraform: Install: This task installs the specified version of Terraform on the build agent
• Terraform: Init: This task initializes your Terraform working directory, downloads providers, and sets up the backend for storing the Terraform state
• Terraform: Validate: This task validates your Terraform configuration files for syntax and configuration errors
• Terraform: Plan: This task generates an execution plan showing which resources will be created, modified, or destroyed
• Terraform: Apply: This task applies the changes to your infrastructure based on the execution plan

trigger:
  branches:
    include:
    - main

pool:
  vmImage: 'ubuntu-latest'


steps:
- task: TerraformTaskV4@0
  displayName: 'Install Terraform'
  inputs:
    terraformVersion: 'latest'

- checkout: self
  displayName: 'Checkout repository'

- task: TerraformTaskV4@1
  displayName: 'Terraform Init'
  inputs:
    command: 'init'
    workingDirectory: '$(Build.SourcesDirectory)'
    backendType: 'azurerm'
    ensureBackend: true
    backendServiceArm: '<service_connection_name>'
    resourceGroupName: '<resource_group_name>'
    storageAccountName: '<storage_account_name>'
    containerName: '<container_name>'
    key: '<state_file_key>'

- task: TerraformTaskV3@3
  displayName: 'Terraform Validate'
  inputs:
    command: 'validate'
    workingDirectory: '$(Build.SourcesDirectory)'

- task:TerraformTaskV3@3
  displayName: 'Terraform Plan'
  inputs:
    command: 'plan'
    workingDirectory: '$(Build.SourcesDirectory)'
    environmentServiceName: '<service_connection_name>'
    commandOptions: '-out=tfplan'

- task:TerraformTaskV3@3
  displayName: 'Terraform Apply'
  inputs:
    command: 'apply'
    workingDirectory: '$(Build.SourcesDirectory)'
    environmentServiceName: '<service_connection_name>'
    commandOptions: '-auto-approve -input=false tfplan'

Running the pipeline:

Save and run the pipeline to deploy your infrastructure. You’ll be able to see the progress of each task and view detailed logs for each Terraform command.

Managing Terraform state in Azure:

Terraform stores the state of your infrastructure in a state file. By default, this file is stored locally, but it is recommended to store it in a remote backend like Azure Blob Storage. To configure the remote backend, add the following code to your Terraform configuration file:

terraform {
  backend "azurerm" {
    resource_group_name  = "<resource_group_name>"
    storage_account_name = "<storage_account_name>"
    container_name       = "<container_name>"
    key                  = "<state_file_key>"
  }
}

Replace the placeholders with your Azure Blob Storage details. You’ll need to create a storage account and a container in the specified resource group before running the pipeline.

Best practices for Terraform in Azure DevOps pipelines:

• Use a separate repository or branch for your Terraform configuration files to isolate changes and maintain version control.
• Store sensitive data such as credentials and API keys in Azure Key Vault or use environment variables in the pipeline to avoid exposing them in the configuration files.
• Use Terraform workspaces to manage multiple environments (e.g., dev, staging, production) with the same configuration files.
• Lock your Terraform state file during pipeline execution to prevent concurrent modifications.
• Review and approve changes using pull requests and require peer reviews for critical infrastructure changes.
• Implement a CI/CD process with automated testing to validate your infrastructure changes before applying them.
• Regularly update the Terraform extension and the Terraform binary to take advantage of new features, bug fixes, and provider updates.

Conclusion:

Integrating Terraform with Azure DevOps pipelines is an excellent way to automate the provisioning and management of your infrastructure resources. By following this guide, you’ll have a solid foundation for using Terraform in your Azure DevOps environment. As you become more familiar with Terraform and Azure DevOps, you can further explore more advanced features and techniques to improve your infrastructure management practices.