avatarGuillermo Musumeci

Summary

The provided content outlines a detailed guide on deploying Azure OpenAI with a private endpoint and ChatGPT using Terraform, including prerequisites, configuration steps, and troubleshooting common errors.

Abstract

The article "How to Deploy Azure OpenAI with Private Endpoint and ChatGPT using Terraform" is a comprehensive tutorial that guides readers through the process of setting up Azure OpenAI services with enhanced security features. It begins by explaining the advantages of using Azure OpenAI, such as compatibility with OpenAI models and Azure's enterprise security. The guide covers prerequisites, including enabling OpenAI in an Azure subscription, requesting access to GPT-4, and setting up Customer Managed Keys. It then delves into defining the Azure provider, creating a resource group, configuring networking components, and deploying Azure Cognitive Services and OpenAI models. The article also addresses potential issues and errors that may arise during deployment, providing solutions to common problems such as feature enablement, network access configuration, and role assignment errors. The code examples and explanations aim to facilitate a secure and efficient setup of Azure OpenAI services for users.

Opinions

  • The author emphasizes the importance of checking Microsoft's documentation or error sections for the most up-to-date prerequisites and troubleshooting steps.
  • There is an opinion that the feature set of Azure OpenAI, including private networking, regional availability, and responsible AI content filtering, is superior for enterprise use compared to standard OpenAI offerings.
  • The author suggests that the Terraform code provided is a clear and maintainable way to deploy Azure OpenAI services, implying that Terraform is a preferred tool for infrastructure as code (IaC) in this context.
  • The article conveys that managing access through private endpoints and network access control lists (ACLs) is crucial for securing Azure OpenAI deployments.
  • The author provides a subjective recommendation to contact Microsoft or use provided links to request features such as customer-managed keys, indicating a potential gap in service capabilities that may require direct engagement with the provider.
  • There is an acknowledgment that the provided Terraform code handles specific configuration steps automatically, which may imply that the code is robust and user-friendly.
  • The author's inclusion of a .NET application to test the Azure OpenAI Chat GPT API suggests a preference or endorsement of using .NET for interacting with Azure OpenAI services.

How to Deploy Azure OpenAI with Private Endpoint and ChatGPT using Terraform

In this story, we will learn how to deploy Azure OpenAI and ChaptGPT using Terraform in Azure.

The Azure OpenAI Service gives customers access to managed AI services using OpenAI GPT-4, GPT-3, Codex, DALL-E, and Whisper models with Azure's security and enterprise promise.

Azure OpenAI co-develops the APIs with OpenAI, ensuring compatibility and a smooth transition from one to the other.

With Azure OpenAI, customers get the security capabilities of Microsoft Azure while running the same models as OpenAI.

Azure OpenAI offers private networking, regional availability, and responsible AI content filtering.

The code uses Azure Cognitive Services, which will also help deploy other models, such as ada, cabbage, curie, text-embedding-ada, cushman, davinci, etc.

1. Prerequisites

Before running our Terraform code, there are some extra steps required.

Note: steps listed below were required at the moment of writing this article. Maybe are not required when you deploy the code. Check Microsoft documentation or see the error section at the end to identify error caused by these prerequisites.

1.1. Enabling OpenAI in our Azure Subscription

Before running our Terraform code, we must request access to Azure OpenAI in our desired Azure subscription.

Note: Currently, access to this service is granted only by application. We can apply for access to Azure OpenAI by completing the form at https://aka.ms/oai/access.

1.2. Enabling GPT-4

GPT-4 models are the latest available models. Due to high demand, this model series is only available by request. To request access, existing Azure OpenAI customers can apply by filling out this form.

Note: this step was required at the moment of writing this article.

1.3. Enabling Customer Managed Keys

To request access, fill and submit the Cognitive Services & Applied AI Customer Managed Keys and Bring Your Own Storage access request form at https://aka.ms/cogsvc-cmk

Note: this step was required at the moment of writing this article.

2. OpenAI Models

Currently, we have three models for ChaptGT in development: the “gpt-35-turbo”, “gpt-4”, and “gpt-4–32k” models. When deploying these models, we must specify a model version.

For GPT-35-Turbo, depending on the region, only version “0301" is available, for GPT-4 models is the version “0314” and for GPT-4–32k, the version is “0613”.

Note: We can find information about OpenAI models on the Microsoft OpenAI models page.

3. Defining the Azure Provider

First, we will define Azure authentication variables.

We will use a Service Principal with a Client Secret. Check the link below for more info about Azure authentication for Terraform: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret

variable "azure_subscription_id" {
  type        = string
  description = "Azure Subscription ID"
}

variable "azure_client_id" {
  type        = string
  description = "Azure Client ID"
}

variable "azure_client_secret" {
  type        = string
  description = "Azure Client Secret"
}

variable "azure_tenant_id" {
  type        = string
  description = "Azure Tenant ID"
}

Then, we will configure Terraform and the Azure provider:

# Define Terraform provider
terraform {
  required_version = "~> 1.5"
}

# Configure the Azure provider
provider "azurerm" { 
  features {}  
  environment     = "public"
  subscription_id = var.azure-subscription-id
  client_id       = var.azure-client-id
  client_secret   = var.azure-client-secret
  tenant_id       = var.azure-tenant-id
}

4. Creating a Resource Group for OpenAI Resources

In this step, we will create an Azure Resource Group to store all the Azure resources created by our Terraform code.

We will start defining the location variable in the “variables.tf” file:

# Azure Region
variable "location" {
  type        = string
  description = "The region in which this module should be deployed"
  default     = "north europe"
}

And then the code to create the resource group code in the “main.tf” file:

# Create Resource Group
resource "azurerm_resource_group" "this" {
  name     = "kopicloud-openai-rg"
  location = var.location
}

5. User Assigned Identity

We are going to add the following code to our “main.tf” file to access information about an existing User Assigned Identity:

# Access information about an existing User Assigned Identity
resource "azurerm_user_assigned_identity" "this" {
  name     = "${lower(replace(var.company," ","-"))}-${var.app_name}-${var.environment}-${var.shortlocation}-identity"
  resource_group_name = azurerm_resource_group.this.name
  location            = azurerm_resource_group.this.location

  tags = {
    application = var.app_name
    environment = var.environment
  }
}

6. Creating a VNET and Subnet for OpenAI

Now, we will create the variables to configure the network:

# VNET CIDR
variable "vnet_address_space" {
  type        = string
  description = "VNET for OpenAI VNET"
}

# Subnet CIDR
variable "subnet_address_space" {
  type        = string
  description = "Subnet for OpenAI Public Endpoint"
}

And then the code to create the VNET and the Subnet:

Note #1: the line private_endpoint_network_policies_enabled = true in the resource “azurerm_subnet”; this setting is used to Enable or Disable network policies for the private endpoint on the subnet.

Note #2: the line service_endpoints = [“Microsoft.CognitiveServices”] is very important. Do not forget it!

# Create the VNET for Private Endpoint
resource "azurerm_virtual_network" "this" {
  name                = "kopicloud-openai-vnet"
  address_space       = [var.vnet_address_space]
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

# Create the Subnet for Private Endpoint
resource "azurerm_subnet" "this" {
  name                 = "kopicloud-openai-subnet"
  resource_group_name  = azurerm_resource_group.this.name
  virtual_network_name = azurerm_virtual_network.this.name
  address_prefixes     = [var.subnet_address_space]

  private_endpoint_network_policies_enabled = true

  service_endpoints = ["Microsoft.CognitiveServices"]
}

7. Private DNS Zone for OpenAI

We must register the private zone if we don’t have one for OpenAI.

# Create the Resource Group for DNS Zone
resource "azurerm_resource_group" "openai_dns_zone" {
  name     = "kopicloud-dns-rg"
  location = var.location
}

# Create Private DNS Zone for OpenAI
resource "azurerm_private_dns_zone" "openai" {
  name                = "privatelink.openai.azure.com"
  resource_group_name = azurerm_resource_group.dns_zone.name
}

Or, if we have an existing private DNS zone, we need to use a data resource to access the DNS zone.

# Access the private DNS Zone
data "azurerm_private_dns_zone" "openai_dns_zone" {
  name                = "privatelink.openai.azure.com"
  resource_group_name = "kopicloud-core-dns-rg"
}

8. Creating a Private Endpoint for OpenAI

In this step, we will create the Private DNS Zone Virtual Network Link and the Private Endpoint for OpenAI.

Note: remember to update the private_dns_zone_ids in the code below if you use data to access an existing private DNS zone (current example) or if you create a new private DNS zone.

# Create Private DNS Zone Virtual Network Link
resource "azurerm_private_dns_zone_virtual_network_link" "this" {
  name                  = "kopicloud-openai-vnet-link"
  resource_group_name   = azurerm_private_dns_zone.openai_dns_zone.resource_group_name
  private_dns_zone_name = azurerm_private_dns_zone.openai_dns_zone.name
  virtual_network_id    = azurerm_virtual_network.this.id
}

# Create the Private Endpoint
resource "azurerm_private_endpoint" "this" {
  name                = "kopicloud-openai-pe"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
  subnet_id           = azurerm_subnet.this.id

  private_service_connection {
    name                           = "kopicloud-openai-pe-psc"
    private_connection_resource_id = azurerm_cognitive_account.this.id
    subresource_names              = ["account"]
    is_manual_connection           = false
  }

  private_dns_zone_group {
    name                 = "default"
    private_dns_zone_ids = [data.azurerm_private_dns_zone.openai_dns_zone.id]
  }
}

9. Creating Azure OpenAI Variables

We will create some variables to deploy Azure Cognitive Services and OpenAI in the variables.tf file:

// Azure Cognitive Services Variables

variable "cognitive_account_sku_name" {
  type        = string
  description = "Specifies the SKU Name for this Cognitive Service Account"
  default     = "S0"
}

variable "cognitive_account_kind_name" {
  type        = string
  description = "Specifies the SKU Name for this Cognitive Service Account"
  default     = "OpenAI"
}

variable "cognitive_deployment_name" {
  type        = string
  description = "The name of the Cognitive Services Account Deployment model"
  default     = "gpt-35-turbo"
}

variable "cognitive_deployment_version" {
  type        = string
  description = "The version of the Cognitive Services Account Deployment model"
  default     = "0301"
}

variable "cognitive_deployment_scale_type" {
  type        = string
  description = "Deployment scale type"
  default     = "Standard"
}

10. Deploying the Azure Cognitive Account

We will start the Azure OpenAI deployment with the Azure Cognitive Account instance:

resource "azurerm_cognitive_account" "this" {
  name                = "kopicloud-openai-aca"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
  kind                = var.cognitive_account_kind_name
  sku_name            = var.cognitive_account_sku_name

  public_network_access_enabled      = true
  outbound_network_access_restricted = true

  custom_subdomain_name = "kopicloud-openai"

  network_acls {
    default_action = "Deny"

    virtual_network_rules {
      subnet_id = azurerm_subnet.this.id
    }
  }

  identity {
    type         = "SystemAssigned, UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.this.id]
  }
}

Important: When we enable a private endpoint, it is important to configure the public access as in the example above, or the OpenAI Studio will not be able to communicate with our OpenAI instance.

Changing network rules can impact our applications’ ability to connect to Azure AI services. Setting the default network rule to deny blocks all access to the data unless specific network rules that grant access are also applied. Before we change the default rule to deny access, we need to grant access to any allowed networks by using network rules. If we allow listing for the IP addresses for our on-premises network, we need to add all possible outgoing public IP addresses from our on-premises network.

For more information, refer to the following Microsoft link.

11. Creating the Azure Cognitive Deployment

And the final step is to deploy our OpenAI GPT35 instance:

# Create the Azure Cognitive Deployments
resource "azurerm_cognitive_deployment" "this" {
  name                 = "gpt35"
  cognitive_account_id = azurerm_cognitive_account.this.id
  
  model {
    format  = "OpenAI"
    name    = var.cognitive_deployment_name
    version = var.cognitive_deployment_version
  }

  scale {
    type = var.cognitive_deployment_scale_type
  }
}

Note: the code deploys only one Azure Cognitive Deployment per Azure Cognitive Account, which is, for clarity, only. We will probably deploy multiple deployments per account. See the repo for a more complex example that deploys multiple deployments.

12. Troubleshooting and Errors

Below is a list of errors we got during our test of Azure OpenAI.

12.1. BringOwnFeatureNotEnabled: Bring your own feature is not enabled for Subscription/SKU/Kind

When trying to use the customer-managed key, you get “Unexpected status 400 with error: BringOwnFeatureNotEnabled: Bring your own feature is not enabled for Subscription/SKU/Kind error:

module.openai.azurerm_cognitive_account_customer_managed_key.this: Creating…
╷
│ Error: adding Customer Managed Key for Account (
Subscription: "aaaaa-bbbb–cccc–dddd-ffffffffff"Resource Group Name: "kopi-testai-rg"Account Name: "kopi-testai-aca"): unexpected status 400 with error: 
BringOwnFeatureNotEnabled: Bring your own feature is not enabled for 
Subscription/SKU/Kind.
│
│ with module.openai.azurerm_cognitive_account_customer_managed_key.this,
│ on ..\terraform-azurerm-openai\openai_account.tf line 19, in resource 
"azurerm_cognitive_account_customer_managed_key" "this":
│ 19: resource "azurerm_cognitive_account_customer_managed_key" "this" {

Solution:

This error means the customer-managed key feature is not enabled in your Azure subscription. Contact Microsoft or use one of the links above to request the feature.

12.2. InvalidResourceProperties: The specified SKU ‘Standard’ of account deployment is not supported in this region ‘westeurope’.

When I’m trying to deploy GPT4 in the West Europe region, I got this error:

 Error: creating Deployment (Subscription: "aaaaa-bbbb–cccc–dddd-ffffffffff"Resource Group Name: "kopi-testai-rg"Account Name: "kopi-testai-aca"Deployment Name: "kopi-testai-gpt4"): performing CreateOrUpdate: 
unexpected status 400 with error: InvalidResourceProperties: 
The specified SKU 'Standard' of account deployment is not supported in 
this region 'westeurope'.

Solution:

At the moment of writing (October 2023), GPT-4 is available to customers in these regions:

  • Australia East
  • Canada East
  • East US 2
  • Japan East
  • UK South

12.3. Error: Public access is disabled. Please configure private endpoint on Azure OpenAI Studio Chat Playground

When we try to use Azure OpenAI Chat from the Azure OpenAI Studio Chat Playground with a private endpoint enabled, we will receive the following message: Error: Public access is disabled. Please configure private endpoint.

Also, we can receive this error: Failed to retrieve fine tuned models — Public access is disabled. Please configure private endpoint.

Solution:

We must allow the Azure OpenAI to access the public network, restricting access only to our VNET and Subnet.

We open the Azure OpenAI service console and click on the Networking link.

Then click on the Selected Networks and Private Endpoints option.

  • Click on Add existing virtual network, and select the VNET and subnet(s) where Azure OpenAI is configured.
  • In the Firewall section, add the recommended external IP address.

Note: the Terraform code automatically takes care of these two steps above.

12.4. Error: Principal does not have access to API/Operation on Azure OpenAI Studio Chat Playground

When we try to use Azure OpenAI Chat from the Azure OpenAI Studio Chat Playground, we receive the error Principal does not have access to API/Operation:

Solution:

We need to assign the role “Cognitive Services OpenAI Contributor” or “Cognitive Services Contributor” to the user in the Azure portal or using Terraform code (see below).

Get the Object ID or Principal ID of the user on Microsoft Entra ID (Azure Portal):

Then use to assign to the “Cognitive Services OpenAI Contributor” role:

# Assign OpenAI Contributor Role to the Azure Portal User
resource "azurerm_role_assignment" "openai_contributor" {
  scope                = azurerm_cognitive_account.this.id
  role_definition_name = "Cognitive Services OpenAI Contributor"
  principal_id         = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"  
}

Or the “Cognitive Services Contributor” role:

# Assign Cognitive Services Contributor Role to the Azure Portal User
resource "azurerm_role_assignment" "contributor" {
  scope                = azurerm_cognitive_account.this.id
  role_definition_name = "Cognitive Services Contributor"
  principal_id         = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"  
}

Finally, log out from the Azure OpenAI Studio and log back.

12.5. Error: The Terraform Execution failed when Executing “azurerm_role_assignment”

When we run the Terraform code to assign permissions to a Principal ID or Object ID, we get the following error:

│ Error: authorization.RoleAssignmentsClient#Create: Failure responding to request: StatusCode=403 — Original Error: autorest/azure: Service returned an error. Status=403 Code=”AuthorizationFailed” Message=”The client ‘xxxx’ with object id ‘yyyyyyyyyyyyyyy’ does not have authorization to perform action ‘Microsoft.Authorization/roleAssignments/write’ over scope ‘/subscriptions/zzzzzzzzzzzz/resourceGroups/kopicloud-core-dev-we-rg/providers/Microsoft.CognitiveServices/accounts/kopicloud-core-dev-we-aca/providers/Microsoft.Authorization/roleAssignments/wwwwwwww’ or the scope is invalid. If access was recently granted, please refresh your credentials.” │ │ with azurerm_role_assignment.contributor, │ on openai-account.tf line 55, in resource “azurerm_role_assignment” “contributor”: │ 55: resource “azurerm_role_assignment” “contributor” {

Solution:

We need to add our Terraform service principal to the Owner role in the Azure portal.

The complete code is available at https://github.com/KopiCloud/terraform-azure-openai-private-endpoint

Also, check the code of a small .NET app I wrote to test the Azure OpenAI Chat GPT API at https://github.com/KopiCloud/dotnet-azure-openai-test

And that’s all, folks. If you liked this story, please show your support by 👏 this story. Thank you for reading!

Terraform
Azure
OpenAI
ChatGPT
Openai Chatgpt
Recommended from ReadMedium