Deploy Cloud Functions on GCP with Terraform (2nd Gen Environment)

Set up a Cloud Function — Gen2 in Google Cloud Platform (GCP) that is triggered whenever a file is uploaded to a specific Google Cloud Storage (GCS) bucket.

Cloud Functions offers two product versions: Cloud Functions (1st gen), the original version, and Cloud Functions (2nd gen), a new version built on Cloud Run and Eventarc to provide an enhanced feature set.

Cloud Function — Generation 2: Uses Cloud Run in the background!

The 2nd generation of Cloud Functions is a significant advancement in the serverless computing space. It builds upon the success of the first generation and introduces several key enhancements that empower developers to build more robust and scalable applications. Let’s go deeper into the 2nd generation of Cloud Function.

1. 1st Generation: First version
2. 2nd Generation: New version built on Cloud Run and Eventarc

1st Generation vs 2nd Generation:

Differences as below:

Key Enhancements in 2nd Generation:

• Longer request processing times: Cloud Functions 2nd Generation can handle HTTP requests that take up to 60 minutes to execute.
• Larger instance sizes: Cloud Functions 2nd Generation can use instances with up to 16 GiB of RAM and 4 virtual CPUs.
• Improved concurrency: Cloud Functions 2nd Generation can handle up to 1000 concurrent requests per function instance.
• Traffic management: Cloud Functions 2nd Generation supports multiple function revisions and traffic splitting
• Eventarc integration: Cloud Functions 2nd Generation has native support for Eventarc triggers, bringing all 90+ event sources supported by Eventarc to Cloud Functions.
• Broader CloudEvents support: Cloud Functions 2nd Generation supports industry-standard CloudEvents in all language runtimes, providing a consistent developer experience.

The console of cloud function as shown below:

NOTE: Will focus on creating the function using terraform.

In this tutorial, you will learn how to set up a Cloud Function — Gen2 in Google Cloud Platform (GCP) that is triggered whenever a file is uploaded to a specific Google Cloud Storage (GCS) bucket.

Prerequisites

• Terraform installed on your local machine
• Google Cloud SDK installed on your local machine
• Google Cloud Platform project set up
• Enable the Cloud Functions API

If API's are not enable the use the below commands. 
# gcloud services enable cloudfunctions.googleapis.com
# gcloud services enable eventarc.googleapis.com
# gcloud services enable cloudresourcemanager.googleapis.com
# gcloud services enable cloudrun.googleapis.com
# gcloud services enable pubsub.googleapis.com
# gcloud services enable cloudstorage.googleapis.com
# gcloud services enable cloudlogging.googleapis.com
# gcloud services enable cloudmonitoring.googleapis.com
# gcloud services enable cloudbuild.googleapis.com
# gcloud services enable cloudcode.googleapis.com

We will focus on deploying the following resources using Terraform:

• Bucket for file uploads: Cloud Storage bucket that will be used to upload files to. It provides a scalable and durable storage solution for storing files.
• Bucket for Cloud Function source code: Bucket that will store the source code for the Cloud Function. The Cloud Function will be triggered by file uploads to the first bucket.
• Cloud Function: Serverless function that runs in response to events.

Folder Structure:

• src folder contains the Python source code of the cloud function
• terraform folder contains the configuration files

Create Python function:

src/main.py: The fileUpload function is the entry point of the Cloud Function. It takes two parameters: event and context. The event parameter contains information about the file upload event, such as the bucket name, file name, and file size. You can extract and use this information in your function.

def fileUpload(event, context):
    file_data = event

    # Extract relevant information from the event
    bucket_name = file_data['bucket']
    file_name = file_data['name']
    file_size = file_data['size']

    # Perform desired operations on the uploaded file
    # For example, you can process the file, store metadata, or trigger other actions

    print(f"File uploaded: {file_name} in bucket: {bucket_name}")
    print(f"File size: {file_size} bytes")

    # Add your custom logic here

    # Return a response (optional)
    return "File processing completed"

Create Terraform Infrastructure:

• provider.tf: To declare the connection to the Google provider in Terraform, you need to specify the provider block in your Terraform configuration file.

terraform {
  required_version = ">= 1.0"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 4.69.1"
    }
  }
}

# provider "google" {
#   credentials = file("gcp-project.json")
#   project     = var.project_id
#   region      = var.region
# }

• backend.tf: To configure the backend for storing and retrieving the Terraform state.

terraform {
  backend "gcs" {
    bucket = "gcp-cloud-function-terraform-bucket-" # GCS bucket name to store terraform tfstate
    prefix = "function"               # Prefix name should be unique for each Terraform project having same remote state bucket.
  }
  # backend "local" {}
}

Note: Terraform stores the state file locally in the same directory as the Terraform configuration files. choose ‘local’ to stored in the local directory.

• variables.tf: Declare the variables used in the Terraform files.

variable "project_id" {
  type = string
  default = "<YOUR-PROJECT-ID>"
}

variable "region" {
  type = string
  default = "europe-west2"
}

variable "service_account_email" {
  type = string
  default = "your-service-account-email@your-project-id.iam.gserviceaccount.com"
  
}

NOTE: make sure service-account have all the relevent premsissions.

• storage-bucket.tf: Cloud Storage buckets to store the code of the Cloud Function and to upload files.

resource "google_storage_bucket" "Cloud_function_bucket" {
  name          = "cloud-function-${var.project_id}"
  location      = var.region
  project       = var.project_id
  force_destroy = true
  uniform_bucket_level_access = true
}

resource "google_storage_bucket" "input_bucket" {
  name          = "input-${var.project_id}"
  location      = var.region
  project       = var.project_id
  force_destroy = true
  uniform_bucket_level_access = true
}

• cloudfunctiongen2.tf: Declare the Cloud Function

# Generates an archive of the source code compressed as a .zip file.
data "archive_file" "source" {
  type        = "zip"
  source_dir  = "${path.module}/src"
  output_path = "${path.module}/tmp/function.zip"
}

# Add source code zip to the Cloud Function's bucket (Cloud_function_bucket) 
resource "google_storage_bucket_object" "zip" {
  source       = data.archive_file.source.output_path
  content_type = "application/zip"
  name         = "src-${data.archive_file.source.output_md5}.zip"
  bucket       = google_storage_bucket.Cloud_function_bucket.name
  depends_on = [
    google_storage_bucket.Cloud_function_bucket,
    data.archive_file.source
  ]
}

resource "google_cloudfunctions2_function" "function" {
  name = "Cloud-function-trigger-using-terraform-gen-2"
  location = var.region
  description = "Cloud function gen2 trigger using terraform "

  build_config {
    runtime     = "python39"
    entry_point = "helloGET" 
    environment_variables = {
      BUILD_CONFIG_TEST = "build_test"
    }
    source {
      storage_source {
        bucket = google_storage_bucket.Cloud_function_bucket.name
        object = google_storage_bucket_object.zip.name
      }
    }
  }

  service_config {
    max_instance_count  = 3
    min_instance_count = 1
    available_memory    = "256M"
    timeout_seconds     = 60
    environment_variables = {
        SERVICE_CONFIG_TEST = "config_test"
    }
    ingress_settings = "ALLOW_INTERNAL_ONLY"
    all_traffic_on_latest_revision = true
    service_account_email = var.service_account_email
  }

  event_trigger {
    trigger_region = var.region
    event_type = "google.cloud.storage.object.v1.finalized"
    retry_policy = "RETRY_POLICY_RETRY"
    service_account_email = var.service_account_email
    event_filters {
      attribute = "bucket"
      value = google_storage_bucket.input_bucket.name
    }
  }
  depends_on = [
    google_storage_bucket.Cloud_function_bucket,
    google_storage_bucket_object.zip
  ]
}

Deploy cloud function:

• Start with initializing the Terraform workspace. Aterraform initdownloads all the required providers and plugins. Run a Terraform plan creates an execution plan. The execution plan looks good, so let’s move ahead and apply this plan.

$ terraform init
$ terraform fmt
$ terraform validate
$ terraform apply -auto-approve

Cloud-Function- Gen2:

Storage buckets:

To test if everything is working correctly, follow these steps:

1. Open the Google Cloud Console and log in to your project.
2. Navigate to the Google Cloud Storage browser.
3. Click on the bucket named input-<YOUR-PROJECT-ID>.
4. Upload any file into the bucket to trigger the Cloud Function.
5. To verify that the Cloud Function was triggered, go to the Cloud Functions list in the Google Cloud Console.

This test ensures that the Cloud Function — Gen2 is successfully triggered, whenever a file is uploaded to the bucket input-<YOUR-PROJECT-ID>, and you can confirm its execution by checking the logs.

• Destroy: To destroy Terraform-provisioned infrastructure.

 $ terraform destroy --auto-approve

Reference: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloudfunctions2_function#example-usage---cloudfunctions2-full

I trust that you have found this user-friendly.

Please share your thoughts and experiences after following the steps outlined. Your feedback is valuable and helps us improve the quality.

Topics:

• Deploy Cloud Functions on GCP with Terraform- (Environment -1st Gen)
• Provision GKE Cluster with Terraform Using Module
• Provision EKS Cluster with Terraform Using Modules
• Terraform Tools That You Need

Do not forget the 👏✌️❤️ if you like this content! Also, I will be glad if you hit the follow button so you get notified of my new posts. You can also follow me on LinkedIn! Thank you!

Further Reading: