Summary

The provided content is a comprehensive guide on building a webhook endpoint using FastAPI to receive GitHub webhook events, including steps for local testing with ngrok and securing the endpoint with HMAC signatures.

Abstract

The article details the process of creating a webhook endpoint with FastAPI, a Python web framework known for its high performance and asynchronous capabilities. It explains how to set up an endpoint to receive GitHub webhook events, such as pushes, and demonstrates the use of ngrok to expose a local server to the internet for testing purposes. The guide also emphasizes the importance of securing the webhook endpoint by verifying incoming requests using HMAC signatures included in the request headers. Additionally, it provides insights into handling request payloads directly rather than through Pydantic models to ensure signature consistency, and it concludes with pointers to further resources for deploying the endpoint to a production environment and integrating with GitHub Apps.

Opinions

The author suggests that FastAPI is an efficient choice for creating a Python application due to its asynchronous and high-performance nature.
Using ngrok is recommended for developers to test webhook endpoints without the need for deployment to a production environment.
The article advocates for the security of webhook endpoints by implementing HMAC signatures to authenticate requests, thus preventing potential malicious attacks.
Directly using the Request object to access the raw payload is advised to maintain the integrity of the payload and ensure accurate signature verification.
The author values the reusability and sharing of code, providing a link to a GitHub repository with a complete example of the implementation discussed.
The guide is presented as part of a series, with the author intending to build upon the foundational knowledge by discussing advanced topics such as deploying to API Gateway and Cloud Run, and building GitHub Apps with FastAPI.

Build a Webhook Endpoint with FastAPI

FastAPI is an asynchronous and high-performance framework that can let you create your Python application efficiently. In this post, we are going to demonstrate how to build a webhook endpoint with FastAPI to receive webhook events from the internet.

For demonstration purpose, we will build an application that can receive GitHub Webhook events.

Create an endpoint for receiving webhook events

Let’s create an endpoint for receiving POST payload from GitHub. First, we define a POST endpoint, /webhook. The request argument is for storing the POST request data, we'll discuss it later. Here we return 202 to GitHub which means our server has successfully received the event.

Put localhost to the internet

We can use ngrok to expose localhost to the internet, so we don’t need to deploy to the production to receive the webhook events. Instead, we can directly test our application from localhost.

After you install ngrok, enter the following command, then it will create a tunnel that connects your localhost to the internet. 8000 is the port number that our server will listen to for webhook events.

ngrok http 8000

You will see the following line in your terminal,

Forwarding https://28c0-31-220-193-162.ngrok.io -> http://localhost:8000

Therefore, https://28c0-31-220-193-162.ngrok.io/webhook is the temporary endpoint for receiving POST payload from GitHub.

Set up webhook in your repository

Go to Settings page of your repository, select Webhooks on the left panel, then click Add webhook button. You will see a webhook configuration page.

Payload URL

First, paste the https://28c0-31-220-193-162.ngrok.io/webhook to Payload URL field, so the webhook event will be POST to this endpoint.

Content Type

Choose the default one, application/json. You can of course based on your use case to select the best fit one.

Secret

We will discuss this in the later section.

Which events would you like to trigger this webhook?

By default, only the push event will trigger the webhook event. You can select Send me everything, so you can easily trigger webhook events for development purposes. Last, click Add webhook button to enable it for your repository.

Test receiving ping event

When creating a new webhook, GitHub will send you a ping event to your endpoint, and let you know the webhook has been set up correctly. By clicking Recent Deliveries, you can see the history of delivered webhook events. In the below image, you can see the ping event has successfully POST to our endpoint and received 202 response. Not bad, we can receive webhook events now!

Secure your webhook endpoint

If we don’t authenticate incoming requests, we may receive some malicious requests which may break our service. First, go to webhook configuration page, and create a secret value in Secret field. Remember, do not store the secret value in your code! Instead, you can put it in an environment variable file and don’t track it in your git.

# key.txt

WEBHOOK_SECRET=<secret_value>

When running your service, specify the path to --env-file argument, e.g.

uvicorn src.main:app --reload --env-file key.txt

Then, you can use os module to get the environment variable.

GitHub will use this secret value and payload to generate a hash signature, then put it in request headers, X-Hub-Signature and X-Hub-Signature-256. The front one is generated using SHA-1, the later one is using SHA-256. Here, we will use SHA-1 version for demonstration.

To begin with, create a generate_hash_signature function that takes two positional arguments, secret value and request payload; digest_method by default is hashlib.sha1. hmac.new will return an HMAC object. Since GitHub uses HMAC hex digest to compute hash, we call hexdigest method of the HMAC object to return hexadecimal digits.

Subsequently, in webhook function, get environment variable, WEBHOOK_SECRET, then pass secret value and payload to generate_hash_signature, it will return a hash signature. Compare the generated hash signature with x_hub_signature, if they are equal, which means this is an authenticated request, our server is happy for handling it; on the other hand, if they are not equal, which means it might be a malicious request, we'll need to raise an exception with 401 status code representing this is an unauthorized request. The below screenshot is an example that a request that failed to authenticate while sending a webhook event to our server.

Convert HTTP headers

Some people may notice that we add x_hub_signature argument to webhook function. Declare variable with Header parameter, FastAPI will automatically convert mapped header name to the python variable. You just need to declare the variable name as it is in HTTP headers (case-insensitive and snake_case), then the header value will be assigned to the declared variable. See more information here.

Why using Request directly, not a Pydantic model?

WE can of course define a Pythantic model to convert request body to a Pydantic object. However, the x_hub_signature is generated with secret value and payload, so we should use the same payload to re-generate the hash signature, any space or key order difference may cause the generated result different. That's why we have payload = await request.body() to make sure we use the same request body as GitHub's.

Conclusion

It is easy and fast to build a webhook endpoint with FastAPI. First, declare a variable with Request parameter and retrieve the raw request body; Second, build a function to generate a hash signature with secret value and raw request payload; Third, authenticate the request by comparing the signatures.

For testing webhook, use ngrok to create a temporary tunnel for receiving webhook events from the internet.

Deploy a Webhook Endpoint Built with FastAPI to API Gateway and Cloud Run is the follow-up article to demonstrate how to expose a public webhook endpoint through API Gateway with API Key, hence, we can securely receive webhook events from the internet.

We’ll have a further article to introduce how to use the GitHub Webhook event to build our own integration, such as GitHub Apps. [UPDATED 11/06/2022] Build a GitHub App with FastAPI.

Thanks for your reading. If you have any questions or thoughts please feel free to leave a message. You can also find the complete example in this [repository](https://github.com/Shawnice/sample-fastapi-webhook).