How to use an RDS proxy to manage connection pools?

What is RDS Proxy?

RDS Proxy is a service from AWS which will act as a proxy for RDS services. This managed service from AWS is highly available and helps with increasing the scalability, resilience & security of your applications.

At the time of writing this article, Amazon RDS Proxy is available for Amazon Aurora with MySQL compatibility, Amazon Aurora with PostgreSQL compatibility, Amazon RDS for MariaDB, Amazon RDS for MySQL, Amazon RDS for PostgreSQL, and Amazon RDS for SQL Server.

Why Do I need RDS Proxy?

Consider this simple API design using API Gateway, Lambda & RDS —

As you can see, the API gateway is passing on requests to Lambda which in turn makes a connection to the RDS, but since this is a serverless architecture, lambda has no way of guaranteeing that it will be able to reuse connections. Hence, each individual request will end up making a new connection. This is inefficient and we will end up hitting the maximum number of connections allowed.

How to determine the maximum number of connections allowed?

This can vary by the type of DB Engine you are using and you can find them here on AWS’s documentation page — https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html#RDS_Limits.MaxConnections

At the moment, this is how the limits on connection look like

So, for example, if you are running a MySql instance on a t2 micro with 1GB RAM the max_connections could be (1024*1024*1024)/12582880 ~= 85. This is not a very scalable setup and will cause performance issues for the end user once it reaches its limit.

How does the RDS proxy solve this problem?

RDS proxy is an AWS service, which will sit in front of the RDS service and manage a connection pool. Our applications (like the Lambda function) will connect to the RDS Proxy service instead of the actual RDS itself, and from there onwards, this Proxy service will manage the connections via the connection pool.

Some of the core benefits of using proxy service are —

• Connection shares between various request calls will reduce stresses on the CPU & Memory
• Higher availability of services
• Increase in efficiency
• Better control of authentication by managing DB credentials via AWS’s secret manager

How to implement RDS Proxy?

Step 1: Create a new secret to store credentials for your RDS database

Log into AWS, and create a new secret for your RDS database

Make sure that you select the correct database from the list

Make a note of the Secret ARN, we will need it in our next step

Step 2: Create an IAM role which will allow the RDS proxy to read this secret.

You can do that by creating a new IAM role and adding the following permissions

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetResourcePolicy",
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret",
        "secretsmanager:ListSecretVersionIds"
      ],
      "Resource": [
        "ARN_OF_SECRET_FROM_STEP1"
      ]
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetRandomPassword",
        "secretsmanager:ListSecrets"
      ],
      "Resource": "*"
    }
  ]
}

Then, add the following trust policy to the role so that the RDS is allowed to assume the role

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Sid": "",
   "Effect": "Allow",
   "Principal": {
    "Service": "rds.amazonaws.com"
   },
   "Action": "sts:AssumeRole"
  }
 ]
}

Now, again take note of the ARN of the newly created role, we will need it when we attach this role to our lambda proxy.

Step 3: Add RDS proxy to lambda function

head over to Lambda in the AWS console and open the function you wish to attach the lambda with. Then, click on the configuration tab and find database proxies. Click on add database proxy

here, enter the relevant details from steps 1 and 2 and click add.

Make a note of the proxy endpoint, we will use it to make the connection from our lambda function

Step 4: Adapt your code to use a proxy endpoint to make the connection. the following is a nodeJS example using Sequelize ORM —

import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
import { Signer } from "@aws-sdk/rds-signer";
import { Sequelize } from "sequelize";

const signer = new RDS.Signer({
  region: process.env.REGION,
  hostname: process.env.DB_PROXY_ENDPOINT,
  port: process.env.DB_PROXY_PORT,
  username: process.env.DB_USERNAME,
});

//Use this sequelize object to perform DB queries
const sequelize = new Sequelize({
  process.env.DB_USERNAME,
  process.env.DB_PROXY_ENDPOINT,
  process.env.DB_PROXY_PORT,
  dialect: "mysql",
  dialectOptions: {
    ssl: "Amazon RDS",
    authPlugins: {
      mysql_clear_password: () => () => signer.getAuthToken(),
    },
  },
});

Thanks for reading and if you like my content and want to support me, then please follow me and help all of us on this platform grow and produce useful content by becoming a member.