VPC with a NAT and a VPC Endpoint Deployed With CloudFormation

ACM.276 Allow a Lambda function in a private network to access GitHub via a NAT and AWS CodeCommit via a VPC Endpoint

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Part of my series on Automating Cybersecurity Metrics. The Code.

🔒 Related Stories: Git Security | Application Security | Secure Code | AWS Security | Network Security

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~In the last post I pondered how I could further simplify code to consistently get IDs or ARNs for resources created by a CloudFormation stack. Hopefully that will simplify my code and in general, help me get things done faster.

Now let’s see if I can finish this basic network deployed for my repository syncing process. I’ve laid a lot of the groundwork already in this post that deploys a public and private subnet in a VPC. I’m going to build on and add to this code.

Essentially the end goal is this — A VPC with a public and private subnet. A public NAT allows traffic to reach the Internet from the private subnet. In the private subnet we have a VPC Endpoint that allows a Lambda function to reach an AWS Service — Code Commit — without traversing the Internet or leaving the AWS Network.

As noted in prior posts this is an initial test and I have a more grandiose long term solution, but for someone starting out this should work fine.

I’ve been reusing a lot of the code I wrote in prior posts.

NAT Instance vs. NAT Gateway

Just to get this out of the way, you used to have to install NATs as EC2 instance and do all kinds of funky coding to create two NAT instances that shared route information for failover. It was not fun. I did it for Capital One. I cannot recommend it. Use a NAT Gateway. Here’s more information on that if you care to explore it further:

So any time I say “NAT” in my posts I mean an AWS NAT Gateway.

When the NAT Gateway became available, there was much rejoicing at re:Invent. I am not sure if anyone still deploys a NAT as an EC2 instance. The only reason I can think of is the underlying way you set up a NAT may still be used by some firewall devices you can deploy on AWS, but when using that mechanism they will not be very scalable compared to a load-balanced implementation with a proxy and reverse proxy or something to that effect.

NAT versus Bastion Host

I remember when I was first learning about NATs and Bastion Hosts. I knew that both allowed traffic for a private network. I remember asking someone the difference and they couldn’t explain it. It’s so simple.

OUT (Egress): A NAT is typically allowing resources in your network to send traffic out of the private network.

IN (Ingress): A Bastion Host (or jump box) is used to allow people to login to that host and then from there, connect to things in the private network.

Just thought I’d explain that in case anyone else is trying to understand the use case for these two resources in your security architecture.

NAT Gateway CloudFormation Resource and Properties

The CloudFormation to add a NAT Gateway looks like this:

Allocation ID: Since we are creating a public NAT Gateway we need to first create an Elastic IP and then specify the ID here.

ConnectivityType: We’ll use the default which is Public.

MaxDrainDurationSeconds: We’ll leave this at the default This is how long before an IP is forcibly released when connections are still in progress.

PrivateIpAddress: N/A

SecondaryAllocationIds: We’ll not use this for now but read the documentation and consider whether you need this for your scenario.

SecondaryPrivateIpAddressCount: N/A

SubnetId: We’ll specify the ID of the subnet we create for the NAT.

Tags: We’ll tag the NAT with a Name since there’s no name property for a NAT Gateway.

Sandbox Network Deployment Script and Caveats

I’m going to to test this deployment in my Sandbox account. As a reminder the network code in my repository is located here.

SecurityMetricsAutomation/Network/stacks

I already revised my scripts so I could create a VPC with a private and public route table and associated subnets. The deployment script I created is in the above folder and named:

deploy_sandbox.sh

Now I need to add a few things:

• NAT
• Route for NAT
• VPC Endpoint to reach CloudFormation
• Security Groups for my Lambda Function
• Security Group for the VPC endpoint

Just to be complete:

You cannot associate a security group with a NAT gateway.

NAT VPC, Subnets, NACLS, and Route Tables

As mentioned above I already deployed the NAT VPC, Subnets, NACLS, and Route Tables in this post:

We’re going to build on that here.

Security group to access GitHub

The Github prefix list and Security Group I already created should come in handy. We need to allow our Lambda function to access only GitHub on the Internet with a security group that allows that access. We already created one of those in a prior post.

You can read more about that here in the story about prefix lists.

Elastic IP Address for the NAT

Next, I want an elastic IP address for the NAT so I can allow that single IP to access my GitHub repositories.

This is one template and code I didn’t have to really change.

I need an elastic IP address for my NAT and I have a function to create an EIP here.

The template looks like this.

NAT CloudFormation Template

I’m going to use the outputs from the EIP and Subnet CloudFormation stacks to populate the related properties in my NAT teampate.

• EIP ID output from the EIP stack.
• Subnet ID from the Subnet.yaml template for the public subnet.

I’m going to add parameters for the names of those output names.

NAT Function

Next I add a function to deploy the NAT to my network functions.

NAT Deployment

Now I can add a call to that function in the Sandbox deployment script, passing in the public subnet name.

Add a Route for the NAT

Now I need to add a route for our NAT to the route table. I’m going to send all traffic (0.0.0.0/0) to the NAT ID.

I’m going to make use of the functionality I’m using to match CloudFormation outputs with an ARN or ID to resource names. The name of my NAT is SandBoxNAT and the output with the same name from my NAT stack returns the ID.

I’m going to create an optional parameter to pass in the GatewayName which will then retrieve the GatewayID using the ImportValue function.

I add a new route type called ‘NAT’ to the AllowedValues list and create an optional parameter for the gateway name:

I add a new condition for the NAT route table type:

I basically copy and modify the code that adds an IGW route if the route type is Public and instead point the traffic to the NAT Gateway ID. I can remove the dependency on the VPCAttachment.

Note that I had to change GatewayID to NatGatewayID. This takes a very log time to timeout. It seems like CloudFormation could figure out the value passed in is not an IGW or vice versa and report a better error message sooner. #awswishlist

I deploy the NAT route table after the NAT is created. I have to pass in the name of the NAT.

I move my code to deploy the private subnet after the deployment of the NAT route and change the route table type to NAT.

I can remove the code that deployed the private route table.

GitHub Prefix List

I thought the GitHub Prefix List would just work, but as noted I had to fix a few things in that code as it decided to fail for whatever reason.

VPC Endpoint

I needed to resolve a number of issues with the VPC Endpoint code, the first of which I wrote about in this post.

The VPC Endpoint code I wrote earlier was too specific to CloudFormation so I revised the policy document to be more generic. I also had to remove a lot of hard coded resource ARNS and look up ARNs using exports. That led me to a more simplified and generic way to look up IDs and ARNs using CloudFormation exports.

Basically, I’m making sure that all templates going forward have an output name that matches the resource name and returns an ID or ARN. This consistency will hopefully help me write less code and remember output names more easily to get things done faster.

The other thing I had to do was decide which endpoint I want to connect to in my code.

FIPS2 compliant VPC endpoints for CodeCommit

AWS started offering FIPS2 compliant endpoints earlier this year in certain regions. I looked around to see if that cost more and I didn’t see any difference in pricing. However, those are only available in certain regions. They are available in my region so I’ll use them if I can. You can view all available FIPS2 endpoints here:

Git endpoints for CodeCommit

Also there are two different types of endpoint — endpoint for git commands only and endpoints to API commands. I am going to use the git option. Recall that I showed you how to enforce use of MFA with git commands with codecommit here:

At the time of this writing these are the endpoints:

Based on my above criteria I am going to use this. I’ll test it to see if it works.

com.amazonaws.region.git-codecommit-fips

I basically had to do a lot of reformulation of the code to support all of the above and the new policy document changes. It took a while and I won’t go into all the details. Here’s the working code. Finally!

I made a new VPCEndpoint-Generic.yaml template:

Changes include:

• Pass in service name (CodeCommit)
• Pass in endpoint service name (codecommit-git-fips)
• Pass in all the values to formulate the PolicyDocument.
• Pass in the export names which now match the names of the VPC, Subnet, and Security Group.
• Add a tag name tag.

Tags or no tags?

I want to match the name of the export that returns the ID to the name of the resource in the tag. I thought VPC Endpoints supported tags based on this announcement:

But apparently they are not added to CloudFormation yet.

The function to call this template:

The code in the deploy script to call the function:

Note that code also deploys the security groups.

I created a new function to deploy VPCE security group rules.

VPCEndpointAccess rules CloudFormation to support any service:

VPCEndpointInterface rules CloudFormation to support any service:

Note that I use this method temporarily to get the arn for the WebAdmin user. This is a preliminary test and I still need to go back and finalize my IAM infrastructure. I have other plans for that and just wanted to get this done.

acct=$(aws sts get-caller-identity --profile SandboxAdmin --output text --profile $profile | cut -f1)
readprincipals="arn:aws:iam::$acct:user/WebAdmin"

Verify the Deployment

I had to deploy and redeploy and fix what felt like a million things before this finally all worked. At this point I have all the resources deployed.

That doesn’t mean this is going to work as I expect.

Note: I fixed an issue in this post that occurred after I added the NAT to the route table CloudForamtion template.

If you are having problems with your VPC Endpoint check out these posts:

Now I have to deploy my Lambda function, test, and troubleshoot any issues. That’s up next.

I’ll publish this code when I’m done with this sub-series on static websites in S3 buckets.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2023

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab