AWS FIPS CodeCommit VPC Endpoint Returns Public IP Address

May be as designed but if not, someone should check this — and if by design it makes network security configuration difficult

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics | Code.

🔒 Related Stories: Bugs | AWS Security | Secure Code | CloudFormation

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Every day I’m just trying to test some simple thing and I stumble across things that make me spend time troubleshooting things I wasn’t expecting and writing tangentially related blog posts. This is one of them.

I’m testing mirroring a GitHub repo to an AWS CodeCommit repo.

I’ve written many posts about VPC endpoints and private networks, and how if you don’t configure them correctly, your traffic still may be traversing the public Internet.

But here’s the thing. Sometimes AWS uses public IP addresses and the traffic still remains on the AWS backbone. It’s just hard to tell if you send your traffic off to a public IP address if it really does stay on the AWS backbone or not. I guess it’s possible that you send something to a private IP and it somehow ends up on the public Internet if AWS has something misconfigured, but it feels less likely to me.

So I’m testing cloning an AWS CodeCommit repo and I setup a FIPS-compliant VPC endpoint (versus the standard CodeCommit VPC endpoint).

When you use a FIPS compliant VPC Endpoint you have to use a different URL for git commands. For example, in us-east-2 this is my fips clone command:

 aws_repo='https://git-codecommit-fips.'$region'.amazonaws.com/'$version'/repos/'$repo'.git'

This is the non-fips command:

 aws_repo='https://git-codecommit.'$region'.amazonaws.com/'$version'/repos/'$repo'.git'

Here are the endpoints in us-east-2 and us-east-1, for example:

My network is tightly locked down to Internet traffic even though I have an Internet Gateway in my route table.

When I tried to leverage the FIPS compliant endpoint, my command just started to hang eternally. So I check the IP address that was returned for the FIPS CodeCommit endpoint. It happens to return a public IP address.

Hmm. Seems odd. I have a private DNS endpoint setup:

I wonder what I get if I set up a non-FIPS VPC endpoint.

Waiting for that to kick in I get no response.

Once it finally kicks in, I get a private IP address with the standard git option, as expected.

Well, this seems odd to me. And honestly, I’m more concerned about the private network versus the encryption standard so I’ll switch back to the standard git option. Hopefully this is by design and not a bug but I’m tagging it as one because I think it really should return a private IP address when you’re using a VPC endpoint. Otherwise I have to open up my network to public IPs in places where I don’t want to do that.

Additionally, how does this work if you don’t have an Internet Gateway in your route table? I presume this is some magic AWS routing where it stays off the Internet even though it has a public IP? But I still have to change my NACLs and I don’t want to have to do that.

I’m not sure if I have something misconfigured but it doesn’t seem like it — and if I do — I wish there was some kind of warning on the VPC Endpoint dashboard to explain that my traffic is not actually all going to be private as a result. I have needed this in other cases besides this one where misconfigured DNS led to traffic taking the Internet instead of private route. Seems like AWS could warn customers about that on the VPC Endpoints dashboard. #awswishlist

Note that later I discovered many issues that cause public IPs to be returned by AWS services even though private DNS is configured. If you are attempting cross-account access from one region to a different region, you need to set up private DNS resolution in the region from whence you are coming to get a private IP address (apparently — I read it in a blog post but didn’t try it due to the likely high expense. You night also be able to do something with a Transit Gateway.)

I also discovered an issue when my resource was not deployed in the same AZ as the subnet with the network interface. The EC2 instance or other compute resource likely can’t reach the VPC endpoint in the other private subnet and will try to reach the Internet instead if there is a public gateway.

In short, be very careful if you are using a VPC and subnet with both a public and private route. Your traffic might not be taking the path you think it is and you might not notice if you don’t have very stringent NACL rules.

Note that I had an interesting experience with a KMS FIPS VPC Endpoint here. You have to specify a different endpoint in the AWS CLI command or your traffic doesn’t go to the endpoint. If you have a wide open network, you might think your traffic is FIPS compliant and it’s not.

I discovered that because I block some unwanted public network traffic.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2023

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab