Do You Need AWS Traffic Mirroring?

ACM.223 Taking a look at the pros, cons, and costs

Part of my series on Automating Cybersecurity Metrics and Network Security. The Code.

Free Content on Jobs in Cybersecurity | Sign up for the Email List

In the last post I wrote about moving accounts between AWS Organizations.

Now, although I love networking and network security, I am going to explain why I’m not going to rush out and implement Traffic Mirroring in this series, just to get this topic covered.

Back when everyone said no organizations would move to the cloud because it isn’t secure one of the big issues was lack of packet capture capabilities. You can’t inspect the packets so how can you possibly be secure in the cloud?

I wrote a paper around that time to show that you actually could capture packets on AWS you just had to do it in a different way.

But now there are other options….read on…

AWS has since provided a service called Traffic Mirroring.

Let’s take a quick look at the benefits and challenges of using AWS Traffic Mirroring, as well as why I’m not rushing to get out and implement this immediately for my AWS Organization.

Traffic Mirroring with Open Source Tools

If you’ve ever taken a SANS class in network intrusion detection, you’ve probably used some open source network traffic and packet analysis tools.

I’ve taken a few. :)

These network traffic analysis tools like NIDS, NIPS, HIDS, and HIPS help you inspect traffic on your network and alert or block various types of attacks found therein.

You can deploy tools like Zeek and Suricata to work with Traffic Mirroring.

Deploying traffic mirroring is not a simple list of steps. You will need to think about how you want to deploy your architecture and what you want to mirror. For our initial use case, let’s say I want to deploy a VM in my SandBox account and mirror the traffic of the entire VPC. I’d need to figure out how to ensure all the traffic flows get to AWS Mirroring for inspection. Do I want to capture the flows from that single host? Or the entire VPC?

Of course we also want to know the cost of traffic mirroring. Here’s the traffic mirroring pricing from the VPC page under network analysis:

This sounds like it could add up fast.

Notice that the price is per ENI (Network Interface). If we can structure our network to limit the ENIs that may help. Note, however, that ENIs do not only exist for EC2 instances. If you want to capture ALL your traffic you’ll want to use a VPC and you have to make sure every ENI is associated with a VPC. See this post on Lambda networking, for example:

However, if we are going to direct all our traffic to traffic mirroring, which is probably a good idea if we plan to use it, we’re likely going to employ VPC Peering to connect and redirect traffic to the monitoring ENI. We’ll want to understand the cost of VPC Peering as well.

AWS updated one aspect of the cost of VPC Peering a while back which is nice:

Starting May 1st 2021, all data transfer over a VPC Peering connection that stays within an Availability Zone (AZ) is now free.

If we can keep our peering within a single AZ that will help. Otherwise here’s the cost of VPC peering:

So that’s another consideration for your network architecture if you choose to implement packet mirroring.

Otherwise your peering costs will be in addition to data transfer costs your instances are incurring in addition to Traffic Mirroring.

…more regions listed in the above documentation.

The hosts you want to mirror must be Nitro instances according to this post using Traffic Monitoring with Security Onion:

But that appears to be changing:

You will still need to understand exactly which instance types are supported and are not supported if you intend to use Traffic Mirroring for network traffic intrusion detection analysis and incident response.

Why AWS Traffic Mirroring, with some caveats

A lot of security folks we will want full packet capture due to the visibility it provides for attacks at the network layer.

There are certain attacks you cannot easily see without digging into the details of a packet.

Trust me, I’ve taken a few tests on that topic and wrote some information about dissecting packets in this series on cybersecurity math (hex to binary to decimal, etc.)

DNS exfiltration, ICMP and DNS tunnels, packet fragmentation and IP options attacks. A whole new slew of IPv6 protocol attacks. etc. etc. etc.

But what about when the traffic is encrypted in such a way that your traffic mirroring tool can’t see into the packets? I’ve mentioned this before, but one of the issues in the Capital One breach was the implementation of mutual TLS — which is great for authenticating hosts on both sides of a connection, but can lead to visibility issues for packet inspection devices.

What about new methods of encryption in IPv6 encryption, HTTP2, TLS, QUIC, and DNS over HTTPS? How will these protocol changes affect your traffic visibility? Are you going to test all that prior to use of this technology to make sure you have the visibility you think you do?

Are you capturing ALL The packets? Can some resource in your cloud environment bypass your desired network path? What about traffic to and from the cloud provider itself? How about inter-VPC and container to container traffic on a host?

These are things I discuss with clients at IANS Research who want to implement cloud security. Where should we start? What is most important?

Some considerations when deciding to use Traffic Mirroring:

Here are some of the things you’ll want to consider if you decide to move forward with AWS Traffic Mirroring.

• The cost.
• Support by various AWS compute resources.
• If applications have an encrypted channel between the application and the client and you can’t man-in-the-middle the traffic you won’t have access to the payload.
• Consider the use of mutual TLS and how it affects visibility into your traffic.
• DNS over HTTPS (DoH) limits the ability to inspect DNS traffic for malicious domain names.
• Some applications and protocols may encrypt the payload inside the packet so only the application itself can decrypt the data irrespective of TLS or not as a means of transferring the data between hosts.
• VPN traffic may be encrypted in a way that it is not visible to your packet capture.
• You won’t have visibility to traffic between containers that doesn’t traverse the ENI on which you are mirroring network traffic. You’ll need a different solution for that.
• Certain AWS traffic may not be visible because the way that traffic works on AWS does not traverse the ENI on your compute resource.
• You will only see the traffic traversing whatever ENI has mirrored traffic. For example, if you are mirroring outbound traffic, you won’t see traffic sent between private subnets.
• Anything else that does not traverse the ENI where you capture the traffic may not be visible in your traffic mirroring logs.
• You might be more easily able to find a lot of threats through simpler means so you’ll want to start with things like GuardDuty, CloudTrail logs, and VPC FlowLogs before implementing Traffic Mirroring. Do those things first.

NOTE: I am not saying you should not use AWS Traffic Mirroring!

There are some types of attacks that will be difficult to capture without full packet capture. Or perhaps you want to use Traffic Mirroring during certain security incidents with certain ENIs. Depending on a host to capture this information won’t work when a host is compromised.

I am just explaining that you need to understand when it does and does not help you and considerations to keep in mind when architecting your solution. And consider where you will get the most return on your security investment when you are starting down the path to secure your cloud. I wouldn’t start with AWS Traffic Mirroring, and I’m not starting with that as I redeploy an AWS Organization in this series.

I’m starting with governance. Prevent the egregious actions in the first place rather than inspecting minute details in network packets.

Make sure you have VPC Flow Logs enabled on all your VPCs and a way to query that traffic. You can get a lot of information from network flows alone.

Alert on egregious IAM actions.

Implement a way to analyze your cloud for misconfigurations — block them — and reduce the time it gets to find them and fix them.

Turn on simpler and effective security services like GuardDuty and other security monitoring tools provided by AWS or that you create yourself to find the most common attacks in cloud environments. Make sure are monitoring and responding to them effectively. If you aren’t monitoring, alerting on, and responding to GuardDuty findings, how are you going to respond to additional alerts from AWS Traffic Mirroring?

One thing you might want to do is try out AWS Traffic Mirroring just to understand how it works for future implementation, or how it might be useful during a security incident. But the time and cost to rely on it as a solution for monitoring networks — maybe should wait until you have a few other security controls in place if you are short on time and resources and have to prioritize where you start.

More on Network Security here:

Or check out the whole series where series when I’m setting up security for an AWS organization with the end goal of monitoring security metrics.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2023

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab