I Created a Fake Entry In AWS VPC Flow Logs

ACM.462 How to enter log events from the AWS Console and (hopefully) how to prevent it

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics | Code.

🔒 Related Stories: Network Security | Data Breaches | AWS Security

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the last post I wrote about public and private ASNs.

As you can probably tell I’m working on something network related. While I was doing this, I noticed something that disturbs me, personally. But I’ll let you decide. You can insert events into CloudWatch logs created by AWS services. I’ll explain how to do it and how to prevent it (untested as I don’t have time at the moment.) Also I couldn’t find the related entry for this action in CloudTrail.

How I created a fake entry in VPC Flow Logs

Open up a log stream for your VPC Flow Logs in the CloudWatch console.

Click Actions > Create log event.

Enter a message:

I was able to insert a test event into a VPC Flow Log.

My entry is obvious. But I could have easily replicated a valid VPC Flow Logs entry.

I copied one of my valid VPC Flow Logs entries. I added an exclamation point onto the end of REJECT so I can find my entry. Was it added?

Yes. Yes it was.

Anyone working in security might be freaking out a little bit right now.

But wait, there’s more!

You might think, oh we’ll just go look up who created those entries in CloudTrail and correlate. Here’s the next problem. I can’t find the action I took in CloudTrail.

I do not understand the purpose of this functionality.

How are you going to trust your logs now if anyone with permission to do so can insert fake logs into your log streams?

One of the reasons I initially liked and advocated for AWS at a large financial institution where I worked was due to the separation of duties between what was in the service logs and the people who used the services that create those logs. I had concerns about people with too much access potentially manipulating logs related to financial transactions. This separation does not exist unless you further configure your cloud environment to disallow this action.

Of course, you don’t have to only worry about malicious admins. You also could have malware completely changing your logs unbeknownst to you.

Has this been possible this whole time via automation? I never really checked. I presumed it wasn’t but perhaps you have been able to do this for a while just not through the AWS console. I generally try to write policies to restrict write access to logs. But if the logs is not written by anything the customer controls initially (an AWS service) why can a customer write to that log at all?

Now I understand people can insert things into logs on systems you host that are creating logs for the applications you write, but I personally do not see any reason that anyone should be able to insert something into a VPC Flow Log or any log produced by an AWS service where the customer does not control the logging by the service.

In the case of a security incident — can AWS say that the customer may have changed the logs and there’s no way to prove otherwise? Hopefully AWS has other logs internally that would concretely pinpoint what happened.

I would recommend a Service Control Policy to limit this action across your organization until you are able to figure out if anyone needs it and then create a more fine-grained policy only for the logs where people are allowed to do this.

I was going to use the CloudTrail entry to create the SCP but as noted I can’t find the entry in CloudTrail.

So where can I find the IAM action that I need to add to my policy?

I took a look at CloudWatch Logs actions.

This is the closest action I can find:

I can’t verify that this is the correct action without actually testing it. Deny this action and see if it prevents the ability for the user to create the log entry as I did above. If that action doesn’t deny adding a log entry, you can add and remove actions one by one until you figure out which action does prevent rogue entries in CloudWatch logs.

I don’t have time to do that today but hopefully will get to it in the future and can provide more specific guidance — or maybe AWS will via a support request.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2024

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab