Security & Machine Learning — Part 2
Applying ML to autonomous cars and security problems
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Related Stories: AI & Machine Learning
💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Part one of this blog series on Security & Machine Learning introduced some machine learning terminology. Those terms help you get an idea of machine learning concepts that are applied to datasets to try to predict outcomes.
Many security products offer to solve all your security problems with machine learning, so I wanted to write about my thoughts after speed-reading (maybe a bit of scanning) the end of the book I mentioned: Machine Learning & Security: Protecting Systems with Data and Algorithms. As with any technology, apply it to the correct problems with the proper amount of resources to be effective. We’ll want to avoid choosing technology for technology’s sake.
As I mentioned in the last post, I started getting into all of this because we were hosting a series of meetups at Seattle AWS Architects & Engineers, where we will virtually race AWS DeepRacers. If you are reading this before Monday, October 5th, you still have time to submit a model and win prizes! As mentioned in the last post, you can find the link to our race and more information on the final machine learning meetup page.
I was pretty new to machine learning when we started. I knew the concepts. We’ve had various incredible machine learning experts at our AWS meetup. I played around with AWS QuickSight, which lets you create dashboards based on machine learning insights, and went to a Google meetup on the topic. I knew about Jupyter notebooks and the need for a substantial amount of data to get accurate predictions but not exactly why or how it all worked in practice. That’s where the meetup, creating the model, and reading the security book in this post comes in. I’ll get back to security in a minute, but it’s helpful to see the options for the car as an example.
The summary of steps to create a model to submit to our race:
- Log in to AWS and go to the DeepRacer service.
- Click Create a Model.
- Enter a name (and description if you want).
4. Choose a track. We’re racing on the re:invent 2018 Wide track:
5. Build or select an existing car.
- If you already built a car, it will appear on the list.
- To build a car, you can go to Your garage on the Deep Race home page.
- When you build a car, you choose the features of your physical vehicle, but if you don’t have one, you can choose what you want. The car I have does not have the LIDAR sensor or stereo camera, so if you plan to participate in a future in-person meetup where we test out this car, then you’ll want to choose those options. We hope to have an in-person race one of these days!
- Choose actions on the next page — these actions influence your model:
Note: If you read my last post, you may recognize some words here, such as continuum of actions (continuous), but we’re defining a discrete action space here. That means the model has a defined limit to the actions it may take. But what should I choose? I have no idea.
Based on my last post, the way to improve your machine learning algorithms — is to test them. That means you try different data sets, various methods, and in this case, other discrete actions the vehicle can take and test it to see how it performs. As mentioned in my last post, various issues exist, such as training to a specific dataset and overfitting and not having enough data to properly train a model a generalized problem set (multiple tracks).
In our case, the AWS DeepRacer uses a specific training method: Reinforcement Learning, so that’s one less decision I’ll need to make. I don’t believe the book I read covered this, but I could have missed it since I read it one day! Reinforcement learning is an alternative to supervised and unsupervised. The concept is that automated agents (our car, in this case) will be rewarded for specific actions and adjust to maximize the reward. If you want to read more about it, there’s a link with more information in the AWS DeepRace console.
OK great. So what’s a machine learning noob to do?? Well, I could fiddle with this to figure out which options work best and run many different tests. I can also try to shortcut the process by looking at the many AWS DeepRacer models and car features posted by others on GitHub. To be honest, if I were trying to win, I’d do a lot of research into what others have done and their results to try to improve my outcomes and spend less money. But for this race, I am in the AWS DeepRacer free tier. I don’t have a lot of time to invest by tomorrow, so I’ll scan some of those and submit something just to see how it works out.
6. Name your vehicle.
7. Choose a color and click Done.
Back to our model…
8. Choose Time Trial (to race with us) and choose your car. Click Next.
9. Create your reward function.
Here’s where the magic happens! You try to change various parameters or features to try to optimize your results. I’m not going to go into all of this here, but this is where you design a function that causes your car to react based on rewards for certain behaviors. As I posted on Twitter @teriradichel already, there’s a nice step-by-step walkthrough of setting up a vehicle and creating a model here that has a lot more detail.
Our first attempt at this included me choosing the wrong track and @kolbyallen choosing features that sent his car spinning in circles. Clearly, neither of those was beneficial for our desired results! In my case, I would be training data to work on the wrong track. In Kolby’s case, he set up conflicting rewards that completely confused the poor car! I’ve read some people start with the base options and then tweak from there. I’ll just make a few select guesses but not go too crazy this time.
A few other options exist, such as automatically submitting your race to a league, but I’m not going to choose to do that. I’ll train my model and submit it to our meetup league after.
Note the cost! There’s a free tier but then…
10. Click Create model. At first, you’ll see it is initializing.
11. Then you’ll get to watch your car learn…
Hopefully, you won’t watch your vehicle drive backward off the track too much as my car just did :-/ or run into a wall.
12. Wait.
At this point, we wait, or as Drake Loud put it in his machine learning presentation to the meetup, “watch the Turkey bake.” When I asked him how long we have to let it cook, he said there’s no right answer. Again this comes down to testing, guesstimating, and trial and error.
In this case, if I guess right on all these options, I’ll come up with a model optimized for this track to win the race. If I were trying to win on any track, I’d have to go further and test my model on all the different tracks to try to determine the best generalized approach that works on all of them. The last post on machine learning terminology defined the concept of overtraining your model.
13. Click, you can click Start evaluation to see how your model performs.
14. Choose our track and Time trial.
15. Accept the terms and submit to the general AWS league if you want.
16. Click start evaluation. Note the cost.
Well, my car didn’t even complete the track. I’ll need to make some adjustments or train longer. I trained for about an hour. I think Drake said he trained for two.
Enter the Seattle AWS Architects & Engineers Race
1. Click the link in the meetup to submit your model.
2. You’ll get to our race page. Click Enter race.
3. Choose and submit your model.
Machine learning and security — what have we learned?
I will not proclaim myself to be a machine learning expert by any means, but here are my thoughts on how machine learning applies to security problems and products.
What we’ve learned at this point is that there are a variety of methods to train a machine learning model. Various methods are more or less applicable to different types of problems. The security book I mentioned covers many security use cases throughout the remainder of the book and shows how different methods may work for different types of security problems. It also includes sample code if you want to try it out. Some of the use cases include spam detection, fuzzing, and threat intelligence.
As shown above, we can train our model to optimize for a specific track or work more generally across several different tracks. That applies to security products as well. How was the data trained? Is the training applicable to your specific use case and environment where you will use the product? Was it overtrained to the vendor’s test data or made so general that it will miss your particular high-risk attack vectors?
How many variations of methods and inputs did the vendor test? As I was creating my car model in our first meetup, I noticed that I could change a set number of factors. I couldn’t change the input categories or features, but I could change their values. During the meetup, I immediately started speculating that Amazon was working on an autonomous car, and all this test data would help them. It turns out, they bought such a company called Zoox.
As we pondered the potential use of data in these models, we started thinking there are probably many other factors to consider that would affect the outcome. I’ll leave all that terminology to the car guys. They had to explain to me what a chassis is when I was trying to put together the physical car live. Of course, I disconnected a cable I shouldn’t have in the process, and we had to figure out how to put it back together. I’ll stick to the cloud! They mentioned other things a self-driving car would need to consider in the real world, not included in the potential inputs to an AWS DeepRacer.
I was thinking, what if it’s driving up a mountain on a really curvy road or a straight highway through Montana? What if the car is driving downtown a city street and a person on a bicycle falls in front of it? What if it’s on the freeway and another car’s tire falls off? How about a car that needs to drive down a dead-end street that is too narrow to get out so has to back out? What if a road closure exists that is not on the map? What if it is raining hard, very foggy, or snowy? So many considerations!
All the factors not included in the potential inputs to this car relates to the inputs used to make predictions about security threats in your environment. What factors did a vendor use to make the predictions, and was something missing that will matter in a future new type of attack, or is there something specific to your environment for which the model should be optimizing?
Attackers can trick machine learning algorithms into doing things that it shouldn’t, just like any other software. What if an attacker understands the model and uses it against the product? It may overload the data to skew it in one direction so the attacker can sneak out in broad daylight through an acceptable path.
Is machine learning any good at all?
Definitely! Machine learning in a cloud environment is especially useful as the cloud provider has a vast amount of data to baseline activity and detect anomalies. It also has a massive data set to generalize data to make classification and clustering predictions. This large amount of data and appropriate algorithms may help make better decisions for complex problems.
As I mentioned in the last post, cloud providers are well suited to use machine learning. In addition A LOT of data, many of their customers use systems in somewhat similar ways. They offer virtualized resources and services that will work the same for every customer at the infrastructure layer. These commonalities are a benefit from a security perspective because that makes it easier to write automation that works across the platform.
My only caveat would be that machine learning alone is not a silver bullet that you drop into your environment and let run without any additional monitoring. Patterns and predictions are great, but as Nassim Nicholas Taleb warns in one of my favorite books, Fooled by Randomness, too much dependence on the past or a particular model may cause us to miss a game-changing anomaly that doesn’t fit anywhere in the model. Case in point, did anyone expect COVID-19? I wonder if any machine learning algorithms out there calculating business risk that took this type of event into account.
Though machine learning models are helpful, it’s still a good idea to do your own threat hunting and log analysis in conjunction with your vendor products. You also may need to adjust your monitoring for some factor specific to your environment that leads to less false positives, less false negatives, and doesn’t miss true positives! I explain those concepts in my cybersecurity book, and the machine learning book in this post covers those topics as well. Security products generally need human tuning and analysis for optimal results.
On top of all that, the fact that a generalized model must work across many different environments may mean that creating something more specific to your own environment might work better if you have the resources. If you create your own model, you can create your own inputs and tune it to your particular setting.
Additionally, you may not need machine learning at all for some problems. I wrote a web application firewall (WAF) after my first data breach. I didn’t use machine learning or train a model. I looked at the specific possible threats and wrote an algorithm to address them. I studied the logs each day and tweaked my function to optimize results. For example, I learned what an enterprise proxy is and why it causes may hits from the same IP address. Oops. Sorry.
My WAF wasn’t a simple matching filter or rate-limiting tool. It looked for variations on specific data points in web requests. For example, what is the length of the user agent? Does the CIDR block start with a known-bad network range? Tracking my own list of CIDR prefixes was more efficient than looking up every IP or calculating CIDR blocks, the range conformed to my needs.
It tried to identify some web attack patterns such as directory traversal (.. in the URL). I can’t remember everything it had in it, but it contained numerous checks for different types of malformed requests. I converted each one to XML (JSON wasn’t a thing back then) and analyzed any anomalies or errors to see what was going on. For some variables is used a partial whitelist. I also looked up every IP address to determine where the data was coming from if it didn’t look normal. Yes, really.
My approach was more of a customized algorithm based on a logic flow with many different factors based on what I witnessed in the logs. I didn’t know anything about machine learning at the time. I didn’t use any fancy machine learning terms, or known algorithms or methods. I just wrote code that identified bad things. Then I tried to optimize my algorithm’s decision making capabilities and performance over time by analyzing system performance and traffic logs.
Machine learning will definitely help identify threats hitting many different organizations in the same manner or leveraging typical behavior. It may also spot outliers. I’ve used a product called Cisco StealthWatch Cloud that identified anomalies very well in two different accounts where I tested it. I know they use baselines and anomalies. However, if you have the resources to do so, it’s still a good idea to look at and analyze your logs for threats, in addition to any machine learning products you purchase, to find threats specific to your systems and workflows.
Check out part three of this series where I address optimization of machine learning models.
Follow for updates.
Teri Radichel | © 2nd Sight Lab 2020
About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight LabNeed Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for PresentationFollow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab