Measuring Security

How do you create metrics for something with so many variables?

One of my posts on Cybersecurity and Governance.

Free Content on Jobs in Cybersecurity | Sign up for the Email List

I sat in a meeting once where an executive at a financial institution was asking a member of the security team what requirements our cloud engineering team needed to meet to install a proxy in the cloud. (I’m talking about the type of proxy enterprises use to allow or disallow certain URLs from entering a network, not the kind attackers use to bore through your firewalls.)

Although I was a security person, a technical person, a cloud and a software engineer, I could understand the frustration on the executive’s part when trying to get a straight answer on requirements. The security professional was hemming and hawing with a lot of, “well that depends,” and “if thens” and by the end of the call we had no clear guidance as to how we could proceed.

Simple strategies and caveats

That conversation and others like it, in part, are why I wrote Cybersecurity for Executives in the Age of Cloud. I wanted to provide concrete guidance executives could understand as to what are the top priorities you should be concerned about when creating a cybersecurity strategy for an organization. I wanted to tell them why it matters and the questions to ask to ensure people within the organization have implemented basic cyber-hygiene. I also wanted executives to be able to cut through conversations like these to what matters — and be able to obtain actionable guidance on system implementation.

However, in addition to basics, the book goes into some of the nuances. These nuances include some of the “what-ifs” and “it-depends” that cybersecurity professionals might bring into a discussion and help an executive understand and make sense of it to some degree. The book will help drill down to what matters most by helping executives understand there are some caveats.

For example, you should encrypt data, but understand when encryption doesn’t help. You should implement MFA, but ensure that your implementation and configuration are robust. I write about different types of MFA in my book. In a recent publication, NIST spells out MFA requirements, over and above MFA alone.

Note: In the above document, password rotation is disallowed, but I write about a scenario in my book as to why you may wish to continue to rotate passwords. I personally continue to rotate passwords regularly because when I see old passwords in a phishing attack I know I don’t have to worry about it.

Frameworks Upon Frameworks

So many frameworks exist already. Why am I writing a book on cybersecurity fundamentals — doesn’t all this information already exist? Yes, many details exist about cybersecurity, though in some cases not enough. I explain why we need more specifics in data breach reports in order for them to be useful.

We have many cybersecurity frameworks of many varieties and flavors. All of them are useful. I leverage much of the information from NIST, CSA, OWASP, MITRE, ISACA, ENISA, ACSC, and CIS when making recommendations to customers. In addition to all the general guidance customers need to be aware of vendor-specific guidance and implementation details for any technology they use.

What is an executive or even a cybersecurity professional supposed to do with this information? The number of lists, guidance, recommendations, and attacks can be overwhelming, as I explain in this post where I was digging into the latest version of MITRE ATT&CK:

As I explained in my first book we can make sense of mountains of data, such as a pile of system errors, by abstracting out the things that are similar that we can deal with the same way. That way, we have a shorter, categorized list to deal with. When dealing with this shorter list, we still need to be cognizant of variations and details. Although the abstracted list may be useful, it too comes with caveats that I covered in the book and will dive into deeper when providing strategies for cloud governance and cybersecurity governance in general.

A more focused approach to measuring cybersecurity

I had hoped that my last book would provide a starting point and guidance for how to measure cybersecurity, and ways to automate that. The things to measure in that book focus on what causes data breaches and what allows them to occur in the first place. It also focuses on not being able to quickly spot them and mitigate the damage they may cause.

In my post on MITRE ATT&CK I reiterate a concept in the book — that some items on the cybersecurity lists we use today are very important, but they are subtasks of the actual item that stops a data breach. Sometimes people get lost in the weeds trying to find all their systems and categorize all their data when they have glaring misconfigurations and vulnerabilities in their environment. Which really should be the priority? Everything needs to focus on the fastest path to reducing the chance of a cyberattack.

Turning ideas into implementation

As I speak to more and more clients on consulting calls through IANS Research and perform security assessments and penetration tests through 2nd Sight Lab I am discovering that additional information might be helpful. Although I presented 20 questions you should ask your cybersecurity team I didn’t go into detail and spell out a plan to get it done. It’s hard to provide one single set of steps with so many variations in the way organizations operate and different technologies work. However, we can abstract out some general principles and guidance.

That’s what a new book I’m working on aims to do. I’ll cover the material in my last book but provide the steps to go about using it within your organization. I’ll also attempt to drill down into the details and caveats presented in the book and how to deal with them. I’m constantly thinking about ways to reduce risk, simplify security, and making it easier to manage. I won’t have all the answers (no one does). But I can provide some actionable steps any organization can use to reduce risk.

Larger organizations that are more secure may end up skipping over the basic starting points. But you should find out if your organization can measure the things I’m presenting here or not. If not, why?

Smaller organizations may not be doing some of the things in my last book at all. For this reason, I’ll start with the basics, and move on to more complex topics from there. If you don’t understand why should be measuring the things in my list, that’s covered in my last book, so start there.

Cybersecurity may feel overwhelming at times. At the end of my last book, I talked about another book I read that explains how small changes can add up to success: Atomic Habits. The key is to get started. You may not be able to fix everything at once, but you can start fixing something. Small improvements reduce risk, though eventually, you will want to prioritize your efforts on the things that matter most. You’ll also want to understand where gaps may exist in your metrics.

For now, I’ll start with one simple list of metrics and move towards the more complex. I’ll go into more detail to explain how you can measure these things in cloud environments and where it’s difficult in traditional on-premises data centers and offices. I’ll show how you can take a hybrid approach in some cases, to get the answers to the questions.

Different methods to measure security

In Cybersecurity for Executives in the Age of Cloud, I wrote about using percentages to measure risk. While talking to a recent client, I was sure the metrics the client was bringing up came from my book, because they sounded so familiar. He was using percentages to measure certain aspects of cybersecurity. As it turns out, they came from another, which I have yet to read: How to Measure Anything in Cybersecurity Risk. I’ll have to check that out before I finish my book and may reference it.

I also recently ran across an episode of a podcast by Rafal Los. The first question aligned nicely with something I wrote about. Metrics with caveats. I learned in a cybersecurity 101 class that you should measure all the attacks your software equipment blocks and show that to your executives to prove you are doing your job.

Those metrics are interesting and useful to some degree, but after working on a quarterly security report for a vendor, I have some insight into those types of metrics. They tell you what got blocked, not what security problems exist in your environment that could lead to an attack, what was missed, or attacks that exist currently in your environment. Those metrics don’t help you stop a future attack.

I’m listening to another book at the moment as well: Cybersecurity Program Development: The Essential Planning Guide. I like the concepts in the book and they are all highly relevant. But once again, I considered if someone trying to formulate all the information into a plan might find the details overwhelming.

This book came out in 2018 and some of the information about cloud might need an update. I’m less of a fan of meetings than this author, and more of a fan of automation to solve security problems (when appropriate). But overall this book has an amazing amount of cybersecurity information. One of my favorite parts so far is the information about compliance and cybersecurity laws.

I haven’t gotten to the end of that book yet so maybe the end includes more specific metrics and steps to take. But like my book, that one is a wealth of information that needs to be considered. I dove into some of the technical aspects of attacks and cybersecurity a bit more and focus less on administration, more on metrics. In other words, I focused more on the end result and less on how to get there as that may vary. But with all that information where should a company start?

What if you want a roll-up executive level of cybersecurity reporting to take to the board? Let’s say you have a small to medium-size business and you are just want to know what you can do to start improving cybersecurity right away?What if you want to ensure your company has implemented the key components of a cybersecurity program that will prevent the most common causes of data breaches?

Here’s a starting point to help you measure your security. I dove into these questions in Cybersecurity for Executives in the Age of Cloud as they are not all as simple as they appear on this list. But if you aren’t doing these things at all, just start. Then iterate on improving the details.

I rearranged the order of the questions a bit to ensure policies cover all the prior questions. I’m also working on making the information more succinct. Organizations and executives, such as the one I worked with, may want a concrete list of actionable steps to ensure they have appropriate governance and risk management in place.

Here I’ve summarized all the questions in a chart. You can type this up into a spreadsheet to get started. Can you measure these things — yes or no — in your organization?

Other topics that may appear in future blog posts or the book include quantifying security, drilling into the details, gathering the information, handling caveats, technology-specific metrics, architecting systems that provide metrics and prevent non-compliant systems, and capturing useful data free from bias and manipulation.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2022

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab