Testing Your Cybersecurity
Penetration tests, assessments, audits, red teams, and bug bounties
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Related Stories: Cybersecurity for Executives | Penetration Testing
💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Leveraging pentesting, assessments, audits, and bug bounties is my next of 20 questions to ask your security team in my series of posts on Cybersecurity for Executives. Perfect timing as I’m speaking on the topic of cloud pentesting at AWS re:Invent in session DVC01 in just over a week. The next few blog posts talk about this topic.
Validating security controls are in place — and working
Organizations should periodically validate that their efforts to implement cybersecurity are working. Although your teams may be working diligently to implement secure solutions, gaps may exist due to lack of training, knowledge about new cybersecurity threats, or because someone made a mistake. The other risk is that an insider may be intentionally misconfiguring systems. Having an internal or outside team or third party test your systems can help you find gaps and misconfigurations you might not otherwise know they exist. Additionally, sometimes, just having a person with a different perspective analyze the systems helps find things that your team may otherwise miss.
Companies can test systems for cybersecurity weaknesses using different levels of validation and testing. Penetration tests, assessments, and audits allow internal teams or third parties to perform various activities to validate security. The evaluators provide a report back to the company on the findings — and hopefully also the mitigations or things you can do to fix the problems they found. You have various options for testing the effectiveness of your cybersecurity controls.
Cybersecurity audits
A cybersecurity audit tests a set of controls to see if and how the company has implemented them. The controls may be defined externally by a particular standard, or the auditors may extrapolate a set of controls for the audit from those standards or internal policies. ISACA (Information Systems Audit and Control Association) defines a cybersecurity audit as follows:
THE OBJECTIVE OF A CYBER SECURITY AUDIT IS TO PROVIDE MANAGEMENT WITH AN ASSESSMENT OF AN ORGANIZATION’S CYBER SECURITY POLICIES AND PROCEDURES AND THEIR OPERATING EFFECTIVENESS. ADDITIONALLY, CYBER SECURITY AUDITS IDENTIFY INTERNAL CONTROL AND REGULATORY DEFICIENCIES THAT COULD PUT THE ORGANIZATION AT RISK
https://m.isaca.org/About-ISACA/advocacy/Documents/CyberSecurityAudit_mis_Eng_1017.pdf
For example, do you encrypt data at rest? Do you protect and rotate the encryption key? An audit may or may not get into the details as to whether or not a control is effective. They often tell you if you have the control — yes or no — based on interviews and reviewing system reports. Some may go farther into testing systems, but sometimes auditors are not given direct access to the systems themselves. The other point to keep in mind is that auditors do not have time to check everything, so they typically select sample systems to evaluate. The process by which auditors select systems, processes, or teams to evaluate hopefully helps find some of the riskiest systems in your organization based on the type of data stored and known potential mismanagement.
Sometimes auditors hire a Subject Matter Expert (SME) to come in and provide technical expertise to help formulate the questions and controls the auditors inspect and test whether they are effective. For example, as a SME on an audit, I helped a company evaluate some of their SAAS (software as a service) systems. The company wanted to know how well they were protecting their data that was put into third-party systems. Since they were unfamiliar with cloud technologies, they wanted someone who could explain the technology and risks and find potentially weaknesses they might have otherwise missed.
Auditors follow industry-standard best practices for their profession, though each company may implement its audits in different ways. There are different types of audits, as well. Sarbanes-Oxley, which includes a cybersecurity component, is an example of a type of audit mandated by regulations in the United States. Other countries have their own regulations companies must follow related to cybersecurity. Companies may also perform operational audits within their company as an internal audit to see if people are implementing cybersecurity best practices.
Cybersecurity risk assessments
Cybersecurity risk assessments are similar to audits but may go a step farther to try to determine the effectiveness of security controls and the impact of the risk. Instead of just asking if a company implements encryption, an assessment may look at whether the implementation and overall architecture of systems prevent breaches. As I wrote about in another blog post, The Encryption Fallacy, just having encryption doesn’t prevent someone from reading your data. Organizations need to understand how and when the data may reside in an unencrypted state, such as while a system is processing it in memory, and who might be able to access it at that point. Additionally, if keys are not protected or decryption permissions are too broad, the encryption does no good.
NIST (National Institute of Standards and Technology) characterizes a risk assessment as follows:
The purpose of risk assessments is to inform decision makers and support risk responses by identifying: (i) relevant threats to organizations or threats directed through organizations against other organizations; (ii) vulnerabilities both internal and external to organizations;(iii) impact (i.e., harm) to organizations that may occur given the potential for threats exploiting vulnerabilities; and (iv) likelihood that harm will occur.
https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-30r1.pdf
Some cybersecurity assessment frameworks offer mechanisms for testing controls that are in place. Testing the control is actually in place, and working is better than simply checking a box or asking open-ended questions where people say a control exists. For example, the Center for Internet Security (CIS) offers benchmarks for securing many different types of technologies. Each recommended best practice generally includes tests to determine if the control is actually in place or not. Although complete analysis of controls, risks, and security gaps still requires humans, evaluation using measurable and, better yet, automated tests help improve the effectiveness of audits and assessments.
A specific compliance requirement mandated by a particular organization or government organization may require an assessment. An assessor may be certified to perform a specific type of assessment related to compliance with a particular standard such as the Payment Card Industry (PCI) compliance standard mandated and administered by the Payment Card Industry Security Standards Council. Organizations that accept credit cards as a form of payment must maintain compliance to continue to do so.
Other types of assessments may not be compliance-related but have some other objective, such as an evaluation of the use of cloud systems to look for security gaps or evaluation of the architecture of an on-premises system to look for security flaws. Some companies specialize in the analysis of cryptography implementations or industry-specific requirements such as health care systems or critical infrastructure like power plants. Assessments may involve interviewing people, reviewing documentation, architecture and code reviews, evaluating system reporting, or running scans on systems to find vulnerabilities, among other things. The goal, as always, should be to try to find security gaps in your environment.
Penetration tests or pentesting
Penetration tests, sometimes called pentests or pen tests, involve intentional attacks on systems to try to break into systems or show that an attacker could access sensitive data. This type of testing originated in the U.S. government. Security researchers started researching how attackers might exploit systems. The government started hiring what they called Tiger Teams to break into systems to discover flaws. If the teams could break into the systems, then the government would know what to fix by patching systems or taking other measures to prevent the threat from occurring.
The primary difference between the penetration test and audits or assessments is the fact that the pentester is attempting to exploit system vulnerabilities instead of just showing that they exist. Instead of looking at systems from the outside only, a pentest tries to break into systems, and show an organization how an attacker might compromise a system, and potentially pivot and get to other systems or exfiltrate data. A pentester may expose other types of vulnerabilities that allow an attacker to obtain credentials of administrative users to get into systems or allow one customer to get into another customer’s data.
An organization can get many different types of penetration tests at different price points. Penetration testers operate in different ways to provide value to customers. Some companies may only perform a scan of systems and report vulnerabilities. These scans are generally the least expensive types of “pentest,” though, in reality, it’s an assessment, not a pentest. I have had companies approach me for a pentest after they tried to obtain SOC 2 (Service Organization Control) compliance with these types of pentests have been instructed to improve the quality of their pentests before they could meet the SOC2 requirements. Although these are not full-fledged pentests, they are a starting point for companies new to security. You can even learn to perform basic vulnerability scanning yourself. Performing these scans in advance of a penetration test and fixing those vulnerabilities to help avoid lengthy reports and basic security flaws reported as high-risk findings.
Some penetration companies have a team of people that perform numerous activities in a short amount of time using a standardized approach to test each client. Other companies, like mine, spread out a penetration test over a few weeks to have time to research and perform more customized tests aligned with the systems in scope for the test. Some companies have programmers on staff who know how to reverse engineer systems. Some focus on and specialize in a subset of the various types of penetration tests.
Penetration tests may include testing websites, cloud systems, on-premises systems, networking, IOT devices, and security appliances, among other things. I met someone who pentests airplanes — very cool, but much responsibility as well! In these situations, people’s lives are at stake, so testing must be comprehensive. Testing live hospital systems could cause something to break that affects patient health. Testing a nuclear facility might cause an unwanted explosion. Pentesters must be extremely careful when working with live, production systems, or testing things that have no alternatives if something is damaged, such as expensive hospital equipment. I prefer to test non-production systems and validate results in a production environment, whenever possible.
Red Teams
Some people also use a term called Red Teaming interchangeably with penetration testing or with a distinct definition. I am not one to fight over words and definitions, so I leave it to someone else articulate or argue about the difference between red teams and penetration testers. If you want to get into the differences, other articles explain that in more depth. For this series of posts, when you see the phrase penetration test, it also applies to red teaming.
One definition from references in the Wikipedia article on the topic (which lacks some citations) comes from a Wayback machine copy of an army web site:
The Army defines Red Teaming as a “structured, iterative process executed by trained, educated and practiced team members that provides commanders an independent capability to continuously challenge plans, operations, concepts, organizations and capabilities in the context of the operational environment and from our partners’ and adversaries’ perspectives.”
Red teams are similar penetration testers in that they exploit systems, though they may have different or additional services, functions, or objectives. I am only speculating here, but likely they cost more due to this additional work, hence the attempt for some companies to re-brand themselves as red teams. They are performing more work than someone who scans your systems and gives you a canned report, for example. The service is more valuable than a scan, hence the higher cost.
What is more important than whether you are getting something called a red team engagement or a penetration test is that you get the services you require to determine the security of systems to the level you desire. Whatever services you need, regardless of the name, should be defined in the scope, permissions, and process, as will be explained in my next post on this topic. This definition of services and whether you achieve your target objectives is more important than the name the company applied to the service.
Bug Bounties
Some companies are leveraging bug bounties in place of, or in addition to penetration tests. A bug bounty is a way for a company to allow people to try to break into their systems and then pay them for individual findings. Sometimes companies run bug bounties internally. Sometimes companies get help from other companies to run bug bounties such as Bugcrowd and HackerOne.
Benefits of a bug bounty program include the ability to have many different penetration testers and security researchers attack systems. The cost of individual findings could be lower than a full-fledged pentest. Penetration tests performed by the same people year after year may be missing new types of attacks. Each penetration tester may look at the system in a new way or try to find different things based on his or her personal experience and expertise. Running a bug bounty opens up your systems to many different individuals with different backgrounds and skills.
One downside of bug bounties includes the fact that you are opening up systems to unknown individuals who may have nefarious motives. They may not tell you about the finding but save it for another purpose, such as hacking competitions and selling it to someone who pays more. A recent hacking competition in Chengdu, China, called the Tianfu Cup exposed many zero-day software flaws in software like popular web browsers, mobile phone operating systems, and the underlying software for some cloud platforms. People I follow on Twitter speculated that researchers save up vulnerabilities they find for these competitions. They also do not submit everything they brought to a competition if they can’t get the top prize.
Those with lower ethical standards may also sell them on forums designed for such purposes — and sometimes the buyer is unknown. If you are going to offer a bug bounty, offer enough money to make it worth people’s time — and as you can see from the prizes in the hacking competition, it could be a lot. Some zero-day bugs have been known to go for hundreds of thousands, if not millions of dollars. Many security researchers complain that some large organizations do not offer bug bounties or do not pay enough. Hopefully, they are still reporting these vulnerabilities through the proper channels and to the appropriate people.
Another issue with bug bounties is that often, the people who participate in them don’t make enough to earn a living if they do it full time. Often you hear about the people who make crazy amounts of money on bug bounties. They are few and far between. I do know someone who received $18,000 for a bug he reported to Google. However, it appears that bug bounty income was not sustainable because he eventually returned to his day job. The higher-skilled individuals want a guaranteed payout. However, some security researchers use bug bounties to test out new exploits for penetration tests, perform testing in their spare time, or perform automated mechanisms to find hard to find exploits with high payouts.
One thing people do who participate in bug bounties is to scan systems to find low-hanging fruit quickly. Then they drop out. The problem with investing a significant amount of time into these bug bounties is that at the same time, another person could be reporting the same bug. Whoever gets there first gets the payout in most cases. That means a person could spend a great deal of time working on a bug and in the end, get paid nothing. Some companies running bug bounties have reported they start with a lower payout to get the low hanging problems and then increase the payout over time. George Gerchow, CISO of Sumo Logic, told me they use a bug bounty and are very happy with the results using this approach.
Whether or not you offer a bug bounty, you should have an effective way for people to report vulnerabilities to your company. Even if you do not have a bug bounty, security researchers may contact your company to report a vulnerability and request money in exchange for their findings. You need to be ready for this so you can handle the report effectively. Be aware that once you pay someone for a finding, they might continue to search for findings on your system and request payments for their services. If you do not pay or threaten to sue the researcher, this also could have adverse consequences.
Which type of security evaluations and testing do I need?
All of these forms of security testing are valid and cover different aspects of your security. If possible, do all of them. You may be required to do one or the other to meet a particular compliance requirement. Regardless of which you choose, ensure your testing is focused on the appropriate objective — finding the highest risk security flaws in your environment so you can fix them. The next blog post on effective security testing goes into more detail about defining the services you need and hiring the appropriate person to do the job.
Follow for updates.
Teri Radichel | © 2nd Sight Lab 2019
About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight LabNeed Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for PresentationFollow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab
