Before You Open the Cloud Account
Multicloud.3 Assessing your vendors for security gaps and risks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
⚙️ Check out my series on Automating Cybersecurity Metrics. The Code.
🔒 Related Stories: Multi-Cloud Security | AWS Security | Azure Security | GCP & Google Security.
💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In the last post I wrote about visibility in an enterprise environment for multiple cloud accounts.
In this post, I’ll provide some insight into what you should do before you sign a contract with a cloud vendor or almost any vendor for that matter.
Minimum Security Requirements
Define the minimum standards that vendors must meet to become one of your vendors. Most people just presume AWS, Azure and GCP are used by everyone and they are really big so they must be secure.
However, over time things change. CEOs change. Key personnel change. Internal policies change. Staff changes. Documentation and services change.
Define a set of assessment questions that are applicable to different types of vendors. For example, as a training vendor who doesn’t even collect the real corporate addresses for students in my classes (unless they use them even when I tell them not to do so), how stringent does your security review need to be?
The review for a penetration test may have separate requirements. Perhaps you want to know how I secure the systems I use for penetration tests. Do I use MFA to access roles used to assess your cloud environment? How is my network configured? Are the hosts encrypted where your data is stored? Are documents encrypted? Is MFA required to access them? Hardware or virtual MFA?
It may be that you have different levels of consistency for different types of vendors. But you may have certain controls that, if a vendor cannot meet, or are a no-go. For example, one company I worked with required vendors to federate to their identity management platform to authenticate users. Another required control was support for MFA. Perhaps you require a particular type of MFA — like a hardware key. Additionally, many companies have minimum standards for encryption algorithms. Another control that may be required for sensitive data is secure access via a private network.
Periodic Security Assessments
Not only should you be performing an assessment prior to using any cloud provider, you should revisit that assessment on a regular basis. This initial and periodic review should be well defined, easy to read, documented, and available to anyone who is authorized to ask for it. That includes auditors.
Squarespace is about to take over Google Domains. Do you have any domains registered with Google Domains? Now is the time to perform a security assessment on SquareSpace.
If you come to the event in Atlanta in October where I’ll be speaking, people can share information and stories about how they assess vendors and challenges they face. (Sorry, no vendors allowed in these sessions, but vendors can sponsor and speak at the event. Contact IANS for more information.)
Defining Requirements
If you are assessing a vendor, one of the things you can review are the vendor assessments in the CSA Star Registry. I’ve written and talked about this before so I won’t go into it here but searching for and reviewing these documents will help you understand actions vendors take to secure their systems.
I reviewed one of these documents in my series on Okta and unfortunately it was a bit light. If I were performing an assessment on a vendor I would have other questions. Sometimes I modify the list of questions available from the CSA Questionnaire, which is now called the Cloud Controls Matrix.
Some of the questions I find lacking are generally around deployment systems though the controls have improved over time. Also, in the past I found some controls a bit wordy or not applicable to a particular client. You can take these questions and align them to your needs, but the questions you ask each vendor should be consistent across different tiers or types of vendors.
Some companies have a central team for handling risk assessments but that can become a bottleneck. You might want to develop a basic list of requirements that people throughout the organization can give to vendors to see if they even meet your minimum requirements before they get added to the risk assessment queue. Communication is also key, something I always end up discussing on calls with IANS research clients. I know what it feels like on both sides of the bottle neck.
Vulnerability Scans
Another thing you’ll want to do is run an external scan on the vendor’s systems, with permission of course. If your vendor will not let you do that then at a minimum you should ask for a penetration test. But what was the scope of the penetration test or assessment? Did it cover all your concerns? When I perform a penetration test I may or may not have access to test or assess the deployment systems. With more information about how the company deploys assets and guards the keys that can deploy the assets, I might understand or find avenues of attack that might not be obvious or possible to find in a short penetration test covering limited resources.
This aspect of scanning for exposed systems is critical. I recently spoke to someone who found customer PII online in a cloud environment for a vendor they reviewed. However, they were likely under NDA and could not tell another vendor who had hired that company they had a problem from what I could ascertain from the conversation. Unfortunately it seems that I may also be under NDA in my conversation with them. I have asked for a contract review and permission to disclose from the company I work with but until then, make sure you are properly assessing your vendors!
In this instance, the vendor was apparently not resolving the problem five weeks after they were informed about it. They had not reported the breach or finding to any authorities. When I posed the problem on my Twitter account, another IANS faculty member mentioned that his contracts always have an out to report any breaches of customer information as required by law. That would be a good clause to include in your contract. If you are performing an assessment and find sensitive data you need to be able to report it as required by law to the proper authorities or regulators.
Severity of vulnerabilities — do they matter?
Another thing you’ll want to understand is whether the vulnerabilities you find matter. I’m going to go into the topic of tools that report an “F” on websites when most of the findings are not applicable. They either reference functionality not used on the website, are outdated and deprecated in some cases, and in other cases would be very difficult to exploit.
Even if you could exploit some of the problems some scanners find— the only data you can steal is all the data that is already public on the website. The worst that could happen is a reflection to some other site and lost business for the site owner or simply reflecting data for some other unknown reason — as is probably happening on some of your cloud systems right now if you haven’t been paying attention.
So if you have someone telling you “Oh my goodness! Look at this website! It got an F!” without digging into the actual functionality on the website, then perhaps you should consider why you are taking that person’s advice. Ask them how those vulnerabilities could be leveraged in an actual breach or attack that would harm your company. On the other hand, an “F” when there is a login form, potential access to customer data and possible cross site scripting attacks is another matter.
When you find vulnerabilities on vendor systems, make sure you communicate them to the vendor and ask them about those problems. The vendor may have a good explanation for them, mitigation for known vulnerabilities, or perhaps it is even a honeypot or honey token to help them monitor their systems, data, and networks.
Compliance Regulations
Your company may need to be in compliance with various standards based on your industry, country, or your agreements with other vendors. Make sure the vendors you use are also in compliance with any such regulations.
Also be aware that on a large platform like AWS, Azure, or Google, there are many different services and features you can use. Each individual feature may or may not be compliant with the regulations you require. Additionally, the feature may only be compliant in certain regions. Each cloud provider has a way to check for compliance of their platform to meet your needs.
On AWS you can check out their compliance program here:
Find Azure’s compliance documentation here:
Google’s compliance documentation:
Licensing
Review the licensing terms and agreements. I’ve recently seen some complaints on my Twitter (now X) feed that people are getting hit with Microsoft licensing fees for being out of compliance. One person claimed that someone at Microsoft told them they were compliant. Microsoft’s licensing model is that you get a little bit for free for a short period of time but almost every security service you want bumps you up to the most expensive license.
If you haven’t gathered, I’m not really a fan of the Microsoft licensing model. I’m not even going to try to explain it, other than you’re probably going to need the most expensive option to get the best security features most companies want.
Contrast that with AWS and GCP which are often pay as you go models. This model is much nicer for startups and small businesses — that may be big business someday! Their cloud usage grows as they grow. Their cost (hopefully) scales with their revenue. And they can afford security at every size.
However, not everything is pay as you go. Which services do you plan to use? How much redundancy do you need for storage? The level of redundancy you need may bump you to a higher price point, for example. Using a Windows license on GCP was a minimum of $35 per month last time I checked.
Do you pay for the entire organization as a flat fee, or for each account, subscription, or project you create? I’ll explain more about what those are in an upcoming post. You’ll need to understand something about the architecture you plan to deploy and which security services you want to use to understand the costs.
I’ll cover costs and pricing in more detail in a separate post but make sure you understand what level of security you want and need and how much that might cost. Understand what kind of licensing requirements exist and the fees for non-compliance.
Service Level Agreements
One of the lovely jobs I had when I first got to the Capital One Cloud team was documenting the service level agreements for each and every service. Oh joy. I’m a technical person. I wanted to BUILD THINGS! But if you are going to build things, you need to understand when the service level agreement does not does not apply and how that will affect the organization in the case of an outage.
Do you know that to obtain certain EC2 service level agreement credits you must build systems that span two availability zones?
For Amazon EC2 with all running instances deployed concurrently across two or more AZs in the same region (or at least two regions if there is only one AZ in a given region), AWS will use commercially reasonable efforts to make Amazon EC2 available for each AWS region with a Monthly Uptime Percentage of at least 99.99%, in each case during any monthly billing cycle (the “Region-Level SLA”). In the event Amazon EC2 does not meet the Region-Level SLA, you will be eligible to receive a Service Credit as described below.
That agreement has changed since I first reviewed it and some other options exist now which are pretty nice. Do you understand how you would report a service outage and what information you would be required to provide?
For each service and region, you might want to look at the SLA and incorporate your findings into your architecture and plans as required by your disaster recovery and business continuity plan. If the amount paid by the SLA is far below the monetary losses should an outage occur, the business may want to think carefully about using that vendor. Alternatively, the savings compared to the on-going cost of maintaining in-house systems may offset any short term outage losses. Each business must analyze these risks and costs according to their specific business needs.
How likely is an outage?
Well, as far as I can tell some cloud providers seem to have a lot more outages than others. I wrote about the critical nature of cloud systems when it comes to outages here since so many businesses operating on them. In this post I randomly looked at a list of outages and causes for a certain cloud provider. Are the causes of the outages preventable? What is the vendor doing to reduce future outages?
That is something you will likely want to investigate.
System Status
Does the company have an online system for telling you if a service is having a problem? That’s a good place to start to investigate the health of the system and the outage history. Major cloud platforms will have a dashboard with outages and an explanation of each outage hopefully. You might also notice that outages are more frequent in a particular region or for a particular service.
Although the online dashboards are nice you can also check third-party reports to find past outages. Search in Google for “AWS Outage” or “Azure Outage” or “Google Outage” and then click the news link. Filter on different time periods to try to get a sense of the stability of the cloud platform.
You can also check the cloud-specific dashboards below. What if your cloud or service doesn’t offer a dashboard? You will definitely want to speak to the vendor about how you will be notified in the case of an outage — and a data breach!
Another way to gauge system outages is to use Down Detector.
Here’s what Microsoft 365 looks like at the moment.
AWS Health Dashboard
https://health.aws.amazon.com/health/status
Azure status
Unfortunately you need to login to get more information. I’m not sure how that would work out if the system that facilitates your login is down.
https://azure.status.microsoft/
Microsoft has a separate dashboard for O365 and as fate would have it, they are having some issues at the moment with Outlook at Teams. I honestly wasn’t even sure what I was going to write in this post until I started writing it and this is purely a coincidence.
https://portal.office.com/servicestatus
That said, Microsoft seems to be having a lot of issues lately. So many issues that a senator has called them out on it.
I have personally had many more problems with Microsoft than either Google or AWS, both of which I use more. However, it could be that Microsoft systems are more under attack since they may be more heavily used by government agencies and large corporations. That makes them a big target.
Google Status
Google has a workspace status. Google Workspace is where you manage identities and use services like Gmail and Google Docs if your organization allows people to use those.
https://www.google.com/appsstatus/dashboard/
There is a separate link for Google Cloud status.
https://status.cloud.google.com/
Bug Bounty
Do you care? I personally feel that companies that expose themselves via a bug bounty are being more transparent and exposing themselves to more security testers. I don’t think bug bounties can or will find everything but they are a good way to see how many findings exist and if companies are addressing and fixing those findings quickly.
Microsoft has a bug bounty.
https://www.microsoft.com/en-us/msrc/bounty
Google has a bug bounty. Love the stats.
https://bughunters.google.com/
AWS does not have a bug bounty at the time of this writing.
You can obviously report vulnerabilities but there’s no established method of payment. The problem is that people will not work at proving something is a vulnerability, even if they find something, as they need to get compensated for their time. They may report and move on and focus on things that pay them money instead. I’m really curious why AWS still does not have a bug bounty.
https://aws.amazon.com/security/vulnerability-reporting
Curiously, Amazon does have a bug bounty on HackerOne.
https://hackerone.com/amazonvrp?type=team
AWS did try to launch something that some people called a bug bounty but it’s not exactly a security bug bounty.
I would give you a link to that article but turns out there’s an image coming from a Chinese source on that page which is blocked on my network. But back to the topic. The program AWS launched is called “Bug Bust” and you can read about it here. It makes a game out of finding and fixing bugs but it’s not actually a bug bounty that lets you report security findings on AWS products and get paid for it. Although AWS announced they would pay money for some bugs I don’t see anything about that here at first glance, but perhaps it is in the fine print.
Review Vulnerabilities
There are a number of different ways you can review vulnerabilities for cloud platforms. You can search for vulnerabilities in online CVE tracking systems like NIST or Mitre.
You can check CISA advisories for vulnerabilities and known exploited vulnerabilities. They also have a mailing list.
If your vendor participates in a bug bounty program you can possibly see reported findings, if the vendor allowed them to be disclosed. This gives you an idea if the vendor is responding to and fixing vulnerabilities promptly and paying a fair price. If vendors do not pay a fair price for vulnerabilities, someone else might — and it might not be someone that is very ethical.
Once again, you can search the news for vulnerabilities and data breaches the same way I showed you how to do that for outages above. Search Google news links for different time periods and see what type of issues arise.
In addition you can take a look at vulnerabilities tracked and reported by the vendor themself. Here are the links for the top three:
Amazon Security Bulletins
https://aws.amazon.com/security/security-bulletins/
Microsoft Security Bulletins
https://learn.microsoft.com/en-us/security-updates/
Google Security Bulletins
https://cloud.google.com/support/bulletins
Understand the complexity
For each new cloud service you use, you need to understand all the security related configurations that you will need to set, monitor and maintain for that new service. For every new service you add you are increasing the complexity in your environment and the attack surface that attackers can go after to try to get into your systems and data. It also increases the number of log sources and the volume of logs you need to monitor. (Hopefully the vendor has good logs right?!) Consider the time and work that goes into maintaining and monitoring this new environment. Hopefully I can provide some tips for making that easier in AWS, Azure, and GCP.
Try before you buy
Many types of cloud platforms will give you a free or low risk trial. Use it! Try out the security features you are planning to use and see how they work. Use this opportunity to ask the vendor questions. The support is often much better before you buy than after you are a customer. The nice thing about pay-as-you-go cloud services is that you don’t have to shell out a lot of money or lock yourself into an expensive license or price tier to try almost any feature.
Too many times clients tell me on calls that they purchased a service or cloud security product and they can’t get it working correctly and the vendor is not very responsive. Get help before you sign the contact (without exposing sensitive networks, data, or credentials.)
Document and communicate the process for vendor assessments
One problem companies often have is that they have a process and a policy that exists in a vacuum. People are not aware of the policy and the security team gets mad when people don’t follow it. Communication is key. So is repetition.
People are busy and you have to make the policy clear and obvious to anyone who wants to sign up for a cloud vendor. Do not email the policy out and expect people to read it. You will have to do three things:
- Tell them face to face.
- Remind them.
- Ensure that if they forget, they will be stopped by the legal or finance team.
Let people know that payments to cloud vendors may not be expensed and that people should not put corporate assets in cloud systems without prior approval. Your network should be blocking this to at least some degree, but it is difficult to enforce. In some cases, you have to trust your employees to do the right thing once they have been informed. Also, you need to make sure they can get their jobs done easily within the boundaries of the policy.
Ensure the CEO and Board Enforce the Policy
The process for bringing on new cloud vendors must be enforced. If an executive or anyone who doesn’t like the policy can go around it by planting a bug in the CEO’s ear, then the policy is useless. The CEO, board, legal, and finance teams must be on board and understand why the policy exists. If they do not actually agree with or enforce the policy, that is likely going to lead to a lot of wasted time for security teams and potentially a data breach.
Exceptions may exist. There may be a truly valid reason for bypassing the policy in some cases. When that happens, the person who wanted and approved the exception should sign their name on a document showing it was their wish and choice to accept the associated risk. This is related to my post on Security Teams are Not Enforcers. Often the security team is blamed after a breach, but the security team was not the one who wanted the risky vendor or vendor configuration. It was someone else.
What happens if you want to say goodbye?
Make sure you understand the rules and steps around closing an account — and who can close the account. More on that in the next post.
Did the vendor promise something?
Get the terms of your agreement in writing, in a contract. I learned this the hard way in my first business. I’m going to expose how naive I was in the past now, even after reading books on the subject, and why I use a lawyer for almost everything these days.
First, when I ran my e-commerce business before my current business, I had an agreement with a company to backup my systems. You and I might assume that means ALL THE SYSTEMS managed by that vendor. But when a Linux system failed, the vendor tried to claim they only had to back up the Windows systems. Luckily I did not have to go after the vendor for compensation and prove or disprove that theory because my client was thankfully kind and found a way to get their data back.
In another case, I had, in writing in an email, a promise that a client was going to do this and that for me. In the end, none of what she wrote mattered. My client made promises she couldn’t keep because she did not have the authority to make those promises. And in the end she was pretty self-serving even though I had helped her build her business up from almost nothing as the sole technical person for most of the life of the business to that date. Good intentions and promises only matter when they are in the contract. People don’t always appreciate your contributions and will take advantage of your desire to do good work at your own expense.
During the dot-bomb meltdown in 2000, my e-commerce company had a $90,000 contract that would have possibly helped us ride out the storm. We had a 33% non-refundable retainer. Someone new took over the organization we were working with and used all manner of tricks to essentially give that contract to her best friend’s business that was hurting (weren’t we all.) Luckily, we got to keep the retainer but I still had to pay my employee’s health insurance out of my own savings. I found out later one of them had cancer and didn’t tell us, so I’m glad I did that. Evaluate the contract for loopholes and consider the consequences if something goes wrong.
In yet another case, a customer signed a contract and when we landed in small claims court, the judge seemingly did not read the very long and detailed contract based on the time she spent looking at it. She didn’t understand a thing about technology. The customer breached the contract in multiple ways by trying to essentially hack and break the e-commerce platform and yet all the judge said was “you didn’t meet the customer’s expectations” (which were basically insane). On my part, I just copied some online contract and modified it and didn’t have a lawyer. I had no money and no clue what I was doing. Live and learn.
Although I had paid money out of my own pocket to try to show that client how to make money on their website, it was all for naught. It was one of those clients where you knew you never should have taken them on in the first place. I did, in fact, turn them down multiple times before I finally accepted. They were in a dispute with their prior hosting company. What a nightmare and a mistake. My ex-business partner wanted me to fight the end result but I just never wanted to see those people again so I paid them and moved on. But I did learn a valuable lesson.
The final example of what not to do, is when a salesperson promised me that if I signed a lease on an office and needed to get out of it that it wouldn’t be a problem. That was a lie. I spent months completely stressing over that situation and could not get out of that contract until I got my own lawyer — and as it turns out that company was breaking California state law. If I had just paid a lawyer in the first place, they would have advised me not to sign it, and also gotten me out of it more quickly.
Never believe or trust a sales person’s promises. Always have a lawyer review your contracts and advise on legal risks — unless you really want to take your chances. On occasions I have just wanted to get in or out of a situation and I feel that the potential losses are worth the risk. For example, I took a risk to get paid a lot of money and get a particular type of experience even though my lawyer was correct in his assessment of a particular clause in a contract. His prediction was accurate, but the amount of money I made and the experience I got in the process was well worth that risk. I don’t regret that. I have hired risky contractors in my home for projects. Sometimes it works out and sometimes it doesn’t. The risk you are willing to take correlates with your potential losses and what you are trying to achieve.
For the most part, you should abide by these principles. Especially if you are planning to sign something that might result in significant business expense or loss if something goes wrong:
- If they said it, it doesn’t matter unless it’s in your contract.
- Have a lawyer review any contract before you sign it, or live with the consequences.
- Understand the terms for if and when that agreement can change.
- Don’t say anything without talking to your lawyer first in the face of a dispute.
These are the things that I have sadly learned due to people who were unethical or at least self-serving in their approach to business, sales, and agreements. Of course, there are also cases where everyone is just doing the best they can and mistakes happen. But just because you read something on a web site or just because a sales person told you something about a product doesn’t mean you can bank on those promises. If you think the cloud provider is going to tell you about vulnerabilities or breaches or stay in compliant with a particular regulation you also must follow, get it in writing.
Involve your legal team.
Don’t give the vendor any sensitive data or information until the legal, financial, and security review is complete.
From a security perspective, your contract should outline what your responsibilities are and what security requirements the vendor must fulfill. I wrote about how that applies to AWS here.
If you are not familiar with contract law, check out this book.
Ok now that I’ve exposed how completely naive I was when I started my e-commerce business let’s move on. By the way, most of the time, people will treat you right, I have found, if you have a lawyer review your contracts and get everything clearly communicated up front. People have proper expectations going into the agreement. It also helps that I’ve had wonderful customers since I started 2nd Sight Lab. Thank you!
Follow for updates.
Teri Radichel | © 2nd Sight Lab 2023
About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight LabNeed Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for PresentationFollow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab