Love Your QA Team

They prevent customers like me from finding YOUR bugs in production

In the last post, I wrote about importing the default components deployed outside of CloudFormation when you create a new VPC so you can monitor and manage them from within CloudFormation.

Now, as I’m trying to get work done outside of this blog so I can get back to blogging, I have to stop and write a very quick post about something that is currently slowing me down and led me to a few memories that may help companies and developers improve the quality of their products and code. The quality of your code and your products affects everyone down the line if you produce tools that other people use to get their work done.

Feel free to pass this along to people to help them understand the importance of QA in your organization!

Why you should love your QA team

When I was managing a team that was performing a cloud deployment there was initially a clash between my top DevOps person and my top QA person. The DevOps person came from an IT background and was not used to having people test his work. He was so super duper amazing smart that any finding of a problem in his work was considered an insult, I think.

I mean, the guy was amazingly smart.

But so was the QA person. He was very particular as well, whereas the DevOps person could solve problems at lightning speed and get a solution in place — kind of like I used to be — people accused me of going to too fast, except this guy was probably smarter than me. But the DevOps person would go so fast that he’d miss things.

The QA person would find them.

I tend to go fast when I’m fleshing out an idea or trying to get something done in a hurry. I’ve been known to annoy QA people when I was in a rush. 😁 Sorry. Too many bugs.

This is the beauty of QA. You can go fast but then someone else can come along who is super detailed and find your mistakes. You might think that some of the QA person’s findings are not a big deal and too minor to bother with and get annoyed — but that’s the thing. It’s a balance between being detailed and being sort of haphazardly fast.

I like to be a bit fast so I can get to the solution quickly to see if the idea is even going to work. But then you have to clean it up — make perfect as it needs to be.

Whether something is perfect enough is not a decision for the DevOps person or the QA person. It is typically a decision for the business owner — the person who owns the system — or an architect, or a product manager. It is a question for the person who will be responsible for how the system operates and how customers experience it.

The beauty of high quality QA and why I loved my QA people so much is that they kept MY bugs out of production! I didn’t want mistakes in production. They allowed me to go fast because I knew they would help me find all the problems later so we could have a perfect system go to production. And I worked with some amazing QA people who were very smart on various financial systems who helped ensure that all the numbers added up perfectly.

Nobody’s perfect — and that’s OK

I made a big mistake once and a QA person missed it — in part because a product manager came over and told the QA person they only needed to check this, this and this. I think if the project manager had never gotten involved that would not have happened. She was trying to be helpful so I appreciate that and don’t blame her — it was still my mistake and the QA person should have tested it more thoroughly — but in the end, the result was a disaster for me — a significant customer facing bug which was very rare over the course of my career when it comes to financial systems.

I try to swing back around and test things in the end alongside the QA folks but I took a particular piece of code for granted that I should not have. And when it comes to financial systems and people’s money — you don’t want to make a mistake.

In that case I was so focused on testing the underlying system and how the money added up with minute transactions occurring daily in a system processing billions of dollars of assets under management that I skipped over testing of a small template change that caused a number to look wrong on an invoice — though it was correct in the underlying system. The problem occurred when another team was changing at the same time and somehow we had a conflict in our source control system. It got resolved incorrectly and I missed it.

And that’s why you should always merge your code first, then test it, as I wrote about in another post:

Treat your QA team well and understand the value of their work

Good QA team members are life savers. If QA people feel beat up and under appreciated in an organization — the good ones will leave — because they can. And you need excellent, smart QA people to have a good product. I’ve seen people say that developers can test their own code over the years, but many developers and QA people have a different mind set, and you have to look at the system in a different way when you are developing and when you are performing QA.

A developer’s goal is to “make something work.”

A QA person’s goal is to “find all the flaws.”

These are two very different ways to look at a system. You are likely in one mind set or another at any given time, not both. No matter how hard we try, our brain can literally only do one thing at a time. I’ve read about this and unfortunately don’t have a reference, but that’s how our brain works — similar to how a computer CPU works. It can only process one thing at a time but instead is just context switching — while it’s waiting for one task to complete like a network response to come back, it can do something else.

So you are likely either trying to make something work or trying to find all the ways you can break it at any given time, not both.

Good Quality Assurance people study their craft. You need to understand how to test all the logical paths, get code coverage, and perform bounds checking. If you are simply testing to see if something works or not — that is not complete quality assurance. That’s simply functional testing which is only a subset of the QA tests you should be performing on systems.

A good QA person will understand all the different types of QA tests they need to perform to ensure a system meets a particular quality standard. They study techniques and strategies to find and improve the quality of products and try to get better as QA professionals, just like a software engineer or developer studies the latest programming languages and best practices to try to write better code that is efficient and gets things done more quickly.

Your QA person is your best friend if you are a developer. They are the person who will help you find the flaws that you were too busy to test while trying to get the system working. They will help you ensure you have no mistakes in production like the ones I find on my bugs that bite blog in systems I use almost daily.

Don’t let me find your bugs! Make sure you have a solid quality assurance program that prevents these errors before you get to production.

No one is perfect — especially me as I have already pointed out above. So I’m sure I will keep finding bugs like the ones in my bugs that bite blog. But hopefully I’m helping people find the bugs they missed so they can get them fixed, and maybe in the process they can think about how to improve QA to prevent those bugs in the future.

Teamwork is essential to improve processes

If a bug is missed — it’s not the QA person’s fault and it’s not the developer’s fault or the project manager’s fault. It’s a team effort. Everyone needs to learn and improve going forward rather than focusing on someone’s mistakes. The goal is to prevent them in the future. That requires everyone on the team to work together to prevent future mistakes — as much as possible because they will still happen. We are still human, after all.

In addition to trying to improve QA, consider how you might use methods I’ve been writing about in this blog such as Microtemplates. Once you get a particular microtemplate built and tested — if you are using it for the same purposes for which it was originally created, that should help prevent future errors. That means your QA team can focus less on testing that particular item and more on the new and different things you deploy.

This process of abstracting out what is common applies to any type of software that you can turn into a reusable component using the principle of abstraction.

As a team, you can think of ways to speed up testing without missing things along the way.

Penetration testing in QA and the efficacy of security automation

I’ve seen people say that penetration testing is not QA, but really it is. Penetration testing is a form of testing software for flaws and problems just like other types of QA. I would say it is an advanced subset of quality assurance. Whenever I perform a penetration test, I try to provide the steps to reproduce the problem that any QA person can follow to test for that type of vulnerability in the future.

Some tests are more complicated than others, but when I find a simple problem, I know the QA team isn’t testing for security flaws at all — or the company is depending on a scanner to find all the security flaws which doesn’t work. Don’t believe me? If you are using a security scanner alone, hire me for a penetration test and see what I can find. Especially if you have complex applications with various interactions with different systems. I can help you test the efficacy of your scanner.

I can also help your QA team get better by providing information about the bugs I found and tell them how to find those bugs in the future. In addition, I often find a lot of non-security bugs which I put in the appendix.

If you want more customers and to keep your existing customers, love your QA team

Your QA team helps you prevent bugs from getting to production — love and appreciate them because the job of a real QA person is not easy. They are not simply performing functional testing. They are thinking much more deeply about the internal workings of the system and how it might break. They are making the developers and the products at the company look good and work smoothly for customers — which will keep them coming back.

If things break repeatedly or don’t work right, customers will be frustrated. They might look for alternative products.

Products that constantly break or don’t work as advertised frustrate customers — and most of all developers who use products and want to go fast. As I’ve mentioned above, that is the typical M.O. of a software developer, including myself.

I read once that a company was very focused on marketing and it wasn’t working. They couldn’t market hard enough to get new customers. They shifted their focus to the quality of the product. When they improved the product — that’s how they got more customers. Marketing a broken product might get you short term success. A quality product keeps customers for the long term. In order to have a quality product — you need a great quality assurance team. Love them! 🩷

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2023

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab