Test Automation for Deployments

ACM.26 Making sure code changes don’t break something else

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics | The Code

🔒 Related Stories: Application Security | Secure Code

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the last post, I wrote about why you might want to consider having separate IAM administrators dedicated to the task of assigning permissions for users and compute resources on AWS. We left the policies associated with application roles in the hands potentially of application developers or an AppSec team. We’ll delve into that in more detail and consider who should handle resource policies in upcoming posts.

This post is going to be short and sweet. If you follow on Twitter you might have noticed that I finally got covid and it hasn’t exactly been “no symptoms”. Hopefully, I am on the mend. The bugs in my code will have to wait.

In my last post I explained how and why I centralized the IAM scripts to a central location. However, I left the policies for specific batch jobs in their own folders. The idea is that the IAM team can handle the core functionality and the batch-job specific policies may be handled by others. Even if they are not, I expect the core functionality will require few changes in the future, such as who can assume the roles, versus a new policy document we need to create for each new batch job in accordance with zero-trust security policies.

Even though I don’t have a lot of code yet, those changes were extensive enough that I was worried about introducing an error. Any time you change a piece of code there’s a chance of error. So I wanted to test it all out. When I did that I realized that it was getting a bit tricky to remember which roles go where. I would be better off documenting this now, and the best way to document it is via some test scripts.

I added a test.sh file in all the pertinent folders and then call all those test scripts from the root folder. That way I can test independent folders or all the code at once.

For example if you look at the updated codebase in GitHub now you’ll see the following:

• There’s a test.sh file inside the iam folder.
• Inside the iam folder I have folders for each of the iam identities I’ve created so far (users or roles.)
• If you’ve been following along you’ll notice that I consistently put a deploy.sh file in each folder where I deploy something so each of the iam folders has a deploy.sh file in it.

So my test.sh script is pretty simple:

The one difference you might notice is that for one of the scripts where I have to pass in an ARN I look that up from the outputs of the template. If and when I move things into separate accounts I’ll need to think about how I’m going to to deploy that, but for now it allows me to test the code and helps me remember who is supposed to be allowed to do what and what arguments I need to pass to which scripts.

In my root folder I test all the scripts in the subfolders using another test.sh file.

Now whenever I re-run my code I can validate that I haven’t broken something else in the process.

Note: The test automation code in this post uses to AWS CLI profiles for the IAM and KMS user and role that require MFA. I’ll explain how to do that in an upcoming post as some of what is in this repo I haven’t written about yet. I also have one failing test at the moment I’ll fix before I publish the related post. I haven’t written about that code yet.

Test your deployments

It is very important to test not only the functionality of applications but also deployment code.

Test in a separate environment

Although I’m testing in my own account hopefully I have used appropriate parameters so this runs in any account. I’ll test that later. When you are deploying code to another environment later, such as a production environment vs. a dev environment, you want to test your deployment code first in some other environment to make sure the deployment works. Ideally you have a separate staging environment that mirrors production but at a minimum test your deployment in a QA environment.

Test Automation

It would be very tedious for me to go into every file and test it separately to make sure my changes haven’t broken the code. Test automation helps you check every time you make a change that you haven’t broken something else.

This is very simple code. We had much more complicated code when writing stored procedures for banking systems with complex parameters and logic. Whenever possible I try to break things up into smaller components that are easier to test when you can. Too often those complex unit tests would be ignored or turned off for the sake of getting projects out the door. However, sometimes it is not possible to make things simpler.

Testing user interfaces that change a lot can also be complicated. Each time the code changes, the test needs to be updated. For this reason it might best to do your test automation on a user interface after it is somewhat stable.

Test automation is not easy and I’m not a stickler like some other people that say you must have a unit test for every single code change, but whenever you can, test automation will help you prevent errors by quickly validating that a code change hasn’t broken something.

Testing for Accuracy

One thing I’m not doing in my tests is testing for the accuracy of what has been deployed. I could add that later as time allows.

Code on Github:

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2022

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab