Simplifying Deployment Scripts While Maintaining Segregation of Duties

ACM.367 Eliminating some of the context switching in my first iteration of securing deployment code

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics | Code.

🔒 Related Stories: AWS Security | Secure Code | Cybersecurity | IAM

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In my last post I revisited my function that validates parameters as part of the reorganization and cleanup of my code base for this series.

In this post, I’m going to explain how I’m changing my deployment scripts and why. In my initial iteration I started putting deploy.sh scripts in each folder and running them as needed. That turned out to be painful later and here’s why.

Bash and file paths

When I started programming in Java, file paths were the bane of my existence. Do I set the home directory to the Java folder, the bin folder, or where my code is located? Does it have a slash at the end? Forward slash or backslash (depending on the operating system). If I pass the path in some kind of programming language do I have to escape the slashes?

I think I spent more time on file paths than any time I spent actually writing code, which was the simple part. I don’t want anyone to deal with that in my code base. I’m not going to make you set any magic environment variables (which can be modified by malicious people as well) in my code base. All paths will be relative.

The problem is that bash determines the location of files from the point where the code gets executed, not based on the location of the file. So if you try to run a script in one folder that uses a script that was initially executed from another folder while testing and it sources a file, things will break.

To get around this I had this whole switching function that would switch to the correct folder for the type you’re trying to deploy but that was not feasible. Just look at the folders I generated for different resource types here.

Yeah, no.

A common deployment script folder

That problem is why I’m changing my code to use a common deployment script folder, and all files will be executed from the root of this code base, which is currently:

SecurityMetricsAutomation/v1/

I’ve been using various roles ending with “admin” in the name to deploy things. That will remain consistent. I’m going to organize my folders by the administrator name that owns and runs a particular script.

Different administrators exist that can deploy different resources. For example, when the account first gets created, the initial root user in the first account has to run a script to create a new user. That script will go here.

/deploy/root/rootadmin.sh

Note that I decided to rename my orgroot user to rootadmin for consistency. The initial rootadmin user can deploy the initial structure. I haven’t fully worked this out but it will likely start with an OrgAdmin account where the administrators who deploy things across the organization will exist. I am planning to test using Okta further but I’ll start with AWS IAM users.

Here are some of the admins I’m thinking about creating:

billingadmin - billing, budgets, and creating new accounts
orgadmin - create OUs, move accounts, and service control policies
iamadmin - deploy the IDP in the root account and roles across the org
kmsadmin - deploy KMS keys in a KMS account with corresponding KMS policy
networkadmin - deploy networking across the organization
securityadmin - deploy security services and configure logs

Each of the above admins may have their own scripts for deploying resources. Those scripts use the resource templates and functions.

All files will be executed from the root. So for example, if I need to deploy an environment script owned by the orgadmin it might look like this:

./deploy/orgadmin/environment.sh

If that file needs to source other files it would do so like this:

source resources/ssm/parameter/functions.sh

I already explained how the script that deploys CloudFormation stacks can now calculate template paths using this approach and a consistent naming convention.

Environment Deployments

How will environments get deployed? There’s a series of steps to deploy an environment which consists of running scripts from multiple admins. Perhaps there’s a script in the orgadmin folder to create an environment that uses scripts from other admins but the script can only get executed by switching to the other roles. Or perhaps that user must use scripts from other folders and code it can’t change but is allowed to execute a job to do those things.

I wrote about what I’m planning to put into environments here:

I’ll work this out more concretely as I deploy an environment.

Application Deployments

Along similar lines, there needs to be an application admins in each environment that deploy applications and application resources. Those resources may include scripts from other admins’ directories. The application admins may have permission to run the scripts and pass in variables but not change the code.

Who can change the CloudFormation templates?

I wrote before about how you could use my code by checking it out and referencing it in your own scripts here.

It might (or might not) be overkill to have a separate repository for every resource type depending on the size of your organization and how you break out duties. But you could do that and limit who can change the code for different resource types. Of course this gets complicated with EC2 which should have never included networking, in my opinion.

However you break apart your repositories, you can rearrange and concatenate them when you create a container for deployment purposes. And going forward, I’m going to almost always use containers as I will demonstrate.

You can divvy up responsibility for this code any way you want and put it in different repositories and then pull in the required resources into a container as needed. The deployment scripts would be easy to break out into different repositories. The CloudFormation resources are a bit trickier but doable.

The other thing you could do is have a team of architects (the nice kind, not the blocking kind that doesn’t have time to explain why things are the way they are or try to take over people’s projects when it is not necessary) oversee the resources repository. People can create pull requests on that repository and ask to push changes up for things they need. Along the way, the architects can explain why or why not.

That’s something I proposed when I was at Capital one and it sort of got implemented. The problem was when people pushed up a whole series of complex changes at once that were essentially impossible to review. So there has to be a good mechanism for discussion as to what the person is trying to achieve and why they are making so many changes. People also need to understand that if they deviate too far from a common template the review will take much longer to evaluate.

I wrote about code reviews here:

However you break my monolithic repository down, consider who you want changing what code and don’t just make it a free-for-all. Unless you like data breaches and misconfigurations. 😊

Now I need to start migrating over the code, adjusting, and using it.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2023

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab