Easier CloudFormation
How AWS re:Invent 2014 GameDay got me started
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Related Stories: AWS Security | Security Automation
💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When I started getting serious about AWS, I immediately understood how CloudFormation could help organizations to improve cybersecurity. More on that later. First, some tips to make CloudFormation easier to learn and use. When I hired a particular individual at a company who is brilliant, he didn’t like CloudFormation much. He already knew how to write it. I shared some of the structural tips at the end with him, and then he liked it a lot better, so now I’m sharing them with you. The general tips about how to structure and troubleshoot templates also apply to Terraform, Azure, and Google Cloud Platform templates.
Where to start
The first time you look at a long involved AWS CloudFormation template, it may seem overwhelming. I felt the same way. Even though I had many years of experience as a software engineer and knew many software languages, it was something new, and I had to figure out where to start. I wasn’t allowed to use AWS at work when I got seriously interested in using it for production workloads. I was using AWS in a personal account just to get started and didn’t get around to learning CloudFormation immediately, but I knew I needed it to take full advantage of the AWS cloud capabilities.
Resources ~ the building blocks
There was one simple thing that allowed me to learn CloudFormation quickly. I went to my first AWS re:Invent conference in I believe 2014. It’s hard to remember. It was not even close to how big it is today, but it was still overwhelming. On one of the days, I was wandering around, and I stumbled on AWS GameDay. I was able to get in and join a team. Luckily I landed on a team with some people experienced on AWS. One of them, named Matt, knew CloudFormation well and even had some scripts from work he could modify to help us. At some point, a photo of us was floating around the Internet to promote the next re:Invent, but unfortunately, I don’t have it to share here.
It was on that day that I got the most valuable tip that helped me start using CloudFormation. Although there are many sample scripts, you need first to understand the fundamental constructs. I asked him how he knew how to create the code for a particular thing on AWS and pointed me to the AWS CloudFormation resources page.
Got it! This single web page made all the difference. Each AWS resource such as an EC2 instance, VPC, or IAM role has an associated block of code in the template, and the CloudFormation resources page tells you how to write the code for each resource. Tip: You can find this page easily by searching for CloudFormation Resources in Google.
Let’s look at an S3 Bucket as an example.
The top of the page shows the name, format, and all possible properties for the resource. As you see below you can use these properties to define things like the name of the bucket, tags, the logging configuration, and a bucket policy that defines who can access the files (objects) in the bucket.
The next section describes each property and tells you important details: is it required or optional and what values are allowed. For example, here are the details for the BucketName property listed above.
The last section has working* examples. Here is the example on that page at the time of this writing for an S3 bucket with encryption on by default.
* Occasionally, I find a code sample doesn’t work, but typically you can copy and paste examples with very few modifications for your account, and you have working code.
The CloudFormation Template
Now you have the building blocks to put into a CloudFormation template, but it needs to be structured correctly so the CloudFormation engine can deploy your resources.
You can write CloudFormation in JSON or YAML ~ two different file formats the CloudFormation parsing engine understands. I’m going to use YAML in my examples. YAML requires proper indents and spacing. It’s picky! Find yourself an IDE (integrated development environment) that understands whichever format you choose and shows you when your formatting is off. You can try out AWS Cloud9, for example, but most IDEs have a configuration that validates JSON or YAML.
There are two things you need to create your first template. The first line is always pretty much the same. It looks like this, and you can just copy and paste this at the top of your file:
The only other thing that is required is the resources section. Add the heading for the resources section.
Each resource will start with a name used to reference the resource within the template. In our S3 example above, the name for the S3 bucket resource is EncryptedS3Bucket. You can name your resources whatever you want, within the rules for CloudFormation resource names. The name will be followed by the type, which in this case, is an S3 bucket.
Now put it all together. Add those lines to a file — with the proper spacing. Note the indentation and space after “Type:”. This matters. The Description section is optional, but it helps people understand the purpose of your template and it will show up in the AWS Console. Add the properties you want to set for your S3 bucket.
You have created a CloudFormation Template. You can deploy it by using the AWS CloudFormation console:
You can also use the AWS SDKs or CLI to deploy CloudFormation stacks. See the bottom of this page for an example:
Making CloudFormation easier
That was a really simple example, but sometimes you are going to see much more complicated templates with many different parts and pieces. You also may be wondering how to create a complicated stack that involves networking, EC2 Instances, IAM roles, and many other components.
The biggest tip I have is this: Start from the beginning. Learn each of the components before trying to deploy a complicated template with no understanding of the individual parts. For example, there are other sections of a CloudFormation template. Take the time to understand the purpose and structure of each section, such as parameters (data you can pass into a template), mappings (data structures you can create and subsequently use in your template), and outputs (data you can pass out of the template.
CloudFormation has some special intrinsic functions that you can use to transform data in your templates. Learn how to use and read these functions:
Pseudo Parameters help you create reusable templates. Don’t ever hard-code things like your region or account number into a template. CloudFormation can retrieve these for you. Then your template can run in different accounts and regions without modification.
Leverage custom parameters to create reusable templates. Abstract out anything hard-coded in your template that prevents you from using it for multiple applications if you can. Pass variable values in at deployment time.
Break monolithic templates into manageable components that you can deploy separately. Typically you deploy the network and do not change it much. That should be in a separate template. You will probably deploy your web tier changes separately from the data tier. Best practice for microservices includes the ability to make changes and deploy them independently.
Leverage outputs to pass data out of one template into another template as a parameter. For example, when you create your networking templates, you can use outputs such as your VPC ID and security group ID as an input to another template.
Don’t try to write CloudFormation templates for a stack all at once and then test it. Write a template for one or two resources at a time. Test it. Then add another resource. Test it. This iterative and methodical approach makes troubleshooting much easier!
Another thing that developers sometimes struggle with is that they think of CloudFormation as executable code. It’s not. CloudFormation template code is data that defines the resources you want to deploy. The executable code that deploys your resources is in the CloudFormation engine that takes your data as input. As with any technical construct, use the correct technology appropriately for the purpose it serves.
I tend to like to keep my executable code and data separate as much as possible, but there are cases where they intertwine. One example of this in CloudFormation is the UserData section of an EC2 Instance resource. In this section, you can pass commands in CloudFormation to the template that gets executed on your virtual machine when it starts. This section is a bit tricky because you need to escape special characters so the CloudFormation engine can process the code correctly. I mentioned the built-in AWS CloudFormation functions above. You’ll typically see two of those in the UserData section: one to transform this code to Base64 encoding and one to join the individual lines of code together. As I mentioned earlier, take CloudFormation step by step and learn each piece to make it easier. You’ll find an example of EC2 UserData and more information on this page:
The CloudFormation engine is better than writing AWS CLI scripts or custom code to deploy resources in most cases because it understands the dependencies between resources within a single template. There are other benefits I’ll mention in an upcoming blog post. When you deploy multiple templates you may need to ensure one template completes before you deploy another. Most of the time, CloudFormation knows that it needs to wait before deploying one resource until its dependencies are ready for resources in a single template. It also knows to delete resources in the opposite order when you delete a CloudFormation stack. I’ve occasionally found exceptions to this, and that’s when you need to use the DependsOn attribute. If you indicate one resource depends on another, CloudFormation will understand that it needs to deploy the dependency first.
Most of the time, I have no problem writing CloudFormation from scratch, but some things get tricky. One of those is networking. I used to be on the team of five people deploying networking for all the applications for 11,000 developers at Capital One. We had repeatable patterns, and I quickly decided I needed to write some code to auto-assign IPs and generate standard templates without hand-coded errors. There are some tools out there like the CDK that generates your CloudFormation code for you. Using tools that generate code is fine as long as the code they produce follows security best practices. You need to make sure the generated code isn’t deploying too much network access, such as leaving the default outbound rules open in a security group. In the end, you can still analyze the output (CloudFormation) in a structured data format before deploying it to production.
Hopefully, some of those tips help you get started with CloudFormation or make it easier to deploy if you are already using it. More on CloudFormation security in an upcoming post.
Follow for updates.
Teri Radichel | © 2nd Sight Lab 2018
About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight LabNeed Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for PresentationFollow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab
