Cannot Pass Spaces to AWS CloudFormation Deploy Parameter-Overrides

A convoluted combination of Bash and CloudFormation that caused obnoxiously confusing errors

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics. The Code.

🔒 Related Stories: AWS Security | Application Security | CloudFormation

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This post was updated on 9/9/2022 after I reported the spammers who are blatantly copying it word or word because I came across another scenario I had to fix. If you see people plagiarizing content please take the time to report it. All of the authors in the world thank you.

I spent way, way too long trying to figure out how to pass a KMS Key description with spaces into a CloudFormation stack with parameter overrides.

It doesn’t help that AWS seems to have different and what seem to be incorrect answers all over the place.

Somewhere in the documentation I came across an example like this:

aws cloudformation deploy [...] --parameter-overrides key=value key2=value

I read all these pages that say you should be able to pass in parameters by quoting the key and value like this:

"key=value"

However, that wasn’t really clear. What if you have multiple parameters to override? Do the quotes go around all the parameters like this?

"key=value key2=value"

or just each key, value pair like this?

"key=value" "key2=value"

[And why is it called “override” anyway instead of just parameters — that doesn’t really make sense. I’m not overriding anything I’m passing in parameters…but I digress]

I tried both ways and got errors saying my stack couldn’t deploy and/or the space threw something out of whack so each word in my string with spaces was used for each key-value pair. Neither one of those formats alone worked for me.

Then I tried this:

"key='something with spaces' key2='something else'"

With the above format when I looked in the CloudFormation console at the parameters the single quotes got included in my parameters which is not what I want. It appears the single quotes are in the value instead of surrounding it.

The solution here didn’t work either though apparently it works in SAM:

I tried a whole bunch of variations of the solution at the bottom of this post, but I’m trying not to pass in a JSON string. YAML is cleaner and avoiding JSON as much as possible.

This problem has been reported over and over. And in the end absolutely none of the answers were right in my case…but read on as I have special case. Of course I do.

I just found this post which is interesting:

the thing is that each key value pair is in single quotes except the one with spaces:

"TAG2='Test Project'"

I was putting quotes around the whole string of parameters not each individual key value pair. Let’s try it.

Well at least I get a different error:

JSON passed to --parameter-overrides must be one of the formats: ["Key1=Value1","Key2=Value2", ...] , [{"ParameterKey": "Key1", "ParameterValue": "Value1"}, ...] , ["Parameters": {"Key1": "Value1", "Key2": "Value2", ...}]

This?

parameters="[\"EncryptArnParam=$encryptarn\",\"DecryptArnParam=$decryptarn\",\"KeyAliasParam=$keyalias\",\"DescParam=$desc\"]"

Nope. How about this?

parameters="{[\"EncryptArnParam=$encryptarn\",\"DecryptArnParam=$decryptarn\",\"KeyAliasParam=$keyalias\",\"DescParam=$desc\"]}"

Nope. Maybe this?

parameters='"EncryptArnParam=$encryptarn","DecryptArnParam=$decryptarn","KeyAliasParam=$keyalias","DescParam=$desc"'

When all else fails…read the documentation. I found this page:

This example shows putting \\ after a value with spaces in it.

--parameter-overrides ParameterKey=MaximumExecutionFrequency,ParameterValue=TwentyFour_Hours\\,Twelve_Hours

Well that’s different. Let’s try it. The example is not for the deploy command but you would expect consistency for switches named “parameter-overrides” for CloudFormation, no? No.

It doesn’t work.

This doesn’t work:

Because then I get this:

This:

Tells me that some of my parameters don’t have values.

Ok I have this working elsewhere. Looking at where it works I have this:

So that should work but it doesn’t seem to work with spaces. Let’s revisit that error message above once again. It contradicts documentation elsewhere because the documentation I found had spaces between key value pairs and no brackets. It also didn’t have curly braces.

I found but the first example in the above error message easy enough to replicate.

This “appears” to work.

I don’t get an error but I think that’s because I moved the problematic parameter to the end I think. Maybe not after I figured out a final solution. When I go to the AWS CloudFormation console the description only has the first word:

What I realized after printing out my parameter string is that the parameter in the parameter string in two different functions was short. I pass the variable holding the value with spaces into a function that formulates the parameter string. Then I pass the parameter string to another function.

Here’s what happened with that. The value got truncated right from the start in bash when passing it to the other function. I needed to put quotes around the value of the variable in the initial function call because bash also delineates arguments by quotes.

So this:

deploy_key $profile $encryptarn $decryptarn $keyalias $desc

became this:

deploy_key $profile $encryptarn $decryptarn $keyalias "$desc"

I already knew that I had to put values are the passed in arguments in the next function but at some point while testing I had removed them. So this:

deploy_key(){

  profile=$1
  encryptarn=$2
  decryptarn=$3
  keyalias=$4
  desc=$5

Needed to be this:

deploy_key(){

profile=$1
  encryptarn=$2
  decryptarn=$3
  keyalias=$4
  desc="$5" 

When I add the above quotes, now one of my parameter values has spaces in it and CloudFormation croaks in may ways. Rather than try to fix my already overly-complicated parameter string I forced my value to have single quotes around it like this:

desc="'$desc'"

Then when it gets added to my parameter string the value has single quotes in it within this lovely convoluted structure and I don’t have to try to sort out and escape more quotes here:

parameters='["EncryptArnParam='$encryptarn'","DecryptArnParam='$decryptarn'","KeyAliasParam='$keyalias'","DescParam='$desc'"]'

Next, I also need to put quotes around the parameters variable I pass into the next function:

deploy_stack $profile $keyalias $resourcetype $template "$parameters"

And finally….I have to add quotes around the argument I retrieve in the last function (I had at some point quoted all these arguments in frustration at some point but only the parameters argument has spaces in it.

deploy_stack () {
  profile="$1"
  resourcename="$2"
  resourcetype="$3"
  template="$4"
  parameters="$5"

And, I do some other things in between but finally I call and use my parameters. I quoted this value as well.

aws cloudformation deploy \
  --profile $profile \
  --stack-name $stackname \
  --template-file $template \
  --parameter-overrides \
    "$parameters"

Then when I check CloudFormation, I can see that my value with strings is present.

PHEW!!!

I thought for sure I would not be able to get this working in the middle of all this. The main thing that helped was a useful, though somewhat confusing, error message. The only problem is that it took me a long time and a lot of poking and prodding to get that error message. Perhaps there is a way to provide a better error message earlier based on the other inputs. Maybe AWS can add those and any other manner of convoluted combinations of quotes to their test cases.

[Update:] I found another issue when I changed the code in the function I am calling to this:

I started getting errors again due to the way I was constructing my string in the above. I seriously wonder if there’s some kind of escaping bug here not the issue every one is just blowing off but I don’t have time to explore it further. I hope it does not lead to any security problems because this behavior is just odd. I haven’t had time to think through and definitely not time to look at the underlying code. Maybe AWS can. At any rate here’s how I fixed the second problem.

I wrapped the value I’m passing in with spaces with quotes like this:

I thought that fixed the problem but it didn’t. It put the variable name in the CloudFormation Parameters.

Back to the drawing board.

If I had time to think about this more it wouldn’t be such a guessing game but this is not what I want to be spending my time on at the moment. I hope this helps someone and maybe someone at AWS who works on EC2 instances can look at the OS if this can’t be improved in CloudFormation. This behavior just seems odd.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2022

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab