Why I’m Not Using Lambda Step Functions (Yet)

ACM.355 A summary of prior posts on feature requests for AWS Step Functions

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics | Code.

🔒 Related Stories: AWS Security | Secure Code | IAM

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the last post I wrote about creating a base container for Lambda functions. Lambda containers are a great way to create layers of functionality that you can include across Lambda functions.

In this post, I just want to reiterate why I’m not using Step Functions for some of the functionality I’m creating. I’ve written about this before but did not call it out specifically in the title.

If you’re not familiar with step functions I wrote about them in my post below but it’s a way to use Lambda functions to run a series of steps. It’s a really cool concept but missing a few features for my use case.

The first issue is that you cannot create separate roles for each step in a process. I want to be able to give different steps in my process permission via different roles as explained in this post:

Update: Because someone commented without reading the prior post and doesn’t want to create a Medium account so apparently does not value the work people put in here, I will clarify that I was trying to use real CloudFormation in that post. I just also discovered that Twitter(X) no longer has the option to limit messages to people who follow you now. You can contact me via other options at the bottom of the post.

Was able to find a way to do this via Task Definitions via the relevant portion of the comment, but that’s an join statement with concatenated code put into a single CloudFormation property. I would love to see that become a first class citizen in CloudFormation so roles and the rest of the properties in the Task Definition are specified directly in CloudFormation.

I already put in a request for that with AWS at the AWS heroes event in Seattle in August, so maybe it’s on the way. But perhaps I needed to be more specific — I would like to specify roles for each step in the process in real CloudFormation.

I didn’t realize you could assign a role since it doesn’t appear in the CloudFormation definition and when I asked about it no one at the event said including the person speaking said, “Oh you can do that like this…” There was a room full of AWS Heroes who have been using AWS for years, so they either didn’t know you could do this either, it’s new since last time I looked at this, or they understood what I meant when I said I wanted to be able to assign roles in CloudFormation. The product manager told someone to write it down. I don’t know but this doesn’t look right to me. Maybe a better solution is in the works?

Using the method above the S3 team could have defined CloudFormation for S3 like this:

Properties:
  BucketDefinition: {
    a blob of concatenated JSON for all the bucket properties.
  }

#the end

I’m wondering why all of these cannot be CloudFormation properties? It seems like any resource on AWS could just implement a single property where you pass in a blob of JSON but that’s not really CloudFormation is it?

On that note I was asking why IAM Policy documents cannot be first class citizens as well. Would solve a lot of problems and reduce errors I bet.

Besides clarifying all that…

I may also need separate network restrictions for each task. This is something I’ll need to consider more in the future, but for my initial use case, one VPC with the same security group for each step may be fine. That may not always be the case, however.

I also want to be able to require MFA to kick off a job — every time it runs. Note that my solution requires a different code every time the the function runs. It’s not simply requiring MFA for Lambda actions.

I demonstrated how to do the latter in a Service Control Policy (SCP), and some issues with CloudFormation that broke said SCP due to the service logging a domain name in the IPAddress field which should not be allowed. That makes writing policies for IP addresses in accounts where CloudFormation deploys Lambda functions impossible, but I digress.

While testing out this solution, I had multiple issues. I figured out after some trial and tribulation why my Lambda function was taking 10 minutes to execute due to a VPC Endpoint configuration issue.

Then I got to the end and Lambda reports that my role does not have permissions it has. The error message is inaccurate. I have proof because the role works fine in a Local container with the Lambda Runtime Interface Emulator. I may know what is going on as noted in the post but didn’t want to monkey with the AWS Lambda home directory to get around it or spend even more time on this.

I ended up just creating a container I could start up in an EC2 instance instead as demonstrated in the post below. Not sure how well this will work long term but this proof of concept worked.

I could put steps in the above or in multiple containers running in the same host. I’m not sure yet. But I have plans for building out deployment steps with the above solution and a related revamp of my code.

For now, I am going to finish up the deployment of a static website and then go back and revamp and simplify a few things.

It’s too bad Lambda doesn’t support the above because I’ve got a nice base container created in the last post for running AWS CLI commands in Bash and it would be cool to string a few containers together for a deployment.

I would need to be able to also control the logic such that steps could not get run out of order but that needs to be managed in the underlying customer data and logic with proper validation and error handling. I’ll address that in an upcoming post. Separation of concerns.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2023

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab