Testing Lambda Functions Locally Outside of Lambda

ACM.302 Using the AWS Lambda Runtime Interface Emulator (RIE) and how system architectures affect it

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics | Code.

🔒 Related Stories: Lambda | Container Security | Application Security

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the last post I did some container troubleshooting and showed you how to fix some errors. I also explained how redeploying a Lambda that uses a container works.

It seems like some additional documentation and functionality may be forthcoming. At this point I’m wondering how many people are using containers with Lambda. It seems super cool and has been around since 2020. I can’t believe it is not all the rage.

Anyway, moving on, at the end of the post I tried to use the Lambda Runtime Interface Emulator (RIE) but it didn’t work. I wrote about Lambda and container build systems architectures in the previous post and presume the issue is that I’m using the wrong architecture version of the RIE.

What is the Lambda Runtime Interface Emulator (RIE)?

The Lambda Runtime Interface Emulator mimics running a container in Lambda but you run it on your local machine. To better understand what the Lambda runtime is and does, I’m going to write a post on creating a custom runtime in a future post, but essentially the Lambda Runtime helps Lambda know what code to write when you invoke your function.

When you’re running a container, it should be obvious what code you want to run by virtue of how the container is built with the Dockerfile. The Dockerfile tells a container what code to run when it starts. Therefore, if you are using a container then Lambda should be able to figure out what code to run based on the container configuration. And it does, but you can also override those settings.

You can specify the following in a CloudFormation template when you deploy the container:

• ENTRYPOINT — Specifies the absolute path to the entry point of the application.
• CMD — Specifies parameters that you want to pass in with ENTRYPOINT.
• WORKDIR — Specifies the absolute path to the working directory.
• ENV — Specifies an environment variable for the Lambda function.

Security professionals reviewing Lambda functions need to understand this so they aren’t just looking at the containers but also the configuration of the Lambda function when assessing execution paths and attack vectors.

So if Lambda knows what code to run, then why do we need the Lambda Runtime API? Well, what does it do? As I wrote about in another post, it interacts with the Lambda service to send events back to it such as Telemetry and other notifications.

This makes me curious. What if something was wrong with the Lambda Runtime included in our container in the last post and that’s why the status never changed? Hmm.

I was talking to Ben Kehoe about all these runtimes at the AWS Heroes event in Seattle and he said the difference between the runtimes used by Kubernetes and other systems is that they call into your container versus having your container call out. I haven’t looked into all that specifically, but if the code inside the container isn’t correctly calling the Lambda Runtime API you could have problems, I suppose. More research is needed on this point. You should definitely be testing that your containers correctly send the necessary data to Lambda and it shows up where expected.

The Lambda Runtime Interface Emulator (RIE) is a mock of the Lambda Runtime API. It allows you to run your Lambda function outside the Lambda service instead of first uploading the container to Lambda to test it. It emulates running your container as if you had deployed it to Lambda. Ok great. But if it’s just a container running some code, then what’s the difference? Let’s find out.

Testing our container with the Lambda RIE

For now, we will focus on the Lambda Runtime Interface Emulator to test our container locally as if it was running in Lambda.

As noted in the last post, it looks like I’ve pulled the x86 version of the emulator when I need the arm64 version. That’s because I was using an example off the AWS blog which I’m guessing was written before the Arm version existed.

Which line of our docker file is pulling the RIE in?

Aha:

That should be pretty simple to fix. Documentation from the last post:

The runtime interface emulator is available on the AWS GitHub repository. There are separate packages for the x86–64 and arm64 architectures.

So I head up to the Github repository:

Scroll down for architecture-specific instructions. I need the arm version for Linux.

Really, I only need to change my Add line above to use this:

https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie-arm64

And actually all I really needed to change was this:

Rebuild the container as I did in prior posts.

./build.sh

Re-run the local test…well, I get something different. It looks like it’s trying to do something with Lambda.

But…it’s not working. 🥲

I’m guessing what the problem is here, without actually taking the time to test it. I will test this further later but not with this container. It seems like whatever is going on here could provide better error messages because it’s very confusing. But I’m guessing this is what is happening without actually verifying it.

Here’s where we need to understand a bit more about Docker containers. Every Dockerfile contains a FROM statement.

The FROM statement tells the container to start from a particular base image. Someone else build an image, the same way we are doing in this series, and you are building on top of that image — unless you specify to build “FROM scratch” (no base container) which is a whole lot of work and potentially error prone.

If you take a look at the Dockerfile above, it is starting “FROM” a particular container image and building on top of that.

Alpine Linux is a version of Linux in a container that many people use as a starting point for their containers. Alpine Linux describes itself as follows:

Alpine Linux is a security-oriented lightweight Linux distribution based on musl libc and busybox.

If you want to build a container based on Alpine Linux you can use the following FROM statement in your docker file with a specific version that you want to use, or the latest.

FROM alpine:3.14

In the Docker file above it’s starting from a container that has Alpine Linux and python built into it. The versions are passed in as parameters.

Ok, so how were those images built? Here we are back to architectures again.

We need to build from a base image that matches our selected architecture. Alpine Linux offers different base images for Arm and x86. The corresponding Python libraries also need to work on the respective architecture. I’m guessing that’s what is going on here.

Also, this may be why the status never changed in the last post. Something wasn’t working right when the container redeployed. That could lead to odd problems. I can only imagine…

I’m actually surprised the container executed correctly on the Lambda service if the underlying container layer was not the correct architecture. More detailed error handling and architecture verification might help. But I’ll leave that to AWS.

One easy test to verify what I am surmising would be to build everything on an x86 architecture and use the correct x86 RIE library and it might all work.

But instead, I’m going to move on to something I think is more interesting. I want to be able to run my container that clones a GitHub repo on Lambda.

This experiment is taking too long and not actually what I want to be doing. And I still haven’t answered all the questions about container runtimes. More on that in subsequent posts.

Update: Working version of the Lambda Runtime Interface Emulator using a container that runs Bash via a custom Lambda runtime with a better explanation of how to run it here:

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2024

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity for Executives in the Age of Cloud
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Appication Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presenation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Medium: Teri Radichel
❤️ Sign Up For Email
❤️ Twitter: @teriradichel
❤️ Mastodon: @[email protected]
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab