Mastering Kubernetes One Task at a Time — Ephemeral Storage Volumes with ‘emptyDir’

3 working examples demonstrate three ways of configuring emptyDir Volumes meeting three specific purposes. Learn all of them right here!

Table of Contents

1. Introduction
2. Kubernetes Documentation References for emptyDir
3. Create and Access KIND Kubernetes Cluster on EC2
4. OBJECTIVE 1 — Communicate Between Containers Using emptyDir Shared Volume
5. OBJECTIVE 2 — Implment Memory or RAM-backed emptyDir Volume
6. OBJECTIVE 3 — Setting emptyDir Volume sizeLimit and Ephemeral Storage Resource Requests and Limits
7. Conclusion

Introduction

Persistent Storage Volumes with ‘hostPath’ post last week broke ground in the Mastering Kubernetes One Task at a Time series by showing how to define volumes in a pod, configure volumeMounts inside containers, and use the hostPath type to meet persistent storage needs using the cluster node filesystem. Having acquired the skills, we’re ready to showcase many other types of Volumes and apply them to meet various needs in everyday Kubernetes applications. Please follow the link below to read and try out the hands-on examples.

This post lines up the next logical Volumes type we want to cover, called emptyDir, an important implementation of Kubernetes that exists for a few special reasons outlined in the next section with associated references. This Volumes type is ephemeral, which means its lifetime is limited to a Pod’s lifetime, and all containers in the Pod can read and write the same files in the emptyDir Volume. When a pod ceases to exist, Kubernetes destroys all ephemeral volumes.

Here I’ll cover the following 3 demonstrations or objectives that I think are must-know initial skills to take full advantage of emptyDir volumes.

1. Enable inter-container communication in the same Pod using a shared Volume where data should not persist outside the Pod’s existence.
2. Implement a memory or RAM-backed filesystem using emptyDir because data is not permitted to be written on the disk.
3. Show how to restrict emptyDir volume size limit as well as how to specify ephemeral storage resource requests size and resource limit size.

Kubernetes Documentation References for emptyDir

The following links from the kubernetes.io documentation site provide sufficient ground to understand the purpose, use cases, and implementation tips and examples. Next, refer to my summary below for a quick grasp. As always, it’s best to read up first and then try the hands-on examples to comprehend and internalize the concepts and the constructs fully.

1. Kubernetes Volumes emptyDir type — https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
2. Types of Ephemeral Volumes —https://kubernetes.io/docs/concepts/storage/ephemeral-volumes/#types-of-ephemeral-volumes
3. Volumes Local / Temporary Directory API Spec — https://kubernetes.io/docs/reference/kubernetes-api/config-and-storage-resources/volume/#local-temporary-directory

In practice, emptyDir Volumes types are commonly used for—

• Scratch space requirements, such as for a disk-based merge sort
• Checkpointing a long computation for recovery from crashes by creating log files
• Holding files that a content-manager container (a sidecar container pattern) fetches while a webserver container serves the data

One of my hands-on examples below shows the use of emptyDir.medium: Memory setting, which lets Kubernetes mount a tmpfs (RAM-backed filesystem). The Kubernetes resource secret, usually used to pass sensitive information, such as passwords, can be mounted to Pods as Volumes. The secret volumes are backed by tmpfs (memory) so they are never written to non-volatile storage.

Finally, as resource management best practice, ephemeral storage resource requests, limits, and emptyDir size limit settings are set, similar to CPU and memory resource management. The third objective hands-on example shows how to implement and test these features.

Create and Access KIND Kubernetes Cluster on EC2

It turns out our faithful KIND cluster, as detailed in the blog post The AWS Way — IaC in Action — A Docker and KIND Ready Amazon EC2 Node, is fully able to support all the Volumes-related hands-on activities. So, without further ado, let’s use the code below to fire up the cluster and access it to execute our code, including my demonstration objective by objective.

NOTE: I have a Kubernetes Cluster (1 control plane and 2 worker nodes) with Kubeadm using Terraform and AWS EC2 that can also be used for hands-on exercises as an alternative.

OBJECTIVE 1 — Communicate Between Containers Using emptyDir Shared Volume

The code here illustrates the most common use case of the emptyDir Volumes type, mainly driven by the need for sharing data among the containers of a Pod in compliance with non-persistent requirements, as opposed to persistent type using hostPath type covered in the previous post.

As already stated, emptyDir being ephemeral storage, the Volume in this example provides a way for Containers to communicate during the life of the Pod. Any data stored in the shared Volume is lost if the Pod is deleted and recreated.

OBJECTIVE 2— Implment Memory or RAM-backed emptyDir Volume

In general, emptyDir volumes’ actual medium depends on whatever disk medium the node filesystem uses, such as SSD, network storage, etc. By setting the emptyDir.medium field to "Memory", a tmpfs directory is mounted to the filesystem with actual storage being backed by RAM.

The code example below shows a concrete example, including the listing showing the tmpfs directory and associated metadata with the df command.

OBJECTIVE 3— Setting emptyDir Volume sizeLimit and Ephemeral Storage Resource Requests and Limits

Setting requests and limits for local ephemeral storage documentation page explains the theory really well behind the code example below.

I’ve provided the additional commands to perform the stress test by exceeding the sizeLimit specification and the timely Kubernetes cluster response of the limit violation.

Conclusion

In this Ephemeral Storage Volumes with ‘emptyDir’ task, as part of the Mastering Kubernetes One Task at a Time series, I’ve provided three common use case examples of emptyDir Volumes type.

All of these examples can be practiced on any available Kubernetes cluster you have access to. I’ve tested the code both on the KIND cluster as well as the Kubeadm-based cluster mentioned at the top of the post, and they worked perfectly.

The code demonstrating the three objectives above can be found at https://github.com/jdluther2020/jdluther-kubernetes-io-tasks/tree/main/ephemeral-volume-with-emptydir. As I’ve done with the rest of the series tasks, this repo comes pre-installed on the KIND cluster node.

Thank you for following the series and staying in touch.

Please check out the next post in the series — DaemonSet Demonstration.

If you benefited from reading the post, please 👏 a few times before parting, and help others by sharing it; I highly appreciate that!

Please follow to stay in touch, track, and be the first to get notified of all future writings on AWS Cloud, Containers, Kubernetes, and Machine Learning. Also, check all my stories on The AWS Way publication.