Mastering Kubernetes One Task at a Time — Know Thy Nodes!
6 key objectives with code examples to master and maneuver your interaction with Kubernetes cluster nodes like a Pro.
Table of Contents
- Introduction
- Kubernetes Nodes Resource Reference
- Create and Access KIND Kubernetes Cluster on EC2
- Objective 1 — Check Cluster Nodes
- Objective 2 — Examine a Node Manifest
- Objective 3 — Describe a Node’s Status Details
- Objective 4 — Run Commands and check Components inside a Node
- Objective 5 — Schedule Pods on a Specific Node
- Objective 6 — Schedule Pods on all Nodes (Worker + Control Plane)
- Conclusion
Introduction
Pods, Deployments, and ReplicaSets, the first of the four Kubernetes built-in Workload Resources groups covered in my last post, teach key skills to build and manage stateless application workloads on a cluster. Please see the link below if you haven’t checked it out yet or want to revisit it.
Now is an excellent time to learn some theory on Kubernetes nodes and master the key hands-on skills to be able to interact with node objects, understand how pods are scheduled on nodes, and even go inside a node to issue commands and observe the critical components running inside each node, up, close and personal. These skills play an essential role in daily cluster management, operations, and application debugging, and we must master these basics.
Kubernetes Nodes Resource Reference
Most of the concepts and codes I will cover here are well covered in the following three Kubernetes.io official documentation pages.
- Kubernetes and Node Components (Architecture Overview)— https://kubernetes.io/docs/concepts/overview/components/
- Node Components and Details — https://kubernetes.io/docs/concepts/architecture/nodes/
- Node Object Manifest and Description Lookup for Troubleshooting Clusters — https://kubernetes.io/docs/tasks/debug/debug-cluster/
When I walk through the working code demonstration, and also in the future, it’s good to have these pages handy for reference. If not in the beginning, over time, as the familiarity with Kubernetes grows, these concepts will become entrenched in mind and make a lot of sense at a glance.
At this point, we’re ready to roll with the code.
Create and Access KIND Kubernetes Cluster on EC2
Like the previous posts and their associated hands-on code examples, we will practice on the KIND cluster explained in the blog post The AWS Way — IaC in Action — A Docker and KIND Ready Amazon EC2 Node. Use the code below to fire up the cluster and access it to execute our code.
Objective 1 — Check Cluster Nodes
The introduction to a brand new cluster for a Kubernetes operator always beings with the commands shown below by checking out the nodes., inspecting the node objects, seeing their ready or not-ready status, and most importantly, finding out which ones represent the Control Plane nodes vs. Worker nodes.
By looking at the -o wide output, we also discern the IPs, OS, and container run times running inside the node. The — show-labels is also a handy option frequently used to check the labels attached, as one of the examples in a coming section will illustrate.
Objective 2 — Examine a Node Manifest
The commands in this section and the following show how to get detailed information about each node. To a large extent, both contain overlapping information, and both are frequently used, describe more so than get, as starting points for Troubleshooting Clusters.
The commands below help obtain and understand the constitution of a node manifest that the kubelet agent uses to create and register a node object to the cluster. Not that you’ll ever need to create a node resource yourself, but the information is helpful for debugging and using it to perform application-specific decisions and actions.
And I can’t emphasize enough the usefulness of kubectl explain command to get deep-dive information about any API resource and its associated resource fields.
Two useful tips while at it:
1. Add the
— recursiveflag to display all of the fields at once, the full spec, without verbose descriptions. (e.g.kubectl explain node — recursive).
2. Use
kubectl api-resourcesfor a complete list of supported resources.
Objective 3 — Describe a Node’s Status Details
As mentioned before, kubectl describe is used more commonly to retrieve detailed information about nodes for debugging and maintenance and for gathering key information to help make application and workload specific decisions, including scheduling. I’d publish a few dedicated posts showing debugging and maintenance use cases in the future because of their constant usefulness.
Suffice it to say, in the daily life of Kubernetes, these commands are used ad nauseam for good and unavoidable reasons.
Objective 4 — Run Commands and check Components inside a Node
Many a time, we need to log in to specific nodes for a variety of reasons, including cluster debugging, maintenance, and resource inspection reasons. There are many use cases to cover, but for the context of this post, it’s good to know these commands and their capabilities, including inspecting individual containers related to different types of resources.
It’s worth pointing out that since we’re dealing with a KIND cluster, I am logging in to the container using docker exec. If it were a server instance, ssh would be the way to log in.
Objective 5 — Schedule Pods on a Specific Node
These commands are used to cover two highly common and frequent use cases:
- Scheduling pods on one or more nodes controlled by labels.
- Scheduling pods on a specific node by the name of the node.
The code below illustrates both.
Objective 6 — Schedule Pods on all Nodes (Worker + Control Plane)
I’ll cover the Taints and Tolerations topic separately, but in the context of nodes, the code below shows how to inspect nodes for any taints that prevent a node from scheduling pods and how to handle that using toleration. It’s good to know the mechanism.
By default, the Control Plane nodes are tainted to avoid scheduling regular workloads, but if, for any reason, we must run a pod on a control plane node, either through a Deployment or a DaemonSet resource, the code below shows how to do it.
Conclusion
The Know Thy Nodes! topic of the ongoing Mastering Kubernetes One Task at a Time series provides six different working examples to help learn how to interact with nodes in a Kubernetes cluster for debugging purposes, gather information to make application design decisions, and perform other cluster activities.
In addition to providing end-to-end working code demonstration for each of the six examples, I’ve also provided a single script file on my GitHub repo at
Thank you for following the series and staying in touch.
Please check out the next post in the series — Persistent Storage Volumes with ‘hostPath’.
If you benefited from reading the post, please 👏 a few times before parting, and help others by sharing it; I highly appreciate that!
Please follow to stay in touch, track, and be the first to get notified of all future writings on AWS Cloud, Containers, Kubernetes, and Machine Learning. Also, check all my stories on The AWS Way publication.
