Setting Up a Kubernetes 1.24 Cluster using kubeadm on 3 Ubuntu Severs

Step-by-step guide to install a Kubernetes cluster with kubeadm

To create a Kubernetes cluster with kubeadm I use 3 different virtual machines, 1 as Control Plane and the other 2 as Worker Nodes.

Using kubeadm, you can create a minimum viable Kubernetes cluster that conforms to best practices. In fact, you can use kubeadm to set up a cluster that will pass the Kubernetes Conformance tests. kubeadm also supports other cluster lifecycle functions, such as bootstrap tokens and cluster upgrades.

The kubeadm tool is good if you need:

A simple way for you to try out Kubernetes, possibly for the first time.

A way for existing users to automate setting up a cluster and test their application.

A building block in other ecosystems and/or installer tools with a larger scope.

You can install and use kubeadm on various machines: your laptop, a set of cloud servers, a Raspberry Pi, and more. Whether you're deploying into the cloud or on-premises, you can integrate kubeadm into provisioning systems such as Ansible or Terraform. [kubernetes.io]

First of all, you need 3 virtual machines. I am using 3 instances of a virtual machine with 2 virtual CPUs and 4GiB Memory and Ubuntu 20.04 as operation system.

Setting Up Hostnames

After starting the virtual machines and switching over to the terminals, you assign new hostnames to all machines. For the Control Plane machine you can assign a name like k8s-control-node, and the Worker Nodes can be named k8s-worker-node-n.

$ sudo hostnamectl set-hostname k8s-control-node
$ sudo hostnamectl set-hostname k8s-worker-node-1
$ sudo hostnamectl set-hostname k8s-worker-node-2

This is very convenient when defining and identifying the machines in the host files later on.

Installing kubeadm on all 3 Machines

The main goal of this section is to install kubeadm, kubelet, and kubectl on all machines. For this, you need to go through a series of steps.

kubeadm is the command to bootstrap the cluster. kubelet is the component that runs on all of the machines in your cluster and does things like starting pods and containers. And kubectl is the command line util to talk to your cluster.

First, you need to set up the host files and add entries of the private IP addresses of all machines.

$ sudo nano /etc/hosts

In the host files you need to append the hostnames of the servers to the corresponding IP addresses, so the machines can talk to each other using the hostnames which you assigned a step earlier. This could look like this:

172.31.117.239 k8s-control-node
172.31.125.235 k8s-worker-node-1
172.31.118.191 k8s-worker-node-2

The next step is to install containerd, the container runtime for the Kubernetes cluster to run containers in Pods. For this, you need to install some kernel modules first:

$ cat << EOF | sudo tee /etc/modules-load.d/containerd.conf 
overlay 
br_netfilter 
EOF

To use them without a restart, you run the following command to set up the kernel modules immediately:

$ sudo modprobe overlay
$ sudo modprobe br_netfilter

After that, you need to set some system level settings, so that Kubernetes networking works as expected:

$ cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

Then you need to run the following command, so that the previous steps take effect immediately:

$ sudo sysctl --system

After setting up these kernel modules and system settings, you can install and configure containerd:

$ sudo apt-get update && sudo apt-get install -y containerd 

Now, you can create the configuration file for containerd and configure it:

$ sudo mkdir -p /etc/containerd 
$ sudo containerd config default | sudo tee /etc/containerd/config.toml 

To be sure the new configuration file is used, you need to restart containerd:

$ sudo systemctl restart containerd

After this you need to disable SWAP:

$ sudo swapoff -a

To check if there is no entry in this file, you can run this:

$ sudo cat /etc/fstab

// returns 
// LABEL=cloudimg-rootfs / ext4 defaults,discard 0 0

Before the servers are ready to install the Kubernetes packages, you need to install some required packages first — Transport-https and Curl:

$ sudo apt-get update && sudo apt-get install -y apt-transport-https curl

After that, you need to add the gpg key for the Kubernetes package repository:

$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

Then, set up the Kubernetes repository entry and update the packages to get the latest releases:

$ cat << EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
$ sudo apt-get update

And now you can install the Kubernetes packages: kubelet, kubeadm and kubectl. You need to select the same version for all these packages. kubeadm will not install the packages of kubelet or kubectl for you, so you need to ensure their versions match:

$ sudo apt-get install -y kubelet=1.24.0-00 kubeadm=1.24.0-00 kubectl=1.24.0-00

To disable the automated updates for these packages, you need to run the following apt-mark command:

$ sudo apt-mark hold kubelet kubeadm kubectl

Important: You need to run all these steps on the Control Plane and both Worker Nodes.

Control Plane Node Settings

On the Control Plane server you need to initialize the Kubernetes cluster with the same version that you used for kubelet, kubeadm, and kubectl. This step takes a little bit of time:

$ sudo kubeadm init --pod-network-cidr 192.168.0.0/16 --kubernetes-version 1.24.0

After this, you need to configure the kube-config. There are some hints in the kubeadm init command output that could be helpful:

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

After this, you can run the following command to see that you have one node for the Kubernetes cluster initialized, but in a NotReady state:

kubectl get nodes

To get a Ready state, you need to setup the Kubernetes networking with Calico.

Calico is an open source networking and network security solution for containers, virtual machines, and native host-based workloads. Calico supports a broad range of platforms including Kubernetes, OpenShift, Mirantis Kubernetes Engine (MKE), OpenStack, and bare metal services.

To apply Calico as network solution, you need to run the following command:

$ kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

Now you need to join the Worker Nodes to the cluster. They have a Kubernetes installation on it, but are not part of the cluster, yet. As shown above, only the Control Plane server is part of the cluster.

First, you need to generate a kubeadm token in the Control Plane node. With the command that is shown as output of the following command you can add Worker Nodes to the cluster.

$ kubeadm token create --print-join-command

An example for the output could be this:

$ kubeadm join 172.31.117.239:6443 --token gfgp1z.6y1o6cvzijlke1aa --discovery-token-ca-cert-hash sha256:383908a3cd658ded8d000650a3b49d3b7b1091e284f1965854316f2b685a2966

Worker Node Settings

Now, you need to run the output command from the previous step on the Worker Nodes as administrator. This can look like this:

$ sudo kubeadm join 172.31.117.239:6443 --token gfgp1z.6y1o6cvzijlke1aa --discovery-token-ca-cert-hash sha256:383908a3cd658ded8d000650a3b49d3b7b1091e284f1965854316f2b685a2966

That is is all. You can run the following command on the Control Plane node to see, if all Worker Nodes are joined.

$ kubectl get nodes

The output should be something like this:

k8s-control-node  Ready  control-plane,master 8m21s v1.24.0
k8s-worker-node-1 Ready  <none>               2m13s v1.24.0
k8s-worker-node-2 Ready  <none>               38s   v1.24.0

Conclusion

This was a quick Hands-On how to build up a Kubernetes cluster. kubeadm is a very helpful tool to make this possible in a simple way.

Follow me on Medium, or Twitter, or subscribe here on Medium to read more about DevOps, Agile & Development Principles, Angular and other useful stuff. Happy Coding! :)

Resources

Learn More