Karpenter: A Powerful Tool for Managing Kubernetes Clusters on AWS | By M. Sharma

Introduction

Karpenter is an open-source Kubernetes-native autoscaling solution that runs on top of Kubernetes clusters and automates the management of nodes on cloud infrastructure providers like Amazon EC2. It was developed by the team at Upbound, a cloud-native computing company. Karpenter is designed to solve the problem of node management in Kubernetes clusters. In a Kubernetes cluster, nodes are the worker machines that run containerized applications. Scaling the number of nodes up and down based on demand is a critical function of Kubernetes, but it can be challenging to manage manually, especially when running Kubernetes clusters on cloud infrastructure like EC2. Karpenter automates the process of scaling up and down nodes in Kubernetes clusters by implementing custom resource definitions (CRDs) in Kubernetes that describe the desired state of the cluster. Karpenter monitors the workload and resource utilization of the cluster and automatically provisions and deprovisions EC2 instances as needed to maintain the desired state. Karpenter supports multiple EC2 instance types and allows users to define custom instance types with specific configurations. It also provides advanced features like node pre-warming, which proactively provisions new nodes before they are needed, reducing the time it takes to scale up.

Architecture and components of Karpenter

Karpenter is a Kubernetes-native autoscaling solution that consists of several components that work together to automate the management of nodes in Kubernetes clusters.

+-------------------------------------+
|                Kubernetes Cluster   |
+-------------------------------------+
                 |
+-------------------------------------+
|                Karpenter API Server |
+-------------------------------------+
|             |                       |
|             | watches               |
|             v                       |
| +---------------------------------+ |
| |         Karpenter Controller    | |
| +---------------------------------+ |
|             |                       |
|             | prov/deprov EC2       |
|             v                       |
| +---------------------------------+ |
| |        Karpenter Scheduler      | |
| +---------------------------------+ |
|             |                       |
|             | validate admission    |
|             v                       |
| +---------------------------------+ |
| |         Karpenter Webhook       | |
| +---------------------------------+ |
|                                     |
|             Metrics and Logging     |
|                                     |
+-------------------------------------+

In this diagram, the Kubernetes cluster is at the centre, and Karpenter is deployed as a set of components that run on top of the cluster. The Karpenter API Server is responsible for defining the desired state of the cluster using custom resource definitions (CRDs). The Karpenter Controller watches for changes to the Kubernetes API and makes decisions about when to scale up or down based on the current demand and available resources. The Karpenter Scheduler is responsible for selecting the best EC2 instance type for the workload, and the Karpenter Webhook validates the admission of new Kubernetes objects. Metrics and logging are provided by Karpenter to monitor the health and performance of the cluster.

Here are the main components of Karpenter:

Karpenter Controller: This is the core component of Karpenter that manages the node lifecycle. It watches for changes to the Kubernetes API and makes decisions about when to scale up or down based on the current demand and available resources. The Karpenter Controller is responsible for provisioning and de-provisioning EC2 instances as needed to maintain the desired state of the cluster.
Karpenter API Server: This component provides an API that allows users to define the desired state of the cluster using custom resource definitions (CRDs). The Karpenter API Server also exposes metrics that can be used to monitor the health and performance of the cluster.
Karpenter Scheduler: This component is responsible for selecting the best EC2 instance type for the workload based on factors like cost, availability, and performance. The Karpenter Scheduler uses a pluggable architecture, which allows users to define custom scheduling strategies that are tailored to their specific use case.
Karpenter Webhook: This component is responsible for validating the admission of new Kubernetes objects that are created or updated by users. It ensures that these objects are in the correct format and complies with the policies defined by Karpenter.
Metrics and Logging: Karpenter also provides built-in metrics and logging that allow users to monitor the health and performance of the cluster. Metrics are exposed through the Kubernetes API, and logging is stored in a configurable location.

Overall, Karpenter is designed to be highly scalable and configurable, allowing users to tailor the solution to their specific needs. The pluggable architecture and custom resource definitions make it easy to extend and integrate with existing Kubernetes clusters, while the built-in metrics and logging provide visibility into the cluster’s health and performance.

How Karpenter optimizes cluster resource utilization and reduces costs

Karpenter optimizes cluster resource utilization and reduces costs in several ways:

Right-sizing instances: Karpenter continuously monitors the resource utilization of the cluster and selects the appropriate EC2 instance types for the workload. This ensures that instances are right-sized for the workload, which minimizes waste and reduces costs.
Automated scaling: Karpenter automates the scaling of EC2 instances based on the demand of the workload. This ensures that the cluster is always appropriately sized, which maximizes resource utilization and reduces costs.
Instance lifecycle management: Karpenter manages the lifecycle of EC2 instances by automatically provisioning and de-provisioning instances as needed. This eliminates the need for manual intervention and ensures that the cluster is always optimized for cost and performance.
Spot instance optimization: Karpenter supports the use of AWS Spot Instances, which can provide significant cost savings compared to On-Demand Instances. Karpenter automatically manages the lifecycle of Spot Instances and ensures that they are used effectively to minimize costs.
Smart scheduling: Karpenter uses a pluggable scheduling strategy that selects the best instance type for the workload. This ensures that the cluster is optimized for performance and cost, and reduces waste by avoiding over-provisioning.

Overall, Karpenter’s optimization strategies ensure that Kubernetes clusters running on EC2 instances are always optimized for cost and performance, which can significantly reduce the overall cost of running a cluster.

Step-by-step guide for deploying a Karpenter-managed Kubernetes

Prerequisites:

An AWS account with sufficient permissions to create EKS clusters
The awscli command-line interface installed
The eksctl command-line utility installed
The kubectl command-line utility installed
A VPC and subnet in the region where you want to create the EKS cluster

Step 1: Install the awscli command-line interface

To install the awscli command-line interface, follow the instructions for your operating system provided in the official AWS documentation:

https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html

Step 2: Install the eksctl command-line utility

To install the eksctl command-line utility, follow the instructions for your operating system provided in the official EKS documentation:

https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html

Step 3: Install the kubectl command-line utility

To install the kubectl command-line utility, follow the instructions for your operating system provided in the official Kubernetes documentation:

https://kubernetes.io/docs/tasks/tools/

Step 4: Create an EKS cluster with eksctl

To create an EKS cluster with eksctl, create a YAML file that defines your cluster configuration, for example:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: my-eks-cluster
  region: us-west-2
vpc:
  cidr: 10.0.0.0/16
  subnets:
    public:
      cidr: 10.0.0.0/24
      availabilityZones: ["us-west-2a"]
    private:
      cidr: 10.0.1.0/24
      availabilityZones: ["us-west-2a"]
nodeGroups:
  - name: karpenter-nodes
    instanceType: t2.medium
    desiredCapacity: 2
    ssh:
      allow: true

In this example, the EKS cluster is named my-eks-cluster and is created in the us-west-2 region. The VPC CIDR block is 10.0.0.0/16, and there are two subnets: a public subnet with CIDR 10.0.0.0/24 in availability zone us-west-2a, and a private subnet with CIDR 10.0.1.0/24 in availability zone us-west-2a. The cluster has a node group named karpenter-nodes, which uses t2.medium instances and has a desired capacity of 2.

To create the cluster, run the following command:

eksctl create cluster -f my-eks-cluster.yaml

This command will create an EKS cluster with the configuration defined in the YAML file.

Step 5: Verify the EKS cluster

To verify that the EKS cluster was created successfully, run the following command:

kubectl get nodes

This command will show the list of nodes in the cluster.

Step6: Cluster & Karpenter configuration

Create a basic cluster with eksctl. The following cluster configuration will:

Use CloudFormation to set up the infrastructure needed by the EKS cluster.
Create a Kubernetes service account and AWS IAM Role, and associate them using IRSA to let Karpenter launch instances.
Add the Karpenter node role to the aws-auth configmap to allow nodes to connect.
Use AWS EKS managed node groups for the kube-system and karpenter namespaces. Uncomment fargateProfiles settings (and comment out managedNodeGroups settings) to use Fargate for both namespaces instead.
Set KARPENTER_IAM_ROLE_ARN variables.
Create a role to allow spot instances.
Run helm to install karpenter

export CLUSTER_NAME="${USER}-karpenter-demo"
export AWS_DEFAULT_REGION="us-west-2"
export AWS_ACCOUNT_ID="$(aws sts get-caller-identity --query Account --output text)"
export TEMPOUT=$(mktemp)
export KARPENTER_VERSION=v0.26.1


curl -fsSL https://karpenter.sh/"${KARPENTER_VERSION}"/getting-started/getting-started-with-eksctl/cloudformation.yaml  > $TEMPOUT \
&& aws cloudformation deploy \
  --stack-name "Karpenter-${CLUSTER_NAME}" \
  --template-file "${TEMPOUT}" \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides "ClusterName=${CLUSTER_NAME}"

eksctl create cluster -f - <<EOF
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: ${CLUSTER_NAME}
  region: ${AWS_DEFAULT_REGION}
  version: "1.24"
  tags:
    karpenter.sh/discovery: ${CLUSTER_NAME}

iam:
  withOIDC: true
  serviceAccounts:
  - metadata:
      name: karpenter
      namespace: karpenter
    roleName: ${CLUSTER_NAME}-karpenter
    attachPolicyARNs:
    - arn:aws:iam::${AWS_ACCOUNT_ID}:policy/KarpenterControllerPolicy-${CLUSTER_NAME}
    roleOnly: true

iamIdentityMappings:
- arn: "arn:aws:iam::${AWS_ACCOUNT_ID}:role/KarpenterNodeRole-${CLUSTER_NAME}"
  username: system:node:{{EC2PrivateDNSName}}
  groups:
  - system:bootstrappers
  - system:nodes

managedNodeGroups:
- instanceType: m5.large
  amiFamily: AmazonLinux2
  name: ${CLUSTER_NAME}-ng
  desiredCapacity: 2
  minSize: 1
  maxSize: 10

## Optionally run on fargate
# fargateProfiles:
# - name: karpenter
#  selectors:
#  - namespace: karpenter
EOF

export CLUSTER_ENDPOINT="$(aws eks describe-cluster --name ${CLUSTER_NAME} --query "cluster.endpoint" --output text)"
export KARPENTER_IAM_ROLE_ARN="arn:aws:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-karpenter"

echo $CLUSTER_ENDPOINT $KARPENTER_IAM_ROLE_ARN

aws iam create-service-linked-role --aws-service-name spot.amazonaws.com || true
# If the role has already been successfully created, you will see:
# An error occurred (InvalidInput) when calling the CreateServiceLinkedRole operation: Service role name AWSServiceRoleForEC2Spot has been taken in this account, please try a different suffix.

docker logout public.ecr.aws
helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter --version ${KARPENTER_VERSION} --namespace karpenter --create-namespace \
  --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \
  --set settings.aws.clusterName=${CLUSTER_NAME} \
  --set settings.aws.defaultInstanceProfile=KarpenterNodeInstanceProfile-${CLUSTER_NAME} \
  --set settings.aws.interruptionQueueName=${CLUSTER_NAME} \
  --wait

Step7: Create Karpenter Provisioner

A Karpenter Provisioner is a custom resource definition (CRD) in Karpenter that allows users to specify how the cluster should provision nodes to satisfy unfulfilled demand. The provisioner CRD specifies the target capacity of the node group, the instance type and launch template for the nodes, and the strategy for provisioning and scaling the nodes.

Once a provisioner is created, Karpenter continuously monitors the cluster for unscheduled pods and uses the provisioner’s specifications to launch new nodes to satisfy demand. When demand decreases, Karpenter will scale down the node group by terminating unused nodes according to the specified strategy. This helps to ensure efficient resource utilization and cost savings.

Learn more about Karpenter provisioner CRD here: https://karpenter.sh/v0.26.1/concepts/provisioners/

# This example will use spot instance type for all
# provisioned instances

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot"]
  providerRef:
    name: my-provider
---
apiVersion: karpenter.k8s.aws/v1alpha1
kind: AWSNodeTemplate
metadata:
  name: my-provider
spec:
  subnetSelector:
    karpenter.sh/discovery: "${CLUSTER_NAME}" # replace with your cluster name
  securityGroupSelector:
    karpenter.sh/discovery: "${CLUSTER_NAME}" # replace with your cluster name

Step8: Sample Deployment

Here’s a sample code with Karpenter watcher that monitors the Kubernetes cluster for unfulfilled pod requests and launches nodes to satisfy the demand:

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: my-provisioner
spec:
  targetCapacity: 10
  strategy:
    type: spot
  constraints:
    maxAge: 1h
  template:
    spec:
      iamInstanceProfile:
        name: my-instance-profile
      instanceType: t3.small
      spotPrice: "0.01"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: karpenter-controller
  namespace: karpenter
spec:
  replicas: 1
  selector:
    matchLabels:
      app: karpenter
  template:
    metadata:
      labels:
        app: karpenter
    spec:
      containers:
        - name: karpenter
          image: <your-image>:latest
          command: ["/<your-command>"]
          args: ["start", "--log-level=debug", "--watch"]
          env:
            - name: AWS_REGION
              value: us-west-2
            - name: AWS_NODE_TAINTS
              value: "special=true:NoSchedule"
          volumeMounts:
            - name: aws-credentials
              mountPath: /root/.aws
            - name: kubeconfig
              mountPath: /root/.kube/config
              subPath: config
      volumes:
        - name: aws-credentials
          secret:
            secretName: aws-credentials
        - name: kubeconfig
          configMap:
            name: kubeconfig
            items:
              - key: config
                path: config

In this example, we define a Provisioner called “my-provisioner” that will launch nodes of type “t3.small” with a maximum capacity of 10 instances and a spot price of $0.01. The provisioner will use the “spot” strategy, which will launch spot instances to satisfy demand. Additionally, the provisioner specifies a constraint of maximum node age of 1 hour to ensure that nodes are not running for too long.

The Deployment section defines the Karpenter controller that will watch the cluster for pod demand and launch nodes as needed. We specify the latest version of the container image, set the log level to debug, and enable the watch mode. We also set the AWS_REGION environment variable and mount the kubeconfig and AWS credentials secret to the container.

Once you have the Provisioner and Karpenter controller set up with the watcher, Karpenter will continuously monitor the cluster for unfulfilled pod requests. When it detects that there are not enough resources to satisfy the demand, it will launch new nodes automatically to satisfy the demand. This means that as your workload grows and demands more resources, Karpenter will automatically scale your cluster up to meet the demand. Similarly, when demand decreases, Karpenter will scale down the cluster to save resources and reduce costs. You can also monitor the Karpenter logs to see how the Provisioner is scaling the cluster up or down in response to demand changes. Additionally, Karpenter provides a set of metrics that can be used to track node utilization and capacity, allowing you to monitor the performance of your cluster over time.

Best practices for Karpente

Here are some best practices for using Karpenter on AWS EKS:

Use managed node groups: Managed node groups are recommended for EKS clusters as they are fully managed by AWS, providing automated updates, scaling, and health checks.
Use the correct instance types: Choosing the correct instance types for your workload can improve performance and reduce costs. You can use Karpenter’s NodeGroupOverrides configuration to specify the instance types that Karpenter should use when launching nodes.
Use spot instances: Spot instances can significantly reduce costs while providing the same performance as on-demand instances. Karpenter supports launching nodes on spot instances using the SpotInstanceConfig configuration.
Use the correct node labels: Karpenter uses node labels to decide which nodes to use when scheduling pods. You should label nodes based on their characteristics, such as CPU and memory capacity, to ensure that the right nodes are used for each workload.
Use node taints and tolerations: Node taints and tolerations can be used to specify which pods should be scheduled on which nodes. This can be used to ensure that nodes are used efficiently and that the right pods are scheduled on the right nodes.
Use Kubernetes Pod Priority and Preemption: Pod Priority and Preemption can be used to prioritize certain pods over others and ensure that critical workloads are always running. This can be used in combination with Karpenter to ensure that the right nodes are always available for critical workloads.
Use Kubernetes Pod Disruption Budgets: Pod Disruption Budgets can be used to ensure that critical workloads are not disrupted when nodes are removed from the cluster. This can be used in combination with Karpenter to ensure that nodes are scaled down in a controlled manner without disrupting critical workloads.
Use auto-scaling: Auto-scaling can be used to automatically scale the number of nodes up or down based on the demand. This can be used in combination with Karpenter to ensure that the right number of nodes are always available for your workloads.

By following these best practices, you can ensure that your cluster is configured for optimal performance and scalability when using Karpenter on AWS EKS.

Conclusion

In conclusion, Karpenter is a valuable tool for managing Kubernetes clusters on Amazon EC2 or EKS. It helps optimize resource utilization and reduce costs by automatically scaling nodes up or down based on demand. Deploying a Karpenter-managed cluster on Amazon EC2 or EKS involves several steps, including setting up infrastructure, creating Kubernetes service accounts and IAM roles, and installing Karpenter via Helm. Once the cluster is set up, it is important to follow best practices to ensure optimal performance and scalability, such as properly configuring node pools and setting resource limits. With proper configuration and management, Karpenter can help organizations efficiently run Kubernetes workloads on AWS.