Summary

The web content provides a step-by-step guide on setting up a private, insecure Docker image registry within a K3s Kubernetes cluster, including necessary configurations and tests to ensure functionality.

Abstract

The article "How To Setup A Private Image Registry On K3s" outlines the process of deploying a private Docker registry within a K3s cluster. It emphasizes the importance of using a private registry for container images when public registries are not suitable. The guide includes the creation of Kubernetes resources such as Ingress, Service, and Deployment manifests to establish the registry. It also covers the necessary domain name configuration and the creation of a registries.yaml file on K3s nodes to recognize the private registry. The tutorial advises restarting K3s services on server and agent nodes to apply changes and verifies the registry's addition through crictl info. For local workstation integration, the article provides instructions for configuring Docker Desktop on macOS. The testing phase involves building a simple Nginx image, pushing it to the private registry, and then removing and re-pulling the image to confirm the registry's functionality. The article concludes with a request for support via a "buy me a coffee" link and references additional resources for further reading.

Opinions

The author notes that the provided manifest will result in the removal of all images when the registry resources are deleted, indicating a need for caution and future improvements (TODO).
The author highlights that the initial setup of the registry is unsecured and suggests further steps to secure it, which are beyond the scope of the tutorial.
The author expresses uncertainty about the necessity of configuring K3s agent nodes but proceeds with the configuration on both server and agent nodes.
The author encourages readers to support the content by purchasing a coffee, indicating a preference for direct reader support over other monetization methods.

How To Setup A Private Image Registry On K3s

Photo by Maksym Kaharlytskyi on Unsplash

It is not always appropriate to push ones own container images to a public registry. This post shows a quick way to create a private image registry inside a K3s Kubernetes cluster.

Please note, that with the following manifest, when the registry resources are being removed from the cluster, all images will be removed as well. There is a TODO in the very last line of the manifest that addresses this.

Also important to note: The registry is unsecured. Further steps have to be taken to secure it with methods like username/password or certificates which is not scope of this tutorial.

Kubernetes Resources

Before this manifest gets applied, the domain name under spec:rules:hostneeds to be changed accordingly.

---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: docker-registry-ingress
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  rules:
  - host: registry.domain.de
    http:
      paths:
      - path: /
        backend:
          serviceName: docker-registry-service
          servicePort: 5000

---
apiVersion: v1
kind: Service
metadata:
  name: docker-registry-service
  labels:
    run: docker-registry
spec:
  selector:
    app: docker-registry
  ports:
    - protocol: TCP
      port: 5000

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: docker-registry
  labels:
    app: docker-registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: docker-registry
  template:
    metadata:
      labels:
        app: docker-registry
    spec:
      containers:
      - name: docker-registry
        image: registry
        ports:
        - containerPort: 5000
          protocol: TCP
        volumeMounts:
        - name: storage
          mountPath: /var/lib/registry
        env:
        - name: REGISTRY_HTTP_ADDR
          value: :5000
        - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
          value: /var/lib/registry
      volumes:
      - name: storage
        emptyDir: {} # TODO -make this more permanent later

The manifest gets applied with:

kubectl apply -f registry.yaml

Settings

On the K3s nodes, the file /etc/rancher/k3s/registries.yaml needs to be created with the following content. The domain name needs to match the one from above manifest.

mirrors:
  "registry.domain.de":
    endpoint:
      - "http://registry.domain.de"

I did this on server and agent nodes. I am not sure if it really has to be done on the agents, I just did.

After that, server and agents need to be restarted.

On the server execute:

systemctl restart k3s

And on the agent node:

systemctl restart k3s-agent

If the changes applied can be checked with:

crictl info

There is a section called registry that should list the newly created private registry.

The local workstation also needs to know about the new registry. I am working on macOS with Docker Desktop. Under “Preferences -> Docker Engine”, the settings had to be extended with the following entry:

{
  ...
  
  "insecure-registries": [
    "registry.domain.de"
  ]
}

Test the Registry

In order to test if an image can be pushed to the new registry, I have built a small image just containing Nginx with a custom HTML-file:

<html>
<head><title>Hello World!</title>
  <style>
    html {
      font-size: 500.0%;
    }
    div {
      text-align: center;
    }
  </style>
</head>
<body>
  <div>Hello World!</div>
</body>
</html>

The Dockerfile for the new image:

FROM nginx:alpine
COPY index.html /usr/share/nginx/html

Build and tag the new image according to your registry domain:

docker build -t registry.domain.de/hello:latest .

Finally the image can be pushed.

docker push registry.domain.de/hello:latest

Hopefully that worked without any issues.

For an additional test, the image can also be removed from the local workstation and then be pulled again from the private registry.

docker rmi registry.domain.de/hello:latest
docker pull registry.domain.de/hello:latest

Feel free to buy me a coffee if you liked this post!