Local Docker registry running in Kubernetes / k3s

README.md

Install a docker registry in a k3s kubernetes cluster

k3s doesn't have a builtin plugin for a local registry, so this is how to install the docker registry. The registry will be exposed via the default traefik ingress (on port 80 and 443).

Assumptions

• resolvectl query registry.localhost returns 127.0.0.1 - on most Linux distros it does. If not, enable and configure systemd-resolved
• K3s comes with traefik ingress controller by default, so the ingress for the registry service is configured for that.

Install the registry in k3s

kubectl create namespace docker-registry
kubectl apply -f docker-registry.yaml -n docker-registry

Use from the host

We need to make sure both docker/podman are able to push to the local unsecure registry and to make k3s be ablet to use the registry hosted inside of it.

docker build -t registry.localhost/test:latest .
docker push registry.localhost/test:latest

Note

configure http://registry.localhost as insecure registry: https://docs.docker.com/registry/insecure/

podman build -t registry.localhost/test:latest .
podman push --tls-verify=false registry.localhost/test:latest

Note

configure http://registry.localhost as insecure registry: https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md

For setting up as insecure registry in k3s see: https://docs.k3s.io/installation/private-registry and the /etc/rancher/k3s/registries.yaml file in this gist (last file).

docker-registry.yaml

# Local Docker registry running in Kubernetes - for k3s
#
# kubectl create namespace docker-registry
# kubectl apply -f docker-registry.yaml -n docker-registry
#
# docker build -t registry.localhost/test:latest .
# docker push registry.localhost/test:latest
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: docker-registry-ingress
spec:
  ingressClassName: traefik
  rules:
  - host: registry.localhost
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: docker-registry-service
            port:
              number: 5000

---
apiVersion: v1
kind: Service
metadata:
  name: docker-registry-service
  labels:
    app: docker-registry
spec:
  selector:
    app: docker-registry
  ports:
    - protocol: TCP
      port: 5000

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: docker-registry-pvc
  labels:
    app: docker-registry
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: local-path
  resources:
    requests:
      storage: 10Gi

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: docker-registry
  labels:
    app: docker-registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: docker-registry
  template:
    metadata:
      labels:
        app: docker-registry
    spec:
      containers:
      - name: docker-registry
        image: registry
        ports:
        - containerPort: 5000
          protocol: TCP
        volumeMounts:
        - name: storage
          mountPath: /var/lib/registry
        env:
        - name: REGISTRY_HTTP_ADDR
          value: :5000
        - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
          value: /var/lib/registry
      volumes:
      - name: storage
        persistentVolumeClaim:
          claimName: docker-registry-pvc

registries.yaml

# /etc/rancher/k3s/registries.yaml
#
# No need for TLS/HTTPS for the local registry
# https://docs.k3s.io/installation/private-registry
#
mirrors:
  registry.localhost:
    endpoint:
      - "http://registry.localhost:80"

of-course another alternative is to just use http://ttl.sh/ but that will create both outbound and inbound traffic over the internet for no reason.