Based on k3d.io
- docker
- kubectl
curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash- Create a volume folder
mkdir -p "${HOME}/kubernetes/volume"- Create a k3d Cluster
k3d cluster create CKAD \
--servers 3 \
--agents 3 \
--k3s-node-label topology.kubernetes.io/zone=zone-a@agent:0 \
--k3s-node-label topology.kubernetes.io/zone=zone-a@agent:1 \
--k3s-node-label topology.kubernetes.io/zone=zone-a@agent:2 \
--api-port 6550 \
-p "8000:80@loadbalancer" \
--volume "${HOME}/kubernetes/volume:/data@agent:*"The above command creates a Kubernetes cluster using k3d with 3 server and 3 agent nodes, assigns zone labels to the agents, configures API access, and sets up port forwarding and a volume mount for shared storage across all agent nodes.
cluster create CKAD
This part creates a new k3d cluster with the name CKAD.
--servers 3
This option specifies that the cluster should have 3 server nodes (control-plane). In Kubernetes, control-planes nodes are responsible for managing the cluster and running key services such as the API server, scheduler, and controller manager.
--agents 3
This specifies that the cluster should have 3 agent nodes (also known as worker nodes). These are responsible for running the application workloads (Pods).
--k3s-node-label topology.kubernetes.io/zone=zone-a@agent:*
This sets a Kubernetes node label on each agent nodes. The label topology.kubernetes.io/zone=zone-a helps identify that this agent is part of "zone-a". Labels can be used later for things like scheduling workloads to specific zones for availability.
--api-port 6550
--api-port $port specifies that the Kubernetes API should be accessible via port 6550 on the host machine. If you want to interact with the cluster using kubectl or any other Kubernetes management tool, this is the port to use.
-p "8000:80@loadbalancer"
This -p or --port sets up port forwarding from the host machine to the cluster. In this case, it's mapping port 80 on the host machine to port 80 on the load balancer node inside the k3d cluster. This is useful for exposing applications running in the cluster to the outside world.
--volume "${HOME}/kubernetes/volume:/data@agent:*"
This mounts a volume from your local machine (${HOME}/kubernetes/volume) to the /data directory on all agent nodes (@agent:*). The * indicates that this volume should be mounted on all agent nodes. This is useful for sharing data between your host and the Kubernetes cluster's worker nodes.
In lighter environments, modify the k3d command to have 1 control plane (server) and 3 agents (worker nodes), and configure your cluster to schedule deployments only on the agent nodes:
k3d cluster create k8s \
--servers 1 \
--agents 2 \
--k3s-node-label topology.kubernetes.io/zone=zone-a@agent:0 \
--k3s-node-label topology.kubernetes.io/zone=zone-a@agent:1 \
--api-port 6550 \
-p "8000:80@loadbalancer" \
--volume "${HOME}/kubernetes/volume:/data@agent:*"- Taint the control plane node to prevent it from running Pods
kubectl taint nodes <control-plane-node-name> node-role.kubernetes.io/control-plane:NoSchedule
-
Example
kubectl get nodes NAME STATUS ROLES AGE VERSION k3d-k8s-agent-0 Ready <none> 41s v1.29.6+k3s2 k3d-k8s-agent-1 Ready <none> 40s v1.29.6+k3s2 k3d-k8s-agent-2 Ready <none> 41s v1.29.6+k3s2 k3d-k8s-server-0 Ready control-plane,master 48s v1.29.6+k3s2
kubectl taint nodes k3d-k8s-server-0 node-role.kubernetes.io/control-plane:NoSchedule
node/k3d-k8s-server-0 tainted
Additional example using k3d Args
k3d cluster create k8s \ --image rancher/k3s:v1.31.2-k3s1-amd64 \ --port 8080:80@loadbalancer \ --port 8443:443@loadbalancer \ --servers 1 \ --agents 3 \ --k3s-node-label topology.kubernetes.io/zone=zone-a@agent:0 \ --k3s-node-label topology.kubernetes.io/zone=zone-b@agent:1 \ --k3s-node-label topology.kubernetes.io/zone=zone-c@agent:2 \ --api-port 6543 \ --volume "${KUBEVOLUME}:/mnt/data@agent:*" \ --verboseK3s Images
https://hub.docker.com/r/rancher/k3s/tags