Set up a multi-master (HA) Kubernetes Cluster. Building a HA, multi-master (server) cluster
To set up a high availability (HA) Kubernetes cluster using k3d on two Windows machines with WSL2.
Here are the steps you can follow to achieve this:
-
Install
k3don both Windows machines (WLS2) by following the instructions in the k3d documentation. -
On the first machine (Server 1), create a new k3d cluster with the
--cluster-initflag to initialize the first control plane node.
- Server 1
➜ sudo k3d cluster create my-ha-cluster --servers 1 --k3s-arg '--cluster-init@server:0'
➜ sudo k3d kubeconfig get my-ha-cluster > .kube/config
➜ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k3d-my-ha-cluster-server-0 Ready control-plane,etcd,master 11m v1.25.7+k3s1
# Print k3s cluster token
➜ sudo k3d cluster list my-ha-cluster --token
NAME SERVERS AGENTS LOADBALANCER TOKEN
my-ha-cluster 1/1 0/0 true mynodetoken- Server 2
➜ sudo k3d node create worker1 --cluster https://192.168.1.128:33893 --token mynodetoken --k3s-arg "--node-external-ip=172.17.246.34"
➜ sudo k3d cluster list
NAME SERVERS AGENTS LOADBALANCER
https://192.168.1.128:33893 0/0 1/1 false- Server 1
➜ kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k3d-my-ha-cluster-server-0 Ready control-plane,etcd,master 69m v1.25.7+k3s1 172.18.0.3 <none> K3s dev 6.3.0-oleksis-microsoft-standard-WSL2 containerd://1.6.15-k3s1
k3d-worker1-0 Ready <none> 15m v1.25.7+k3s1 172.18.0.2 172.17.246.34 K3s dev 5.15.90.1-microsoft-standard-WSL2 containerd://1.6.15-k3s1
➜ kubectl create deployment nginx --image=nginx:latest --port=80 --replicas=2
➜ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-cd55c47f5-ckcl6 1/1 Running 0 3m51s 10.42.1.3 k3d-worker1-0 <none> <none>
nginx-cd55c47f5-xmx4w 1/1 Running 0 3m50s 10.42.0.16 k3d-my-ha-cluster-server-0 <none> <none>
➜ sudo k3d node list
NAME ROLE CLUSTER STATUS
k3d-my-ha-cluster-server-0 server my-ha-cluster running
k3d-my-ha-cluster-serverlb loadbalancer my-ha-cluster running
k3d-my-ha-cluster-tools my-ha-cluster running
- Server 2 wirt `k3d`
➜ k3d cluster create my-server2 -p "80:80@loadbalancer"
INFO[0000] portmapping '80:80' targets the loadbalancer: defaulting to [servers:*:proxy agents:*:proxy]
INFO[0000] Prep: Network
INFO[0005] Created network 'k3d-my-server2'
INFO[0006] Created image volume k3d-my-server2-images
INFO[0006] Starting new tools node...
INFO[0008] Creating node 'k3d-my-server2-server-0'
INFO[0008] Starting Node 'k3d-my-server2-tools'
INFO[0011] Creating LoadBalancer 'k3d-my-server2-serverlb'
INFO[0015] Using the k3d-tools node to gather environment information
INFO[0021] HostIP: using network gateway 172.19.0.1 address
INFO[0021] Starting cluster 'my-server2'
INFO[0021] Starting servers...
INFO[0021] Starting Node 'k3d-my-server2-server-0'
INFO[0062] All agents already running.
INFO[0062] Starting helpers...
INFO[0063] Starting Node 'k3d-my-server2-serverlb'
INFO[0077] Injecting records for hostAliases (incl. host.k3d.internal) and for 2 network members into CoreDNS configmap...
INFO[0082] Cluster 'my-server2' created successfully!
INFO[0083] You can now use it like this:
kubectl cluster-info
➜ k3d cluster list
NAME SERVERS AGENTS LOADBALANCER
https://192.168.1.128:33893 0/0 1/1 false
my-server2 1/1 0/0 true
➜ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k3d-my-server2-server-0 Ready control-plane,master 3m39s v1.25.7+k3s1 172.19.0.2 <none> K3s dev 5.15.90.1-microsoft-standard-WSL2 containerd://1.6.15-k3s1
➜ kubectl get service -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 5m29s <none>
➜ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES
8b1f4800eed7 ghcr.io/k3d-io/k3d-proxy:5.4.9 "/bin/sh -c nginx-pr…" 17 minutes ago Up 17 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:33829->6443/tcp k3d-my-server2-serverlb
f610e4ed2200 rancher/k3s:v1.25.7-k3s1 "/bin/k3s server --t…" 18 minutes ago Up 17 minutes k3d-my-server2-server-0
3b204737019d rancher/k3s:v1.25.7-k3s1 "/bin/k3s agent --no…" About an hour ago Up 58 minutes
➜ docker exec -it k3d-worker1-0 /bin/sh
ps -ef | grep containerd
ip ad
wget -O - -S 10.42.1.3
...
<title>Welcome to nginx!</title>
...
➜ kubectl create deployment nginx --image=nginx:latest --port=80
deployment.apps/nginx created
➜ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-cd55c47f5-9zpqx 1/1 Running 0 111s 10.42.0.9 k3d-my-server2-server-0 <none> <none>
➜ kubectl port-forward --address 0.0.0.0 nginx-cd55c47f5-9zpqx 8080:80- Servers with
k3sK3S on Windows Subsystem for Linux
export DOCKER_HOST="unix:///mnt/wsl/docker-desktop/shared-sockets/guest-services/docker.sock"
sudo ln -s /mnt/wsl/docker-desktop/shared-sockets/guest-services/docker.sock /var/run/docker.sock
