Deploy a lightweight 3-nodes K8s cluster with K3s and k3sup on your laptop

Official links

What I used in this lab ;)

Requirements

A minimum powerful laptop, I have the following configuration :

CPU : i7 4720HQ
RAM : 16GB
Disk : SSD 1TB

I use the following configuration for the virtual machines :

1 master : 2vCPU - 1GB RAM
2 workers : 2vCPU - 4GB RAM each node

Let's go !

Virtual machines setup

Firstly, you need to setup virtualization on your laptop if you're not using KVM on your laptop. You can also use VirtualBox, VMware Workstation/Player or Hyper-V.

For QEMU/KVM :
- Check virtualization (vt-x) is enabled in your BIOS.
- Install qemu, libvirt and Virt-Manager packages (packages depends of your distribution).
- Enable the libvirtd services and you can also add your user to the libvirt group (to avoid enter your password each time you start Virt-Manager).
Next, Amazon Linux 2. You can download the image for your prefered hypervisor here : https://docs.aws.amazon.com/en_pv/AWSEC2/latest/UserGuide/amazon-linux-2-virtual-machine.html#amazon-linux-2-virtual-machine-download
Once it's downloaded, you can create your virtual machines by using a copy of the image you've downloaded previously. For this lab, I use the disk default size and I used the hardware configuration above.
Create two files, user-data and meta-data. These files are used to create the cloud-init ISO file in order to configure the virtual machines at boot. Here are my two files for one node :

meta-data

local-hostname: k3sm
network-interfaces: |
  auto eth0
  iface eth0 inet static
  address 192.168.122.10
  network 192.168.122.0
  netmask 255.255.255.0
  gateway 192.168.122.1
  dns-nameservers 192.168.122.1

user-data

#cloud-config
#vim:syntax=yaml
users:
  - name: user
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    lock_passwd: false
    ssh-authorized-keys:
      - your ssh public key
chpasswd:
  list: |
    user:yourpassword

write_files:
  - path: /etc/cloud/cloud.cfg.d/80_disable_network_after_firstboot.cfg
    content: |
      network:
        config: disabled

More details on Amazon Linux 2 documentation above.

After filling these files, you can create an iso file for each node. Don't forget to adjust settings for each of your nodes (IP addresses, hostname) !

genisoimage -output seed-node-x.iso -volid cidata -joliet -rock user-data meta-data
Ok, now you're ready. Attach these ISO files to your virtual machines by using a virtual CD-ROM.
Boot the virtual machines and check your settings such as user and network. Prefer SSH for administration.
You can install the updates in order to be ready :

sudo yum update -y && sudo reboot

Install your k3s cluster with k3sup

Install k3sup

curl -sLS https://get.k3sup.dev | sh
sudo install k3sup /usr/local/bin/

Install master node

k3sup install --ip 192.168.122.10 --user user --ssh-key /path/to/ssh_keys --ssh-port 22 --k3s-version v0.9.0 --k3s-extra-args '--no-deploy servicelb --no-deploy traefik'

I want a vanilla cluster, so I didn't install the additionnal components provided by k3s such as Traefik (Ingress Controller) and Service LB. Here, "--ip" is the IP of my master node.

Install workers nodes

k3sup join --ip 192.168.122.11 --server-ip 192.168.122.10 --user user --ssh-key /path/to/ssh_keys --k3s-version v0.9.0

Here, "--ip" is the IP of one of my worker and "--server-ip" is the IP of my master node.

Test it !

Once it's done, you can add kubeconfig configuration details into your config file if it already exists ($HOME/.kube/config). I use kubectx (https://github.com/ahmetb/kubectx) to switch between my different contexts.

For testing, I tested with a "crappy" deployment :p

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 10
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80

Here is the result :

core@jupiter: Kubernetes » k get pods                                                                                                                                               
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-7bffc778db-sd6z2   1/1     Running   0          10s
nginx-deployment-7bffc778db-8bz2w   1/1     Running   0          10s
nginx-deployment-7bffc778db-zlntt   1/1     Running   0          10s
nginx-deployment-7bffc778db-hx7jq   1/1     Running   0          10s
nginx-deployment-7bffc778db-d6bvq   1/1     Running   0          10s
nginx-deployment-7bffc778db-r5cf6   1/1     Running   0          10s
nginx-deployment-7bffc778db-h4lfw   1/1     Running   0          10s
nginx-deployment-7bffc778db-v9h7t   1/1     Running   0          10s
nginx-deployment-7bffc778db-zjrdq   1/1     Running   0          10s
nginx-deployment-7bffc778db-dn64n   1/1     Running   0          10s

f-bn/3-nodes-local-k8s-cluster-k3sup.md

Official links

k3s (by Rancher Labs @Rancher_Labs)

k3sup (by Alex Ellis @alexellisuk)

Amazon Linux 2

QEMU/KVM