Install latest Raspbian by following these instructions:
https://desertbot.io/blog/headless-raspberry-pi-3-bplus-ssh-wifi-setup
Installation steps for running K3s on a Raspberry PI cluster. My RPIs:
# Model
pi@k3s-master-1:~ $ cat /sys/firmware/devicetree/base/model
Raspberry Pi 3 Model B Plus Rev 1.3
pi@k3s-worker-2:~ $ cat /sys/firmware/devicetree/base/model
Raspberry Pi 4 Model B Rev 1.1
pi@k3s-worker-3:~ $ cat /sys/firmware/devicetree/base/model
Raspberry Pi 4 Model B Rev 1.1
# OS version
pi@k3s-master-1:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
pi@k3s-master-1:~ $ uname -a
Linux k3s-master-1 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux
Install latest Rapsbian on all RPIs using standard procedures. Configure password for user pi, change hostname and expand file system using sudo raspi-config. I use the following naming standard for hostnames:
- master: k3s-master-1
- worker: k3s-worker-1, k3s-worker-2, k3s-worker-3
Add hostnames to /etc/hosts on all RPIs.
Update to latest OS version on all RPIs:
sudo apt update && sudo apt dist-upgrade
sudo apt clean
sudo reboot
Currently (20190927) on there is a bug with incompatibility issues between Kube proxy and IP tables v1.8. kubernetes/kubernetes#71305 Therefor to get networking to work the following work-around is needed on all RPIs in the cluster (it will configure legacy mode for IP tables):
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo sysctl net.bridge.bridge-nf-call-iptables=1
# Check version
iptables -V
iptables v1.8.2 (legacy)
Turm off swap:
sudo dphys-swapfile swapoff && sudo dphys-swapfile uninstall && sudo update-rc.d dphys-swapfile remove
We also need to configure Rasbian kernel to support container features, edit /boot/cmdline.txt on all RPIs and add the following to the end of the line:
cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory
Then reboot all RPIs again.
K3s uses K3d (containerd) as default container runtime (use can use Docker with the --docker swtich), but since the my applications uses Docker we will install the Docker daemon and docker-compose on the master node:
# Install Docker:
curl -sSL https://get.docker.com | sh
# Install docker-compose via pip, it takes a while
sudo apt-get install libffi-dev libssl-dev
sudo apt install python3-dev
sudo apt-get install -y python3 python3-pip
With pip3 and python3 installed, it’s possible to install Docker-Compose via the command;
sudo pip3 install docker-compose
Currently (20190927) there is a bug with docker push: docker/for-linux#711
To fix that and also to prepare for having a local docker registry we'll add the following lines to /etc/docker/daemon.json (NOTE: If the daemon.json does not exist, then create it. Change to another hostname if you do not follow my naming standard):
pi@k3s-worker-2:~ $ cat /etc/docker/daemon.json
{
"max-concurrent-uploads": 1,
"insecure-registries" : ["k3s-master-1:5000"]
}
Restart docker-daemon:
sudo systemctl restart docker.service
Time to install K3s, on master node (NOTE:: Instructions tested on K3S version v1.18.9+k3s1)
curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644
Check that the k3s service is running:
sudo systemctl status k3s
Find out K3s master node join code:
sudo cat /var/lib/rancher/k3s/server/node-token
The join code is used to grant access for all worker nodes. On all worker nodes (NOTE: If hostname not present in /etc/hosts, use master node's IP below):
export K3S_URL="https://k3s-master-1:6443"
export K3S_TOKEN="K10e9fd828806f453fa2293e4fa7e5db51696b29565819b2a5a0320c3b620572b01::node:519d266f594a54673f1f52bef47443e1"
curl -sfL https://get.k3s.io | sh -
# Repeat for all working nodes
Optional but convenient, add following aliases to .bashrc on master node, then source via . bashrc:
alias k='kubectl'
alias ks='kubectl -n kube-system'
Set roles 'worker' on the worker nodes:
kubectl label nodes k3s-worker-2 node-role.kubernetes.io/worker=worker
kubectl label nodes k3s-worker-3 node-role.kubernetes.io/worker=worker
Check that master and all worker nodes are running (NOTE: Examples below requires alias being set):
k get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k3s-master-1 Ready master 9h v1.15.4-k3s.1 192.168.1.150 <none> Raspbian GNU/Linux 10 (buster) 4.19.66-v7+ containerd://1.2.8-k3s.1
k3s-worker-2 Ready worker 9h v1.15.4-k3s.1 192.168.1.161 <none> Raspbian GNU/Linux 10 (buster) 4.19.66-v7l+ containerd://1.2.8-k3s.1
k3s-worker-3 Ready worker 9h v1.15.4-k3s.1 192.168.1.149 <none> Raspbian GNU/Linux 10 (buster) 4.19.66-v7l+ containerd://1.2.8-k3s.1
If you want to be able to run kubectl from your local computer and not from the cluster master node, then install kubectl on your local computer:
https://kubernetes.io/docs/tasks/tools/install-kubectl/
Then create the file ~/.kube/config on your local computer and paste the contents of the file cat /etc/rancher/k3s/k3s.yaml on your master node. NOTE: You need to replace 127.0.0.1 with the IP of your master node.
Then use @ruan's excellent tutorial (skip the docker installation part) to try your new K3s cluster out! :-) https://sysadmins.co.za/develop-build-and-deploy-a-golang-app-to-k3s/