Skip to content

Instantly share code, notes, and snippets.

@StephenSorriaux
Created October 25, 2018 18:47
Show Gist options
  • Save StephenSorriaux/fa07afa57c931c84d1886b08c704acfe to your computer and use it in GitHub Desktop.
Save StephenSorriaux/fa07afa57c931c84d1886b08c704acfe to your computer and use it in GitHub Desktop.
Install Kubernetes on bare-metal ArchLinux host

Installing Kubernetes on ArchLinux

Packages

pacman -S curl docker ebtables ethtool wget unzip

Also cfssl is needed but available on AUR, using pacaur

pacaur -S cfssl

Configuring Docker

Add --iptables=false and --ip-masq=false parameters to the dockerd daemon in the docker systemd service (/usr/lib/systemd/system/docker.service).

Allow bridged IPV4 traffic to iptables' chains using:

sysctl net.bridge.bridge-nf-call-iptables=1

If Docker was previsously used, clean the iptables rules using:

iptables -F
iptables -t nat -F

Start or restart Docker. systemctl enable docker && systemctl restart docker

Installing CNI

export CNI_VERSION="v0.6.0"
mkdir -p /opt/cni/bin
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-amd64-${CNI_VERSION}.tgz" | tar -C /opt/cni/bin -xz

Installing CRI

export CRICTL_VERSION="v1.11.1"
mkdir -p /opt/bin
curl -L "https://github.com/kubernetes-incubator/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-amd64.tar.gz" | tar -C /opt/bin -xz

Installing kubeadm, kubelet, kubectl

RELEASE="$(curl -sSL https://dl.k8s.io/release/stable.txt)"

mkdir -p /opt/bin
cd /opt/bin
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
chmod +x {kubeadm,kubelet,kubectl}

curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes/${RELEASE}/build/debs/kubelet.service" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service
mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes/${RELEASE}/build/debs/10-kubeadm.conf" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
systemctl enable kubelet && systemctl start kubelet

Creating the cluster

kubeadm init --pod-network-cidr=10.244.0.0/16
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

At this point you should be able to use

$ kubectl get no
NAME                 STATUS   ROLES    AGE   VERSION
stephen-arch-linux   Ready    master   31m   v1.12.1

Adding Flannel

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml

Once Flannel is up, your cluster is up and running.

Making the master schedulable

kubectl taint nodes --all node-role.kubernetes.io/master-

Accessing your k8s services from outside the cluster

Install an Ingress controller, for instance the NGINX Ingress Controller:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml

Dont forget the k8s service:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml

Using the Ingress object you will be able to access your services.

Using local volume as PersistentVolumeClaim

Create a storageClass (this object is not namespaced):

---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

Make it the default one:

kubectl annotate storageclass local-storage storageclass.kubernetes.io/is-default-class=true

For each PersistentVolumeClaim, you will need to manually create a PersistentVolume:

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /etc/kubernetes/local
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: beta.kubernetes.io/os
          operator: In
          values:
          - linux

Be sure that the spec.local.path exists on the host.

@vi2co
Copy link

vi2co commented Feb 24, 2020

Hi @zebralight, I had the the issue with Kube cluster "NotReady" status. In my case it was wrong CNI installation folder (/usr/bin instead of /opt/cni/bin). But you can troubleshoot this issue using journalctl -u kubelet -r command ( -r means latest logs are on top of the screen).

One of the discrepancies between the guide and what I needed to do is to run kubectl as root.

Just run:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config`

The full list of commands which for Manjaro Awesome 18.1.3 (5.3.11-1-MANJARO kernel ):

############### SUDO SHELL ###############################
pacman -S ebtables ethtool socat
modprobe br_netfilter
echo "br_netfilter" > /etc/modules-load.d/br-netfilter.conf
sysctl net.bridge.bridge-nf-call-iptables=1
#### Replace Docker service ExecStart /usr/lib/systemd/system/docker.service with the following:
ExecStart=/usr/bin/dockerd --exec-opt native.cgroupdriver=systemd --iptables=false --ip-masq=false -H fd://
systemctl daemon-reload

export CNI_VERSION="v0.8.5"
mkdir -p /opt/cni/bin
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-amd64-${CNI_VERSION}.tgz" | tar -C /opt/cni/bin -xz

export CRICTL_VERSION="v1.17.0"
mkdir -p /opt/cri
curl -L "https://github.com/kubernetes-incubator/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-amd64.tar.gz" | tar -C /opt/cri -xz

cd /usr/bin
RELEASE="$(curl -sSL https://dl.k8s.io/release/stable.txt)"
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
chmod +x {kubeadm,kubelet,kubectl}

curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes/${RELEASE}/build/debs/kubelet.service" > /etc/systemd/system/kubelet.service
mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes/${RELEASE}/build/debs/10-kubeadm.conf" > /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
systemctl enable kubelet

sudo kubeadm init --pod-network-cidr=10.244.0.0/24 --apiserver-advertise-address=0.0.0.0

echo 'KUBELET_KUBEADM_ARGS="--cgroup-driver=systemd  --runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1"' > /var/lib/kubelet/kubeadm-flags.env
################# Normal user shell ###################################
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml`

@krisnova
Copy link

krisnova commented May 1, 2020

Sup nerds.

OFC the download directory changed in 1.18 of Kubernetes.

The new tree is for debian packages is here and we can steal the kubelet.service unit file and 10-kubeadm.conf as follows:

RELEASE="$(curl -sSL https://dl.k8s.io/release/stable.txt)"

mkdir -p /opt/bin
cd /opt/bin
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
chmod +x {kubeadm,kubelet,kubectl}

curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service
mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
systemctl enable kubelet && systemctl start kubelet

The commit with the changes: kubernetes/website@5cf0d19

Also I think the Arch docs suggests /usr/lib/systemd/system for installed packages - which is where I am putting mine. But whatevs. https://wiki.archlinux.org/index.php/systemd#Writing_unit_files

@StefanLobbenmeier
Copy link

StefanLobbenmeier commented Jun 7, 2020

@kris-nova small type in the snippet - ${RELEASE_VERSION} should be ${RELEASE} right?

Also
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE}/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" still gives me 404 with the current Release of v1.18.3: https://raw.githubusercontent.com/kubernetes/release/v1.18.3/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service

Found it: https://github.com/kubernetes/website/blob/master/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md

RELEASE_VERSION="v0.2.7"
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service
mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment