containerd + kubernetes, and making clr-examples do what i want on Bionic

notes.md

Quick guide for getting Kata+containerd (using v2 shim) up and running super quick on bionic

Installation of Kube stuff on bionic:

Use the following sh:

sudo bash -c "cat <<EOF > /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial-unstable main
EOF"
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

sudo -E apt update
sudo -E apt install -y kubelet kubeadm kubectl

VERSION="1.2.5"
echo "Install Containerd ${VERSION}"
wget -q https://storage.googleapis.com/cri-containerd-release/cri-containerd-${VERSION}.linux-amd64.tar.gz
sudo tar -C / -xzf cri-containerd-${VERSION}.linux-amd64.tar.gz
sudo systemctl start containerd
sudo swapoff -a
sudo modprobe br_netfilter

Helper script for starting/resetting:

Grab cloud-native-setup:

git clone https://github.com/clearlinux/cloud-native-setup.git

Apply patch, which I still need to upstream:

Update to use containerd instead of CRIO, and make minimal more minimal

cd cloud-native-setup
wget  https://gist.githubusercontent.com/egernst/138def9599a42376f409fbce2e95de5d/raw/9e1b952b75f92fc125f65a068563f0286e159f60/z0001-Use-containerd-and-let-me-configure-Kata.patch
git am z0001-Use-containerd-and-let-me-configure-Kata.patch  

Launch kube:

pushd cloud-native-setup/cl-k8s-examples
./create-stack.sh minimal

Setup Kata latest inside the running cluster:

Use kata deploy, and register the runtimeclass, kata:

kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-rbac.yaml
kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-deploy.yaml
kubectl apply -f https://raw.githubusercontent.com/clearlinux/cloud-native-setup/master/clr-k8s-examples/8-kata/kata-qemu-runtimeClass.yaml

Run Kata example:

Here's a simple example pod. Of not is the runtimeClass definition

kubectl apply -f https://gist.githubusercontent.com/egernst/9cbbb3d6ba566afca98be2a4e29328d6/raw/0f6e332ba1b7306ed895e5ff91b2939ca3f0bb42/two-pod.yaml

z0001-Use-containerd-and-let-me-configure-Kata.patch

From aa5abd9ab621e5ca2dd7a1e0adfcd5d4469f0bd2 Mon Sep 17 00:00:00 2001
From: Eric Ernst <[email protected]>
Date: Fri, 12 Apr 2019 03:51:57 +0000
Subject: [PATCH 1/1] Use containerd, and let me configure Kata

Signed-off-by: Eric Ernst <[email protected]>
---
 clr-k8s-examples/create_stack.sh |  2 +-
 clr-k8s-examples/kubeadm.yaml    |  2 +-
 clr-k8s-examples/reset_stack.sh  | 10 +++++-----
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/clr-k8s-examples/create_stack.sh b/clr-k8s-examples/create_stack.sh
index ffd811f..55d5677 100755
--- a/clr-k8s-examples/create_stack.sh
+++ b/clr-k8s-examples/create_stack.sh
@@ -98,7 +98,7 @@ function miscellaneous() {
 function minimal() {
        cluster_init
        cni
-       kata
+       #kata
        metrics
 }
 
diff --git a/clr-k8s-examples/kubeadm.yaml b/clr-k8s-examples/kubeadm.yaml
index de436b2..92defd7 100644
--- a/clr-k8s-examples/kubeadm.yaml
+++ b/clr-k8s-examples/kubeadm.yaml
@@ -1,7 +1,7 @@
 apiVersion: kubeadm.k8s.io/v1beta1
 kind: InitConfiguration
 nodeRegistration:
-  criSocket: /var/run/crio/crio.sock
+  criSocket: /var/run/containerd/containerd.sock
 ---
 apiVersion: kubelet.config.k8s.io/v1beta1
 kind: KubeletConfiguration
diff --git a/clr-k8s-examples/reset_stack.sh b/clr-k8s-examples/reset_stack.sh
index 77a5118..08d6e32 100755
--- a/clr-k8s-examples/reset_stack.sh
+++ b/clr-k8s-examples/reset_stack.sh
@@ -3,7 +3,7 @@
 set -o nounset
 
 #Cleanup
-sudo -E kubeadm reset -f --cri-socket="/var/run/crio/crio.sock"
+sudo -E kubeadm reset -f --cri-socket="/var/run/containerd/containerd.sock"
 
 for ctr in $(sudo crictl ps --quiet); do
        sudo crictl stop "$ctr"
@@ -17,7 +17,7 @@ done
 #Forcefull cleanup all artifacts
 #This is needed is things really go wrong
 sudo systemctl stop kubelet
-sudo systemctl stop crio
+sudo systemctl stop containerd
 sudo pkill -9 qemu
 sudo pkill -9 kata
 sudo pkill -9 kube
@@ -39,8 +39,8 @@ sudo -E bash -c "rm -r /var/run/kata-containers/*"
 sudo rm -rf /var/lib/rook
 
 sudo systemctl daemon-reload
-sudo systemctl enable kubelet crio
-sudo systemctl restart crio
+sudo systemctl enable kubelet containerd
+sudo systemctl restart containerd
 sudo systemctl restart kubelet
 
-sudo -E kubeadm reset -f --cri-socket="/var/run/crio/crio.sock"
+sudo -E kubeadm reset -f --cri-socket="/var/run/containerd/containerd.sock"
-- 
2.17.1