K3s, Docker & Core V2

readme.md

K3s-based setup for Core V2 with Docker runtime

Context

Using docker runtime with K3s allows to re-use local Docker images. This give better experience when working with big Docker images (like Triton) as one can avoid:

• docker pulls from inside kind cluster
• still expensive kind load ... operations

Setting up K3s with Docker

Start K3s cluster disabling servicelb, traefik and using docker runtime

curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=v1.24 sh -s - --docker --disable servicelb --disable traefik

If using Fedora switch off (or configure) firewalld as discussed here:

systemctl disable firewalld --now

References:

• https://docs.k3s.io/upgrades/manual#release-channels
• https://docs.k3s.io/advanced#using-docker-as-the-container-runtime
• https://docs.k3s.io/networking#traefik-ingress-controller
• https://docs.k3s.io/networking#disabling-the-service-lb

Access K3s kubeconfig as default one

By default kubeconfig will be located under /etc/rancher/k3s/k3s.yaml. You can use sudo k3s kubectl to interact with the K3s cluster.

Following allows to move kubeconfig to default location to use kubectl command as usual.

sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config && sudo chown $USER:$USER ~/.kube/config

Note: this will overwrite your ~/.kube/config. If you like to merge configs you could follow a solution similar to the one discussed here.

Reference: https://docs.k3s.io/cluster-access

Install MetalLB

MetalLB is useful to get local IPs.

Install

helm repo add metallb https://metallb.github.io/metallb
helm install metallb metallb/metallb -n metallb-system --create-namespace

Create L2Advertisement and IPAdressPool

cat <<EOF | kubectl apply -f -
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: example
  namespace: metallb-system
spec:
  ipAddressPools:
  - local-pool

---

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: local-pool
  namespace: metallb-system
spec:
  addresses:
  - 172.25.255.1-172.25.255.255
EOF

Install Seldon Core

Pull Big Images

First, pull big images to avoid timeouts later, see caveat at the end.

docker pull docker.io/seldonio/mlserver:1.2.3
docker pull nvcr.io/nvidia/tritonserver:22.11-py3

Install with Ansible

Now install Core V2 with Ansible as usual.

ansible-playbook playbooks/setup-ecosystem.yaml
ansible-playbook playbooks/setup-seldon.yaml

Caveats

ImagePullBackOff

For some reasons, when using --docker runtime with K3s one gets ImagePullBackOff after 2 mins. Did not find solution to it yet. However, simple workaround is to just first docker pull ... the image on the laptop before deploying to K3s.

Uninstall does not remove containers

When running K3s with --docker runtime you will see all containers under your docker ps. For some reason when uninstalling K3s with the usual /usr/local/bin/k3s-uninstall.sh script these containers are left behind.