By default, K3S will run with flannel as the CNI and use custom directories to store CNI plugin binaries and config files(You can inspect the kubelet args K3S uses via journalctl -u k3s|grep cni-conf-dir
).
So you need to configure that properly When deploying Multus CNI.
For example given the official Multus manifests in https://github.com/intel/multus-cni/blob/36f2fd64e0965e639a0f1d17ab754f0130951aba/images/multus-daemonset.yml
, the following changes are needed:
volumes:
- name: cni
hostPath:
path: /var/lib/rancher/k3s/agent/etc/cni/net.d
- name: cnibin
hostPath:
path: /var/lib/rancher/k3s/data/<replace-with-your-hash>/bin
containers:
- name: kube-multus
image: nfvpe/multus:v3.4.1
command: ["/entrypoint.sh"]
args:
- "--multus-conf-file=auto"
- "--cni-version=0.3.1"
# Add the following arg
- "--multus-kubeconfig-file-host=/var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig"
Once Multus CNI is deployed properly it works as normal in K3S.
I too ran into some issues installing multus on k3s (although in my case, on a Raspberry Pi 4, not RKE). This post helped point me in the right direction (thank you), so I figured I'd post my learnings here.
Like the original post alludes to, k3s uses non-standard file locations for its CNI configurations and binaries. Multus expects those configurations and binaries to be in the same place. Trying to update the multus DaemonSet to accommodate this was pretty messy, so I instead created a DaemonSet that symlinks the k3s file locations to the standard ones, and this allows the standard multus DaemonSet to run correctly.
Keep in mind, you must run this DaemonSet BEFORE installing multus, otherwise multus will create actual directories where the symlinks would be. If you already ran multus prior, you can clean up your host nodes with:
WARNING: if the above commands are executed in the incorrect context, they may completely hose your Kubernetes nodes. The same goes for multus misconfiguration in general.
Also, I've recently started using ArgoCD to deploy multus - you can check out the WIP here. It works well, but I am currently changing my argocd configuration frequently, so ymmv.