Multus CNI with k3s and RKE

01-multus-k3s.md

Using Multus CNI in K3S

By default, K3S will run with flannel as the CNI and use custom directories to store CNI plugin binaries and config files(You can inspect the kubelet args K3S uses via journalctl -u k3s|grep cni-conf-dir). So you need to configure that properly When deploying Multus CNI.

For example given the official Multus manifests in https://github.com/intel/multus-cni/blob/36f2fd64e0965e639a0f1d17ab754f0130951aba/images/multus-daemonset.yml, the following changes are needed:

volumes:
  - name: cni
    hostPath:
      path: /var/lib/rancher/k3s/agent/etc/cni/net.d
  - name: cnibin
    hostPath:
      path: /var/lib/rancher/k3s/data/<replace-with-your-hash>/bin

containers:
  - name: kube-multus
    image: nfvpe/multus:v3.4.1
    command: ["/entrypoint.sh"]
    args:
      - "--multus-conf-file=auto"
      - "--cni-version=0.3.1"
      # Add the following arg
      - "--multus-kubeconfig-file-host=/var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig"

Once Multus CNI is deployed properly it works as normal in K3S.

02-multus-rke.md

Using Multus CNI in RKE

1. Provision RKE cluster with automatic CNI deployment disabled

RKE cluster.yaml:

...
network:
    plugin: none
services:
  kubelet:
    extra_args:
      cni-conf-dir: "/etc/cni/net.d"
      cni-bin-dir: "/opt/cni/bin"
...

2. Install Multus

Use the following Helm chart for conveniance:

https://github.com/ibrokethecloud/multus-helm

https://github.com/ibrokethecloud/cni-plugins https://ubuntu.com/kubernetes/docs/cni-sriov

03-external.md

• k8snetworkplumbingwg/multus-cni#539
• https://github.com/kubernetes-sigs/kubespray/blob/master/docs/multus.md
• https://medium.com/@sarpkoksal/multus-installation-on-kubernetes-a0a6ef04972f
• https://github.com/k8snetworkplumbingwg/reference-deployment/blob/a3b4d3066ac0e130deee7d20fa02a72d67e9e683/multus-calico/multus-crio-daemonset.yml
• https://github.com/rancher/kontainer-driver-metadata/blob/497e5d720660bf9c8dc7ab0c7b44ee6daca28534/rke/templates/calico.go
• rancher/rke#873 (comment)

Hello is there a new technique to deploy multus cni on k3s Am stuck with error

this is what I did that worked for me. adding the clusterNetwork line made it work. before that I manually installed flannel over the built-in version. Created a new test cluster and ran this helmchart like this and it worked fine.

# k3s multus install
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: multus
  namespace: kube-system
spec:
  repo: https://rke2-charts.rancher.io
  chart: rke2-multus
  targetNamespace: kube-system
  # createNamespace: true
  valuesContent: |-
    config:
      cni_conf:
        confDir: /var/lib/rancher/k3s/agent/etc/cni/net.d
        clusterNetwork: /var/lib/rancher/k3s/agent/etc/cni/net.d/10-flannel.conflist
        binDir: /var/lib/rancher/k3s/data/current/bin/
        kubeconfig: /var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig