fluxcd + akri + conbee ii

README.md

This is a quick documentation on how I've setup akri in my kubernetes cluster to make my Conbee II stick available to zigbee2mqtt as a remote device. This allows zigbee2mqtt to run on any node in the cluster. The advantage to using Akri over node-feature-detection and node affinity rules to run zigbee2mqtt on a given node is that when a device is unplugged Akri will automatically remove the pod. So I can move the stick from one node to another and less than a minute later the stick is ready to be used again.

Eventually Kubernetes plans to add requiredDuringSchedulingRequiredDuringExecution affinity rules which in theory could give us a similar effect where zigbee2mqtt could connect directly to the device rather than connecting remotely, but we could simply move the stick and the zigbee2mqtt pod would follow it.

With this configuration, you should have a kind: Service that gets added which will have an endpoint going to the broker pod running ser2net on port 2000. Akri sets the Service name to <Configuration.metadata.name>-svc. So lets say our Configuration is named conbee-ii in the host-system namespace in zigbee2mqtt and the standard kubernetes service domain confiugration, you would configure zigbee2mqtt to use tcp://conbee-ii-svc.host-system.svc.cluster.local:2000.

Note: This should also work with other sticks. For example, my Aeotec Gen 5 stick would get matched with udev this rule:

ATTRS{bDeviceClass}=="02", ATTRS{idVendor}=="0658", ATTRS{idProduct}=="0200", SUBSYSTEM=="tty"

However, at the moment, I haven't swapped it over yet. I'm testing the waters with my zigbee stuff as zigbee just has my lights and sensors. My door locks are on z-wave and I want to make sure that stays stable.

---

Your HelmRelease for akri may or may not need the patches I have applied. The reason I have applied the patches is that Akri relies on crictl and by default it is looking on the host system for it. In my case I'm running talos, where access to the OS is via a CLI + API. It prevents shell access so it's a little difficult to get something installed on the host. But in reality it doesn't NEED to be on the host, so I've patched it to just download crictl to an emptydir volume shared in the pod. I suppose I could have alternatively still mounted a host path and download it there and include the script to check if its already present.

---

The Configuration CRD is what will actually tell Akri to match a specific device and create a broker pod, which I've defined for now as just an alpine image that runs a shell script that installs and starts ser2net. I didn't see any recently updated images that run ser2net, so I just did this for now. Perhaps I should create a ser2net docker image that can be updated automatically with renovate ✨.

---

It's also worth noting that currently there appears to be a bug in at least the deconz driver for zigbee2mqtt where if the service ip changes or in otherways times out, the driver blows up before it tries to re-connect. Specifically this block of code needs to be updated to handle both this.serialPort and this.socketPort because it throws a Cannot read property 'isOpen' of undefined error when performing this.serialPort.isOpen. This an issue that went stale around this. I've opened a PR to fix this here.

configuration.conbee-ii.yaml

apiVersion: akri.sh/v0
kind: Configuration
metadata:
  name: conbee-ii
  namespace: host-system
spec:
  # how many things can claim a found device resource
  capacity: 1

  discoveryHandler:
    name: udev
    discoveryDetails: |+
      udevRules:
        - ATTRS{bDeviceClass}=="02", ATTRS{idVendor}=="1cf1", ATTRS{idProduct}=="0030", SUBSYSTEM=="tty"
  
  brokerPodSpec:
    containers:
      - name: conbee-ii-broker
        image: alpine:3.14.0
        imagePullPolicy: IfNotPresent
        securityContext:
          privileged: true
        command:
          - sh
          - -c
          - |-
            set -euxo pipefail
            echo Installing ser2net
            apk add --no-cache ser2net
            ser2net -d -C 2000:raw:0:$UDEV_DEVNODE:9600
        ports:
          - name: device
            containerPort: 2000
        resources:
          requests:
            "{{PLACEHOLDER}}": '1'
            cpu: 10m
            memory: 50Mi
          limits:
            "{{PLACEHOLDER}}": '1'
            cpu: 100m
            memory: 200Mi

  configurationServiceSpec:
    type: ClusterIP
    ports:
      - name: device
        port: 2000
        protocol: TCP
        targetPort: device

helm-release.akri.yaml

---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: akri
  namespace: host-system
spec:
  interval: 5m
  chart:
    spec:
      # renovate: registryUrl=https://deislabs.github.io/akri/
      chart: akri
      version: 0.6.5
      sourceRef:
        kind: HelmRepository
        name: akri-charts
        namespace: flux-system
      interval: 5m
  values:
    agent:
      host:
        dockerShimSock: /run/containerd/containerd.sock
      resources:
        memoryRequest: 15Mi
        cpuRequest: 30m
        memoryLimit: 100Mi
        cpuLimit: 100m
    prometheus:
      enabled: true
    udev:
      discovery:
        enabled: true
        resources:
          memoryRequest: 11Mi
          cpuRequest: 10m
          memoryLimit: 100Mi
          cpuLimit: 200m
  postRenderers:
    - kustomize:
        patchesJson6902:
          - target:
              kind: DaemonSet
              name: akri-agent-daemonset
            patch:
              # change usr-bin-crictl volume to an emptyDir
              - op: replace
                path: /spec/template/spec/volumes/2
                value:
                  name: usr-bin-crictl
                  emptyDir:
              # add an updated HOST_CRICTL_PATH environment variable
              # because the original path is now a directory
              - op: add
                path: /spec/template/spec/containers/0/env/0
                value:
                  name: HOST_CRICTL_PATH
                  value: /host/usr/bin/crictl/crictl
              # add an init container that will download crictl into the emptyDir shared mount
              - op: add
                path: /spec/template/spec/initContainers
                value:
                  - name: install-crictl
                    image: curlimages/curl:7.77.0
                    volumeMounts:
                      - name: usr-bin-crictl
                        mountPath: /mnt
                    command:
                      - sh
                      - -c
                      - |
                        set -euxo pipefail
                        curl --silent --fail -L https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.21.0/crictl-v1.21.0-linux-amd64.tar.gz --output /tmp/crictl.tar.gz
                        tar zxvf /tmp/crictl.tar.gz -C /tmp
                        rm -f /tmp/crictl.tar.gz
                        mv /tmp/crictl /mnt/crictl
                        ls -la /mnt
                        test -x /mnt/crictl
                        exit $?

Thanks for the detailed description and for sharing your configuration! I will try it out soon in my cluster. I have a similar situation with two Pis in a local k3s cluster. And I also want to have the option that zigbee2mqtt can run on any of these two nodes. But I use ArgoCD instead of fluxcd, so I have to make a few adjustments. I will add another comment with an update on my progress soon.

Have a nice day!

Hey again!
It took some time before I got back to this topic. But I now have it up and running. So if someone is using ArgoCD and k3s, this values.yaml file could be helpful to you.

Note: I use this file with a Chart.yaml file. In the Chart.yaml akri is set as a dependency. Therefore, the root key is set to akri.

values.yaml:

akri:
  agent:
    host:
      crictl: /usr/local/bin/crictl
      dockerShimSock: /run/k3s/containerd/containerd.sock
    resources:
      memoryRequest: 15Mi
      cpuRequest: 30m
      memoryLimit: 100Mi
      cpuLimit: 100m
  prometheus:
    enabled: true
  udev:
    discovery:
      enabled: true
      resources:
        memoryRequest: 11Mi
        cpuRequest: 10m
        memoryLimit: 100Mi
        cpuLimit: 200m
    configuration:
      enabled: true
      name: conbee
      discoveryDetails:
        udevRules:
          - ATTRS{bDeviceClass}=="02", ATTRS{idVendor}=="1cf1", ATTRS{idProduct}=="0030", SUBSYSTEM=="tty"
      capacity: 1
      brokerPod:
        image:
          repository: alexbabel/ser2net
          tag: 1.0.0
          pullPolicy: IfNotPresent
        resources:
          memoryRequest: 10Mi
          cpuRequest: 10m
          memoryLimit: 30Mi
          cpuLimit: 29m
      createInstanceServices: false
      createConfigurationService: true
      configurationService:
        port: 80
        targetPort: 8080

Chart.yaml:

apiVersion: v2
description: Akri
type: application
name: akri
version: 0.0.0
appVersion: 0.0.0
dependencies:
  - name: akri
    version: 0.7.0
    repository: https://project-akri.github.io/akri/

In Zigbee2MQTT you just use this:

serial:
  adapter: deconz
  port: tcp://conbee-svc.akri.svc.cluster.local:80

👍🏻 awesome! Glad this could be helpful. Thanks for sharing for ArgoCD too!