Cilium Service Mesh

This instruction set:

represents the 2nd installment of a two-part series (Part 1 here)
assumes Docker Engine on macOS with the latest KinD client installed (0.20.0 -> k8s v1.27.3)
covers Cilium's ability to replace kube-proxy, as detailed in this blog

Create a cluster

k8s_name=cilium-full-$(date +"%y%m%d%H%M")
cat <<EOF | kind create cluster --config -
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: ${k8s_name}
nodes:
  - role: control-plane
    kubeadmConfigPatches:
    - |
      kind: InitConfiguration
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "ingress-ready=true"
    extraPortMappings:
    - containerPort: 80
      hostPort: 80
      listenAddress: 127.0.0.1
      protocol: TCP
    - containerPort: 443
      hostPort: 443
      listenAddress: 127.0.0.1
      protocol: TCP
  - role: worker
  - role: worker
networking:
  disableDefaultCNI: true  # do not install kindnet
  kubeProxyMode: none      # do not run kube-proxy instances
EOF

The combined effect of the networking settings above is that neither the kindnet CNI or kube-proxy workloads will be running.

kubectl -n kube-system get pods

Install the Cilium and Hubble clients

If required, follow the instructions here to install the clients for your local OS:

Deploy Cilium in cluster

Deploy the cilium daemonset in the cluster as follows

helm upgrade --install --namespace kube-system --repo https://helm.cilium.io cilium cilium --values - <<EOF
kubeProxyReplacement: strict
k8sServiceHost: $(kubectl config current-context | cut -d'-' -f2-)-control-plane
k8sServicePort: 6443
hostServices:
  enabled: false
externalIPs:
  enabled: true
nodePort:
  enabled: true
hostPort:
  enabled: true
image:
  pullPolicy: IfNotPresent
ipam:
  mode: kubernetes
hubble:
  enabled: true
  relay:
    enabled: true
  ui:
    enabled: true
    ingress:
      enabled: true
      annotations:
        kubernetes.io/ingress.class: nginx
      hosts:
        - hubble-ui.127.0.0.1.nip.io
EOF

Install Ingress Controller

kubectl apply -f \
  https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.4/deploy/static/provider/kind/deploy.yaml

Navigate to Hubble

Hubble UI should be browsable at http://hubble-ui.127.0.0.1.nip.io