Rancher 2 HA using Helm and self signed certificate (certificate from files)

This will only cover the part of installing Rancher on a RKE built cluster, see https://rancher.com/docs/rancher/v2.x/en/installation/ha/ how to get there.

Note: make sure kubeconfig is configured correctly

The commands are for Linux, if you are using Mac then you can use md5 instead of md5sum and base64 -D instead of base64 -d.

Generate certificates

In this example, we use a simple utility to create self signed certificates based on paulczar/omgwtfssl.

DOMAIN=rancher.mydomain.com
curl https://gist.githubusercontent.com/superseb/b2c1d6c9baa32609a49ee117a27bc700/raw/7cb196e974e13b213ac6ec3105971dd5e21e4c66/selfsignedcert.sh | bash -s -- $DOMAIN

This will place the certificates in /certs in the current working directory.

Install Rancher

Check if these instructions are still up2date with the instructions in the linked page above.

DOMAIN=rancher.mydomain.com
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
kubectl create namespace cattle-system
helm install rancher rancher-latest/rancher --namespace cattle-system --set hostname=$DOMAIN --set ingress.tls.source=secret --set privateCA=true
kubectl -n cattle-system create secret tls tls-rancher-ingress --cert=certs/cert.pem --key=certs/key.pem
cp certs/ca.pem certs/cacerts.pem
kubectl -n cattle-system create secret generic tls-ca --from-file=certs/cacerts.pem

Wait for Rancher to be ready

kubectl -n cattle-system rollout status deploy/rancher
...
deployment "rancher" successfully rolled out

Check certificates

DOMAIN=rancher.mydomain.com
docker run superseb/ranchercheck https://$DOMAIN
...
Certificate chain is complete, connection to https://rancher.mydomain.com established successfully

Debugging

Check mounted file inside rancher/rancher containers

for pod in $(kubectl -n cattle-system get pods -l app=rancher -o custom-columns=NAME:.metadata.name --no-headers); do echo "Pod: $pod"; kubectl -n cattle-system exec $pod -- /bin/bash -c "cat /etc/rancher/ssl/cacerts.pem | openssl x509 -noout -subject -issuer -dates"; done

for pod in $(kubectl -n cattle-system get pods -l app=rancher -o custom-columns=NAME:.metadata.name --no-headers); do echo "Pod: $pod"; kubectl -n cattle-system exec $pod -- /bin/bash -c "cat /etc/rancher/ssl/cacerts.pem | md5sum"; done

Check created tls-ca secret

kubectl -n cattle-system get secret tls-ca -o "jsonpath={.data['cacerts\.pem']}" | base64 -d | openssl x509 -noout -subject -issuer -dates

kubectl -n cattle-system get secret tls-ca -o "jsonpath={.data['cacerts\.pem']}" | base64 -d | md5sum

Check /v3/settings/cacerts

Requires jq

curl -sLk https://rancher.mydomain.com/v3/settings/cacerts | jq -r .value | openssl x509 -noout -subject -dates -issuer

curl -sLk https://rancher.mydomain.com/v3/settings/cacerts | jq -r .value | md5sum

Check caCerts inside listenconfig cli-config

kubectl get listenconfig.management.cattle.io cli-config -o "jsonpath={.caCerts}" | openssl x509 -noout -subject -issuer -dates

kubectl get listenconfig.management.cattle.io cli-config -o "jsonpath={.caCerts}" | md5sum

When switching certificates, the value of caCerts can be stuck to the old value, and this will be copied to settings->cacerts. To sync them up, run:

SINGLELINECERT=$(kubectl -n cattle-system get secret tls-ca -o "jsonpath={.data['cacerts\.pem']}"  | base64 -d | awk 1 ORS='\\n')
kubectl patch listenconfig.management.cattle.io cli-config --type=merge --patch='{"caCerts": "'"$SINGLELINECERT"'"}'

superseb/rancher2-ha-helm-selfsigned-certificate.md