Recover cluster.rkestate file from controlplane node

restore-rkestate-file.md

Recover cluster.rkestate file from controlplane node

RKE

Run on controlplane node, uses any found hyperkube image

k8s 1.19 and higher

docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=org.opencontainers.image.source=https://github.com/rancher/hyperkube.git) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .' > cluster.rkestate

k8s 1.18 and lower

docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=org.label-schema.vcs-url=https://github.com/rancher/hyperkube.git) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .' > cluster.rkestate

Rancher v2.2.x

Run on controlplane node, uses rancher/rancher-agent image

docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=io.cattle.agent=true) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml get configmap -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .' > cluster.rkestate

Hello, more-than-late on this thread, I'd like to contribute "my 2 cents" (I'm, in fact, using it on a downstream cluster).

# use the desired context (kubie is a great tool!)
kubie ctx mycontext
# get kubeconfig file (this one has only *one* context inside)
kubectl config view --flatten > kube_config_cluster.yml
# get cluster.yml (this one references *master nodes only*)
kubectl get configmap -n kube-system full-cluster-state -o "jsonpath={.data.full-cluster-state}" | python3 -c 'import json, yaml, sys; yaml.safe_dump(json.load(sys.stdin).get("currentState", []).get("rkeConfig",[]), sys.stdout)' > cluster.yml
# get rkestate
rke util get-state-file
# backup etcd
rke etcd snapshot-save --name 20220323-mycluster

I'm using kubectl config view --flatten initially to obtain the kubeconfig file "thru Rancher server", while using rke util get-kubeconfig afterwards will obtain one that bypasses Rancher (direct cluster access).

this was excellent, thank you!