superseb

Rancher/RKE etcd snapshot intro

Brief description of what is done where/how/what etc

Repositories/tools involved:

• https://github.com/rancher/rke-tools The repository containing the the source for the rke-tools image which is used for all snapshot operations. See https://github.com/rancher/rke-tools/blob/master/main.go#L36 for available flags.
• https://github.com/rancher/rke RKE has an etcd subcommand to run operations on etcd. See https://github.com/rancher/rke/blob/master/cmd/etcd.go for options

superseb

#!/usr/bin/env bash
if [ $# -eq 0 ]; then
    # Check if run on controlplane node, we can use that kubeconfig
    if [ -f /opt/rke/etc/kubernetes/ssl/kube-controller-manager.pem ]; then
      KUBECTLCERT=/opt/rke/etc/kubernetes/ssl/kube-controller-manager.pem
    elif [ -f /etc/kubernetes/ssl/kube-controller-manager.pem ]; then
      KUBECTLCERT=/etc/kubernetes/ssl/kube-controller-manager.pem
    fi
    if [ -f /opt/rke/etc/kubernetes/ssl/kube-controller-manager-key.pem ]; then
      KUBECTLKEY=/opt/rke/etc/kubernetes/ssl/kube-controller-manager-key.pem

superseb

Retrieve full-cluster-state from etcd directly

etcdctl

docker exec -e ETCDCTL_ENDPOINTS=$(docker exec etcd /bin/sh -c "etcdctl member list | cut -d, -f5 | sed -e 's/ //g' | paste -sd ','") etcd etcdctl get /registry/configmaps/kube-system/full-cluster-state | tail -n1 | tr -c '[:print:]\t\r\n' '[ *]' | sed 's/^.*{"desiredState/{"desiredState/'  | docker run -i oildex/jq:1.6 jq -r . > cluster.rkestate 2>/dev/null

curl

superseb

Check certificate state on Rancher v2.2 clusters

Steps to verify certificate state for custom clusters in v2.2

Check certificates locally on nodes

Run script below to output certificate checksums for certificates present on the node.

#!/bin/sh

superseb

Recover cluster.rkestate file from controlplane node

RKE

Run on controlplane node, uses any found hyperkube image

k8s 1.19 and higher

docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=org.opencontainers.image.source=https://github.com/rancher/hyperkube.git) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .' > cluster.rkestate

superseb

#!/bin/bash
if [ "$#" -ne 2 ]; then
    echo "Usage: $0 fqdn email"
    exit 1
fi

docker run -p 80:80 -p 443:443 -v /etc/letsencrypt:/etc/letsencrypt certbot/certbot certonly   --standalone   --agree-tos   --reinstall   --force-renewal   --non-interactive   --text   --rsa-key-size 4096   --email $2 --domains $1                                                          

mkdir -p /root/.minio/certs
cp /etc/letsencrypt/live/$1/fullchain.pem /root/.minio/certs/public.crt

superseb

Check certificate DNS/IP on etcd in Rancher custom cluster

Requirements:

• kubectl installed
• SSH access available to nodes

Retrieve kubeconfig from custom cluster

Use Kubeconfig File in the UI when in cluster dashboard and configure it for kubectl (either place it in ~/.kube/config or set KUBECONFIG=/tmp/kubeconfig if /tmp/kubeconfig is the file containing the kubeconfig)

superseb

Troubleshooting Kubernetes commands

Commands belonging to the Rancher webinar Troubleshooting Kubernetes

etcd

Check etcd members

docker exec etcd etcdctl member list

jlesage

#!/bin/bash

#
# Post-processing script for Plex DVR.
#
# This script is used to convert the video file produced by the DVR to a smaller
# version.  The conversion is performed by a separate Docker container.
#
# For example, the HandBrake docker container is capable of automatically
# converting files put in a specific folder, called the "watch" folder.  The

janeczku

Follow these steps to update the SSL certificate of the ingress in a Rancher High Availability installation or switch from the default self-signed to a custom certificate:

1. Create or update the tls-rancher-ingress k8s secret resource with the new certificate and private key
2. Create or update the tls-ca k8s secret resource with the root CA certificate (only required when using a private CA)
3. Update Rancher installation using Helm CLI
4. Reconfigure Rancher Agents to trust the new CA certificate

Detailed steps

1. Create/Update the certificate secret resource