Murali Paluru leodotcloud

Identify RKE / Rancher Launched Kubernetes / Imported clusters

This will describe how to identify clusters that are created by RKE, created by Rancher or managed by Rancher

RKE

Clusters created by RKE CLI have:

Cluster state stored as Configmap called full-cluster-state in namespace kube-system (cluster-state before RKE v0.2.0)
kubectl -n kube-system get configmap full-cluster-state

go dumps for Rancher 2.x support

If you are experiencing random crashing of your Rancher 2.x pods or docker container, sometimes Rancher support will ask you to take a go routine dump or a memory dump. Below are the commmands you need to run inside of your Rancher container or pod.

#exec into Rancher pod or container first
mkdir dumps
curl localhost:6060/debug/pprof/goroutine -o dumps/goroutine
curl localhost:6060/debug/pprof/heap -o dumps/heap
curl localhost:6060/debug/pprof/threadcreate -o dumps/threadcreate
curl localhost:6060/debug/pprof/block -o dumps/block
curl localhost:6060/debug/pprof/mutex -o dumps/mutex

lssh function

Changelog

06.03.2020 - added zsh function that works for WSL
04.21.2020 - updated mac os x code to work with zsh and improved instructions.
01.02.2020 - added Windows Subsystem for Linux and broke out each OS into its own section for easy copy and paste
12.10.2019 - added support for Rancher 1.6 tar.gz files (requires gtar on mac)
12.06.2019 - made command lazier by not requiring user to paste the IP.

Description

Quick bash function to make my life easier when sshing into Rancher nodes. Make sure to update your default web browser download directory by modifying line 2 of the script. For mac: brew install findutils

Cloud-Init Debugging

Cloud-init combined with terraform can be a powerful tool to provision instances on startup. Debugging scripts that are run by cloud-init however are not the easiest to debug.

Logs

Usually on an Ubuntu machine a lot of what is happening can be found in the syslog

cat /var/log/syslog

Vagrant Cloud API Example

Create box

curl \
  --header "X-Atlas-Token: $VAGRANT_CLOUD_TOKEN" \
  --header "Content-Type: application/json" \
  https://app.vagrantup.com/api/v1/boxes \
  --data '{"username":"justin_hashicorp", "name":"test"}'

	#!/bin/sh

	PUBLIC_IP=$(curl ifconfig.io)

	# export INSTALL_RKE2_VERSION="v1.20.5+rke2r1"

	curl -sfL https://get.rke2.io \| sh -

	provider_id="$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)/$(curl -s http://169.254.169.254/latest/meta-data/instance-id)"

	#!/bin/bash
	# Execute this script in a cattle-node-agent container
	# Successful websocket handshake is indicated by following line being printed:
	# HTTP/1.1 101 Switching Protocols

	TOKEN=$(cat /cattle-credentials/token)
	URL="$(cat /cattle-credentials/url)/v3/connect/register"

	echo "Using TOKEN: $TOKEN"
	echo "Using URL: $URL"

	#!/bin/sh
	# Backup your data
	# Use at your own risk
	# Usage ./extended-cleanup-rancher2.sh
	# Include clearing all iptables: ./extended-cleanup-rancher2.sh flush
	docker rm -f $(docker ps -qa)
	docker rmi -f $(docker images -q)
	docker volume rm $(docker volume ls -q)
	for mount in $(mount \| grep tmpfs \| grep '/var/lib/kubelet' \| awk '{ print $3 }') /var/lib/kubelet /var/lib/rancher; do umount $mount; done
	cleanupdirs="/etc/ceph /etc/cni /etc/kubernetes /opt/cni /opt/rke /run/secrets/kubernetes.io /run/calico /run/flannel /var/lib/calico /var/lib/etcd /var/lib/cni /var/lib/kubelet /var/lib/rancher/rke/log /var/log/containers /var/log/pods /var/run/calico"

	{
	"hosts": ["unix:///var/run/docker.sock", "tcp://0.0.0.0:2375"],
	"registry-mirrors": [],
	"insecure-registries": [
	"172.16.197.20:5000"
	]
	}

	a4b.amazonaws.com
	access-analyzer.amazonaws.com
	account.amazonaws.com
	acm-pca.amazonaws.com
	acm.amazonaws.com
	airflow-env.amazonaws.com
	airflow.amazonaws.com
	alexa-appkit.amazon.com
	alexa-connectedhome.amazon.com
	amazonmq.amazonaws.com