Canh Ngo canhnt

Check numbers of templatecontent custom resource objects in the etcd DB:

etcdctl get  /registry/management.cattle.io/templatecontents --keys-only --prefix | sed '/^\s*$/d' | wc -l

A bug in Rancher prior v2.0.8 and v2.1.0 added a lot of html files in templatecontent objects, which fills up etcd db and may cause K8s master nodes unavailable.

Delete all templatecontent custom resource objects:

etcdctl del /registry/management.cattle.io/templatecontents/ --prefix

Get current cluster.yaml

kc get cm cluster-state -n kube-system -o jsonpath='{.data.cluster-state}'

Get user-addon settings:

kc get cm  -n kube-system rke-user-addon -o jsonpath='{.data}'

Check free storage in gluster nodes

for i in $(seq 1 9); do node="829k8xlstorage$i.company.org"; echo "$node checking:"; ssh $node 'df -h'; done 2>/dev/null | grep -E '829k8xlstorage|centos-root'

If any node free is lower than 18%, delete logs:

rm -f /var/log/glusterfs/gluster*
rm -f /var/log/glusterfs/bricks/*

	#!/usr/bin/env bash

	# Find invalid LVs that exist in physical (via lvscan) but not managed by gluster

	vgs=$1
	host_address='10.165.210.xxx'
	valid_lvs=$(gluster volume info \| grep $vgs \| grep ${host_address} \| sed -e 's/.brick_\(.[a-z0-9]\)\/brick/\1/')

	all_lvs=$(lvscan \| grep $vgs \| grep 'brick_' \| awk -F'brick_' '{ print substr($2,0,32)}')

	# Ref https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/maintenance.md#space-quota
	export ETCDCTL_API=3
	etcdctl endpoint status --write-out="json"
	etcdctl endpoint status --write-out="table"

	etcdctl compact $rev
	etcdctl defrag

	etcdctl endpoint status --write-out="table"
	etcdctl alarm list

	openssl req -new -key cngo.key -out cngo-csr.pem -subj "/CN=cngo/O=app1/O=infra-team"

	openssl x509 -req -in cngo-csr.pem -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cngo.crt -days 730

	export AWS_CA_BUNDLE=$path_private_ca_cert

	aws --endpoint-url https://$minioserver s3 cp $filename s3://$bucketname