KlusterD Attempt 1

READEME.md

Hack 1 - Replace scheduler

Process #1 - build replacement scheduler

• NOTE - if control changes IPs, code will need updates (for api, currently not ENV or file)
• On a machine with Docker
• git clone http://github.com/ronaldpetty/scheduler.git (derived from kelseyhightower/scheduler)
• cd ~/scheduler
• git checkout tls_healthz
• build image - sudo docker build -t k8s.gcr.io/kube-scheduler:v1.22.4 --no-cache .
• extract image - sudo docker save k8s.gcr.io/kube-scheduler:v1.22.4 -o hightower.tar
• copy image over to control-plane-1 (eg. scp’d to my laptop; copy to S3, curl to control plane)

Process #2 - replace scheduler

• safety first - think backups
• cp scheduler log file (so we can spoof it later) - cp /var/log/containers/kube-scheduler-* .)
• back up scheduler manifest - cp /etc/kubernetes/manifests/kube-scheduler.yaml kube-scheduler.yaml
• remove scheduler pod - rm /etc/kubernetes/manifests/kube-scheduler.yaml (confirm via get pod or ps)
• remove image(s)
  • ctr -n k8s.io image ls | grep scheduler
  • ctr -n k8s.io image rm k8s.gcr.io/kube-scheduler:v1.22.4
  • ctr -n k8s.io image rm k8s.gcr.io/kube-scheduler@sha256:35e7fb6d7e570caa10f9545c46f7c5d852c7c23781efa933d97d1c12dbcd877b
• upload - ctr -n k8s.io image import hightower.tar (confirm ctr -n k8s.io image ls | grep scheduler)
• replace pod - cp kube-scheduler.yaml /etc/kubernetes/manifests/kube-sceduler.yaml (confirm get pod -- takes time due to probe)

Process #3 - cover tracks

• append (dont replace) log file (the name will change so be careful)
• delete files (new scheduler tar)
• delete kubectl / ctr history
• replace other pods (not etcd) so ages match

To Undo Hack:

• remove scheduler manifest
• remove images (see above)
• replace scheduler manifest

Hack 2 - limit number of pods in a namespace

Process #1 - Set hard limit on number of pods

• kubectl create quota default -n default --hard=pods=2 (delete or raise to restore prior behavior)

To Undo Hack:

• Delete the default quota: kubectl delete quota default -n default

Hack 3 - revoke cm ability to create a pod

Process #1 - Edit the replicaset controller cluster role

• kubectl edit clusterrole system:controller:replicaset-controller
  • Remove - create under pods

To Undo Hack:

• Edit the replicaset controller cluster role: kubectl edit clusterrole system:controller:replicaset-controller
  • Add - create under pods