Skip to content

Instantly share code, notes, and snippets.

@TBeijen
Created February 11, 2025 14:39
Show Gist options
  • Save TBeijen/028556fea2a62623f455590d82dabb46 to your computer and use it in GitHub Desktop.
Save TBeijen/028556fea2a62623f455590d82dabb46 to your computer and use it in GitHub Desktop.
CKS Study

Commands

Misc

Linux misc:

# Where is the file
find / -name "docker.sock" 2>/dev/null

# Remove user ubuntu from group floppy
gpasswd -d ubuntu floppy

Checking connectivity:

nc -v <ip-address> <port>

# timeout 1 sec
nc -v -w1 <ip-address> <port>
# nc -v -w1 1.2.3.4 80
# nc: connect to 1.2.3.4 port 80 (tcp) timed out: Operation now in progress

# curl with max timeout 1 sec
curl -m 1 http://my-svc.my-ns.svc.cluster.local

# If needing to specify what to resolve
curl --resolve "hello-world.example:80:$( minikube ip )" -i http://hello-world.example

# Checksums
echo '9016f6048ff9827ef58934e98f28a8026634c10b4e6fcc1df49451038a23a9aa kube-apiserver' | 
sha256sum --check
sha512sum binaries/* > checksums.txt

CRICTL

crictl ps 
crictl ps -a
watch -n2 crictl ps -a
# find container based on e.g. falco container_id
crictl ps -id f86cd629e71c
# find pod info based on pod id found in container info
crictl pods -id cab6dafd045d5

Falco

# Asked for formatting displaying 'time in nanoseconds'
falco --list |grep nano

# Unbuffered
falco -U

# Unbuffered, events of particular kind
falco -U | grep "Package management process"

Docker

man Dockerfile
man docker-run

# Run in PID kernel namespace of other container
docker run -d --name=c2 --pid=container:c1 nginx:alpine sleep infinity

# Run in process host namespace (e.g. run htop in container, showing entire host)
docker run --pid=host some-image-with-htop htop

TLS OpenSSL

cd /etc/kubernetes/pki
# show certificate
openssl x509 -text -noout -in ca.crt

# Check if certificate is signed by certificate authority
openssl verify -verbose -CAfile ca.crt apiserver.crt

# Sign certificate with Kubernetes CA (creates valid cert to set up user in kubeconfig)
openssl x509 -req -in /root/60099.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out /root/60099.crt -days 500

AppArmor

apparmor_status
apparmor_parser <profile-file>

Trivy

trivy image httpd:2.4.39-alpine |grep -E 'CVE-2016-9841|CVE-this|CVE-that'

ETCD (encryption)

# -n: Create 16-char length base64 pw without newline being appended to value
echo -n "this-is-very-sec" |base64

# re-encrypt all secrets
k get secret -A -o yaml | k replace -f -

# read secret from etcd
# Lookup connection params from docs
ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/apiserver-etcd-client.crt --key=/etc/kubernetes/pki/apiserver-etcd-client.key --endpoints=127.0.0.1:2379 get /registry/secrets/one/s2

Processes

# from name to pod, to container, to command, to PID
crictl pods --name collector1
crictl ps --pod a61e29997e607
crictl inspect e18e766d288ac | grep args -A1
ps aux |grep <command-executed-in-pod>

# trace ls command
strace ls

# strace with summary of calls and number of calls
strace -cw ls

# attach to pid 1945, following forks
# break after while (ctrl+c), then view summary
strace -cw -p 1945 -f

ls -al /proc/1945
cd /proc/1945

# root filesystem of container 
ls -al /proc/1945/root/

# File descriptors
ls -al /proc/1945/fd/

# The executable
ls -al /proc/1945/exe

# Apparmor settings
cat /proc/1/attr/current

# Examine contents of file descriptors (e.g. name of k create secret generic <name-of-secret>)
cat ./fd/10 | grep <name-of-secret>
# Binary file (standard input) matches
cat ./fd/10 | strings | grep <name-of-secret>
cat ./fd/10 | strings | grep <name-of-secret> -B5 -A15

# Exploring capabilities
cat /proc/1234/status | grep Cap
# decode CapBnd set:
capsh --decode=0000003fffffffff

# env
cat environ

# Examine process tree including pids
pstree -p

# Find process listening on port 1234
#
# Using netstat, to install
apt install net-tools
#
# -u adds UDP
# -a all sockets (default: connected)
netstat -tulpan | grep 1234
netstat -plnt | grep 1234

# using lsof
lsof -i :1234

Package management

# Update repositories
apt update
apt-get update

# Search package to install
apt-cache search kubetail
apt install kubetail

# List & examine
apt list
apt show kube-bench

# Uninstall
apt remove kube-bench

Linting

k kustomize overlays/accept | kube-linter lint -
k kustomize overlays/accept | kubesec scan -

Kubectl

kubectl get --raw "/api/v1/nodes/cks8930-node1/proxy/configz" | jq

Docs

Docs that can be used during exam (source)

Topics added oct 2024

https://blog.techiescamp.com/cks-exam-update/

New tools:

Topics

Admission controller (ImagePolicyWebhook)

Docs: search 'admission' or 'ImagePolicyWebhook' -> reference / API access control / Admission controllers (containing list of all admission plugins, including ImagePolicyWebhook)

Moving parts:

  • Api server arg: --admission-control-config-file

  • Api server arg: --enable-admission-plugins, e.g. NodeRestriction,ImagePolicyWebhook

  • Volume mount kube api server: Containing the admission control config file, but also

    • kubeconfig referenced in policy config file
    • certs referenced in kubeconfig file (used to authenticate to webhook and validate webhook)

AppArmor

Moving parts:

  • AppArmor profiles, load via apparmor_parser and check installed via apparmor_status
  • Pod spec securityContext.appArmorProfile
  • Syslogs, cat /var/log/syslog |grep -i apparmor
  • Proc attr, from within container: kubectl exec nginx-pod -- cat /proc/1/attr/current

SecComp

Moving parts:

Tutorial: https://kubernetes.io/docs/tutorials/security/seccomp/

Audit logs

Docs: search 'audit' -> result 'debug cluster / audit'

Moving parts:

  • Kube apiserver arg --audit-policy-file=<policy-file>
  • Kube apiserver arg --audit-log-path-<log-file>
  • Policy file contents, apiVersion: audit.k8s.io, kind: Policy

Cilium

https://docs.cilium.io/

Services or endpoints:

Currently Kubernetes Services without a Selector are supported when defined by their name and namespace or label selector. For services backed by pods, use Endpoints Based rules on the backend pod labels.

Concepts:

  • CiliumNetworkPolicy
  • CiliumClusterwideNetworkPolicy
  • DefaultAllow (ingress/egress), switched to DefaultDeny once having matching rule, unless enableDefaultDeny
  • Layer 3, 4, 7
  • NodeSelector, only usable in CiliumClusterwideNetworkPolicy, see Host Policies
  enableDefaultDeny:
    egress: false
    ingress: false

ETCD encryption

Docs: search 'etcd encrypt rest' -> https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#verifying-that-data-is-encrypted

(contains registry get example)

Moving parts:

  • Encryption config. Kind: EncryptionConfiguration

  • Kube apiserver manifest:

    • --encryption-provider-config=<file-path> arg
    • VolumeMount
    • Volume
  • Within encryption config: Providers.

    • Can be multiple
    • First key of first provider is used for encryption
    • All providers and keys of providers are attempted in order for decryption
    • idendity provider does not encrypt/decrypt, use for being able to read unencrypted content
  • Know how to:

    • Configure (Mind the echo -n!!)
    • Check in etcd
    • Re-encrypt secrets

Falco

Moving parts:

  • Rules: /etc/falco/*, e.g. /etc/falco/falco_rules.yaml
  • Picks up rules without restart
  • systemctl status falco

gvisor

Keywords:

  • Kind: RuntimeClass
  • pod.spec.runtimeClassName

Moving parts:

  • Containerd config: /etc/containerd/config.toml
  • RuntimeClass handler: runsc
  • systemctl status containerd

Kube-bench

Moving parts:

  • Cli kube-bench: kube-bench run --targets=master --check='1.3.2'
  • Targets in cfg dir, e.g. /etc/kube-bench/cfg/cis-1.20/: controlplane,etcd,master,node,policies

Tip:

  • kube-bench run > kube-bench-result.txt, then easily grep through that file for various issues

Kubeadm: Reconfiguring cluster components

Docs:

Moving parts (example: kubelet):

  • Configmap kube-system:kubelet-config
  • Cli kubeadm
  • Local kubelet config file, determine via ps aux |grep kubelet, e.g. /var/lib/kubelet/config.yaml
  • Cli systemctl

Kubeadm: Upgrading cluster

Docs: search 'kubeadm upgrade'. Follow steps.

Open Policy Agent Gatekeeper (OPA)

Moving parts:

  • ConstraintTemplate, defines a new constraint CRD kind and parameters to configure the CRD, e.g.:
  • K8sImageRegistry. Create custom resources of this kind, and populate the parameters. e.g.:
apiVersion: templates.gatekeeper.sh/v1beta1
kind: K8sImageRegistry
metadata:
  name: image-registry-whitelist
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  parameters:
    allowedRegistries:
      - "trusted-registry.example.com"
      - "another-trusted-registry.example.com"

Pod Security Standards (PSS)

Docs: Search

Moving parts:

  • Admission controller PodSecurity
  • Default settings at cluster level (optional). kind: PodSecurityConfiguration, Defaults & exemptions.
  • Namespace labels for enforcing at namespace level
  • SecurityContext settings mandated by the PSS
  • Enforce mode applies only to pods (blocks), audits and warns at workload objects (deployment, job, etc.)

SBOM tools

Moving parts (various tools)

  • bom: Create SPDX SBOMs. Example: bom generate --image=<image> --format=json -o <destination-file
  • trivy: Generate CycloneDX SBOM (and other formats). Example: trivy image --format=cyclonedx --output <destination-file>
  • trivy: Scan SPDX SBOM. Example: trivy sbom <sbom-json-file> --format=json -o <destination-file>

Seccomp

TODO

Trivy

Purpose:

  • Container scanning
  • Generating and parsing of SBOMs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment