How to use debug containers

Following K8s feature-gate must be enabled in kind (kubeadm) config file to access the feature

featureGates:
  EphemeralContainers: true

First we start debug / emphemeral container and attach it to contour shutdown-manager container.
The target container is "distroless": it has just single binary /bin/contour and no shell at all. Our debug container is just standard alpine image.

$ kubectl -n projectcontour debug --target=shutdown-manager envoy-dwdcp -it --image=alpine
Targeting container "shutdown-manager". If you don't see processes from this container it may be because the container runtime doesn't support this feature.
Defaulting debug container name to debugger-hvgcj.
If you don't see a command prompt, try pressing enter.
~ $

The debug container is sharing process namespace with the target shutdown-manager container

~ $ ps -ef
PID   USER     TIME  COMMAND
    1 nobody    0:00 /bin/contour envoy shutdown-manager
   25 nobody    0:00 /bin/sh
   31 nobody    0:00 ps -ef

But we have our own filesystem with all needed debug tools available.

~ $ cat /etc/alpine-release
3.14.2

The process is restricted by the security context. I don't know if it is possible to set security context and runAsUser: 0 via kubectl debug.

~ $ id
uid=65534(nobody) gid=65534(nobody)

The filesystem of the target container is accessed via /proc/<pid>/root/

~ $ ls -l /proc/1/root/
total 12
drwxrwxrwx    2 root     root          4096 Oct  8 13:55 admin
drwxr-xr-x    2 root     root          4096 Oct  8 09:52 bin
drwxr-xr-x    5 root     root           360 Oct  8 13:55 dev
drwxr-xr-x    2 root     root          4096 Oct  8 13:55 etc
dr-xr-xr-x  627 root     root             0 Oct  8 13:55 proc
dr-xr-xr-x   13 root     root             0 Oct  8 13:55 sys
~ $ ls -l /proc/1/root/bin/contour 
-rwxr-xr-x    1 root     root      41590784 Oct  8 09:52 /proc/1/root/bin/contour

tsaarni/kubernetes-debug-containers.md

How to use debug containers

Links