Skip to content

Instantly share code, notes, and snippets.

@alban
Last active January 31, 2019 10:21
Show Gist options
  • Save alban/6b6eee36e042d947c0c550b0dacced52 to your computer and use it in GitHub Desktop.
Save alban/6b6eee36e042d947c0c550b0dacced52 to your computer and use it in GitHub Desktop.
kubectl-trace demo FOSDEM

To enable cgroup-v2 in a Kubernetes cluster.

To enable the hybrid mode on Flatcar Linux:

  • sudo vi /usr/share/oem/grub.cfg
  • set linux_append="...... systemd.unified_cgroup_hierarchy=false systemd.legacy_systemd_cgroup_controller=false"
  • sudo reboot

To enable this in Docker:

$ echo 'DOCKER_OPTS="--exec-opt native.cgroupdriver=systemd"' |  sudo tee -a /run/metadata/torcx

Or alternatively:

$ sudo mkdir  /etc/systemd/system/docker.service.d
$ printf '[Service]\nEnvironment=DOCKER_OPTS="--exec-opt native.cgroupdriver=systemd"\n' | sudo tee /etc/systemd/system/docker.service.d/10-docker.conf

And then:

$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
$ sudo docker run -ti --rm busybox cat /proc/self/cgroup |grep ^[01]:
1:name=systemd:/system.slice/docker-5631ac3634cfd17a2ae5d9068c4121fbe2f392020b9db3d780e571c6b8954db3.scope
0::/system.slice/docker-5631ac3634cfd17a2ae5d9068c4121fbe2f392020b9db3d780e571c6b8954db3.scope

If you use the kubelet on the node, ensure it is started with --cgroup-driver=systemd:

vim /etc/systemd/system/kubelet.service
...

If you use containerd on the node, you need:

sudo mount -o remount,rw /run/torcx/unpack/
printf '\n\n[plugins.cri]\nsystemd_cgroup = true\n' | sudo tee -a /run/torcx/unpack/docker/usr/share/containerd/config.toml
sudo sed -i 's/^disabled_plugins.*$/disabled_plugins = []/g' /run/torcx/unpack/docker/usr/share/containerd/config.toml
sudo systemctl restart containerd

(it will work for processes started in the container, but not for processes entering via kubectl exec since containerd will not setup the cgroup-v2 there)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment