Eric Ernst egernst

Description

qboot is a simple x86 firmware that can boot Linux.

It is Paolo Bonzini's answer to the Clear Containers claim that QEMU takes too long to boot a Linux kernel.

qboot only works as a QEMU firmware as it's bound to QEMU's fw_cfg NVRAM firmware configuration device emulation mode.

Containerd runtimes

A runtime in containerd does not stand at the runc level, but at the platform one. There are separate Linux and Windows runtimes, and they register as plugins:

const (
	runtimeName    = "linux"
	configFilename = "config.json"

Hot plugging devices using ACPI, SHPC and native

This gist is to show different ways to hot plug devices in a Virtual Machine

Set up the environment

Create a device mapper

dd if=/dev/zero of=devmap.img count=1 bs=50M

Create a functional Clear Container image using osbuilder

sudo -E FS_TYPE="ext4" USE_DOCKER=1 DISTRO=clearlinux AGENT_INIT=no EXTRA_PKGS="dbus-bin dbus-autostart util-linux-bin p11-kit-bin bash shadow ca-certs dist-pam-configs xz-bin tar-bin grep-bin sed-bin pigz-bin iproute2-bin procps-ng-bin psstop-bin htop-bin curl nano make-bin coreutils-bin pciutils strace-bin e2fsprogs-extras" make -e rootfs
sudo -E FS_TYPE="ext4" USE_DOCKER=1 DISTRO=clearlinux make -e image

WARNING: Comment "The list of systemd units and files that are not needed in Kata Containers" in image-builder/image_builder.sh

Running Openshift on top of Clear Containers 3.0

Overview

This documents how to get Openshift 3.6 on top of Kubernetes running with Clear Containers. An ansible playbook is used to provision a Fedora server 25 machine. Clear Containers is then installed and a basic test of deploying Clear Container and runc based pods is detailed.

Openshift

OpenShift is a public cloud application development and hosting platform which builds on top of the orchestration provided by Kubernetes.

Tracing QEMU-KVM Interactions

But default in linux you can figure out how many times and for what reasons there is a VM Exit from a VM into the kvm kernel module. However given the ubiquity of vhost and the ability of kvm to emulate most device models directly in the kernel, most of those VM exits do not result in a transition from host kernel into the QEMU. The transitions from VM -> kvm -> QEMU are typically the most expensive.

Here we try to figure out how many of the VM Exits result in the invocation of QEMU.

Tracking VM-KVM Interactions

This can be done very simply with perf

Install FC25 server

Password less root ssh

dnf install git ansible

git clone https://github.com/runcom/openshift-origin-cri-o

git clone https://github.com/openshift/openshift-ansible

	# XXX: Warning: this file is auto-generated from file "config/configuration.toml.in".

	[hypervisor.qemu]
	path = "/usr/bin/qemu-lite-system-x86_64"
	kernel = "/usr/share/clear-containers/vmlinuz.container"
	image = "/usr/share/clear-containers/clear-containers.img"
	machine_type = "pc"
	# Optional space-separated list of options to pass to the guest kernel.
	# For example, use `kernel_params = "vsyscall=emulate"` if you are having
	# trouble running pre-2.15 glibc

	kubeletArguments:
	container-runtime:
	- remote
	container-runtime-endpoint:
	- /var/run/crio.sock
	experimental-cri:
	- 'true'
	image-service-endpoint:
	- /var/run/crio.sock
	node-labels:

	Hotplugging cpu:

	Start QEMU with QMP socket available and with startup amount of CPUs less than maxcpus:
	./qemu-system-x86_64 -qmp unix:/tmp/qmp-sock,server,nowait -smp 2,maxcpus=8

	Connect to qmp socket and add cpu:
	socat /tmp/qmp-sock -
	{ "execute": "cpu-add", "arguments": { "id": 2 } }

	Hotpluggable Cpu's can be queried as: