sboeuf

# The "crio" table contains all of the server options.
[crio]

# root is a path to the "root directory". CRIO stores all of its data,
# including container images, in this directory.
root = "/var/lib/containers/storage"

# run is a path to the "run directory". CRIO stores all of its state
# in this directory.

sboeuf

From 995119a1dc3161e13a9039625a33f9b630fcb1ef Mon Sep 17 00:00:00 2001
From: Sebastien Boeuf <[email protected]>
Date: Mon, 27 Feb 2017 11:46:40 -0800
Subject: [PATCH 0/4] Return unused memory to the host system

This series tries to solve a common issue in the cloud context, that
is being able to prevent any virtual machine running on a host system
from keeping unused resources and particularly unused memory.

Ballooning mechanism exits but it is not enough efficient as it waits

sboeuf

# Taint the master node to allow pods to be scheduled
sudo -E kubectl taint nodes --all node-role.kubernetes.io/master-

# Remove CNI stuff to make sure kube-dns will start
rm -r /var/lib/cni/networks/*

# Restart the cluster
sudo kubeadm reset --cri-socket /var/run/crio/crio.sock
sudo systemctl stop kubelet
sudo systemctl stop crio

sboeuf

sudo ./x86_64-softmmu/qemu-system-x86_64 \
    -nographic \
    -nodefaults \
    -L . \
    -machine virt,accel=kvm,kernel_irqchip,nofw \
    -smp sockets=1,cpus=4,cores=2,maxcpus=8 -cpu host \
    -m 2G,slots=3,maxmem=16G \
    -kernel /home/sebastien/workloads/linux_4_17/linux-4.17.14/arch/x86/boot/compressed/vmlinux.bin \
    -append 'console=hvc0 single iommu=false root=/dev/ram0' \
    -device virtio-serial-pci,id=virtio-serial0 \

sboeuf

Global concepts

Addressable space

Depends on the processor capabilities, which can be around 36~40 bits for recent Intel processors. If taking 39 bits as an example, this means the total addressable space will be 1 TiB of available addresses. The guest RAM is part of this addressable space, same as devices, PCI holes, ...

BAR (Base Address Register)

A base address register is part of the PCI configuration space of each PCI device, and it declares an extra memory region related to the device that can be found at this specific base address.

sboeuf

Prepare the host

Get the image clear-XXXXX-kvm.img

wget https://download.clearlinux.org/releases/30080/clear/clear-30080-kvm.img.xz
unxz clear-30080-kvm.img.xz

Add intel_iommu=on to the kernel boot parameters

mkdir mount_dir

sboeuf

In order to set the Internet connectivity for our cloud-hypervisor VMM, we need to manually setup a few things.

1. Set guest's IP address

# ip addr add 192.168.249.2/24 dev enp0s3

2. Set default route

# ip route add default via 192.168.249.1

sboeuf

This document is a list of ways we can properly test the network is functional when using vhost-user-net. The document assumes we'll be using vhost-user-bridge program from qemu/tests as the vhost-user-net daemon.

1. Communication between 2 VMs

The point in this case is to validate communication between 2 virtual machines by connecting each VM to its own vhost-user-net daemon. Each daemon read/write to each other ports, based on the socat port forwarding.

          +-----------+                      +-----------+
          |           |                      |           |
          |           |                      |           |
          |    VM1    |                      |    VM2    |

sboeuf

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKQpd8fWXinkn3TgdAbLb6hceUGNjRGbr0Cpj4h3q2jNd06GzKyVhbjZWkygvPyC/aMfpaW6OSIClfa6M3ToVWPnRf/fBvzhkrEXNpkbk0GCstKt1Rlu8E0dXQXMM4XAglib2X1tCwOP94o4fh3pfvrk/4eMWhqwghBQu1of1DavNchMkoHmuO9X3hMFH4ljs4/vCgeyiln5gRZQ6n7RqD1/Lg2ne4SEK9zFrrLu7y3pE64R/q3M2aWjoaA3OgwejVoIQ9OuTpmB4/TGj5DWHQnlYw68koWNQ0MHWChx0oi6ElJziXaXQOR7PUa1DJcSaL3KrIBsZ0wmMJIwyn94R/ sebastien@sboeuf-mobl3

sboeuf

diff --git a/init/main.c b/init/main.c
index 0ead83e86b5a..d040c58bf8cd 100644
--- a/init/main.c
+++ b/init/main.c
@@ -832,6 +832,8 @@ asmlinkage __visible void __init start_kernel(void)
        char *command_line;
        char *after_dashes;
 
+       outb(6, 0xf4);
+