The instructions cover cloning and building Linux kernel and the useful qemu wrapper virtme-ng. After performing these actions, a virtual machine is started with the newly built Linux kernel and the current local rootfs - same user and files. This allows building and testing kernel modules and user-space apps with a custom/specific kernel.
The kernel compilation phase can be made very simple with virtme-ng itself, so be sure to check out its capabilities by running vng --help after installing it. Here the kernel building instructions more manual and detailed to radically reduce the build time by removing certain features you normally don't care about in such a setup.
While virtme-ng makes it easy to run a kernel on the local root filesystem for debugging, it is not your original system. In most cases today your Linux runs with systemd init, which is quite a complex environment. The virtme-ng prroject relaces your init with a simplified and small init written in Rust, so don't expect using systemd's features like systemctl. This environment won't be suitable for testing of kernel features or modules in relation to systemd and will not allow you to fully use your rootfs as usual.
Caution
In the following setup the kernel runs inside a VM, so if your kernel gets stuck it won't affect the physical machine. Nevertheless, the file system is the original one on your physical disk, so any changes are permanent. Treat it with care.
Note
Everything is done in CLI, no graphical environment is involved. Tested on:
- Native Debian 12 Bookworm, x86-64
- Windows WSL2, Ubuntu 22.04, x86-64
mkdir ~/work && cd ~/work
Have the necessary packages for kernel build installed. This is not a comprehensive list. Adapt to other package managers as required.
sudo apt update
sudo apt install git build-essential libncurses-dev bison flex libssl-dev libelf-dev bc dwarves
Your distro may have it in the package manager, check out the full instructions here.
git clone --recurse-submodules https://github.com/arighi/virtme-ng.git
cd virtme-ng
sudo apt install python3-pip python3-argcomplete flake8 pylint cargo rustc qemu-system-x86
BUILD_VIRTME_NG_INIT=1 pip3 install --break-system-packages --verbose -r requirements.txt .
Note
Use updated setuptools to avoid build errors: python3 -m pip install --upgrade setuptools
source ~/.profile
cd ~/work
Stable (check out https://kernel.org/ for stable version links):
wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.6.28.tar.xz
tar xf linux-6.6.28.tar.xz
cd linux-6.6.28
git ls-remote --tags https://github.com/torvalds/linux.git | grep 6.6$
git clone --branch v6.6 --depth 1 https://github.com/torvalds/linux.git
cd linux
This is where things become flexible. We'll be running in a VM with a very self-contained init system, so the kernel configuration should depend on what you want to test or debug.
To have your kernel as similar to the currently running one, a local config file must be obtained. Here are some examples (as this is distro-dependant), one should work on modern Linux distros:
cp /boot/config-$(uname -r) .configzcat /proc/config.gz > .configThen prepare it for the build, as the new kernel version is probably different:
make olddefconfig
For a quick start, to vastly improve kernel build time, the following removes many kernel features that may be skipped for the purpose of this build. Begin with either the full config like in option 1, or with a minimal one by running make defconfig. Then, two lists are provided in the attached disable.conf and enable.conf files. After creating or downloading them in the kernel directory, run the following commands to apply the changes.
Note
While many more features can be removed keeping the kernel functional in a virtual machine, curate the list depending on your goals or skip this entirely.
while read -r line; do scripts/config --disable "$line"; done < disable.conf
while read -r line; do scripts/config --enable "$line"; done < enable.conf
make olddefconfig
Tip
To change anything manually in the configuration at this point:
make menuconfig
This can take anywhere between 5 minutes and 2 hours depending on previous choices and the HW.
make -j$(nproc) && make modules
I prefer to install the modules locally and then reference this custom location in the kernel module Makefile, to keep things encapsulated. Instead you may choose to install them as usual by make modules_install.
mkdir -p modules && make modules_install INSTALL_MOD_PATH=modules
Do it in the current kernel directory. Most arguments are quite obvious, feel free to adjust as needed. If vng can't be found, verify you have souced the profile with source ~/.profile or call it directly with ~/.local/bin/vng.
vng --cpus 2 --memory 2G --rw --pwd --user $USER --name my_machine --net user
Tip
To exit the VM at any stage, press Ctrl+d or Ctrl+a followed by x. For more VM keyboard shortcuts see https://www.qemu.org/docs/master/system/mux-chardev.html
Kindly contribute to this guide by correcting mistakes, adding other distros you tested it on, and curating the kernel configs to diasble and enable. Good luck!