With its latest reales qemu added the Venus patches so that virtio-gpu now support venus encapsulation for vulkan. This is one more piece to the puzzle towards full Vulkan support.
An outdated blog post on clollabora described in 2021 how to enable 3D acceleration of Vulkan applications in QEMU through the Venus experimental Vulkan driver for VirtIO-GPU with a local development environment. Following up on the outdated write up, this is how its done today.
Let's start with the brief description of the projects mentioned in the post & extend them:
- QEMU is a machine emulator
- VirGL is an OpenGL driver for VirtIO-GPU, available in Mesa.
- Venus is an experimental Vulkan driver for VirtIO-GPU, also available in Mesa.
- Virglrenderer is a library that enables hardware acceleration to VM guests, effectively translating commands from the two drivers just mentioned to either OpenGL or Vulkan.
- libvirt is an API for managing platform virtualization
- virt-manager is a desktop user interface for managing virtual machines through libvirt
Merged Patches:
- 2024-08-14 venus: make cross-device optional merged in mesa 24.2
- 2024-11-25 KVM: Stop grabbing references to PFNMAP'd pages merged in linux 6.13
- 2024-11-12 Support blob memory and venus on qemu merged in qemu 9.2.0
Work in progress:
- libvirt Add support for more virtio-vga-gl arguments #638
- virt-manager Add support for Venus / Vulkan VirtIO-GPU driver #362
Make sure you have the proper version installed on the host:
- linux kernel >= 6.13 built with CONFIG_UDMABUF
- working Vulkan and kvm setup
- qemu >= 9.2.0
- virglrenderer with enabled venus support
- mesa >= 24.2.0
You can verify this like so:
$ uname -r
6.13.0
$ ls /dev/udmabuf
/dev/udmabuf
$ ls /dev/kvm
/dev/kvm
$ qemu-system-x86_64 --version
QEMU emulator version 9.2.0
Copyright (c) 2003-2024 Fabrice Bellard and the QEMU Project developers
Check your distros package sources how they build virglrenderer.
For Vulkan to work you need the proper drivers to be installed for your graphics card.
To verfiy your setup, install vulkan-tools. Make sure mesa >= 24.2.0 and test vkcube:
$ vulkaninfo --summary | grep driverInfo
driverInfo = Mesa 24.2.3-1ubuntu1
driverInfo = Mesa 24.2.3-1ubuntu1 (LLVM 19.1.0)
...
$ vkcube
Selected GPU x: ..., type: ...
If your distro doesn't (yet) ship and updated version of qemu, you can build it yourself from source:
wget https://download.qemu.org/qemu-9.2.0.tar.xz
tar xvJf qemu-9.2.0.tar.xz
cd qemu-9.2.0
mkdir build && cd build
../configure --target-list=x86_64-softmmu \
--enable-kvm \
--enable-opengl \
--enable-virglrenderer \
--enable-gtk \
--enable-sdl
make -j4
The configuration step will throgh errors if packages are missing. Check the qemu wiki for further info what to install: https://wiki.qemu.org/Hosts/Linux
Create an image & fetch the distro of your choice:
ISO=ubuntu-24.10-desktop-amd64.iso
wget https://releases.ubuntu.com/oracular/ubuntu-24.10-desktop-amd64.iso
IMG=ubuntu-24-10.qcow2
qemu-img create -f qcow2 $IMG 16GRun a live version or install the distro
qemu-system-x86_64 \
-enable-kvm \
-M q35 \
-smp 4 \
-m 4G \
-cpu host \
-net nic,model=virtio \
-net user,hostfwd=tcp::2222-:22 \
-device virtio-vga-gl,hostmem=4G,blob=true,venus=true \
-vga none \
-display gtk,gl=on,show-cursor=on \
-usb -device usb-tablet \
-object memory-backend-memfd,id=mem1,size=4G \
-machine memory-backend=mem1 \
-hda $IMG \
-cdrom $ISO
Adjust the parameters accordingly:
- smp: number of cpu cores
- m: RAM
- hostmem,size: VRAM
Install mesa-utilites and vulkan-tools to test the setup:
$ glxinfo -B
$ vkcube
Selected GPU x: ..., type: ...
If the deive is llvmpipe somehting is wrong. The device should be virgl (...).
- (host) add
-d guest_errorsto show error messages from the guest - (guest) try installing vulkan virtio drivers and mesa
- check the original blog post
This is how you do it on Ubuntu
Install mainline: https://github.com/bkw777/mainline
sudo add-apt-repository ppa:cappelikan/ppa
sudo apt update
sudo apt install mainline
find the latest kernel (>= 6.13), at the time of writing 6.13 is a release candidate, so include those:
$ mainline check --include-rc
Install kernel:
$ sudo mainline install 6.13-rc1
Verfify installed kernels:
$ mainline list-installed
mainline 1.4.10
Installed Kernels:
linux-image-6.11.0-13-generic
linux-image-generic-hwe-24.04
linux-image-unsigned-6.13.0-061300rc1-generic
mainline: done
reboot into new kernel
verify running kernel
$ uname -r
6.13.0-061300rc1-generic
the ubuntu package is not compiled with the proper flags.
If installed remove it: $ sudo apt-get remove libvirglrenderer-dev
download, build & install from source with venus enabled
wget https://gitlab.freedesktop.org/virgl/virglrenderer/-/archive/1.1.0/virglrenderer-1.1.0.tar.gz
sudo apt-get install python3-full ninja-build libvulkan-dev libva-dev
python3 -m venv venv
venv/bin/pip install meson
venv/bin/meson build -Dvideo=true -Dvenus=true
ninja -C build
ninja install
install qemu >= 9.2.0, at the time of writing ubuntu has not yet packaged it
Install build depdencies: https://wiki.qemu.org/Hosts/Linux
sudo apt-get install build-essential pip libslirp-dev slirp
sudo apt-get install git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ninja-build
sudo apt-get install git-email
sudo apt-get install libaio-dev libbluetooth-dev libcapstone-dev libbrlapi-dev libbz2-dev
sudo apt-get install libcap-ng-dev libcurl4-gnutls-dev libgtk-3-dev
sudo apt-get install libibverbs-dev libjpeg8-dev libncurses5-dev libnuma-dev
sudo apt-get install librbd-dev librdmacm-dev
sudo apt-get install libsasl2-dev libsdl2-dev libseccomp-dev libsnappy-dev libssh-dev
sudo apt-get install libvde-dev libvdeplug-dev libvte-2.91-dev libxen-dev liblzo2-dev
sudo apt-get install valgrind xfslibs-dev
sudo apt-get install libnfs-dev libiscsi-dev
build and run as described
-- work in progress --
Currently this is work in progress, so there is no option to add vulkan support in virt-manager. There are no fields to configure this. Also xml doesnt work, because libvirt doesn't know about these options either, so xml validation fails. There is however an option for QEMU command-line passthrough which bypasses the validation.
If you setup a default machine with 4G of memory, you can do this:
<qemu:commandline>
<qemu:arg value="-device"/>
<qemu:arg value="virtio-vga-gl,hostmem=4G,blob=true,venus=true"/>
<qemu:arg value="-object"/>
<qemu:arg value="memory-backend-memfd,id=mem1,size=4G"/>
<qemu:arg value="-machine"/>
<qemu:arg value="memory-backend=mem1"/>
<qemu:arg value="-vga"/>
<qemu:arg value="none"/>
</qemu:commandline>Which gives this error:
qemu-system-x86_64: virgl could not be initialized: -1
Changing the number from 4G to 4194304k (same as memory) leds to this error:
qemu-system-x86_64: Spice: ../spice-0.15.2/server/red-qxl.cpp:435:spice_qxl_gl_scanout: condition `qxl_state->gl_draw_cookie == GL_DRAW_COOKIE_INVALID' failed
to be further investigated.
If you have not patched the kernel on the host as indicated above I'm surprised Vulkan test applications (vkcube and vkgears) are showing you anything at all with an NVIDIA card.