Vagrant on Qubes-OS

(Vagrant)[https://www.vagrantup.com] "Development Environments Made Easy"

(Qubes-OS)[https://www.qubes-os.org] "A Reasonably Secure Operating System."

This is a guide on to using vagrant on qubes-os with qemu using the libvirt provider. Because qubes-os does not support nested virtualization, you'r stuck with emulation. If you want performance, use a system with a proper vagrant setup.

Template Setup

Currently, this only works with a (debian-9)[https://www.qubes-os.org/doc/template/debian/upgrade-8-to-9/] templatevm. fedora25 and debian8 have conflicting xen libraries.

From an appvm, download vagrant from https://vagrantup.com , check the sha256sum and gpg signature, and move it to your template vm. You can use gdebi to install it on the command line and pull in any dependencies.

Do the following steps as root.

In the template vm, we need all the packages for qemu, libvirt, and vagrant-libvirt. Dont forget to remove /home/user/QubesIncoming from the template after installing vagrant.

apt install qemu-kvm libvirt-clients libvirt-daemon-system \
  bash-completion debhelper gem2deb libvirt-dev pkg-config \
  rake libvirt-daemon ebtables dnsmasq libxslt-dev \
  libxml2-dev libvirt-dev zlib1g-dev ruby-dev virt-manager gdebi

gdebi vagrant-XXX.deb

Add user to the libvirt and qemu groups,

adduser user libvirt
adduser user libvirt-qemu

Add libvirt to persistent storage. see https://www.qubes-os.org/doc/bind-dirs/ for more on that.

mkdir -p /rw/config/qubes-bind-dirs.d
cat << EOF >> /rw/config/qubes-bind-dirs.d/50_user.conf    
binds+=( '/etc/libvirt' )
binds+=( '/var/lib/libvirt' )  
EOF

Do the following as user. This installs and sets the libvirt plugin to use qemu in emulation instead of virtualization mode.

vagrant plugin install vagrant-libvirt

cat << EOF > ~/.vagrant.d/Vagrantfile
Vagrant.configure("2") do |config|
  config.vm.provider "libvirt" do |libvirt|
    libvirt.driver = "qemu"
    libvirt.cpu_mode = "custom"
    libvirt.cpu_model = "qemu64"
  end
end
EOF

Finally, add these two lines to users ~/.bashrc

export LIBVIRT_DEFAULT_URI="qemu:///system"
export VAGRANT_DEFAULT_PROVIDER=libvirt

Thats all you need in the template vm. Shutdown so appvms can inherit the changes.

Appvm Setup

Set the private storage max size higher, at least 20gigs. you'll want more memory too.

If you want to use an existing appvm, you'll need to enable persistent storage of libvirt and make all the user changes above.

A brief diversion to virt-manager

Once rebooted try virt-manager. when you first start it, it will complain about not being able to connect to xen. Delete that connection by right clicking on "xen" under "Name" and delete. Then make a new one with File/"Add connection" and qemu/kvm. The generated uri should be "qemu:///system"

Virt-manager is not strictly needed by vagrant, but it makes it easier to see whats going on, and to use the console of desktop vagrant boxes. You can also use virt-manager for remote sessions, to hopefully faster libvirt hosts.

Back to Vagrant

Heres a sample Vagrantfile you can try. its an updated one from the vagrant-libvirt readme. make a folder and a file called Vagrantfile with this in it.

Vagrant.configure("2") do |config|
  config.vm.box = "fedora/26-cloud-base"
end

Then

vagrant up

and in a few minutes, the prompt should return and you can "vagrant ssh".

time for vagrant up:

real        2m41.135s
user        0m2.956s
sys         0m0.345s

time on a real linux box:

real        0m37.065s
user        0m7.900s
sys	        0m2.226s

Other Approaches

Virtualbox in an HVM. Limited to 32 bit guests, also emulation instead of virtualization.
LXC provider. Should be the same as any other linux. Havent tried, as its limited to linux.
Ssh to a linux box running libvirt with kvm. This is what i actually do. Faster and you get the benefits of remote tmux.

datatypevoid/vagrant-qubes.md