How to run docker on Qubes-OS

Docker-on-qubes.md

Docker on Qubes

Since this page is apparently the top result on google, Heres a link to how to do it.

https://martingladdish.co.uk/technology/setting-up-docker-under-qubesos/

That page has more detail, but here are the instructions in case its down.

1. Install docker engine, following the instruction on https://www.docker.com. NOT DESKTOP as that wont work in Qubes (unless you enable nested virtualization)
2. Make the below /etc/docker/daemon.json
3. Add "dockerd &" to your appvms (NOT TEMPLATE) /rw/config/rc.local

/etc/docker/daemon.json

{
    "data-root": "/home/user/docker",
    "group": "user"
}

Tried this with the debian-11 template on August 23, 2023. Martin did this with fedora-33 in 2021.

If you have trouble, you can ask for help at https://forum.qubes-os.org/

To do this in Qubes-OS 3.x, look at the history of this page.

Qubes OS seems promising I am excited to try it out.

@xahare for Qubes 4 you need to do the bind dirs config in the appvm. I don't know if this changed with Qubes 4. I made some changes in my fork.

Thanks for the instructions. This was very helpful. I now have a Debian 10 based Docker TemplateVM.

Regarding the problem with Qubes 4, it looks like https://www.qubes-os.org/doc/bind-dirs/ says you need to use /etc/qubes-bind-dirs.d for TemplateVM-wide configurations, and /rw/config/qubes-bind-dirs.d is per AppVM.

@xahare @coderpatros actually the location of the qubes-bind-dirs.d moved to /usr/lib in r4.0 - if you add 50_user.conf there instead it should all work (it does for me on a debian-based template). The new instructions for r4.0 becomes:

sudo mkdir -p /usr/lib/qubes-bind-dirs.d # This may not be necessary - the directory already existed in my template
sudo cat << EOF > /usr/lib/qubes-bind-dirs.d/50_user.conf
binds+=( '/var/lib/docker' )
binds+=( '/etc/docker' )
EOF

Refer to https://www.qubes-os.org/doc/bind-dirs/#other-configuration-folders for all support configuration paths for bind-dirs.d.

For anyone encountering issues with package: aufs-dkms on install of docker-ce, run;

sudo apt-get -y install docker-ce --no-install-recommends

Reference: https://github.com/freedomofpress/securedrop/pull/5269/commits

/etc/qubes-bind-dirs.d is still the valid place to put this configuration in Qubes 4.x.

sudo mkdir -p /etc/qubes-bind-dirs.d
sudo tee /etc/qubes-bind-dirs.d50_user.conf << EOF > /dev/null
binds+=( '/var/lib/docker' )
binds+=( '/etc/docker' )
EOF

You must run Docker at least once in the template to have it create the folders, for instance docker ps, or create the folders yourself. The bind-dirs script skips non-existent binds.

Is this description still fine or is it better to use the official debian one: https://docs.docker.com/engine/installation/linux/docker-ce/debian/

@TobiasSchotter If your in qubes-os, you can use the official debian install in a standalone, or bind-dirs as above. I haven't tried it. I use a dedicated vagrant machine over ssh for all that now.

This is the first thread that pops-up when you google about Docker on Qubes, but I found this tutorial more helpfull;
https://martingladdish.co.uk/technology/setting-up-docker-under-qubesos/

configuring /etc/docker/daemon.json as follows

{ "data-root": "/home/user/docker", "group": "user" }

makes bind-dirs unneccesary, which is more reliable in my opinion.

I will try to contribute more as I learn, maybe make some tutorials myself.
As of now, just a noob trying to help other noobs.

@IOZZYS Thanks! Didn't know anyone still saw this old gist.

NOTE: For anyone looking for an alternative method which works-as-intended, see also gist:Install Docker on Qubes OS 4.2. The following are the condensed steps required, without the verbose explanations accompanying each step.

Since there is not a documented reliability concern for "bind-dirs" facility, mainly just a matter of "that's a whole lotta words, too bad I'm not readin' 'em" when it comes to documentation, immediately followed by struggles to understand why things are not working; here we provide "the Qubes way" of solving for both that does not leave "bind-dirs" in a broken state.

Configure TemplateVM

sudo apt install -y docker.io
sudo systemctl disable containerd
sudo systemctl disable docker
[[ $(getent group docker) == '' ]] && sudo groupadd docker;
sudo usermod -aG docker user;
[ ! -d /etc/docker ] && sudo mkdir -p /etc/docker && sudo chmod 755 /etc/docker;
[ ! -d /var/lib/docker ] && sudo mkdir -p /var/lib/docker && sudo chmod 710 /var/lib/docker;
sudo mkdir -p /etc/qubes-bind-dirs.d
sudo touch /etc/qubes-bind-dirs.d/30_docker.conf
sudo chmod 644 /etc/qubes-bind-dirs.d/30_docker.conf
echo "binds+=( '/var/lib/docker' )" | sudo tee --append /etc/qubes-bind-dirs.d/30_docker.conf
echo "binds+=( '/etc/docker' )" | sudo tee --append /etc/qubes-bind-dirs.d/30_docker.conf

Enable docker service for AppVM(s)

Remember to change $QUBENAME to be the name of your target AppVM/Qube:

qvm-service --enable $QUBENAME docker

Verification inside AppVM

Exit the dom0 shell.
Shutdown your TemplateVM, saving all changes.
Launch a terminal in your AppVM to perform verifications:

systemctl status containerd
systemctl status docker
docker run --rm -it busybox

After exiting busybox, you can verify "bind-dirs" is working as intended by restarting the AppVM (sudo reboot), then launching a new AppVM terminal verify the busybox image persisted between restarts (optionally removing the image):

docker images
docker rmi busybox

Why would I do it this way?

1. It preserves the privilege model and run time environment intended by docker devs and/or package authors.
2. It leverages package-supplied units for service start-up and configuration, which may change over time.
3. It is DispVM friendly.

Generally, this eliminates any configuration steps from being "inside the AppVM", any time we have to modify /rw/ directly from "inside" an AppVM we're taking an "advanced approach" to administration of our Qubes meant for solving problems that can't be solved by any regular means. To illustrate: Why bootstrap docker from the TemplateVM? Why not bootstrap everything from rc.local? Because it is inconvenient, and not necessary, unless you are trying to keep everything outside of the TemplateVM and only need docker in a single AppVM.

These means and reasons apply to more than just docker.

EDIT: removed unnecessary packages from apt install command and corrected incorrect package name, added commands to disable containerd and docker daemons in the TempalteVM, removed qvm-service --enable call for containerd as suggested by @runephilosof-abtion

I agree with @sw4k. A few corrections though.

For debian, the docker package from https://docs.docker.com/engine/install/debian/#install-using-the-repository is called docker-ce.
They recommend installing these sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
In Debian the services are enabled by default when you install them (in Fedora they are disabled), so you will have to disable them in the template after install, if you want it only enabled through qvm-service.
You don't need to enable containerd, since the docker service depends on it.

Thanks for this snippet. Not sure how accurate this is but in case you have issues with performing the post installation steps, try performing them in the Template VM vs the App VM