In order to run the tests autonatically at each PR, we need a continuous integration service hooked on GitHub. Rocket needs to be able to do bind mounts, change mount options (such as MS_REC|MS_SLAVE), create namespaces, mknod, mount cgroupfs.
Travis does not give CAP_SYS_ADMIN. So mounts are not allowed, creating new namespaces are not allowed.
clone(child_stack=0, flags=CLONE_NEWNS|0x2c000000|SIGCHLD) = -1 EPERM (Operation not permitted)
Drone gives CAP_SYS_ADMIN. However, changing mount options and bind mounts are restricted:
mount(NULL, "/", NULL, MS_REC|MS_SLAVE, NULL) = -1 EACCES (Permission denied)
mount("/tmp/dira", "/tmp/dirb", 0x4101e4, MS_MGC_VAL|MS_BIND, NULL) = -1 EACCES (Permission denied)
Codeship currently does not give root.
They give CAP_SYS_ADMIN. But changing mount options is restricted:
mount(NULL, "/", NULL, MS_REC|MS_SLAVE, NULL) = -1 EACCES (Permission denied)
CircleCI gives CAP_SYS_ADMIN. Changing mount options and creating bind mounts work fine. However, mknod is restricted (we have CAP_MKNOD but they use either device cgroups or AppArmor to restrict it).
Rocket needs mknod, either to populate the device files from an ACI (docker://busybox contains a device node):
mknod("/var/lib/rkt/containers/prepare/e08d3094-c8ce-4d0c-82b2-f98f4a9c2c89/stage1/rootfs/opt/stage2/sha512-c4010045aec65aefa74770ef2bb648d9/rootfs/dev/console", S_IFCHR|0410000622, makedev(5, 1) <unfinished ...>
<... mknod resumed> ) = -1 EPERM (Operation not permitted)
Or to allow systemd-nspawn to populate basic /dev device nodes:
mknod("/var/lib/rkt/containers/run/07d5d727-cbf1-48da-b600-c4532cb37ce9/stage1/rootfs/dev/null", S_IFCHR|0666, makedev(1, 3)) = -1 EPERM (Operation not permitted)
systemd-nspawn does not really need to call mknod and could use bind mounts instead. I suggested this patch to avoid the problem: nspawn: fallback on bind mount when mknod fails
I ported the patch to systemd v215 and rebuilt Rocket with the patch (RKT_STAGE1_USR_FROM=src). Then, as long as the archive does not contain a device node (docker://busybox), the restriction on mknod is no longer an issue.
But it still failed because the CircleCI environment does not have the cgroup filesystems mounted and mounting them is restricted. Instead, the cgroup filesystems are mounted in a separate mount namespace by cgmanager and the only interaction with cgroups allowed is through the cgproxy socket. Systemd in stage1 requires direct access to the cgroup filesystems and cannot use the cgproxy socket. So this cannot work.
Do you know other continuous integration service that would use VMs or that would allow access to cap_sys_admin, mounts, cgroups, etc.?