Skip to content

Instantly share code, notes, and snippets.

@drdaeman

drdaeman/rkt-fly-on-wsl.md

Last active January 19, 2022 10:28
Show Gist options
  • Save drdaeman/84f10a7b9da9d2fe152e36b426bde00d to your computer and use it in GitHub Desktop.
Save drdaeman/84f10a7b9da9d2fe152e36b426bde00d to your computer and use it in GitHub Desktop.
Download ZIP
Running Docker containers on Windows, natively, using rkt (WSL aka Bash on Windows)
Raw
rkt-fly-on-wsl.md

Just a heads up for those who are looking into running Docker containers natively on WSL. While it's not usable for anything serious (even your toy project development), there is a way.

Nuances:

  • We'll be using rkt and not Docker. So, no docker-compose for you.
  • There will be some source code patching. Nothing serious, just a few lines.
  • Very limited isolation and networking capabilities. Think of always --privileged, --net-host and no DNS.

WSL doesn't have cgroups at the moment, and probably won't have them anytime soon, so no way Docker would work. Unless you know a trick to replace runc/containerd with something.

However, rkt has a "rkt fly" feature that uses simple chroot environments.

Here's what I did:

  1. Installed Go 1.8.1 using godeb (I was using older 14.04 OS image). I guess apt-get install golang should also work, but not sure.

  2. Checked out rkt git repository: git clone https://github.com/rkt/rkt.git and cd rkt

  3. Installed necessary build dependencies with apt-get install autoconf build-essential libacl1-dev realpath

  4. There are few issues that prevent rkt from just running. I've patched the following files:

    • In pkg/sys/sys_linux.go in func Syncfs(int) error there is a SYS_SYNCFS syscall that is not implemented in WSL. Let's ignore ENOSYS:

                 if err != 0 {
   +               if err == syscall.ENOSYS {
   +                       return nil
   +               }
                   return syscall.Errno(err)
           }

    • In stage1_fly/run/main.go there will be some mounting failures. WSL seem to fail when host path is an empty string:

                        if err := mounter.Mount(mount.HostPath, absTargetPath, mount.Fs, mount.Flags, ""); err != nil {
                          log.PrintE(fmt.Sprintf("can't mount %q on %q with flags %v", mount.HostPath, absTargetPath, mount.Flags), err)
  -                       return 254
  +                       if mount.HostPath != "" || mount.Flags != (syscall.MS_REC | syscall.MS_SHARED) {
  +                               return 254
  +                       }
                  }
          }

    • And I've had to patch pkg/mountinfo/mountinfo.go, removing sort.Sort(podMounts) line completely, and a "sort" import line (since sort isn't used anywhere else). I don't know why this was necessary, and too lazy to debug, but rkt stage1 run binary just hang otherwise, after read call, not yet calling close. I don't think rkt actually relies on results' ordering, anyway.

  5. With those modifications I was able to build rkt normally:

     ./autogen.sh && ./configure --with-stage1-flavors=fly --disable-tpm --disable-sdjournal && make

  6. Then I was able to run a container. Here is a normal rkt run invocation:

     $ sudo ./build-rkt-1.25.0+git/target/bin/rkt run --interactive --insecure-options=image docker://hello-world:latest
 Downloading sha256:78445dd4522 [=================================] 971 B / 971 B
 run: disabling overlay support: "overlay entry not present in /proc/filesystems"
 run: can't mount "" on "/dev" with flags 1064960: invalid argument
 run: can't mount "" on "/proc" with flags 1064960: invalid argument
 run: can't mount "" on "/sys" with flags 1064960: invalid argument

 Hello from Docker!
 This message shows that your installation appears to be working correctly.
 ...

  7. There is a clean-up problem, though - when a pod (container) exists, mounted directories (dev, proc, sys and tmp) aren't unmounted. I haven't yet figured why, but I've had to run the following:

     grep -F pods/run /proc/mounts | awk '{ print $2 }' | sort --reverse | xargs -I '{}' sudo umount '{}'

    (Put pod ID after pods/run, e.g. pods/run/8ae2ec64 to target a specific pod only - this is important if you have some pods still running. The reverse sorting is important, because there are some nested mounts.)

    After this, pod can be garbage collected:

     $ sudo ./build-rkt-1.25.0+git/target/bin/rkt gc --grace-period=1s
 Garbage collecting pod "8ae2ec64-69fe-412a-a9f4-8e4aabd06405"

  8. Empty volumes (kind=empty) are broken as well, probably because of same no-empty-source limitation. Only kind=host seem to work.

Of course, this is very fragile and extremely limited. But... it works! Well, sort of.

Here's a more realistic example:

$ sudo ./build-rkt-1.25.0+git/target/bin/rkt run --interactive \
>    --volume=volume-var-lib-postgresql-data,kind=host,source=$HOME/tmp \
>    --insecure-options=image docker://postgres:latest
run: disabling overlay support: "overlay entry not present in /proc/filesystems"
run: can't mount "" on "/dev" with flags 1064960: invalid argument
run: ignoring this mount issue
run: can't mount "" on "/proc" with flags 1064960: invalid argument
run: ignoring this mount issue
run: can't mount "" on "/sys" with flags 1064960: invalid argument
run: ignoring this mount issue
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... sysv
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

****************************************************
WARNING: No password has been set for the database.
         This will allow anyone with access to the
         Postgres port to access your database. In
         Docker's default configuration, this is
         effectively any other container on the same
         system.

         Use "-e POSTGRES_PASSWORD=password" to set
         it in "docker run".
****************************************************
waiting for server to start....LOG:  database system was shut down at 2017-04-24 23:25:02 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
 done
server started
ALTER ROLE


/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

LOG:  received fast shutdown request
LOG:  aborting any active transactions
LOG:  autovacuum launcher shutting down
waiting for server to shut down....LOG:  shutting down
WARNING:  could not flush dirty data: Function not implemented
WARNING:  could not flush dirty data: Function not implemented
LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

LOG:  database system was shut down at 2017-04-24 23:25:05 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections

(For some reason, it took a good minute to start, while using CPU a lot. I think Docker in VirtualBox is much faster and less CPU-intensive.)

And in another WSL console window:

$ psql -h 127.0.0.1 -p 5432 -U postgres
psql (9.3.16, server 9.6.2)
WARNING: psql major version 9.3, server major version 9.6.
         Some psql features might not work.
Type "help" for help.

postgres=# SELECT version();
                                         version
------------------------------------------------------------------------------------------
 PostgreSQL 9.6.2 on x86_64-pc-linux-gnu, compiled by gcc (Debian 4.9.2-10) 4.9.2, 64-bit
(1 row)

:-)

Raw
rkt-on-wsl-c767336b.patch
diff --git a/pkg/mountinfo/mountinfo.go b/pkg/mountinfo/mountinfo.go
index f227902..557bb36 100644
--- a/pkg/mountinfo/mountinfo.go
+++ b/pkg/mountinfo/mountinfo.go
@@ -20,7 +20,6 @@ import (
"fmt"
"io"
"os"
- "sort"
"strconv"
"strings"
@@ -50,7 +49,6 @@ func ParseMounts(pid uint) (Mounts, error) {
return nil, err
}
defer mi.Close()
-
return parseMountinfo(mi)
}
@@ -136,6 +134,5 @@ func parseMountinfo(mi io.Reader) (Mounts, error) {
if err := sc.Err(); err != nil {
return nil, errwrap.Wrap(errors.New("problem parsing mountinfo"), err)
}
- sort.Sort(podMounts)
return podMounts, nil
}
diff --git a/pkg/sys/sys_linux.go b/pkg/sys/sys_linux.go
index f273826..4b4b3f5 100644
--- a/pkg/sys/sys_linux.go
+++ b/pkg/sys/sys_linux.go
@@ -19,6 +19,10 @@ import "syscall"
func Syncfs(fd int) error {
_, _, err := syscall.RawSyscall(SYS_SYNCFS, uintptr(fd), 0, 0)
if err != 0 {
+ if err == syscall.ENOSYS {
+ // Let it fail
+ return nil
+ }
return syscall.Errno(err)
}
return nil
diff --git a/stage1_fly/run/main.go b/stage1_fly/run/main.go
index e3ac294..620d5ce 100644
--- a/stage1_fly/run/main.go
+++ b/stage1_fly/run/main.go
@@ -434,7 +434,9 @@ func stage1(rp *stage1commontypes.RuntimePod) int {
if err := mounter.Mount(mount.HostPath, absTargetPath, mount.Fs, mount.Flags, ""); err != nil {
log.PrintE(fmt.Sprintf("can't mount %q on %q with flags %v", mount.HostPath, absTargetPath, mount.Flags), err)
- return 254
+ if mount.HostPath != "" || mount.Flags != (syscall.MS_REC | syscall.MS_SHARED) {
+ return 254
+ }
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment