Skip to content

Instantly share code, notes, and snippets.

@arno01
Last active October 24, 2024 13:24
Show Gist options
  • Save arno01/ebf570af208e28c1a0cf78da4f63bc9c to your computer and use it in GitHub Desktop.
Save arno01/ebf570af208e28c1a0cf78da4f63bc9c to your computer and use it in GitHub Desktop.
Docker on Android

WORK IN PROGRESS

Docker on Android

Setup:

Samsung Galaxy Tab S5e SM-T720
Android Pie on Linux 4.9.112 (not rooted)
Termux
golang 1.12

Client

This will install the docker client to your ~/go/bin/ directory.

go get github.com/docker/cli/cmd/docker

The client is working, you can export DOCKER_HOST value to work with the dockerd, for example:

# export DOCKER_HOST=unix://$HOME/docker.sock
export DOCKER_HOST=tcp://192.168.X.Y:2376
docker run hello-world

Server

go get -u -d github.com/docker/docker/cmd/dockerd

rm -vf ~/go/src/github.com/docker/docker/daemon/graphdriver/register/register_btrfs.go
rm -vf ~/go/src/github.com/docker/docker/daemon/graphdriver/register/register_devicemapper.go

cd ~/go/src/github.com/docker/docker/cmd/dockerd
go install

containerd

containerd is the container runtime used by dockerd.

go get -u -d github.com/containerd/containerd/cmd/containerd
rm -vf ~/go/src/github.com/containerd/containerd/cmd/containerd/builtins_btrfs_linux.go
cd ~/go/src/github.com/containerd/containerd/cmd/containerd
go install

rootless docker

  1. Install rootlesskit
source ~/go/src/github.com/docker/docker/hack/dockerfile/install/rootlesskit.installer
REFIX=$GOPATH/bin _install_rootlesskit
  1. Install slirp4netns
git clone -b v0.3.0 https://github.com/rootless-containers/slirp4netns.git
cd slirp4netns
./autogen.sh
./configure --prefix=$PREFIX
make
make install
  1. Run rootless dockerd
~/go/src/github.com/docker/docker/contrib/dockerd-rootless.sh --experimental

Issue:

Apparently non-rooted Android is not permitting using the namespaces, probably due to SELinux rules or any other means such as unprivileged_userns_clone set to 0.. please try this if you have rooted Android.

+ exec rootlesskit --net=slirp4netns --mtu=65520 --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run /data/data/com.termux/files/home/go/src/github.com/docker/docker/contrib/dockerd-rootless.sh --experimental
WARN[0000] "builtin" port driver is experimental
[rootlesskit:parent] error: failed to start the child: fork/exec /proc/self/exe: operation not permitted
$ strace rootlesskit --net=slirp4netns --mtu=65520 --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run bash |& grep CLONE_NEWUSER
clone(child_stack=NULL, flags=CLONE_NEWUSER|SIGCHLD) = -1 EPERM (Operation not permitted)

$ strace unshare -U id |& grep PERM
unshare(CLONE_NEWUSER)                  = -1 EPERM (Operation not permitted)

Refs

my go env

The defaults I have on my config:

$ go env
GOARCH="arm64"
GOBIN=""
GOCACHE="/data/data/com.termux/files/home/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="android"
GOOS="android"
GOPATH="/data/data/com.termux/files/home/go"
GOPROXY=""
GORACE=""
GOROOT="/data/data/com.termux/files/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/data/data/com.termux/files/usr/lib/go/pkg/tool/android_arm64"
GCCGO="gccgo"
CC="aarch64-linux-android-clang"
CXX="aarch64-linux-android-clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/data/data/com.termux/files/usr/tmp/go-build067260183=/tmp/go-build -gno-record-gcc-switches"
@FreddieOliveira
Copy link

The fetaures you enabled in the kernel and the bug that causes the crash are not related. There's no specific order on which you should do first. Just make sure you did both of them (enabling the features using make menuconfig and aplying the patch) before compiling the kernel.

@hnmn
Copy link

hnmn commented Oct 1, 2020

@FreddieOliveira thanks again for quick reply.
i have compile kernel but problem i don't khow where to modify source code (i.e patch) . as you mentioned we need to modify n complile various
packages which are discussed in various issues .i just want to make gist or repo where all the pieces are present that would be easy .

@mad-ady
Copy link

mad-ady commented Dec 29, 2020

Hey guys!
I'm trying to get Docker running in a Linux chroot (Ubuntu 18 via Linux Deploy) running on Lineage 17 on a Odroid N2 (with a kernel compiled with docker support: http://paste.ubuntu.com/p/TbXrX4yWHN/).
I followed your steps, except I ran them in the Linux chroot, not in termux.
I am able to start containerd:

# ~/go/bin/containerd
INFO[2020-12-29T13:23:06.634962961+02:00] starting containerd                           revision= version=1.4.0+unknown
INFO[2020-12-29T13:23:06.714363003+02:00] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
INFO[2020-12-29T13:23:06.714504920+02:00] loading plugin "io.containerd.snapshotter.v1.aufs"...  type=io.containerd.snapshotter.v1
WARN[2020-12-29T13:23:06.714572878+02:00] failed to load plugin io.containerd.snapshotter.v1.aufs  error="invalid aufs configuration"
INFO[2020-12-29T13:23:06.714609753+02:00] loading plugin "io.containerd.snapshotter.v1.devmapper"...  type=io.containerd.snapshotter.v1
WARN[2020-12-29T13:23:06.714656253+02:00] failed to load plugin io.containerd.snapshotter.v1.devmapper  error="devmapper not configured"
INFO[2020-12-29T13:23:06.714688420+02:00] loading plugin "io.containerd.snapshotter.v1.native"...  type=io.containerd.snapshotter.v1
INFO[2020-12-29T13:23:06.714752128+02:00] loading plugin "io.containerd.snapshotter.v1.overlayfs"...  type=io.containerd.snapshotter.v1
INFO[2020-12-29T13:23:06.714978170+02:00] loading plugin "io.containerd.snapshotter.v1.zfs"...  type=io.containerd.snapshotter.v1
WARN[2020-12-29T13:23:06.715087586+02:00] failed to load plugin io.containerd.snapshotter.v1.zfs  error="invalid zfs configuration"
INFO[2020-12-29T13:23:06.715121795+02:00] loading plugin "io.containerd.metadata.v1.bolt"...  type=io.containerd.metadata.v1
WARN[2020-12-29T13:23:06.715164420+02:00] could not use snapshotter zfs in metadata plugin  error="invalid zfs configuration"
WARN[2020-12-29T13:23:06.715201336+02:00] could not use snapshotter aufs in metadata plugin  error="invalid aufs configuration"
WARN[2020-12-29T13:23:06.715227461+02:00] could not use snapshotter devmapper in metadata plugin  error="devmapper not configured"
INFO[2020-12-29T13:23:06.715257628+02:00] metadata content store policy set             policy=shared
INFO[2020-12-29T13:23:06.718472336+02:00] loading plugin "io.containerd.differ.v1.walking"...  type=io.containerd.differ.v1
INFO[2020-12-29T13:23:06.718630503+02:00] loading plugin "io.containerd.gc.v1.scheduler"...  type=io.containerd.gc.v1
INFO[2020-12-29T13:23:06.718803128+02:00] loading plugin "io.containerd.service.v1.introspection-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.718951336+02:00] loading plugin "io.containerd.service.v1.containers-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.719033836+02:00] loading plugin "io.containerd.service.v1.content-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.719083003+02:00] loading plugin "io.containerd.service.v1.diff-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.719135628+02:00] loading plugin "io.containerd.service.v1.images-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.719185961+02:00] loading plugin "io.containerd.service.v1.leases-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.719237670+02:00] loading plugin "io.containerd.service.v1.namespaces-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.719289211+02:00] loading plugin "io.containerd.service.v1.snapshots-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.719367170+02:00] loading plugin "io.containerd.runtime.v1.linux"...  type=io.containerd.runtime.v1
INFO[2020-12-29T13:23:06.721884711+02:00] loading plugin "io.containerd.runtime.v2.task"...  type=io.containerd.runtime.v2
INFO[2020-12-29T13:23:06.724095461+02:00] loading plugin "io.containerd.monitor.v1.cgroups"...  type=io.containerd.monitor.v1
INFO[2020-12-29T13:23:06.729475295+02:00] loading plugin "io.containerd.service.v1.tasks-service"...  type=io.containerd.service.v1
INFO[2020-12-29T13:23:06.729670045+02:00] loading plugin "io.containerd.internal.v1.restart"...  type=io.containerd.internal.v1
INFO[2020-12-29T13:23:06.730064461+02:00] loading plugin "io.containerd.grpc.v1.containers"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.730214878+02:00] loading plugin "io.containerd.grpc.v1.content"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.730340920+02:00] loading plugin "io.containerd.grpc.v1.diff"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.730444878+02:00] loading plugin "io.containerd.grpc.v1.events"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.730535711+02:00] loading plugin "io.containerd.grpc.v1.healthcheck"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.730626003+02:00] loading plugin "io.containerd.grpc.v1.images"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.730715086+02:00] loading plugin "io.containerd.grpc.v1.leases"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.730793670+02:00] loading plugin "io.containerd.grpc.v1.namespaces"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.730864170+02:00] loading plugin "io.containerd.internal.v1.opt"...  type=io.containerd.internal.v1
INFO[2020-12-29T13:23:06.731665753+02:00] loading plugin "io.containerd.grpc.v1.snapshots"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.732201920+02:00] loading plugin "io.containerd.grpc.v1.tasks"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.732691336+02:00] loading plugin "io.containerd.grpc.v1.version"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.733175170+02:00] loading plugin "io.containerd.grpc.v1.introspection"...  type=io.containerd.grpc.v1
INFO[2020-12-29T13:23:06.735870420+02:00] serving...                                    address=/run/containerd/containerd.sock.ttrpc
INFO[2020-12-29T13:23:06.736949170+02:00] serving...                                    address=/run/containerd/containerd.sock
INFO[2020-12-29T13:23:06.737416045+02:00] containerd successfully booted in 0.112886s

... and dockerd ...

# ~/go/bin/dockerd --iptables=false
INFO[2020-12-29T13:24:40.216266256+02:00] Starting up
INFO[2020-12-29T13:24:40.220255673+02:00] parsed scheme: "unix"                         module=grpc
INFO[2020-12-29T13:24:40.220327714+02:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2020-12-29T13:24:40.220379298+02:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}  module=grpc
INFO[2020-12-29T13:24:40.220406673+02:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2020-12-29T13:24:40.225100923+02:00] parsed scheme: "unix"                         module=grpc
INFO[2020-12-29T13:24:40.225250214+02:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2020-12-29T13:24:40.225378256+02:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}  module=grpc
INFO[2020-12-29T13:24:40.225419923+02:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2020-12-29T13:24:40.260867881+02:00] [graphdriver] using prior storage driver: overlay2
WARN[2020-12-29T13:24:40.272595506+02:00] Your kernel does not support CPU CFS scheduler
WARN[2020-12-29T13:24:40.272657589+02:00] Unable to find blkio cgroup in mounts
INFO[2020-12-29T13:24:40.272974631+02:00] Loading containers: start.
INFO[2020-12-29T13:24:40.511562756+02:00] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address
INFO[2020-12-29T13:24:40.552736881+02:00] Loading containers: done.
INFO[2020-12-29T13:24:40.698796464+02:00] Docker daemon                                 commit=library-import graphdriver(s)=overlay2 version=library-import
INFO[2020-12-29T13:24:40.699200131+02:00] Daemon has completed initialization
INFO[2020-12-29T13:24:40.764358423+02:00] API listen on /var/run/docker.sock

ERRO[2020-12-29T13:30:53.216723267+02:00] stream copy error: reading from a closed fifo
ERRO[2020-12-29T13:30:53.216799475+02:00] stream copy error: reading from a closed fifo
ERRO[2020-12-29T13:30:53.378941059+02:00] a253f1025602c94c70f57e9e674d3d2a939ee43e9c5cbed828e9c8af979a5f63 cleanup: failed to delete container from containerd: no such container
ERRO[2020-12-29T13:30:53.379082517+02:00] Handler for POST /v1.41/containers/a253f1025602c94c70f57e9e674d3d2a939ee43e9c5cbed828e9c8af979a5f63/start returned error: runtime "io.containerd.runc.v2" binary not installed "containerd-shim-runc-v2": file does not exist: unknown

The errors reported by it are generated when I try to run docker run hello-world

# ~/go/bin/docker run hello-world
docker: Error response from daemon: runtime "io.containerd.runc.v2" binary not installed "containerd-shim-runc-v2": file does not exist: unknown.
ERRO[0000] error waiting for container: context canceled

Most likely I still need to start some components manually. Any idea what I'm missing?

@FreddieOliveira
Copy link

@mad-ady I don't think you need to start containerd yourself, just starting dockerd should be enough. The docker daemon starts its dependencies by itself. About your error it looks you don't have runc installed. You can apt install runc or compile it from source. Also, running dockerd with --debug flag gives more verbosity.

@mad-ady
Copy link

mad-ady commented Dec 31, 2020 via email

@FreddieOliveira
Copy link

@mad-ady I'm finishing writing the step by step guide I promised a while ago and ran into the same issue as you. I realized it's not the binary runc that's missing, it's actually a binary called containerd-shim-runc-v2. It's part of containerd and strangely enough this binary is not compiled by default when running make, you have to explicitly compile it with make bin/containerd-shim-runc-v2.

@hessam94
Copy link

hessam94 commented Jan 8, 2021

@FreddieOliveira I coudnt find time to run your patch , i will do it in 2 weeks thanks

@FreddieOliveira
Copy link

Don't worry, I intend to post the tutorial by the end of this week. You can use it as a reference.

@FreddieOliveira
Copy link

The step by step tutorial is finally done. Whoever is interested in get docker running on its own phone can check here for instructions.

@mad-ady
Copy link

mad-ady commented Jan 12, 2021

Thank you for your work. Hope to give it a try soon and provide feedback!

@hessam94
Copy link

hessam94 commented Feb 9, 2021

I am recompiling the android on a ubuntu Linux, would you help us how to apply the patch (segmentation fault patch) on android kernel before compilation a new kernel? I got the entire android files on linux and trying to build a new one
thanks

@xy815661276
Copy link

xy815661276 commented May 13, 2021

We open the Android container, which can run the docker container on the Android operating system. Not only can run Docker containers, but also other containers, such as podman. In addition, we have also enabled the container to support checkpoint and restore functions, and support the migration of containers across architectures and operating systems. See https://github.com/CGCL-codes/Android-Container for details.

@jelllyam
Copy link

jelllyam commented Jun 3, 2022

For those who are interested, docker runs fine in android as long as you have an appropriated kernel (use https://github.com/moby/moby/blob/master/contrib/check-config.sh to check it). I'm running it right now, here's the prove:

This is a screenshot of docker running in my Redmi Note 7 device in termux. No chroot into a Linux rootfs and no qemu emulation of a Linux distro. This is docker in pure android.

But, there's a problem. When you pull a container, docker checks your device architecture and.operating system to download a container that matches it. To do so, it checks the container's manifest and see what architectures and OS's are supported by it and then pulls the correct one. Despite aarch64 is largely supported by most of the containers, the Android OS is not. This makes docker useless in Android, since there's no container that runs on it:

Screenshot_20200614-044554_Termux

For more info and alternatives on how to run docker on Android check this thread: termux/termux-root-packages#60

Can you share your kernel in redmi note 7

@Sokan2004
Copy link

Verify Github on Galaxy. gid:wzXiXzmeFjYF4QKxtCUfnP

@FreddieOliveira
Copy link

@jelllyam, here's the kernel I'm currently using. It's based on LineageOS 17.1 (Android 10): https://ufile.io/a6guqaku

And out of curiosity, here's my kernel config file used when compiling it: https://ufile.io/5u3q10qu

The download links will be valid for 30 days.

Be aware that this kernel is for Android 10. Flashing it on a different version may cause boot loop. In case this happens, just reflash your ROM. You won't lose any of your data flashing the kernel or your ROM.

To learn how to install and use docker after you successfully flash the kernel see https://gist.github.com/FreddieOliveira/efe850df7ff3951cb62d74bd770dce27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment