00_docker_on_lima_setup_guide.md

Docker on Lima (lima-vm/lima) Setup Guide

In recent months, Lima has become de facto standard software as a free and open source alternative to Docker Desktop for Mac. Several projects (such as Colima, Rancher Desktop, or Finch) for similar purpose also uses Lima as their backend. These projects will make it easier to set up Lima. But some people may not want to introduce extra components other than Lima. This guide will show you how to set up Lima to be alternative to Docker Desktop for Mac, while using Lima only.

🪣 Note. It is a good practice to uninstall Docker Desktop for Mac before proceeding with the following steps.

1. Install Lima and Docker CLI

You can easily install Lima using Homebrew:

$ brew install lima
$ lima -v
limactl version 0.13.0

Also, you need to install Docker CLI:

$ brew install docker
$ docker -v
Docker version 20.10.21, build baeda1f82a

2. Create and launch Lima VM for Docker engine

1. Download VM template file:

   $ curl -O https://raw.githubusercontent.com/lima-vm/lima/master/examples/docker.yaml

2. Enable CPU emulation: Modify the template file as follows:

        # NOTE: you may remove the lines below, if you prefer to use rootful docker, not rootless
        systemctl disable --now docker
        apt-get install -y uidmap dbus-user-session
   +- mode: system
   +  script: |
   +    #!/bin/bash
   +    set -eux -o pipefail
   +    docker run --privileged --rm tonistiigi/binfmt --install all
    - mode: user
      script: |
        #!/bin/bash

   This provisioning script will enable CPU emulation for Docker engine (i.e. whether you are using Intel Mac or Apple Silicon M1 Mac, you can run both linux/amd64 and linux/arm64 images).
3. Modify the mounts property to make your workspace directory writable. For example:

    mounts:
   -- location: "~"
   -- location: "/tmp/lima"
   -  writable: true
   +- location" "~/mycode"
   +  writable: true

4. Also, modify some other property if you need. The available options for VM template are described in default.yaml.
   • cf. my template file is here.
5. Run the following command to launch your VM:

   $ limactl start ./docker.yaml

6. After the VM is successfully started, run the following command to verify that your docker engine is working:

   $ DOCKER_HOST=$(limactl list docker -f 'unix://{{.Dir}}/sock/docker.sock') docker info
   ...
   Server:
   ...
    Operating System: Ubuntu 22.04.1 LTS
   ...

3. Establish access to the Docker engine

By default, the unix socket to the Docker Engine running on the Lima VM is deployed under the ~/.lima directory:

$ limactl list docker -f 'unix://{{.Dir}}/sock/docker.sock'
unix:///Users/yourname/.lima/docker/sock/docker.sock

To access the Docker Engine via this socket from the Docker CLI, the following methods are possible:

1. Using the environment variable DOCKER_HOST:
   • This method has already been shown in the step 2.6.
2. Using the Docker Context:
   • This method is described in the message from docker.yaml.
3. Create a symbolic link /var/run/docker.sock:
   • The path /var/run/docker.sock is the location of the unix socket referenced by the Docker CLI by default (i.e. for the default context).

When you configure them correctly, they all work equivalently. However, unfortunately some third-party Docker client does not support the method 1 and 2. So I recommend you to use method 3.

1. Save the following contents as /Library/LaunchDaemons/local.lima-docker.socket-symlink.plist:

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
     <dict>
       <key>Label</key>
       <string>local.lima-docker.socket-symlink</string>
       <key>ProgramArguments</key>
       <array>
         <string>/bin/zsh</string>
         <string>-c</string>
         <!-- Note. please change the part of 'your-name' -->
         <string>/bin/ln -s -f /Users/your-name/.lima/docker/sock/docker.sock /var/run/docker.sock</string>
       </array>
       <key>RunAtLoad</key>
       <true/>
       <key>KeepAlive</key>
       <false/>
       <key>AbandonProcessGroup</key>
       <true/>
     </dict>
   </plist>

2. Then, load (enable) it:

   sudo launchctl load /Library/LaunchDaemons/local.lima-docker.socket-symlink.plist

3. Verify that the symlink /var/run/docker.sock exists and its destination is the unix socket to the Docker Engine running on the Lima VM.

   $ ls -l /var/run/docker.sock
   lrwxr-xr-x root daemon 46 B Sun Dec 11 13:56:46 2022 /var/run/docker.sock ⇒ /Users/your-name/.lima/docker/sock/docker.sock
   

✅ Note. The above step install plist to create the symlink /var/run/docker.sock. This plist (namely, the ln command) is executed automatically at each Mac startup. To uninstall the plist, run the following command:

$ sudo launchctl unload /Library/LaunchDaemons/local.lima-docker.socket-symlink.plist
$ sudo rm /Library/LaunchDaemons/local.lima-docker.socket-symlink.plist 

For more detail, see launchd.

4. Start the Lima VM on boot

By default, you need to start your Lima VM manually when your Mac boots. The following steps will configure the Lima VM named docker to start automatically at each Mac startup.

1. Save the following contents as ~/Library/LaunchAgents/local.lima-docker.start-on-boot.plist:

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
     <dict>
       <key>Label</key>
       <string>local.lima-docker.start-on-boot</string>
       <key>ProgramArguments</key>
       <array>
         <string>/bin/zsh</string>
         <string>-lic</string>
         <string>( limactl stop -f docker; sleep 5; limactl start docker ) > ~/Library/LaunchAgents/local.lima-docker.start-on-boot.plist.log 2>&1</string>
       </array>
       <key>RunAtLoad</key>
       <true/>
       <key>KeepAlive</key>
       <false/>
       <key>AbandonProcessGroup</key>
       <true/>
     </dict>
   </plist>

2. Then, load (enable) it:

   $ launchctl load ~/Library/LaunchAgents/local.lima-docker.start-on-boot.plist

5. Verify your setup

Reboot your Mac, then verify your setup.

1. Verify that the Lima VM named docker is automatically started.

   $ limactl list
   NAME      STATUS     SSH                ARCH       CPUS    MEMORY    DISK      DIR
   docker    Running    127.0.0.1:49185    aarch64    8       16GiB     100GiB    /Users/your-name/.lima/docker

2. Verify that the symlink /var/run/docker.sock exists.

   $ ls -l /var/run/docker.sock
   lrwxr-xr-x root daemon 46 B Sun Dec 11 13:56:46 2022 /var/run/docker.sock ⇒ /Users/your-name/.lima/docker/sock/docker.sock
   

3. Verify that the docker command can access to the Docker engine running on the Lima VM.

   $ docker context ls
   NAME        DESCRIPTION                               DOCKER ENDPOINT               KUBERNETES ENDPOINT   ORCHESTRATOR
   default *   Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                         swarm
   
   $ docker info
   ...
   Server:
   ...
    Operating System: Ubuntu 22.04.1 LTS
   ...
   
   $ docker run --rm hello-world
   ...
    2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
       (arm64v8)
   ...
   
   $ docker run --rm --platform linux/amd64 hello-world
   ...
    2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
       (amd64)
   ...
   

Congratulations! All the basic setup is now done 😎

6. Install additional components

You may want to install some additional components:

• Compose
  At the moment, the docker compose command is provided as a Docker CLI plugin. You can install it as follows:

  $ mkdir -p ~/.docker/cli-plugins
  $ curl -fL -o ~/.docker/cli-plugins/docker-compose https://github.com/docker/compose/releases/latest/download/docker-compose-darwin-aarch64
  $ chmod +x ~/.docker/cli-plugins/docker-compose
  $ docker compose version
  Docker Compose version v2.14.0

• Buildx
  At the moment, the docker buildx command is provided as a Docker CLI plugin. You can install it as follows:

  $ mkdir -p ~/.docker/cli-plugins
  $ curl -fL -o ~/.docker/cli-plugins/docker-buildx https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-$(uname -m)
  $ chmod +x ~/.docker/cli-plugins/docker-buildx
  $ docker buildx version
  github.com/docker/buildx v0.9.1 ed00243a0ce2a0aee75311b06e32d33b44729689

docker.yaml

# Example to use Docker instead of containerd & nerdctl
# $ limactl start ./docker.yaml
# $ limactl shell docker docker run -it -v $HOME:$HOME --rm alpine

# To run `docker` on the host (assumes docker-cli is installed):
# $ export DOCKER_HOST=$(limactl list docker --format 'unix://{{.Dir}}/sock/docker.sock')
# $ docker ...

# This example requires Lima v0.8.0 or later
images:
# Try to use release-yyyyMMdd image if available. Note that release-yyyyMMdd will be removed after several months.
- location: "https://cloud-images.ubuntu.com/releases/22.04/release-20221201/ubuntu-22.04-server-cloudimg-amd64.img"
  arch: "x86_64"
  digest: "sha256:8a814737df484d9e2f4cb2c04c91629aea2fced6799fc36f77376f0da91dba65"
- location: "https://cloud-images.ubuntu.com/releases/22.04/release-20221201/ubuntu-22.04-server-cloudimg-arm64.img"
  arch: "aarch64"
  digest: "sha256:8a0477adcbdadefd58ae5c0625b53bbe618aedfe69983b824da8d02be0a8c961"
# Fallback to the latest release image.
# Hint: run `limactl prune` to invalidate the cache
- location: "https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img"
  arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-arm64.img"
  arch: "aarch64"

# CPUs: if you see performance issues, try limiting cpus to 1.
# Builtin default: 4
cpus: 8

# Memory size
# Builtin default: "4GiB"
memory: "16GiB"

mounts:
- location: "~/mycode"
  writable: true
# containerd is managed by Docker, not by Lima, so the values are set to false here.
containerd:
  system: false
  user: false
provision:
- mode: system
  # This script defines the host.docker.internal hostname when hostResolver is disabled.
  # It is also needed for lima 0.8.2 and earlier, which does not support hostResolver.hosts.
  # Names defined in /etc/hosts inside the VM are not resolved inside containers when
  # using the hostResolver; use hostResolver.hosts instead (requires lima 0.8.3 or later).
  script: |
    #!/bin/sh
    sed -i 's/host.lima.internal.*/host.lima.internal host.docker.internal/' /etc/hosts
- mode: system
  script: |
    #!/bin/bash
    set -eux -o pipefail
    command -v docker >/dev/null 2>&1 && exit 0
    export DEBIAN_FRONTEND=noninteractive
    curl -fsSL https://get.docker.com | sh
    # NOTE: you may remove the lines below, if you prefer to use rootful docker, not rootless
    systemctl disable --now docker
    apt-get install -y uidmap dbus-user-session
- mode: system
  script: |
    #!/bin/bash
    set -eux -o pipefail
    docker run --privileged --rm tonistiigi/binfmt --install all
- mode: user
  script: |
    #!/bin/bash
    set -eux -o pipefail
    systemctl --user start dbus
    dockerd-rootless-setuptool.sh install
    docker context use rootless
probes:
- script: |
    #!/bin/bash
    set -eux -o pipefail
    if ! timeout 30s bash -c "until command -v docker >/dev/null 2>&1; do sleep 3; done"; then
      echo >&2 "docker is not installed yet"
      exit 1
    fi
    if ! timeout 30s bash -c "until pgrep rootlesskit; do sleep 3; done"; then
      echo >&2 "rootlesskit (used by rootless docker) is not running"
      exit 1
    fi
  hint: See "/var/log/cloud-init-output.log". in the guest
hostResolver:
  # hostResolver.hosts requires lima 0.8.3 or later. Names defined here will also
  # resolve inside containers, and not just inside the VM itself.
  hosts:
    host.docker.internal: host.lima.internal
portForwards:
- guestSocket: "/run/user/{{.UID}}/docker.sock"
  hostSocket: "{{.Dir}}/sock/docker.sock"
message: |
  To run `docker` on the host (assumes docker-cli is installed), run the following commands:
  ------
  docker context create lima-{{.Name}} --docker "host=unix://{{.Dir}}/sock/docker.sock"
  docker context use lima-{{.Name}}
  docker run hello-world
  ------

local.lima-docker.socket-symlink.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>local.lima-docker.socket-symlink</string>
    <key>ProgramArguments</key>
    <array>
      <string>/bin/zsh</string>
      <string>-c</string>
      <!-- Note. please change the part of 'your-name' -->
      <string>/bin/ln -s -f /Users/your-name/.lima/docker/sock/docker.sock /var/run/docker.sock</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <false/>
    <key>AbandonProcessGroup</key>
    <true/>
  </dict>
</plist>

local.lima-docker.start-on-boot.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>local.lima-docker.start-on-boot</string>
    <key>ProgramArguments</key>
    <array>
      <string>/bin/zsh</string>
      <string>-lic</string>
      <string>( limactl stop -f docker; sleep 5; limactl start docker ) > ~/Library/LaunchAgents/local.lima-docker.start-on-boot.plist.log 2>&1</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <false/>
    <key>AbandonProcessGroup</key>
    <true/>
  </dict>
</plist>

Thanks a lot! I'm wondering how to upgrade docker engine with lima setup?

@ruibinx Since the above template uses get.docker.com script to install the Docker engine, we may refer to the following guides:

• https://docs.docker.com/engine/install/ubuntu/#upgrade-docker-after-using-the-convenience-script
• https://docs.docker.com/engine/install/ubuntu/#upgrade-docker-engine

In short, run apt-get commands in the Lima guest machine:

# login to the Lima guest machine...
host-macos$ limactl shell docker
# upgrade the Docker engine and related packages...
lima-guest-linux$ sudo apt-get update \
  && sudo apt-get install \
    docker-ce \
    docker-ce-cli \
    containerd.io \
    docker-buildx-plugin \
    docker-compose-plugin

(but I have not tested yet...)

Thanks. Works pretty well for me after restart the guest machine.

Great guide.Many thanks for it!