Skip to content

Instantly share code, notes, and snippets.

@gorshkov-leonid

gorshkov-leonid/docker-in-windows.md

Last active December 13, 2024 10:42
Show Gist options
  • Save gorshkov-leonid/e88685dd263944dca2a66b7b8d1070a3 to your computer and use it in GitHub Desktop.
Save gorshkov-leonid/e88685dd263944dca2a66b7b8d1070a3 to your computer and use it in GitHub Desktop.
Download ZIP
Docker in Windows without Docker Desktop
Raw
docker-in-windows.md

See items marked with ❤️‍🔥 to use winning variants

Docker in Windows without Docker Desktop

❤️‍🔥 Way 1. Using original docker binaries

Install docker client in Windows

  • Download necessary version of docker binaries from https://docs.docker.com/engine/install/binaries/ and extract archive to c:/Program Files, for example, using script in powershell. Run powershell as Administrator and call:

    Expand-Archive .\docker-20.10.9.zip -DestianationPath $Env:ProgramFiles

    Files docker.exe and dockerd.exe will be here c:/Program Files/docker/.

    We need only docker.exe and do not need dockerd.exe because it can start only Windows containers. To check it you can try to call as Admin dockerd.exe and then docker.exe pull ubuntu. Expected result: no matching manifest for windows/amd64 10.0.19043 in the manifest list entries

  • Add path c:/Program Files/docker to environment variable PATH. It could be done with this powershell script:

    [Environment]::SetEnvironmentVariable("Path",$env:Path + ";c:/Program Files/docker", "Machine")

❤️‍🔥 Install Docker Daemon in WSL

  • Install docker under Ubuntu WSL using these instructions https://docs.docker.com/engine/install/ubuntu/

  • To provide ability to call docker in WSL without sudo add current user to docker group

    sudo groupadd docker
sudo usermod -aG docker $USER

  • Configure dockerd to run (one of)...

    👉 over http with access from Windows

    ⤵️ Edit /etc/profile

    function rundocker(){
  ## Use this variant in case when `localhostForwarding` in `/etc/wsl.conf` is unset or equal to `false`
  #DOCKER_HOST=`ifconfig eth0 | grep -E "inet ([0-9]{1,3}.){3}[0-9]{1,3}" | head -1 | awk '{ print $2 }'`
  DOCKER_HOST=localhost
  WIN_CLIENT="/mnt/c/Program Files/docker/docker.exe"
  "$WIN_CLIENT" context rm -f wsl > /dev/null
  "$WIN_CLIENT" context create wsl --description "dockerd in WSL" --docker "host=tcp://$DOCKER_HOST:2375"
  "$WIN_CLIENT" context use wsl
  sudo dockerd -H $DOCKER_HOST -H "unix:///var/run/docker.sock" --tls=false
}
export -f rundocker
    Details...

    ⚠️ Be aware of this warnings:

    Binding to IP address without --tlsverify is insecure and gives root access on this machine to everyone who has access to your network.  host="tcp://172.27.117.140:2375"
Binding to an IP address, even on localhost, can also give access to scripts run in a browser. Be safe out there!  host="tcp://172.27.117.140:2375"

    ⤵️ ❤️‍🔥 Run docker daemon on Windows startup using Task Scheduler (taskschd.msc)

    $task = New-ScheduledTaskAction -Execute "C:\Windows\System32\wsl.exe" -Argument '-u root bash -c "(source /etc/profile && rundocker &) && sleep 10"'
$trigger = New-ScheduledTaskTrigger -AtLogon
$settings = New-ScheduledTaskSettingsSet -DontStopIfGoingOnBatteries -AllowStartIfOnBatteries
Register-ScheduledTask RunDocker -Action $task -Trigger $trigger -Settings $settings

    ⤵️ Run docker daemon on Windows startup using Startup folder

    Open Startup folder Win+R -> shell:common startup (usually C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp) and create .bat script with content

    C:\Windows\System32\wsl.exe g -u root bash -c "(source /etc/profile && rundocker &) && sleep 10"

    ⤵️ To stop docker daemon manually

    sudo kill $(ps faux | grep 'dockerd' | grep -vw grep | awk '{ print $2 }')

    ⤵️ To run docker daemon manually

    rundocker &

    👉 without access from Windows, using systemd

    ⚠️ this method raises questions. Firstly, I do not know how to get access from Windows, e.g. cmd.exe. Secondly, systemd is a kind of overhead.

    ⚠️ TBD try to enable systemd: https://youtu.be/DmfuJzX6vJQ?t=459

    ⤵️ Add systemd support (about script, about Linux Software Repository for Microsoft Products)

    wget --content-disposition \
"https://gist.githubusercontent.com/djfdyuruiry/6720faa3f9fc59bfdf6284ee1f41f950/raw/952347f805045ba0e6ef7868b18f4a9a8dd2e47a/install-sg.sh"
  
chmod +x ./install-sg.sh
  
# use here appropriate version from https://packages.microsoft.com/ubuntu/
sed -i -E 's/(UBUNTU_VERSION)=".+"/\1=21.04/g' ./install-sg.sh
  
./install-sg.sh

    Then Exit the WSL terminal and shutdown the WSL env:

    wsl --shutdown

    To open a new WSL terminal with systemd enabled, run:

    wsl genie -s
    Details...

    ⚠️ In you have an error like

    $ wsl genie -s
Timed out waiting for systemd to enter running state.
This may indicate a systemd configuration error.
Attempting to continue.
Failed units will now be displayed (systemctl list-units --failed):
  UNIT                       LOAD   ACTIVE SUB    DESCRIPTION
● systemd-remount-fs.service loaded failed failed Remount Root and Kernel File Systems

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
1 loaded units listed.

    then disable this service

    sudo systemctl mask systemd-remount-fs.service

    then wsl --shutdown and retry wsl genie -s

    ⚠️ In you have an error (reproduced in Ubuntu 21.10) like

    WARN[2021-11-05T17:20:57.078243654+04:00] grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock 0 }. Err :connection error: desc = "transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout". Reconnecting... module=grpc
failed to start daemon: Error initializing network controller: error obtaining controller instance: unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain: (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.7 (nf_tables): RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1
(exit status 4))

    then switch from nf_tables to ip_tables (see link1, link2):

    $sudo rm -rd /var/run/docker*
# Ubuntu 21.10 uses nf_tables intead of ip_tables
$/sbin/iptables --version
$/usr/sbin/iptables-legacy --version
$ sudo update-alternatives --set iptables /usr/sbin/iptables-legacy

    ⤵️ Run docker daemon on Windows startup using Task Scheduler (taskschd.msc)

    $task = New-ScheduledTaskAction -Execute "C:\Windows\System32\wsl.exe" -Argument 'genie -s'
$trigger = New-ScheduledTaskTrigger -AtLogon
$settings = New-ScheduledTaskSettingsSet -DontStopIfGoingOnBatteries -AllowStartIfOnBatteries
Register-ScheduledTask RunDocker -Action $task -Trigger $trigger -Settings $settings

    ⤵️ Run docker daemon on Windows startup using Startup folder

    Open Startup folder Win+R -> shell:common startup (usually C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp) and create .bat script with content

    C:\Windows\System32\wsl.exe genie -s

    ⤵️ How to run docker daemon manually

    C:\Windows\System32\wsl.exe genie -s

    👉 over https with access from Windows too

    TBD

    👉 over channels?

    TBD

Way 2. Using nerdctl from Rancher Desktop instead of docker

⚠️ this way is not completely functional. Firstly, I do knot know how to make docker available everywhere - not only in bash under WSL. Secondly, nerdctl is not docker, so it could bring unexpected problems (e.g. nerdctl does not have --ip param for nerdctl run, who knows what else difference exist). But Rancher Desktop can be used as separate container orchestrator.

⚠️ I run nerdctl -n docker run -p 3000:3000 grafana/grafana, checked and then stopped container via ctrl+c or stop. After it, I cannot run it again until rebooting.

  • Install Rancher Desktop
  • Run and choose some version of Kubernetes
  • Tick WSL Integration -> Ubuntu. Link /home/<wsl-user>/.kube/config -> /mnt/c/Users/<win-user>/.kube/config will be created.
  • This step is required if you are using appendWindowsPath=false 
    mkdir -p ~/.local/bin
WIN_USER=<type-him>
ln -s /mnt/c/Users/$WIN_USER/AppData/Local/Programs/Rancher\ Desktop/resources/resources/win32/bin/nerdctl.exe /home/$USER/.local/bin/nerdctl
#ln -s /mnt/c/Users/$WIN_USER/AppData/Local/Programs/Rancher\ Desktop/resources/resources/win32/bin/kubectl.exe /home/$USER/.local/bin/kubectl
#ln -s /mnt/c/Users/$WIN_USER/AppData/Local/Programs/Rancher\ Desktop/resources/resources/win32/bin/helm.exe /home/$USER/.local/bin/helm
#ln -s /mnt/c/Users/$WIN_USER/AppData/Local/Programs/Rancher\ Desktop/resources/resources/win32/bin/kuberlr.exe /home/$USER/.local/bin/kuberlr
#ln -s /mnt/c/Users/$WIN_USER/AppData/Local/Programs/Rancher\ Desktop/resources/resources/win32/bin/kim.exe /home/$USER/.local/bin/kim
source ~/.bashrc
    Thereby nerdctl will be available in WSL for current user. Also you can link programs to /usr/bin/nerdctl to share it for all users.
  • Lets create alias docker -> nerdctl with hardcoded namespace name (in Rancher Desktop) where all images will be pull in. Add to ~/.bash_aliases or /etc/aliases: 
    #nerdctl as docker
alias docker='nerdctl -n docker'
    Thereby nerdctl can be used as docker but this is possible only in bash under WSL (think about how to make link or script to fix parameter).

❤️‍🔥 Docker Compose

sudo apt-get install -y python3 python3-pip
pip3 install --user docker-compose

⚠️ Do not forget to set variable DOCKER_HOST in ~/.bashrc if it has not still done.

export DOCKER_HOST=tcp://localhost:2375

⚠️ On error TypeError: load_config() got an unexpected keyword argument 'config_dict' try to call scripts:

pip3 uninstall docker-compose
pip3 uninstall docker-py docker
pip3 install --user -U docker-compose

See also link1, link2, link3

See Setting Up Docker for Windows and WSL to Work Flawlessly

☢️ Problem with host.docker.internal

I caught error Error: net::ERR_CONNECTION_REFUSED at http://host.docker.internal:6006?path=... with docker-chromium having run under jet-puppeteer-docker in Jetbrains Idea (only there). The only way to fix it was found:

  • Install Docker Desktop and clone folder c:\Program Files\Docker as c:\Program Files\Docker_. After that Docker Desktop can be removed.
  • Then make steps above to unpack binaries. Then copy with a replace files docker.exe and com.docker.cli.exe from c:\Program Files\Docker_\Docker\resources\bin\ to c:\Program Files\Docker.
  • I do not know why it is so. Despite docker.exe is just client to dockerd in WSL, but it makes some difference. When we use not-patched version of docker.exe then /etc/hosts does not contains needed alias. But when we use patched version of docker.exe from Docker Desktop (with required com.docker.cli.exe) then it gets OK. I suggested that a problem is hidden in difference of versions and checked it. Probably, Nope.
    • Docker 20.10.10, build b485636 from Docker Desktop
    • Docker 20.10.9, build c2ea9bc from binaries
    • Docker 20.10.10, build b485636 from binaries
  • At the same time do not touch docker-compose.exe, leave original version
@gorshkov-leonid
Copy link
Author

gorshkov-leonid commented Nov 6, 2021 •
edited
Loading

  • Configure dockerd to run...
    • ❌ on Windows startup using as Service: 
      $scvBinaryPath = '"' + "C:\Windows\System32\wsl.exe -u root bash -c "+'\"' + "source /etc/profile && rundocker &"+'\"' + '"'
New-Service -Name "Docker" -BinaryPathName $scvBinaryPath
## remove:
# sc.exe delete Docker
      But it cannot be started as service

      Service applications implement the Windows Services API (start running, stop running, report status, pause, resume running, prepare for system shutdown) — a set of operations used to control and monitor services. Windows uses the API to interact with Service Applications. Unfortunately regular desktop applicationsdo not support the Windows Services API... as well as WSL...

@gorshkov-leonid
Copy link
Author

gorshkov-leonid commented Nov 6, 2021 •
edited
Loading

are "npipe:////./pipe/docker_engine" and "unix:///var/run/docker.sock" compatible?
https://developpaper.com/the-tutorial-of-installing-docker-in-ubuntu-wsl-of-win10-subsystem/
https://github.com/jstarks/npiperelay
https://docs.microsoft.com/en-us/virtualization/community/team-blog/2017/20171208-wsl-interoperability-with-docker
https://github.com/jstarks/npiperelay/blob/master/scripts/docker-relay

@gorshkov-leonid
Copy link
Author

gorshkov-leonid commented Nov 7, 2021 •
edited
Loading

How to install k8s dashboard

GITHUB_URL=https://github.com/kubernetes/dashboard/releases
VERSION_KUBE_DASHBOARD=$(curl -w '%{url_effective}' -I -L -s -S ${GITHUB_URL}/latest -o /dev/null | sed -e 's|.*/||')
kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/${VERSION_KUBE_DASHBOARD}/aio/deploy/recommended.yaml

cat | tee ./dashboard.admin-user.yml > /dev/null << 'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
EOF

cat | tee ./dashboard.admin-user-role.yml > /dev/null << 'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard
EOF

kubectl create -f dashboard.admin-user.yml -f dashboard.admin-user-role.yml
kubectl -n kubernetes-dashboard describe secret admin-user-token | grep '^token'
kubectl proxy

Open http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ and use login admin-user and obtained berer token as password.

@gorshkov-leonid
Copy link
Author

gorshkov-leonid commented Jan 29, 2023 •
edited
Loading

Ubuntu 22.04

WARN[2023-01-29T20:53:30.973064814+04:00] grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout". Reconnecting...  module=grpc
failed to start daemon: Error initializing network controller: error obtaining controller instance: unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain:  (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.7 (nf_tables):  RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1

microsoft/WSL#6655 (comment)

Solution:

sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

@gorshkov-leonid
Copy link
Author

docker run and $
https://superuser.com/questions/1628497/docker-exec-with-dollar-variable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment