k3s on WSL2

README.md

Instructions to hack up WSL2 on Windows 10 Build 18917 to run k3s (Kubernetes) and rio

Install WSL2

https://docs.microsoft.com/en-us/windows/wsl/wsl2-install

I already had Ubuntu-18.04 installed in wsl 1. So I just did wsl --set-version Ubuntu-18.04 2

Compile Kernel

Using Ubuntu 18.04 (I'm sure any distro will work), inside WSL2 download https://thirdpartysource.microsoft.com/download/Windows%20Subsystem%20for%20Linux%20v2/May%202019/WSLv2-Linux-Kernel-master.zip and extract to a folder. The latest version of the kernel source is available at (https://github.com/microsoft/WSL2-Linux-Kernel)

In the extracted kernel source run

# Install basically "compile kernel" crap
sudo apt install build-essential flex bison libssl-dev libelf-dev

# Setup kernel config
cp Microsoft/config-wsl .config
cat >> .config << EOF
CONFIG_BRIDGE_NETFILTER=y
CONFIG_NETFILTER_XT_MATCH_COMMENT=y
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=y
CONFIG_NETFILTER_XT_MATCH_OWNER=y
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=y
CONFIG_VXLAN=y
CONFIG_GENEVE=y
EOF
make oldconfig

# Compile and drink beverage of choice
make -j $(nproc)

Update WSL kernel

1. Copy arch/x86/boot/bzImage to somewhere outside WSL2 (/mnt/c/Users/${USER}/Desktop)
2. Shutdown WSL2 wsl --shutdown
3. Copy bzImage to c:\Windows\System32\lxss\tools\kernel. Make sure you backup the existing kernel file (although the original is still in another location). I had to copy through explorer.exe because of permissions issues.

Install K3s

You can't install k3s using the curl script because there is no supervisor (systemd or openrc) in WSL2.

1. Download k3s binary from https://github.com/rancher/k3s/releases/latest
2. chmod +x k3s
3. Run k3s sudo ./k3s server

Setup access

1. Copy /etc/rancher/k3s/k3s.yaml from WSL to your home in Windows to %HOME%.kube\config. Edit the copied file and change the server URL from https://localhost:6443 to the IP of the your WSL2 instance (ip addr show dev eth0). So something like https://192.168.170.170:6443.
2. Run kubectl from windows (got to download kubectl.exe from somewhere)

Bonus Install Rio

1. Download rio-windows-amd64 from https://github.com/rancher/rio/releases/tag/v0.2.0-rc1 to rio.exe
2. Run ./rio install
3. Start doing stuff with rio :)

Awesome! ☸️😲👍

Awesome. Just got it to work :)

Made a little script to make the install somewhat easier. It creates a brand new wsl2 named liifi-k3s to avoid collisions. It uses the official docker image to create the WSL2 distro rancher/k3s:v1.0.0

If you run the following on the latest wsl2 enabled windows 10 (make sure you have it set as the default), you should get k3s with rio on it, inside WSL2. It will also add k3s and rio cli using scoop.

iex (new-object net.webclient).downloadstring('https://gist.github.com/juliostanley/622486d82cb8ed334d270ead5f684e89/raw/cc39b22add65d71c32dc4e9f9a227c1ce1e3d7e3/install-k3s-rio-wsl2.ps1')

Here is a README with more details.

Also, you can point VSCode to //wsl$/liifi-k3s/etc/rancher/k3s/k3s.yaml but there is a caveat (you must first modify the file to use localhost instead of 127.0.0.1, the above script will take care of it)

k3s server runs nicely on WSL 2 - kernel configs not needed anymore.

Combining it with microsoft/WSL#4150 forwarding port 6443 from the host to WSL makes your server accessible for agents in the network.

if you're looking for the cleanup scripts, this is the way how to get them

# download installer
curl -fsSL https://raw.githubusercontent.com/rancher/k3s/release-1.19/package/rpm/install.sh -o install.sh
# make it executable
chmod +x install.sh
# comment out non-relevant steps at the end of the file, preserve setup_env, create_killall, create_uninstall steps
sed -e '/verify_system$/ s/^#*/#/' -i install.sh
sed -e '/download_and_verify$/ s/^#*/#/' -i install.sh
sed -e '/create_symlinks$/ s/^#*/#/' -i install.sh
sed -e '/systemd_disable$/ s/^#*/#/' -i install.sh
sed -e '/create_env_file$/ s/^#*/#/' -i install.sh
sed -e '/create_service_file$/ s/^#*/#/' -i install.sh
sed -e '/service_enable_and_start$/ s/^#*/#/' -i install.sh
# generate cleanup scripts
INSTALL_K3S_BIN_DIR="." ./install.sh
# uninstall
./k3s-uninstall.sh

If you take a few pieces out of this post you can get systemd running and then k3s's regular install just works. No k3s workflow modifications needed.
https://forum.snapcraft.io/t/running-snaps-on-wsl2-insiders-only-for-now/13033

• apt-get install daemonize
• /usr/sbin/start-systemd-namespace
• /usr/sbin/enter-systemd-namespace
• sudoers
• /etc/bash.bashrc
• Windows env

Now to get this started somehow automagically when windows restarts (probably have to auto log in user and lock the computer). (windows is a required platform for a few of our customers)

unfortunately neither do not work with curl -sfL https://get.k3s.io | sh -s - --docker (docker runtime)

this sets up systemd in a way that persists between reboots: https://github.com/DamionGans/ubuntu-wsl2-systemd-script

however, because microsoft/WSL#4150 doesn't support UDP, and the default CNI is flannel VXLAN which requires UDP, I don't think it's possible to run k3s on WSL without changing the CNI to something else.

haven't tried other CNIs yet. not sure which options, if any, are UDP-free.

Thanks!
please note that '/etc/rancher/k3s/k3s.yaml' should be copied to %USERPROFILE%.kube\config
and ip address of WSL can be get by 'hostname -I' too

Thanks! please note that '/etc/rancher/k3s/k3s.yaml' should be copied to %USERPROFILE%.kube\config and ip address of WSL can be get by 'hostname -I' too

is it hostname -i ?