[Minikube on WSL2 with Podman] Tips for running a local development Kubernetes cluster on WSL2 with Podman #wsl2 #minikube #podman #kubernetes #docker

minikube-on-wsl2-with-podman.md

minikube on WSL2 with Podman

Tips for running a single-node Kubernetes cluster on WSL2 with Podman

Instructions

1. Install Podman

2. Edit /etc/containers/containers.conf:

   • Set cgroup_manager to cgroupfs
   • Set events_logger to file

3. Install minikube

4. Run minikube start --driver=podman

Local Registry

1. Enable minikube local registry addon:

    minikube addons enable registry
   

2. Install docker-ce-cli package for your distro

3. Configure the environment to use minikube's Docker daemon (ie. eval $(minikube -p minikube docker-env))

4. Push/pull images to/from localhost:5000/imagename

Note: Instead of pushing/pulling images into/from the registry, you can build and tag an image and set containers' imagePullPolicy to IfNotPresent or Never

Reaching minikube from Windows (Minikube <= v1.11.x)

1. Make sure minikube is started (ie. run minikube start)

2. Open an elevated PowerShell prompt and execute the following commands:

    # Run "wsl --list" and make sure the default WSL distro is set correctly
    
    $minikube_ip = "$(wsl minikube ip)"
    $wsl_ip = "$(wsl sh -c "ip addr show `$(ip route list default | grep -Po ' dev \K\w+') | grep -Po ' inet \K[\d.]+'")"
    route delete $minikube_ip 2>&1 | Out-Null
    route add $minikube_ip mask 255.255.255.255 $wsl_ip
   

Note: You can also route the whole podman network subnet instead of only the minikube ip

Note: Instructions above should be followed after every restart

Troubleshooting

Podman does not play nicely with iptables-nft

Some distros like Debian, use the nftables framework by default. So in case of getting error messages like:

Error adding network: failed to list chains: running [...] iptables [...] (nf_tables)

Try switching to the legacy version of iptables.

Podman might not be able to pull the kicbase image in 240 seconds

The timeout value for timedCreateHost is currently hardcoded in the Minikube source-code!

In case minikube terminates with an error message like:

StartHost failed, but will try again: creating host: create host timed out in 240.000000 seconds

Try pulling the image manually before starting minikube:

BASE_IMAGE="$(minikube start --help | sed -En "s:^.*base-image='([^@]+).*:\1:p")"
sudo podman pull $BASE_IMAGE

Can you please expand on what all these settings do? Podman on Windows using WSL works great without minikube;
what does step # 2 do? [Edit /etc/containers/containers.conf]
why is docker-ce-cli needed if minikube supports podman? https://minikube.sigs.k8s.io/docs/drivers/podman/

Hi @dancohenenterprise

Thank you for your comment. This is an old gist and as you mentioned Docker works perfectly fine nowadays in WSL thanks to the built-in systemd support. IIRC the reason I used podman back then was to work around the issues I had with running the Docker Engine daemon.

@fardjad interesting; I would like to update the gist; I would like to stay away from docker due to the enterprise licensing requirements. I'd like to use WSL, Podman for Windows (uses WSL machine) and Any good K8s for Desktop distribution (minikube, etc). If you have guidance, I would much appreciate it; I have tried rootless podman + minikube on Windows, but this does not seem to work; I will try rootful config next

I’ll be happy to update this gist if you find out how to make it work or please feel free to fork it and add your own instructions.

As far as I know, Docker engine (not Docker Desktop) is still free to use for enterprises and you can enable systemd (see the link in my previous comment) and simply install docker engine within your WSL distro.