docker-for-mac k8s issue

k8s-isuue.md

After my DockerForMac update itself to Version 17.12.0-ce-rc4-mac44 (21438) k8s is not starting.

Exited containers

after nsentering the VM, i found 2 containers exiting:

/ # docker ps --filter status=exited
CONTAINER ID        IMAGE                                           COMMAND                  CREATED             STATUS                            PORTS               NAMES
b2b6f7eba42e        gcr.io/google_containers/kube-apiserver-amd64   "kube-apiserver --ad…"   2 minutes ago       Exited (255) About a minute ago                       k8s_kube-apiserver_kube-apiserver-docker-for-desktop_kube-system_9999fff5022435b112a99158001bcaa3_10
396e8e638970        gcr.io/google_containers/etcd-amd64             "etcd --listen-clien…"   3 minutes ago       Exited (2) 3 minutes ago                              k8s_etcd_etcd-docker-for-desktop_kube-system_8cf2571a4c3f64cf3429d6f601861eec_10

apiserver is dependant on etcd, so first i've looked etcds log:

panic: pgid (137438953476) above high water mark (586)

goroutine 1 [running]:
panic(0xd460a0, 0xc82039db20)
	/usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/coreos/etcd/cmd/vendor/github.com/boltdb/bolt.(*node).spill(0xc8201e62a0, 0x0, 0x0)
	/home/gyuho/go/src/github.com/coreos/etcd/cmd/vendor/github.com/boltdb/bolt/node.go:375 +0x54e
github.com/coreos/etcd/cmd/vendor/github.com/boltdb/bolt.(*node).spill(0xc8201e6230, 0x0, 0x0)
	/home/gyuho/go/src/github.com/coreos/etcd/cmd/vendor/github.com/boltdb/bolt/node.go:350 +0x12c
github.com/coreos/etcd/cmd/vendor/github.com/boltdb/bolt.(*node).spill(0xc8201e61c0, 0x0, 0x0)
	/home/gyuho/go/src/github.com/coreos/etcd/cmd/vendor/github.com/boltdb/bolt/node.go:350 +0x12c

Then I checked the /var/lib/etcd/ dir, and found some strange files:

/ # find  /var/lib/etcd/member/ -type f
/var/lib/etcd/member/wal/000000000000001c-000000000048d057.wal
/var/lib/etcd/member/wal/000000000000001d-00000000004b6d44.wal.broken

Let's just delete them all, what can go wrong ? ¯\_(ツ)_/¯

find  /var/lib/etcd/member/ -type f|xargs rm -f

Restart

After the restart etcd seems fine. But api server still has issues:

E0104 12:12:49.245521       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:73: Failed to list *settings.PodPreset: the server could 
not find the requested resource (get podpresets.settings.k8s.io)

so its still not working ...

This gist will be updated ...