kube-vip crash loop

time="2022-08-19T20:41:01Z" level=info msg="server started"
time="2022-08-19T20:41:01Z" level=info msg="Starting Kube-vip Manager with the ARP engine"
time="2022-08-19T20:41:01Z" level=info msg="Namespace [kube-system], Hybrid mode [true]"
time="2022-08-19T20:41:01Z" level=info msg="Beginning cluster membership, namespace [kube-system], lock name [plndr-svcs-lock], id [Rancher-Master-P4-1]"
I0819 20:41:01.712210 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-svcs-lock...
time="2022-08-19T20:41:01Z" level=info msg="Beginning cluster membership, namespace [kube-system], lock name [plndr-cp-lock], id [Rancher-Master-P4-1]"
I0819 20:41:01.712944 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-cp-lock...
time="2022-08-19T20:41:01Z" level=info msg="Node [Rancher-Master-P4-2] is assuming leadership of the cluster"
time="2022-08-19T20:41:01Z" level=info msg="new leader elected: Rancher-Master-P4-2"
E0819 20:41:16.625540 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: leases.coordination.k8s.io "plndr-svcs-lock" is forbidden: User "system:serviceaccount:kube-system:kube-vip" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-system"
E0819 20:41:17.223180 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: leases.coordination.k8s.io "plndr-cp-lock" is forbidden: User "system:serviceaccount:kube-system:kube-vip" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-system"
E0819 20:41:17.873042 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: leases.coordination.k8s.io "plndr-svcs-lock" is forbidden: User "system:serviceaccount:kube-system:kube-vip" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-system"
E0819 20:41:18.658726 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: leases.coordination.k8s.io "plndr-cp-lock" is forbidden: User "system:serviceaccount:kube-system:kube-vip" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-system"
E0819 20:41:19.562728 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: leases.coordination.k8s.io "plndr-svcs-lock" is forbidden: User "system:serviceaccount:kube-system:kube-vip" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-system"
E0819 20:41:20.697186 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: leases.coordination.k8s.io "plndr-cp-lock" is forbidden: User "system:serviceaccount:kube-system:kube-vip" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-system"
E0819 20:41:20.917367 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: leases.coordination.k8s.io "plndr-svcs-lock" is forbidden: User "system:serviceaccount:kube-system:kube-vip" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-system"
E0819 20:41:22.058775 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Unauthorized
E0819 20:41:22.825286 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: Unauthorized
E0819 20:41:23.311379 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Unauthorized
E0819 20:41:24.871039 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: Unauthorized
E0819 20:41:25.152659 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Unauthorized
E0819 20:41:26.192032 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Unauthorized
E0819 20:41:26.504418 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: Unauthorized
E0819 20:41:27.386466 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Unauthorized
E0819 20:41:28.237904 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: Unauthorized
E0819 20:41:29.338939 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: Unauthorized
E0819 20:41:29.561762 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Unauthorized
E0819 20:41:30.637328 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Unauthorized
E0819 20:41:31.056973 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-svcs-lock: Unauthorized
time="2022-08-19T20:41:32Z" level=info msg="Received termination, signaling shutdown"
time="2022-08-19T20:41:32Z" level=info msg="Received termination, signaling shutdown"
panic: close of nil channel
goroutine 7 [running]:
github.com/kube-vip/kube-vip/pkg/cluster.(*Cluster).Stop(...)
/src/pkg/cluster/cluster.go:53
github.com/kube-vip/kube-vip/pkg/manager.(*Manager).startARP.func1()
/src/pkg/manager/manager_arp.go:34 +0x9e
created by github.com/kube-vip/kube-vip/pkg/manager.(*Manager).startARP
/src/pkg/manager/manager_arp.go:30 +0x134

Hi! I came across your Reddit post when doing a Google search. I'm experiencing exactly the same problem - did you ever find a fix??

@CaptainStealthy As this was awhile ago I don't remember the exact fix. IIRC, I ended up having to kill the entire cluster and start over, but I could be wrong on that.

At any rate, the most helpful thing I found was to join the TechnoTim discord server. They have guys that do this all the time and can really help you diagnose issues. They were instrumental in helping me. https://l.technotim.live/discord