zalando-ingress-controller-notes.md

zalando-ingress-controller-notes.md

Links

1. The controller https://github.com/zalando-incubator/kube-ingress-aws-controller
2. External-dns: https://github.com/kubernetes-sigs/external-dns
3. stack creation/update issue: zalando-incubator/kube-ingress-aws-controller#296
4. stack namings: zalando-incubator/kube-ingress-aws-controller#297
5. How it works: https://gist.github.com/icy/9996ef070b0f3c28001f6f2474330c0e

Problems or Goals

1. We need to update zalando controller settings
2. We want to have zero-downtime migration

What makes migration hard?

1. External-dns may be so fast to update ingress dns while the LB is not ready yet
2. Zalando controller may create new cloudformation stack at the same time it would delete the old load balance (if your stack is not protected.) This may cause a situation when new LB is not ready but the old LB is gone
3. When changing ingress class, the dnsname may not be ready and your service is still resolved to the old LB address, you would encounter in 404 issues.

Zero-downtime migration

1. Remove external-dns deployment before you update zalando controller settings for sure
2. When you changing ingress class, be sure to have a temporary ingress controller of the old class name that haves your service address as server alias. Then wait a few minutes/hours until your dns resolvers clearly see new addresses, then you can safely delete temporary ingresses.

The statement Zalando controller may create new cloudformation stack at the same time it would delete the old load balance (if your stack is not protected.) This may cause a situation when new LB is not ready but the old LB is gone is wrong.
We mark the stack as to be deleted, before we really delete it after 30 minutes ('default value can be changed).
See also zalando-incubator/kube-ingress-aws-controller#296 (comment)