Skip to content

Instantly share code, notes, and snippets.

@aojea

aojea/README.md

Created January 30, 2023 17:55
Show Gist options
  • Save aojea/30ee9ff53022d6b3e6156c4d2e570852 to your computer and use it in GitHub Desktop.
Save aojea/30ee9ff53022d6b3e6156c4d2e570852 to your computer and use it in GitHub Desktop.
Download ZIP
Tolerate unready endpoints using annotations with Kubernetes (deprecated in 1.24)
Raw
README.md

Version 1.23

$ kubectl apply -f repro.yaml
deployment.apps/server-deployment created
service/service-annotated unchanged
service/service-field unchanged

$ kubectl get pods
NAME                                 READY   STATUS    RESTARTS   AGE
server-deployment-7b5f976b95-bpbnd   0/1     Running   0          2m41s
server-deployment-7b5f976b95-ht8qz   0/1     Running   0          2m41s

$ kubectl get services
NAME                TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes          ClusterIP   10.56.16.1     <none>        443/TCP   26m
service-annotated   ClusterIP   10.56.24.235   <none>        80/TCP    2m47s
service-field       ClusterIP   10.56.25.214   <none>        80/TCP    2m47s

The annotation works for Endpoints

$ kubectl get endpoints
NAME                ENDPOINTS                   AGE
kubernetes          10.154.0.18:443             26m
service-annotated   10.60.1.4:80,10.60.2.5:80   2m50s
service-field       10.60.1.4:80,10.60.2.5:80   2m49s

$ kubectl get endpointslices
NAME                      ADDRESSTYPE   PORTS   ENDPOINTS             AGE
kubernetes                IPv4          443     10.154.0.18           26m
service-annotated-nn2cd   IPv4          80      10.60.2.5,10.60.1.4   2m53s
service-field-h8jg7       IPv4          80      10.60.2.5,10.60.1.4   2m52s

The annotation DOES NOT WORK for EndpointSlices, see the conditions.ready field is false for annotation

$ kubectl get endpointslices service-annotated-nn2cd  -o yaml
addressType: IPv4
apiVersion: discovery.k8s.io/v1
endpoints:
- addresses:
  - 10.60.2.5
  conditions:
    ready: false
    serving: false
    terminating: false
  nodeName: gke-aojea-122-default-pool-be26bb2c-wcsg
  targetRef:
    kind: Pod
    name: server-deployment-7b5f976b95-ht8qz
    namespace: default
    resourceVersion: "11515"
    uid: a25a0bab-c557-4b42-8516-639e971aa9a5
  zone: europe-west2-c
- addresses:
  - 10.60.1.4
  conditions:
    ready: false
    serving: false
    terminating: false
  nodeName: gke-aojea-122-default-pool-be26bb2c-2489
  targetRef:
    kind: Pod
    name: server-deployment-7b5f976b95-bpbnd
    namespace: default
    resourceVersion: "11525"
    uid: 10377f3e-cbe6-44a2-a417-9e77eb7575a1
  zone: europe-west2-c
kind: EndpointSlice
metadata:
  creationTimestamp: "2023-01-30T09:04:43Z"
  generateName: service-annotated-
  generation: 3
  labels:
    endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io
    kubernetes.io/service-name: service-annotated
  name: service-annotated-nn2cd
  namespace: default
  ownerReferences:
  - apiVersion: v1
    blockOwnerDeletion: true
    controller: true
    kind: Service
    name: service-annotated
    uid: fc17f7e8-4436-47ec-83e9-02ef5f70a5ce
  resourceVersion: "11529"
  uid: 377fac1b-526c-4b84-97c4-74806b18eb04
ports:
- name: ""
  port: 80
  protocol: TCP

conditions.ready field is true when using the field

$ kubectl get endpointslices service-field-h8jg7 -o yaml
addressType: IPv4
apiVersion: discovery.k8s.io/v1
endpoints:
- addresses:
  - 10.60.2.5
  conditions:
    ready: true
    serving: false
    terminating: false
  nodeName: gke-aojea-122-default-pool-be26bb2c-wcsg
  targetRef:
    kind: Pod
    name: server-deployment-7b5f976b95-ht8qz
    namespace: default
    resourceVersion: "11515"
    uid: a25a0bab-c557-4b42-8516-639e971aa9a5
  zone: europe-west2-c
- addresses:
  - 10.60.1.4
  conditions:
    ready: true
    serving: false
    terminating: false
  nodeName: gke-aojea-122-default-pool-be26bb2c-2489
  targetRef:
    kind: Pod
    name: server-deployment-7b5f976b95-bpbnd
    namespace: default
    resourceVersion: "11525"
    uid: 10377f3e-cbe6-44a2-a417-9e77eb7575a1
  zone: europe-west2-c
kind: EndpointSlice
metadata:
  creationTimestamp: "2023-01-30T09:04:44Z"
  generateName: service-field-
  generation: 3
  labels:
    endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io
    kubernetes.io/service-name: service-field
  name: service-field-h8jg7
  namespace: default
  ownerReferences:
  - apiVersion: v1
    blockOwnerDeletion: true
    controller: true
    kind: Service
    name: service-field
    uid: 3dd86e7f-6b1b-4a97-8fc6-98f4d369b1f1
  resourceVersion: "11527"
  uid: d8ee5ed0-e55a-4af1-bc23-df432c52608d
ports:
- name: ""
  port: 80
  protocol: TCP

Version 1.26

Annotation removed in 1.24, since then does not work for Endpoint neither EndpointSlcies

$ kubectl apply -f repro.yaml
deployment.apps/server-deployment created
service/service-annotated unchanged
service/service-field unchanged

$ kubectl get pods
NAME                                 READY   STATUS    RESTARTS   AGE
server-deployment-69cb976cf6-kpn65   0/1     Running   0          2m
server-deployment-69cb976cf6-r4ndk   0/1     Running   0          2m

$ kubectl get services
NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes          ClusterIP   10.96.0.1       <none>        443/TCP   16d
service             ClusterIP   10.96.8.78      <none>        80/TCP    8d
service-annotated   ClusterIP   10.96.241.189   <none>        80/TCP    10m
service-field       ClusterIP   10.96.182.249   <none>        80/TCP    11m

$ kubectl get endpoints
NAME                ENDPOINTS                      AGE
kubernetes          192.168.8.2:6443               16d
service                                            8d
service-annotated                                  10m
service-field       10.244.0.10:80,10.244.0.9:80   11m

$  kubectl get endpointslices
NAME                      ADDRESSTYPE   PORTS   ENDPOINTS                AGE
kubernetes                IPv4          6443    192.168.8.2              16d
service-5lsb4             IPv4          80      10.244.0.9,10.244.0.10   8d
service-annotated-59kxq   IPv4          80      10.244.0.9,10.244.0.10   11m
service-field-d8hcj       IPv4          80      10.244.0.9,10.244.0.10   12m

$ kubectl get endpointslices service-annotated-59kxq -o yaml
addressType: IPv4
apiVersion: discovery.k8s.io/v1
endpoints:
- addresses:
  - 10.244.0.9
  conditions:
    ready: false
    serving: false
    terminating: false
  nodeName: kind-control-plane
  targetRef:
    kind: Pod
    name: server-deployment-69cb976cf6-r4ndk
    namespace: default
    uid: b4d746bb-2468-4660-a6c7-c19afb5b6da7
- addresses:
  - 10.244.0.10
  conditions:
    ready: false
    serving: false
    terminating: false
  nodeName: kind-control-plane
  targetRef:
    kind: Pod
    name: server-deployment-69cb976cf6-kpn65
    namespace: default
    uid: 78d56981-eaf3-4369-802b-500ad83fbca8
kind: EndpointSlice
metadata:
  creationTimestamp: "2023-01-30T08:44:58Z"
  generateName: service-annotated-
  generation: 10
  labels:
    endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io
    kubernetes.io/service-name: service-annotated
  name: service-annotated-59kxq
  namespace: default
  ownerReferences:
  - apiVersion: v1
    blockOwnerDeletion: true
    controller: true
    kind: Service
    name: service-annotated
    uid: d348cdc1-7afb-43ea-a73d-661a30176bb0
  resourceVersion: "1918618"
  uid: 1aff7ad9-4719-4f40-afed-b28ba2ddd00f
ports:
- name: ""
  port: 80
  protocol: TCP
Raw
reptro.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: server-deployment
labels:
app: MyApp
spec:
replicas: 2
selector:
matchLabels:
app: MyApp
template:
metadata:
labels:
app: MyApp
spec:
containers:
- name: agnhost
image: k8s.gcr.io/e2e-test-images/agnhost:2.21
args:
- netexec
- --http-port=80
- --udp-port=80
ports:
- containerPort: 80
readinessProbe:
exec:
command:
- exit 1
initialDelaySeconds: 5
periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: service-annotated
annotations:
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
type: ClusterIP
selector:
app: MyApp
ports:
- protocol: TCP
port: 80
targetPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: service-field
spec:
type: ClusterIP
selector:
app: MyApp
publishNotReadyAddresses: true
ports:
- protocol: TCP
port: 80
targetPort: 80
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment