kun432/index.md

Last active October 15, 2021 03:58

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/kun432/be181a347d7a5e857d2083a342742305.js"></script>
Save kun432/be181a347d7a5e857d2083a342742305 to your computer and use it in GitHub Desktop.

Download ZIP

service

Raw

index.md

トポロジを意識したservice転送

Topology-aware service routing
- クラスタがリージョンやAZにまたがっていてもserviceは意識せずに転送する
- またがるとレイテンシーやパフォーマンスの低下
- ノード数が多いと起きやすい
externalTrafficPolicy: Localでリスクは下げれるが・・・
- ClusterIPでは不可
- 同一ノード内でしか転送できない、同一ノード内にpodがいなければタイムアウト
Topology-aware service routingにより回避できる
ただし、
- 送信元IPは取れない
- Topology-aware service routingとexternalTrafficPolicy: Localは併用できない

Topology-aware service routingの設定

どのトポロジに転送するかを複数の優先度で判断する
例
- 1. 同一ノード
- 1. 同一ゾーン
- 1. どれか

kindで3node用意してやってみる. alplaなfeatureなのでfeatureGatesで有効にしてあげる

apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
name: sample
nodes:
- role: control-plane
- role: worker
  kubeadmConfigPatches:
  - |
    kind: JoinConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 30080
    hostPort: 30001
    protocol: TCP
  - containerPort: 30443
    hostPort: 30011
    protocol: TCP
- role: worker
  kubeadmConfigPatches:
  - |
    kind: JoinConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 30080
    hostPort: 30002
    protocol: TCP
  - containerPort: 30443
    hostPort: 30012
    protocol: TCP
- role: worker
  kubeadmConfigPatches:
  - |
    kind: JoinConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 30080
    hostPort: 30003
    protocol: TCP
  - containerPort: 30443
    hostPort: 30013
    protocol: TCP
featureGates:
  ServiceTopology: true

topologyKeysを使う

同じ kubernetes.io/hostname のラベルがあるnode内が最優先
そうでなければいずれか（"*"）

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: bootcamp
  name: bootcamp-deployment
spec:
  replicas: 4
  selector:
    matchLabels:
      app: bootcamp
  template:
    metadata:
      labels:
        app: bootcamp
    spec:
      containers:
      - image: gcr.io/google-samples/kubernetes-bootcamp:v1
        name: bootcamp
        ports:
        - name: "http"
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: bootcamp-service
spec:
  type: ClusterIP
  ports:
  - name: "http-port"
    protocol: "TCP"
    port: 8888
    targetPort: 8080
  selector:
    app: bootcamp
  topologyKeys:
  - kubernetes.io/hostname
  - "*"

$ kubectl get pod -o wide
NAME                                  READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE   READINESS GATES
bootcamp-deployment-8d6697479-cjckn   1/1     Running   0          56s   10.244.1.2   sample-worker2   <none>           <none>
bootcamp-deployment-8d6697479-rnp6l   1/1     Running   0          56s   10.244.1.3   sample-worker2   <none>           <none>
bootcamp-deployment-8d6697479-v7qdj   1/1     Running   0          56s   10.244.2.2   sample-worker3   <none>           <none>
bootcamp-deployment-8d6697479-xs8xb   1/1     Running   0          56s   10.244.3.2   sample-worker    <none>           <none>

nodeportにアクセスしてみる. node-1のpodしか応答しない。

$ curl localhost:30001
Hello Kubernetes bootcamp! | Running on: bootcamp-deployment-8d6697479-xs8xb | v=1

$ curl localhost:30001
Hello Kubernetes bootcamp! | Running on: bootcamp-deployment-8d6697479-xs8xb | v=1

$ curl localhost:30001
Hello Kubernetes bootcamp! | Running on: bootcamp-deployment-8d6697479-xs8xb | v=1

複数podが立っているnode-2にアクセスしてみる。node-2のpodのどちらかが応答する

$ curl localhost:30002
Hello Kubernetes bootcamp! | Running on: bootcamp-deployment-8d6697479-rnp6l | v=1

$ curl localhost:30002
Hello Kubernetes bootcamp! | Running on: bootcamp-deployment-8d6697479-cjckn | v=1

$ curl localhost:30002
Hello Kubernetes bootcamp! | Running on: bootcamp-deployment-8d6697479-cjckn | v=1

$ curl localhost:30002
Hello Kubernetes bootcamp! | Running on: bootcamp-deployment-8d6697479-rnp6l | v=1

topologyKeysに指定可能なラベルは以下の3種類

kubernetes.io/hostname
topology.kubernetes.io/zone
topology.kubernetes.io/region

"*"を指定する場合は最後に指定すること

Author

kun432 commented Oct 13, 2021 •

edited

Loading

Headless Service

対象となる個々のpodのIPアドレスがかえってくるservie
- 他のserviceのような負荷分散のためのエンドポイントを返さない
- DNSラウンドロビンとして使用する
- statefulsetに限り、pod名での名前解決が可能
条件
- ClusterIP serviceのみ
- spec.clusterIPがNone
- statefulsetの場合
  - serviceのmetadata.nameとstatefulsetのspec.servicenameが同じ

statefulset以外（deployment）の場合

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: bootcamp
  name: bootcamp-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: bootcamp
  template:
    metadata:
      labels:
        app: bootcamp
    spec:
      containers:
      - image: gcr.io/google-samples/kubernetes-bootcamp:v1
        name: bootcamp
        ports:
        - name: "http"
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: bootcamp-service
spec:
  type: ClusterIP
  clusterIP: None
  ports:
  - name: "http-port"
    protocol: "TCP"
    port: 8888
    targetPort: 8080
  selector:
    app: bootcamp

headlessではclusterIPがない

$ kubectl get service
NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
bootcamp-service   ClusterIP   None         <none>        8888/TCP   2m8s
kubernetes         ClusterIP   10.96.0.1    <none>        443/TCP    29m

名前解決してみる

$ kubectl run --restart=Never dnstest --rm --image=tutum/dnsutils -ti --command -- bash

root@dnstest:/# dig bootcamp-service.default.svc.cluster.local

; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> bootcamp-service.default.svc.cluster.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21625
;; flags: qr aa rd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bootcamp-service.default.svc.cluster.local. IN A

;; ANSWER SECTION:
bootcamp-service.default.svc.cluster.local. 30 IN A 10.244.3.3
bootcamp-service.default.svc.cluster.local. 30 IN A 10.244.2.4
bootcamp-service.default.svc.cluster.local. 30 IN A 10.244.2.3

;; Query time: 9 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Tue Oct 12 05:58:26 UTC 2021
;; MSG SIZE  rcvd: 245

nodeportとかの場合だとこうなるので違いがわかる

root@dnstest:/# dig bootcamp-service.default.svc.cluster.local

; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> bootcamp-service.default.svc.cluster.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37979
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bootcamp-service.default.svc.cluster.local. IN A

;; ANSWER SECTION:
bootcamp-service.default.svc.cluster.local. 23 IN A 10.96.170.115

;; Query time: 1 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Tue Oct 12 06:03:46 UTC 2021
;; MSG SIZE  rcvd: 129

statefulsetだとpod名で名前解決可能. statefulsetのspec.serviceNameとserviceのmetadata.nameが同じである必要がある。

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: bootcamp
  name: bootcamp-statefulset
spec:
  replicas: 3
  selector:
    matchLabels:
      app: bootcamp
  serviceName: bootcamp-service
  template:
    metadata:
      labels:
        app: bootcamp
    spec:
      containers:
      - image: gcr.io/google-samples/kubernetes-bootcamp:v1
        name: bootcamp
        ports:
        - name: "http"
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: bootcamp-service
spec:
  type: ClusterIP
  clusterIP: None
  ports:
  - name: "http-port"
    protocol: "TCP"
    port: 8888
    targetPort: 8080
  selector:
    app: bootcamp

$ kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
bootcamp-statefulset-0   1/1     Running   0          54s
bootcamp-statefulset-1   1/1     Running   0          53s
bootcamp-statefulset-2   1/1     Running   0          52s

$ kubectl run --restart=Never dnstest --rm --image=tutum/dnsutils -ti --command -- bash

root@dnstest:/# dig bootcamp-service.default.svc.cluster.local

; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> bootcamp-service.default.svc.cluster.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20399
;; flags: qr aa rd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bootcamp-service.default.svc.cluster.local. IN A

;; ANSWER SECTION:
bootcamp-service.default.svc.cluster.local. 30 IN A 10.244.2.10
bootcamp-service.default.svc.cluster.local. 30 IN A 10.244.1.10
bootcamp-service.default.svc.cluster.local. 30 IN A 10.244.3.9

;; Query time: 10 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Fri Oct 15 03:47:01 UTC 2021
;; MSG SIZE  rcvd: 245

root@dnstest:/# dig bootcamp-statefulset-0.bootcamp-service.default.svc.cluster.local

; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> bootcamp-statefulset-0.bootcamp-service.default.svc.cluster.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51492
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bootcamp-statefulset-0.bootcamp-service.default.svc.cluster.local. IN A

;; ANSWER SECTION:
bootcamp-statefulset-0.bootcamp-service.default.svc.cluster.local. 30 IN A 10.244.2.11

;; Query time: 1 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Fri Oct 15 03:54:54 UTC 2021
;; MSG SIZE  rcvd: 175

pod名.service名やpod名.service名.namespace名でもひける

root@dnstest:/# cat /etc/resolv.conf
search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.96.0.10
options ndots:5

root@dnstest:/# dig bootcamp-statefulset-0.bootcamp-service.default +search +short
10.244.2.11
root@dnstest:/# dig bootcamp-statefulset-0.bootcamp-service +search +short
10.244.2.11

kun432/index.md

kun432 commented Oct 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kun432 commented Oct 13, 2021 •

edited

Loading