Container-native LB

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: neg-demo-app # Label for the Deployment
  name: neg-demo-app # Name of Deployment
spec: # Deployment's specification
  replicas: 16
  minReadySeconds: 60 # Number of seconds to wait after a Pod is created and its status is Ready
  selector:
    matchLabels:
      run: neg-demo-app
  template: # Pod template
    metadata:
      labels:
        run: neg-demo-app # Labels Pods from this Deployment
    spec: # Pod specification; each Pod created by this Deployment has this specification
      containers:
      - image: k8s.gcr.io/serve_hostname:v1.4 # Application to run in Deployment's Pods
        name: hostname # Container name
        ports:
        - containerPort: 9376 # Port used by containers running in these Pods
          protocol: TCP
      terminationGracePeriodSeconds: 60

svc1.yaml

apiVersion: v1
kind: Service
metadata:
  name: neg-demo-svc1 # Name of Service
  annotations:
    cloud.google.com/neg: '{"ingress": true}' # Creates an NEG after an Ingress is created
spec: # Service's specification
  selector:
    run: neg-demo-app # Selects Pods labelled run: neg-demo-app
  ports:
  - port: 80 # Service's port
    protocol: TCP
    targetPort: 9376
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: neg-demo-ing
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
    certmanager.k8s.io/acme-http01-edit-in-place: "true"
    kubernetes.io/ingress.global-static-ip-name: hostname
spec:
  rules:
  - host: 35.227.249.202.nip.io
    http:
      paths:
      - backend:
          serviceName: neg-demo-svc1
          servicePort: 80
        path: /
  tls:
  - hosts:
    - 35.227.249.202.nip.io
    secretName: neg-demo-cert1

svc2.yaml

apiVersion: v1
kind: Service
metadata:
  name: neg-demo-svc2
spec:
  type: NodePort
  selector:
    run: neg-demo-app # Selects Pods labelled run: neg-demo-app
  ports:
  - port: 80 # Service's port
    protocol: TCP
    targetPort: 9376
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: neg-demo-ing2
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
    certmanager.k8s.io/acme-http01-edit-in-place: "true"
    kubernetes.io/ingress.global-static-ip-name: hostname2
spec:
  rules:
  - host: 35.244.182.8.nip.io
    http:
      paths:
      - backend:
          serviceName: neg-demo-svc2
          servicePort: 80
        path: /
  tls:
  - hosts:
    - 35.244.182.8.nip.io
    secretName: neg-demo-cert2

test-environments.md

Kubernetes Cluster:

• Version: 1.11.2-gke.18
• Node size: 4
• Pod size: 16
• Zone: us-central1-a
• Other workloads: Helm + cert-manager
• There is no auto scaler.

Loader VM:

• Machine Type: n1-standard-8

• Zone: us-central1-a

• OS: SMP Debian 4.9.130-2 (2018-10-27) x86_64

• There's nothing but wrk tool here. ( https://github.com/wg/wrk/wiki/Installing-Wrk-on-Linux#ubuntudebian-clean-box )

• All tests were run separately.

tests_with_10_nodes_64_pods.md

Test 1

wrk -c100 -t 100

Container-Native activated service (NEG)

$ wrk -c100 -t 100 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.26ms    3.50ms 211.28ms   95.45%
    Req/Sec   350.53     62.39   565.00     69.28%
  4190504 requests in 2.00m, 715.35MB read
Requests/sec:  34893.57
Transfer/sec:      5.96MB

$ wrk -c100 -t 100 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.14ms   27.74ms   1.97s    99.44%
    Req/Sec   369.68     62.93   550.00     68.07%
  4411305 requests in 2.00m, 753.04MB read
Requests/sec:  36732.99
Transfer/sec:      6.27MB

Standard Ingress

$ wrk -c100 -t 100 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.37ms   26.19ms   1.92s    99.04%
    Req/Sec   342.13     63.23   540.00     71.07%
  4083403 requests in 2.00m, 697.07MB read
  Socket errors: connect 0, read 0, write 0, timeout 2
Requests/sec:  33999.71
Transfer/sec:      5.80MB

$ wrk -c100 -t 100 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.55ms    4.98ms 214.81ms   96.04%
    Req/Sec   343.21     62.37   520.00     68.22%
  4102295 requests in 2.00m, 700.29MB read
  Socket errors: connect 0, read 1, write 0, timeout 0
Requests/sec:  34157.19
Transfer/sec:      5.83MB

---

Test 2

wrk -c50 -t 50

Container-Native activated service (NEG)

$ wrk -c50 -t 50 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.74ms    3.98ms 273.29ms   98.10%
    Req/Sec   413.24     67.40   595.00     67.02%
  2469329 requests in 2.00m, 421.53MB read
  Socket errors: connect 0, read 1, write 0, timeout 0
Requests/sec:  20560.68
Transfer/sec:      3.51MB

$ wrk -c50 -t 50 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.65ms   38.59ms   1.94s    99.42%
    Req/Sec   408.30     74.15   600.00     64.24%
  2428956 requests in 2.00m, 414.64MB read
  Socket errors: connect 0, read 0, write 0, timeout 3
Requests/sec:  20225.14
Transfer/sec:      3.45MB

Standard Ingress

$ wrk -c50 -t 50 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.66ms    2.40ms 210.16ms   97.38%
    Req/Sec   395.31     65.52   550.00     62.13%
  2362629 requests in 2.00m, 403.32MB read
Requests/sec:  19672.17
Transfer/sec:      3.36MB

$ wrk -c50 -t 50 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.42ms   37.97ms 892.21ms   99.27%
    Req/Sec   388.05     68.64   565.00     68.43%
  2305891 requests in 2.00m, 393.63MB read
Requests/sec:  19199.93
Transfer/sec:      3.28MB

---

Test 3

wrk -c25 -t 25

Container-Native activated service (NEG)

$ wrk -c25 -t 25 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.34ms    7.92ms 209.93ms   97.49%
    Req/Sec   417.01     89.13   606.00     71.33%
  1245058 requests in 2.00m, 212.54MB read
Requests/sec:  10367.14
Transfer/sec:      1.77MB

$ wrk -c25 -t 25 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.03ms   56.97ms   1.90s    99.37%
    Req/Sec   425.10     78.38   595.00     69.60%
  1262018 requests in 2.00m, 215.44MB read
  Socket errors: connect 0, read 0, write 0, timeout 2
Requests/sec:  10508.11
Transfer/sec:      1.79MB

Standard Ingress

$ wrk -c25 -t 25 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.65ms    2.12ms 211.83ms   97.46%
    Req/Sec   393.14     58.35   545.00     65.23%
  1174696 requests in 2.00m, 200.53MB read
Requests/sec:   9780.97
Transfer/sec:      1.67MB

$ wrk -c25 -t 25 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.56ms    2.44ms 202.24ms   97.78%
    Req/Sec   410.18     63.38   560.00     64.76%
  1225606 requests in 2.00m, 209.22MB read
Requests/sec:  10204.93
Transfer/sec:      1.74MB

tests_with_4_nodes_16_pods.md

Test 1

wrk -c100 -t 100

Container-Native activated service (NEG)

$ wrk -c100 -t 100 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.80ms   11.85ms 215.76ms   88.76%
    Req/Sec   185.10     63.15   470.00     67.34%
  2213497 requests in 2.00m, 377.86MB read
  Socket errors: connect 0, read 1, write 0, timeout 0
Requests/sec:  18430.56
Transfer/sec:      3.15MB

$ wrk -c100 -t 100 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.83ms   12.11ms 226.32ms   89.39%
    Req/Sec   184.17     62.93   500.00     67.87%
  2202635 requests in 2.00m, 376.01MB read
Requests/sec:  18340.12
Transfer/sec:      3.13MB

Standard Ingress

$ wrk -c100 -t 100 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.06ms   14.60ms 257.41ms   92.02%
    Req/Sec   188.40     62.49   474.00     71.59%
  2248085 requests in 2.00m, 383.77MB read
Requests/sec:  18718.29
Transfer/sec:      3.20MB

$ wrk -c100 -t 100 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.79ms   16.02ms 213.45ms   91.69%
    Req/Sec   183.26     64.62   470.00     70.67%
  2184460 requests in 2.00m, 372.90MB read
Requests/sec:  18188.59
Transfer/sec:      3.10MB

---

Test 2

wrk -c50 -t 50

Container-Native activated service (NEG)

$ wrk -c50 -t 50 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.63ms    4.66ms 208.08ms   95.13%
    Req/Sec   333.31     61.13   530.00     67.83%
  1991720 requests in 2.00m, 340.00MB read
Requests/sec:  16584.06
Transfer/sec:      2.83MB

$ wrk -c50 -t 50 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.45ms   13.76ms 412.77ms   97.72%
    Req/Sec   322.78     64.72   550.00     70.09%
  1925014 requests in 2.00m, 328.61MB read
Requests/sec:  16028.63
Transfer/sec:      2.74MB

Standard Ingress

$ wrk -c50 -t 50 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.77ms    4.97ms 211.51ms   95.39%
    Req/Sec   321.80     59.14   520.00     67.26%
  1923275 requests in 2.00m, 328.32MB read
  Socket errors: connect 0, read 1, write 0, timeout 0
Requests/sec:  16014.28
Transfer/sec:      2.73MB

$ wrk -c50 -t 50 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.90ms    5.87ms 406.57ms   96.02%
    Req/Sec   315.83     56.61   500.00     71.20%
  1887250 requests in 2.00m, 322.17MB read
Requests/sec:  15714.00
Transfer/sec:      2.68MB

---

Test 3

wrk -c25 -t 25

Container-Native activated service (NEG)

$ wrk -c25 -t 25 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.84ms    4.04ms 214.21ms   98.02%
    Req/Sec   393.30     78.33   565.00     63.07%
  1174669 requests in 2.00m, 200.53MB read
Requests/sec:   9780.80
Transfer/sec:      1.67MB

$ wrk -c25 -t 25 -d 120 https://35.227.249.202.nip.io/
Running 2m test @ https://35.227.249.202.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.75ms    6.86ms 429.02ms   98.80%
    Req/Sec   424.54     70.71   616.00     66.42%
  1267814 requests in 2.00m, 216.43MB read
Requests/sec:  10556.78
Transfer/sec:      1.80MB

$ wrk -c25 -t 25 -d 200 https://35.227.249.202.nip.io/
Running 3m test @ https://35.227.249.202.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.72ms    3.83ms 249.99ms   98.07%
    Req/Sec   409.12     71.32   590.00     67.39%
  2036329 requests in 3.33m, 347.62MB read
  Socket errors: connect 0, read 1, write 0, timeout 0
Requests/sec:  10178.01
Transfer/sec:      1.74MB

Standard Ingress

$ wrk -c25 -t 25 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.78ms    3.08ms 291.44ms   97.52%
    Req/Sec   387.88     65.90   545.00     62.60%
  1158797 requests in 2.00m, 197.82MB read
Requests/sec:   9648.70
Transfer/sec:      1.65MB

$ wrk -c25 -t 25 -d 120 https://35.244.182.8.nip.io/
Running 2m test @ https://35.244.182.8.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.65ms    2.50ms 209.45ms   96.93%
    Req/Sec   404.80     64.42   560.00     64.64%
  1209309 requests in 2.00m, 206.44MB read
Requests/sec:  10069.81
Transfer/sec:      1.72MB

$ wrk -c25 -t 25 -d 200 https://35.244.182.8.nip.io/
Running 3m test @ https://35.244.182.8.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.52ms    2.61ms 204.06ms   97.79%
    Req/Sec   421.27     67.22   585.00     63.37%
  2097070 requests in 3.33m, 357.99MB read
Requests/sec:  10480.21
Transfer/sec:      1.79MB

ismail@instance-3:~$ wrk -c25 -t 25 -d 600 https://35.244.182.8.nip.io/
Running 10m test @ https://35.244.182.8.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.86ms   30.98ms   1.98s    99.65%
    Req/Sec   388.74     56.44   545.00     68.31%
  5788862 requests in 10.00m, 0.97GB read
  Socket errors: connect 0, read 0, write 0, timeout 4
Requests/sec:   9646.59
Transfer/sec:      1.65MB
ismail@instance-3:~$ wrk -c25 -t 25 -d 600 https://35.227.249.202.nip.io/
Running 10m test @ https://35.227.249.202.nip.io/
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.07ms   35.61ms   1.64s    98.98%
    Req/Sec   412.43     80.30   595.00     71.65%
  6128558 requests in 10.00m, 1.02GB read
  Socket errors: connect 0, read 0, write 0, timeout 4
Requests/sec:  10212.66
Transfer/sec:      1.74MB
ismail@instance-3:~$ wrk -c100 -t 100 -d 600 https://35.227.249.202.nip.io/
Running 10m test @ https://35.227.249.202.nip.io/
  100 threads and 100 connections
   Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.23ms   23.59ms   1.93s    99.23%
    Req/Sec   356.64     65.53   555.00     66.80%
  21230334 requests in 10.00m, 3.54GB read
  Socket errors: connect 0, read 1, write 0, timeout 15
Requests/sec:  35378.03
Transfer/sec:      6.04MB
ismail@instance-3:~$  wrk -c100 -t 100 -d 600 https://35.244.182.8.nip.io/
Running 10m test @ https://35.244.182.8.nip.io/
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.44ms   23.96ms   1.93s    98.93%
    Req/Sec   340.24     63.19   530.00     68.96%
  20266568 requests in 10.00m, 3.38GB read
  Socket errors: connect 0, read 2, write 0, timeout 21
Requests/sec:  33771.97
Transfer/sec:      5.77MB