Installing Istio 1.4 on PKS with restrictive Pod Security Policy

ISTIO 1.4 on PKS with PSPs.md

What does this GIST do or not do

1. Shows you how to use Istio 1.4.0 - 1.4.2 on Kubernetes 1.14+ with a modicum of runtime security for your workloads.
2. Specifically it installs Istio with CNI support, and allows the use of restrictive PodSecurityPolicies for your workloads. Istio 1.4.3 fixes this issue. However, this may still be useful due to a need to add a PSP privileged role to the Istio helm charts!
3. It is designed for VMware PKS, but doesn't require it ... (just change the CNI bin dir and excluded namespaces in values-cni.yml, also swap the ClusterRole pks-privileged and pks-restricted mentioned throughout these files with your own PSP roles).
4. It doesn't fix the need for Istio itself to run as root, but that should be fixed in a future Istio release as it's already fixed in trunk.
5. Update I've also included a workaround for the CNI race condition bug in Istio CNI if you're using a DaemonSet-based CNI. See istio/istio#14327

Prerequisites

1. You are logged into your cluster as a cluster admin, K8s 1.14 at least
2. You have the HELM CLI installed and Kustomize 3.x installed
   • https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv3.4.0
   • https://github.com/helm/helm/releases/tag/v2.16.0
   • This should work with HELM v3 but this isn't tested
3. Create a working directory
4. If you're running DaemonSet CNI e.g. Calico policy, stay tuned for more information.

Instructions

1. Add the Istio Helm Repo: helm repo add istio.io https://storage.googleapis.com/istio-release/releases/1.4.0/charts/

2. Copy the values-*.yml in this GIST to your working directory.

3. Make a sub-working directory mkdir -p istio-kustomize/base

4. Copy kustomization.yaml, psp.yml, and sidcar-injector-configmap.yml from this GIST into ./istio-kustomize.

5. If you're using a DaemonSet based CNI such as Calico,

• copy race-cni-patch.yml from this GIST into ./istio-kustomize
• If your CNI is not Calico, edit cni-race-patch.yml and set a pod selector in CNI_SELECTOR env variable that matches your CNI pods
• edit kustomization.yaml adding - path: race-cni-patch.yml to the end of the file.

This injects an init container into the istio-CNI to wait for Calico to come up before Istio.

6. Fetch/generate your Istio YAMLs

   helm fetch istio.io/istio
   helm fetch istio.io/istio-init
   helm fetch istio.io/istio-cni
   
   helm template ./istio-1.4.0.tgz -f values-istio.yml --namespace istio-system > istio-kustomize/base/istio-1.4.0.yml
   helm template ./istio-init-1.4.0.tgz --namespace istio-system > istio-kustomize/base/istio-init-1.4.0.yml
   helm template ./istio-cni-1.4.0.tgz -f values-cni.yml --namespace kube-system > istio-kustomize/base/istio-cni-1.4.0.yml
   

7. Create namespace

   kubectl create ns istio-system
   

8. Run Kustomize to generate your Istio install kustomize build ./istio-kustomize > istio-full.yml

9. Install istio kubectl apply -f istio-full.yml

Results

1. Sidecar will run non-privileged due to Istio CNI
2. Pod will match PSP because the securityContext of the istio-proxy sidecar should match the PSP's clauses

What did all of this Kustomize stuff do?

1. Insert pks-privileged PSP into all generated roles for istio-system pods.
   PSPs are supported for istio-cni's helm chart but not yet for istio's main helm chart.

2. Swap out the SidecarInjector ConfigMap helm template w/ a modified one to insert more of possible default securityContext values in the sidecar istio-proxy. See the sidecar injector diff for specifics, see for istio/istio#17427 explanation

3. Ensure the AppArmor annotations are injected appropriately (the helm chart templates this properly in Istio 1.4.0 but since I'm swapping out the results, my copy needs to also do it). See istio/istio#17334 for details

4. If using Calico or other DaemonSet based CNI, injects an init container to wait for that parent CNI to come up before Istio. See istio/istio#14327 for details

Other FAQs

1. Why does Bookinfo still require root?

Because it was built that way. Sorry. This is supposedly fixed upstream on Oct 18, but not yet pulled into the current release. See https://github.com/istio/istio/commit/53e2f40dafa7ffef3d54b9c46b831055bc9e8469 for commit, also istio/istio#14667

2. Why does Istio itself require root?

Because it was built that way. Sorry. This was fixed on Oct 16 and was supposed to make Istio 1.4 but I couldn't get it to work w/ a restrictive PSP. Note that Istio itself doesn't even support PSPs with the helm chart yet (or the operator?). istio/istio#16014

3. Why is any of this necessary?

A historical analogy: In the 1980's, PC software evolved quickly, with few restrictions for safety and security, thus software (and even the whole OS) crashed a lot, was hard to use, hard to integrate, and conflicted with each other ("DOS ain't done until Lotus won't run"). This insanity continued for about 20 years, and circa the rise of protected memory operating systems such as Linux, Windows 2000, FreeBSD, and MacOS X, things got better. Mainframers chuckled at us and kept maintaining their secure/scalable/performant COBOL, RPG, PL/1, ADA, and NATURAL programs to run most of the world until the mid-late 1990s when the Y2K date bug forced a rewrite.

That's the Kubernetes ecosystem in 2019. Good luck, enjoy the ride, hope you have protective gear.

kustomization.yaml

resources:
- base/istio-cni-1.4.0.yml
- base/istio-init-1.4.0.yml
- base/istio-1.4.0.yml
patches:
- path: psp.yml
  target:
    group: rbac.authorization.k8s.io
    version: v1
    kind: ClusterRole
- path: psp.yml
  target:
    group: rbac.authorization.k8s.io
    version: v1
    kind:  Role
- path: sidecar-injector-configmap.yml # comment out for istio 1.4.3+

psp.yml

- op: add
  path: "/rules/-"
  value:
    apiGroups:
    - extensions
    resourceNames:
    - pks-privileged
    resources:
    - podsecuritypolicies
    verbs:
    - use

race-cni-patch.yaml

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: istio-cni-node
  namespace: kube-system
  labels:
    k8s-app: istio-cni-node
    chart: istio-cni-1.4.0
    release: release-name
    version: 1.4.0
    heritage: Tiller
spec:
  template:
    spec:
      initContainers:
      - name: wait-for-parent
        image: parasubvert/istio-cni:1.4.0-sc2
        command: [ "sh", "-c"]
        args:
        - until kubectl get pods -n kube-system -l ${CNI_SELECTOR} -o json | jq -e -r ".items[] | select(.status.phase == \"Running\" and ([ .status.conditions[] | select(.type == \"Ready\" and .status == \"True\") ] | length ) == 1 ) | select (.spec.nodeName == \"${NODE_NAME}\") | .metadata.namespace + \"/\" + .metadata.name "; do
            echo "waiting for $NODE_NAME to come up";
            sleep 2;
          done;
        env:
        - name: CNI_SELECTOR
          value: "k8s-app=calico-node"
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName

sidecar-injector-configmap.yml

---
# Source: istio/templates/sidecar-injector-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: istio-sidecar-injector
  namespace: istio-system
  labels:
    app: istio
    chart: istio
    heritage: Tiller
    release: release-name
    istio: sidecar-injector
data:
  values: |-
    {"certmanager":{"enabled":false},"galley":{"enableAnalysis":false,"enableServiceDiscovery":false,"enabled":true,"global":{"arch":{"amd64":2,"ppc64le":2,"s390x":2},"certificates":[],"configValidation":true,"controlPlaneSecurityEnabled":false,"defaultNodeSelector":{},"defaultPodDisruptionBudget":{"enabled":true},"defaultResources":{"requests":{"cpu":"10m"}},"defaultTolerations":[],"disablePolicyChecks":true,"enableHelmTest":false,"enableTracing":true,"hub":"docker.io/istio","imagePullPolicy":"IfNotPresent","imagePullSecrets":[],"k8sIngress":{"enableHttps":false,"enabled":false,"gatewayName":"ingressgateway"},"localityLbSetting":{"enabled":true},"logging":{"level":"default:info"},"meshExpansion":{"enabled":false,"useILB":false},"meshID":"","meshNetworks":{},"monitoringPort":15014,"mtls":{"auto":false,"enabled":false},"multiCluster":{"clusterName":"","enabled":false},"network":"","oneNamespace":false,"operatorManageWebhooks":false,"outboundTrafficPolicy":{"mode":"ALLOW_ANY"},"policyCheckFailOpen":false,"priorityClassName":"","proxy":{"accessLogEncoding":"TEXT","accessLogFile":"","accessLogFormat":"","autoInject":"enabled","clusterDomain":"cluster.local","componentLogLevel":"","concurrency":2,"dnsRefreshRate":"300s","enableCoreDump":false,"enableCoreDumpImage":"ubuntu:xenial","envoyAccessLogService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyMetricsService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyStatsd":{"enabled":false},"excludeIPRanges":"","excludeInboundPorts":"","excludeOutboundPorts":"","image":"proxyv2","includeIPRanges":"*","includeInboundPorts":"*","init":{"resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"10m","memory":"10Mi"}}},"kubevirtInterfaces":"","logLevel":"","privileged":false,"protocolDetectionTimeout":"100ms","readinessFailureThreshold":30,"readinessInitialDelaySeconds":1,"readinessPeriodSeconds":2,"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"statusPort":15020,"tracer":"zipkin"},"proxy_init":{"image":"proxyv2"},"sds":{"enabled":false,"token":{"aud":"istio-ca"},"udsPath":""},"tag":"1.4.0","tracer":{"datadog":{"address":"$(HOST_IP):8126"},"lightstep":{"accessToken":"","address":"","cacertPath":"","secure":true},"stackdriver":{"debug":false,"maxNumberOfAnnotations":200,"maxNumberOfAttributes":200,"maxNumberOfMessageEvents":200},"zipkin":{"address":""}},"trustDomain":"","trustDomainAliases":[],"useMCP":true},"image":"galley","nodeSelector":{},"podAnnotations":{},"podAntiAffinityLabelSelector":[],"podAntiAffinityTermLabelSelector":[],"replicaCount":1,"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%","tolerations":[]},"gateways":{"enabled":true,"global":{"arch":{"amd64":2,"ppc64le":2,"s390x":2},"certificates":[],"configValidation":true,"controlPlaneSecurityEnabled":false,"defaultNodeSelector":{},"defaultPodDisruptionBudget":{"enabled":true},"defaultResources":{"requests":{"cpu":"10m"}},"defaultTolerations":[],"disablePolicyChecks":true,"enableHelmTest":false,"enableTracing":true,"hub":"docker.io/istio","imagePullPolicy":"IfNotPresent","imagePullSecrets":[],"k8sIngress":{"enableHttps":false,"enabled":false,"gatewayName":"ingressgateway"},"localityLbSetting":{"enabled":true},"logging":{"level":"default:info"},"meshExpansion":{"enabled":false,"useILB":false},"meshID":"","meshNetworks":{},"monitoringPort":15014,"mtls":{"auto":false,"enabled":false},"multiCluster":{"clusterName":"","enabled":false},"network":"","oneNamespace":false,"operatorManageWebhooks":false,"outboundTrafficPolicy":{"mode":"ALLOW_ANY"},"policyCheckFailOpen":false,"priorityClassName":"","proxy":{"accessLogEncoding":"TEXT","accessLogFile":"","accessLogFormat":"","autoInject":"enabled","clusterDomain":"cluster.local","componentLogLevel":"","concurrency":2,"dnsRefreshRate":"300s","enableCoreDump":false,"enableCoreDumpImage":"ubuntu:xenial","envoyAccessLogService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyMetricsService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyStatsd":{"enabled":false},"excludeIPRanges":"","excludeInboundPorts":"","excludeOutboundPorts":"","image":"proxyv2","includeIPRanges":"*","includeInboundPorts":"*","init":{"resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"10m","memory":"10Mi"}}},"kubevirtInterfaces":"","logLevel":"","privileged":false,"protocolDetectionTimeout":"100ms","readinessFailureThreshold":30,"readinessInitialDelaySeconds":1,"readinessPeriodSeconds":2,"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"statusPort":15020,"tracer":"zipkin"},"proxy_init":{"image":"proxyv2"},"sds":{"enabled":false,"token":{"aud":"istio-ca"},"udsPath":""},"tag":"1.4.0","tracer":{"datadog":{"address":"$(HOST_IP):8126"},"lightstep":{"accessToken":"","address":"","cacertPath":"","secure":true},"stackdriver":{"debug":false,"maxNumberOfAnnotations":200,"maxNumberOfAttributes":200,"maxNumberOfMessageEvents":200},"zipkin":{"address":""}},"trustDomain":"","trustDomainAliases":[],"useMCP":true},"istio-egressgateway":{"autoscaleEnabled":true,"autoscaleMax":5,"autoscaleMin":1,"cpu":{"targetAverageUtilization":80},"enabled":false,"env":{"ISTIO_META_ROUTER_MODE":"sni-dnat"},"labels":{"app":"istio-egressgateway","istio":"egressgateway"},"nodeSelector":{},"podAnnotations":{},"podAntiAffinityLabelSelector":[],"podAntiAffinityTermLabelSelector":[],"ports":[{"name":"http2","port":80},{"name":"https","port":443},{"name":"tls","port":15443,"targetPort":15443}],"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%","secretVolumes":[{"mountPath":"/etc/istio/egressgateway-certs","name":"egressgateway-certs","secretName":"istio-egressgateway-certs"},{"mountPath":"/etc/istio/egressgateway-ca-certs","name":"egressgateway-ca-certs","secretName":"istio-egressgateway-ca-certs"}],"serviceAnnotations":{},"tolerations":[],"type":"ClusterIP"},"istio-ilbgateway":{"autoscaleEnabled":true,"autoscaleMax":5,"autoscaleMin":1,"cpu":{"targetAverageUtilization":80},"enabled":false,"labels":{"app":"istio-ilbgateway","istio":"ilbgateway"},"loadBalancerIP":"","nodeSelector":{},"podAnnotations":{},"ports":[{"name":"grpc-pilot-mtls","port":15011},{"name":"grpc-pilot","port":15010},{"name":"tcp-citadel-grpc-tls","port":8060,"targetPort":8060},{"name":"tcp-dns","port":5353}],"resources":{"requests":{"cpu":"800m","memory":"512Mi"}},"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%","secretVolumes":[{"mountPath":"/etc/istio/ilbgateway-certs","name":"ilbgateway-certs","secretName":"istio-ilbgateway-certs"},{"mountPath":"/etc/istio/ilbgateway-ca-certs","name":"ilbgateway-ca-certs","secretName":"istio-ilbgateway-ca-certs"}],"serviceAnnotations":{"cloud.google.com/load-balancer-type":"internal"},"tolerations":[],"type":"LoadBalancer"},"istio-ingressgateway":{"applicationPorts":"","autoscaleEnabled":true,"autoscaleMax":5,"autoscaleMin":1,"cpu":{"targetAverageUtilization":80},"enabled":true,"env":{"ISTIO_META_ROUTER_MODE":"sni-dnat"},"externalIPs":[],"labels":{"app":"istio-ingressgateway","istio":"ingressgateway"},"loadBalancerIP":"","loadBalancerSourceRanges":[],"meshExpansionPorts":[{"name":"tcp-pilot-grpc-tls","port":15011,"targetPort":15011},{"name":"tcp-mixer-grpc-tls","port":15004,"targetPort":15004},{"name":"tcp-citadel-grpc-tls","port":8060,"targetPort":8060},{"name":"tcp-dns-tls","port":853,"targetPort":853}],"nodeSelector":{},"podAnnotations":{},"podAntiAffinityLabelSelector":[],"podAntiAffinityTermLabelSelector":[],"ports":[{"name":"status-port","port":15020,"targetPort":15020},{"name":"http2","nodePort":31380,"port":80,"targetPort":80},{"name":"https","nodePort":31390,"port":443},{"name":"tcp","nodePort":31400,"port":31400},{"name":"https-kiali","port":15029,"targetPort":15029},{"name":"https-prometheus","port":15030,"targetPort":15030},{"name":"https-grafana","port":15031,"targetPort":15031},{"name":"https-tracing","port":15032,"targetPort":15032},{"name":"tls","port":15443,"targetPort":15443}],"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%","sds":{"enabled":false,"image":"node-agent-k8s","resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}}},"secretVolumes":[{"mountPath":"/etc/istio/ingressgateway-certs","name":"ingressgateway-certs","secretName":"istio-ingressgateway-certs"},{"mountPath":"/etc/istio/ingressgateway-ca-certs","name":"ingressgateway-ca-certs","secretName":"istio-ingressgateway-ca-certs"}],"serviceAnnotations":{},"tolerations":[],"type":"LoadBalancer"}},"global":{"arch":{"amd64":2,"ppc64le":2,"s390x":2},"certificates":[],"configValidation":true,"controlPlaneSecurityEnabled":false,"defaultNodeSelector":{},"defaultPodDisruptionBudget":{"enabled":true},"defaultResources":{"requests":{"cpu":"10m"}},"defaultTolerations":[],"disablePolicyChecks":true,"enableHelmTest":false,"enableTracing":true,"hub":"docker.io/istio","imagePullPolicy":"IfNotPresent","imagePullSecrets":[],"k8sIngress":{"enableHttps":false,"enabled":false,"gatewayName":"ingressgateway"},"localityLbSetting":{"enabled":true},"logging":{"level":"default:info"},"meshExpansion":{"enabled":false,"useILB":false},"meshID":"","meshNetworks":{},"monitoringPort":15014,"mtls":{"auto":false,"enabled":false},"multiCluster":{"clusterName":"","enabled":false},"network":"","oneNamespace":false,"operatorManageWebhooks":false,"outboundTrafficPolicy":{"mode":"ALLOW_ANY"},"policyCheckFailOpen":false,"priorityClassName":"","proxy":{"accessLogEncoding":"TEXT","accessLogFile":"","accessLogFormat":"","autoInject":"enabled","clusterDomain":"cluster.local","componentLogLevel":"","concurrency":2,"dnsRefreshRate":"300s","enableCoreDump":false,"enableCoreDumpImage":"ubuntu:xenial","envoyAccessLogService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyMetricsService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyStatsd":{"enabled":false},"excludeIPRanges":"","excludeInboundPorts":"","excludeOutboundPorts":"","image":"proxyv2","includeIPRanges":"*","includeInboundPorts":"*","init":{"resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"10m","memory":"10Mi"}}},"kubevirtInterfaces":"","logLevel":"","privileged":false,"protocolDetectionTimeout":"100ms","readinessFailureThreshold":30,"readinessInitialDelaySeconds":1,"readinessPeriodSeconds":2,"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"statusPort":15020,"tracer":"zipkin"},"proxy_init":{"image":"proxyv2"},"sds":{"enabled":false,"token":{"aud":"istio-ca"},"udsPath":""},"tag":"1.4.0","tracer":{"datadog":{"address":"$(HOST_IP):8126"},"lightstep":{"accessToken":"","address":"","cacertPath":"","secure":true},"stackdriver":{"debug":false,"maxNumberOfAnnotations":200,"maxNumberOfAttributes":200,"maxNumberOfMessageEvents":200},"zipkin":{"address":""}},"trustDomain":"","trustDomainAliases":[],"useMCP":true},"grafana":{"enabled":false},"istio_cni":{"enabled":true},"istiocoredns":{"enabled":false},"kiali":{"enabled":false},"mixer":{"adapters":{"kubernetesenv":{"enabled":true},"prometheus":{"enabled":true,"metricsExpiryDuration":"10m"},"stdio":{"enabled":false,"outputAsJson":true},"useAdapterCRDs":false},"env":{"GOMAXPROCS":"6"},"global":{"arch":{"amd64":2,"ppc64le":2,"s390x":2},"certificates":[],"configValidation":true,"controlPlaneSecurityEnabled":false,"defaultNodeSelector":{},"defaultPodDisruptionBudget":{"enabled":true},"defaultResources":{"requests":{"cpu":"10m"}},"defaultTolerations":[],"disablePolicyChecks":true,"enableHelmTest":false,"enableTracing":true,"hub":"docker.io/istio","imagePullPolicy":"IfNotPresent","imagePullSecrets":[],"k8sIngress":{"enableHttps":false,"enabled":false,"gatewayName":"ingressgateway"},"localityLbSetting":{"enabled":true},"logging":{"level":"default:info"},"meshExpansion":{"enabled":false,"useILB":false},"meshID":"","meshNetworks":{},"monitoringPort":15014,"mtls":{"auto":false,"enabled":false},"multiCluster":{"clusterName":"","enabled":false},"network":"","oneNamespace":false,"operatorManageWebhooks":false,"outboundTrafficPolicy":{"mode":"ALLOW_ANY"},"policyCheckFailOpen":false,"priorityClassName":"","proxy":{"accessLogEncoding":"TEXT","accessLogFile":"","accessLogFormat":"","autoInject":"enabled","clusterDomain":"cluster.local","componentLogLevel":"","concurrency":2,"dnsRefreshRate":"300s","enableCoreDump":false,"enableCoreDumpImage":"ubuntu:xenial","envoyAccessLogService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyMetricsService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyStatsd":{"enabled":false},"excludeIPRanges":"","excludeInboundPorts":"","excludeOutboundPorts":"","image":"proxyv2","includeIPRanges":"*","includeInboundPorts":"*","init":{"resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"10m","memory":"10Mi"}}},"kubevirtInterfaces":"","logLevel":"","privileged":false,"protocolDetectionTimeout":"100ms","readinessFailureThreshold":30,"readinessInitialDelaySeconds":1,"readinessPeriodSeconds":2,"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"statusPort":15020,"tracer":"zipkin"},"proxy_init":{"image":"proxyv2"},"sds":{"enabled":false,"token":{"aud":"istio-ca"},"udsPath":""},"tag":"1.4.0","tracer":{"datadog":{"address":"$(HOST_IP):8126"},"lightstep":{"accessToken":"","address":"","cacertPath":"","secure":true},"stackdriver":{"debug":false,"maxNumberOfAnnotations":200,"maxNumberOfAttributes":200,"maxNumberOfMessageEvents":200},"zipkin":{"address":""}},"trustDomain":"","trustDomainAliases":[],"useMCP":true},"image":"mixer","nodeSelector":{},"podAnnotations":{},"podAntiAffinityLabelSelector":[],"podAntiAffinityTermLabelSelector":[],"policy":{"autoscaleEnabled":true,"autoscaleMax":5,"autoscaleMin":1,"cpu":{"targetAverageUtilization":80},"enabled":true,"replicaCount":1,"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%"},"telemetry":{"autoscaleEnabled":true,"autoscaleMax":5,"autoscaleMin":1,"cpu":{"targetAverageUtilization":80},"enabled":true,"loadshedding":{"latencyThreshold":"100ms","mode":"enforce"},"replicaCount":1,"reportBatchMaxEntries":100,"reportBatchMaxTime":"1s","resources":{"limits":{"cpu":"4800m","memory":"4G"},"requests":{"cpu":"1000m","memory":"1G"}},"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%","sessionAffinityEnabled":false},"tolerations":[]},"nodeagent":{"enabled":false},"pilot":{"autoscaleEnabled":true,"autoscaleMax":5,"autoscaleMin":1,"configSource":{},"cpu":{"targetAverageUtilization":80},"enableProtocolSniffingForInbound":false,"enableProtocolSniffingForOutbound":true,"enabled":true,"env":{"PILOT_PUSH_THROTTLE":100},"global":{"arch":{"amd64":2,"ppc64le":2,"s390x":2},"certificates":[],"configValidation":true,"controlPlaneSecurityEnabled":false,"defaultNodeSelector":{},"defaultPodDisruptionBudget":{"enabled":true},"defaultResources":{"requests":{"cpu":"10m"}},"defaultTolerations":[],"disablePolicyChecks":true,"enableHelmTest":false,"enableTracing":true,"hub":"docker.io/istio","imagePullPolicy":"IfNotPresent","imagePullSecrets":[],"k8sIngress":{"enableHttps":false,"enabled":false,"gatewayName":"ingressgateway"},"localityLbSetting":{"enabled":true},"logging":{"level":"default:info"},"meshExpansion":{"enabled":false,"useILB":false},"meshID":"","meshNetworks":{},"monitoringPort":15014,"mtls":{"auto":false,"enabled":false},"multiCluster":{"clusterName":"","enabled":false},"network":"","oneNamespace":false,"operatorManageWebhooks":false,"outboundTrafficPolicy":{"mode":"ALLOW_ANY"},"policyCheckFailOpen":false,"priorityClassName":"","proxy":{"accessLogEncoding":"TEXT","accessLogFile":"","accessLogFormat":"","autoInject":"enabled","clusterDomain":"cluster.local","componentLogLevel":"","concurrency":2,"dnsRefreshRate":"300s","enableCoreDump":false,"enableCoreDumpImage":"ubuntu:xenial","envoyAccessLogService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyMetricsService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyStatsd":{"enabled":false},"excludeIPRanges":"","excludeInboundPorts":"","excludeOutboundPorts":"","image":"proxyv2","includeIPRanges":"*","includeInboundPorts":"*","init":{"resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"10m","memory":"10Mi"}}},"kubevirtInterfaces":"","logLevel":"","privileged":false,"protocolDetectionTimeout":"100ms","readinessFailureThreshold":30,"readinessInitialDelaySeconds":1,"readinessPeriodSeconds":2,"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"statusPort":15020,"tracer":"zipkin"},"proxy_init":{"image":"proxyv2"},"sds":{"enabled":false,"token":{"aud":"istio-ca"},"udsPath":""},"tag":"1.4.0","tracer":{"datadog":{"address":"$(HOST_IP):8126"},"lightstep":{"accessToken":"","address":"","cacertPath":"","secure":true},"stackdriver":{"debug":false,"maxNumberOfAnnotations":200,"maxNumberOfAttributes":200,"maxNumberOfMessageEvents":200},"zipkin":{"address":""}},"trustDomain":"","trustDomainAliases":[],"useMCP":true},"image":"pilot","keepaliveMaxServerConnectionAge":"30m","nodeSelector":{},"podAnnotations":{},"podAntiAffinityLabelSelector":[],"podAntiAffinityTermLabelSelector":[],"resources":{"requests":{"cpu":"500m","memory":"2048Mi"}},"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%","sidecar":true,"tolerations":[],"traceSampling":1},"prometheus":{"contextPath":"/prometheus","enabled":true,"global":{"arch":{"amd64":2,"ppc64le":2,"s390x":2},"certificates":[],"configValidation":true,"controlPlaneSecurityEnabled":false,"defaultNodeSelector":{},"defaultPodDisruptionBudget":{"enabled":true},"defaultResources":{"requests":{"cpu":"10m"}},"defaultTolerations":[],"disablePolicyChecks":true,"enableHelmTest":false,"enableTracing":true,"hub":"docker.io/istio","imagePullPolicy":"IfNotPresent","imagePullSecrets":[],"k8sIngress":{"enableHttps":false,"enabled":false,"gatewayName":"ingressgateway"},"localityLbSetting":{"enabled":true},"logging":{"level":"default:info"},"meshExpansion":{"enabled":false,"useILB":false},"meshID":"","meshNetworks":{},"monitoringPort":15014,"mtls":{"auto":false,"enabled":false},"multiCluster":{"clusterName":"","enabled":false},"network":"","oneNamespace":false,"operatorManageWebhooks":false,"outboundTrafficPolicy":{"mode":"ALLOW_ANY"},"policyCheckFailOpen":false,"priorityClassName":"","proxy":{"accessLogEncoding":"TEXT","accessLogFile":"","accessLogFormat":"","autoInject":"enabled","clusterDomain":"cluster.local","componentLogLevel":"","concurrency":2,"dnsRefreshRate":"300s","enableCoreDump":false,"enableCoreDumpImage":"ubuntu:xenial","envoyAccessLogService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyMetricsService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyStatsd":{"enabled":false},"excludeIPRanges":"","excludeInboundPorts":"","excludeOutboundPorts":"","image":"proxyv2","includeIPRanges":"*","includeInboundPorts":"*","init":{"resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"10m","memory":"10Mi"}}},"kubevirtInterfaces":"","logLevel":"","privileged":false,"protocolDetectionTimeout":"100ms","readinessFailureThreshold":30,"readinessInitialDelaySeconds":1,"readinessPeriodSeconds":2,"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"statusPort":15020,"tracer":"zipkin"},"proxy_init":{"image":"proxyv2"},"sds":{"enabled":false,"token":{"aud":"istio-ca"},"udsPath":""},"tag":"1.4.0","tracer":{"datadog":{"address":"$(HOST_IP):8126"},"lightstep":{"accessToken":"","address":"","cacertPath":"","secure":true},"stackdriver":{"debug":false,"maxNumberOfAnnotations":200,"maxNumberOfAttributes":200,"maxNumberOfMessageEvents":200},"zipkin":{"address":""}},"trustDomain":"","trustDomainAliases":[],"useMCP":true},"hub":"docker.io/prom","image":"prometheus","ingress":{"enabled":false,"hosts":["prometheus.local"]},"nodeSelector":{},"podAntiAffinityLabelSelector":[],"podAntiAffinityTermLabelSelector":[],"replicaCount":1,"retention":"6h","scrapeInterval":"15s","security":{"enabled":true},"service":{"annotations":{},"nodePort":{"enabled":false,"port":32090}},"tag":"v2.12.0","tolerations":[]},"security":{"citadelHealthCheck":false,"createMeshPolicy":true,"enableNamespacesByDefault":true,"enabled":true,"env":{},"global":{"arch":{"amd64":2,"ppc64le":2,"s390x":2},"certificates":[],"configValidation":true,"controlPlaneSecurityEnabled":false,"defaultNodeSelector":{},"defaultPodDisruptionBudget":{"enabled":true},"defaultResources":{"requests":{"cpu":"10m"}},"defaultTolerations":[],"disablePolicyChecks":true,"enableHelmTest":false,"enableTracing":true,"hub":"docker.io/istio","imagePullPolicy":"IfNotPresent","imagePullSecrets":[],"k8sIngress":{"enableHttps":false,"enabled":false,"gatewayName":"ingressgateway"},"localityLbSetting":{"enabled":true},"logging":{"level":"default:info"},"meshExpansion":{"enabled":false,"useILB":false},"meshID":"","meshNetworks":{},"monitoringPort":15014,"mtls":{"auto":false,"enabled":false},"multiCluster":{"clusterName":"","enabled":false},"network":"","oneNamespace":false,"operatorManageWebhooks":false,"outboundTrafficPolicy":{"mode":"ALLOW_ANY"},"policyCheckFailOpen":false,"priorityClassName":"","proxy":{"accessLogEncoding":"TEXT","accessLogFile":"","accessLogFormat":"","autoInject":"enabled","clusterDomain":"cluster.local","componentLogLevel":"","concurrency":2,"dnsRefreshRate":"300s","enableCoreDump":false,"enableCoreDumpImage":"ubuntu:xenial","envoyAccessLogService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyMetricsService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyStatsd":{"enabled":false},"excludeIPRanges":"","excludeInboundPorts":"","excludeOutboundPorts":"","image":"proxyv2","includeIPRanges":"*","includeInboundPorts":"*","init":{"resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"10m","memory":"10Mi"}}},"kubevirtInterfaces":"","logLevel":"","privileged":false,"protocolDetectionTimeout":"100ms","readinessFailureThreshold":30,"readinessInitialDelaySeconds":1,"readinessPeriodSeconds":2,"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"statusPort":15020,"tracer":"zipkin"},"proxy_init":{"image":"proxyv2"},"sds":{"enabled":false,"token":{"aud":"istio-ca"},"udsPath":""},"tag":"1.4.0","tracer":{"datadog":{"address":"$(HOST_IP):8126"},"lightstep":{"accessToken":"","address":"","cacertPath":"","secure":true},"stackdriver":{"debug":false,"maxNumberOfAnnotations":200,"maxNumberOfAttributes":200,"maxNumberOfMessageEvents":200},"zipkin":{"address":""}},"trustDomain":"","trustDomainAliases":[],"useMCP":true},"image":"citadel","nodeSelector":{},"podAnnotations":{},"podAntiAffinityLabelSelector":[],"podAntiAffinityTermLabelSelector":[],"replicaCount":1,"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%","selfSigned":true,"tolerations":[],"workloadCertTtl":"2160h"},"sidecarInjectorWebhook":{"alwaysInjectSelector":[],"enableNamespacesByDefault":false,"enabled":true,"global":{"arch":{"amd64":2,"ppc64le":2,"s390x":2},"certificates":[],"configValidation":true,"controlPlaneSecurityEnabled":false,"defaultNodeSelector":{},"defaultPodDisruptionBudget":{"enabled":true},"defaultResources":{"requests":{"cpu":"10m"}},"defaultTolerations":[],"disablePolicyChecks":true,"enableHelmTest":false,"enableTracing":true,"hub":"docker.io/istio","imagePullPolicy":"IfNotPresent","imagePullSecrets":[],"k8sIngress":{"enableHttps":false,"enabled":false,"gatewayName":"ingressgateway"},"localityLbSetting":{"enabled":true},"logging":{"level":"default:info"},"meshExpansion":{"enabled":false,"useILB":false},"meshID":"","meshNetworks":{},"monitoringPort":15014,"mtls":{"auto":false,"enabled":false},"multiCluster":{"clusterName":"","enabled":false},"network":"","oneNamespace":false,"operatorManageWebhooks":false,"outboundTrafficPolicy":{"mode":"ALLOW_ANY"},"policyCheckFailOpen":false,"priorityClassName":"","proxy":{"accessLogEncoding":"TEXT","accessLogFile":"","accessLogFormat":"","autoInject":"enabled","clusterDomain":"cluster.local","componentLogLevel":"","concurrency":2,"dnsRefreshRate":"300s","enableCoreDump":false,"enableCoreDumpImage":"ubuntu:xenial","envoyAccessLogService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyMetricsService":{"enabled":false,"tcpKeepalive":{"interval":"10s","probes":3,"time":"10s"},"tlsSettings":{"mode":"DISABLE","subjectAltNames":[]}},"envoyStatsd":{"enabled":false},"excludeIPRanges":"","excludeInboundPorts":"","excludeOutboundPorts":"","image":"proxyv2","includeIPRanges":"*","includeInboundPorts":"*","init":{"resources":{"limits":{"cpu":"100m","memory":"50Mi"},"requests":{"cpu":"10m","memory":"10Mi"}}},"kubevirtInterfaces":"","logLevel":"","privileged":false,"protocolDetectionTimeout":"100ms","readinessFailureThreshold":30,"readinessInitialDelaySeconds":1,"readinessPeriodSeconds":2,"resources":{"limits":{"cpu":"2000m","memory":"1024Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"statusPort":15020,"tracer":"zipkin"},"proxy_init":{"image":"proxyv2"},"sds":{"enabled":false,"token":{"aud":"istio-ca"},"udsPath":""},"tag":"1.4.0","tracer":{"datadog":{"address":"$(HOST_IP):8126"},"lightstep":{"accessToken":"","address":"","cacertPath":"","secure":true},"stackdriver":{"debug":false,"maxNumberOfAnnotations":200,"maxNumberOfAttributes":200,"maxNumberOfMessageEvents":200},"zipkin":{"address":""}},"trustDomain":"","trustDomainAliases":[],"useMCP":true},"image":"sidecar_injector","injectedAnnotations":{"container.apparmor.security.beta.kubernetes.io/istio-proxy":"runtime/default"},"neverInjectSelector":[],"nodeSelector":{},"podAnnotations":{},"podAntiAffinityLabelSelector":[],"podAntiAffinityTermLabelSelector":[],"replicaCount":1,"rewriteAppHTTPProbe":false,"rollingMaxSurge":"100%","rollingMaxUnavailable":"25%","tolerations":[]},"tracing":{"enabled":false}}

  config: |-
    policy: enabled
    alwaysInjectSelector:
      []
    neverInjectSelector:
      []
    template: |-
      rewriteAppHTTPProbe: {{ valueOrDefault .Values.sidecarInjectorWebhook.rewriteAppHTTPProbe false }}
      {{- if or (not .Values.istio_cni.enabled) .Values.global.proxy.enableCoreDump }}
      initContainers:
      {{ if ne (annotation .ObjectMeta `sidecar.istio.io/interceptionMode` .ProxyConfig.InterceptionMode) `NONE` }}
      {{- if not .Values.istio_cni.enabled }}
      - name: istio-init
      {{- if contains "/" .Values.global.proxy_init.image }}
        image: "{{ .Values.global.proxy_init.image }}"
      {{- else }}
        image: "{{ .Values.global.hub }}/{{ .Values.global.proxy_init.image }}:{{ .Values.global.tag }}"
      {{- end }}
        command:
        - istio-iptables
        - "-p"
        - "15001"
        - "-z"
        - "15006"
        - "-u"
        - 1337
        - "-m"
        - "{{ annotation .ObjectMeta `sidecar.istio.io/interceptionMode` .ProxyConfig.InterceptionMode }}"
        - "-i"
        - "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/includeOutboundIPRanges` .Values.global.proxy.includeIPRanges }}"
        - "-x"
        - "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/excludeOutboundIPRanges` .Values.global.proxy.excludeIPRanges }}"
        - "-b"
        - "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/includeInboundPorts` `*` }}"
        - "-d"
        - "{{ excludeInboundPort (annotation .ObjectMeta `status.sidecar.istio.io/port` .Values.global.proxy.statusPort) (annotation .ObjectMeta `traffic.sidecar.istio.io/excludeInboundPorts` .Values.global.proxy.excludeInboundPorts) }}"
        {{ if or (isset .ObjectMeta.Annotations `traffic.sidecar.istio.io/excludeOutboundPorts`) (ne .Values.global.proxy.excludeOutboundPorts "") -}}
        - "-o"
        - "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/excludeOutboundPorts` .Values.global.proxy.excludeOutboundPorts }}"
        {{ end -}}
        {{ if (isset .ObjectMeta.Annotations `traffic.sidecar.istio.io/kubevirtInterfaces`) -}}
        - "-k"
        - "{{ index .ObjectMeta.Annotations `traffic.sidecar.istio.io/kubevirtInterfaces` }}"
        {{ end -}}
        imagePullPolicy: "{{ .Values.global.imagePullPolicy }}"
      {{- if .Values.global.proxy.init.resources }}
        resources:
          {{ toYaml .Values.global.proxy.init.resources | indent 4 }}
      {{- else }}
        resources: {}
      {{- end }}
        securityContext:
          runAsUser: 0
          runAsNonRoot: false
          runAsGroup: 0
          allowPrivilegeEscalation: {{ .Values.global.proxy.privileged }}
          readOnlyRootFilesystem: false
          capabilities:
            add:
            - NET_ADMIN
            drop:
            - ALL
          {{- if .Values.global.proxy.privileged }}
          privileged: true
          {{- else }}
          privileged: false
          {{- end }}
          capabilities:
            add:
            - NET_ADMIN

        restartPolicy: Always
      {{- end }}
      {{  end -}}
      {{- if eq .Values.global.proxy.enableCoreDump true }}
      - name: enable-core-dump
        args:
        - -c
        - sysctl -w kernel.core_pattern=/var/lib/istio/core.proxy && ulimit -c unlimited
        command:
          - /bin/sh
        image: {{ $.Values.global.proxy.enableCoreDumpImage }}
        imagePullPolicy: IfNotPresent
        resources: {}
        securityContext:
          runAsUser: 0
          runAsNonRoot: false
          privileged: true
          runAsGroup: 0
          allowPrivilegeEscalation: true
          readOnlyRootFilesystem: false
      {{ end }}
      {{- end }}
      containers:
      - name: istio-proxy
      {{- if contains "/" (annotation .ObjectMeta `sidecar.istio.io/proxyImage` .Values.global.proxy.image) }}
        image: "{{ annotation .ObjectMeta `sidecar.istio.io/proxyImage` .Values.global.proxy.image }}"
      {{- else }}
        image: "{{ annotation .ObjectMeta `sidecar.istio.io/proxyImage` .Values.global.hub }}/{{ .Values.global.proxy.image }}:{{ .Values.global.tag }}"
      {{- end }}
        ports:
        - containerPort: 15090
          protocol: TCP
          name: http-envoy-prom
        args:
        - proxy
        - sidecar
        - --domain
        - $(POD_NAMESPACE).svc.{{ .Values.global.proxy.clusterDomain }}
        - --configPath
        - "{{ .ProxyConfig.ConfigPath }}"
        - --binaryPath
        - "{{ .ProxyConfig.BinaryPath }}"
        - --serviceCluster
        {{ if ne "" (index .ObjectMeta.Labels "app") -}}
        - "{{ index .ObjectMeta.Labels `app` }}.$(POD_NAMESPACE)"
        {{ else -}}
        - "{{ valueOrDefault .DeploymentMeta.Name `istio-proxy` }}.{{ valueOrDefault .DeploymentMeta.Namespace `default` }}"
        {{ end -}}
        - --drainDuration
        - "{{ formatDuration .ProxyConfig.DrainDuration }}"
        - --parentShutdownDuration
        - "{{ formatDuration .ProxyConfig.ParentShutdownDuration }}"
        - --discoveryAddress
        - "{{ annotation .ObjectMeta `sidecar.istio.io/discoveryAddress` .ProxyConfig.DiscoveryAddress }}"
      {{- if eq .Values.global.proxy.tracer "lightstep" }}
        - --lightstepAddress
        - "{{ .ProxyConfig.GetTracing.GetLightstep.GetAddress }}"
        - --lightstepAccessToken
        - "{{ .ProxyConfig.GetTracing.GetLightstep.GetAccessToken }}"
        - --lightstepSecure={{ .ProxyConfig.GetTracing.GetLightstep.GetSecure }}
        - --lightstepCacertPath
        - "{{ .ProxyConfig.GetTracing.GetLightstep.GetCacertPath }}"
      {{- else if eq .Values.global.proxy.tracer "zipkin" }}
        - --zipkinAddress
        - "{{ .ProxyConfig.GetTracing.GetZipkin.GetAddress }}"
      {{- else if eq .Values.global.proxy.tracer "datadog" }}
        - --datadogAgentAddress
        - "{{ .ProxyConfig.GetTracing.GetDatadog.GetAddress }}"
      {{- end }}
      {{- if .Values.global.proxy.logLevel }}
        - --proxyLogLevel={{ .Values.global.proxy.logLevel }}
      {{- end}}
      {{- if .Values.global.proxy.componentLogLevel }}
        - --proxyComponentLogLevel={{ .Values.global.proxy.componentLogLevel }}
      {{- end}}
        - --dnsRefreshRate
        - {{ .Values.global.proxy.dnsRefreshRate }}
        - --connectTimeout
        - "{{ formatDuration .ProxyConfig.ConnectTimeout }}"
      {{- if .Values.global.proxy.envoyStatsd.enabled }}
        - --statsdUdpAddress
        - "{{ .ProxyConfig.StatsdUdpAddress }}"
      {{- end }}
      {{- if .Values.global.proxy.envoyMetricsService.enabled }}
        - --envoyMetricsService
        - '{{ protoToJSON .ProxyConfig.EnvoyMetricsService }}'
      {{- end }}
      {{- if .Values.global.proxy.envoyAccessLogService.enabled }}
        - --envoyAccessLogService
        - '{{ protoToJSON .ProxyConfig.EnvoyAccessLogService }}'
      {{- end }}
        - --proxyAdminPort
        - "{{ .ProxyConfig.ProxyAdminPort }}"
        {{ if gt .ProxyConfig.Concurrency 0 -}}
        - --concurrency
        - "{{ .ProxyConfig.Concurrency }}"
        {{ end -}}
        - --controlPlaneAuthPolicy
        - "{{ annotation .ObjectMeta `sidecar.istio.io/controlPlaneAuthPolicy` .ProxyConfig.ControlPlaneAuthPolicy }}"
      {{- if (ne (annotation .ObjectMeta "status.sidecar.istio.io/port" (valueOrDefault .Values.global.proxy.statusPort 0 )) `0`) }}
        - --statusPort
        - "{{ annotation .ObjectMeta `status.sidecar.istio.io/port` .Values.global.proxy.statusPort }}"
        - --applicationPorts
        - "{{ annotation .ObjectMeta `readiness.status.sidecar.istio.io/applicationPorts` (applicationPorts .Spec.Containers) }}"
      {{- end }}
      {{- if .Values.global.trustDomain }}
        - --trust-domain={{ .Values.global.trustDomain }}
      {{- end }}
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_POD_PORTS
          value: |-
            [
            {{- range $index1, $c := .Spec.Containers }}
              {{- range $index2, $p := $c.Ports }}
                {{if or (ne $index1 0) (ne $index2 0)}},{{end}}{{ structToJSON $p }}
              {{- end}}
            {{- end}}
            ]
        - name: ISTIO_META_CLUSTER_ID
          value: "{{ valueOrDefault .Values.global.multiCluster.clusterName `Kubernetes` }}"
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INSTANCE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: SERVICE_ACCOUNT
          valueFrom:
            fieldRef:
              fieldPath: spec.serviceAccountName
        {{- if .Values.global.mtls.auto }}
        - name: ISTIO_AUTO_MTLS_ENABLED
          value: "true"
        {{- end }}
      {{- if eq .Values.global.proxy.tracer "datadog" }}
        - name: HOST_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
      {{- if isset .ObjectMeta.Annotations `apm.datadoghq.com/env` }}
      {{- range $key, $value := fromJSON (index .ObjectMeta.Annotations `apm.datadoghq.com/env`) }}
        - name: {{ $key }}
          value: "{{ $value }}"
      {{- end }}
      {{- end }}
      {{- end }}
        - name: ISTIO_META_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_CONFIG_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: SDS_ENABLED
          value: {{ $.Values.global.sds.enabled }}
        - name: ISTIO_META_INTERCEPTION_MODE
          value: "{{ or (index .ObjectMeta.Annotations `sidecar.istio.io/interceptionMode`) .ProxyConfig.InterceptionMode.String }}"
        - name: ISTIO_META_INCLUDE_INBOUND_PORTS
          value: "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/includeInboundPorts` (applicationPorts .Spec.Containers) }}"
        {{- if .Values.global.network }}
        - name: ISTIO_META_NETWORK
          value: "{{ .Values.global.network }}"
        {{- end }}
        {{ if .ObjectMeta.Annotations }}
        - name: ISTIO_METAJSON_ANNOTATIONS
          value: |
                 {{ toJSON .ObjectMeta.Annotations }}
        {{ end }}
        {{ if .ObjectMeta.Labels }}
        - name: ISTIO_METAJSON_LABELS
          value: |
                 {{ toJSON .ObjectMeta.Labels }}
        {{ end }}
        {{- if .DeploymentMeta.Name }}
        - name: ISTIO_META_WORKLOAD_NAME
          value: {{ .DeploymentMeta.Name }}
        {{ end }}
        {{- if and .TypeMeta.APIVersion .DeploymentMeta.Name }}
        - name: ISTIO_META_OWNER
          value: kubernetes://apis/{{ .TypeMeta.APIVersion }}/namespaces/{{ valueOrDefault .DeploymentMeta.Namespace `default` }}/{{ toLower .TypeMeta.Kind}}s/{{ .DeploymentMeta.Name }}
         {{- end}}
        {{- if (isset .ObjectMeta.Annotations `sidecar.istio.io/bootstrapOverride`) }}
        - name: ISTIO_BOOTSTRAP_OVERRIDE
          value: "/etc/istio/custom-bootstrap/custom_bootstrap.json"
        {{- end }}
        {{- if .Values.global.sds.customTokenDirectory }}
        - name: ISTIO_META_SDS_TOKEN_PATH
          value: "{{ .Values.global.sds.customTokenDirectory -}}/sdstoken"
        {{- end }}
        {{- if .Values.global.meshID }}
        - name: ISTIO_META_MESH_ID
          value: "{{ .Values.global.meshID }}"
        {{- else if .Values.global.trustDomain }}
        - name: ISTIO_META_MESH_ID
          value: "{{ .Values.global.trustDomain }}"
        {{- end }}
        {{- if eq .Values.global.proxy.tracer "stackdriver" }}
        - name: STACKDRIVER_TRACING_ENABLED
          value: "true"
        - name: STACKDRIVER_TRACING_DEBUG
          value: "{{ .ProxyConfig.GetTracing.GetStackdriver.GetDebug }}"
        {{- if .ProxyConfig.GetTracing.GetStackdriver.GetMaxNumberOfAnnotations }}
        - name: STACKDRIVER_TRACING_MAX_NUMBER_OF_ANNOTATIONS
          value: "{{ .ProxyConfig.GetTracing.GetStackdriver.GetMaxNumberOfAnnotations }}"
        {{- end }}
        {{- if .ProxyConfig.GetTracing.GetStackdriver.GetMaxNumberOfAttributes }}
        - name: STACKDRIVER_TRACING_MAX_NUMBER_OF_ATTRIBUTES
          value: "{{ .ProxyConfig.GetTracing.GetStackdriver.GetMaxNumberOfAttributes }}"
        {{- end }}
        {{- if .ProxyConfig.GetTracing.GetStackdriver.GetMaxNumberOfMessageEvents }}
        - name: STACKDRIVER_TRACING_MAX_NUMBER_OF_MESSAGE_EVENTS
          value: "{{ .ProxyConfig.GetTracing.GetStackdriver.GetMaxNumberOfMessageEvents }}"
        {{- end }}
        {{- end }}
        imagePullPolicy: {{ .Values.global.imagePullPolicy }}
        {{ if ne (annotation .ObjectMeta `status.sidecar.istio.io/port` (valueOrDefault .Values.global.proxy.statusPort 0 )) `0` }}
        readinessProbe:
          httpGet:
            path: /healthz/ready
            port: {{ annotation .ObjectMeta `status.sidecar.istio.io/port` .Values.global.proxy.statusPort }}
          initialDelaySeconds: {{ annotation .ObjectMeta `readiness.status.sidecar.istio.io/initialDelaySeconds` .Values.global.proxy.readinessInitialDelaySeconds }}
          periodSeconds: {{ annotation .ObjectMeta `readiness.status.sidecar.istio.io/periodSeconds` .Values.global.proxy.readinessPeriodSeconds }}
          failureThreshold: {{ annotation .ObjectMeta `readiness.status.sidecar.istio.io/failureThreshold` .Values.global.proxy.readinessFailureThreshold }}
        {{ end -}}
        securityContext:
          {{- if .Values.global.proxy.privileged }}
          privileged: true
          {{- else }}
          privileged: false
          {{- end }}
          {{- if ne .Values.global.proxy.enableCoreDump true }}
          readOnlyRootFilesystem: true
          {{- else }}
          readOnlyRootFilesystem: false
          {{- end }}
          capabilities:
          {{ if eq (annotation .ObjectMeta `sidecar.istio.io/interceptionMode` .ProxyConfig.InterceptionMode) `TPROXY` -}}
            add:
            - NET_ADMIN
          {{- end }}
            drop:
            - ALL
          runAsGroup: 1337
          runAsUser: 1337
          allowPrivilegeEscalation: {{ .Values.global.proxy.privileged }}
          runAsNonRoot: true
        resources:
          {{ if or (isset .ObjectMeta.Annotations `sidecar.istio.io/proxyCPU`) (isset .ObjectMeta.Annotations `sidecar.istio.io/proxyMemory`) -}}
          requests:
            {{ if (isset .ObjectMeta.Annotations `sidecar.istio.io/proxyCPU`) -}}
            cpu: "{{ index .ObjectMeta.Annotations `sidecar.istio.io/proxyCPU` }}"
            {{ end}}
            {{ if (isset .ObjectMeta.Annotations `sidecar.istio.io/proxyMemory`) -}}
            memory: "{{ index .ObjectMeta.Annotations `sidecar.istio.io/proxyMemory` }}"
            {{ end }}
        {{ else -}}
      {{- if .Values.global.proxy.resources }}
          {{ toYaml .Values.global.proxy.resources | indent 4 }}
      {{- end }}
        {{  end -}}
        volumeMounts:
        {{ if (isset .ObjectMeta.Annotations `sidecar.istio.io/bootstrapOverride`) }}
        - mountPath: /etc/istio/custom-bootstrap
          name: custom-bootstrap-volume
        {{- end }}
        - mountPath: /etc/istio/proxy
          name: istio-envoy
        {{- if .Values.global.sds.enabled }}
        - mountPath: /var/run/sds
          name: sds-uds-path
          readOnly: true
        - mountPath: /var/run/secrets/tokens
          name: istio-token
        {{- if .Values.global.sds.customTokenDirectory }}
        - mountPath: "{{ .Values.global.sds.customTokenDirectory -}}"
          name: custom-sds-token
          readOnly: true
        {{- end }}
        {{- else }}
        - mountPath: /etc/certs/
          name: istio-certs
          readOnly: true
        {{- end }}
        {{- if and (eq .Values.global.proxy.tracer "lightstep") .Values.global.tracer.lightstep.cacertPath }}
        - mountPath: {{ directory .ProxyConfig.GetTracing.GetLightstep.GetCacertPath }}
          name: lightstep-certs
          readOnly: true
        {{- end }}
          {{- if isset .ObjectMeta.Annotations `sidecar.istio.io/userVolumeMount` }}
          {{ range $index, $value := fromJSON (index .ObjectMeta.Annotations `sidecar.istio.io/userVolumeMount`) }}
        - name: "{{  $index }}"
          {{ toYaml $value | indent 4 }}
          {{ end }}
          {{- end }}
      volumes:
      {{- if (isset .ObjectMeta.Annotations `sidecar.istio.io/bootstrapOverride`) }}
      - name: custom-bootstrap-volume
        configMap:
          name: {{ annotation .ObjectMeta `sidecar.istio.io/bootstrapOverride` "" }}
      {{- end }}
      - emptyDir:
          medium: Memory
        name: istio-envoy
      {{- if .Values.global.sds.enabled }}
      - name: sds-uds-path
        hostPath:
          path: /var/run/sds
      - name: istio-token
        projected:
          sources:
            - serviceAccountToken:
                path: istio-token
                expirationSeconds: 43200
                audience: {{ .Values.global.sds.token.aud }}
      {{- if .Values.global.sds.customTokenDirectory }}
      - name: custom-sds-token
        secret:
          secretName: sdstokensecret
      {{- end }}
      {{- else }}
      - name: istio-certs
        secret:
          optional: true
          {{ if eq .Spec.ServiceAccountName "" }}
          secretName: istio.default
          {{ else -}}
          secretName: {{  printf "istio.%s" .Spec.ServiceAccountName }}
          {{  end -}}
        {{- if isset .ObjectMeta.Annotations `sidecar.istio.io/userVolume` }}
        {{range $index, $value := fromJSON (index .ObjectMeta.Annotations `sidecar.istio.io/userVolume`) }}
      - name: "{{ $index }}"
        {{ toYaml $value | indent 2 }}
        {{ end }}
        {{ end }}
      {{- end }}
      {{- if and (eq .Values.global.proxy.tracer "lightstep") .Values.global.tracer.lightstep.cacertPath }}
      - name: lightstep-certs
        secret:
          optional: true
          secretName: lightstep.cacert
      {{- end }}
      {{- if .Values.global.podDNSSearchNamespaces }}
      dnsConfig:
        searches:
          {{- range .Values.global.podDNSSearchNamespaces }}
          - {{ render . }}
          {{- end }}
      {{- end }}
      podRedirectAnnot:
         sidecar.istio.io/interceptionMode: "{{ annotation .ObjectMeta `sidecar.istio.io/interceptionMode` .ProxyConfig.InterceptionMode }}"
         traffic.sidecar.istio.io/includeOutboundIPRanges: "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/includeOutboundIPRanges` .Values.global.proxy.includeIPRanges }}"
         traffic.sidecar.istio.io/excludeOutboundIPRanges: "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/excludeOutboundIPRanges` .Values.global.proxy.excludeIPRanges }}"
         traffic.sidecar.istio.io/includeInboundPorts: "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/includeInboundPorts` (includeInboundPorts .Spec.Containers) }}"
         traffic.sidecar.istio.io/excludeInboundPorts: "{{ excludeInboundPort (annotation .ObjectMeta `status.sidecar.istio.io/port` .Values.global.proxy.statusPort) (annotation .ObjectMeta `traffic.sidecar.istio.io/excludeInboundPorts` .Values.global.proxy.excludeInboundPorts) }}"
      {{ if or (isset .ObjectMeta.Annotations `traffic.sidecar.istio.io/excludeOutboundPorts`) (ne .Values.global.proxy.excludeOutboundPorts "") }}
         traffic.sidecar.istio.io/excludeOutboundPorts: "{{ annotation .ObjectMeta `traffic.sidecar.istio.io/excludeOutboundPorts` .Values.global.proxy.excludeOutboundPorts }}"
      {{- end }}
         traffic.sidecar.istio.io/kubevirtInterfaces: "{{ index .ObjectMeta.Annotations `traffic.sidecar.istio.io/kubevirtInterfaces` }}"
    injectedAnnotations:
      "container.apparmor.security.beta.kubernetes.io/istio-proxy": "runtime/default"

values-cni.yml

cniBinDir: /var/vcap/jobs/kubelet/packages/cni/bin
excludeNamespaces: ['pks-system', 'kube-system',  'kube-node-lease', 'kube-public', 'istio-system']
psp_cluster_role: pks-privileged

values-istio.yml

istio_cni:
  enabled: true
sidecarInjectorWebhook:
  injectedAnnotations:
    container.apparmor.security.beta.kubernetes.io/istio-proxy: runtime/default

zzillustrative diff

72a73,75
>           runAsGroup: 0
>           allowPrivilegeEscalation: {{ .Values.global.proxy.privileged }}
>           readOnlyRootFilesystem: false
75a79,80
>             drop:
>             - ALL
77a83,84
>           {{- else }}
>           privileged: false
78a86,89
>           capabilities:
>             add:
>             - NET_ADMIN
> 
95a107,109
>           runAsGroup: 0
>           allowPrivilegeEscalation: true
>           readOnlyRootFilesystem: false
309a324,325
>           {{- else }}
>           privileged: false
312a329,330
>           {{- else }}
>           readOnlyRootFilesystem: false
314d331
<           {{ if eq (annotation .ObjectMeta `sidecar.istio.io/interceptionMode` .ProxyConfig.InterceptionMode) `TPROXY` -}}
315a333
>           {{ if eq (annotation .ObjectMeta `sidecar.istio.io/interceptionMode` .ProxyConfig.InterceptionMode) `TPROXY` -}}
318,321d335
<           runAsGroup: 1337
<           {{ else -}}
<           {{ if .Values.global.sds.enabled }}
<           runAsGroup: 1337
322a337,339
>             drop:
>             - ALL
>           runAsGroup: 1337
324c341,342
<           {{- end }}
---
>           allowPrivilegeEscalation: {{ .Values.global.proxy.privileged }}
>           runAsNonRoot: true