Cilium envoy config (CiliumEnvoyConfig) example for 1.13.1, with multiple ports, header-based traffic split, and header-based dynamic routing

cilium-envoy-config-multiple-ports-header-split.yaml

apiVersion: cilium.io/v2
kind: CiliumEnvoyConfig
metadata:
  name: mesh-grpc-server-helloworld
  namespace: etalon-grpc
spec:
  backendServices:
  # this is my target service, and this is it's main port, that we'll be processing
  - name: grpc-server-helloworld
    namespace: etalon-grpc
    number:
    - "50051"
  # this is it's second port, it will be just proxied
  - name: grpc-server-helloworld
    namespace: etalon-grpc
    number:
    - "50052"
  # this is blue-green version of my service, it's available for tests during rollout, but unavailable for users. we'll make it available with special header
  - name: grpc-server-helloworld-next
    namespace: etalon-grpc
    number:
    - "50051"
  # this is dev environment with my app. i'll route traffic there by separate header and it's value
  - name: grpc-server-helloworld
    namespace: etalon-grpc-devel
    number:
    - "50051"
  resources:
  - '@type': type.googleapis.com/envoy.config.listener.v3.Listener
    filterChains:
    - filterChainMatch:
        # this is required if service is having multiple ports, but we want to process just one of them
        destination_port: 50051
        transportProtocol: raw_buffer
      filters:
      - name: envoy.filters.network.http_connection_manager
        typedConfig:
          '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          httpFilters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          rds:
            routeConfigName: listener-route
          statPrefix: mesh-grpc-server-helloworld
    - filterChainMatch:
        # simple tcp proxy for ports that we don't want to route by some special rules, they should just live as they did
        destination_port: 50052
        transportProtocol: raw_buffer
      filters:
      - name: envoy.filters.network.tcp_proxy
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
          stat_prefix: mesh-grpc-server-helloworld
          cluster: etalon-grpc/grpc-server-helloworld:50052
    # new versions add this listener to envoy as "ns/svcname/listenername" so it can be the same inside all CEC's, while they are applied to different services
    name: listener
    socketOptions:
    - description: Enable TCP keep-alive (default to enabled)
      intValue: "1"
      level: "1"
      name: "9"
      state: STATE_LISTENING
    - description: TCP keep-alive idle time (in seconds) (defaults to 10s)
      intValue: "10"
      level: "6"
      name: "4"
      state: STATE_LISTENING
    - description: TCP keep-alive probe intervals (in seconds) (defaults to 5s)
      intValue: "5"
      level: "6"
      name: "5"
      state: STATE_LISTENING
    - description: TCP keep-alive probe max failures.
      intValue: "10"
      level: "6"
      name: "6"
      state: STATE_LISTENING
  - '@type': type.googleapis.com/envoy.config.route.v3.RouteConfiguration
    name: listener-route
    virtualHosts:
    - domains:
      - '*'
      name: '*'
      routes:
      - match:
          headers:
          # if this header present - let's route traffic to new version during b/g rollout
          - name: x-mycompany-route-tests
            present_match: true
          prefix: /
        route:
          cluster: etalon-grpc/grpc-server-helloworld-next:50051
          maxStreamDuration:
            maxStreamDuration: 0s
      - match:
          headers:
          # if there is this header
          - name: x-mycompany-route-cluster
            present_match: true
          # let's route to the cluster from this header's value. 
          # this double header scheme can be replaced with single header x-mycompany-route-for-sha1("<ns>/<svcname>")[-6:] that can be used then in "cluster_header" instruction
          - name: x-mycompany-route-for
            string_match:
              contains: etalon-grpc/grpc-server-helloworld
          prefix: /
        route:
          cluster_header: x-mycompany-route-cluster
          maxStreamDuration:
            maxStreamDuration: 0s
      # default route leads to the main port original destination
      - match:
          prefix: /
        route:
          cluster: etalon-grpc/grpc-server-helloworld:50051
          maxStreamDuration:
            maxStreamDuration: 0s
  # all clusters are described the same. this can be templated as "{% for ... %}" :)
  - '@type': type.googleapis.com/envoy.config.cluster.v3.Cluster
    connectTimeout: 5s
    name: etalon-grpc/grpc-server-helloworld:50051
    outlierDetection:
      consecutiveLocalOriginFailure: 2
      splitExternalLocalOriginErrors: true
    type: EDS
    typedExtensionProtocolOptions:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        '@type': type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        useDownstreamProtocolConfig:
          http2ProtocolOptions: {}
  - '@type': type.googleapis.com/envoy.config.cluster.v3.Cluster
    connectTimeout: 5s
    name: etalon-grpc/grpc-server-helloworld:50052
    outlierDetection:
      consecutiveLocalOriginFailure: 2
      splitExternalLocalOriginErrors: true
    type: EDS
  - '@type': type.googleapis.com/envoy.config.cluster.v3.Cluster
    connectTimeout: 5s
    name: etalon-grpc/grpc-server-helloworld-next:50051
    outlierDetection:
      consecutiveLocalOriginFailure: 2
      splitExternalLocalOriginErrors: true
    type: EDS
    typedExtensionProtocolOptions:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        '@type': type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        useDownstreamProtocolConfig:
          http2ProtocolOptions: {}
  - '@type': type.googleapis.com/envoy.config.cluster.v3.Cluster
    connectTimeout: 5s
    name: etalon-grpc-devel/grpc-server-helloworld:50051
    outlierDetection:
      consecutiveLocalOriginFailure: 2
      splitExternalLocalOriginErrors: true
    type: EDS
    typedExtensionProtocolOptions:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        '@type': type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        useDownstreamProtocolConfig:
          # dunno if it is required for http or grpc
          http2ProtocolOptions: {}
  services:
  # cilium selects first listener in resources so you shouldn't really point this out(actually, if you set it to let's say "listener", cilium will say that such listener is not found - because in reality it is named "<ns>/<svcname>/<listenername>"
  - listener: ""
    name: grpc-server-helloworld
    namespace: etalon-grpc

Something that might have changed in Cilium 1.18+ is that the filter_chain_match.destination_port seems to match against the Envoy listener port which we do not have control over...

# cilium-dbg envoy admin listeners
some-listener::127.0.0.1:11415
some-listener::[::1]:11415

Best alternative I was able to come up with is to use a frontend service per port and map each service to a dedicated listener in the CiliumEnvoyConfig services section. Then each listener can do whatever it needs to do and route it to an endpoint (cluster).