Test nodePort ExternalTrafficPolicy

gistfile1.txt

Verifying OVN-K externalTrafficPolicy Feature
1. 2 CLient pods on two different nodes 

[astoycos@nfvsdn-03 demo]$ kubectl get pods -n logging -o wide --show-labels
NAME      READY   STATUS    RESTARTS   AGE    IP           NODE          NOMINATED NODE   READINESS GATES   LABELS
client    1/1     Running   0          112m   10.244.0.4   ovn-worker2   <none>           <none>            app=webserver
client2   1/1     Running   0          106m   10.244.2.6   ovn-worker    <none>           <none>            app=webserver

2. 2 Nodeport services pointing to those pods, one with ETP=Local the other with ETP=Cluster(the default) 

[astoycos@nfvsdn-03 demo]$ cat test_service_nodeport.yaml 
apiVersion: v1
kind: Service
metadata:
  name: webserver
spec:
  type: NodePort
  selector:
    app: webserver
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
  externalTrafficPolicy: Local

[astoycos@nfvsdn-03 demo]$ cat test_service_nodeport2.yaml 
apiVersion: v1
kind: Service
metadata:
  name: webserver2
spec:
  type: NodePort
  selector:
    app: webserver
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      
[astoycos@nfvsdn-03 demo]$ kubectl get svc -o wide -n logging
NAME         TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE     SELECTOR
webserver    NodePort   10.96.219.134   <none>        8080:30964/TCP   2m24s   app=webserver
webserver2   NodePort   10.96.201.220   <none>        8080:31269/TCP   14m     app=webserver     

[astoycos@nfvsdn-03 demo]$ kubectl get nodes -A -o wide
NAME                STATUS   ROLES                  AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                                     KERNEL-VERSION           CONTAINER-RUNTIME
ovn-control-plane   Ready    control-plane,master   3h47m   v1.20.0   172.18.0.4    <none>        Ubuntu Groovy Gorilla (development branch)   5.8.15-201.fc32.x86_64   containerd://1.4.0
ovn-worker          Ready    <none>                 3h47m   v1.20.0   172.18.0.3    <none>        Ubuntu Groovy Gorilla (development branch)   5.8.15-201.fc32.x86_64   containerd://1.4.0
ovn-worker2         Ready    <none>                 3h47m   v1.20.0   172.18.0.2    <none>        Ubuntu Groovy Gorilla (development branch)   5.8.15-201.fc32.x86_64   containerd://1.4.0


3. Let's test the webserver service with ETP=local

Endpoint pods are only on ovn-worker(172.18.0.3) and ovn-worker2(172.18.0.2) so we should only be able to reach
backends at those VIPS 

## CURLING NODEPORT WEBSERVER FROM UNDERLYING HOST (172.18.0.1) 
[astoycos@nfvsdn-03 demo]$ curl -m 2 172.18.0.3:30964
Hello Kubernetes!
[astoycos@nfvsdn-03 demo]$ curl -m 2 172.18.0.2:30964
Hello Kubernetes!
[astoycos@nfvsdn-03 demo]$ curl -m 2 172.18.0.4:30964   -------> Traffic Correctly blackholes 
curl: (28) Connection timed out after 2001 milliseconds

## MAKE SURE SRC IP IS MAINTAINED (Using ACL audit logging feature) 
2021-03-31T18:31:44.144Z|00039|acl_log(ovn_pinctrl0)|INFO|name="allow-all-ingress", verdict=allow, severity=alert: tcp,vlan_tci=0x0000,dl_src=0a:58:0a:f4:00:01,dl_dst=0a:58:0a:f4:00:04,nw_src=172.18.0.1,nw_dst=10.244.0.4,nw_tos=0,nw_ecn=0,nw_ttl=62,tp_src=60592,tp_dst=8080,tcp_flags=ack
## SRC IP 172.18.0.1 is maintained 

4. Let's test webserver2 service with ETP=cluster

Nodeport should be reachable via all VIPS 

## CURLING NODEPORT WEBSERVER2 FROM UNDERLYING HOST (172.18.0.1)
[astoycos@nfvsdn-03 demo]$ curl -m 2 172.18.0.3:31269
Hello Kubernetes!
[astoycos@nfvsdn-03 demo]$ curl -m 2 172.18.0.2:31269
Hello Kubernetes!
[astoycos@nfvsdn-03 demo]$ curl -m 2 172.18.0.4:31269
Hello Kubernetes!                                      --------> Traffic was now reachable via all VIPs

### SRC IP IS NOW NOT MAINTIANED, PACKETS ARE SNATED TO GATEWAY ROUTER IP 
2021-03-31T18:36:05.289Z|00039|acl_log(ovn_pinctrl0)|INFO|name="allow-all-ingress", verdict=allow, severity=alert: tcp,vlan_tci=0x0000,dl_src=0a:58:0a:f4:02:01,dl_dst=0a:58:0a:f4:02:06,nw_src=100.64.0.4,nw_dst=10.244.2.6,nw_tos=0,nw_ecn=0,nw_ttl=62,tp_src=49866,tp_dst=8080,tcp_flags=ack


4. Checking the OVN entities 

We can see the new "local" loadbalancr on gateway router correctly does not SNAT and has an empty endpoint list for the node without any backends

_uuid               : fa1e210b-4616-4719-a7a4-079c73e4e698
external_ids        : {TCP_lb_gateway_router=GR_ovn-control-plane_local}
health_check        : []
ip_port_mappings    : {}
name                : ""
options             : {lb_skip_snat="true"}
protocol            : tcp
selection_fields    : []
vips                : {"172.18.0.4:30964"=""}

While the loadalancer with a backend only contains the one local to itself 

_uuid               : d7779a35-8c3e-4f12-a25c-67538b871e3a
external_ids        : {TCP_lb_gateway_router=GR_ovn-worker_local}
health_check        : []
ip_port_mappings    : {}
name                : ""
options             : {lb_skip_snat="true"}
protocol            : tcp
selection_fields    : []
vips                : {"172.18.0.3:30964"="10.244.2.6:8080"}