anfernee

Overview

The DPDK implements a run to completion model for packet processing, where all resources must be allocated prior to calling Data Plane applications, running as execution units on logical processing cores. The model does not support a scheduler and all devices are accessed by polling. The primary reason for not using interrupts is the performance overhead imposed by interrupt processing.

ref: https://doc.dpdk.org/guides/prog_guide/overview.html

Core components

anfernee

All subsystem defined in: https://elixir.bootlin.com/linux/v4.20/source/include/linux/cgroup_subsys.h

Implementation defined in (e.g. cpuset): https://elixir.bootlin.com/linux/v4.20/source/kernel/cgroup/cpuset.c

Freezer is a very simple example (only has 3 files) https://elixir.bootlin.com/linux/v4.20/source/kernel/cgroup/freezer.c

anfernee

#!/bin/bash

NODE_COMMANDS=( \
  "uptime"                           \
  "df --all --inodes"                \
  "ip addr"                          \
  "sudo iptables-save --counters"    \
  "mount"                            \
  "ip route list table all"          \
  "top -bn1"                         \

anfernee

	// creates the connection
	config, err := clientcmd.BuildConfigFromFlags(master, kubeconfig)
	if err != nil {
		klog.Fatal(err)
	}

	var rt2 *http2.Transport
	var once sync.Once
	config.WrapTransport = func(rt http.RoundTripper) http.RoundTripper {
		fmt.Println("wrapper")

anfernee

vagrant@k8s1:~/go/src/github.com/cilium/cilium$ sudo tcpdump -i enp0s8 icmp6 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes




18:11:10.397859 IP6 k8s2 > ff02::1:ff00:2c69: ICMP6, neighbor solicitation, who has f00d::a0f:0:0:2c69, length 32
18:11:11.478191 IP6 k8s2 > ff02::1:ff00:2c69: ICMP6, neighbor solicitation, who has f00d::a0f:0:0:2c69, length 32

anfernee

diff --git a/bpf/lib/icmp6.h b/bpf/lib/icmp6.h
index 3446b727e..4fa0aa57e 100644
--- a/bpf/lib/icmp6.h
+++ b/bpf/lib/icmp6.h
@@ -9,6 +9,8 @@
 #include "common.h"
 #include "eth.h"
 #include "drop.h"
+#include "eps.h"
+#include "dbg.h"

anfernee

# First rule SNAT the packet that comes out of the gw node
-A CILIUM_POST_nat -s 10.11.1.0/24 ! -d 10.11.0.0/16 ! -o cilium_+ -m comment --comment "cilium masquerade non-cluster" -j MASQUERADE

-A CILIUM_POST_nat ! -o cilium_host -m comment --comment "exclude non-cilium_host traffic from masquerade" -j RETURN
-A CILIUM_POST_nat -m mark --mark 0xa00/0xe00 -m comment --comment "exclude proxy return traffic from masquarade" -j ACCEPT
-A CILIUM_POST_nat ! -s 10.11.1.253/32 ! -d 10.11.1.0/24 -o cilium_host -m comment --comment "cilium host->cluster masquerade" -j SNAT --to-source 10.11.1.253
-A CILIUM_POST_nat -s 127.0.0.1/32 -o cilium_host -m comment --comment "cilium host->cluster from 127.0.0.1 masquerade" -j SNAT --to-source 10.11.1.253
-A CILIUM_POST_nat -o cilium_host -m mark --mark 0xf00/0xf00 -m conntrack --ctstate DNAT -m comment --comment "hairpin traffic that originated from a local pod" -j SNAT --to-source 10.11.1.253

anfernee

apiVersion: apps/v1
kind: Deployment
metadata:
  name: windows-powershell
  labels:
    app: windows-powershell
spec:
  replicas: 1
  selector:
    matchLabels:

anfernee

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURDekNDQWZPZ0F3SUJBZ0lRSm5MME9SRktmNGdlQ3NXcU5jWnpPREFOQmdrcWhraUc5dzBCQVFzRkFEQXYKTVMwd0t3WURWUVFERXlRek4yVTJZakEyWkMwMlltWXhMVEV4WldJdFlUa3hZUzFoTmpCbFl6SmhPV0V6T0RBdwpIaGNOTWpFd01qRXdNakkwTWpFeldoY05Nall3TWpBNU1qSTBNakV6V2pBdk1TMHdLd1lEVlFRREV5UXpOMlUyCllqQTJaQzAyWW1ZeExURXhaV0l0WVRreFlTMWhOakJsWXpKaE9XRXpPREF3Z2dFaU1BMEdDU3FHU0liM0RRRUIKQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUURvTkd1RURFSlhLUlEwai9MSUFZK3pyMC9FMDhtcS9reTI3K21WeVB4TgprOXEzekRnSVllTTU1ZG4vZXIvK2ZNN3g1aGhjM3pzTDA3L3NFSDNuNG55T0hYVDg3TWQ1OEY3dWZkeE5QVXNNCjJRSGtqdjJ6ekxoZTNYRm9YWm9icVRjVEoyeFJlUnAyQmtGTzVxb0ppemNTdUY3aUlnbmNPdDVhem40RVhkZ3kKUFN4Z0dCUFlRdWozL0FVQXZOYnhtZWtMMTNibnFMUWdNdG9CTVhKUEVqYk9lYmc3YjZCK0JtN0dITGpZMzMrVQp0MHRwNndHd2llMmFobEJJRDhDTDVLK21Wb2Q5TmcwZkNRZmcrZkh6L01CQXpvNm1UMHk5K0liek1peFNKTzk3ClVDZ0pubk5VbUQ2ZXRpbWl1KzRoa1NkUVFGMlJHbkRNY3dySU9TeWdXSC8xQWdNQkFBR2pJekFoTUE0R0ExVWQKRHdFQi93UUVBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUEw

anfernee

Node/Pod	IP
Source Pod	10.11.1.103
Source Node	192.168.11.12
Gateway Node	192.168.11.11
External Service	192.168.11.13

iptables

Added the following on both source and gateway node. Egress gateway datapath still works.