Skip to content

Instantly share code, notes, and snippets.

View anfernee's full-sized avatar

Yongkun Anfernee Gui anfernee

  • Google
  • San Francisco Bay area
View GitHub Profile

Create 3 nodes with the third node as outside cluster node:

sudo K8S_NODES=3 NO_CILIUM_ON_NODE=k8s3 NETNEXT=true ginkgo --focus "K8s.*fragments" -v -- --cilium.provision=true

Issues

cilium-agent failed to start

Node/Pod IP
Source Pod 10.11.1.103
Source Node 192.168.11.12
Gateway Node 192.168.11.11
External Service 192.168.11.13

iptables

Added the following on both source and gateway node. Egress gateway datapath still works.

apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURDekNDQWZPZ0F3SUJBZ0lRSm5MME9SRktmNGdlQ3NXcU5jWnpPREFOQmdrcWhraUc5dzBCQVFzRkFEQXYKTVMwd0t3WURWUVFERXlRek4yVTJZakEyWkMwMlltWXhMVEV4WldJdFlUa3hZUzFoTmpCbFl6SmhPV0V6T0RBdwpIaGNOTWpFd01qRXdNakkwTWpFeldoY05Nall3TWpBNU1qSTBNakV6V2pBdk1TMHdLd1lEVlFRREV5UXpOMlUyCllqQTJaQzAyWW1ZeExURXhaV0l0WVRreFlTMWhOakJsWXpKaE9XRXpPREF3Z2dFaU1BMEdDU3FHU0liM0RRRUIKQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUURvTkd1RURFSlhLUlEwai9MSUFZK3pyMC9FMDhtcS9reTI3K21WeVB4TgprOXEzekRnSVllTTU1ZG4vZXIvK2ZNN3g1aGhjM3pzTDA3L3NFSDNuNG55T0hYVDg3TWQ1OEY3dWZkeE5QVXNNCjJRSGtqdjJ6ekxoZTNYRm9YWm9icVRjVEoyeFJlUnAyQmtGTzVxb0ppemNTdUY3aUlnbmNPdDVhem40RVhkZ3kKUFN4Z0dCUFlRdWozL0FVQXZOYnhtZWtMMTNibnFMUWdNdG9CTVhKUEVqYk9lYmc3YjZCK0JtN0dITGpZMzMrVQp0MHRwNndHd2llMmFobEJJRDhDTDVLK21Wb2Q5TmcwZkNRZmcrZkh6L01CQXpvNm1UMHk5K0liek1peFNKTzk3ClVDZ0pubk5VbUQ2ZXRpbWl1KzRoa1NkUVFGMlJHbkRNY3dySU9TeWdXSC8xQWdNQkFBR2pJekFoTUE0R0ExVWQKRHdFQi93UUVBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUEw
@anfernee
anfernee / win-pod.yaml
Created January 23, 2021 01:39
Useful k8s yamls
apiVersion: apps/v1
kind: Deployment
metadata:
name: windows-powershell
labels:
app: windows-powershell
spec:
replicas: 1
selector:
matchLabels:
# First rule SNAT the packet that comes out of the gw node
-A CILIUM_POST_nat -s 10.11.1.0/24 ! -d 10.11.0.0/16 ! -o cilium_+ -m comment --comment "cilium masquerade non-cluster" -j MASQUERADE

-A CILIUM_POST_nat ! -o cilium_host -m comment --comment "exclude non-cilium_host traffic from masquerade" -j RETURN
-A CILIUM_POST_nat -m mark --mark 0xa00/0xe00 -m comment --comment "exclude proxy return traffic from masquarade" -j ACCEPT
-A CILIUM_POST_nat ! -s 10.11.1.253/32 ! -d 10.11.1.0/24 -o cilium_host -m comment --comment "cilium host->cluster masquerade" -j SNAT --to-source 10.11.1.253
-A CILIUM_POST_nat -s 127.0.0.1/32 -o cilium_host -m comment --comment "cilium host->cluster from 127.0.0.1 masquerade" -j SNAT --to-source 10.11.1.253
-A CILIUM_POST_nat -o cilium_host -m mark --mark 0xf00/0xf00 -m conntrack --ctstate DNAT -m comment --comment "hairpin traffic that originated from a local pod" -j SNAT --to-source 10.11.1.253
diff --git a/bpf/lib/icmp6.h b/bpf/lib/icmp6.h
index 3446b727e..4fa0aa57e 100644
--- a/bpf/lib/icmp6.h
+++ b/bpf/lib/icmp6.h
@@ -9,6 +9,8 @@
#include "common.h"
#include "eth.h"
#include "drop.h"
+#include "eps.h"
+#include "dbg.h"
vagrant@k8s1:~/go/src/github.com/cilium/cilium$ sudo tcpdump -i enp0s8 icmp6 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes




18:11:10.397859 IP6 k8s2 > ff02::1:ff00:2c69: ICMP6, neighbor solicitation, who has f00d::a0f:0:0:2c69, length 32
18:11:11.478191 IP6 k8s2 > ff02::1:ff00:2c69: ICMP6, neighbor solicitation, who has f00d::a0f:0:0:2c69, length 32
// creates the connection
config, err := clientcmd.BuildConfigFromFlags(master, kubeconfig)
if err != nil {
klog.Fatal(err)
}
var rt2 *http2.Transport
var once sync.Once
config.WrapTransport = func(rt http.RoundTripper) http.RoundTripper {
fmt.Println("wrapper")
#!/bin/bash
NODE_COMMANDS=( \
"uptime" \
"df --all --inodes" \
"ip addr" \
"sudo iptables-save --counters" \
"mount" \
"ip route list table all" \
"top -bn1" \