Skip to content

Instantly share code, notes, and snippets.

Design and Implementation of K8s Services Proxy using eBpf

Goals and Priorities

  • Build an eBpf based implementation of Kubernetes Services (ClusterIP, NodePort, LoadBalancer) to replace Kube-proxy/ iptables and CNI based implementations of Kubernetes services.
  • The goal is not "use as much eBpf" as possible but rather to use eBpf selectively and opportunistically and also to leverage standard kernel datapaths as much as possible unless there is a good reason to do otherwise.
  • Since iptables packages are being deprecated in the Linux kernel and RHEL, it is necessary to have an implementation of kube-proxy that does not depend on iptables. See iptables deprecation
  • Primary design requirement is to retain end user experience for stability and debuggability when replacing the kube-proxy/ iptables based datapath. This requirement is more important that flat out data plane performanc
@enixdark
enixdark / promtail_docker_logs.md
Created November 2, 2022 16:36 — forked from ruanbekker/promtail_docker_logs.md
Docker Container Logging using Promtail
@enixdark
enixdark / benchmark
Created September 20, 2022 12:40 — forked from aifeiasdf/benchmark
benchmark-apisix with envoy
对比 Apache APISIX 和 envoy 的一些性能参数:网关延迟、QPS 和是否线性扩展
系统环境:微软云 Linux (ubuntu 18.04), Standard D13 v2 (8 vcpus, 56 GiB memory)
压力测试客户端:wrk
@enixdark
enixdark / elasticsearch.rake
Created September 5, 2022 18:22 — forked from tlikai/elasticsearch.rake
Rebuild elasticsearch index with zero downtime
require 'elasticsearch/rails/tasks/import'
namespace :elasticsearch do
desc 'Rebuild index with zerodown time'
task rebuild: :environment do
size = (ENV['size'].presence || 1000).to_i
model = ENV['model'].to_s.classify.constantize
delete_old_index = ENV['delete'].present?
es = model.__elasticsearch__
@enixdark
enixdark / zero_downtime_reindexing.py
Created September 5, 2022 18:01 — forked from ikeikeikeike/zero_downtime_reindexing.py
Elasticsearch Zero Downtime Reindexing using elasticsearch-dsl-py ref: https://www.elastic.co/blog/changing-mapping-with-zero-downtime
from datetime import datetime
from elasticsearch_dsl import (
DocType,
String,
Integer,
Float
)
def _suffix():
@enixdark
enixdark / Dockerfile
Created August 7, 2022 15:47 — forked from AlwxSin/Dockerfile
Sentry docker-compose with telegram integration
FROM sentry:onbuild
@enixdark
enixdark / 01-trace.md
Created July 17, 2022 11:09 — forked from anfernee/01-trace.md
Kernel-Note
# First rule SNAT the packet that comes out of the gw node
-A CILIUM_POST_nat -s 10.11.1.0/24 ! -d 10.11.0.0/16 ! -o cilium_+ -m comment --comment "cilium masquerade non-cluster" -j MASQUERADE

-A CILIUM_POST_nat ! -o cilium_host -m comment --comment "exclude non-cilium_host traffic from masquerade" -j RETURN
-A CILIUM_POST_nat -m mark --mark 0xa00/0xe00 -m comment --comment "exclude proxy return traffic from masquarade" -j ACCEPT
-A CILIUM_POST_nat ! -s 10.11.1.253/32 ! -d 10.11.1.0/24 -o cilium_host -m comment --comment "cilium host->cluster masquerade" -j SNAT --to-source 10.11.1.253
-A CILIUM_POST_nat -s 127.0.0.1/32 -o cilium_host -m comment --comment "cilium host->cluster from 127.0.0.1 masquerade" -j SNAT --to-source 10.11.1.253
-A CILIUM_POST_nat -o cilium_host -m mark --mark 0xf00/0xf00 -m conntrack --ctstate DNAT -m comment --comment "hairpin traffic that originated from a local pod" -j SNAT --to-source 10.11.1.253
Node/Pod IP
Source Pod 10.11.1.103
Source Node 192.168.11.12
Gateway Node 192.168.11.11
External Service 192.168.11.13

iptables

Added the following on both source and gateway node. Egress gateway datapath still works.

vagrant@k8s1:~/go/src/github.com/cilium/cilium$ sudo tcpdump -i enp0s8 icmp6 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes




18:11:10.397859 IP6 k8s2 > ff02::1:ff00:2c69: ICMP6, neighbor solicitation, who has f00d::a0f:0:0:2c69, length 32
18:11:11.478191 IP6 k8s2 > ff02::1:ff00:2c69: ICMP6, neighbor solicitation, who has f00d::a0f:0:0:2c69, length 32