Andre Fredette anfredette

Multiple Active Gateways

Summary

Submariner currently only allows a single gateway to be active at any one time in a given cluster. This enhancement proposes that there be an option to allow more than one gateway to be active in a given cluster to enhance both performance and resiliency. This feature is sometimes referred to as “active-active gateways”; however, the goal is to allow two or more active gateways to be used.

Proposal

tc-bpf behavior

tc is a utility provided by iproute2 that can be used to interact with the Linux Traffic Control (TC) Subsystem. Support was added in kernel version 4.1 to allow BPF programs to be loaded as TC classifiers (or filters) and actions.

This document describes testing done on the behavior of adding and removing BPF programs using the tc-bpf tool.

The main goal was to understand how tc-bpf behaves in order to determine whether it could be added as an option for bpfd. The specific initial goals documented here were to understand how BPF programs can be added and deleted, and also how to control the order of execution of multiple programs added to the same interface and qdisc.

Setup

Description

Below is the output from running the following commands using ns-watch test code.

sudo ip netns add n1
sudo ip link add dev br0 type bridge
sudo ip link set br0 up
sudo ip netns exec n1 ip link add name vethin type veth peer name vethout
sudo ip netns exec n1 ip link set vethout netns 1

This the kubectl output from the initial implementation of the new bpfman-operator design intended to support the load/attach split.

The code currently only supports cluster-scoped XDP programs.

Add a program:

$ kubectl apply -f /home/afredette/go/src/github.com/bpfman/bpfman-operator/config/samples/bpfman.io_v1alpha1_bpfapplication.yaml
bpfapplication.bpfman.io/bpfapplication-sample created

// The [KubeBuilder documentation]
// (https://book.kubebuilder.io/reference/markers/crd-validation.html?highlight=required#crd-validation)
// states that if you include the kubebuilder:validation:Required marker at the
// packate level as shown below, all fields in the package are required unless
// explicitly marked optional.  However, it doesn't seem to make a difference.
// Without it, all fields that aren't explicitly marked as optional are treated
// as required anyway.
// +kubebuilder:validation:Required
package v1alpha1

	[[local\|localrc]]
	# put the log files in a dir different than the source so they can be manipulated independently
	LOGFILE=/opt/stack/logs/stack.sh.log
	SCREEN_LOGDIR=/opt/stack/logs
	LOG_COLOR=False
	# flip OFFLINE and RECLONE to lock (RECLONE=no) or update the source.
	#OFFLINE=False
	#RECLONE=yes
	OFFLINE=True
	RECLONE=no

	[[local\|localrc]]
	# put the log files in a dir different than the source so they can be manipulated independently
	LOGFILE=/opt/stack/logs/stack.sh.log
	SCREEN_LOGDIR=/opt/stack/logs
	LOG_COLOR=False
	# flip OFFLINE and RECLONE to lock (RECLONE=no) or update the source.
	#OFFLINE=False
	#RECLONE=yes
	OFFLINE=True
	RECLONE=no

	OFPST_FLOW reply (OF1.3) (xid=0x2):
	cookie=0x8000000, duration=1206.948s, table=0, n_packets=393, n_bytes=34491, priority=4,in_port=2 actions=load:0->NXM_OF_IN_PORT[],write_metadata:0x10000000000/0xffffff0000000001,goto_table:17
	cookie=0x8000000, duration=1195.219s, table=0, n_packets=871, n_bytes=139660, priority=4,in_port=3 actions=load:0->NXM_OF_IN_PORT[],write_metadata:0x20000000000/0xffffff0000000001,goto_table:17
	cookie=0x8000001, duration=1174.195s, table=0, n_packets=461713, n_bytes=33360031, priority=5,in_port=4 actions=write_metadata:0x50000000001/0x1fffff0000000001,goto_table:36
	cookie=0x8000000, duration=1156.113s, table=0, n_packets=12, n_bytes=2098, priority=4,in_port=5 actions=load:0->NXM_OF_IN_PORT[],write_metadata:0x60000000000/0xffffff0000000001,goto_table:17
	cookie=0x8000000, duration=1144.734s, table=0, n_packets=32, n_bytes=2264, priority=4,in_port=6 actions=load:0->NXM_OF_IN_PORT[],write_metadata:0x70000000000/0xffffff0000000001,goto_table:17
	cookie=0x8000000, duration=1092.449s, t

	OFPST_FLOW reply (OF1.3) (xid=0x2):
	cookie=0x8000000, duration=1267.670s, table=0, n_packets=32, n_bytes=2376, priority=4,in_port=2 actions=load:0->NXM_OF_IN_PORT[],write_metadata:0x30000000000/0xffffff0000000001,goto_table:17
	cookie=0x8000001, duration=1266.256s, table=0, n_packets=1943, n_bytes=135606, priority=5,in_port=3 actions=write_metadata:0x40000000001/0x1fffff0000000001,goto_table:36
	cookie=0x8000000, duration=1217.676s, table=0, n_packets=29, n_bytes=2418, priority=4,in_port=4 actions=load:0->NXM_OF_IN_PORT[],write_metadata:0x80000000000/0xffffff0000000001,goto_table:17
	cookie=0x8000000, duration=1183.966s, table=0, n_packets=6422877, n_bytes=518620067, priority=1,in_port=1 actions=load:0->NXM_OF_IN_PORT[],write_metadata:0x90000000001/0xffffff0000000001,goto_table:17
	cookie=0x8040000, duration=1266.386s, table=17, n_packets=32, n_bytes=2376, priority=6,metadata=0xc000030000000000/0xffffff0000000000 actions=write_metadata:0xe000031388000000/0xfffffffffffffffe,goto_table:50
	cookie=0x8040000, du

	jordan:~/go/src/github.com/submariner-io/submariner-operator$ make deploy using=lighthouse DEPLOY_ARGS='--globalnet --cable_driver vxlan'
	docker network create -d bridge kind
	Error response from daemon: network with name kind already exists
	make: [Makefile.dapper:42: deploy] Error 1 (ignored)
	./.dapper -- make --debug=b deploy
	Sending build context to Docker daemon 498.2MB
	Step 1/8 : ARG BASE_BRANCH
	Step 2/8 : FROM quay.io/submariner/shipyard-dapper-base:${BASE_BRANCH}
	---> ee8080f4e530
	Step 3/8 : ARG PROJECT