Here we present the steps taken for setting up a test that uses both
the ETF qdisc and the TAPRIO one. That means that we'll use a (Qbv-like)
port scheduler with a fixed Tx schedule for traffic classes (TC), while
using Time-based transmission for controlling the Tx time of packets within
each TC.
The 'talker' side of the example described below will have 2 applications
transmitting time-sensitive traffic following a strict cyclic schedule.
In addition to that, iperf3 is used to transmit best-effort traffic on the
port. The port schedule is thus comprised by 3 time-slices, with a total
cycle-time of 1 millisecond allocated as:
- Traffic Class 0 (TC 0): duration of 300us, 'strict txtime' is used.
- Traffic Class 1 (TC 1): duration of 300us, 'deadline txtime' is used.
- Traffic Class 2 (TC 2): duration of 400us, best-effort traffic.
The system is configured so the application enqueueing packets for
TC 0 will set its packets *Tx time* with an offset of 250us within its
time-slice.
The application enqueueing packets for TC 1 will set its packets *deadline*
with an offset of 250us within its time-slice. However, because this TC is
using the deadline mode of SO_TXTIME + etf, then a packet maybe transmitted
at anytime within its time slice that is before its deadline.
Best-effort traffic is transmitted at anytime during the third time slice.
A away to visualize this cycle and its time-slices is:
|______x_|......D_|bbbbbbbbbb|
0 299 599 999us
The application for each time-sensitive traffic class will transmit a packet
every 1ms. The packet's txtime is set through the SO_TXTIME api, and is
copied into the packet's payload.
At the 'listener' side, we capture traffic and then post-process it to
verify if packets are arriving outside of the time-slice they belong to.
ptp4l is used for synchronizing the PHC clocks over the network and
phc2sys is used on the 'talker' size for synchronizing the system
clock to the PHC.
CLOCK_TAI is the reference clockid used throughout the example for the
qdiscs and the applications.
# LISTENER #
1) Setup network
sudo ip addr add 192.168.0.78/4 broadcast 192.168.0.255 dev enp3s0
2) Start time sync (ptp master)
sudo ./setup_clock_sync.sh -i enp3s0 -m -v
3) Start iperf server
iperf3 -s
4) Prepare 'dump-classifier' files. Please refer to
README.classifier for further information.
--filters--
talker_strict :: udp port 7788
talker_deadline :: udp port 7798
--gates.sched--
S 0x1 300000
S 0x2 300000
S 0xc 400000
5) Start capturing traffic:
sudo tcpdump -c 600000 -i enp3s0 -w tmp.pcap -j adapter_unsynced \
-tt --time-stamp-precision=nano
6) Use the talkers to transmit packets as described on the next section.
6) After traffic was captured, check if packets arrived outside of their
time-slices. The base-time comes from the udp_tai for TC 0 minus the
250us txtime offset as used below. For example:
./dump-classifier -b 1528320726000000000 -d tmp.pcap -f filter \
-s gates.sched | grep -v ontime
# TALKER #
1) Setup network
sudo ip addr add 192.168.0.77/4 broadcast 192.168.0.255 dev enp3s0
2) Setup qdiscs
2.0) Prepare sched file for taprio
--gates.sched--
S 0x1 300000
S 0x2 300000
S 0xc 400000
2.1) Setup taprio with a base-time starting in 2min from now rounded down.
We must add the 37s UTC-TAI offset to the timestamp we get with 'date'.
i=$((`date +%s%N` + 37000000000 + (2 * 60 * 1000000000))) ; \
base=$(($i - ($i % 1000000000))) ; \
tc qdisc add dev enp3s0 parent root handle 100 taprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 queues 0 1 2 2 \
sched-file gates.sched base-time $base clockid CLOCK_TAI
We can read the above as:
- there are 3 traffic classes (num_tc 3);
- SO_PRIORITY value 3 maps to TC 0, while value 2 maps to TC 1.
Everything else maps to the other (best-effort) traffic classes;
- "queues 0 1 2 2" is a positional argument, meaning that TC 0 maps
to queue 0, TC 1 maps to queue 1 and TC 2 maps to queues 2 and 3.
- gates.sched is used as schedule file;
- the reference clock is CLOCK_TAI;
2.2) Setup etf for queue TC 0:
tc qdisc replace dev enp3s0 parent 100:1 etf clockid CLOCK_TAI \
delta 200000 offload
2.3) Setup etf in deadline mode for TC 1:
tc qdisc replace dev enp3s0 parent 100:2 etf clockid CLOCK_TAI \
delta 200000 offload deadline_mode
3) Start time sync (ptp slave):
sudo ./setup_clock_sync.sh -i enp3s0 -s -v
4) Start iperf3 client:
iperf3 -c 192.168.0.78 -t 600 --fq-rate 100M
5) Start udp_tai for TC 0. Use a base-time starting in 1min from now + a
250us offset for txtime:
now=`date +%s%N` ; i=$(($now + 37000000000 + (60 * 1000000000))) ; \
base=$(($i - ($i % 1000000000) + 250000)) ; \
sudo ./udp_tai -i enp3s0 -b $base -P 1000000 -t 3 -p 90 -d 600000 \
-u 7788
6) Start udp_tai in deadline mode for TC 1. Use the txtime computed for
the previous traffic class (above) and add 300us so it falls under the
second time slice (TC 1). For example, if the instance of udp_tai executed
on the previous step printed
"txtime of 1st packet is: 1528320726000250000", then the now we should do:
sudo ./udp_tai -i enp3s0 -t 2 -p 90 -D -d 600000 \
-b 1528320726000550000 -u 7798