X2522 benchmark, 2024-08-13

x2522.20240813.log

# One-way, overall
#
# 0.973/1.093/3.042us (0/50/99p, 256 byte packets)


# Port-to-port (switch hop)
#  
# 680+/-20ns
#
# - Mellanox Spectrum SN2700 w/ Cumulus 5.7
# - 200 byte packets
# - 1 Gbps packet rate
# 
#
$ ./nt_latency -t 2 -r 3 -x 1000 -s 200 -l 5
Rate: 1000Mbps, Packet Size: 200, Tx Port: 2, Rx Port: 3, running time: 5s
...
               Count             Bad             Min             Max             Avg            Mean          StdDev            Last
6.           2820408               0             694             712             698             698               1             701


# One-way, including switch hop
# 
# 1.653/1.773/3.722us (0/50/99, 256 byte packets)
#
# - AMD EPYC 7543P
# - AMD Xilinx XtremeScale X2522-25G-PLUS25 GbE
# - OpenOnload 8.0.0.34
# - host -> switch -> host
# - disable FEC, tasket/cpuset, interrupt steering, etc.
# - see https://www.reddit.com/r/highfreqtrading/comments/1eyrhso/lowlatency_tuning_guide/
#
#
 	size	mean	min	median	max	%ile	stddev	iter
	0	    1613	1487	1603	13821	1853	69	923000
	1	    1617	1492	1608	9418	1853	65	922000
	2	    1616	1497	1603	10630	1853	65	922000
	4	    1613	1512	1603	4298	1848	64	924000
	8	    1614	1487	1603	11717	1853	67	923000
	16	    1613	1507	1603	3977	1853	64	924000
	32	    1676	1537	1663	3707	1923	71	883000
	64	    1704	1568	1693	13010	1948	74	868000
	128	    1777	1643	1768	4052	2008	68	832000
	256	    1789	1653	1773	3722	2003	65	826000
	512	    2112	1983	2099	11983	2314	67	705000
	1024	2309	2179	2294	4318	2534	74	644000
	1472	2801	2655	2780	3907	3025	67	533000
	1473	9754	9197	9723	24266	10595	252	154000
	2048	9877	9207	9838	14567	10670	258	152000
	4096	11511	10650	11467	15895	12524	325	131000
	8192	16514	15154	16491	20925	17774	488	91000
	16384	25906	24031	25879	40282	27402	598	58000
	32768	40886	38488	40828	46213	43107	785	37000