DPDK resources

dpdk_resources.md

• DPDK Documentation
• DPDK repositories
• Building DPDK
• Understanding DPDK [SLIDES]
• Introduction to DPDK: Architecture and Principles
• Learning DPDK : Huge pages
• How do I check for hugepages usage and what is using it?
• Use DPDK to send packets between platforms
• Testing DPDK Performance and Features with TestPMD
• NVIDIA Mellanox Bluefield-2 SmartNIC Hands-On Tutorial: “Rig for Dive” — Part VI: DPDK Performance
• Building a high performance — Linux Based Traffic generator with DPDK
• [Video] Run DPDK testpmd application to send tens millions of packet
• Performance Tuning for Mellanox Adapters
• Ethtool Commands
• Linux Kernel vs DPDK: HTTP Performance Showdown
• Mellanox OFED cheat sheet
• Measuring Round Trip Time using DPDK
• Building DPDK application
• Selectively init DPDK ports
• rte_eth_tx_burst() descriptor/mbuf management guarantees vs. free thresholds
• Are the DPDK functions rte_pktmbuf_alloc and rte_pktmbuf_free thread safe?
• Running multiple threads on same DPDK lcore
• Memory in DPDK, Part 1: General Concepts
• Memory in DPDK Part 2: Deep Dive into IOVA
• DPDK Deep Dive [VIDEO]
• Kernel-bypass techniques for high-speed network packet processing [VIDEO]
• Exploring new DPDK memory subsystem
• Learning DPDK : Symmetric RSS
• Multiple queue and RSS in DPDK [Chinese]
• [SO] DPDK - RSS Offloads (Hash Calculations): ETH_RSS_IPV6_EX - Explanation
• Scaling in the Linux Networking Stack
• DPDK Design Tips (Part 1 - RSS)
• Scaling in the Linux Networking Stack
• Queues, RSS, interrupts and cores
• Building a high performance — Linux Based Traffic generator with DPDK
• How to generate RANDOM traffic via Pktgen-DPDK?
• Tuning the buffers: a practical guide toreduce or avoid packet loss in DPDK applications
• How memory types affect DPDK application performance—case study
• How to generate 100Mpps traffic of 64B packet size with Pktgen?
• dpdk-pktgen: How to change rate of packet transmission?
• [dpdk-users] PKTGEN option to limit sending packets per second
• DPDK 18.11 flow director API
• HW Timestamping example

Issues

• [dpdk-users] Linking to multiple DPDK .so builds
• DPDK program initialization encounters "tailq is already registered" error
• Build and install DPDK and Pktgen-DPDK
• [SO] How debug DPDK with GDB
• Writing a PMD for DPDK
• Understanding mlx5 ethtool Counters
• rte flow rule not clear with DPDK 21.11 and Intel E810
• [dpdk-users] multi core performance problem
• What does "cacheline aligned" mean?
• [dpdk-dev] Random numbers at line-rate
• Seed in multicore

MLX Specific

• NVIDIA MLX5 Common Driver
• NVIDIA MLX5 Ethernet Driver

Using DPDK with Mellanox devices

1. Install mellanox ofed driver [REQUIRED].
2. Compile DPDK
3. Done!

Refs:

• Using mellanox NICs with DPDK

See devices:

$ mst status -s

Find the DPDK port number attached to a specific PCIE:

• Use: testpmd -l 8-15 -n 4 -b -- --rxq=2 --txq=2

Port number are probed based on physical PCIE address: https://stackoverflow.com/a/68457392

Handy tools

• ethtool - query and change device configuration.
• mst - Mellanox NIC too.
• mlxconfig - Configure mellanox devices.

Generating packets with packetgen

• Installing pktgen

• Generating packets

sudo ./usr/local/bin/pktgen -n 1 -b 0000:07:00.0 -- -T -P -m 1.1 -f send_pktgen.pkt

send_ptkgen.pkt

stop 1
set 1 proto udp
set 1 sport 1212
set 1 dport 2323
set 1 src mac 9c:dc:71:5d:90:e1
set 1 dst mac 9c:dc:71:56:fd:d5
set 1 src ip 10.10.1.1/24
set 1 dst ip 10.10.1.2
set 1 size 64

Using port 1 to send packets.

• Receiving packets

sudo ./usr/local/bin/pktgen -n 1 -b 0000:07:00.0 -- -T -P -m 1.1 -f -f recv_pktgen.pkt

recv_pktgen.pkt

stop 1
set 1 proto udp
set 1 dport 1212
set 1 sport 2323
set 1 src mac 9c:dc:71:56:fd:d5
set 1 dst mac 9c:dc:71:5d:90:e1
set 1 src ip 10.10.1.2
set 1 dst ip 10.10.1.1/24
set 1 size 64

Receive packets from port 1

Adjust the dpdk port, ip and macs.

issue: Can't receive packets more that 500 MBit/s. Improved by increasing packet size. With 1500 byte packets got 13GBit/s. Increased all the way to (max MTU supported by NIC)/2 which is 32Kbytes. Still same results.

Configuring Mellanox NICs with DPDK:

• Install Mellanox OFED driver
• Install DPDK
• Find out the DPDK port number for the PCIE device using testpmd by adding the control port with -b option.

Refs: https://dpdk.readthedocs.io/en/v17.11/nics/mlx5.html

How to find out the DPDK port number mapped to an interface (Needed for bifurcated drivers)

Run testpmd with -b option to specifying the control link PCIE address.

set 0 rate 50 - 50% of the link rate
set 0 count 50 - Send 50 packets
set 0 seq_cnt 2 - This will generate 2 different sequence of packets.
sequence 0 0 dst 02:00:00:00:00:00 src b8:00:00:00:00:00 dst 10.10.1.2 src 10.10.1.1/24 sport 1234 dport 5656 ipv4 udp vlan 0 size 64 - Configure sequence 0
sequence 1 0 dst 02:00:00:00:00:00 src b8:00:00:00:00:00 dst 10.10.1.2 src 10.10.1.1/24 sport 1235 dport 5656 ipv4 udp vlan 0 size 64 - Configure sequence 1 [Only source port is different in this case]