Contains notes about the U200_100G_ether_duplex design transmitting bad Ethernet packets using the CMAC, discovered running the cmac_loopback_test program.
Related:
- 000034731 - AXI4-Stream Data FIFO - Data is not always output in packet mode for short 2 beat packets -Dropped packet seen with 100G Ethernet IP describes the symptoms.
- Determining AXI4-stream Data FIFO size, understanding Packet Mode? query about packet mode
- https://electronics.stackexchange.com/q/511965/101063 question about the xpm_vhdl library.
- Where in Vivado to check for module parameter/generics and status of hard blocks (RAMs, DSPs) how to check what parameters have been used.
- XPM_FIFO_AXIS documentation for Parameterized Macro: AXI Stream FIFO
The following is an example of the test passing:
$ cmac_ethernet/cmac_loopback_test -n 1:0
Link ready
Testing U200_100G_ether_duplex Tx port 1 Rx Port 0 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 1 statistics (over 0.045 secs):
CYCLE_COUNT : 14566402
TX_TOTAL_PACKETS : 9537
TX_TOTAL_GOOD_PACKETS : 9537
TX_TOTAL_BYTES : 46082784
TX_TOTAL_GOOD_BYTES : 46082936
TX_PACKET_65_127_BYTES : 63
TX_PACKET_128_255_BYTES : 129
TX_PACKET_256_511_BYTES : 256
TX_PACKET_512_1023_BYTES : 512
TX_PACKET_1024_1518_BYTES : 495
TX_PACKET_1519_1522_BYTES : 4
TX_PACKET_1523_1548_BYTES : 26
TX_PACKET_1549_2047_BYTES : 499
TX_PACKET_2048_4095_BYTES : 2048
TX_PACKET_4096_8191_BYTES : 4096
TX_PACKET_8192_9215_BYTES : 1024
TX_PACKET_LARGE : 385
TX_UNICAST : 9537
RX_RSFEC_CW_INC : 882812
U200_100G_ether_duplex port 0 statistics (over 0.045 secs):
CYCLE_COUNT : 14566329
RX_TOTAL_PACKETS : 9537
RX_TOTAL_GOOD_PACKETS : 9537
RX_TOTAL_BYTES : 46082784
RX_TOTAL_GOOD_BYTES : 46082784
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 63
RX_PACKET_128_255_BYTES : 128
RX_PACKET_256_511_BYTES : 256
RX_PACKET_512_1023_BYTES : 512
RX_PACKET_1024_1518_BYTES : 495
RX_PACKET_1519_1522_BYTES : 4
RX_PACKET_1523_1548_BYTES : 26
RX_PACKET_1549_2047_BYTES : 499
RX_PACKET_2048_4095_BYTES : 2048
RX_PACKET_4096_8191_BYTES : 4096
RX_PACKET_8192_9215_BYTES : 1024
RX_PACKET_LARGE : 385
RX_UNICAST : 9537
RX_RSFEC_CW_INC : 882808
Total byte including FCS: 46082784
Loopback test: PASS
For the rest of the output, only recorded when a test failed, since is intermittent. Haven't attempted to record the failure rate.
The initial design which reports as User access build timestamp : CA34F4E0 - 25/04/2026 15:19:32.
DAC cable between ports 0 and 1.
Port 1 to 0 can fail:
$ cmac_ethernet/cmac_loopback_test -n 1:0
Link ready
Testing U200_100G_ether_duplex Tx port 1 Rx Port 0 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 1 statistics (over 0.000 secs):
CYCLE_COUNT : 105883
TX_TOTAL_PACKETS : 18
TX_TOTAL_GOOD_PACKETS : 18
TX_TOTAL_BYTES : 1457
TX_TOTAL_GOOD_BYTES : 1305
TX_PACKET_64_BYTES : 1
TX_PACKET_65_127_BYTES : 17
TX_UNICAST : 18
RX_RSFEC_CW_INC : 6417
U200_100G_ether_duplex port 0 statistics (over 0.000 secs):
CYCLE_COUNT : 105819
RX_BAD_CODE : 9
RX_TOTAL_PACKETS : 19
RX_TOTAL_GOOD_PACKETS : 18
RX_TOTAL_BYTES : 1441
RX_TOTAL_GOOD_BYTES : 1305
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 17
RX_PACKET_128_255_BYTES : 1
RX_BAD_FCS : 1
RX_PACKET_BAD_FCS : 1
RX_UNICAST : 18
RX_RSFEC_CW_INC : 6413
Total byte including FCS: 1387
Loopback test: FAIL
0000:31:00.0 C2H channel 0 failure : Rx transfer_len=132, expected 78
Port 0 to 1 can fail:
$ cmac_ethernet/cmac_loopback_test -n 0:1
Link ready
Testing U200_100G_ether_duplex Tx port 0 Rx Port 1 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 0 statistics (over 0.000 secs):
CYCLE_COUNT : 111580
TX_TOTAL_PACKETS : 9
TX_TOTAL_GOOD_PACKETS : 9
TX_TOTAL_BYTES : 764
TX_TOTAL_GOOD_BYTES : 612
TX_PACKET_64_BYTES : 1
TX_PACKET_65_127_BYTES : 8
TX_UNICAST : 9
RX_RSFEC_CW_INC : 6762
U200_100G_ether_duplex port 1 statistics (over 0.000 secs):
CYCLE_COUNT : 111520
RX_BAD_CODE : 9
RX_TOTAL_PACKETS : 10
RX_TOTAL_GOOD_PACKETS : 9
RX_TOTAL_BYTES : 748
RX_TOTAL_GOOD_BYTES : 612
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 8
RX_PACKET_128_255_BYTES : 1
RX_BAD_FCS : 1
RX_PACKET_BAD_FCS : 1
RX_UNICAST : 9
RX_RSFEC_CW_INC : 6759
Total byte including FCS: 685
Loopback test: FAIL
0000:31:00.0 C2H channel 1 failure : Rx transfer_len=132, expected 69
This added a AXI Stream Switch between the XDMA and CMAC ports, to allow loopback between the CMAC ports, without needing to use the XDMA to send/receive packets from software. For these tests the AXI Stream Switch is configured with the defaults to connect the CMAC ports to XDMA.
The design reports as User access build timestamp : 4C34ACD9 - 09/08/2026 10:51:25
Port 1 to 0 can fail:
$ cmac_ethernet/cmac_loopback_test -n 1:0
Device 0000:31:00.0 design U200_100G_ether_duplex routes updated
Waiting to link to be ready to receive.
Link ready (latched rx_status initial 0x000000D8 last 0x00000003)
Testing U200_100G_ether_duplex Tx port 1 Rx Port 0 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 1 statistics (over 0.000 secs):
CYCLE_COUNT : 130248
TX_TOTAL_PACKETS : 22
TX_TOTAL_GOOD_PACKETS : 22
TX_TOTAL_BYTES : 1791
TX_TOTAL_GOOD_BYTES : 1639
TX_PACKET_64_BYTES : 1
TX_PACKET_65_127_BYTES : 21
TX_UNICAST : 22
RX_RSFEC_CW_INC : 7894
U200_100G_ether_duplex port 0 statistics (over 0.000 secs):
CYCLE_COUNT : 130240
RX_BAD_CODE : 9
RX_TOTAL_PACKETS : 23
RX_TOTAL_GOOD_PACKETS : 22
RX_TOTAL_BYTES : 1775
RX_TOTAL_GOOD_BYTES : 1639
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 21
RX_PACKET_128_255_BYTES : 1
RX_BAD_FCS : 1
RX_PACKET_BAD_FCS : 1
RX_UNICAST : 22
RX_RSFEC_CW_INC : 7894
Total byte including FCS: 1725
Loopback test: FAIL
0000:31:00.0 C2H channel 0 failure : Rx transfer_len=132, expected 82
Port 0 to 1 can fail:
$ cmac_ethernet/cmac_loopback_test -n 0:1
Waiting to link to be ready to receive.
Link ready (latched rx_status initial 0x000000C8 last 0x00000003)
Testing U200_100G_ether_duplex Tx port 0 Rx Port 1 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 0 statistics (over 0.000 secs):
CYCLE_COUNT : 117883
TX_TOTAL_PACKETS : 16
TX_TOTAL_GOOD_PACKETS : 16
TX_TOTAL_BYTES : 1296
TX_TOTAL_GOOD_BYTES : 1144
TX_PACKET_64_BYTES : 1
TX_PACKET_65_127_BYTES : 15
TX_UNICAST : 16
RX_RSFEC_CW_INC : 7144
U200_100G_ether_duplex port 1 statistics (over 0.000 secs):
CYCLE_COUNT : 117678
RX_BAD_CODE : 9
RX_TOTAL_PACKETS : 17
RX_TOTAL_GOOD_PACKETS : 16
RX_TOTAL_BYTES : 1280
RX_TOTAL_GOOD_BYTES : 1144
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 15
RX_PACKET_128_255_BYTES : 1
RX_BAD_FCS : 1
RX_PACKET_BAD_FCS : 1
RX_UNICAST : 16
RX_RSFEC_CW_INC : 7132
Total byte including FCS: 1224
Loopback test: FAIL
0000:31:00.0 C2H channel 1 failure : Rx transfer_len=132, expected 76
This added a DRP bridge to allow access to the CMAC RDP interfaces (e.g. allow reading or changing the min/max receive packet lengths).
The design reports as User access build timestamp : 8434D48A - 16/08/2026 13:18:10
Port 1 to 0 can fail:
$ cmac_ethernet/cmac_loopback_test -n 0:1
Link ready
Testing U200_100G_ether_duplex Tx port 0 Rx Port 1 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 0 statistics (over 0.000 secs):
CYCLE_COUNT : 127516
TX_TOTAL_PACKETS : 30
TX_TOTAL_GOOD_PACKETS : 30
TX_TOTAL_BYTES : 2507
TX_TOTAL_GOOD_BYTES : 2355
TX_PACKET_64_BYTES : 1
TX_PACKET_65_127_BYTES : 29
TX_UNICAST : 30
RX_RSFEC_CW_INC : 7728
U200_100G_ether_duplex port 1 statistics (over 0.000 secs):
CYCLE_COUNT : 127454
RX_BAD_CODE : 9
RX_TOTAL_PACKETS : 31
RX_TOTAL_GOOD_PACKETS : 30
RX_TOTAL_BYTES : 2491
RX_TOTAL_GOOD_BYTES : 2355
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 29
RX_PACKET_128_255_BYTES : 1
RX_BAD_FCS : 1
RX_PACKET_BAD_FCS : 1
RX_UNICAST : 30
RX_RSFEC_CW_INC : 7725
Total byte including FCS: 2449
Loopback test: FAIL
0000:31:00.0 C2H channel 1 failure : Rx transfer_len=132, expected 90
Port 1 to 0 can fail:
$ cmac_ethernet/cmac_loopback_test -n 1:0
Waiting to link to be ready to receive.
Link ready (latched rx_status initial 0x000000C8 last 0x00000003)
Testing U200_100G_ether_duplex Tx port 1 Rx Port 0 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 1 statistics (over 0.000 secs):
CYCLE_COUNT : 135307
TX_TOTAL_PACKETS : 26
TX_TOTAL_GOOD_PACKETS : 26
TX_TOTAL_BYTES : 2141
TX_TOTAL_GOOD_BYTES : 1989
TX_PACKET_64_BYTES : 1
TX_PACKET_65_127_BYTES : 25
TX_UNICAST : 26
RX_RSFEC_CW_INC : 8201
U200_100G_ether_duplex port 0 statistics (over 0.000 secs):
CYCLE_COUNT : 135120
RX_BAD_CODE : 9
RX_TOTAL_PACKETS : 27
RX_TOTAL_GOOD_PACKETS : 26
RX_TOTAL_BYTES : 2125
RX_TOTAL_GOOD_BYTES : 1989
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 25
RX_PACKET_128_255_BYTES : 1
RX_BAD_FCS : 1
RX_PACKET_BAD_FCS : 1
RX_UNICAST : 26
RX_RSFEC_CW_INC : 8189
Total byte including FCS: 2079
Loopback test: FAIL
0000:31:00.0 C2H channel 0 failure : Rx transfer_len=132, expected 86
A System ILA was added to monitor the AXI stream transmit to the CMAC, and triggered on either the cmac_usplus_1_tx_ovfout or cmac_usplus_1_tx_unfout errors from the CMAC. This design identifies as User access build timestamp : B4356EB2 - 22/08/2026 22:58:50
Port 1 to 0 can fail, albeit seems to happen less often with this design:
$ cmac_ethernet/cmac_loopback_test -n 1:0
Link ready
Testing U200_100G_ether_duplex Tx port 1 Rx Port 0 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 1 statistics (over 0.001 secs):
CYCLE_COUNT : 172225
TX_TOTAL_PACKETS : 65
TX_TOTAL_GOOD_PACKETS : 65
TX_TOTAL_BYTES : 6392
TX_TOTAL_GOOD_BYTES : 6240
TX_PACKET_64_BYTES : 1
TX_PACKET_65_127_BYTES : 63
TX_PACKET_128_255_BYTES : 1
TX_UNICAST : 65
RX_RSFEC_CW_INC : 10438
U200_100G_ether_duplex port 0 statistics (over 0.001 secs):
CYCLE_COUNT : 172158
RX_BAD_CODE : 9
RX_TOTAL_PACKETS : 66
RX_TOTAL_GOOD_PACKETS : 65
RX_TOTAL_BYTES : 6376
RX_TOTAL_GOOD_BYTES : 6240
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 63
RX_PACKET_128_255_BYTES : 2
RX_BAD_FCS : 1
RX_PACKET_BAD_FCS : 1
RX_UNICAST : 65
RX_RSFEC_CW_INC : 10434
Total byte including FCS: 6369
Loopback test: FAIL
0000:31:00.0 C2H channel 0 failure : Rx transfer_len=132, expected 125
The ILA capture when the above failure happened:
The ILA capture shows:
- The CMAC reported a transmit underflow error.
- The AXI stream has a packet with two beats, with one clock cycle between the beats. That explains why the CMAC reported a transmit underflow.
- Looking at the TKEEP shows 125 bytes, which matches the expected value reported for the failed test.
Port 0 to 1 can fail:
$ cmac_ethernet/cmac_loopback_test -n 0:1
Link ready
Testing U200_100G_ether_duplex Tx port 0 Rx Port 1 with 9537 packet lengths (including FCS) from 64 to 9600 bytes
U200_100G_ether_duplex port 0 statistics (over 0.000 secs):
CYCLE_COUNT : 94481
TX_TOTAL_PACKETS : 5
TX_TOTAL_GOOD_PACKETS : 5
TX_TOTAL_BYTES : 482
TX_TOTAL_GOOD_BYTES : 330
TX_PACKET_64_BYTES : 1
TX_PACKET_65_127_BYTES : 4
TX_UNICAST : 5
RX_RSFEC_CW_INC : 5726
U200_100G_ether_duplex port 1 statistics (over 0.000 secs):
CYCLE_COUNT : 94420
RX_BAD_CODE : 9
RX_TOTAL_PACKETS : 6
RX_TOTAL_GOOD_PACKETS : 5
RX_TOTAL_BYTES : 466
RX_TOTAL_GOOD_BYTES : 330
RX_PACKET_64_BYTES : 1
RX_PACKET_65_127_BYTES : 4
RX_PACKET_128_255_BYTES : 1
RX_BAD_FCS : 1
RX_PACKET_BAD_FCS : 1
RX_UNICAST : 5
RX_RSFEC_CW_INC : 5722
Total byte including FCS: 399
Loopback test: FAIL
0000:31:00.0 C2H channel 1 failure : Rx transfer_len=132, expected 65
The ILA capture when the above failure happened:
The ILA capture shows:
- The CMAC reported a transmit underflow error.
- The AXI stream has a packet with two beats, with one clock cycle between the beats. That explains why the CMAC reported a transmit underflow.
- Looking at the TKEEP shows 65 bytes, which matches the expected value reported for the failed test.
The actual packet length on failure seems to be between 65 and 128 bytes, albeit haven't keep a note of every failure. When checked on the System ILA have allways seen:
- TKEEP matching the expected value reported by the loopback test.
- Two beats in a packet, with a clock cycle between them.
- Transmit underflow reported by the CMAC.
This does match the issue in 000034731 - AXI4-Stream Data FIFO - Data is not always output in packet mode for short 2 beat packets -Dropped packet seen with 100G Ethernet IP:
Description
When the AXI4-Stream Data FIFO is configured to be both asynchronous and in packet mode, short two beat packets can occasionally be output with a clock cycle gap between the beats.
If used with a block that requires data to be presented as full packet without any gap such as the 100G Ethernet IP (CMAC), this can cause the packet to be corrupted.
And the occasionally matches the observed intermittent failure.
The AXI-4-Stream Data FIFO's used for the CMAC transmit are configured using the following. They were configured in the block diagram, and this shows what was written by write_project_tcl:
# Create instance: axis_data_fifo_0, and set properties
set axis_data_fifo_0 [ create_bd_cell -type ip -vlnv xilinx.com:ip:axis_data_fifo:2.0 axis_data_fifo_0 ]
set_property -dict [list \
CONFIG.FIFO_DEPTH {1024} \
CONFIG.FIFO_MODE {2} \
CONFIG.IS_ACLK_ASYNC {1} \
] $axis_data_fifo_0
# Create instance: axis_data_fifo_1, and set properties
set axis_data_fifo_1 [ create_bd_cell -type ip -vlnv xilinx.com:ip:axis_data_fifo:2.0 axis_data_fifo_1 ]
set_property -dict [list \
CONFIG.FIFO_DEPTH {1024} \
CONFIG.FIFO_MODE {2} \
CONFIG.IS_ACLK_ASYNC {1} \
] $axis_data_fifo_1While from the product guide for AXI4-Stream Infrastructure IP Suite LogiCORE IP Product Guide (PG085) can't seem to find a definition of FIFO_MODE, the ipshared/e33d/hdl/axis_data_fifo_v2_0_rfs.v file has:
// +---------------------------------------------------------------------------------------------------------------------+
// | PACKET_FIFO | String | Allowed values: false, true. Default value = false. |
// |---------------------------------------------------------------------------------------------------------------------|
// | |
// | "true"- Enables Packet FIFO mode |
// | "false"- Disables Packet FIFO mode |
localparam LP_PACKET_FIFO = C_FIFO_MODE == 2 ? "true" : "false";Which confirms a FIFO_MODE of 2 is Packet FIFO Mode
The following is all properties reported for the AXIS Data FIFO for the CMAC port 0 transmit:
report_property -all -verbose [get_ips U200_100G_ether_duplex_axis_data_fifo_0_0]
Property Type Read-only Value
ALLOWED_SIM_MODELS string* true rtl
BASE_BOARD_PART string true
BOARD string true
BOARD_CONNECTIONS string true
CLASS string true ip
CONFIG.ACLKEN_CONV_MODE string false 0
CONFIG.Component_Name string true U200_100G_ether_duplex_axis_data_fifo_0_0
CONFIG.ENABLE_ECC string false 0
CONFIG.FIFO_DEPTH string false 1024
CONFIG.FIFO_MEMORY_TYPE string false auto
CONFIG.FIFO_MODE string false 2
CONFIG.HAS_AEMPTY string false 0
CONFIG.HAS_AFULL string false 0
CONFIG.HAS_ECC_ERR_INJECT string false 0
CONFIG.HAS_PROG_EMPTY string false 0
CONFIG.HAS_PROG_FULL string false 0
CONFIG.HAS_RD_DATA_COUNT string false 0
CONFIG.HAS_TKEEP string false 1
CONFIG.HAS_TLAST string false 1
CONFIG.HAS_TREADY string false 1
CONFIG.HAS_TSTRB string false 0
CONFIG.HAS_WR_DATA_COUNT string false 0
CONFIG.IS_ACLK_ASYNC string false 1
CONFIG.M_AXIS.INSERT_VIP string false 0
CONFIG.M_CLKIF.FREQ_HZ string false 322265625
CONFIG.M_CLKIF.INSERT_VIP string false 0
CONFIG.PROG_EMPTY_THRESH string false 5
CONFIG.PROG_FULL_THRESH string false 11
CONFIG.SYNCHRONIZATION_STAGES string false 3
CONFIG.S_AXIS.INSERT_VIP string false 0
CONFIG.S_CLKIF.FREQ_HZ string false 250000000
CONFIG.S_CLKIF.INSERT_VIP string false 0
CONFIG.S_RSTIF.INSERT_VIP string false 0
CONFIG.TDATA_NUM_BYTES string false 64
CONFIG.TDEST_WIDTH string false 0
CONFIG.TID_WIDTH string false 0
CONFIG.TUSER_WIDTH string false 1
CORE_REVISION long true 18
DCP_RESOURCE_DATA string true LUT 206 LUTAsLogic 206 LUTAsMem 0 REG 389 CARRY8 4 F7MUX 0 F8MUX 0 F9MUX 0 BRAM 16 URAM 0 DSP 0 BUFGCE 0 BUFGCE_DIV 0 BUFG_GT 0 BUFGCTRL* 0 PLL 0 MMCM 0 GTYE4_CHANNEL 0 GTYE4_COMMON 0 SLL_SLR1-SLR2 0 SLL_SLR2-SLR1 0 SLL_SLR0-SLR1 0 SLL_SLR1-SLR0 0
DELIVERED_TARGETS string* true synthesis simulation
DESIGN_TOOL_CONTEXTS string* true HDL IPI Sysgen
IPDEF string true xilinx.com:ip:axis_data_fifo:2.0
IP_CORE_CONTAINER string true
IP_DIR string true /home/mr_halfword/fpga_sio/fpga_tests/U200_100G_ether_duplex/U200_100G_ether_duplex/U200_100G_ether_duplex.srcs/sources_1/bd/U200_100G_ether_duplex/ip/U200_100G_ether_duplex_axis_data_fifo_0_0
IP_FILE string true /home/mr_halfword/fpga_sio/fpga_tests/U200_100G_ether_duplex/U200_100G_ether_duplex/U200_100G_ether_duplex.srcs/sources_1/bd/U200_100G_ether_duplex/ip/U200_100G_ether_duplex_axis_data_fifo_0_0/U200_100G_ether_duplex_axis_data_fifo_0_0.xci
IS_BD_CONTEXT bool true 1
IS_BOARD_AWARE bool true 0
IS_LOCKED bool true 0
KNOWN_TARGETS string* true instantiation_template synthesis simulation example changelog
LOCK_DETAILS string* true
NAME string true U200_100G_ether_duplex_axis_data_fifo_0_0
PART string true xcu200-fsgd2104-2-e
REQUIRES_VIP bool true
SCOPE string true /home/mr_halfword/fpga_sio/fpga_tests/U200_100G_ether_duplex/U200_100G_ether_duplex/U200_100G_ether_duplex.srcs/sources_1/bd/U200_100G_ether_duplex/U200_100G_ether_duplex.bd
SELECTED_SIM_MODEL enum false rtl
STALE_TARGETS string* true
SUPPORTED_TARGETS string* true instantiation_template synthesis simulation example changelog
SUPPORTS_MODREF bool true 1
SW_VERSION string true 2026.1
UNSUPPORTED_SIMULATORS string* true
UPGRADE_RESULT string true
UPGRADE_VERSIONS string* true
USED_LICENSE_KEYS string* true
USER_LOCKED bool false 0
There are no properties which appear to control the USE_ADV_FEATURES mentioned in 000034731 - AXI4-Stream Data FIFO - Data is not always output in packet mode for short 2 beat packets -Dropped packet seen with 100G Ethernet IP for the xpm_fifo
The /opt/Xilinx/2026.1/Vivado/data/ip/xilinx/axis_data_fifo_v2_0/xgui/axis_data_fifo.tcl has the following code to populate the C_USE_ADV_FEATURES parameter for the underlying xpm_fifo:
proc update_MODELPARAM_VALUE.C_USE_ADV_FEATURES {MODELPARAM_VALUE.C_USE_ADV_FEATURES \
PARAM_VALUE.HAS_PROG_FULL \
PARAM_VALUE.HAS_WR_DATA_COUNT \
PARAM_VALUE.HAS_AFULL \
PARAM_VALUE.HAS_PROG_EMPTY \
PARAM_VALUE.HAS_RD_DATA_COUNT \
PARAM_VALUE.HAS_AEMPTY \
PARAM_VALUE.FIFO_MODE \
} {
set has_prog_full [ get_property value ${PARAM_VALUE.HAS_PROG_FULL}]
set has_wr_data_count [ get_property value ${PARAM_VALUE.HAS_WR_DATA_COUNT}]
set has_afull [ get_property value ${PARAM_VALUE.HAS_AFULL}]
set has_prog_empty [ get_property value ${PARAM_VALUE.HAS_PROG_EMPTY}]
set has_rd_data_count [ get_property value ${PARAM_VALUE.HAS_RD_DATA_COUNT}]
set has_aempty [ get_property value ${PARAM_VALUE.HAS_AEMPTY}]
set fifo_mode [get_property value ${PARAM_VALUE.FIFO_MODE}]
# Force has_aempty, has_afull on when packet mode is enabled.
if {$fifo_mode == 2} {
set has_aempty 1
set has_afull 1
}
set val_0 [scan [format %X [expr {2*$has_prog_full+4*$has_wr_data_count+8*$has_afull}]] %c]
set val_1 [scan 0 %c]
set val_2 [scan [format %X [expr {2*$has_prog_empty+4*$has_rd_data_count+8*$has_aempty}]] %c]
set val_3 [scan 1 %c]
set val [format {0x%x%x%x%x} ${val_3} ${val_2} ${val_1} ${val_0}]
# puts "val is $val"
# send_msg_id axis_data_fifo-1 info "val is $val: [format %d $val]"
# set_property value $val ${MODELPARAM_VALUE.C_USE_ADV_FEATURES}
set_property value [format %d $val] ${MODELPARAM_VALUE.C_USE_ADV_FEATURES}
}Bit 13 is hard coded as zero.
Opened the elaborated design, and browsed the structure under axis_data_fifo_0:
- inst:
LP_USE_ADV_FEATURES(integer): 825765944
- gen_fifo.xpm_fifo_axis_inst:
EN_ADV_FEATURE_AXIS(string): 16'b0001100000001000EN_ADV_FEATURE_AXIS_INT(string): 16'b0001100000001000PKT_SIZE_LT8(string): 1'b0USE_ADV_FEATURES(integer): 825765944USE_ADV_FEATURES_INT(integer): 825765944
The USE_ADV_FEATURES value is 825765944 decimal which is 0x31383038 hexdecimal. This is the 4 ASCII characters 1808 which matches the EN_ADV_FEATURE_AXIS value of 16'b0001100000001000 binary. Looks like the decimal value of USE_ADV_FEATURES gets (implictly?) converted to a 32-bit vector. /opt/Xilinx/2026.1/data/ip/xpm/xpm_fifo/hdl/xpm_fifo.sv has the following code starting from line 2607 which converts the USE_ADV_FEATURES parameter into the fields which control the FIFO:
// Function to convert ASCII value to binary
function [3:0] str2bin;
input [7:0] str_val_ascii;
if((str_val_ascii == 8'h30) || (str_val_ascii == 8'h31) ||
(str_val_ascii == 8'h32) || (str_val_ascii == 8'h33) ||
(str_val_ascii == 8'h34) || (str_val_ascii == 8'h35) ||
(str_val_ascii == 8'h36) || (str_val_ascii == 8'h37) ||
(str_val_ascii == 8'h38) || (str_val_ascii == 8'h39) ||
(str_val_ascii == 8'h41) || (str_val_ascii == 8'h42) ||
(str_val_ascii == 8'h43) || (str_val_ascii == 8'h44) ||
(str_val_ascii == 8'h45) || (str_val_ascii == 8'h46) ||
(str_val_ascii == 8'h61) || (str_val_ascii == 8'h62) ||
(str_val_ascii == 8'h63) || (str_val_ascii == 8'h64) ||
(str_val_ascii == 8'h65) || (str_val_ascii == 8'h66) ||
(str_val_ascii == 8'h00)) begin
if (!str_val_ascii[6])
str2bin = str_val_ascii[3:0];
else begin
str2bin [3] = 1'b1;
str2bin [2] = str_val_ascii[2] | (str_val_ascii[1] & str_val_ascii[0]);
str2bin [1] = str_val_ascii[0] ^ str_val_ascii[1];
str2bin [0] = !str_val_ascii[0];
end
end
else
$error("Found Invalid character while parsing the string, please cross check the value specified for either READ_RESET_VALUE_A|B or MEMORY_INIT_PARAM (if initialization of memory through parameter is used). XPM_MEMORY supports strings (hex) that contains characters 0-9, A-F and a-f.");
endfunction
// Function that parses the complete reset value string
function logic [15:0] hstr2bin;
input [16*8-1 : 0] hstr_val;
integer rst_loop_a;
localparam integer rsta_loop_iter = 16;
logic [rsta_loop_iter-1 : 0] rst_val_conv_a_i;
for (rst_loop_a=1; rst_loop_a <= rsta_loop_iter/4; rst_loop_a = rst_loop_a+1) begin
rst_val_conv_a_i[(rst_loop_a*4)-1 -: 4] = str2bin(hstr_val[(rst_loop_a*8)-1 -: 8]);
end
return rst_val_conv_a_i[15:0];
endfunction
// <<snip>>
localparam [15:0] EN_ADV_FEATURE_AXIS = hstr2bin(USE_ADV_FEATURES);
localparam EN_ALMOST_FULL_INT = (PACKET_FIFO == "true") ? 1'b1 : EN_ADV_FEATURE_AXIS[3];
localparam EN_ALMOST_EMPTY_INT = (PACKET_FIFO == "true") ? 1'b1 : EN_ADV_FEATURE_AXIS[11];
localparam EN_DATA_VALID_INT = 1'b1;
localparam [15:0] EN_ADV_FEATURE_AXIS_INT = {EN_ADV_FEATURE_AXIS[15:13], EN_DATA_VALID_INT, EN_ALMOST_EMPTY_INT, EN_ADV_FEATURE_AXIS[10:4], EN_ALMOST_FULL_INT, EN_ADV_FEATURE_AXIS[2:0]};
localparam USE_ADV_FEATURES_INT = bin2hstr(EN_ADV_FEATURE_AXIS_INT);
localparam PKT_SIZE_LT8 = EN_ADV_FEATURE_AXIS[13];PKT_SIZE_LT8 having the value zero is what think causes packets with length 65..128 to sometimes get corrupted. Can't see any way in the IP integrator to cause PKT_SIZE_LT8 to be one.
The design is rev 02 plus a System ILA on each side of the transmit packet FIFOs, which identifies as User access build timestamp : E4352DB1 - 28/08/2026 18:54:49.
The initial implementation of cmac_switch_test, using a fixed frame size of 1518 bytes runs successfully when using a DAC cable between both ports, and running two instances of the test each a different direction:
[mr_halfword@haswell-alma release]$ cmac_ethernet/cmac_switch_test -n 1:0 -p 1-4 -l -r 100000 -s
Device 0000:31:00.0 design U200_100G_ether_duplex routes updated
Waiting to link to be ready to receive. Press Ctrl-C to abort.
Link ready
Waiting 2 seconds after link came up for switch to accept packets (latched rx_status initial 0x000000D8 last 0x00000003)
Bit rate on interface to injection switch = 100000 (Mbps)
Requested bit rate to be generated on each switch port under test = 100000.00 (Mbps)
Not limiting frame rate, as bit-rate on interface to injection switch doesn't exceed the total across all switch ports under test
Writing per-port counts to 20260829T155652_per_port_counts_linux.csv
Using design U200_100G_ether_duplex device 0000:31:00.0 Tx port 1 Rx port 0
Test interval = 10 (secs)
Frame debug enabled = No
Expect CMAC loopback = Yes
Disable CMAC port statistics = Yes
Press Ctrl-C to stop test at end of next test interval
15:57:04.740
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10836535 10836486 0 0 0 0 1083653.5 3341.99
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
15:57:14.740
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10659628 10659628 0 0 0 0 1065962.8 3287.43
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
<<snip>>>
16:10:14.740
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10483726 10483727 0 0 0 0 1048372.6 3233.18
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
^C
16:10:24.740
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10650830 10650829 0 0 0 0 1065083.0 3284.72
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
Max pending rx frames = 5 out of 276
[mr_halfword@haswell-alma release]$ cmac_ethernet/cmac_switch_test -n 0:1 -p 1-4 -l -r 100000 -s
Waiting to link to be ready to receive. Press Ctrl-C to abort.
Link ready
Waiting 2 seconds after link came up for switch to accept packets (latched rx_status initial 0x000000C8 last 0x00000003)
Bit rate on interface to injection switch = 100000 (Mbps)
Requested bit rate to be generated on each switch port under test = 100000.00 (Mbps)
Not limiting frame rate, as bit-rate on interface to injection switch doesn't exceed the total across all switch ports under test
Writing per-port counts to 20260829T155654_per_port_counts_linux.csv
Using design U200_100G_ether_duplex device 0000:31:00.0 Tx port 0 Rx port 1
Test interval = 10 (secs)
Frame debug enabled = No
Expect CMAC loopback = Yes
Disable CMAC port statistics = Yes
Press Ctrl-C to stop test at end of next test interval
15:57:06.811
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10284179 10284130 0 0 0 0 1028417.9 3171.64
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
15:57:16.811
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10761186 10761186 0 0 0 0 1076118.6 3318.75
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
<<snip>>
16:10:06.811
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10510734 10510735 0 0 0 0 1051073.4 3241.51
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
16:10:16.811
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10531996 10531995 0 0 0 0 1053199.6 3248.07
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
^C
16:10:26.811
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
10990102 10990102 0 0 0 0 1099010.2 3389.35
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 ...
2 . ..
3 .. .
4 ...
Total test intervals with failures = 0
Max pending rx frames = 5 out of 276
And a sample of statistics while the test was running:
16:10:18.109 collection number 81
U200_100G_ether_duplex port 0 statistics (over 10.000 secs):
CYCLE_COUNT : 3222629631
TX_TOTAL_PACKETS : 10562792
TX_TOTAL_GOOD_PACKETS : 10562792
TX_TOTAL_BYTES : 16076569424
TX_TOTAL_GOOD_BYTES : 16076569424
TX_PACKET_1519_1522_BYTES : 10562792
TX_UNICAST : 10562792
TX_VLAN : 10562792
RX_TOTAL_PACKETS : 10562666
RX_TOTAL_GOOD_PACKETS : 10562666
RX_TOTAL_BYTES : 16076377804
RX_TOTAL_GOOD_BYTES : 16076377652
RX_PACKET_1519_1522_BYTES : 10562666
RX_UNICAST : 10562666
RX_VLAN : 10562666
RX_RSFEC_CW_INC : 195310887
U200_100G_ether_duplex port 1 statistics (over 10.000 secs):
CYCLE_COUNT : 3222629626
TX_TOTAL_PACKETS : 10562667
TX_TOTAL_GOOD_PACKETS : 10562667
TX_TOTAL_BYTES : 16076379174
TX_TOTAL_GOOD_BYTES : 16076379174
TX_PACKET_1519_1522_BYTES : 10562667
TX_UNICAST : 10562667
TX_VLAN : 10562667
RX_TOTAL_PACKETS : 10562792
RX_TOTAL_GOOD_PACKETS : 10562792
RX_TOTAL_BYTES : 16076569424
RX_TOTAL_GOOD_BYTES : 16076569424
RX_PACKET_1519_1522_BYTES : 10562792
RX_UNICAST : 10562792
RX_VLAN : 10562792
RX_RSFEC_CW_INC : 195310887
The cmac_switch_test program was modified to add the -f command line option to allow testing with varying packet lengths.
This uses:
- The U200_100G_ether_duplex design identified as
E4352DB1 - 28/08/2026 18:54:49. - DAC cable between the U200 ports
- Tx port 1, rx port 0
- Testing packets lengths over 64 .. 9596 bytes (excluding FCS). This is the possible range with the default min/max receive packet lengths in the CMAC configuration.
Ran for 800 seconds:
$ cmac_ethernet/cmac_switch_test -n 1:0 -p 1-4 -l -r 100000 -f 60:9596
Link ready
Bit rate on interface to injection switch = 100000 (Mbps)
Requested bit rate to be generated on each switch port under test = 100000.00 (Mbps)
Not limiting frame rate, as bit-rate on interface to injection switch doesn't exceed the total across all switch ports under test
Writing per-port counts to 20260906T104433_per_port_counts_linux.csv
Using design U200_100G_ether_duplex device 0000:31:00.0 Tx port 1 Rx port 0
Test interval = 10 (secs)
Frame debug enabled = No
Expect CMAC loopback = Yes
Disable CMAC port statistics = No
Min packet len (excluding FCS) = 60
Max packet len (excluding FCS) = 9596
Press Ctrl-C to stop test at end of next test interval
10:44:43.470
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
9256760 9253427 0 0 3284 3284 925676.0 8982.76
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 SSS
2 S SS
3 SS S
4 SSS
Total test intervals with failures = 1 : last failure NOW
10:44:53.470
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
9133437 9130188 0 0 3249 3249 913343.6 8863.09
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 SSS
2 S SS
3 SS S
4 SSS
Total test intervals with failures = 2 : last failure NOW
<<snip>>
10:57:43.470
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
9166834 9163547 0 0 3287 3287 916683.3 8895.50
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 SSS
2 S SS
3 SS S
4 SSS
Total test intervals with failures = 79 : last failure NOW
^C
10:57:53.470
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
9171778 9168448 0 0 3330 3330 917177.9 8900.29
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 SSS
2 S SS
3 SS S
4 SSS
Total test intervals with failures = 80 : last failure NOW
U200_100G_ether_duplex port 1 statistics (over 800.000 secs):
CYCLE_COUNT : 257810601842
TX_TOTAL_PACKETS : 733554243
TX_TOTAL_GOOD_PACKETS : 733554243
TX_TOTAL_BYTES : 3545782084857
TX_TOTAL_GOOD_BYTES : 3545782084857
TX_PACKET_64_BYTES : 76943
TX_PACKET_65_127_BYTES : 4387470
TX_PACKET_128_255_BYTES : 9928066
TX_PACKET_256_511_BYTES : 19820093
TX_PACKET_512_1023_BYTES : 39396108
TX_PACKET_1024_1518_BYTES : 38087280
TX_PACKET_1519_1522_BYTES : 307776
TX_PACKET_1523_1548_BYTES : 2000544
TX_PACKET_1549_2047_BYTES : 38395056
TX_PACKET_2048_4095_BYTES : 157581312
TX_PACKET_4096_8191_BYTES : 315160908
TX_PACKET_8192_9215_BYTES : 78789632
TX_PACKET_LARGE : 29623055
TX_UNICAST : 733554243
TX_VLAN : 733554243
RX_RSFEC_CW_INC : 15624884960
U200_100G_ether_duplex port 0 statistics (over 800.000 secs):
CYCLE_COUNT : 257810601749
RX_BAD_CODE : 2351993
RX_TOTAL_PACKETS : 733811802
RX_TOTAL_GOOD_PACKETS : 733550115
RX_TOTAL_BYTES : 3545778032105
RX_TOTAL_GOOD_BYTES : 3545742318990
RX_PACKET_64_BYTES : 76943
RX_PACKET_65_127_BYTES : 4609628
RX_PACKET_128_255_BYTES : 10086676
RX_PACKET_256_511_BYTES : 19697664
RX_PACKET_512_1023_BYTES : 39395328
RX_PACKET_1024_1518_BYTES : 38087280
RX_PACKET_1519_1522_BYTES : 307776
RX_PACKET_1523_1548_BYTES : 2000544
RX_PACKET_1549_2047_BYTES : 38395056
RX_PACKET_2048_4095_BYTES : 157581312
RX_PACKET_4096_8191_BYTES : 315160908
RX_PACKET_8192_9215_BYTES : 78789632
RX_PACKET_LARGE : 29623055
RX_BAD_FCS : 261687
RX_PACKET_BAD_FCS : 261687
RX_UNICAST : 733550115
RX_VLAN : 733550115
RX_RSFEC_CW_INC : 15624884955
Max pending rx frames = 10 out of 276
There were some failures on every test iteration. Looking at the 20260906T104433_per_packet_len_linux.csv and filtering on only packet lengths with failures shows:
| packet len (bytes excluding FCS) |
Num tx frames | Num missed frames |
|---|---|---|
| 65 | 76944 | 4152 |
| 66 | 76944 | 3964 |
| 67 | 76944 | 4110 |
| 68 | 76944 | 4310 |
| 69 | 76944 | 3980 |
| 70 | 76944 | 3899 |
| 71 | 76944 | 4144 |
| 72 | 76944 | 4337 |
| 73 | 76944 | 4064 |
| 74 | 76944 | 3990 |
| 75 | 76944 | 4207 |
| 76 | 76944 | 4226 |
| 77 | 76944 | 4054 |
| 78 | 76944 | 3965 |
| 79 | 76944 | 3971 |
| 80 | 76944 | 4369 |
| 81 | 76944 | 4005 |
| 82 | 76944 | 3931 |
| 83 | 76944 | 3967 |
| 84 | 76944 | 4348 |
| 85 | 76944 | 4172 |
| 86 | 76944 | 3957 |
| 87 | 76944 | 4024 |
| 88 | 76944 | 4247 |
| 89 | 76944 | 4043 |
| 90 | 76944 | 3957 |
| 91 | 76944 | 4058 |
| 92 | 76944 | 4295 |
| 93 | 76944 | 4011 |
| 94 | 76944 | 3864 |
| 95 | 76944 | 4117 |
| 96 | 76944 | 4382 |
| 97 | 76944 | 4078 |
| 98 | 76944 | 3859 |
| 99 | 76944 | 4189 |
| 100 | 76944 | 4175 |
| 101 | 76944 | 3960 |
| 102 | 76944 | 3845 |
| 103 | 76944 | 4062 |
| 104 | 76944 | 4287 |
| 105 | 76944 | 4107 |
| 106 | 76944 | 4007 |
| 107 | 76944 | 4167 |
| 108 | 76944 | 4281 |
| 109 | 76944 | 4133 |
| 110 | 76944 | 4029 |
| 111 | 76944 | 4010 |
| 112 | 76944 | 4200 |
| 113 | 76944 | 4015 |
| 114 | 76944 | 3897 |
| 115 | 76944 | 4128 |
| 116 | 76944 | 4191 |
| 117 | 76944 | 4077 |
| 118 | 76944 | 3905 |
| 119 | 76944 | 4015 |
| 120 | 76944 | 4290 |
| 121 | 76944 | 4038 |
| 122 | 76944 | 3968 |
| 123 | 76944 | 4096 |
| 124 | 76944 | 4190 |
| 125 | 76944 | 4143 |
| 126 | 76944 | 3919 |
| 127 | 76944 | 3971 |
| 128 | 76944 | 4335 |
All packets lengths from 65 to 128 bytes have intermittent missed frames. Those packet lengths require two beats on the AXI-4 stream to the CMAC transmit. Across the different packets lengths the percentage missed is from 5.0% to 5.7%.
The total number of missed frames reported by the cmac_switch_test software is 261687, which matches the value of RX_BAD_FCS and RX_PACKET_BAD_FCS reported in the CMAC statistics.
The ratio of RX_BAD_CODE / RX_BAD_FCS is 8.99.
Repeated the above test with the same design, this time just swapped the transmit and receive ports and left running for longer:
[mr_halfword@haswell-alma release]$ cmac_ethernet/cmac_switch_test -n 0:1 -p 1-4 -l -r 100000 -f 60:9596
Link ready
Bit rate on interface to injection switch = 100000 (Mbps)
Requested bit rate to be generated on each switch port under test = 100000.00 (Mbps)
Not limiting frame rate, as bit-rate on interface to injection switch doesn't exceed the total across all switch ports under test
Writing per-port counts to 20260906T183905_per_port_counts_linux.csv
Using design U200_100G_ether_duplex device 0000:31:00.0 Tx port 0 Rx port 1
Test interval = 10 (secs)
Frame debug enabled = No
Expect CMAC loopback = Yes
Disable CMAC port statistics = No
Min packet len (excluding FCS) = 60
Max packet len (excluding FCS) = 9596
Press Ctrl-C to stop test at end of next test interval
18:39:15.707
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
8997619 8997169 0 0 400 400 899761.9 8731.29
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 SSS
2 S SS
3 SS S
4 SSS
Total test intervals with failures = 1 : last failure NOW
18:39:25.707
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
9160627 9160234 0 0 394 394 916062.6 8889.47
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 SSS
2 S SS
3 SS S
4 SSS
Total test intervals with failures = 2 : last failure NOW
<<snip>>
20:31:05.707
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
9129545 9129159 0 0 386 386 912954.5 8859.31
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 SSS
2 S SS
3 SS S
4 SSS
Total test intervals with failures = 672 : last failure NOW
^C
20:31:15.707
Tx Test Rx Test Rx Unexpected Rx Flooded Rx Other missed frames tx rate (Hz) per port Mbps
9113093 9112690 0 0 403 403 911309.3 8843.35
Summary of missed frames : '.' none missed 'S' some missed 'A' all missed
Source Destination ports --->
port
1234
1 SSS
2 S SS
3 SS S
4 SSS
Total test intervals with failures = 673 : last failure NOW
U200_100G_ether_duplex port 0 statistics (over 6730.000 secs):
CYCLE_COUNT : 2168830658207
TX_TOTAL_PACKETS : 6189698891
TX_TOTAL_GOOD_PACKETS : 6189698891
TX_TOTAL_BYTES : 29909865703993
TX_TOTAL_GOOD_BYTES : 29909865703993
TX_PACKET_64_BYTES : 649046
TX_PACKET_65_127_BYTES : 40421904
TX_PACKET_128_255_BYTES : 83165345
TX_PACKET_256_511_BYTES : 166279310
TX_PACKET_512_1023_BYTES : 332312089
TX_PACKET_1024_1518_BYTES : 321278265
TX_PACKET_1519_1522_BYTES : 2596188
TX_PACKET_1523_1548_BYTES : 16875222
TX_PACKET_1549_2047_BYTES : 323874453
TX_PACKET_2048_4095_BYTES : 1329248256
TX_PACKET_4096_8191_BYTES : 2658494408
TX_PACKET_8192_9215_BYTES : 664622080
TX_PACKET_LARGE : 249882325
TX_UNICAST : 6189698891
TX_VLAN : 6189698891
RX_RSFEC_CW_INC : 131444282316
U200_100G_ether_duplex port 1 statistics (over 6730.000 secs):
CYCLE_COUNT : 2168830658119
RX_BAD_CODE : 2349974
RX_TOTAL_PACKETS : 6189956316
RX_TOTAL_GOOD_PACKETS : 6189694708
RX_TOTAL_BYTES : 29909861651401
RX_TOTAL_GOOD_BYTES : 29909825956485
RX_PACKET_64_BYTES : 649046
RX_PACKET_65_127_BYTES : 40652740
RX_PACKET_128_255_BYTES : 83315237
RX_PACKET_256_511_BYTES : 166156032
RX_PACKET_512_1023_BYTES : 332312064
RX_PACKET_1024_1518_BYTES : 321278265
RX_PACKET_1519_1522_BYTES : 2596188
RX_PACKET_1523_1548_BYTES : 16875222
RX_PACKET_1549_2047_BYTES : 323874453
RX_PACKET_2048_4095_BYTES : 1329248256
RX_PACKET_4096_8191_BYTES : 2658494408
RX_PACKET_8192_9215_BYTES : 664622080
RX_PACKET_LARGE : 249882325
RX_BAD_FCS : 261608
RX_PACKET_BAD_FCS : 261608
RX_UNICAST : 6189694708
RX_VLAN : 6189694708
RX_RSFEC_CW_INC : 131444282310
Max pending rx frames = 8 out of 276
There were some failures on every test iteration. Looking at the 20260906T183905_per_packet_len_linux.csv and filtering on only packet lengths with failures shows:
| packet len (bytes excluding FCS) |
Num tx frames | Num missed frames |
|---|---|---|
| 65 | 649047 | 3231 |
| 66 | 649047 | 4738 |
| 67 | 649047 | 3581 |
| 68 | 649047 | 4627 |
| 69 | 649047 | 3267 |
| 70 | 649047 | 4762 |
| 71 | 649047 | 3594 |
| 72 | 649047 | 4887 |
| 73 | 649047 | 3202 |
| 74 | 649047 | 4864 |
| 75 | 649047 | 3525 |
| 76 | 649047 | 4603 |
| 77 | 649047 | 3256 |
| 78 | 649047 | 4798 |
| 79 | 649047 | 3547 |
| 80 | 649047 | 4658 |
| 81 | 649047 | 3244 |
| 82 | 649047 | 4751 |
| 83 | 649047 | 3602 |
| 84 | 649047 | 4692 |
| 85 | 649047 | 3356 |
| 86 | 649047 | 4738 |
| 87 | 649047 | 3566 |
| 88 | 649047 | 4760 |
| 89 | 649047 | 3307 |
| 90 | 649047 | 4788 |
| 91 | 649047 | 3603 |
| 92 | 649047 | 4620 |
| 93 | 649047 | 3241 |
| 94 | 649047 | 4845 |
| 95 | 649047 | 3663 |
| 96 | 649047 | 4630 |
| 97 | 649047 | 3216 |
| 98 | 649047 | 4854 |
| 99 | 649047 | 3695 |
| 100 | 649047 | 4642 |
| 101 | 649047 | 3216 |
| 102 | 649047 | 4826 |
| 103 | 649047 | 3526 |
| 104 | 649047 | 4858 |
| 105 | 649047 | 3146 |
| 106 | 649047 | 4862 |
| 107 | 649047 | 3583 |
| 108 | 649047 | 4673 |
| 109 | 649047 | 3248 |
| 110 | 649047 | 4775 |
| 111 | 649047 | 3541 |
| 112 | 649047 | 4685 |
| 113 | 649047 | 3367 |
| 114 | 649047 | 4823 |
| 115 | 649047 | 3586 |
| 116 | 649047 | 4504 |
| 117 | 649047 | 3235 |
| 118 | 649047 | 4849 |
| 119 | 649047 | 3644 |
| 120 | 649047 | 4811 |
| 121 | 649047 | 3274 |
| 122 | 649047 | 4899 |
| 123 | 649047 | 3648 |
| 124 | 649047 | 4698 |
| 125 | 649047 | 3231 |
| 126 | 649047 | 4739 |
| 127 | 649047 | 3675 |
| 128 | 649047 | 4733 |
All packets lengths from 65 to 128 bytes have intermittent missed frames. Those packet lengths require two beats on the AXI-4 stream to the CMAC transmit. Across the different packets lengths the percentage missed is from 0.48% to 0.75%. This is a lower percentage of missed frames compared to when were transmitting in the other direction on the ports.
The total number of missed frames reported by the cmac_switch_test software is 261608, which matches the value of RX_BAD_FCS and RX_PACKET_BAD_FCS reported in the CMAC statistics.
The ratio of RX_BAD_CODE / RX_BAD_FCS is 8.98. Not sure what causes RX_BAD_CODE to increment, but is the same ratio as the previous test in the other port direction which had a lower percentage of missed frames. Does the CMAC deliberately output a bad code after getting a transmit underflow?
Since the U200_100G_ether_duplex design doesn't yet have a mechanism to report the CMAC receive errors to the software, the software doesn't count packets with a bad FCS. On each iteration, the number of "Rx Other" packets matches the number of "missed frames" so frames with a bad FCS do get passed to the software, which then discards them since aren't the expected test frames. Looking at some frame debug results for "Rx Other"
- Have extracted a test VLAN ID from the start of the packet.
- The majority of packets lengths are 132 bytes, but have seen other values 74, 85, 86, 92, 94, 109, 188