Contains some notes about using the X722 Ethernet controller built in to the HP Z6 G4, which uses the C622 chipset. Related is Intel X722 usage which used an Intel X722-DA2 PCIe card.
The PC doesn't have the HP Z 2-Port 10GbE Network Module (1QL49AA) option fitted, which would add two 1Gbps/10Gbps ports via a dedicated connector to the motherboard - see Networking Adapters for HP Workstations
As a result there are two RJ45 Ethernet ports:
- One using a X722
- One using a I219-LM, which also supports AMT
When booted into Windows 11 Pro installed on a NVMe both Ethernet ports were functional when connected to a 1GbE switch. Booted AlmaLinux 9.5 live image to check the functionality before deciding which Linux distrubution to install.
The output of ethool for the X722 port shows only 1000baseT/full is shown as supported:
$ ethtool enp9s0f2
Settings for enp9s0f2:
Supported ports: [ TP ]
Supported link modes: 1000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
MDI-X: Unknown
netlink error: Operation not permitted
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Whereas for the I219-LM port lower speeds are supported:
$ ethtool eno1
Settings for eno1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 2
Transceiver: internal
MDI-X: Unknown (auto)
netlink error: Operation not permitted
Current message level: 0x00000007 (7)
drv probe link
Link detected: no
Installed libibverbs-utils to get the tools for checking RDMA. iWARP is enabled on the X722 port:
$ ibv_devinfo -v
hca_id: irdma0
transport: iWARP (1)
fw_ver: 0.2
node_guid: cad9:d2ff:fe1f:5ff9
sys_image_guid: cad9:d2ff:fe1f:5ff9
vendor_id: 0x8086
vendor_part_id: 14289
hw_ver: 0x9
phys_port_cnt: 1
max_mr_size: 0x200000000000
page_size_cap: 0x201000
max_qp: 32765
max_qp_wr: 895
device_cap_flags: 0x00220000
MEM_WINDOW
MEM_MGT_EXTENSIONS
max_sge: 3
max_sge_rd: 1
max_cq: 65533
max_cqe: 1048574
max_mr: 2097150
max_pd: 32765
max_qp_rd_atom: 63
max_ee_rd_atom: 0
max_res_rd_atom: 0
max_qp_init_rd_atom: 127
max_ee_init_rd_atom: 0
atomic_cap: ATOMIC_NONE (0)
max_ee: 0
max_rdd: 0
max_mw: 2097150
max_raw_ipv6_qp: 0
max_raw_ethy_qp: 0
max_mcast_grp: 32768
max_mcast_qp_attach: 8
max_total_mcast_qp_attach: 262144
max_ah: 0
max_fmr: 0
max_srq: 0
max_pkeys: 0
local_ca_ack_delay: 0
general_odp_caps:
rc_odp_caps:
NO SUPPORT
uc_odp_caps:
NO SUPPORT
ud_odp_caps:
NO SUPPORT
xrc_odp_caps:
NO SUPPORT
completion_timestamp_mask not supported
core clock not supported
device_cap_flags_ex: 0x220000
tso_caps:
max_tso: 0
rss_caps:
max_rwq_indirection_tables: 0
max_rwq_indirection_table_size: 0
rx_hash_function: 0x0
rx_hash_fields_mask: 0x0
max_wq_type_rq: 0
packet_pacing_caps:
qp_rate_limit_min: 0kbps
qp_rate_limit_max: 0kbps
tag matching not supported
num_comp_vectors: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 1
port_lmc: 0x00
link_layer: Ethernet
max_msg_sz: 0x7fffffff
port_cap_flags: 0x00050000
port_cap_flags2: 0x0000
max_vl_num: invalid value (0)
bad_pkey_cntr: 0x0
qkey_viol_cntr: 0x0
sm_sl: 0
pkey_tbl_len: 0
gid_tbl_len: 1
subnet_timeout: 0
init_type_reply: 0
active_width: 1X (1)
active_speed: 2.5 Gbps (1)
iWARP is active on the X722 port at the same time as the Linux network stack, albeit haven't attempted any iWARP communication yet.
There are two functions on the same device which report X722 identities:
$ lspci -nn | grep 722
09:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 [8086:37cc] (rev 09)
09:00.2 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 1GbE [8086:37d1] (rev 09)
And the detailed output:
$ sudo lspci -vvv -nn -s 09:00
[sudo] password for liveuser:
09:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 [8086:37cc] (rev 09)
Subsystem: Hewlett-Packard Company Device [103c:81c6]
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 255
NUMA node: 0
Region 0: Memory at 383ffe000000 (64-bit, prefetchable) [disabled] [size=16M]
Region 3: Memory at 383fff808000 (64-bit, prefetchable) [disabled] [size=32K]
Expansion ROM at <ignored> [disabled]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable- Count=129 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00001000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [e0] Vital Product Data
Product Name: Example VPD
Read-only fields:
[V0] Vendor specific:
[RV] Reserved: checksum good, 0 byte(s) reserved
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 2
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
IOVSta: Migration-
Initial VFs: 32, Total VFs: 32, Number of VFs: 0, Function Dependency Link: 00
VF offset: 16, stride: 1, Device ID: 37cd
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000383fff400000 (64-bit, prefetchable)
Region 3: Memory at 0000383fff890000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1a0 v1] Transaction Processing Hints
Device specific mode supported
No steering table available
Capabilities: [1b0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
09:00.2 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 1GbE [8086:37d1] (rev 09)
Subsystem: Hewlett-Packard Company Device [103c:81c6]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 19
NUMA node: 0
Region 0: Memory at 383ffd000000 (64-bit, prefetchable) [size=16M]
Region 3: Memory at 383fff800000 (64-bit, prefetchable) [size=32K]
Expansion ROM at <ignored> [disabled]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=129 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00001000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [e0] Vital Product Data
Product Name: Example VPD
Read-only fields:
[V0] Vendor specific:
[RV] Reserved: checksum good, 0 byte(s) reserved
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
IOVSta: Migration-
Initial VFs: 32, Total VFs: 32, Number of VFs: 0, Function Dependency Link: 02
VF offset: 78, stride: 1, Device ID: 37cd
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000383fff000000 (64-bit, prefetchable)
Region 3: Memory at 0000383fff810000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1a0 v1] Transaction Processing Hints
Device specific mode supported
No steering table available
Capabilities: [1b0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: i40e
Kernel modules: i40e
Where:
- The
8086:37ccendpoint at09:00.0has it's BARs disabled and no driver loaded. This is presumably the endpoint for the HP Z 2-Port 10GbE Network Module (1QL49AA) option, which the BIOS might enable when the option card is fitted. - The
8086:37d1endpoint at09:00.2has its BARs enabled and drivers loaded. This is the endpoint for the on-board X722 1GbE port.
After a 1QL49AA was fitted, booting into the BIOS setup showed two additional X722 ports in the UEFI network stack.
Booted into Windows 11 and two Intel(R) Ethernet Connection X722 for 10GBASE-T interfaces appeared, and Windows could asign an IP address for the interfaces by DHCP.
Output when the only Ethernet cable was connected between one of the X722 for 10GBASE-T ports and 1 1G Ethernet switch. I.e. the X722 10GBASE-T ports can work at 1G:
C:\Users\mr_halfword>ipconfig /all
Windows IP Configuration
Host Name . . . . . . . . . . . . : DESKTOP-BVUMP11
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : lan
Ethernet adapter Ethernet:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . : lan
Description . . . . . . . . . . . : Intel(R) Ethernet Connection (3) I219-LM
Physical Address. . . . . . . . . : C8-D9-D2-1F-5F-F6
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Ethernet adapter Ethernet 4:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . : lan
Description . . . . . . . . . . . : Intel(R) Ethernet Connection X722 for 10GBASE-T #2
Physical Address. . . . . . . . . : C8-D9-D2-1F-5F-F7
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Ethernet adapter Ethernet 2:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . : lan
Description . . . . . . . . . . . : Intel(R) Ethernet Connection X722 for 1GbE
Physical Address. . . . . . . . . : C8-D9-D2-1F-5F-F9
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Ethernet adapter Ethernet 3:
Connection-specific DNS Suffix . : lan
Description . . . . . . . . . . . : Intel(R) Ethernet Connection X722 for 10GBASE-T
Physical Address. . . . . . . . . : C8-D9-D2-1F-5F-F8
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::a530:bd54:5bd3:7120%23(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.0.58(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Lease Obtained. . . . . . . . . . : 26 January 2025 15:07:11
Lease Expires . . . . . . . . . . : 27 January 2025 15:07:11
Default Gateway . . . . . . . . . : 192.168.0.1
DHCP Server . . . . . . . . . . . : 192.168.0.1
DHCPv6 IAID . . . . . . . . . . . : 399038930
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2E-DC-B4-A2-C8-D9-D2-1F-5F-F6
DNS Servers . . . . . . . . . . . : 192.168.0.1
NetBIOS over Tcpip. . . . . . . . : Enabled
my msp430 code is working when running with JTAG debugging but not without JTAG
ethtool shows the additional ports support 1000baseT/Full, 10000baseT/Full 1000baseKX/Full and 10000baseKR/Full. Shown here with enp9s0f0 not connected and enp9s0f1 connected to a 1G Ethernet switch:
$ ethtool enp9s0f0
Settings for enp9s0f0:
Supported ports: [ ]
Supported link modes: 1000baseT/Full
10000baseT/Full
1000baseKX/Full
10000baseKR/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
10000baseT/Full
1000baseKX/Full
10000baseKR/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
netlink error: Operation not permitted
Current message level: 0x00000007 (7)
drv probe link
Link detected: no
liveuser@localhost-live:~$ ethtool enp9s0f1
Settings for enp9s0f1:
Supported ports: [ TP ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
MDI-X: Unknown
netlink error: Operation not permitted
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
iWARP is enabled on all 3 X722 ports:
$ ibv_devinfo -v
hca_id: irdma0
transport: iWARP (1)
fw_ver: 0.2
node_guid: cad9:d2ff:fe1f:5ff9
sys_image_guid: cad9:d2ff:fe1f:5ff9
vendor_id: 0x8086
vendor_part_id: 14289
hw_ver: 0x9
phys_port_cnt: 1
max_mr_size: 0x200000000000
page_size_cap: 0x201000
max_qp: 16381
max_qp_wr: 895
device_cap_flags: 0x00220000
MEM_WINDOW
MEM_MGT_EXTENSIONS
max_sge: 3
max_sge_rd: 1
max_cq: 32765
max_cqe: 1048574
max_mr: 1048574
max_pd: 32765
max_qp_rd_atom: 63
max_ee_rd_atom: 0
max_res_rd_atom: 0
max_qp_init_rd_atom: 127
max_ee_init_rd_atom: 0
atomic_cap: ATOMIC_NONE (0)
max_ee: 0
max_rdd: 0
max_mw: 1048574
max_raw_ipv6_qp: 0
max_raw_ethy_qp: 0
max_mcast_grp: 16384
max_mcast_qp_attach: 8
max_total_mcast_qp_attach: 131072
max_ah: 0
max_fmr: 0
max_srq: 0
max_pkeys: 0
local_ca_ack_delay: 0
general_odp_caps:
rc_odp_caps:
NO SUPPORT
uc_odp_caps:
NO SUPPORT
ud_odp_caps:
NO SUPPORT
xrc_odp_caps:
NO SUPPORT
completion_timestamp_mask not supported
core clock not supported
device_cap_flags_ex: 0x220000
tso_caps:
max_tso: 0
rss_caps:
max_rwq_indirection_tables: 0
max_rwq_indirection_table_size: 0
rx_hash_function: 0x0
rx_hash_fields_mask: 0x0
max_wq_type_rq: 0
packet_pacing_caps:
qp_rate_limit_min: 0kbps
qp_rate_limit_max: 0kbps
tag matching not supported
num_comp_vectors: 1
port: 1
state: PORT_DOWN (1)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 1
port_lmc: 0x00
link_layer: Ethernet
max_msg_sz: 0x7fffffff
port_cap_flags: 0x00050000
port_cap_flags2: 0x0000
max_vl_num: invalid value (0)
bad_pkey_cntr: 0x0
qkey_viol_cntr: 0x0
sm_sl: 0
pkey_tbl_len: 0
gid_tbl_len: 1
subnet_timeout: 0
init_type_reply: 0
active_width: 1X (1)
active_speed: 2.5 Gbps (1)
hca_id: irdma1
transport: iWARP (1)
fw_ver: 0.2
node_guid: cad9:d2ff:fe1f:5ff8
sys_image_guid: cad9:d2ff:fe1f:5ff8
vendor_id: 0x8086
vendor_part_id: 14290
hw_ver: 0x9
phys_port_cnt: 1
max_mr_size: 0x200000000000
page_size_cap: 0x201000
max_qp: 16381
max_qp_wr: 895
device_cap_flags: 0x00220000
MEM_WINDOW
MEM_MGT_EXTENSIONS
max_sge: 3
max_sge_rd: 1
max_cq: 32765
max_cqe: 1048574
max_mr: 1048574
max_pd: 32765
max_qp_rd_atom: 63
max_ee_rd_atom: 0
max_res_rd_atom: 0
max_qp_init_rd_atom: 127
max_ee_init_rd_atom: 0
atomic_cap: ATOMIC_NONE (0)
max_ee: 0
max_rdd: 0
max_mw: 1048574
max_raw_ipv6_qp: 0
max_raw_ethy_qp: 0
max_mcast_grp: 16384
max_mcast_qp_attach: 8
max_total_mcast_qp_attach: 131072
max_ah: 0
max_fmr: 0
max_srq: 0
max_pkeys: 0
local_ca_ack_delay: 0
general_odp_caps:
rc_odp_caps:
NO SUPPORT
uc_odp_caps:
NO SUPPORT
ud_odp_caps:
NO SUPPORT
xrc_odp_caps:
NO SUPPORT
completion_timestamp_mask not supported
core clock not supported
device_cap_flags_ex: 0x220000
tso_caps:
max_tso: 0
rss_caps:
max_rwq_indirection_tables: 0
max_rwq_indirection_table_size: 0
rx_hash_function: 0x0
rx_hash_fields_mask: 0x0
max_wq_type_rq: 0
packet_pacing_caps:
qp_rate_limit_min: 0kbps
qp_rate_limit_max: 0kbps
tag matching not supported
num_comp_vectors: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 1
port_lmc: 0x00
link_layer: Ethernet
max_msg_sz: 0x7fffffff
port_cap_flags: 0x00050000
port_cap_flags2: 0x0000
max_vl_num: invalid value (0)
bad_pkey_cntr: 0x0
qkey_viol_cntr: 0x0
sm_sl: 0
pkey_tbl_len: 0
gid_tbl_len: 1
subnet_timeout: 0
init_type_reply: 0
active_width: 1X (1)
active_speed: 2.5 Gbps (1)
hca_id: irdma2
transport: iWARP (1)
fw_ver: 0.2
node_guid: cad9:d2ff:fe1f:5ff7
sys_image_guid: cad9:d2ff:fe1f:5ff7
vendor_id: 0x8086
vendor_part_id: 14290
hw_ver: 0x9
phys_port_cnt: 1
max_mr_size: 0x200000000000
page_size_cap: 0x201000
max_qp: 16381
max_qp_wr: 895
device_cap_flags: 0x00220000
MEM_WINDOW
MEM_MGT_EXTENSIONS
max_sge: 3
max_sge_rd: 1
max_cq: 32765
max_cqe: 1048574
max_mr: 1048574
max_pd: 32765
max_qp_rd_atom: 63
max_ee_rd_atom: 0
max_res_rd_atom: 0
max_qp_init_rd_atom: 127
max_ee_init_rd_atom: 0
atomic_cap: ATOMIC_NONE (0)
max_ee: 0
max_rdd: 0
max_mw: 1048574
max_raw_ipv6_qp: 0
max_raw_ethy_qp: 0
max_mcast_grp: 16384
max_mcast_qp_attach: 8
max_total_mcast_qp_attach: 131072
max_ah: 0
max_fmr: 0
max_srq: 0
max_pkeys: 0
local_ca_ack_delay: 0
general_odp_caps:
rc_odp_caps:
NO SUPPORT
uc_odp_caps:
NO SUPPORT
ud_odp_caps:
NO SUPPORT
xrc_odp_caps:
NO SUPPORT
completion_timestamp_mask not supported
core clock not supported
device_cap_flags_ex: 0x220000
tso_caps:
max_tso: 0
rss_caps:
max_rwq_indirection_tables: 0
max_rwq_indirection_table_size: 0
rx_hash_function: 0x0
rx_hash_fields_mask: 0x0
max_wq_type_rq: 0
packet_pacing_caps:
qp_rate_limit_min: 0kbps
qp_rate_limit_max: 0kbps
tag matching not supported
num_comp_vectors: 1
port: 1
state: PORT_DOWN (1)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 1
port_lmc: 0x00
link_layer: Ethernet
max_msg_sz: 0x7fffffff
port_cap_flags: 0x00050000
port_cap_flags2: 0x0000
max_vl_num: invalid value (0)
bad_pkey_cntr: 0x0
qkey_viol_cntr: 0x0
sm_sl: 0
pkey_tbl_len: 0
gid_tbl_len: 1
subnet_timeout: 0
init_type_reply: 0
active_width: 1X (1)
active_speed: 2.5 Gbps (1)
Not that for ports active_width is reported as 1X (1) and active_speed as 2.5 Gbps (1) even though only one port has a link. The smallest lane rate the active_speed fields in the struct ibv_port_attr encodes is 2.5 Gbps, which is why the actual link speed of 1G isn't reported.
There are three functions on the same device which report X722 identities:
$ lspci -nn | grep 722
09:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 10GBASE-T [8086:37d2] (rev 09)
09:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 10GBASE-T [8086:37d2] (rev 09)
09:00.2 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 1GbE [8086:37d1] (rev 09)
And the detailed output:
$ sudo lspci -vvv -nn -s 09:00
[sudo] password for liveuser:
09:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 10GBASE-T [8086:37d2] (rev 09)
Subsystem: Hewlett-Packard Company Device [103c:81c6]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 19
NUMA node: 0
Region 0: Memory at 383ffe000000 (64-bit, prefetchable) [size=16M]
Region 3: Memory at 383fffc10000 (64-bit, prefetchable) [size=32K]
Expansion ROM at <ignored> [disabled]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=129 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00001000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [e0] Vital Product Data
Product Name: Example VPD
Read-only fields:
[V0] Vendor specific:
[RV] Reserved: checksum good, 0 byte(s) reserved
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 1
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
IOVSta: Migration-
Initial VFs: 32, Total VFs: 32, Number of VFs: 0, Function Dependency Link: 00
VF offset: 16, stride: 1, Device ID: 37cd
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000383fff800000 (64-bit, prefetchable)
Region 3: Memory at 0000383fffd18000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1a0 v1] Transaction Processing Hints
Device specific mode supported
No steering table available
Capabilities: [1b0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: i40e
Kernel modules: i40e
09:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 10GBASE-T [8086:37d2] (rev 09)
Subsystem: Hewlett-Packard Company Device [103c:81c6]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 19
NUMA node: 0
Region 0: Memory at 383ffd000000 (64-bit, prefetchable) [size=16M]
Region 3: Memory at 383fffc08000 (64-bit, prefetchable) [size=32K]
Expansion ROM at <ignored> [disabled]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=129 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00001000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [e0] Vital Product Data
Product Name: Example VPD
Read-only fields:
[V0] Vendor specific:
[RV] Reserved: checksum good, 0 byte(s) reserved
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 2
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
IOVSta: Migration-
Initial VFs: 32, Total VFs: 32, Number of VFs: 0, Function Dependency Link: 01
VF offset: 47, stride: 1, Device ID: 37cd
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000383fff400000 (64-bit, prefetchable)
Region 3: Memory at 0000383fffc98000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1a0 v1] Transaction Processing Hints
Device specific mode supported
No steering table available
Capabilities: [1b0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: i40e
Kernel modules: i40e
09:00.2 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 1GbE [8086:37d1] (rev 09)
Subsystem: Hewlett-Packard Company Device [103c:81c6]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 19
NUMA node: 0
Region 0: Memory at 383ffc000000 (64-bit, prefetchable) [size=16M]
Region 3: Memory at 383fffc00000 (64-bit, prefetchable) [size=32K]
Expansion ROM at <ignored> [disabled]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=129 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00001000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [e0] Vital Product Data
Product Name: Example VPD
Read-only fields:
[V0] Vendor specific:
[RV] Reserved: checksum good, 0 byte(s) reserved
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
IOVSta: Migration-
Initial VFs: 32, Total VFs: 32, Number of VFs: 0, Function Dependency Link: 02
VF offset: 78, stride: 1, Device ID: 37cd
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000383fff000000 (64-bit, prefetchable)
Region 3: Memory at 0000383fffc18000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1a0 v1] Transaction Processing Hints
Device specific mode supported
No steering table available
Capabilities: [1b0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: i40e
Kernel modules: i40e
The output from lshw:
$ sudo lshw -class network
*-network:0
description: Ethernet interface
product: Ethernet Connection X722 for 10GBASE-T
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:09:00.0
logical name: enp9s0f0
version: 09
serial: c8:d9:d2:1f:5f:f7
capacity: 10Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical 1000bt-fd 1000bx-fd 10000bt-fd 10000bx-fd autonegotiation
configuration: autonegotiation=off broadcast=yes driver=i40e driverversion=5.14.0-427.16.1.el9_4.x86_64 firmware=3.2d 0x80000c1b 1.1681.0 latency=0 link=no multicast=yes
resources: iomemory:383f0-383ef iomemory:383f0-383ef irq:19 memory:383ffe000000-383ffeffffff memory:383fffc10000-383fffc17fff memory:383fff800000-383fffbfffff memory:383fffd18000-383fffd97fff
*-network:1
description: Ethernet interface
product: Ethernet Connection X722 for 10GBASE-T
vendor: Intel Corporation
physical id: 0.1
bus info: pci@0000:09:00.1
logical name: enp9s0f1
version: 09
serial: c8:d9:d2:1f:5f:f8
size: 1Gbit/s
capacity: 10Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical tp 1000bt-fd 10000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=i40e driverversion=5.14.0-427.16.1.el9_4.x86_64 duplex=full firmware=3.2d 0x80000c1b 1.1681.0 ip=192.168.0.58 latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s
resources: iomemory:383f0-383ef iomemory:383f0-383ef irq:19 memory:383ffd000000-383ffdffffff memory:383fffc08000-383fffc0ffff memory:383fff400000-383fff7fffff memory:383fffc98000-383fffd17fff
*-network:2
description: Ethernet interface
product: Ethernet Connection X722 for 1GbE
vendor: Intel Corporation
physical id: 0.2
bus info: pci@0000:09:00.2
logical name: enp9s0f2
version: 09
serial: c8:d9:d2:1f:5f:f9
capacity: 1Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical 1000bt-fd autonegotiation
configuration: autonegotiation=off broadcast=yes driver=i40e driverversion=5.14.0-427.16.1.el9_4.x86_64 firmware=3.2d 0x80000c1b 1.1681.0 latency=0 link=no multicast=yes
resources: iomemory:383f0-383ef iomemory:383f0-383ef irq:19 memory:383ffc000000-383ffcffffff memory:383fffc00000-383fffc07fff memory:383fff000000-383fff3fffff memory:383fffc18000-383fffc97fff
*-network
description: Ethernet interface
product: Ethernet Connection (3) I219-LM
vendor: Intel Corporation
physical id: 1f.6
bus info: pci@0000:00:1f.6
logical name: eno1
version: 09
serial: c8:d9:d2:1f:5f:f6
capacity: 1Gbit/s
width: 32 bits
clock: 33MHz
capabilities: pm msi bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=5.14.0-427.16.1.el9_4.x86_64 firmware=0.2-4 latency=0 link=no multicast=yes port=twisted pair
resources: irq:26 memory:94380000-9439ffff
Fitted a DELL SFP-10G-T-MG-30M-G3.5 SFP module into a tp-link T1700G-28TQ switch, and connected to one of the 10G ports on the HP Z6 G4.
ibv_devinfo -v showed a 10.0 Gbps link speed, which is correct:
hca_id: irdma0
transport: iWARP (1)
fw_ver: 0.2
node_guid: cad9:d2ff:fe1f:5ff7
sys_image_guid: cad9:d2ff:fe1f:5ff7
vendor_id: 0x8086
vendor_part_id: 14290
hw_ver: 0x9
phys_port_cnt: 1
max_mr_size: 0x200000000000
page_size_cap: 0x201000
max_qp: 16381
max_qp_wr: 895
device_cap_flags: 0x00220000
MEM_WINDOW
MEM_MGT_EXTENSIONS
max_sge: 3
max_sge_rd: 1
max_cq: 32765
max_cqe: 1048574
max_mr: 1048574
max_pd: 32765
max_qp_rd_atom: 63
max_ee_rd_atom: 0
max_res_rd_atom: 0
max_qp_init_rd_atom: 127
max_ee_init_rd_atom: 0
atomic_cap: ATOMIC_NONE (0)
max_ee: 0
max_rdd: 0
max_mw: 1048574
max_raw_ipv6_qp: 0
max_raw_ethy_qp: 0
max_mcast_grp: 16384
max_mcast_qp_attach: 8
max_total_mcast_qp_attach: 131072
max_ah: 0
max_fmr: 0
max_srq: 0
max_pkeys: 0
local_ca_ack_delay: 0
general_odp_caps:
rc_odp_caps:
NO SUPPORT
uc_odp_caps:
NO SUPPORT
ud_odp_caps:
NO SUPPORT
xrc_odp_caps:
NO SUPPORT
completion_timestamp_mask not supported
core clock not supported
device_cap_flags_ex: 0x220000
tso_caps:
max_tso: 0
rss_caps:
max_rwq_indirection_tables: 0
max_rwq_indirection_table_size: 0
rx_hash_function: 0x0
rx_hash_fields_mask: 0x0
max_wq_type_rq: 0
packet_pacing_caps:
qp_rate_limit_min: 0kbps
qp_rate_limit_max: 0kbps
tag matching not supported
num_comp_vectors: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 1
port_lmc: 0x00
link_layer: Ethernet
max_msg_sz: 0x7fffffff
port_cap_flags: 0x00050000
port_cap_flags2: 0x0000
max_vl_num: invalid value (0)
bad_pkey_cntr: 0x0
qkey_viol_cntr: 0x0
sm_sl: 0
pkey_tbl_len: 0
gid_tbl_len: 1
subnet_timeout: 0
init_type_reply: 0
active_width: 1X (1)
active_speed: 10.0 Gbps (8)
ethtool also shows a 10G link speed:
$ ethtool enp9s0f0
Settings for enp9s0f0:
Supported ports: [ TP ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
MDI-X: Unknown
netlink error: Operation not permitted
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
The output from lshw:
$ sudo lshw -class network
[sudo] password for liveuser:
*-network:0
description: Ethernet interface
product: Ethernet Connection X722 for 10GBASE-T
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:09:00.0
logical name: enp9s0f0
version: 09
serial: c8:d9:d2:1f:5f:f7
size: 10Gbit/s
capacity: 10Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical tp 1000bt-fd 10000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=i40e driverversion=5.14.0-427.16.1.el9_4.x86_64 duplex=full firmware=3.2d 0x80000c1b 1.1681.0 ip=192.168.0.57 latency=0 link=yes multicast=yes port=twisted pair speed=10Gbit/s
resources: iomemory:383f0-383ef iomemory:383f0-383ef irq:19 memory:383ffe000000-383ffeffffff memory:383fffc10000-383fffc17fff memory:383fff800000-383fffbfffff memory:383fffd18000-383fffd97fff
*-network:1
description: Ethernet interface
product: Ethernet Connection X722 for 10GBASE-T
vendor: Intel Corporation
physical id: 0.1
bus info: pci@0000:09:00.1
logical name: enp9s0f1
version: 09
serial: c8:d9:d2:1f:5f:f8
capacity: 10Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical 1000bt-fd 1000bx-fd 10000bt-fd 10000bx-fd autonegotiation
configuration: autonegotiation=off broadcast=yes driver=i40e driverversion=5.14.0-427.16.1.el9_4.x86_64 firmware=3.2d 0x80000c1b 1.1681.0 latency=0 link=no multicast=yes
resources: iomemory:383f0-383ef iomemory:383f0-383ef irq:19 memory:383ffd000000-383ffdffffff memory:383fffc08000-383fffc0ffff memory:383fff400000-383fff7fffff memory:383fffc98000-383fffd17fff
*-network:2
description: Ethernet interface
product: Ethernet Connection X722 for 1GbE
vendor: Intel Corporation
physical id: 0.2
bus info: pci@0000:09:00.2
logical name: enp9s0f2
version: 09
serial: c8:d9:d2:1f:5f:f9
capacity: 1Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical 1000bt-fd autonegotiation
configuration: autonegotiation=off broadcast=yes driver=i40e driverversion=5.14.0-427.16.1.el9_4.x86_64 firmware=3.2d 0x80000c1b 1.1681.0 latency=0 link=no multicast=yes
resources: iomemory:383f0-383ef iomemory:383f0-383ef irq:19 memory:383ffc000000-383ffcffffff memory:383fffc00000-383fffc07fff memory:383fff000000-383fff3fffff memory:383fffc18000-383fffc97fff
*-network
description: Ethernet interface
product: Ethernet Connection (3) I219-LM
vendor: Intel Corporation
physical id: 1f.6
bus info: pci@0000:00:1f.6
logical name: eno1
version: 09
serial: c8:d9:d2:1f:5f:f6
capacity: 1Gbit/s
width: 32 bits
clock: 33MHz
capabilities: pm msi bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=5.14.0-427.16.1.el9_4.x86_64 firmware=0.2-4 latency=0 link=no multicast=yes port=twisted pair
resources: irq:27 memory:94380000-9439ffff
*-network
description: Ethernet interface
product: MT27700 Family [ConnectX-4]
vendor: Mellanox Technologies
physical id: 0
bus info: pci@0000:2d:00.0
logical name: enp45s0np0
version: 00
serial: 98:03:9b:6a:7e:a1
capacity: 40Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pciexpress vpd msix pm bus_master cap_list rom ethernet physical 1000bx-fd 10000bx-fd 25000bx-fd 40000bx-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=mlx5_core driverversion=5.14.0-427.16.1.el9_4.x86_64 firmware=12.28.2006 (MT_2180110032) latency=0 link=no multicast=yes
resources: iomemory:38ff0-38fef irq:400 memory:38fffe000000-38ffffffffff memory:bb900000-bb9fffff