Initialize emulex script

readme.md

About

This script will run a self-test on all be2net interfaces during boot. For my Emulex OCe11102-N-XI (IBM branded, FRU 49Y7942) this results in the interface being initialized and being available to use as a 10GBe NIC on my Arch Linux and Manjaro boxes.

Mind: I have not tried this with headless boot, yet. But it works well on my desktop machine.

Maybe there should be some module magic to actually do this right, but it's old hardware and I'm only using this at home. So I don't feel like peskering Kernel people with this.

This trick was discovered by Jurriaan Pruis. Thanks for blogging about it!

Manual test

You might want to first try manually initializing the card, just run the self-test using ethtool -t $interface (repeat for all ports/SFP+ slots). On the first try the command fails, but the card is then silently initialized and you should see the link go up (that is, if the device is connected to something).

If this fails and you have a multi-port card, make sure to run the command for the correct port!

See the end of the blog article for details.

Optional: Firmware update

Mind that I had success with the pre-installed firmware version 4.2.433.3 (run ethtool -i $interface to see your firmware version) I only dared to upgrade to 4.6.281.8 to avoid bricking the card. You can find that version, among other, newer versions, at ftp://download2.boulder.ibm.com/ecc/sar/CMA/XSA/. Just check the elx_fw_cna files: Each elx_fw_cna_*_linux_32-64.bin is a self-extracting updater, containing the firmeware file. Just pass -x some_temp_dir and it will self-extract without running anything else.

Jurriaan linked to an HPE ISO. That contains an even newer firmware version.

Updating the firmware then works just as Jurriaan describes:

1. Copy the firmware/oc11-[...].ufi to /lib/firmware
2. run ethtool -f $interface oc11-[...].ufi

Careful, heed this instruction from broadcom when updating from a version prior to 10.0.803.37 to that version or newer:

You must perform the firmware update procedure twice to ensure that the flash regions are properly configured, and you must reboot the system after each firmware update.

Installing the script to run during boot

Usage:

1. Install the card, connect a DAC and boot your machine.
2. Make sure it shows up in lspci. When running sudo lspci -vv, mine reports part number 49Y7941 and FRU 49Y7942.
3. Install the script as /opt/initialize-emulex/initialize-emulex.sh (feel free to pick another location, but remember to adjust the unit file)
4. Install the systemd unit file as /etc/systemd/system/emulex-initialization.service
5. Reload systemd and enable the file: systemctl daemon-reload && systemctl enable emulex-initialization.service
6. reboot the system

Your interface should now be automatically up&running after booting.

emulex-initialization.service

[Unit]
Description=Initialize the emulex card by running a self-test

[Service]
Type=oneshot
ExecStart=/bin/bash /opt/initialize-emulex/initialize-emulex.sh

[Install]
WantedBy=network.target

initialize-emulex.sh

# Thanks to Jurriaan: https://jurriaan.pru.is/2020/ten-gigabit-fiber-ethernet-home-setup-on-a-budget/
# This script gets all be2net interfaces and runs the self-test on them
# For my Emulex OCe11102-N-XI (IBM branded, FRU 49Y7942) this does the trick and brings the Ethernet interface up.

set -eu

# Set this to a list of cards to initialize. Else the script tries to guess.
iface_list=""

# Method 1: Guess the right way - though which only seems to work later during booting :(
if [ "$iface_list" == "" ]; then
    echo "Checking /sys for be2net cards..."
    for x in /sys/module/be2net/drivers/pci\:be2net/*/net/*; do
        iface=$(basename $x)
        iface_list="$iface_list $iface"
    done
fi

# Method 2: guess the bad way: assume that any card with a mac starting with 00:00:c9 is an emulex
if [ "$iface_list" == "*" ]; then
    echo "Falling back to parsing output of ip util..."
    iface_list=""
    for iface in $(ip link | grep '^[0-9]\+: ' | sed -e 's@^[0-9]\+: @@' -e 's@: <.*@@'); do
        if [ "$iface" == "lo" ]; then
        continue;
    fi
    
    if ip link show $iface | grep "ether 00:00:c9" >/dev/null 2>/dev/null; then
        iface_list="$iface_list $iface"
    fi
    done
fi

if [ "$iface_list" == "" ]; then
    echo "Did not detect an emulex card."
    exit 0
fi

# Run self tests
for iface in $iface_list; do
    echo "Initializing interface $iface by running self-test..."
    ethtool -t $iface || true
done