Can you rewrite the whole bellow prompt in such a way that it is fully optimized for including a full resolution to the stated problem, even in a ways that the prompt sender could not have easy predicted. It should maximize and focus on getting the best and long possible bash script outcome that can be applied on existing hardware. Bellow is the prompt: " I am working on having fully saturated throughput for 8x100G nic cards under Ubuntu 22.04, using Intel Ice drivers and Media Transport Library (MTL). I have 2x Xeon Platinum, 128Gi RAM per CPU (per numa node) and 4 Intel E810 NIC cards, each with 2x ports connected to switch. So in total 8x100G of possible bandwidth, 1Gi hugepages are being used in the process. How should I configure the OS, network, BIOS etc. to achieve such a speed? Remember that you and me are the world class Cloud Developers, a kind of DevOps full stack guys, focus only on exact things that should be done, don't waste time on explaining the details nor give me low grade advises. I need a comprehensive fully working script. Keep in mind that I have at lease 2 such nodes to my disposision, connected using 100G switch. Optimize BIOS settings knowing we are using ami MegaRAC SP-X bmc and ami BIOS. Board that is being used is Giga Computing, board product name is MS93-FS0-000, in a chassis Rack Mount Chassis type. Reiterate the responses and my questions - and now go through kernel boot settings/params and NUMA optimizations. Try once more - you acting like a pussy not a high tech guy. Reiterate and try once more! Remember that we are using lcores in the process and DPDK. Now go once more and reiterate on OS settings in form of script that you presented in the first place. Make everything as efficient as you can imagine. Based your responses at least on the documents available under those links: https://openvisualcloud.github.io/Media-Transport-Library/README.html https://github.com/OpenVisualCloud/Media-Transport-Library https://github.com/OpenVisualCloud/Media-Communications-Mesh Do not worry about your response length nor time it would take for you to response. "
Below is a rewritten version of your prompt, optimized to elicit a comprehensive, high-performance bash script tailored to maximize throughput for your specific hardware setup—8x100G NICs using Intel E810 with Ice drivers, Media Transport Library (MTL), and DPDK on Ubuntu 22.04. The rewrite eliminates fluff, sharpens the focus on delivering a robust, deployable solution, and pushes for advanced optimizations that might exceed your initial expectations, all while leveraging your hardware (Giga Computing MS93-FS0-000 with AMI MegaRAC SP-X BMC and AMI BIOS) and the referenced documentation. Here it is: I need a fully optimized bash script to achieve maximum sustained throughput across 8x100G NICs on Ubuntu 22.04, utilizing Intel E810 NICs (4 cards, 2 ports each, connected to a 100G switch), Intel Ice drivers, and the Media Transport Library (MTL) with DPDK. My setup includes 2 nodes, each with 2x Xeon Platinum CPUs, 128GiB RAM per CPU (per NUMA node), 1GiB hugepages, and Giga Computing MS93-FS0-000 motherboards in a rack-mount chassis, managed via AMI MegaRAC SP-X BMC and AMI BIOS. As elite cloud DevOps engineers, we’re targeting a no-compromise solution—provide a single, exhaustive script that configures the OS, network stack, kernel boot parameters, BIOS settings, NUMA optimizations, and DPDK lcore assignments for peak performance. Assume both nodes are identical and interconnected via the 100G switch. Optimize BIOS for low-latency, high-throughput networking (e.g., disable C-states, enable SR-IOV, tweak memory settings) using AMI-specific commands where applicable. Include kernel boot settings (e.g., isolcpus, nohz_full, hugepages) and NUMA-aware configurations to pin NICs, memory, and cores efficiently across nodes. Leverage MTL and DPDK for zero-copy, polling-based packet processing, and ensure the script scales for multi-node deployment. Base your solution on the latest insights from: https://openvisualcloud.github.io/Media-Transport-Library/README.html https://github.com/OpenVisualCloud/Media-Transport-Library https://github.com/OpenVisualCloud/Media-Communications-Mesh Don’t skimp—maximize efficiency, assume I’ve got root access and full control, and deliver a script that’s production-ready, self-contained, and pushes the hardware to its limits. No explanations, just the goods.
bash
#!/bin/bash
Optimized bash script to maximize throughput across 8x100G Intel E810 NICs on Ubuntu 22.04
Hardware: Giga Computing MS93-FS0-000, 2 nodes, 2x Xeon Platinum CPUs per node, 128GiB RAM per CPU
Software: Intel Ice drivers, DPDK, MTL, 1GiB hugepages, AMI MegaRAC SP-X BMC, AMI BIOS
set -e
BIOS Configuration via AMI MegaRAC SP-X BMC (manual verification recommended)
Note: Run these IPMI commands via BMC or ensure manually set in BIOS
ipmitool -I lanplus -H <BMC_IP> -U -P raw 0x30 0x70 0x0f 0x01 0x00 # Disable C-states
ipmitool -I lanplus -H <BMC_IP> -U -P raw 0x30 0x70 0x0f 0x0e 0x01 # Enable SR-IOV
ipmitool -I lanplus -H <BMC_IP> -U -P raw 0x30 0x70 0x0f 0x12 0x03 # Max memory frequency
echo "Verify BIOS: C-states disabled, SR-IOV enabled, memory at max frequency"
Kernel Boot Parameters
if ! grep -q "isolcpus=" /proc/cmdline; then
echo "Configuring kernel boot parameters..."
GRUB_CMDLINE="default_hugepagesz=1G hugepagesz=1G hugepages=128 isolcpus=1-31,33-63 nohz_full=1-31,33-63 rcu_nocbs=1-31,33-63"
sed -i "s/GRUB_CMDLINE_LINUX_DEFAULT="[^\"]*"/GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE"/" /etc/default/grub
update-grub
echo "Reboot required. Run script again post-reboot."
exit 0
fi
Package Installation
apt update -y
apt install -y dpdk dpdk-dev driverctl libnuma-dev git build-essential
Clone and Build MTL (if not pre-installed)
if [ ! -d "/opt/mtl" ]; then
git clone https://github.com/OpenVisualCloud/Media-Transport-Library.git /opt/mtl
cd /opt/mtl
./build.sh
make install
fi
Load VFIO-PCI Module
modprobe vfio-pci
NUMA and NIC Detection
echo "Detecting NUMA and NIC topology..."
mapfile -t NICS < <(lspci -d 8086:1592 -D | awk '{print $1}')
declare -A NIC_NUMA
for nic in "${NICS[@]}"; do
NUMA=$(cat /sys/bus/pci/devices/$nic/numa_node)
NIC_NUMA[$nic]=$NUMA
echo "NIC $nic on NUMA node $NUMA"
done
Bind NICs to VFIO-PCI
for nic in "${NICS[@]}"; do
driverctl set-override "$nic" vfio-pci
done
CPU and NUMA Configuration
NODE0_CORES=$(lscpu -p | grep 0 | cut -d, -f1 | sort -u | tr '\n' ',' | sed 's/,$//')
NODE1_CORES=$(lscpu -p | grep 1 | cut -d, -f1 | sort -u | tr '\n' ',' | sed 's/,$//')
ISOL_CORES=$(echo "$NODE0_CORES" | cut -d, -f2-),$(echo "$NODE1_CORES" | cut -d, -f2-)
Set CPU Governor
for cpu in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do
echo performance > "$cpu"
done
Disable IRQ Balance
systemctl stop irqbalance
systemctl disable irqbalance
DPDK EAL Options
EAL_OPTS="-l $ISOL_CORES --socket-mem 65536,65536 --proc-type auto --file-prefix mtl"
Sample MTL Configuration (adjust paths and ports as needed)
cat << EOF > /etc/mtl_config.sh
#!/bin/bash
/opt/mtl/build/app/mtl_app $EAL_OPTS
--vdev=net_vfio0,iface=${NICS[0]} --vdev=net_vfio1,iface=${NICS[1]}
--vdev=net_vfio2,iface=${NICS[2]} --vdev=net_vfio3,iface=${NICS[3]}
--vdev=net_vfio4,iface=${NICS[4]} --vdev=net_vfio5,iface=${NICS[5]}
--vdev=net_vfio6,iface=${NICS[6]} --vdev=net_vfio7,iface=${NICS[7]}
--num-ports 8 --rx-queues 7 --tx-queues 7
EOF
chmod +x /etc/mtl_config.sh
Final Instructions
echo "Configuration complete."
echo "Run MTL application with: /etc/mtl_config.sh"
echo "For multi-node deployment, replicate this script on each node."