Skip to content

Instantly share code, notes, and snippets.

@TimCook1
Forked from FlorianHeigl/01_VEP4600 Infos.md
Last active December 25, 2024 23:04
Show Gist options
  • Save TimCook1/504dc76caacb8f8551886b3d7ce8d8d7 to your computer and use it in GitHub Desktop.
Save TimCook1/504dc76caacb8f8551886b3d7ce8d8d7 to your computer and use it in GitHub Desktop.
VEP4600

There are two user levels

User

  • this is the boot password
  • if it is set, the system only starts with the password)

Admin

  • this is the BIOS password
  • is requested when calling up the BIOS
  • if you just press "enter" you get into the BIOS, but you do not have the "admin" user level
  • the password should normally be SERVICETAG!
  • on the puller with the QR codes at the top right of the front of the case
  • the service tag can also be found on a sticker on the lid

DiagOS itself can be updated using a .deb package

FW Updates

updatetool --dev=BIOS --update -e filename.img
updatetool --dev=CPLD --update -e filename.jed
updatetool --dev=BMC --update -e filename.bmc

If you fail with the BMC password (serial port), the BMC update is a pretty good solution

DiagOS does not get a new IP from the DHCP server if you unplug and plug in the LAN port.

Examples

Preparation

  • Unpack updates / identify files (see below)
  • Log in to DiagOS using TTY, identify active LAN port, read IP.
  • Get new IP via ifdown / ifup if necessary
  • Copy the update files into the booted DiagOS via scp

BMC update

root@dellemc-diag-os:~# updatetool --dev=BMC --update -e VEP4600-BMC-v2.30.ima
Disable device protect
Write image to BMC
INFO: Yafu INI Configuration File not found... Default options will not be applied...

Creating IPMI session via USB...Done

-------------------------------------------------
YAFUFlash - Firmware Upgrade Utility (Version 5.0.0)

-------------------------------------------------
(C)Copyright 2016, American Megatrends Inc.
Image To be updated is (Image-1)
The Module boot major or minor version is different from the one in the image
So,Type (Y/y) to do Full Firmware Upgrade or (N/n) to exit
Enter your option: y
****************************************************************************
 WARNING!
 FIRMWARE UPGRADE MUST NOT BE INTERRUPTED ONCE IT IS STARTED.
 PLEASE DO NOT USE THIS FLASH TOOL FROM THE REDIRECTION CONSOLE.
****************************************************************************
Preserving Env Variables... done
Uploading Firmware Image: 100%... done
Flashing [boot] modules....
Flashing Firmware Image: 100%... done
Verifying Firmware Image: 100%... done
Flashing [conf] modules....
Flashing Firmware Image: 100%... done
Verifying Firmware Image: 100%... done
Flashing [root] modules....
...
Flashing [ast2500e] modules....
Flashing Firmware Image: 100%... done
Verifying Firmware Image: 100%... done
Setting Env variables... done
Resetting the firmware.........
write BMC image success
Enable device protection

Update BMC image success

During the BMC update, the fan control goes off briefly (Full)

###CPLD

This one probably doesn't functions

root@dellemc-diag-os:~# updatetool --dev=CPLD --update -e AZUL_CPLD_V12_20210520.jed

-------------------------------------------------
YAFUFlash - Firmware Upgrade Utility (Version 5.0.0)

-------------------------------------------------
(C)Copyright 2016, American Megatrends Inc.
Beginning CPLD Update...
Uploading Image: 100%... done
Flashing Firmware Image: 0%Error in ECFStatus

Error in updating CPLD image
write CPLD image success
Enable device protection

Update CPLD image success

BIOS

BIOS update does not ask for Yes/No

root@dellemc-diag-os:~# updatetool --dev=BIOS --update -e VEP4600-BIOS-3.41.0.9-23.BIN
Write image to BIOS
+---------------------------------------------------------------------------------+
| AMI Firmware Update Utility Manufacture for Dell Nv5.11.06.1854|
| Copyright (C)2018 American Megatrends Inc. All Rights Reserved. |
+---------------------------------------------------------------------------------+
 Reading flash ............done
 - ME Data Size checking. OK
 - FFS checksums ......... ok
 - Check RomLayout ......... ok.
 Erasing Main Block .......... done
 Updating Main Block ......... 0x0028C000 (13%)
 ...
 - Update success for MER
WARNING : System must power-off to have the changes take effect!
write BIOS image success
Update BIOS image success

At the end, shut down the system with poweroff - or at some point later. In the case of the CPLD update error, it is best to completely disconnect the power. (It worked like that, but I didn't try it without disconnecting the power.)

Overview / Description:

https://infohub.delltechnologies.com/p/dell-technologies-vep4600-bios-firmware-upgrade/

DiagOS and Unified FW Updater

https://www.dell.com/support/manuals/de-de/virtual-edge-platform-4600-8c/vep4600_os_install_pub/run-the-unified-firmware-updater-tool-from-a-linux-server?guid=guid-556d049c-9a4f-4784-99f2-8678a8eebed9&lang=en-us

DiagOS Partitioning issues.issues

[Error Description](https://www.dell.com/support/manuals/de-de/virtual-edge-platform-4600-4c/rns-vep4600-di ag-pub/diag-os-installation-failure-and-resolution?guid=guid-89e5b8bd-a3a2-4633-b69c-4d9b415cfc8e&lang=en-us)

Repartitioning

Real solution was: DiagOS with Rufus on a fresh USB stick. Boot this in UEFI mode.

Instructions: https://www.dell.com/support/manuals/de-de/virtual-edge-platform-4600-8c/ucpe_diag_pub/restore-to-manufacture-diag-os?guid=guid-e0457706-c0c2-4407-8aad-f3b788f53217


Release Notes current version 3.8

Release Notes VEP 4600 DiagOS

Download Link Drivers and Tools, UFW


System for work:

  • Ubuntu 20: Works
  • CentOS Stream: Doesn't work!

Software works normally in a RAM disk, so allocate enough RAM for VM.

Unpack all zip files the ufw_v.x is the binary, call with

./ufwding 192.168.86.xx interactive

NOTE If an IPMI session error is seen while connecting to the BMC, run following command to reset the BMC prior to re-run of UFW.

Creating IPMI session via network with address 192.168.86.xx...Failed

root@ubs18-04:~/# ipmitool -I lanplus -H xxx.xx.xx.xxx -U admin -P admin mc reset cold

need reset!!!!!

[...]
 9. Reset BMC after CPLD update completion and system bootup
 q. Exit



 Enter your choice:9
Reset BMC, in case of BMC is not in correct status (e.g. BMC in Firmware Update Mode will cause firmware update that through CFU failed)
Sending reset to BMC to recover, need to wait for 180 seconds

Reset can also be seen on the BMC console:

[ 9899.290059] ipmi_si IPI0001:00: Maybe ok, but ipmi might run very slowly.

Reset via menu does NOT help

updating BMC image

****************************
* BMC version = 2.0 *
* Backup version = 2.0 *
****************************
updating primary image only..
INFO: Yafu INI Configuration File not found... Default options will not be applied...

Creating IPMI session via network with address 192.168.xx.100...Failed
Image update failed
Upgrade Failed & Exiting
/home/flea/VEP4600/temp

Plugging in the cable seems to search for IP Plugging a cable triggers DHCP refresh.

~ # DHCP monitor: Re[ 366.450000] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
newing eth0 interface for IPv6
RTNETLINK answers: File exists
Sending ICMPv6 packet: Cannot assign requested address
Renewing DNS for eth0 interface.
DHCP monitor: Renewing eth0 interface for IPv6
RTNETLINK answers: File exists
Sending ICMPv6 packet: Cannot assign requested address
Renewing DNS for eth0 interface.

no working DHCP observed

https://www.dell.com/support/manuals/de-de/virtual-edge-platform-4600-8c/vep4600_os_install_pub/configure-bmc-management-address-in-bios?guid=guid-f6405c55-60af- 45ad-8a28-0e0b7f46c4f2&lang=en-us

From the BMC console window, run the following commands to boot into the BIOS:

ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin chassis bootparam set bootflag force_bios
~ # ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin power reset
Chassis power control: Reset
~ # RESET CHASSIS
[ 653.800000] LPC RESET
PDK LPC Reset is invoked

You will definitely have to switch serial ports

Note: ipmi user/pw != bmc

According to the manual, you have to do everything via BIOS.

But it also works with ipmitool locally

ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin lan set 1 ipaddr 10.0.0.1
41~ # ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin lan set 1 netmask 255.255
.255.0
~ # ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin lan set 1 defgw ipaddr 10.0.0.1

https://www.dell.com/support/kbdoc/de-de/000112188/how-to-access-console-port-of-dell-emc-networking-vep4600

Management Port Overview

Port names are in clock-wise order

image

Port Description
Port A Serial Console Port to Access CPU
Port B Serial Console Port to Access BMC
Port C Management Port to Access CPU (igb4)
Port D Management Port to Access BMC
Port E USB-B Console port to Access CPU

If a cable is connected to Port E (referenced above) even if not in use, the CPU RJ45 (Port A) console port will not work.

Port C is an Intel i210 desktop NIC with reduced features intended for management purposes only. The other 1g ports use i350 server NICs. Rarely will the i210 port be enumerated as the first NIC

Access and Default Settings

Accessing the console ports:

Port A/E - Serial/USB-B Console Port to access CPU
 User Name: root
 Password: calvin

The above login credentials are only for the default installed DIAG OS When installed with any other OS, follows vendor directions.

Port B - Serial Console Port to access BMC
 User Name: sysadmin
 Password: superuser

In certain versions the BMC login is different:

 User Name: sysadmin
 Password: SERVICE DAY!
Accessing BMC through HTTPS
 User Name: admin
 Password: admin
Default console settings
Baud rate: 115200
Data Bits: 8
Stop bits: 1
Parity: None
Flow Control: None

Login example:

BMCXXXX login: sysadmin
Password:
[1446 : 1446 INFO]SERIAL Login from IP:127.0.0.1 user:sysadmin

IP setup?

3: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
 link/ether XXXX brd ff:ff:ff:ff:ff:ff
 inet 192.168.0.1/24 brd 192.168.0.255 scope global eth0

image

  1. Power LED
  2. Primary unit indicator LED
  3. System LED
  4. Locator LED
  5. Temperature LED
  6. Fan LED
  7. SFP+ indicator LED
  8. 10/100/1000 BaseT RJ-45 networking link (left) and activity (right) LEDs
  9. 10/100/1000 BaseT RJ-45 networking link (left) and activity (right) LEDs for the processor (left) and for the BMC (right)

LED codes im Dell EMC VEP4600 Installation Guide

Typical configuration without rNDC module

image

Ports assigment:

id/os name id/os name
"9"/igb4 bmc
id/os name id/os name
3/igb1 5/igb3
4/igb0 6/igb2
id/os name id/os name
1 ix0
2 ix1

Internal Ports

There are two more ports facing the internal backplane for 'service chaining' use cases. Those are the ones labeled "for backplane" in the overview

Network cards

SR-IOV is active in the BIOS by default. VT-d etc. are also available

Linux

SR-IOV Settings

  • i210 onboard: Management port and chip do not support it
  • i350 onboard: SR-IOV deactivated and cannot be configured in the BIOS
  • i7xx onboard: SR-IOV deactivated and cannot be configured in the BIOS
  • i7xx on rNDC: SR-IOV can be configured in the BIOS

There are instructions for this

Link?

PCI Passthrough

When using passthrough, only 1 device out of 4 was passed on, the others were then "gone"

VFs

from VMware documentation below (running on KVM)

  • a. Set the spoofcheck off. # ip link set eth1 vf 0 spoofchk off
  • b. Set the Trusted mode on. # ip link set dev eth1 vf 0 trust on
  • c. Set the VLAN, if required. # ip link set eth1 vf 0 vlan 3500
  • d. disable GRO on host # ethtool –K <interface> gro off tx off

ESXi

To check

  • There is a NIC FW update - but only for VMware

It is also mentioned in the VMware SD-WAN guide, so it is very possible that this would help. https://docs.vmware.com/en/VMware-SD-WAN/4.0/vmware-sd-wan-partner-guide.pdf

Security Update

https://www.dell.com/support/kbdoc/en-us/000215718/dsa-2023-251-security-update-for-dell-networking-vep4600-for-intel-ethernet-controllers-and-adapters-advisory-intel-ta-00593

Product Software/Firmware Affected Versions Remediated Versions Link Dell Networking VEP4600 16-CORE Firmware Versions prior to 22.0.9 Version 22.0.9 or later https://www.dell.com/support/home/en-ca/drivers/driversdetails?driverid=9nppg

Relnotes: https://dl.dell.com/FOLDER09675541M/4/fw_release_x710.txt?uid=67a7d282-d9f6-4eb3-56c8-0e3fa7a02476&fn=fw_release_x710.txt

FW update!!! (for the 710)) https://dl.dell.com/FOLDER10201251M/1/vep4600-x722-fw-rn.pdf?uid=d6f44b92-fd82-4244-40bf-353f8d1f97ae&fn=vep4600-x722-fw-rn.pdf

https://dl.dell.com/FOLDER10201235M/1/X722_V6_20_Linux.tar.gz

See the resolved issues here, a lot of things fit there...

Proxmox thread regarding i350 passthrough

https://forum.proxmox.com/threads/nic-card-passthrough-and-essential-things-to-think-about.80805/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment