Quaint HP Servers in 2023

Acquired a few old HP servers for home lab.. this is to document their journey.

Challenge:

Started without using a monitor and a keyboard.
No access to any KVM over IP, or similar tools

Decided to use the constraints and turn it into a fun learning exercise.

Power On and security override

Plugged servers in. And connected the ethernet port of the iLO to my network to be able to manage them.
Override systems maintenance switch to be able to reset passwords. HP servers have a diagram inside the cover of the server that shows how to do this.. Example

Network and connecting to ILO

Use my network to find the current IPs of the iLO and connect to them by getting an IP on the same network
My network switch leaked the IP of the devices connected to it, it was then really simple to set a static IP on the same network to access the ILO.

ERR_SSL_VERSION_OR_CIPHER_MISMATCH

Constraints:

Accessing the iLO from a browser: http://$ILO_IP/, yields the following error.
```
    ERR_SSL_VERSION_OR_CIPHER_MISMATCH
```

Workaround:

Using a browser from a really old iso (centos 6 live CD) that will accept TLS 1.0.
Decided not to use this workaround, instead will use HP's SMash and scripting tools (as I find out about them).
For iLO4, after the upgrade I've been able to access the iLO through the browser.

Protocol -> TLS 1.2
Key exchange -> ECDHE_RSA with P-384
Server signature -> RSA with SHA-512
Cipher -> AES_128_GCM

No Matching Keys

Constraints:

trying to ssh into the ilo the first time and go the following errors:

no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
no matching host key type found. Their offer: ssh-dss

Workaround:

simply ssh by offering what the server is asking for.

    KexAlgorithms +diffie-hellman-group14-sha1
    HostKeyAlgorithms +ssh-dss

Server Type

Once ssh is successful, let's learn about the 2 servers.

poking around I was able to get information about the firmware and names. Here are the commands I used

show /system1/firmware1
show /system1 name
show /map1 name

Server (1) Wkabi

    name=iLO 3 Advanced
    name=ProLiant DL360 G7
    version=P68

Server (2) Mbaku

    name=iLO 4
    name=ProLiant DL380p Gen8
    version=P70

Documents

Here is full documentation on iLO3 and iLO4

Configure server Hostname and DHCP

Once we've poked around, let's set the configure the iLOs with DHCP and set their hostname. I reuse the iLO SN.

cd /map1/dnsendpt1
set Hostname ILOXXXXXX

cd /map1/dhcpendpt1
set EnabledState yes

Now the iLOs can be addressed by name and managed easily managed.

SSH

Create config file for ILO

Host ILOUSEXXXXXXX
    KexAlgorithms +diffie-hellman-group14-sha1
    HostKeyAlgorithms +ssh-dss
    User Administrator
    IdentityFile ~/.ssh/$SSH_USER@$SSH_HOST


Host ILO2M2XXXXXXX
    KexAlgorithms +diffie-hellman-group14-sha1
    HostKeyAlgorithms +ssh-dss
    User Administrator
    Ciphers +aes256-cbc
    IdentityFile ~/.ssh/$SSH_USER@$SSH_HOST

Generate Keys

SSH_HOST=ILO2M2XXXXXXX
SSH_USER=Administrator
ssh-keygen -t dsa -f $SSH_USER@$SSH_HOST -b 1024 -P '' -f ~/.ssh/$SSH_USER@$SSH_HOST

SSH_HOST=ILOUSEXXXXXXX
SSH_USER=Administrator
ssh-keygen -t dsa -f $SSH_USER@$SSH_HOST -b 1024 -P '' -f ~/.ssh/$SSH_USER@$SSH_HOST

Add Keys ~/.ssh/$SSH_USER@$SSH_HOST.pub to the portal

<RIBCL VERSION="2.0">
   <LOGIN USER_LOGIN="adminname" PASSWORD="password">
      <RIB_INFO MODE="write">
         <IMPORT_SSH_KEY>
-----BEGIN SSH KEY-----
ssh-dss
ContentOfYourSSHKey== USER
-----END SSH KEY-----
         </IMPORT_SSH_KEY>
      </RIB_INFO>
   </LOGIN>
</RIBCL>

Firmware upgrade

Download files and extract them
- We use the HP website to find each server's iLO files and BIOS upgrade files.
- For Linux components: Depending on the file format, enter one of the following commands:
```
#sh ./CP00XXXX.scexe --unpack=/tmp/
#rpm2cpio hp-firmware-ilo4-2.xx-1x1.i386.rpm | cpio -id
```

iLO

Download iLO files
- Firmware CD Supplemental Update / Online ROM Flash Component for Linux - HPE Integrated Lights-Out 3
- Online ROM Flash Component for Linux - HPE Integrated Lights-Out 4

Extract iLO files

for iLO4

sh ./CP051872.scexe --unpack=/tmp/iLO4
ls /tmp/iLO4 | grep .bin
ilo4_281.bin

for iLO3

rpm2cpio firmware-ilo3-1.94-1.1.i386.rpm | cpio -id
ls usr/lib/i386-linux-gnu/firmware-ilo3-1.94-1.1/ | grep .bin
ilo3_194.bin

Install iLOfw.bin

load /map1/firmware1 -source http://$HOST/$ILO_BINARY.bin

Verify the installed versions

For iLO4 Before

/map1/firmware1
Targets
Properties
    version=1.94
    date=Dec 06 2020

For iLO4 After

/map1/firmware1
Targets
Properties
    version=2.81
    date=Jul 27 2022
    name=iLO 4

ROM BIOS

Download ROM BIOS
- Online ROM Flash Component for Linux - HP ProLiant DL360 G7 (P68) Servers
- Online ROM Flash Component for Linux - HP ProLiant DL380p Gen8 (P70) Servers

Extract BIOS ROM

for P68

rpm2cpio firmware-system-p68-2018.05.21-1.1.i386.rpm | cpio -id 
3846 blocks
ls ./usr/lib/i386-linux-gnu/firmware-system-p68-2018.05.21-1.1/
CP036591.xml  CPQP6803.4B5  cpqsfp68.cfg  hpsetup

for P70

rpm2cpio firmware-system-p70-2019.05.24-1.1.i386.rpm | cpio -id 
10038 blocks
kenzo@ThinkyT420:~/Downloads$ ls ./usr/lib/i386-linux-gnu/firmware-system-p70-2019.05.24-1.1/
CP040097.xml  CPQP7013.6B8  cpqsfp70.cfg  hpsetup  setup