OpenBSD octeon notes

These are my personal notes on how to run/use OpenBSD on Ubiquiti Edgerouter, specifically for Edgerouter 4. I installed OpenBSD bunch of releases back and keep upgrading. Perhaps some of the issues highlighted below would not happen on fresher install.

Upgrade

The upgrade should be done one release at a time, i.e. from x.y to x.(y+1).

Note: has to be done using serial console (in order to access U-boot)

download the sets (had to use Cloudflare URL since the previous one artfiles.org was flaky and resulted in connect - no route to host)

sysupgrade -n -k https://cloudflare.cdn.openbsd.org/pub/OpenBSD/

The -k option is used to keep the files in the /home/_sysupgrade/ directory as the install process boots into the wrong kernel.

Remove unwanted sets (the installer will cope with this just fine):

rm /home/_sysupgrade/x*.tgz /home/_sysupgrade/game*.tgz

replace bsd.rd and bsd in the U-Boot partition with the one downloaded.

From https://ftp.openbsd.org/pub/OpenBSD/7.1/octeon/INSTALL.octeon

U-Boot partitions defined on the disk will usually show up as partition 'i', 'j' and so on.

This has to be done by replacing because the space on the partition is very limited (it has some 32 MB in total and the kernel has around 8 MB).

mount /dev/sd0i /mnt
cp /home/_sysupgrade/bsd.mp /mnt/bsd
cp /home/_sysupgrade/bsd.rd /mnt/bsd.rd
umount /mnt

Note: It would be probably wiser to replace the /bsd only after the installer finished successfully.

reboot into the installer

in U-boot:

usb start
fatload usb 0 $loadaddr bsd.rd
bootoctlinux rootdev=sd0

Then follow the installer instructions for upgrade. In the recent versions this uses unattended install so it will proceed automatically.

Note: for some reason the extraction of the tarballs takes really long time - many minutes. The speed is something like hundreds of kilobytes per second. Sometimes the extraction stalls completely. The used USB stick is most likely the reason (Sandisk Cruzer Fit 8GB, although it happens also on the newer Verbatim STORE N GO 8GB). The Samsung FIT Plus drive has also pretty small factor however performs much better. Using dd bs=$( echo '1024 * 1024' | bc ) (i.e. 1 MB block size for both read and write) with the 64 GB variant, I was able to write 8 GB image to the flash drive in cca 300 seconds. Using even bigger block size values yields no significant improvement.

cleanup

rm -f /home/_sysupgrade/*

merge configuration that could not be merged automatically in the unattended upgrade:

sysmerge

update packages

Note: make sure to set the correct date/time. Edgerouters lack RTC so the time may be way off, which might throw OCSP/certificate verification that is needed when establishing TLS connection to the package servers.

echo 'https://cloudflare.cdn.openbsd.org/pub/OpenBSD' > /etc/installurl
pkg_add -v -u

The binary packages seem to come from the mips64 flavor, e.g. https://cloudflare.cdn.openbsd.org/pub/OpenBSD/7.7/packages/mips64/

check

Make sure top reports 4 CPUs.

See if there are any changes requiring manual fixups

e.g. for upgrade from 7.6 to 7.7 see https://www.openbsd.org/faq/upgrade77.html

Esp. the section on file removal.

UPS setup

After experiencing a power outage that resulted in corrupted flash and the router unable to boot (it got stuck in ddb due to kernel panic when mounting the root file system), it was clear that a UPS is needed. THe other alternative would be read-only root (ran the solution by Chris Cappuccio for a long time on my Soekris boxes) however the UPS solution is easier.

The UPS is connected to the Edgerouter via USB. Since Edgerouter 4 has just one USB port, a USB hub is used to connect both the flash drive with the system as well as the UPS.

Important things to do / consider when configuring:

see the values reported by the sensor: sysctl -A | grep upd0
enable sensorsd in /etc/rc.conf.local (not /etc/rc.conf, this might be lost on upgrade)
test the shutdown script by running sensorsd in foreground: sensorsd -d -c 1

/etc/sensorsd.conf contents:

# Every sensor update below 30%, run the following script.
# If the sensor value is below a value specified in the script, then gracefully shut down the system.
hw.sensors.upd0.percent0:low=30.00%:command=/root/scripts/ups-shutdown.sh %2

The script can look like this:

#!/bin/ksh

if (( $# != 1 )); then
	echo "Missing argument (needs percentage value)"
	exit
fi

integer percentage=$( echo $1 | cut -d. -f1 )
echo "Current percentage: $percentage"
if (( percentage < 20 )); then
	echo "Shutting down"
	/sbin/shutdown now
fi

Links:

Recovery of corrupted file-system

If the sd0 get corrupted because of e.g. power outage, boot into the installer from the UBOOT prompt:

usb start; fatload usb 0 $loadaddr bsd.rd
bootoctlinux

enter Shell and create the device nodes:

cd /dev
sh MAKEDEV sd0

and fix the file-system:

fsck -y sd0a

vladak/openbsd-octeon.md

OpenBSD octeon notes

Upgrade

Links

UPS setup

Recovery of corrupted file-system