Flag day: OS-7260 SmartOS should support booting with loader

Hi all, with the integration of:

OS-7260 SmartOS should support booting with loader

and related work, there are changes in the way both SmartOS and Triton boot. See RFD 156 for some more background:

https://github.com/joyent/rfd/tree/master/rfd/0156

In general, unless you are re-installing a system, these changes are unlikely to affect you significantly.

Rob and I would especially like to thank Toomas Soome for all his hard work on the new bootloader in illumos, which we depend on.

TL;DR: make sure you're up to date: git pull, sdcadm experimental update-gz-tools, sdcadm platform, sdcadm update, etc. and you should be fine.

Major changes

If booting SmartOS or Triton from media with these changes (for example, a reflashed USB key), the FreeBSD bootloader ("loader") is now used instead of Grub. This supports booting the system in EFI mode. See below for more details on using loader.

In Triton, if you do reflash the head node or newly install a compute node, you must run sdcadm experimental update-gz-tools on the HN first. You should also make sure that you have assigned a new PI in sdcadm platform. You may also want to update your dhcpd service.

If you do not boot from new media, then a grub-based system will still continue to work.

This project has changes across the following Joyent repos: ipxe, sdcboot, sdc-booter, mountain-gorilla, sdc-headnode, triton, smartos-live, and illumos-joyent. In general, if you want to do builds yourself, you will need to update all the related repositories.

Using loader

To temporarily change boot settings, such as loading KMDB or modifying the default console, you can do so in the Configure Boot [O]ptions menu.

Changes can also be made via the file /boot/loader.conf.local:

console="ttya,text"
platform-version="20190221T163050Z"

In Triton, the new sdc-usbkey set-variable sub-command can be used for setting many boot parameters.

If you would just like to temporarily add free-form boot options, you can drop into the loader CLI ([Esc]ape to loader prompt), then, for example:

boot -m verbose

If you need to add a boot module, for example to modify /etc/system, please see:

https://blog.movementarian.org/2019/02/modifying-boot-files-with-smartos-under.html

For situations where the default boot platform image on the usbkey is not bootable, a common way to recover the system is to temporarily boot from an alternate platform image on the key. However, this can be cumbersome because it requires changing multiple Loader settings. A new "set-platform" Forth word has been added to Loader to simplify the process of temporarily selecting an alternate platform image under Triton. This can be used from the Loader prompt like this:

s" /os/20190319T100929Z" set-platform

In this example, we have a PI located on the key under /os/20190319T100929Z/platform. NB: The space between the opening quote and the pathname is intentional. The Loader command prompt is a Forth interpreter and s" is a Forth word for creating a string and pushing it onto the stack.

iPXE

The Joyent version of iPXE has been updated from upstream, bringing a lot of new features and improvements, including new support for the following devices:

Solarflare X2522-10, X2522-25, X2541 and X2542 NICs
Solarflare SFC8XXX NICs
Intel Fortville NICs
Intel X552/X553 NIC
Intel i354 NIC
Intel i219v, 219lm-3, i219lm-4, i219lm-5, i219v-5
Exablaze ExaNIC X40g
Microchip LAN78XX USB ethernet adapters

In addition, Joyent's version of iPXE now supports booting SmartOS/Triton in EFI mode. An updated USB key will contain this version of iPXE. Alternatively, the head node's dhcpd instance should be updated if you want to PXE boot a compute node that lacks a USB key.

Note that the freedos boot image has been retired.

Developer notes

You do not need to read any of this as a user/operator of SmartOS or Triton.

smartos-live

tools/build-iso and tools/build-usb have been merged into a single script, tools/build_boot_image, which can build both USB and ISO SmartOS images, as well as the pro-forma Triton image used as input to an sdc-headnode build. Only one 4Gb pro-forma image is now created.

illumos-joyent

The fix for RICHMOND-16 mitigation ensnares stock loaders should now avoid problems with booting from a non-RICHMOND-16-enabled boot loader.

A new disk_label MDB module provides some basic support for interrogating disks and disk images.

sdc-headnode

With these changes, you must match the version of sdc-headnode with the platform bits (that is, to build with loader support, you must have an up to date sdc-headnode, and vice versa).

mount-usb.sh has gone: use sdc-usbkey mount.

You must use a current version of bin/reflash to flash a loader-based USB image. To re-iterate, you must have run sdcadm experimental update-gz-tools first on the head node you're reflashing.

There's no tool currently to convert a USB key from the old to the new format without a factory reset.

The build.spec setting default-boot-option is now controlled instead by the ipxe setting.

sdcboot / ipxe / mountain-gorilla

The sdcboot repository is now retired. Instead, the ipxe repository delivers these bits. It has also been converted to the new build system.

jlevon/newboot-flagday.md