Skip to content

Instantly share code, notes, and snippets.

@mmzeeman
Last active September 1, 2026 08:54
Show Gist options
  • Select an option

  • Save mmzeeman/eb682017fe2677ecb2f22124d4645902 to your computer and use it in GitHub Desktop.

Select an option

Save mmzeeman/eb682017fe2677ecb2f22124d4645902 to your computer and use it in GitHub Desktop.
Redundant ZFS data and boot

Goal

Setup a Linux with Ubuntu 26.04 with two normal SSD disks which are mirrored with ZFS.

There are two physical disks.

Each disk has:

  • its own EFI System Partition (ESP)
  • a member of the mirrored bpool
  • a member of the mirrored rpool

The important boot redundancy is:

                 ┌── Disk 1 ESP ── GRUB ──┐
UEFI ────────────┤                         ├── mirrored bpool
                 └── Disk 2 ESP ── GRUB ──┘
                                             │
                                      mirrored rpool
                                             │
                                           Ubuntu

Either disk can therefore provide the EFI bootloader and the corresponding ZFS mirror members.

Howto

The disks should be the same size, or if one of the disks is larger, use the smaller disk to do the initial installation.

Boot from USB to install Ubuntu 26.04

Go through the installation steps like, language, keyboard layout, internet untill you get to the install part. Choose "interactive", "erase and install". Select the smallest disk to install the OS on. Go to "Advanced Options" and choose ZFS with or without encryption, depending on your needs. Choose a computer and username.

Wait until the OS is installed.

After the initial installation you can run zpool status. You will see two pools bpool, where the boot related items are stored and an rpool where the user data will live. Later it is possible to createn filesystems for specific needs on the pools.

Copy the partition table from disk 1 to disk 2

Now we are going to mirror both pools in order to get data mirroring.

Login and copy the partition table of disk 1 to disk 2..

In the example I use /dev/sda as disk 1 and /dev/sdb as disk 2

sudo sgdisk --backup=partition-table.sgdisk /dev/sda
sudo sgdisk --load-backup=partition-table.sgdisk /dev/sdb
sudo sgdisk --randomize-guid /dev/sdb

Now this disk 2 is partitioned in the same way as disk 1 and is configured something like this:

$ sudo sgdisk -p /dev/sda
Disk /dev/sda: 976773168 sectors, 465.8 GiB
Model: ST3500630AS
Sector size (Logical/physical): 512/512 bytes
Disk identifier (GUID): C66737AC-BEBD-43D0-A7C4-82461897A0BC
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 2048, last usable sector is 976773134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2063 sectors (1.0 M1B)

Number Start (sector) End (sector)   Size      Code Name
1               2048     2203647   1.0 GiB     EF0Q
2            2203648     6397951   2.0 GiB     8300
3            6397952    14786559   4.0 GiB     8200
4           14786560.  976771071 458.7 GiB     8300

Add partition 2 and partition 4 as mirrors to the zfs pools

After installing ubuntu, with zfs boot, you usually have two ZFS pools. After copying the partition table to disk 2 you can add these partitions to the bpool and the rpool.

First you have to find out what de ids of the disks are. It is important to use these ids, because this makes it possible to connect the device to a different internal port, without the configuration going bad because of hard code device names.

You can find out the ids of the devices by running the following command:

$ ls -la /dev/disk/by-id/

You will see which partition already have been attached to the pools by running

$ zpool status

This will display an overview of the disks..

Now you can attach the right partitions to the right pool by running:

sudo zpool attach bpool <id-part-disk-1> <id-part-disk-2>
sudo zpool attach rpool <id-part-disk-1> <id-part-disk-2>

Check the mirroring status by running

$ zpool status

The bpool and rpool data is now mirrored on the other disk.

Redundant Swap Partition

Add the swap partition to /etc/fstab. It is important to use the id, and use sw,nofail as option. This makes it possible to continue booting when one of the swap partitions is no longer available.

/dev/disk/by-uuid/<SWAP1-UUID> none swap sw,nofail 0 0 
/dev/disk/by-uuid/<SWAP2-UUID> none swap sw,nofail 0 0

After changing fstab

sudo systemctl daemon-reload

Redundant Boot Setup

Setting up the EFI partitions

Booting is done from an efi partition. In order to setup redundant booting efi partition on the second disk needs to be formatted.

sudo mkfs.fat -F 32 /dev/<ESP-partition>

After this you can configure grub to use both esp partitions for booting.

sudo dpkg-reconfigure grub-efi-amd64

During the setup you will get an option to select both partitions to install grub. After doing this grub will automatically update grub when there are changens.

Dracut configuration

This was the important part of making the boot process independent of a particular physical disk.

Create: /etc/dracut.conf.d/99-zfs-redundant-boot.conf

containing:

hostonly="no"

This will keep all device ids out of the initial ramdisk and make rebooting truely redundant.

With host-only dracut generation, the initramfs contained a dependency on the ESP UUID: 69E9-84AC

The initramfs contained entries such as: etc/systemd/system/initrd.target.wants/dev-disk-by\x2duuid-69E9\x2d84AC.device and: var/lib/dracut/hooks/initqueue/finished/devexists-/dev/disk/by-uuid/69E9-84AC.sh

This caused dracut to wait for that particular ESP when its physical disk was disconnected. Changing the fstab entries to noauto,nofail did not remove the dependency from the generated initramfs.

Setting up fstab for the efi partitions

/dev/disk/by-uuid/ /boot/efi vfat defaults,nofail 0 1 /dev/disk/by-uuid/ /boot/efi-alt vfat defaults,nofail 0 1

Make sure the mount point /boot/efi-alt is created

Building the initramfs and grub

After adding the dracut configuration and changing the fstab do:

sudo systemctl daemon-reload
sudo update-initramfs -u -k all
sudo update-grub

You can check both /boot/efi and /boot/efi-alt are the same by running

sudo diff -ru /boot/efi/ /boot/efi-alt/
sudo rsync -aHAX --delete /boot/efi/ /boot/efi-alt/

After this the system should be configured for redundant data and booting.

Disk Replacement / Recovery

This chapter covers replacing a failed disk with a blank one, as opposed to the initial setup where both disks started out identical. The steps look similar to the original mirroring setup, but there are important differences: the pool is running degraded, the new disk has different identifiers, and fstab/GRUB need editing rather than fresh configuration.

Assume /dev/sda (disk 1) has failed and is being replaced. /dev/sdb (disk 2) is the healthy survivor.

0. Confirm the failure

zpool status

Look for a DEGRADED pool state and a member listed as UNAVAIL, FAULTED, or missing entirely. Note which partitions on the failed disk were part of bpool and rpool before continuing.

At this point the pool is running on a single disk. There is no redundancy until the steps below are complete — treat this as a priority, not routine maintenance.

1. Physically replace the disk

Power off, swap the failed disk for a blank one of the same size (or larger), and boot back up on the surviving disk.

2. Restore the partition table

This is where the partition-table.sgdisk backup saved during initial setup pays off. Restore it onto the new disk:

sudo sgdisk --load-backup=partition-table.sgdisk /dev/sda
sudo sgdisk --randomize-guid /dev/sda

Verify the layout matches the surviving disk:

sudo sgdisk -p /dev/sda
sudo sgdisk -p /dev/sdb

If you don't have a saved backup, you can instead clone the partition table directly from the surviving disk:

sudo sgdisk --backup=partition-table-recovery.sgdisk /dev/sdb
sudo sgdisk --load-backup=partition-table-recovery.sgdisk /dev/sda
sudo sgdisk --randomize-guid /dev/sda

3. Find the new disk's identifiers

The replacement disk has a different serial number, and therefore a different /dev/disk/by-id entry than the failed one did. Do not reuse the old id.

ls -la /dev/disk/by-id/

Identify the new disk's bpool and rpool partitions (e.g. partitions 2 and 4, per the original layout) by their new id strings.

4. Attach the new partitions and resilver

sudo zpool attach bpool <id-part-disk2> <id-part-new-disk>
sudo zpool attach rpool <id-part-disk2> <id-part-new-disk>

Unlike the initial mirror setup, the pool has likely accumulated writes while running degraded, so this triggers a resilver rather than a fresh sync. This can take noticeably longer and involves more I/O than the original setup did. Track progress with:

zpool status

The pool remains at reduced redundancy until the resilver finishes — avoid other disruptive operations on the pool until zpool status reports both mirrors healthy.

5. Format the new ESP

sudo mkfs.fat -F 32 /dev/<new-ESP-partition>

Formatting assigns a new UUID to this partition. This is the main place recovery diverges from the initial setup: back then you were writing fstab from scratch, but now you're editing a live, already-working file.

6. Update fstab with the new ESP UUID

Find the new UUID:

sudo blkid /dev/<new-ESP-partition>

Edit /etc/fstab and replace the old disk's ESP UUID with the new one, on whichever line pointed at the failed disk (/boot/efi or /boot/efi-alt, depending on which physical disk failed):

/dev/disk/by-uuid/<NEW-EFI-UUID> /boot/efi-alt vfat defaults,nofail 0 1

If the failed disk also held a swap partition, update its fstab line the same way with the new swap partition's UUID:

/dev/disk/by-uuid/<NEW-SWAP-UUID> none swap sw,nofail 0 0

Reload:

sudo systemctl daemon-reload

This step is easy to skip because it's an edit to something that already works, not a fresh write — but skipping it means the mount unit keeps pointing at a UUID that no longer exists.

7. Reinstall GRUB on the new disk

sudo dpkg-reconfigure grub-efi-amd64

Select both ESPs when prompted. The surviving disk's ESP is already set up; this step specifically ensures GRUB is (re)installed onto the new disk's now-blank ESP, rather than just confirming an existing configuration as it did during initial setup.

8. Rebuild initramfs and grub config

sudo update-initramfs -u -k all
sudo update-grub

No changes should be needed to /etc/dracut.conf.d/99-zfs-redundant-boot.conf itself — hostonly="no" should already be in place from initial setup. This step just confirms the regenerated initramfs and GRUB config are consistent with the new disk's UUIDs.

9. Sync the ESP contents

sudo rsync -aHAX --delete /boot/efi/ /boot/efi-alt/

Adjust source/destination depending on which physical disk (1 or 2) was replaced.

10. Verify

  • zpool status — both pools should show ONLINE with no degraded members.
  • efibootmgr -v — confirm a boot entry exists for the new disk's ESP.
  • sudo diff -ru /boot/efi/ /boot/efi-alt/ — should report no differences.
  • Reboot with only the new disk present (pull the other one temporarily) to confirm it can boot independently, just as tested during initial setup.
  • Update your saved partition-table.sgdisk backup and by-id notes to reflect the new disk, so the next recovery isn't working from stale records.

Summary of what differs from initial setup

Step Initial setup Replacement/recovery
Partition table Copied to a healthy blank disk Restored from backup (or cloned) onto a blank disk
Pool sync Fresh mirror build Resilver of a degraded pool, more I/O, less redundancy in the meantime
Disk id Recorded once Must be re-discovered; changes with the new disk's serial
ESP UUID Written into fresh fstab Must be edited into an existing, working fstab
GRUB Installed on two new, symmetric ESPs Reinstalled onto one specific new ESP
Dracut config Created and tuned Unchanged, only re-verified
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment