Skip to content

Instantly share code, notes, and snippets.

@timofeika

timofeika/Fedora_Root_On_ZFS.md

Forked from p7cq/Fedora_Root_On_ZFS.md
Created May 7, 2021 13:27
Show Gist options
  • Save timofeika/2ef370a1b0760591994a9ebb6e7fd20f to your computer and use it in GitHub Desktop.
Save timofeika/2ef370a1b0760591994a9ebb6e7fd20f to your computer and use it in GitHub Desktop.
Download ZIP
Fedora with Root on ZFS - installation notes
Raw
Fedora_Root_On_ZFS.md

Fedora: Root on ZFS

  • Fedora: 28
  • ZFS on Linux: 0.7.9

Note: for newer Fedora releases see Reliably boot Fedora with root on ZFS.

A. Prepare source

  1. Download Fedora Workstation network installer image.

  2. Boot live media by adding inst.gpt at the end of the boot line and set up with:

    • a boot partition /boot/efi of size 512MB
    • a root partition / of size rest of the disk
    • minimal installation

  3. Reboot and:

    • run dnf -y update
    • set hostname, networking, idmapd
    • output of rpm -qa|grep "grub2-efi-x64\|shim-x64" should contain:
    grub2-efi-x64
 shim-x64
    • install UEFI modules and some required packages:
    dnf -y install grub2-efi-x64-modules kernel-devel gdisk rsync bridge-utils
    • add system wide profile customization in /etc/profile.d/
    • run init 6

  4. Create scripts:

    • to run dracut and make GRUB ZFS aware:
    #!/bin/bash
 kver=$1
 sh -x /usr/bin/dracut -fv --kver $kver
 mount /boot/efi
 grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg
 grubby --set-default-index 0
 mkdir -p /boot/efi/EFI/fedora/x86_64-efi
 cp -a /usr/lib/grub/x86_64-efi/zfs* /boot/efi/EFI/fedora/x86_64-efi
 umount /boot/efi
    • to enter chroot:
    #!/bin/bash
 target=$1
 mount -t proc  proc $target/proc
 mount -t sysfs sys $target/sys
 mount -o bind /dev $target/dev
 mount -o bind /dev/pts $target/dev/pts
 chroot $target /bin/env -i HOME=/root \
     TERM="$TERM" PS1='[\u@chroot \W]\$ ' \
     PATH=/bin:/usr/bin:/sbin:/usr/sbin \
     /bin/bash --login
 echo "Exiting chroot environment..."
 umount $target/dev/pts
 umount $target/dev/
 umount $target/sys/
 umount $target/proc/
    • make both scripts executable

  5. Add ZFS support to GRUB in /etc/default/grub:

GRUB_PRELOADED_MODULES="zfs"

so that it can detect the ZFS filesystem; and

GRUB_DISABLE_OS_PROBER=true

so that grub2-mkconfig does not throw errors.

  1. Fix GRUB's handling of pool device path names by creating /etc/profile.d/grub2_zpool_fix.sh containing
export ZPOOL_VDEV_NAME_PATH=YES

The effect is that zpool status will list the full path of the block device: /dev/disk/by-id/nvme0n1-part2 instead of nvme0n1-part2.

  1. Prepare for future kernel updates:
ln -s /usr/local/sbin/zmogrify /etc/kernel/postinst.d
  1. Add ZFS on Linux repository then install and load ZFS:
dnf install zfs zfs-dracut
modprobe zfs
  1. Disable cache and enable scan:
systemctl disable zfs-import-cache
systemctl enable zfs-import-scan
  1. Regenerate initramfs:
dracut -fv --kver `uname -r`
  1. Reboot

B. Prepare target

  1. Wipe disks:
sgdisk --zap-all /dev/disk/by-id/nvme0n1
sgdisk --zap-all /dev/disk/by-id/nvme1n1
  1. Create boot partition (UEFI) and dedicate the rest to ZFS:
sgdisk -n1:1M:+512M -t1:EF00 /dev/disk/by-id/nvme0n1
sgdisk -n2:0:0 -t2:BF01 /dev/disk/by-id/nvme0n1
sgdisk -n1:1M:+512M -t1:EF00 /dev/disk/by-id/nvme1n1
sgdisk -n2:0:0 -t2:BF01 /dev/disk/by-id/nvme1n1
  1. Format boot partition:
mkfs.fat -F32 /dev/disk/by-id/nvme0n1-part1
mkfs.fat -F32 /dev/disk/by-id/nvme1n1-part1
  1. Create the pool:
zpool create -m none -O canmount=off -o ashift=12 -o cachefile=none rpool mirror /dev/disk/by-id/nvme0n1-part2 /dev/disk/by-id/nvme1n1-part2
  1. Create a container dataset:
zfs create -o canmount=off -o mountpoint=none rpool/ROOT
  1. Create the root dataset:
zfs create rpool/ROOT/default
zfs set acltype=posixacl rpool/ROOT/default # systemd-journald's /var/log/journal
zfs set xattr=sa rpool/ROOT/default
zfs set compression=on rpool/ROOT/default
zfs set atime=off rpool/ROOT/default
  1. Create the home dataset:
zfs create -o compression=on -o atime=off -o xattr=sa -o acltype=posixacl rpool/home
  1. Create a dataset for storing data:
zfs create -o compression=on -o atime=off -o xattr=sa -o acltype=posixacl rpool/local
  1. Create a swap volume:
zfs create -V 16G -b $(getconf PAGESIZE) -o compression=zle -o logbias=throughput -o sync=always -o primarycache=metadata -o secondarycache=none -o com.sun:auto-snapshot=false rpool/swap
mkswap /dev/zvol/rpool/swap
  1. Export pool:
zpool export rpool
  1. Import pool into an alternate mount point:
zpool import rpool -d /dev/disk/by-id -o altroot=/sysroot
  1. Set the mountpoint for the root and for home and local datasets:
zfs set mountpoint=/ rpool/ROOT/default
zfs set mountpoint=/home rpool/home # directory is empty
zfs set mountpoint=/local rpool/local # directory is empty

  1. Check mountpoints.

  2. Transfer data:

rsync -WSHAXhavx / /sysroot/
  1. Copy EFI partition from the source to the target:
cd
mkdir here
mount /dev/disk/by-id/nvme0n1-part1 here
cp -a /boot/efi/* here
umount here
rmdir here
  1. Add boot and swap partitions to fstab:
blkid /dev/disk/by-id/nvme0n1-part1

Replace content of /sysroot/etc/fstab with:

UUID=<UUID from blkid>  /boot/efi  vfat  umask=0077,shortname=winnt  0  2
/dev/zvol/rpool/swap    swap       swap  defaults                    0  0
  1. Enter chroot:
zenter /sysroot
  1. Needed by F28's GRUB is a line containing:
GRUB_DEVICE_BOOT=/dev/disk/by-id/nvme0n1-part2

in /etc/default/grub.

  1. Transform target into a ZFS aware system:
zmogrify \`uname  -r\`
  1. Confirm that GRUB can recognize the ZFS filesystem:
grub2-probe /

should output zfs.

  1. Exit chroot

  2. Export pool:

zpool export rpool

  1. Reboot

  2. Take an initial snapshot:

zfs snapshot rpool/ROOT/default@initial

  1. Create user and group

  2. Complete installation:

dnf -y groupinstall "Fedora Workstation"
systemctl set-default graphical.target
systemctl enable gdm.service
  1. Reboot

References:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment