Skip to content

Instantly share code, notes, and snippets.

@karlbunch

karlbunch/alpine-zfs-grub-uefi.md

Forked from kongkrit/alpine-zfs-grub-uefi.md
Created August 30, 2022 20:19
Show Gist options
  • Save karlbunch/441252336ab12cdd72575dbc086a9874 to your computer and use it in GitHub Desktop.
Save karlbunch/441252336ab12cdd72575dbc086a9874 to your computer and use it in GitHub Desktop.
Download ZIP
Install Alpine Linux on ZFS Root - grub bootloader on UEFI
Raw
alpine-zfs-grub-uefi.md

Alpine Linux Installation on ZFS Root with grub on UEFI

  • References [ ref1 | ref2 | ref3 ]

  • Boot from alpine-extended [ download ]

  • login as root without any password

  • setup network interfaces and start networking:

    setup-interfaces
/etc/init.d/networking start

  • add, config, and enable openssh:

    apk add openssh
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
/etc/init.d/sshd start

  • config root password and show IP so we can ssh in:

    echo "root:abc" | chpasswd # set root password to "abc"
ip addr show # show ip address

  • now we can ssh into the machine

  • pick a mirror from alpine-mirrors and get the latest version

    MIRROR=http://dl-cdn.alpinelinux.org/alpine  # example from mirror above, no trailing slash
ARCH=x86_64 # architecture, 32-bit x86 is "i386" and amd64 is "x86_64"
VERSION=v3.11 # get the latest version from https://alpinelinux.org/ -- do not include last dot

  • setup apk repos and update:

    echo "$MIRROR/$VERSION/main" >> /etc/apk/repositories
echo "$MIRROR/$VERSION/community" >> /etc/apk/repositories
apk update

  • setup udev to get /dev/disk/by-id

    apk add util-linux udev zfs
setup-udev

  • load zfs module and check version

    modprobe zfs      # should return nothing
lsmod | grep zfs  # should list a few modules
zfs --version     # shows version

  • load sgdisk and dosfstools and list disks:

    apk add sgdisk dosfstools
ls -l /dev/disk/by-id  # shows disk by id
ls -l /dev/disk/by-id/ | sed -E 's/^.*[ \t]+([^ \t]+[ \t]+->[ \t]+.*)$/\1/g' # without extraneous info 

    lrwxrwxrwx  1 root  root   9 Mar 15 16:44 ata-VBOX_CD-ROM_VB2-01700376 -> ../../sr0
lrwxrwxrwx  1 root  root   9 Mar 15 16:44 ata-VBOX_HARDDISK_VB0cc84e8a-30407bab -> ../../sda
lrwxrwxrwx  1 root  root   9 Mar 15 16:44 ata-VBOX_HARDDISK_VB5ee32bc3-a631fa00 -> ../../sdb

  • We will use /dev/sda id as $DISK1 and /dev/sdb id as $DISK2 for the following example:

    DISK1=/dev/disk/by-id/ata-VBOX_HARDDISK_VB0cc84e8a-30407bab
DISK2=/dev/disk/by-id/ata-VBOX_HARDDISK_VB5ee32bc3-a631fa00

  • wipe both disks:

    sgdisk --zap-all $DISK1
sgdisk --zap-all $DISK2

    each command should return:

    Creating new GPT entries in memory.
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.

  • create EFI partitions:

    sgdisk -n1:1M:+512M -t1:EF00 $DISK1
udevadm settle
mkfs.fat -F 32 -n EFI ${DISK1}-part1
udevadm settle
sgdisk -n1:1M:+512M -t1:EF00 $DISK2
udevadm settle
mkfs.fat -F 32 -n EFI ${DISK2}-part1
udevadm settle

  • create ZFS partition for boot:

    sgdisk -n2:0:+1G -t2:BF00 $DISK1
sgdisk -n2:0:+1G -t2:BF00 $DISK2
udevadm settle

  • create ZFS partition for root:

    sgdisk -n3:0:0 -t3:BF00 $DISK1
sgdisk -n3:0:0 -t3:BF00 $DISK2
udevadm settle

  • create boot pool (bpool) with features incompatible for grub disabled

    BPOOL=bpool
zpool create -o ashift=12 -d \
  -o feature@async_destroy=enabled \
  -o feature@bookmarks=enabled \
  -o feature@embedded_data=enabled \
  -o feature@empty_bpobj=enabled \
  -o feature@enabled_txg=enabled \
  -o feature@extensible_dataset=enabled \
  -o feature@filesystem_limits=enabled \
  -o feature@hole_birth=enabled \
  -o feature@large_blocks=enabled \
  -o feature@lz4_compress=enabled \
  -o feature@spacemap_histogram=enabled \
  -o feature@userobj_accounting=enabled \
  -o feature@zpool_checkpoint=enabled \
  -o feature@spacemap_v2=enabled \
  -o feature@project_quota=enabled \
  -o feature@resilver_defer=enabled \
  -o feature@allocation_classes=enabled \
  -O acltype=posixacl -O canmount=off -O compression=lz4 -O devices=off \
  -O normalization=formD -O relatime=on -O xattr=sa \
  -O mountpoint=none -R /mnt -f $BPOOL mirror ${DISK1}-part2 ${DISK2}-part2

  • create root pool (use ashift=13 for Samsung SSD):

    RPOOL=rpool
zpool create -o ashift=12 \
  -O acltype=posixacl -O canmount=off -O compression=lz4 \
  -O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
  -O mountpoint=none -R /mnt \
  -f $RPOOL mirror ${DISK1}-part3 ${DISK2}-part3

  • create containers for $BPOOL and $RPOOL

    zfs create -o mountpoint=none -o canmount=off $BPOOL/BOOT
zfs create -o mountpoint=none -o canmount=off $RPOOL/ROOT

  • read create your datasets

    zfs create -o mountpoint=legacy   -o atime=off $BPOOL/BOOT/default
zfs create -o mountpoint=/        -o atime=off $RPOOL/ROOT/alpine
zfs create -o mountpoint=/home    -o atime=off $RPOOL/HOME
zfs create -o mountpoint=/var/log -o atime=off $RPOOL/LOG

  • mount boot filesystem

    # mount -t zfs $RPOOL/ROOT/alpine  /mnt
mkdir -p /mnt/boot
mount -t zfs $BPOOL/BOOT/default /mnt/boot

  • Set the bootfs property on the boot filesystem. This command will only be successful when dnodesize property on $BPOOL/BOOT/default is set to legacy:

    zfs set dnodesize=legacy $BPOOL/BOOT/default
zpool set bootfs=$BPOOL/BOOT/default $BPOOL

Installing Alpine Linux in a chroot [ ref ]

  • Find out the latest apk-tools-static version. 
    echo ${MIRROR}/latest-stable/main/${ARCH}
  • Go to the above URL and find the file apk-tools-static-?????.apk, As of this writing (2020-03-12) the version is apk-tools-static-2.10.4-r3.apk
  • Download and unpack the apk-tools-static package to mount directory: 
    wget ${MIRROR}/latest-stable/main/${ARCH}/apk-tools-static-2.10.4-r3.apk
tar -xzf apk-tools-static-*.apk
  • Install the alpine base onto chroot 
    ./sbin/apk.static -X ${MIRROR}/latest-stable/main -U --allow-untrusted --root /mnt --initdb add alpine-base
  • bind mount /dev, /proc, /sys -- see set up the chroot 
    mount --rbind /dev  /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys  /mnt/sys
  • copy DNS settings and apk repos to to /mnt 
    cp /etc/resolv.conf /mnt/etc/resolv.conf
cp /etc/apk/repositories /mnt/etc/apk/repositories
  • save environment vars for later use: 
    cat > /mnt/root/vars.sh << EOF
DISK1=$DISK1
DISK2=$DISK2
RPOOL=$RPOOL
BPOOL=$BPOOL
ARCH=$ARCH
MIRROR=$MIRROR
VERSION=$VERSION
EOF
  • chroot 
    chroot /mnt /usr/bin/env DISK1=$DISK1 DISK2=$DISK2 RPOOL=$RPOOL BPOOL=$BPOOL /bin/ash -l
  • mount /dev/cdrom and add necessary stuff 
    mount /dev/cdrom /media/cdrom
apk add linux-lts udev util-linux zfs-lts openssh nano
  • perform init process 
    rc-update add devfs sysinit
rc-update add dmesg sysinit
rc-update add mdev sysinit
rc-update add hwdrivers sysinit

rc-update add hwclock boot
rc-update add modules boot
rc-update add sysctl boot
rc-update add hostname boot
rc-update add bootmisc boot
rc-update add syslog boot

rc-update add mount-ro shutdown
rc-update add killprocs shutdown
rc-update add savecache shutdown
  • enable zfs services 
    rc-update add zfs-import sysinit
rc-update add zfs-mount sysinit
  • Edit the /etc/mkinitfs/mkinitfs.conf file and append nvme zfs module to the features parameter: 
    # if you don't need nvme: sed -i -E 's/^(features=.*)([^ \t])"$/\1\2 zfs"/g' /etc/mkinitfs/mkinitfs.conf
sed -i -E 's/^(features=.*)([^ \t])"$/\1\2 nvme zfs"/g' /etc/mkinitfs/mkinitfs.conf
cat /etc/mkinitfs/mkinitfs.conf # check the work
  • rebuild initramfs 
    mkinitfs $(ls /lib/modules/)
  • generate fstab for bpool 
    mv /etc/fstab /etc/fstab.old
cat >> /etc/fstab << EOF
$BPOOL/BOOT/default /boot zfs rw,nodev,noatime,xattr,posixacl 0 0
EOF
  • generate mount points for EFI partitions: 
    mkdir -p /efi1
mkdir -p /efi2
  • add 2 EFI partitions to /etc/fstab 
    echo UUID=$(blkid -s PARTUUID -o value ${DISK1}-part1 | sed -E 's/^.*[ \t]UUID="([^ \t]+)".*$/\1/g') \
  /efi1 vfat nofail,defaults 0 0 >> /etc/fstab
echo UUID=$(blkid -s PARTUUID -o value ${DISK2}-part1 | sed -E 's/^.*[ \t]UUID="([^ \t]+)".*$/\1/g') \
  /efi2 vfat nofail,defaults 0 0 >> /etc/fstab

    echo PARTUUID=$(blkid -s PARTUUID -o value ${DISK1}-part1 | sed -E 's/:.*$//g') \
  /efi1 vfat nofail,defaults 0 0 >> /etc/fstab
echo PARTUUID=$(blkid -s PARTUUID -o value ${DISK2}-part1 | sed -E 's/:.*$//g') \
  /efi2 vfat nofail,defaults 0 0 >> /etc/fstab
  • check /etc/fstab to be of this format: 
    bpool/BOOT/default /boot zfs rw,nodev,noatime,xattr,posixacl 0 0
UUID=F857-B91A /boot/efi1 vfat nofail,defaults 0 0
UUID=F889-5FAE /boot/efi2 vfat nofail,defaults 0 0
  • mount EFI partitions: 
    mount /efi1
mount /efi2
  • install grub-efi 
    apk add efibootmgr grub-efi

    grub-probe will report an error. we will fix later

  • install grub to /efi1 
    mkdir -p /boot/grub
ZPOOL_VDEV_NAME_PATH=1 grub-probe /boot
# grub-probe should return "zfs"
ZPOOL_VDEV_NAME_PATH=1 grub-install --target=x86_64-efi --efi-directory=/efi1 --bootloader-id=GRUB
#ZPOOL_VDEV_NAME_PATH=1 grub-install --target=x86_64-efi --efi-directory=/boot/efi2 --bootloader-id=GRUB2
  • modify /etc/default/grub - add root=ZFS=$RPOOL/ROOT/alpine to GRUB_CMDLINE_LINUX 
    [[ -f /etc/default/grub.original ]] && cp /etc/default/grub.original /etc/default/grub
cp /etc/default/grub /etc/default/grub.original
# add GRUB_CMDLINE_LINUX="root=$RPOOL/ROOT/alpine"
echo "GRUB_CMDLINE_LINUX=\"root=""$RPOOL""/ROOT/alpine rootfstype=zfs\"" >> /etc/default/grub
# add stuff to to GRUB_CMDLINE_LINUX_DEFAULT
echo "GRUB_CMDLINE_LINUX_DEFAULT=\"modules=sd-mod,usb-storage,ext4 nomoodeset\"" >> /etc/default/grub
# remove crap from GRUB_CMDLINE_LINUX_DEFAULT
# sed -i -E 's/^(GRUB_CMDLINE_LINUX_DEFAULT=).*$/\1=""/g' /etc/default/grub
  • make new /boot/grub/grub.cfg 
    ZPOOL_VDEV_NAME_PATH=1 grub-mkconfig -o /boot/grub/grub.cfg
    • it will report an error. BUMMER 2 choices below
    1. run script to fix what's manually done in 2.
    cp /boot/grub/grub.cfg /boot/grub/grub.cfg.original
sed -i -E 's;^([ \t]*linux[ \t]+/BOOT/default@/vmlinuz[^ \t]*[ \t]+root=).*$;\1'"$RPOOL"'/ROOT/alpine rootfstype=zfs modules=sd-mod,usb-storage,ext4 nomodeset;g' /boot/grub/grub.cfg
sed -i -E '/^\/dev\/sd[a-z]3[ \t]+ro[ \t]+root='"$RPOOL"'\/ROOT\/alpine.*$/d' /boot/grub/grub.cfg
    1. Manually fix it. edit near line "echo 'Loading Linux lts ...' followed by linux /BOOT/default@/vmlinuz... so it reads:
    echo 'Loading Linux lts ...'
linux /BOOT/default@/vmlinuz-lts rootfstype=zfs root=$RPOOL/ROOT/alpine ro modules=sd-mod,usb-storage,ext4 nomodeset 
echo 'Loading initial ramdisk ...'
  • rsync both partitions to make both bootable 
    apk add rsync
rsync -Rai --stats --human-readable --delete --verbose --progress /efi1/./ /efi2
  • add EFI boot entry to machine 
    efibootmgr -c -g -d $DISK2 -p 1 -L "GRUB-2" -l '\EFI\GRUB\grubx64.efi'
  • (optional) disable media use for apk installation: 
    sed -i -E 's/^(\/media.*)$/#\1/g' /etc/apk/repositories
  • (optional) permit root login on ssh: 
    sed -i -E 's/^(#PermitRootLogin.*)$/PermitRootLogin yes\n\1/g' /etc/ssh/sshd_config
  • (optional-required for ssh login) set root password: 
    echo "root:abc" | chpasswd
  • set ZFS pool cachefile: 
    zpool set cachefile=/etc/zfs/zpool.cache $RPOOL
  • exit chroot 
    exit

Done with chroot, prepare to reboot

  • unmount all 
    mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
  • unmount zfs and export pools 
    umount /mnt/boot
zfs unmount $RPOOL/LOG
zfs unmount $RPOOL/HOME
zfs unmount -a
  • remove live USB/CD and reboot 
    # remove live media
reboot

Post Installation Setup

  • login as root with or without password (depending on whether you set root password)
  • setup networking: 
    setup-interfaces
/etc/init.d/networking restart
  • Hopefully ethernet access will be on: try ping -c 3 google.com
  • If you haven't enabled sshd or set root password, go back and do it:
  • ssh login to machine with root and password
  • can mount live media for faster apk updates: 
    mount /dev/cdrom /media/cdrom
  • fix grub-efi install error from earlier: 
    cp /boot/grub/grub.cfg /boot/grub/grub.cfg.working
apk fix # this will fix grub-efi error
cp /boot/grub/grub.cfg.working /boot/grub/grub.cfg
  • add some useful software: 
    apk add which htop man man-pages
  • setup the rest of alpine 
    # cp /etc/apk/repositories /etc/apk/repositories.save
setup-udev
setup-keymap
setup-hostname
setup-timezone
setup-proxy
setup-ntp
# setup-apkrepos
# setup-sshd
setup-ntp # use chrony
# setup-alpine # when asked "which disk to use?" answer "none"
# cp /etc/apk/repositories.save /etc/apk/repositories # don't wanna mess with repo file
  • change root to run bash instead of ash: 
    apk add bash bash-completion shadow sudo
usermod -s /bin/bash root
    • logout and login again to take effect
  • add another user 
    useradd -s /bin/bash -m -U username
echo "username:1234" | chpasswd
  • add that user to wheel group so user can sudo 
    EDITOR=nano visudo
    • uncomment either a) # %wheel ALL=(ALL) ALL or b) # %wheel ALL=(ALL) NOPASSWD: ALL
    • if you uncomment a), password is needed for sudo, if you uncomment b), password is not needed for sudo
    • add a user to wheel group so that user can sudo 
      usermod -aG wheel username
  • login as a non-root user and use it to disable direct root login: 
    sudo passwd -l root
    (sudo passwd -u root will unlock root access)
  • reboot and login as root
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment