Building ZFS on Raspberry Pi 3 running Rasbpian

Instructions.md

Introduction

This is a tutorial for building and installing the latest release version (0.7.3 as of writing) of "ZFS on Linux" on a Raspberry Pi 3 running Raspbian Stretch. Specifically, we'll be building the dkms version of ZoL, which saves you the hassle of re-compiling the kernel modules after every kernel update. Even though ZoL added support for building dkms packages for debian in version 0.7.3, the build process on a Raspberry Pi 3 is not quite straight-forward. Hopefully, these instructions will make it easier.

Steps

1. Install the build dependencies.

$ sudo apt-get update
$ sudo apt-get install build-essential autoconf libtool gawk alien fakeroot
$ sudo apt-get install dkms zlib1g-dev uuid-dev libattr1-dev libblkid-dev libselinux-dev libudev-dev libssl-dev parted lsscsi wget ksh

You'll also need the kernel headers. Running apt-get upgrade will take care of both installation and upgradation.

$ sudo apt-get upgrade raspberrypi-kernel raspberrypi-kernel-headers

2. Download the source.

Download the latest release tarballs for zfs and spl from this page and extract them.

$ wget https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.3/spl-0.7.3.tar.gz && tar -xzf spl-0.7.3.tar.gz
$ wget https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.3/zfs-0.7.3.tar.gz && tar -xzf zfs-0.7.3.tar.gz

3. Compile and install spl.

$ cd spl-0.7.3/
$ autoreconf --install --force
$ ./configure

You need to make a small edit to the Makefile generated by configure before you proceed.

$ sed -E 's/(^RPMBUILD = rpmbuild.*)/\1 --target=armhf/' -i Makefile
$ make pkg-utils deb-dkms
$ for deb in *.deb; do sudo dpkg -i "$deb"; done

The final step will take some time as dkms builds spl modules for each kernel version installed in your system.

4. Compile and install zfs.

The build process for zfs is the same as spl, albeit with an extra option --with-config=srpm in ./configure

$ cd ../zfs-0.7.3/
$ autoreconf --install --force
$ ./configure --with-config=srpm
$ sed -E 's/(^RPMBUILD = rpmbuild.*)/\1 --target=armhf/' -i Makefile
$ make pkg-utils deb-dkms
$ for deb in *.deb; do sudo dpkg -i "$deb"; done

The zfs source is much bigger than spl's and hence the build time will also be much longer.

5. Reboot and Enjoy. :)

Sources

• The ZoL Wiki
• rpdom on this thread

I made a tutorial a little while ago to use the debian packages should be every step needed, included tuning for low men usage. Make sure you don't use swap on a normal zpool you need to use a partition or a special zvol.

https://github.com/zfsonlinux/pkg-zfs/wiki/HOWTO-install-Raspbian-to-a-Native-ZFS-Root-Filesystem,-or,-How-I-Learned-to-Love-Data-Integrity

I have added dkms to the set of dependencies.

I should also note that since last few 0.7.x versions of ZoL, while building on this 64-bit debian distribution for pi 3, I haven't had the need to edit the Makefile. Haven't checked on official Raspbian distro yet.

My Raspbian on a Raspberry Pi 3, after using wget ZFS 0.7.11, with the spl for 0.7.11, after reboot reports that the ZFS module is not loaded, and mod probe can’t find it. I followed the above instructions and they all executed properly, but no ZFS on the Raspberry pi.

I installed using this guide, but with 0.7.11... When trying to use any of the utilities I get this

root@rp01:/home/pi# zpool status
The ZFS modules are not loaded.
Try running '/sbin/modprobe zfs' as root to load them.

Running modprobe gives me this:

root@rp01:/home/pi# /sbin/modprobe zfs
modprobe: FATAL: Module zfs not found in directory /lib/modules/4.14.71-v7+

What am I missing? Any hints greatly appreciated!

[edit]
Looks like the kernel modules for 4.14.71-v7+ weren't built... :(

root@rp01:/home/pi# dpkg-reconfigure zfs-dkms
Removing old zfs-0.7.11 DKMS files...

------------------------------
Deleting module version: 0.7.11
completely from the DKMS tree.
------------------------------
Done.
Loading new zfs-0.7.11 DKMS files...
It is likely that 4.14.71-v7+ belongs to a chroot's host
Building for 3.6-trunk-rpi, 4.14.71+ and 4.14.71-v7+
Building initial module for 3.6-trunk-rpi
configure: error: in `/var/lib/dkms/zfs/0.7.11/build':
configure: error: SHUT 'ER DOWN CLANCY, SHE'S PUMPIN' MUD!
See `config.log' for more details
Error! Bad return status for module build on kernel: 3.6-trunk-rpi (armv7l)
Consult /var/lib/dkms/zfs/0.7.11/build/make.log for more information.

[edit #2]
Managed to compile the kernel module after getting rid of the old 3.6-trunk-rpi! Hooray!

thx for the guide, lil sidenote, need some other packages to compile (version 0.8) :
apt install python3-dev python3-cffi python3-setuptools

how is the performance?

Can you provide an image please? - That would be great! :)

Thanks for this guide.
I am on Raspbian GNU/Linux 9.9 (stretch).
I am sharing my experience to help others save time. Here is what worked for me and what not:

• [FAILED] installation from stable repository
• [FAILED] installation from testing repository (dkms builts but zfsutils is missing from the repo)
• [FAILED] this guide for zfs 0.7.3
• [FAILED] this guide for zfs 0.8.0
• [WORKED] this guide for zfs 0.7.13

NOTE: before you begin with this guide you have to do sudo apt remove 3.6-trunk-rpi (thanks to @drdebian)

I'm having issues with the "make pkg-utils" part of this guide when running on raspbian 4.19.65-v7l+.
I've tried it with various versions of zfs, but it's always the same.
The error is:
`make install-data-hook
make[6]: Entering directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[6]: Nothing to be done for 'install-data-hook'.
make[6]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[5]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[4]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[3]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[2]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'

• find /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/lib/arm-linux-gnueabihf -name *.la -exec rm -f {} ;
• exit 0
  Processing files: zfs-0.8.1-1.armv7hnl
  error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/sbin/*
  error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man1/*
  error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man5/*
  error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man8/*
  error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/lib/udev/rules.d/*
  error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/etc/init.d/*
  error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/etc/sudoers.d/*

RPM build errors:
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/sbin/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man1/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man5/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man8/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/lib/udev/rules.d/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/etc/init.d/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/etc/sudoers.d/*
make[1]: *** [Makefile:1234: rpm-common] Error 1
make[1]: Leaving directory '/home/pi/zfs/zfs-0.8.1'
make: *** [Makefile:1193: rpm-utils] Error 2
`
When I try the same on an x86 box, the path turns out to be
/tmp/zfs-build-ubuntu-89OPc67R/BUILDROOT/zfs-0.8.1-1.x86_64

It looks as if the system architecture isn't recognized. But I'm not familiar enough with make and package building to figure this out on my own. Any help appreciated.

Having the same problem as @stefanhinker ... clearly the %{_arch} substitution is not being made in the execution, but this may be a red herring in that the fles that are "not-found" are in fact there:

pi@raspberrypi:~ $ file /tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/bin
/tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/bin: directory
pi@raspberrypi:~ $ ls /tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/bin
splslab.py
pi@raspberrypi:~ $ file /tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/share/man/man5/
/tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/share/man/man5/: directory
pi@raspberrypi:~ $ ls /tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/share/man/man5/
spl-module-parameters.5

zfs-0.8.2 failed to build on
Linux laser-gw 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

  CC       strlcpy.lo
  CC       timestamp.lo
  CC       zone.lo
  CPPAS    atomic.lo
  CCLD     libspl.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libtpool
  CC       thread_pool.lo
  CCLD     libtpool.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libzutil
  CC       zutil_device_path.lo
  CC       zutil_import.lo
  CC       zutil_nicenum.lo
  CC       zutil_pool.lo
  CCLD     libzutil.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libunicode
  CC       u8_textprep.lo
  CC       uconv.lo
  CCLD     libunicode.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libuutil
  CC       uu_avl.lo
  CC       uu_alloc.lo
  CC       uu_dprintf.lo
  CC       uu_ident.lo
  CC       uu_misc.lo
  CC       uu_list.lo
  CC       uu_open.lo
  CC       uu_pname.lo
  CC       uu_string.lo
  CCLD     libuutil.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libnvpair
  CC       libnvpair.lo
  CC       libnvpair_json.lo
  CC       nvpair_alloc_system.lo
  CC       nvpair_alloc_fixed.lo
  CC       nvpair.lo
  CC       fnvpair.lo
  CCLD     libnvpair.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libzpool
  CC       kernel.lo
  CC       taskq.lo
make[3]: *** No rule to make target '../../module/zfs/zfeature_common.c', needed by 'zfeature_common.lo'.  Stop.
make[3]: *** Waiting for unfinished jobs....
  CC       util.lo
Makefile:653: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
Makefile:840: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
Makefile:709: recipe for target 'all' failed
make: *** [all] Error 2

pi@laser-gw:~/dev/zfs$ uname -a
Linux laser-gw 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

At the same time, the "zfs-0.7.13" built successfully:

make[2]: Leaving directory '/usr/src/linux-headers-4.19.66-v7+'
kmoddir=/lib/modules/4.19.66-v7+; \
if [ -n "" ]; then \
        find $kmoddir -name 'modules.*' | xargs rm -f; \
fi
sysmap=/boot/System.map-4.19.66-v7+; \
if [ -f $sysmap ]; then \
        depmod -ae -F $sysmap 4.19.66-v7+; \
fi
make[1]: Leaving directory '/home/pi/dev/zfs/module'
make[1]: Entering directory '/home/pi/dev/zfs'
make[2]: Entering directory '/home/pi/dev/zfs'
make[2]: Nothing to be done for 'install-exec-am'.
 /bin/mkdir -p '/usr/local/src/zfs-0.7.13'
 /usr/bin/install -c -m 644 zfs.release.in zfs_config.h.in '/usr/local/src/zfs-0.7.13'
 /bin/mkdir -p '/usr/local/src/zfs-0.7.13/4.19.66-v7+'
 /usr/bin/install -c -m 644 zfs.release zfs_config.h module/Module.symvers '/usr/local/src/zfs-0.7.13/4.19.66-v7+'
make[2]: Leaving directory '/home/pi/dev/zfs'
make[1]: Leaving directory '/home/pi/dev/zfs'
+ sudo ldconfig
+ sudo depmod -a
+ sudo modprobe zfs
+ sudo zpool import -a

If someone has a solution, pls share!

I just fixed in this gist:
https://gist.github.com/Alexey-Tsarev/d5809e353e756c5ce2d49363ed63df35

I just fixed in this gist:
https://gist.github.com/AlexeySofree/d5809e353e756c5ce2d49363ed63df35

thanks!

I just fixed in this gist:
https://gist.github.com/AlexeySofree/d5809e353e756c5ce2d49363ed63df35

Now appears to be moved to https://gist.github.com/Alexey-Tsarev/d5809e353e756c5ce2d49363ed63df35

@stefanhinker @colettace
The problem can be fixed with the following to build zfs 0.8.3 on a Raspberry Pi 4:

./autogen.sh
./configure --disable-pyzfs
# https://github.com/zfsonlinux/zfs/issues/9504
sed -E 's/(^RPMBUILD = rpmbuild.*)/\1 --target=armv7hnl-linux-gnueabihf/' -i Makefile

make deb-dkms deb-utils

see openzfs/zfs#9504 for more information

There is no need to use sed to modify the Makefile. rpmbuild errrors are caused by the rpmbuild package on raspbian not storing the architecture files in an appropriately named directory. You can fix this permanently by running the following:

cd /usr/lib/rpm/platform; sudo ln -s armv7hnl-linux-gnueabihf armv7hnl-linux