squashfs compression tests using mksquashfs - xz, gzip, zstd, lz4, lz4hc, lzo

squashfs.txt

Just a quick test of squashfs compression ratio using different settings.

I am looking for relatively good and fast compression, that also is quick to decompress.
I don't care about ultimate end size exactly tho.

Input (a Debian testing live build with 6240 installed packages):

$ sudo du -bs ./chroot
26566785410	./chroot   # 26.6GB
$
$ sudo find ./chroot -type f | wc -l
614635
$ sudo find ./chroot -type d | wc -l
55771
$

Input and output on tmpfs.

32 compression threads on AMD TR 2950X

Results:

mksquashfs ./chroot chroot_gzip9_default.squashfs -noappend -comp gzip  # 128kiB blocks, gzip-9           # 168s
mksquashfs ./chroot chroot_xz_default.squashfs -noappend -comp xz  # 128kiB blocks                        # 376s
mksquashfs ./chroot chroot_xz-x86_default.squashfs -noappend -comp xz -Xbcj x86  # 128kiB blocks          # 781s
mksquashfs ./chroot chroot_xz_block32k.squashfs -noappend -comp xz -b 32768  # 32 kiB blocks              # 336s
mksquashfs ./chroot chroot_zstd15_default.squashfs -noappend -comp zstd                                   # 215s
mksquashfs ./chroot chroot_zstd15_block32k.squashfs -noappend -comp zstd -b 32768                         # 159s
mksquashfs ./chroot chroot_zstd22_block32k.squashfs -noappend -b 32768 -comp zstd -Xcompression-level 22  # 650s
mksquashfs ./chroot chroot_zstd22_block128k.squashfs -noappend -comp zstd -Xcompression-level 22          # 934s
mksquashfs ./chroot chroot_lzo9_block32k.squashfs -noappend -b 32768 -comp lzo -Xcompression-level 9      # 180s
mksquashfs ./chroot chroot_lz4_block32k.squashfs -noappend -comp lz4 -b 32768                             #  21s (not a typo!)
mksquashfs ./chroot chroot_lz4hc_block32k.squashfs -noappend -comp lz4 -b 32768 -Xhc                      # 121s

-rw-r--r--  1 root root  7701307392 Jul 24 07:01 chroot_xz-x86_block128k.squashfs
-rw-r--r--  1 root root  7822753792 Jul 24 05:08 chroot_xz_default.squashfs
-rw-r--r--  1 root root  8473190400 Jul 24 06:37 chroot_zstd22_block128k.squashfs
-rw-r--r--  1 root root  8517169152 Jul 24 05:14 chroot_xz_block32k.squashfs
-rw-r--r--  1 root root  8553738240 Jul 24 05:44 chroot_zstd15_default.squashfs
-rw-r--r--  1 root root  9059016704 Jul 24 06:09 chroot_zstd22_block32k.squashfs
-rw-r--r--  1 root root  9148821504 Jul 24 05:53 chroot_zstd15_block32k.squashfs
-rw-r--r--  1 root root  9248616448 Jul 24 06:47 chroot_gzip9_block128k.squashfs
-rw-r--r--  1 root root 10710016000 Jul 24 06:19 chroot_lzo9_block32k.squashfs
-rw-r--r--  1 root root 11628195840 Jul 24 06:14 chroot_lz4hc_block32k.squashfs
-rw-r--r--  1 root root 13007339520 Jul 24 06:11 chroot_lz4_block32k.squashfs

I will stick with `-b 32768 -comp zstd -Xcompression-level 22`, because it is
reasonably fast to compress (but actually slower than xz), reasonably good
compression, and should be really fast to decompress. 32kiB blocks hopefully
reduce some redundant reads for some big files, which hopefully will make
latency lower and total transfered data (on read) smaller, for overall faster
load and better expirience, and slightly more optimized memory usage.