Compile Linux Kernel Raspberry Pi 5 with Huge Pages support

raspi-huge-pages-kernel.md

Out of the box the Linux Kernel from Raspberry Pi OS does not come with activated support for Huge Pages, but it can be activated while compiling your own Linux Kernel.

Instructions on how to compile the Linux Kernel for Raspberry Pi can be mostly found on this website:
https://www.raspberrypi.com/documentation/computers/linux_kernel.html

Huge Pages can improve the speed of crypto mining (like Monero) hashes by about 50% compared to the stock Kernel.

First install some reqirements for building and download the Kernel sources:

sudo su
apt update
apt install -y git bc bison flex libssl-dev make libncurses5-dev
cd /opt
git clone --depth=1 https://github.com/raspberrypi/linux
cd linux

After that you need to configure the Kernel for Raspberry Pi 5:

make bcm2712_defconfig

Now you need to activate Huge Pages supprt:

make menuconfig

First change the kernel name by navigating to General Setup and look for Local Version. Change it to something like v8-16k-huge. Then press ESC two times to go back.
At last go to File systems, then to Pseudo filesystems (at the bottom of the list) and there activate HugeTLB file system support- Then press ESC two times to go back.

Then save the configuration.

From here, the normal compile steps have to be executed, which should take about an hour:

make -j4 Image.gz modules dtbs
make modules_install
cp arch/arm64/boot/dts/broadcom/*.dtb /boot/firmware/
cp arch/arm64/boot/dts/overlays/*.dtb* /boot/firmware/overlays/
cp arch/arm64/boot/dts/overlays/README /boot/firmware/overlays/
cp arch/arm64/boot/Image.gz /boot/firmware/kernel8-huge.img

Now activate the new Kernel:

echo "kernel=kernel8-huge.img" | tee -a /boot/firmware/config.txt

To have a specific Huge Page configuration, you have to edit the file /boot/firmware/cmdline.txt in order for the Kernel to reserve the memory. Add something like this at the end of the line:

default_hugepagesz=2M hugepages=1280 hugepagesz=1G hugepages=3

Then reboot

At time of writing, the Rasperry Kernel was 6.1.x, later rechecked for 6.6.22 (still works for me).

EDIT: updated some file locations, thanks to @Pi-novice for pointing out, I probably mixed things up with my Pi 4 setup

For reference: on my newly arrived Raspberry Pi 5 with 8GB RAM the hashing rate is about 330 H/s with a single thread running XMRIG

Thank you for taking the time to put this together

Since the 3/12/24 update, this doesn't seem to work.

At time of writing, the kernel was 6.1.x, since some weeks this is now based on 6.6.x as default. I have recompile the kernel, and for me it is still working @Pi-novice , no other changes were required.

Finally working.
Are you sure that :
echo "kernel=kernel8-huge.img" | tee -a /boot/config.txt
shouldn't be :
echo "kernel=kernel8-huge.img" | tee -a /boot/firmware/config.txt

I do not know if the last update is when they stopped using /boot/config.txt and started using /boot/firmware/config.txt.
But once I did the above change, it all worked.

need to change from /boot/cmdline.txt to /boot/firmware/cmdline.txt also.

@Pi-novice interesting ... it is working for me, so I am a bit confused. Maybe it is something with my local installation, Initially I had an older RaspberryPiOS image, maybe it has something to do with the way how they are having their files stored now. Glad you found your way ;)

Yes, config.txt has been moved to /boot/firmware.

I have requested hugepages support on the official Raspberry Pi kernel builds. Those interested in having this work by default without needing to recompile the kernel, please feel free to comment there to show your support.
raspberrypi/linux#6297

I have compared the performance before and after enabling hugepages in cmdline.txt.
This table compares the benchmarked values from two xmrig.json files.

algo	Without hugepages	With hugepages
cn/0	27	28
cn/1	18	18
cn/2	18	18
cn/r	18	18
cn/fast	37	36
cn/half	37	36
cn/xao	18	18
cn/rto	18	18
cn/rwz	25	24
cn/zls	25	24
cn/double	9	9
cn/ccx	55	56
cn-lite/0	132	124
cn-lite/1	132	124
cn-heavy/xhv	28	28
cn-pico	1593	1635
cn-pico/tlo	1593	1635
cn/gpu	12	12
rx/0	222	246
rx/arq	3208	4254
rx/graft	201	231
rx/sfx	222	246
panthera	466	463
argon2/chukwav2	593	568
ghostrider	52	53

This is with the moneroocean fork of xmrig.
I found there are more performance benefits from compiling xmrig with -ffast-math (rx/o reached 340 and rx/arq reached 4900), but this causes segfaults on all the cn* algorithms.

@Botspot What Raspberry are you using for your benchmarks? Have you checked if and how the cmdline entries default_hugepagesz , hugepages, hugepagesz and hugepages influence your results?
If I remember correct, Pi4 and Pi5 have different default_hugepagesz settings, and I might not have found the best settings :) just ones that "somehow work"

Pi5 8GB at default clockspeed, and I used your default_hugepagesz=2M hugepages=1280 hugepagesz=1G hugepages=3.
I see no value in playing with the values here. More would go unused and less would mean XMRig cannot reach 100% utilization. Furthermore, all the issues on the xmrig repo I checked were saying to have 3 1gb hugepages for this one algo and 1280 2M hugepages for the other algos.
Why exactly do you feel that adding more hugepages would be of value, when xmrig already reports that it has what it can use?

@FibreFoX If you are expecting higher values, try the same test yourself.
Compile and install xmrig with the same script I'm using (pi-apps), and then allow it to complete its benchmarks uninhibited. Then look at xmrig's config file to see what the benchmarked values were.

If was asking just for the sake of clarity. I've heared that Pi5 with 4GB performs somewhat different than the 8GB versions. In addition, due to the hugepages hogging the RAM, it might influence the running system at a whole. I was more looking into enabling the feature, not getting the best performance :)

Still working on rpi4 8GB, thanks a lot, I used hugepages=6 and get 6GB addressed