Automatic Signing of DKMS-Generated Kernel Modules for Secure Boot

dkms-kmod-auto-mok-signing.md

Automatic Signing of DKMS-Generated Kernel Modules for Secure Boot (Nvidia Driver on CentOS 8 as Example)

First I thank Nvidia for sponsoring the video card.

Secure Boot isn't exactly easy to configure to work with Linux and disabling it isn't really a good idea. Many modern Linux distributions provide the Microsoft-signed shim EFI binary to interpose between Secure Boot and the grub2 bootloader, making booting Linux easy enough if you only ever use kernels and drivers from the official repos. Still, enabling Secure Boot prevents the loading of kernel or modules without a proper digital signature. For example, the propriatary Nvidia GPU driver won't work, unless your distro really went to great lengths to distribute a signed version of the kernel module.

To make Secure Boot play nicely with the driver (i.e. to work at all), we can generate and import a Machine Owner Key (MOK), and use it to sign the kernel module each time it is rebuilt after updating the kernel and/or the driver. The rebuilding of the kernel module is typically automated by DKMS, but the signature process is not -- because the MOK's generation and usage must be initiated manually by the user.

There have been guides (e.g. Nvidia's own guide and 1 2 3) for signing the modules and even automating it with the POST_BUILD directive (e.g. 1 2).

But after digging into DKMS's man page, I found that the SIGN_TOOL directive may provide a simpler way to configure the automatic signing. So here's a full guide, and if you're only interested in the DKMS part, it's in the Step 2 below:

I used CentOS 8; things can vary slightly in other distros.

Note 2022-05-08: Thank you folks for offering advices and fixes to this gist. I recently switched my personal PC from CentOS to Ubuntu LTS (actually, KDE Neon) due to the uncertainties about IBM/RedHat converting CentOS into CentOS stream. I found that for Ubuntu/Debian, well-packaged DKMS drivers such as nvidia (as packaged by Nvidia with CUDA) and v4l2loopback have some hooks handling MOK signing, thus making this gist unnecessary.

However, they are not perfectly functional. My observations/caveats are:

• When installing using the installer, it will prompt you to set up a MOK key if you have Secure Boot on and disk encrypted. I use debootstrap to install Debian-based distros so ain't sure how automated it actually is.

• The hooks do not always trigger when installing the dkms driver packages. If they don't, you can reboot with Secure Boot on, and try to run, e.g., dpkg-reconfigure nvidia-dkms-xxx as root. In addition to building and signing the kernel module, it should also prompt for a passphrase twice, and invoke the MOK import EFI interface on a reboot.

• If you can't boot into the desktop due to unsigned GPU driver -- which could happen if you didn't sign the kernel module, or if you did a BIOS update/reset that erased previous MOKs, just try booting into recovery mode from grub, become root, and do the dpkg-reconfigure fix above.

Step 0. Prequisites

(Click to expand/collapse)

• Make sure the packages openssl, mokutil and dkms are installed. You need to enable the EPEL repo to get dkms for RHEL/CentOS.

• You also need the necessary compilers and other build tools to build your module, like gcc and make. For CentOS users I suggest simply installing them by dnf groupinstall development.

• The kernel source, provided by kernel-devel in Fedora/CentOS, or linux-headers-generic in Debian/Ubuntu.

Step 1. Generating a MOK and Enrolling It in Secure Boot

(Click to expand/collapse)

• Start by becoming root with sudo -i.

• Generate the key and certificate.

  openssl req -new -x509 \
      -newkey rsa:2048 -keyout /root/nvidia-driver.key \
      -outform DER -out /root/nvidia-driver.der \
      -nodes -days 36500 -subj "/CN=Nvidia Driver Kmod Signing MOK"
  

  (The key and the certificate filenames, paths, expiration date and subject can be modified to your liking.)

• Enroll the public key.

  mokutil --import /root/nvidia-driver.der
  

  You'll be prompted to create a password. Enter it twice.

• Reboot the computer. At boot you'll see the MOK Manager EFI interface. Press any key to enter it.

  • "Enroll MOK"
  • "Continue".
  • "Yes".
  • Enter the password you set up just now.
  • Select "OK" and the computer will reboot again.

• After reboot, you should be able to see the new key with cat /proc/keys | grep asymmetri as root.

Step 2. Configuring DKMS to Sign Newly-Built Modules Automatically with the MOK.

(Click to expand/collapse)

• To minimize human effort and troubleshooting, it's best to get the keys, config files and scripts in place before installing any actual drivers.

• Create a text file /etc/dkms/nvidia.conf, or /etc/dkms/<module-name>.conf for other modules (with <module-name> part exactly matching the name of the module), which is a one-liner pointing to the signing script.

  echo "SIGN_TOOL=/root/sign-nvidia-driver.sh" > /etc/dkms/nvidia.conf
  

  (I put the script under /root but obviously you can adjust its path.)

• DKMS will pass to our script the kernel version number as $1 and the full path to module file as $2. We'll use the sign-file tool from the kernel-devel package, which needs to be supplied with more info.

  Create the script /root/sign-nvidia-driver.sh which simply provides the correct argument for sign-file as follows:

  #!/bin/bash
  # sign-nvidia-driver.sh
  
  hash_algo=sha256
  private_key=/root/nvidia-driver.key
  x509_cert=/root/nvidia-driver.der
  
  prefix=/usr/src/kernels/
  # For Debian/Ubuntu, use
  #prefix=/usr/src/linux-headers-
  
  "${prefix}${1}/scripts/sign-file" \
      "${hash_algo}" "${private_key}" "${x509_cert}" "${2}" \
      && echo "Signed newly-built module ${2} with MOK successfully." >&2 \
      && exit 0
  echo "Error signing file ${2}." >&2
  exit 1
  

  Remember to chmod +x /root/sign-nvidia-driver.sh.

  The script returns 0 when the signing succeeds and 1 when it fails. A non-zero return value will cause the DKMS build operation to fail. Corresponding message will be printed to stderr

Step 3. Installing the Nvidia Driver and Registering It With DKMS.

(Click to expand/collapse)

• Become root again. Blacklist nouveau in modprobe.d and your initramfs (the latter will vary with distros).

  echo "blacklist nouveau" > /etc/modprobe.d/blacklist-nouveau.conf
  echo 'omit_drivers+="nouveau"' > /etc/dracut.conf.d/blacklist-nouveau.conf
  dracut -f
  

• It usually suffices to exit the graphical session with systemctl isolate multi-user and login onto a text mode console as root. But if the driver install fails, try rebooting into text mode.

• Install the DKMS driver. For Nvidia GPUs, people typically download, chmod and execute the runfile driver from Nvidia website which will ask you to register it with DKMS.

  I also have success with the package kmod-nvidia-latest-dkms by following Nvidia's instructions to install CUDA. This makes updating easier, but it doesn't come with 32-bit libraries necessary for e.g. Steam.

• After install, run dkms status and you should see the nvidia module in "installed" state with DKMS. lsmod | grep nvidia should show you that the modules are sucessfully loaded.

  Now you can go back to GUI with systemctl isolate graphical. Viola!

• Note 1: If you previously installed some DKMS driver package without setting up proper signing process, the module will be shown as "built" or "installed" by dkms status but the kernel will refuse to load it unless you disable Secure Boot. In this case, simply remove the modules and rebuild them with dkms with proper signing:

  dkms remove nvidia/440.64.00 --all
  dkms add nvidia/440.64.00
  dkms autoinstall
  

  Again, replace the <module-name>/<version> part with your own.

• Note 2: if you are using Nvidia's runfile installer, it is unnecessary to pass the keys as parameters to the installer, because the installer also calls DKMS to do the kmod building which should hook up the signing program automatically with our setup in place. If you did run the installer with Secure Boot on but w/o signing the driver (either by passing keys as parameters or with the method detailed in this gist), the installation would fail because the unsigned module would be rejected by the kernel.

At least for debian users, don't forget that the correct path begins with /usr/src but NOT /usr/src/kernels

When it's time to update drivers or the kernel - which steps of this gist should we repeat?

I just updated the Kernel on a Fedora 34 installation, and my graphics drivers fell back to lvmpipe. Trying to figure out how to re-load Nvidia drivers.

Adding onto @NewRedsquare 's comment, for Debian 11, I had to use "/usr/src/linux-headers-${1}/scripts/sign-file" instead of "/usr/src/kernels/${1}/scripts/sign-file". Also, the SIGN_TOOL value was not being picked up when I was trying to add nvidia-current to dkms. (Note: I have just been installing the Nvidia drivers / CUDA via package managers, not via .sh/.run files.) What ended up working was adding that SIGN_TOOL value to the end of /etc/dkms/framework.conf. I'm pretty sure that sets that to be the SIGN_TOOL for all DKMS modules, which is probably undesirable, but since the NVIDIA driver is the only DKMS module I have installed that requires installation, this worked for me. THANK YOU for making this guide!

P.S. I tried making /etc/dkms/nvidia-current.conf with that SIGN_TOOL value, but it still wasn't being picked up. Not sure why dkms was not reading that conf file / adopting that SIGN_TOOL value, open to any ideas.

In step 3 there is a small typo (extra . after conf.d) in blacklisting nouveau in dracut. The line should be:
echo 'omit_drivers+="nouveau"' > /etc/dracut.conf.d/blacklist-nouveau.conf

@DABH Sorry for replying late (because I've only recently switched to a Debian-based distro). You have to name the config file as nvidia.conf. Its name has to perfectly match the module name. nvidia-current is the name of the package, but not name of the k-mod.

Also, it might not be necessary to do all the tricks described here. Just try running dpkg-reconfigure nvidia-dkms-<ver#> (if you can't boot into the GUI, just select recovery mode in grub), with Secure Boot on, and see if it can set up a MOK for you.

P.S. I tried making /etc/dkms/nvidia-current.conf with that SIGN_TOOL value, but it still wasn't being picked up. Not sure why dkms was not reading that conf file / adopting that SIGN_TOOL value, open to any ideas.

FYI, this was a deliberate change made in 2020. It will only look at the sign_tool variable in framework.conf dkms-project/dkms@597f576

This doesn't seem necessary anymore. In addition to sign_tool being in framework.conf you can just set mok_signing_key and mok_certificate in the same file...

   $sign_file
          This is the path of the sign-file kernel binary that is used to sign the kernel modules. The path for the binary depends on the distribution.

   $mok_signing_key, $mok_certificate
          Location of the key and certificate files used for Secure boot.  mok_signing_key can also be a "pkcs11:..." string for PKCS#11 engine, as long as the sign_file program supports it.

In my case (ZFS on RHEL9) I just had to enroll /var/lib/dkms/mok.pub (which already existed!) and rebuild the module via dkms.

Same for my case on Redhat 8.7 and ext4, I had only have to enrol the /var/lib/dkms/mok.pub (coming from dkms packet). Modules was already build via dkms, no need to rebuild.
See https://github.com/dell/dkms#secure-boot
On second partition, things are little more complex with Debian 11. Dkms was version 2.8.4-3
Then I installed the version dkms-3.0.10 from testing , rebuild the modules via dkms, et voila.