Hardening the USB Armory

As a good crypto nerd, I usually use an entirely encrypted linux FS: / but also /boot using grub LUKS support. It's a good setup but it's not perfect, the BIOS and the bootloader are not protected.

I recently got a USBArmory and I wanted to apply the same (or a better) setup.

I found some useful links but no clear howto. So this is my setup.

This tutorial is for Archlinux ARM but you can use it as a base for other distributions, the principle is the same.

Requirements

USBArmory
USB to TTL cable (used to check the signed /boot and the signed bootloader)
Archlinux ARM

Steps

First of all, we'll use a USBArmory feature in this setup: the Secure Boot. It allow us to sign the bootloader.

The bootloader is u-boot (grub doesn't support usbarmory). U-boot doesn't support LUKS decryption but it supports Verified boot which will check the kernel and some files (usually in /boot).

And of course, the root system will be a LUKS partition.

I strongly suggest you backup your usbarmory before every step.

Encrypted root

To decrypt the LUKS root FS, we need to setup an ssh server when the kernel boots.

This part is Archlinux specific. For Debian, you can use this link: http://hacksr.blogspot.de/2012/05/ssh-unlock-with-fully-encrypted-ubuntu.html (I didn't test it).

We work on the usbarmory, prepare everything and then edit the SD card on a desktop, create two partitions /boot and / and encrypt the / partition.

Install yaourt (see https://archlinux.fr/yaourt-en).
You should choose between the ssh server dropbear or the ssh server tinyssh. Dropbear doesn't support ed25519 and tinyssh doesn't support RSA/DH. Install requirements in user mode:

alarm@usbarmory$ sudo yaourt -Sy mkinitcpio-netconf mkinitcpio-dropbear mkinitcpio-utils

alarm@usbarmory$ sudo yaourt -Sy mkinitcpio-netconf mkinitcpio-tinyssh mkinitcpio-utils

For tinyssh, you should edit the PKGBUILD of tinyssh and ucspi-tcp and add 'armv7h' in arch.

Copy your desktop public key in /etc/dropbear/root_key or /etc/tinyssh/root_key (as in ~/.ssh/authorized_keys).
Edit /etc/mkinitcpio.conf, in the MODULES array, add g_cdc usb_f_acm usb_f_ecm. In HOOKS, before filesystems add netconf dropbear encryptssh or netconf tinyssh encryptssh.
Then, create the ramdisk:

root@usbarmory# mkinitcpio -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img

Edit kernel args to activate network and LUKS root. Edit /boot/boot.txt, replace:

setenv bootargs console=ttyGS0,115200 console=${console} root=PARTUUID=${uuid} rw rootwait

with:

setenv bootargs console=ttyGS0,115200 console=${console} ip=10.0.0.1::10.0.0.2:255.255.255.0::usb0:none cryptdevice=/dev/mmcblk0p2:root root=/dev/mapper/root rw rootwait

The ip= argument is for the default ArchlinuxARM setup: a static ip 10.0.0.1 with gateway 10.0.0.2. For more information: https://wiki.archlinux.org/index.php/Mkinitcpio#Using_net.

Be sure to have uboot-tools installed. Then:

root@usbarmory# cd /boot
root@usbarmory# ./mkscr

Edit /etc/fstab and add:

/dev/mmcblk0p1    /boot    none    0    2

Halt your usbarmory and put the microSD in your computer. Then backup your FS:

desktop# mount /dev/mmcblk0p1 /mnt
desktop# cd /mnt
desktop# tar cf /tmp/armory_backup.tar *
desktop# cd && umount /dev/mmcblk0p1

Now, rewrite your partition table, the first partition must be /boot and the second must be /.

desktop# fdisk /dev/mmcblk0

For example:

Device         Boot    Start      End  Sectors  Size Id Type
/dev/mmcblk0p1          2048   526335   524288  256M 83 Linux
/dev/mmcblk0p2        526336 27789311 27262976   13G 83 Linux

Now we recreate FS:

desktop# mkfs.ext4 /dev/mmcblk0p1
desktop# cryptsetup luksFormat /dev/mmcblk0p2 --hash=sha256
desktop# cryptsetup luksOpen /dev/mmcblk0p2 usb
desktop# mkfs.ext4 /dev/mapper/usb
desktop# mount /dev/mapper/usb /mnt
dekstop# mkdir /mnt/boot
desktop# mount /dev/mmcblk0p1 /mnt/boot
desktop# cd /mnt && tar xf /tmp/armory_backup.tar
desktop# dd if=boot/u-boot.imx of=/dev/mmcblk0 bs=512 seek=seek2 conv=fsync
dekstop# sync
desktop# cd && umount /dev/mmcblk0p1 && umount /dev/mapper/usb
dekstop# cryptsetup luksClose usb

Now fingers crossed and boot your usbarmory. The LED should blink, the kernel should be up and dropbear/tinyssh waiting to unlock your root partition:

dekstop$ ssh [email protected]
Enter passphrase for /dev/mmcblk0p2:
Connection to 10.0.0.1 closed.

If you chose tinyssh, your first ssh will get a warning, add the public key in your ~/.ssh/known_hosts. This happens because tinyssh generates his key when you install it while dropbear copy existing ssh key (/etc/ssh).

Now the kernel boot is done, you can ssh as usual.

Signed /boot

To use the Verified boot feature of u-boot, we need to compile a custom u-boot and create a FIT image which will contains the kernel, the initramfs and the signatures. The public key used to sign the FIT image will be stored in the u-boot binary using a control device tree.

Compile custom u-boot to enable FIT image support and FIT signature. Also modify the boot args and put the /boot/boot.txt contents directly into the u-boot binary.

alarm@usbarmory ~$ git clone https://github.com/u-boot/u-boot.git
alarm@usbarmory ~$ cd u-boot

Use this .patch for custom u-boot, you can put it into /tmp/uboot.patch for instance:

diff --git a/configs/usbarmory_defconfig b/configs/usbarmory_defconfig
index c25d103..2087577 100644
--- a/configs/usbarmory_defconfig
+++ b/configs/usbarmory_defconfig
@@ -1,5 +1,12 @@
 CONFIG_ARM=y
 CONFIG_ARCH_MX5=y
 CONFIG_TARGET_USBARMORY=y
+CONFIG_FIT=y
+CONFIG_FIT_VERBOSE=y
+CONFIG_FIT_SIGNATURE=y
+CONFIG_DM=y
+CONFIG_RSA=y
+CONFIG_OF_CONTROL=y
+CONFIG_OF_SEPARATE=y
 # CONFIG_CMD_IMLS is not set
 # CONFIG_CMD_SETEXPR is not set
diff --git a/include/configs/usbarmory.h b/include/configs/usbarmory.h
index 714e3e2..43d76a8 100644
--- a/include/configs/usbarmory.h
+++ b/include/configs/usbarmory.h
@@ -81,10 +81,10 @@
 #define CONFIG_HOSTNAME                usbarmory
 #define CONFIG_BOOTCOMMAND                                             \
        "run distro_bootcmd; "                                          \
-       "setenv bootargs console=${console} ${bootargs_default}; "      \
-       "ext2load mmc 0:1 ${kernel_addr_r} /boot/uImage; "              \
-       "ext2load mmc 0:1 ${fdt_addr_r} /boot/${fdtfile}; "             \
-       "bootm ${kernel_addr_r} - ${fdt_addr_r}"
+       "setenv bootargs console=ttyGS0,115200 console=${console} ip=10.0.0.1::10.0.0.2:255.255.255.0::usb0:none cryptdevice=/dev/mmcblk0p2:root root=/dev/mapper/root rw rootwait;" \
+       "if load ${devtype} ${devnum}:${bootpart} 0x80000000 /uImage; then" \
+       "      bootm 0x80000000;" \
+       "fi"

 #define BOOT_TARGET_DEVICES(func) func(MMC, mmc, 0)

and apply it:

alarm@usbarmory ~/u-boot$ git apply /tmp/uboot.patch

Now we compile u-boot tools:

alarm@usbarmory ~/u-boot$ make distclean && make usbarmory_config && make ARCH=arm tools

Prepare /boot and /sboot. I suggest you mount your first partition (actual /dev/mmcblk0p1) on /sboot and copy its contents into /boot (in the LUKS partition) so the clear partition /dev/mmcblk0p1 will only contains the signed FIT image and an Archlinux update will not impact this partition. Edit /etc/fstab and replace /boot by /sboot. Then:

root@usbarmory# umount /dev/mmcblk0p1
root@usbarmory# mount -a
root@usbarmory# mv /sboot/* /boot/

Now /sboot is empty, don't reboot !

Create working dir and gen 2048 bits RSA key and cert:

alarm@usbarmory$ mkdir working && cd working
alarm@usbarmory ~/working$ openssl genrsa -F4 -out ubootfit.key 2048
alarm@usbarmory ~/working$ openssl req -batch -new -x509 -key ubootfit.key -out uboot.crt

You can use another name for your key but you should replace ubootfit in the next files by your key name.

Create the DTS (device tree source) of the control device tree which will be stored in the u-boot binary (later in the MBR). Write the following in uboot.dts:

/dts-v1/;

/ {
    model = "Keys";
    compatible = "inversepath,imx53-usbarmory", "fsl,imx53";

    signature {
        key-ubootfit {
            required = "conf";
            algo = "sha256,rsa2048";
            key-name-hint = "ubootfit";
        };
    };
};

Install dtc:

usbarmory# pacman -S dtc

Now, generate the DTB (device tree blob) which will be the control device tree:

alarm@usbarmory ~/working$ dtc -p 0x1000 uboot.dts -O dtb -o uboot.dtb

uboot.dtb is the control device tree, it'll be copied in the u-boot binary when we compile it. uboot.dtb doesn't contain the public key yet.

Copy the kernel, initramfs and usbarmory fdt into the working dir:

alarm@usbarmory ~/working$ cp /boot/zImage ./
alarm@usbarmory ~/working$ cp /boot/initramfs-linux.img ./
alarm@usbarmory ~/working$ cp -r /boot/dtbs ./

Create the FIT image. Write the following in image.fit:

/dts-v1/;

/ {
    description = "USB Armory Kernel";
    #address-cells = <1>;

    images {
        kernel@1 {
            description = "arch kernel";
            data = /incbin/("./zImage");
            type = "kernel";
            arch = "arm";
            os = "linux";
            compression = "none";
            load = <0x70800000>;
            entry = <0x70800000>;
            hash@1 {
                algo = "sha256";
            };
        };

        ramdisk@1 {
            description = "initramfs";
            data = /incbin/("./initramfs-linux.img");
            type = "ramdisk";
            arch = "arm";
            os = "linux";
            compression = "gzip";
            load = <0x73000000>;
            entry = <0x73000000>;
            hash@1 {
                algo = "sha256";
            };
        };

        fdt@1 {
            description = "imx53-usbarmory.dtb";
            data = /incbin/("./dtbs/imx53-usbarmory.dtb");
            type = "flat_dt";
            arch = "arm";
            compression = "none";
            load = <0x71000000>;
            entry = <0x71000000>;
            hash@1 {
                algo = "sha256";
            };
        };
    };

    configurations {
        default = "config@1";

        config@1 {
            description = "default";
            kernel = "kernel@1";
            ramdisk = "ramdisk@1";
            fdt = "fdt@1";
            signature@1 {
                algo = "sha256,rsa2048";
                key-name-hint = "ubootfit";
                sign-images = "kernel", "ramdisk", "fdt";
            };
        };
    };
};

Generate the FIT image:

alarm@usbarmory ~/working$ ~/u-boot/tools/mkimage -D "-I dts -O dtb -p 2000" -f image.its uImage

The FIT image is not signed. This command will sign it and copy the public key into the control device tree:

alarm@usbarmory ~/working$ ~/u-boot/tools/mkimage -F -k ./ -K ./uboot.dtb -r uImage

Now, FIT image is generated, signed and the public key is in the control device tree.

Copy it in /sboot:

alarm@usbarmory ~/working$ sudo cp uImage /sboot/

Compile uboot with the control device tree:

alarm@usbarmory ~/u-boot$ make distclean && make usbarmory_config && make ARCH=arm EXT_DTB=~/working/uboot.dtb

Then copy it on the MBR:

alarm@usbarmory ~/u-boot$ dd if=u-boot-dtb.imx of=/dev/mmcblk0 bs=512 seek=2 conv=fsync
alarm@usbarmory ~/u-boot$ sync

Now remove uboot-usbarmory otherwise a uboot update will overwrite your custom u-boot:

root@usbarmory# pacman -Rns uboot-usbarmory

Reboot, using the USB to TTL cable, you should see:

CPU:   Freescale i.MX53 rev2.1 at 800 MHz
Reset cause: POR
Model: Keys
Board: Inverse Path USB armory MkI
I2C:   ready
DRAM:  512 MiB
MMC:   FSL_SDHC: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   CPU Net Initialization Failed
No ethernet found.
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
9419922 bytes read in 573 ms (15.7 MiB/s)
## Loading kernel from FIT Image at 80000000 ...
   Using 'config@1' configuration
   Verifying Hash Integrity ... sha256,rsa2048:ubootfit+ OK
   Trying 'kernel@1' kernel subimage
   ...

The line Verifying Hash Integrity ... sha256,rsa2048:ubootfit+ OK shows the verification of the FIT image was successful.

Now u-boot will only boot on a signed FIT image.

To update the FIT image, you don't need to recompile u-boot as long as the RSA key doesn't change. To generate a new FIT image and sign it (for example because a new kernel is available, or you need to update your ramdisk):

alarm@usbarmory ~/working$ ~/u-boot/tools/mkimage -D "-I dts -O dtb -p 2000" -f image.its uImage
alarm@usbarmory ~/working$ ~/u-boot/tools/mkimage -F -k ./ -r uImage

Signed bootloader

This part is supposed to sign the u-boot binary using the usbarmory SoC and to be the first link in the chain of trust of this setup.

This step is the most sensitive as you can only write once into the usbarmory SoC and it could brick your device.

I haven't tested this step because the tool provided by Freescale is no longer available. I'll update this step when a replacement is found.

This step is not described because this official link already does: https://github.com/inversepath/usbarmory/wiki/Secure-boot

Mayeu/hardening_usbarmory.md