Building U-Boot and Linux 3.11 from scratch for the BeagleBone, and booting

Author: Philippe Proulx
Date: Fri Sep 13 18:05:26 EDT 2013

Introduction

Here's a really "simple" guide on how to build U-Boot and the 3.11 Linux kernel from scratch for the BeagleBone SoC. This is mostly a reminder of the steps for myself and shows what I recently learned from lots of different sources and by trial and error.

There are faster/automated ways to do all this. Buildroot has a BeagleBone target. Also, the Ångström distribution, the "sort of" official BeagleBone distribution, can be used, but it's probably not using the latest kernel release. This document explains the raw steps to build everything from scratch for didactic purposes.

Download links

As of this date, you can get the needed files from here:

U-Boot: ftp://ftp.denx.de/pub/u-boot/u-boot-latest.tar.bz2
Kernel: git clone git://github.com/beagleboard/kernel.git
Latest Busybox ARMv7 binary: http://www.busybox.net/downloads/binaries/latest/busybox-armv7l
ARM cross-compiling toolchain: there are lots of sources for this; just make sure you get something with a Linux target, like arm-linux-eabi. I used the Buildroot downloaded tools (without actually using Buildroot for the rest of the process).

We cannot use the vanilla kernel as is since some important patches that need to be applied are not mainlined yet. This is why we're using the BeagleBoard GitHub repository (contains valid patches and scripts for BeagleBoard/BeagleBone kernels).

Building U-Boot

The first step is to build the bootloader, U-Boot. Two important files will come out of this process: MLO and u-boot.img. MLO is what the U-Boot community calls the SPL (Secondary Program Loader) and contains executable code for the second boot stage. Here are the stages as a reminder (for an MMC boot):

the on-board ROM initializes a few cores and searches for a file called MLO on the first FAT partition of the first MMC card (the one that has a socket on the BeagleBone), then loads it in memory and executes it
the SPL (MLO) initializes a few other things and searches on the same partition for u-boot.img, which is the third boot stage (the actual complete U-Boot program), loads it in memory and executes it
U-Boot starts and after 1-2 seconds, you can get a prompt

MLO is compiled by U-Boot too and the vanilla U-Boot knows how to produce this SPL (it has a board target for this).

Download U-Boot and extract it:

$ wget ftp://ftp.denx.de/pub/u-boot/u-boot-latest.tar.bz2
$ tar -xjvf u-boot-latest.tar.bz2

All the available boards are listed in boards.cfg. In there you will find am335x_evm, evm meaning evaluation module. am335x_evm is a common name for the SoC used by the BeagleBone boards (exact SoC is an AM3359). Although it's not obvious, this configuration row points to the include file include/configs/am335x_evm.h, defines SERIAL1 and also CONS_INDEX to 1.

This file, am335x_evm.h, contains all the specific configurations U-Boot needs in order to build MLO and u-boot.img files that the SoC will understand. Amongst them is the CONFIG_EXTRA_ENV_SETTINGS definition, which embed environment variables into the U-Boot output image file. The less we put there, the more manual will be the boot process, and that's what we want here since we're doing everything from scratch to understand.

In include/configs/am335x_evm.h, remove the following default environment variables (CONFIG_EXTRA_ENV_SETTINGS):

all the *args variables (including bootargs)
all the *boot variables
loadaddr
fdtaddr
rdaddr
bootfile
fdtfile
loadbootenv
importbootenv
loadramdisk
loaduimagefat
loaduimage
findfdt

Also comment all the CONFIG_BOOTCOMMAND definition block. Save the file.

This way, we have something minimalist and we're happy. To make sure you're using your own version of U-Boot, you may also change the prompt (CONFIG_SYS_PROMPT) for something you'll recognize.

Leave the rest as is, although there's still stuff we don't need. This will at least provide us with a clean U-Boot environment, without predefined stuff we don't want/understand.

We're now ready to build U-Boot's main configuration:

$ make ARCH=arm CROSS_COMPILE=/path/to/your/toolchain/arm-something-something- am335x_evm_config

This basically creates include/config.h which includes relevant files for the am335x_evm configuration. You will also see the additional definitions in boards.cfg are added at the top of include/config.h:

#define CONFIG_SERIAL1 1
#define CONFIG_CONS_INDEX 1

We can now build U-Boot:

$ make ARCH=arm CROSS_COMPILE=/path/to/your/toolchain/arm-something-something- -j4

Wait a few seconds and you will get MLO and u-boot.img in U-Boot's root directory. So far so good.

In order to build the Linux kernel as an uImage, which is a zImage with the U-Boot legacy image header, we need an installed tool called mkimage which a kernel Makefile will call.

This can be built and installed now:

$ make tools
# install tools/mkimage /usr/local/bin

Just make sure you can execute it:

$ mkimage -h

You might need to restart your shell in order to rescan the $PATH binaries if it says something like command not found.

Building the Linux kernel

Let's now build Linux.

First step is to clone the aforementioned GitHub repo:

$ git clone git://github.com/beagleboard/kernel.git

and checkout whatever version tag you want.

Let us checkout the 3.12 branch (which is really a 3.11 kernel):

$ git checkout origin/3.12 -b 3.12

Then, apply the patches (could be a long step because it actually clones Torvald's kernel, checksout the correct version and then applies lots of patches to it, which are in the patches directory):

$ ./patch.sh

After this, you shall see a kernel directory which contains the patched Linux source. Some steps are still required before actually building it.

$ cd kernel

You will see that the current branch is tmp-patching-branch, master still being a clean branch without any patches. Stay in tmp-patching-branch.

The patch will have downloaded firmware/am335x-pm-firmware.bin, which is a needed power management firmware that can be found here.

TI puts default .config files into configs (from the repo's root), so use BeagleBone's default .config like so:

$ cp ../configs/beaglebone .config

We may now compile the kernel:

$ make ARCH=arm CROSS_COMPILE=/path/to/your/toolchain/arm-something-something- -j4 uImage dtbs

Let me explain what important files in arch/arm/boot those two targets will create while this is compiling:

zImage: this is a self-extracting compressed kernel. You execute this file, and it has a decompression algorithm to extract the rest of it, which is the actual kernel to execute.
uImage: this is zImage with a 64-byte U-Boot header so that U-Boot knows a few things when asked to boot this file (the previously installed mkimage tool is used to create this out of zImage).
dts/am335x-bone.dtb: the compiled device tree "blob" which the kernel must have in order to initialize its drivers and SoC-specific routines.

At the end of the process, you will see:

Image arch/arm/boot/uImage is ready

and will be happy about it. mkimage will also report the load address and entry point of the created image, both of which should be 0x80008000 (external DRAM starts at 0x80000000 on the AM335x SoC).

Usually, when booting (we will do this later), you need to load uImage and dts/am335x-bone.dtb at different memory locations, then ask U-Boot to boot by providing those two addresses. But there's another way to give the DTB to Linux. In .config, you will see

CONFIG_ARM_APPENDED_DTB=y

which tells the kernel to look at the end of its zImage for a valid DTB if you don't provide it with a memory address when booting. So you only need to concatenate dts/am335x-bone.dtb to zImage and then use mkimage to create a uImage with appended DTB from this.

This is exactly what the uImage-dtb.am335x-bone target does:

$ make ARCH=arm CROSS_COMPILE=/path/to/your/toolchain/arm-something-something- uImage-dtb.am335x-bone

Now you also have arch/arm/boot/uImage-dtb.am335x-bone. Its size should be the size of zImage + the size of dts/am335x-bone.dtb + 64.

Creating a root filesystem

The kernel will be able to start booting with the above files, but won't be able to finish because it will complain it cannot find any root filesystem. We need a very basic one.

Let us create a minimalist rootfs from scratch.

Create a new directory for the root filesystem:

$ mkdir rootfs
$ cd rootfs

As root, create the basic directory structure:

# mkdir bin dev proc sys

Download the latest Busybox build for ARMv7 and make it executable:

# wget http://www.busybox.net/downloads/binaries/latest/busybox-armv7l -O bin/busybox
# chmod +x bin/busybox

Create Busybox init and sh links:

# ln -s busybox bin/init
# ln -s busybox bin/sh

That's really all we need for Busybox: an init and a shell (in fact, /bin/sh could be the init itself). We'll execute everything else by calling busybox directly once in that shell (think minimalist).

Create a few important /dev nodes:

# mknod dev/console c 5 1
# mknod dev/null c 1 3

So here we have it:

# tree
.
├── bin
│   ├── busybox
│   ├── init -> busybox
│   └── sh -> busybox
├── dev
│   ├── console
│   └── null
├── proc
└── sys

Bringing it all together: creating the SD card

Let's review what we have:

U-Boot's MLO
U-Boot's u-boot.img
Linux's uImage-dtb.am335x-bone
our root filesystem

We need to put all this on a properly formatted SD card. Until the end of this document, my SD card device is /dev/sdb. Yours might be different: be careful here.

Delete all existing SD card partitions. Use fdisk with the d command to delete everything and write it with w. You might need to unplug/replug your SD card for the changes to be seen by the kernel.

Create a FAT partition and an ext4 partition:

# fdisk /dev/sdb <<EOF
n
p
1

+16M
t
4
a
n
p
2


t
2
83
w
EOF

Again, unplug/replug the device.

Format both partitions:

# mkfs.vfat -F16 -v /dev/sdb1
# mkfs.ext4 /dev/sdb2

Mount both partitions:

# mkdir /mnt/bbone_sd1
# mkdir /mnt/bbone_sd2
# mount /dev/sdb1 /mnt/bbone_sd1
# mount /dev/sdb2 /mnt/bbone_sd2

Now, don't ask me why, but MLO needs to be in the very first sectors of the partition. Copying it first ensures this. Everything else can be copied in any order.

Copy files to the FAT partition:

# cd /mnt/bbone_sd1
# cp /path/to/MLO .
# cp /path/to/u-boot.img .
# cp /path/to/uImage-dtb.am335x-bone .
# touch uEnv.txt

The empty file uEnv.txt is read by U-Boot and can be used to dynamically set initial environment variables. We're creating an empty one just to avoid U-Boot complaining it cannot read it.

Copy the root filesystem to the ext4 partition:

# cd /mnt/bbone_sd2
# cp -a /path/to/rootfs/* .

Unmount both partitions:

# cd /mnt
# umount bbone_sd1
# umount bbone_sd2

Ready to boot!

Booting

Plug the USB BeagleBone cable into your host machine and plug the external power into the BeagleBone. On your host machine, look at a TTY named /dev/ttyUSB0 or such. Use screen to connect to it at the correct baud rate (always 115200 bauds):

# screen /dev/ttyUSB0 115200,cs8

Unplug the SD card from your host machine and plug it into the BeagleBone. Press the BeagleBone reset button near the RJ-45 socket.

After a few seconds, you should get a U-Boot prompt.

Now, we need to do 3 important things:

set the kernel boot parameters
copy the kernel image (uImage-dtb.am335x-bone) in memory
boot that memory location

This is really easy using U-Boot.

At the U-Boot prompt, type:

# setenv bootargs "earlyprintk console=ttyO0,115200n8 root=/dev/mmcblk0p2 rootwait ro rootfstype=ext4 init=/bin/init"

You might find it strange that we specify /dev/mmcblk0p2 as the root filesystem. What is /dev/mmcblk0p2 if there's no root filesystem yet? It turns out this is just a convention and it's not actually pointing to a specific file. The kernel understands a lot of device prefixes like this (sd, hd, fd, etc.). mmcblk0p2 is a synonym for the second partition of the first MMC device (i.e. our ext4 partition). Just a note: instead of this arbitrary synonym, we could write the complete (still arbitrary) device number. From Documentation/devices.txt, we can see that the second partition of the first MMC block device is major number 179 and minor number 2 (which means device number 0xb302), so the root parameter could be root=0xb302 as well here.

earlyprintk is a parameter to tell the kernel to print to the console as soon as possible (even before the UART driver is initialized!). This is usually possible because the SoC specific code knows how to talk to its own UART (mostly seen on embedded devices).

ro mounts the root filesystem as read-only (as opposed to rw).

Now, the kernel image must be loaded at its load address (which is 0x80008000). However, if we copy the image contents to this memory location, U-Boot will get an exception from the ARM CPU when trying to boot because of unaligned memory accesses. This is because zImage must be aligned on 0x80008000, not uImage (the encapsulated zImage doesn't know about its uImage shell). So the uImage needs to be copied 64 bytes before 0x80008000 (64 bytes being the U-Boot header size): at 0x80007fc0.

Here's how to do it with U-Boot:

# fatload mmc 0:1 0x80007fc0 uImage-dtb.am335x-bone

which means: load file /uImage-dtb.am335x-bone from the FAT filesystem of the first partition of the first MMC device at memory location 0x80007fc0.

Now we can boot:

# bootm 0x80007fc0

Linux should start right away. After a few seconds, the kernel will execute its init (we specified /bin/init, which points to Busybox) and Busybox's init will ask:

Please press Enter to activate this console.

Do so.

You are now commanding the BeagleBone as root!

We could also have loaded uImage (with no appended DTB) and the DTB at separate memory locations and still use the bootm command this way:

# fatload mmc 0:1 0x80007fc0 uImage
# fatload mmc 0:1 0x81000000 am335x-bone.dtb
# bootm 0x80007fc0 - 0x81000000

The bootm command's second argument, here -, is the address of an initramfs, but we don't have any; bootm reads - as "no argument".

bootz can also be used to boot a zImage directly, although this is not recommended because uImage adds some protection around it (CRC check, etc.):

# fatload mmc 0:1 0x80008000 zImage
# fatload mmc 0:1 0x81000000 am335x-bone.dtb
# bootz 0x80008000 - 0x81000000

As you already noticed, we're loading zImage at 0x80008000, and not 0x80007fc0, since the U-Boot header is not present in zImage.

Please note that U-Boot must be built with bootz support to use it (I believe it's done by default since I may use it and I didn't modify the default build features).

Using Linux

Of course this system from scratch is not really useful. For instance, no command will be found since nothing is in /bin but init and sh. If you want to execute a command, precede it with busybox :

# busybox ls
bin dev lost+found proc sys

Still, if you list /proc or /sys, you will find empty directories. They are mount points, but nothing is mounted. So procfs can be mounted, for example:

# busybox mount -t proc proc /proc
# busybox cat /proc/cmdline 
earlyprintk console=ttyO0,115200n8 root=/dev/mmcblk0p2 rootwait ro rootfstype=ext4 init=/bin/init

Of course, the usual way of doing this is to use mount -a which reads /etc/fstab and mounts everything at once, but we don't have such a file.

Conclusion

So this is the very basis of an embedded Linux system creation. From there, you have something working and can experiment with more useful configurations:

automating the booting process (this can be done using uEnv.txt)
compiling kernel modules and installing them in the root filesystem (kernel targets modules and install_modules)
getting the kernel image using TFTP in U-Boot (U-Boot knows TFTP and even DHCP)
getting the root filesystem using NFS (Linux must be configured accordingly to understand NFS and be able to boot a remote filesystem)
installing a complete official distribution, like Arch Linux ARM, which is easy to do

eepp/BeagleBoneLinuxUBootFromScratch.md