Steps to build Bazel in QEMU environment for PYNQ images

build-bazel-qemu.md

Building Bazel on QEMU

Here you'll learn how to build Bazel targetting PYNQ image (ZCU104 or Pynq-Z1) with QEMU environment.

I am using ZCU104 as an example; Pynq-Z1 is very similar (change ZCU104 to Pynq-Z1 where it is needed in my steps). Other Zynq and Zynq Ultrascale images should also work.

Requirement

• A Ubuntu machine as the building environment for PYNQ images.

Steps

1. Prepare the Building Environment

You need to prepare a QEMU environment. Usually, if you have a PYNQ building environment (by running setup_host.sh), you will already have a QEMU 4.0.0 installed in your Ubuntu system under /opt.

2. Log onto QEMU

An existing PYNQ image can be mounted so you have a QEMU environment of your ZCU104. You can leverage my script, which helps you prepare or clean the mounted images and drives.

Note that you may need to change a few variables in the script, based on your system settings; for example, you may need to change the board name and image location accordingly.

chmod 777 qemu_on_off.sh
./qemu_on_off.sh

In addition, the script will also prepare a large /local/tmp folder in QEMU mounted to /tmp of your host machine; you can also adjust this if you have some other large drives.

3. Chroot environment

Run (change the path accordingly based on your needs):

sudo -E chroot /opt/builds/PYNQ_20180529/sdbuild/build/bionic.ZCU104 bash

Now you have an environment just like you are on the ZCU104 board. From now on, we assume you are on that chroot environment.

Once you are on that chroot environment, you can double-check that you have enough disk space for your /local/tmp folder using df -h. For me, this is 100+ GB.

4. Install Basic Dependencies

Install java now. The default openjdk 11 does not work well with bazel so I have to purge it and install openjdk 8.

apt-get update
apt-get remove --purge openjdk* java-common default-jdk
apt-get autoremove --purge
apt-get install openjdk-8-jdk

You need to make sure you can compile a Java program. Create a file /local/tmp/HelloWorld.java as follows:

public class HelloWorld {
    public static void main(String[] args) {
        // Prints "Hello, World" to the terminal window.
        System.out.println("Hello, World");
    }
}

Compile it and run it to make sure nothing is broken.

cd /local/tmp
javac HelloWorld.java
java HelloWorld

Finally, for cleanliness, make a directory that will hold the Bazel repository.

cd /local/tmp
mkdir work
cd work

5. Make Patches

To build Bazel, we're going to need to download a zip file containing a distribution archive. Let's do that now and extract it into a new directory called bazel:

wget https://github.com/bazelbuild/bazel/releases/download/0.29.1/bazel-0.29.1-dist.zip
unzip -d bazel bazel-0.29.1-dist.zip
cd bazel

Now we need to change the permissions of every files in the bazel project with:

chmod u+w ./* -R

Run the following to set environment:

export JAVA_HOME="$(dirname $(dirname $(realpath $(which javac))))"
export TMPDIR=/local/tmp

Patch 1

Before building Bazel, we need to set the javac maximum heap size for this job, or else we could get an OutOfMemoryError. To do this, we need to make a small addition to bazel/scripts/bootstrap/compile.sh.

vi scripts/bootstrap/compile.sh

Move down to line around 152, where you'll see the following block of code:

run "${JAVAC}" -classpath "${classpath}" -sourcepath "${sourcepath}" \
      -d "${output}/classes" -source "$JAVA_VERSION" -target "$JAVA_VERSION" \
      -encoding UTF-8 "@${paramfile}"

At the end of this block, add in the -J-Xmx2048M flag, which sets the maximum size of the Java heap to 2048 MB:

run "${JAVAC}" -classpath "${classpath}" -sourcepath "${sourcepath}" \
      -d "${output}/classes" -source "$JAVA_VERSION" -target "$JAVA_VERSION" \
      -encoding UTF-8 "@${paramfile}" -J-Xmx2048M

Patch 2

Go to file:

vi src/tools/singlejar/mapped_file_posix.inc

Comment out the line:

#error This code for 64 bit Unix. 

Patch 3

We have a well-known issue that bazel cannot build itself on QEMU. After the following patch, we can fix it.

Go to file:

vi third_party/ijar/mapped_file_unix.cc

Change around line 116 into:

void* mapped = mmap(NULL, mmap_length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

Patch 4

Go to file:

vi src/main/cpp/blaze_util_linux.cc

Locate the following line around line 88:

return "/proc/self/exe";

Change it into:

static char path[PATH_MAX];
return realpath("/proc/self/exe", path);

6. Build Bazel

Now we can build Bazel!

env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" ./compile.sh

This takes a long time (at least 30 mins). When the build finishes, you end up with a new binary output/bazel. Copy that to your /usr/local/bin directory.

cp output/bazel /usr/local/bin/bazel

To make sure it's working properly, run bazel on the command line and verify it prints help text.

cd /local/tmp
bazel

Usage: bazel <command> <options> ...

Available commands:
  analyze-profile     Analyzes build profile data.
  build               Builds the specified targets.
  canonicalize-flags  Canonicalizes a list of bazel options.
  clean               Removes output files and optionally stops the server.
  dump                Dumps the internal state of the bazel server process.
  fetch               Fetches external repositories that are prerequisites to the targets.
  help                Prints help for commands, or the index.
  info                Displays runtime info about the bazel server.
  mobile-install      Installs targets to mobile devices.
  query               Executes a dependency graph query.
  run                 Runs the specified target.
  shutdown            Stops the bazel server.
  test                Builds and runs the specified test targets.
  version             Prints version information for bazel.

Getting more help:
  bazel help <command>
                   Prints help and options for <command>.
  bazel help startup_options
                   Options for the JVM hosting bazel.
  bazel help target-syntax
                   Explains the syntax for specifying targets.
  bazel help info-keys
                   Displays a list of keys used by the info command.

Now you can copy that binary around onto any ZCU104 board with PYNQ.

You can even copy that binary onto any other Zynq Ultrascale board, such as Ultra96, since the binary should be compatible with aarch64 architecture. Similarly, if you build bazel for Pynq-Z1, you can also copy the binary built for Pynq-Z1 to other Zynq 7000 boards, since the binary will then be compatible with armv7l architecture.

Congratulations!

7. Cleaning Up

You can exit the chroot environment now.

exit

Use my script again to clean the mounted images and drives:

./qemu_on_off.sh

Finally check you can check whether you have a clean environment again. The following command should return nothing.

sudo losetup -a