Building Bazel for aarch64

I am building bazel for aarch64 on Amazon A1 platform. This flow is pretty nice and much easier than the QEMU flow which is prone to QEMU bugs.

Requirement

An Amazon A1 platform running Ubuntu 18.04.

For me, I have requested an a1.large instance, which has 2GB RAM. I also adjusted the disk space to 32GB.

A similar flow may be run on Ubuntu 16.04 but I have not verified that.

Steps

1. Prepare the Building Environment

Download Java 1.8

The first step is to make sure Java environment is good.

sudo apt-get update
sudo apt-get install -y openjdk-11-jdk-headless

Verify it is working by

javac -version
java -version

For me, it is 10.0.2.

Install Basic Dependencies

Install some packages that we will need. Some of the packages listed below may not be required for building bazel, but it does not hurt us to install them.

sudo apt-get install -y pkg-config zip g++ zlib1g-dev unzip autoconf automake libtool
sudo apt-get install -y python3-pip python3-numpy swig python3-dev
pip3 install wheel

You should be able to check the gcc/g++ version:

gcc -v
g++ -v

For me, this is 7.3.0.

Finally, for cleanliness, make a directory that will hold the Bazel repository. I am doing this under /opt/builds; you can do it differently.

mkdir /opt/builds
cd /opt/builds

2. Make Patches to Bazel

To build Bazel, we're going to need to download a zip file containing a distribution archive. Let's do that now and extract it into a new directory:

wget https://github.com/bazelbuild/bazel/releases/download/0.21.0/bazel-0.21.0-dist.zip
unzip -d bazel-0.21.0 bazel-0.21.0-dist.zip
cd bazel-0.21.0

Now we need to change the permissions of every files in the bazel project with:

chmod u+w ./* -R

Run the following to set Java environment:

export JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-arm64

Patch 1

Before building Bazel, we need to set the javac maximum heap size for this job, or else we'll get an OutOfMemoryError. To do this, we need to make a small addition to scripts/bootstrap/compile.sh.

vi scripts/bootstrap/compile.sh

Move down to line around 120, where you'll see the following block of code:

run "${JAVAC}" -classpath "${classpath}" -sourcepath "${sourcepath}" \
      -d "${output}/classes" -source "$JAVA_VERSION" -target "$JAVA_VERSION" \
      -encoding UTF-8 "@${paramfile}"

At the end of this block, add in the -J-Xmx2048M flag, which sets the maximum size of the Java heap to 2048 MB; this number depends on how large the memory is for your A1 instance and how much time you want to spend on building as well.

run "${JAVAC}" -classpath "${classpath}" -sourcepath "${sourcepath}" \
      -d "${output}/classes" -source "$JAVA_VERSION" -target "$JAVA_VERSION" \
      -encoding UTF-8 "@${paramfile}" -J-Xmx2048M

Patch 2

Go to file

vi tools/cpp/CROSSTOOL

Add the following lines when you locate linker_flag (around line 67). The flag has to be consistent with your gcc version.

linker_flag: "-Wl,-R/usr/lib/aarch64-linux-gnu"
cxx_builtin_include_directory: "/usr/lib/gcc/aarch64-linux-gnu/7/include"
cxx_builtin_include_directory: "/usr/lib/gcc/aarch64-linux-gnu/7/include-fixed"

4. Build Bazel

Now we can build Bazel! Based on a Github issues, we want to use local JDK without fetching from remote location

env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh

This takes a long time. When the build finishes, you end up with a new binary output/bazel. Copy that to your /usr/local/bin directory.

cp output/bazel /usr/local/bin/bazel

To make sure it's working properly, run bazel on the command line and verify it prints help text.

bazel

Usage: bazel <command> <options> ...

Available commands:
  analyze-profile     Analyzes build profile data.
  build               Builds the specified targets.
  canonicalize-flags  Canonicalizes a list of bazel options.
  clean               Removes output files and optionally stops the server.
  dump                Dumps the internal state of the bazel server process.
  fetch               Fetches external repositories that are prerequisites to the targets.
  help                Prints help for commands, or the index.
  info                Displays runtime info about the bazel server.
  mobile-install      Installs targets to mobile devices.
  query               Executes a dependency graph query.
  run                 Runs the specified target.
  shutdown            Stops the bazel server.
  test                Builds and runs the specified test targets.
  version             Prints version information for bazel.

Getting more help:
  bazel help <command>
                   Prints help and options for <command>.
  bazel help startup_options
                   Options for the JVM hosting bazel.
  bazel help target-syntax
                   Explains the syntax for specifying targets.
  bazel help info-keys
                   Displays a list of keys used by the info command.

Now you can copy that binary around onto any aarch64 board! Congratulations!

Hi yunqu!

I am very honor to learn your various project!

I tried to build Bazel.

So, I followed your steps.

Unfortunately, I got some error when I commanded env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh

I got this error

ERROR: /opt/builds/bazel-0.21.0/src/java_tools/buildjar/java/com/google/devtools/build/buildjar/javac/plugins/BUILD:16:1: Building src/java_tools/buildjar/java/com/google/devtools/build/buildjar/javac/plugins/libdependency.jar (3 source files) and running annotation processors (AutoAnnotationProcessor, AutoValueProcessor) failed (Exit 1): java failed: error executing command (cd /tmp/bazel_S8VWr3OV/out/execroot/io_bazel && \ exec env - \ LC_CTYPE=en_US.UTF-8 \ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin \ external/local_jdk/bin/java -Xverify:none -Xmx512m -XX:+TieredCompilation '-XX:TieredStopAtLevel=1' -jar bazel-out/host/bin/src/java_tools/buildjar/java/com/google/devtools/build/buildjar/bootstrap_VanillaJavaBuilder_deploy.jar @bazel-out/aarch64-opt/bin/src/java_tools/buildjar/java/com/google/devtools/build/buildjar/javac/plugins/libdependency.jar-0.params) Execution platform: @bazel_tools//platforms:host_platform src/java_tools/buildjar/java/com/google/devtools/build/buildjar/javac/plugins/dependency/StrictJavaDepsPlugin.java:180: error: incompatible types: int cannot be converted to com.sun.tools.javac.code.Lint.LintCategory note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output Target //src:bazel_nojdk failed to build INFO: Elapsed time: 1020.244s, Critical Path: 186.74s INFO: 1570 processes: 1341 local, 229 worker. FAILED: Build did NOT complete successfully
Do you have any solution?

Thanks a lot.

yunqu/build-bazel.md