Build and Install Ray on ARM (aarch64)

Ray_ARM64.md

Dependencies

Only dependencies which aren't highly likely to be featured in a ARM development environment are featured here, Obvious dependencies such as Python3, Python2, pip, wheel, GCC etc. aren't covered here.

Bazel

Note : Use bazel 1.x.x for Ray as 2.x.x is not supported.

sudo apt-get install build-essential openjdk-8-jdk unzip
export JAVA_HOME="/usr/bin/java"
wget https://github.com/bazelbuild/bazel/releases/download/1.0.0/bazel-1.0.0-dist.zip
unzip bazel-1.0.0-dist.zip
cd bazel-1.0.0
./compile.sh

py-spy

py-spy is a python profiling library written in rust. It's easier to install it via rust's package manager on ARM.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install py-spy

Download Ray from git

Download ray from git and checkout the commit without PyArrow dependency.

git clone https://github.com/ray-project/ray.git
git checkout fae99ecb8e8d750bddcb3674f720f068541dc15d

Patching BUILD.boost

Update: This patch has been added to the upstream - nelhage/rules_boost#168. So, one could skip this step. If you face the errors discussed in this issue - ray-project/ray#7184, then follow the patching process mentioned below.

The bazel build rules for installing boost library has assembly files which doesn't compile on ARM64 and missing some default conditions. I have prepared a patch to address it - https://gist.github.com/heavyinfo/25cf56fe0b5f8509dd0120257d008d3f

Download the patch and place it in ray/thirdparty/patches.

Add path to the patch here https://github.com/ray-project/ray/blob/fae99ecb8e8d750bddcb3674f720f068541dc15d/bazel/ray_deps_setup.bzl#L138.

Building ray

cd ray/python
python3 setup.py bdist_wheel

Installing ray

cd ray/python/dist
pip install ray-0.9.0.dev0-cp36-cp36m-linux_aarch64.whl

Troubleshooting

Permission errors

If there are any permission related errors, run the above commands with sudo after understanding the risks of doing so and that every installation command after that may require sudo.

Do you happen to have a link to the prebuilt wheel for easier installation? I'm having some trouble with building this on my Jetson. For some reason it fails to detect the py-spy library.

Also, do you think you could make a little how to guide to install JAX on Jetson devices? I think it's a pretty powerful library and the community could benefit from a little how to.

Py-spy can be built easily from the repo. Just

git clone https://github.com/benfred/py-spy
cd py-spy
python3 setup.py bdist_wheel # Or install if you want it on the local system

@smorad: I had to install cargo first to pip install py-spy on ARM where there is not a pre-built wheel.

Py-spy has ARM wheels on pypi now for armv7/aarch64 - this should be as easy as going pip install py-spy to install, and doesn't require building from source anymore: benfred/py-spy#328

Hi
I try to build ray for armv7 but there are still some errors. I started issue in ray discuss but I don't have more ideas how to fix it.
https://discuss.ray.io/t/ray-for-rapberry-pi-is-possible/451/12

I will be gratefull for any suggestions.

@PeterPirog I built it some time ago, the resulting ray wheel and dependencies are here: https://github.com/smorad/arm64-popular-wheels/tree/master/aarch64/ubuntu_20.04

@smorad thank you I will try it.

I have problem with error:

INFO: From Generating C++ proto_library //streaming:streaming_proto:
protobuf/streaming.proto:5:1: warning: Import google/protobuf/any.proto but not used.
ERROR: /home/pi/ray/BUILD.bazel:136:1: C++ compilation of rule '//:worker_rpc' failed (Exit 1) gcc failed: error executing command
  (cd /root/.cache/bazel/_bazel_root/27c0bb57e3de4f4c1e5336c83fd82d62/sandbox/linux-sandbox/1545/execroot/com_github_ray_project_ray && \
  exec env - \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \

Use --sandbox_debug to see verbose messages from the sandbox
In file included from bazel-out/arm-opt/bin/_virtual_includes/grpc_common_lib/ray/rpc/grpc_client.h:22,
                 from bazel-out/arm-opt/bin/_virtual_includes/worker_rpc/ray/rpc/worker/core_worker_client.h:33,
                 from bazel-out/arm-opt/bin/_virtual_includes/worker_rpc/ray/rpc/worker/core_worker_client_pool.h:21,
                 from src/ray/rpc/worker/core_worker_client_pool.cc:1:
bazel-out/arm-opt/bin/_virtual_includes/ray_common/ray/common/ray_config_def.h:200:75: error: integer overflow in expression of type 'long int' results in '-2147483648' [-Werror=overflow]
 RAY_CONFIG(uint64_t, object_manager_max_bytes_in_flight, 2L * 1024 * 1024 * 1024)
                                                          ~~~~~~~~~~~~~~~~~^~~~~~
bazel-out/arm-opt/bin/_virtual_includes/ray_common/ray/common/ray_config.h:35:18: note: in definition of macro 'RAY_CONFIG'
   type name##_ = default_value;               \
                  ^~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
Target //:ray_pkg failed to build
INFO: Elapsed time: 2002.802s, Critical Path: 145.31s
INFO: 1546 processes: 1546 linux-sandbox.
FAILED: Build did NOT complete successfully
Traceback (most recent call last):
  File "setup.py", line 459, in <module

Does anybody have suggestion how to fix the error?

@heavyinfo for few days I have built packges in ray requirements files:
https://github.com/PeterPirog/Raspberry_armv7_builds
but now I’m confused which aproach is correct:

1. try use “python3 setup.py bdist_wheel” command in /ray/python directory
   2) try tu use npn according to Building Ray from Source
   3) try tu use Building Ray (Python Only)
   4) try tu use script build.sh from /ray/build.sh
   5) try tu use “bazel build -c fastbuild //:ray_pkg”
   6) try somehow ./configure and make

The goal is still the same build wheel ray file for python 3.8 armv7
I will be greatefull fot the suggestions, maybe some of options above have no sense so I will stop trying its

For arm64 I believe I used pip wheel, which calls bazel behind the scenes.

For the bazel build, I had to set JAVA_HOME to /usr/lib/jvm/java-8-openjdk-arm64 -- using the /usr/bin/java complains ERROR: JAVA_HOME (/usr/bin/java) is not a path to a working JDK.