Building IREE's compiler through Emscripten

iree_emscripten.md

Goal: built iree-opt through emscripten to run it interactively from documentation websites.

See also: https://gist.github.com/ScottTodd/f30d9d26254de82648de37d5ed445cbc (mlir-opt through emscripten)

References:

• Install emsdk: https://emscripten.org/docs/getting_started/downloads.html
• Read https://emscripten.org/docs/compiling/Building-Projects.html
• LLVM cross compilation: https://llvm.org/docs/HowToCrossCompileLLVM.html
• LLVM CMake options: https://llvm.org/docs/CMake.html

Notes:

• All commands are run from iree root directory (~/code/iree/ on my Linux machine)
• Tested with IREE at https://github.com/google/iree/commit/ccae793f8c7c208a0b1b0baf489cfd121e5f7b4c
• Tested with emsdk at https://github.com/emscripten-core/emsdk/commit/2e7eaf7233144e5e25b1c1338890bbab5d011815
• Tested with cmake version 3.18.4 with clang 11 on Linux (Debian) after first trying on Windows with MSVC

Set up emsdk following those docs (run source ./emsdk_env.sh)

Build host tools:

cmake -G Ninja -B ../iree-build/ -DIREE_BUILD_TESTS=OFF -Wno-dev -DCMAKE_BUILD_TYPE=Release .
cmake --build ../iree-build/
cmake --build ../iree-build/ --target install

Configure for emscripten:

Note: we want to use the host tablegen and other tools throughout the build. Emscripten or LLVM creates a NATIVE/ folder and tries to run tools from there... but the "executable binaries" in that directory are JavaScript files... which obviously can't be executed in a shell environment.

emcmake cmake -G Ninja -B ../iree-build-emscripten/ -DCMAKE_BUILD_TYPE=Release -DLLVM_TABLEGEN=$PWD/../iree-build/third_party/llvm-project/llvm/bin/llvm-tblgen -DMLIR_TABLEGEN_EXE=$PWD/../iree-build/third_party/llvm-project/llvm/bin/mlir-tblgen -DLLVM_TOOLS_BINARY_DIR=$PWD/../iree-build/third_party/llvm-project/llvm/bin/ -DIREE_HOST_BINARY_ROOT=$PWD/../iree-build/install/ -DIREE_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=../iree-build/emscripten/install/ -Wno-dev .

** weird stuff happens here **

try building:

cmake --build ../iree-build-emscripten/ --target iree-opt.js

see errors about llvm-tblgen, mlir-tblgen, etc.

copy from host build to NATIVE/ folder:

cp ../iree-build/third_party/llvm-project/llvm/bin/llvm-tblgen ../iree-build-emscripten/third_party/llvm-project/llvm/NATIVE/bin && chmod +x ../iree-build-emscripten/third_party/llvm-project/llvm/NATIVE/bin/llvm-tblgen && cp ../iree-build/third_party/llvm-project/llvm/bin/mlir-tblgen ../iree-build-emscripten/third_party/llvm-project/llvm/NATIVE/bin && chmod +x ../iree-build-emscripten/third_party/llvm-project/llvm/NATIVE/bin/mlir-tblgen && cp ../iree-build/third_party/llvm-project/llvm/bin/mlir-linalg-ods-gen ../iree-build-emscripten/third_party/llvm-project/llvm/NATIVE/bin && chmod +x ../iree-build-emscripten/third_party/llvm-project/llvm/NATIVE/bin/mlir-linalg-ods-gen && cp ../iree-build/third_party/llvm-project/llvm/bin/mlir-linalg-ods-yaml-gen ../iree-build-emscripten/third_party/llvm-project/llvm/NATIVE/bin && chmod +x ../iree-build-emscripten/third_party/llvm-project/llvm/NATIVE/bin/mlir-linalg-ods-yaml-gen

try building again:

cmake --build ../iree-build-emscripten/ --target iree-opt.js

maybe irrelevant, see errors about node and patch cmakecache:

edit iree-build-emscripten/CMakeCache.txt: before:

CMAKE_CROSSCOMPILING_EMULATOR:UNINITIALIZED="/usr/local/google/home/scotttodd/code/emsdk/node/14.15.5_64bit/bin/node"

after:

CMAKE_CROSSCOMPILING_EMULATOR:UNINITIALIZED=/usr/local/google/home/scotttodd/code/emsdk/node/14.15.5_64bit/bin/node

notice that tblgen paths are not getting set correctly for HLO and IREE, hardcode TableGen.cmake:

  if (${project} STREQUAL "LLVM")
    set(${project}_TABLEGEN_EXE "/usr/local/google/home/scotttodd/code/iree/../iree-build/third_party/llvm-project/llvm/bin/llvm-tblgen")
  elseif(${project} STREQUAL "MLIR")
    set(${project}_TABLEGEN_EXE "/usr/local/google/home/scotttodd/code/iree/../iree-build/third_party/llvm-project/llvm/bin/mlir-tblgen")
  elseif(${project} STREQUAL "IREE")
    set(${project}_TABLEGEN_EXE "/usr/local/google/home/scotttodd/code/iree/../iree-build/install/bin/iree-tblgen")
  endif()

(see https://github.com/google/iree/blob/68c27a08f42ba64d566f7a6e4816550f6e0f57ae/build_tools/cmake/iree_copts.cmake#L449-L452)

wasm-ld: error: initial memory too small, 19770448 bytes needed

add

SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -s INITIAL_MEMORY=134217728")

at the bottom of iree_copts.cmake (number needs tuning)

success ?

ls -la ../iree-build-emscripten/iree/tools/

(should have iree-opt.js and iree-opt.wasm, among other files)

Tried this on Windows, ran into a few issues.

• https://zenn.dev/okuoku/scraps/3ad03e01d81375 has clues

Configuring with:
λ emcmake cmake -G Ninja -B ../iree-build-web/ -DCMAKE_BUILD_TYPE=Release -DLLVM_HOST_TRIPLE=wasm32-unknown-emscripten -DLLVM_TARGETS_TO_BUILD=WebAssembly -DIREE_TARGET_BACKEND_DEFAULTS=OFF -DIREE_TARGET_BACKEND_LLVM_CPU=ON -DIREE_TARGET_BACKEND_LLVM_CPU_WASM=ON -DCMAKE_EXE_LINKER_FLAGS="-g0 -sTOTAL_MEMORY=39321600" -DLLVM_TABLEGEN=D:\dev\projects\iree-build\llvm-project\bin\llvm-tblgen.exe -DMLIR_TABLEGEN_EXE=D:\dev\projects\iree-build\llvm-project\bin\mlir-tblgen.exe -DLLVM_TOOLS_BINARY_DIR=D:\dev\projects\iree-build\llvm-project\bin\ -DIREE_HOST_BIN_DIR=D:\dev\projects\iree-build\install\bin -DIREE_BUILD_TESTS=OFF -Wno-dev .

Building with:
cmake --build ../iree-build-web/ --target iree-compile.js

Latest errors are

[2/3339] Linking CXX executable third_party\iree\llvm-project\bin\lld.js
FAILED: third_party/iree/llvm-project/bin/lld.js
wasm-ld: error: third_party/iree/llvm-project/lib/libLLVMSupport.a(Program.cpp.o): undefined symbol: wait4
wasm-ld: error: third_party/iree/llvm-project/lib/libLLVMSupport.a(Program.cpp.o): undefined symbol: wait4

• https://www.reddit.com/r/LLVM/comments/12za62c/wasmld_error_liblibllvmsupportaprogramcppo/

Maybe lld is less compatible with Emscripten than LLVM/MLIR... could try just VMVX...