This gist shows how to get a fully working LLVM build that runs in a separate sysroot, completely free of any GNU tools. I achieved this by substituting glibc with musl, libstdc++ with libc++ and gcc-libs with compiler-rt. I can't say anything about the stability of this system (as far as I know libc++ is far from finished), but it was fun to get here.
Let's get started.
For this, I'll assume you have a fully working LLVM compiler (clang) as well as LLVM linker (lld) on your system.
I myself am on Artix Linux currently, and I'm running clang-19.1.7-2 and llvm-19.1.7-2.
First, let's grab all the resources we need:
$ mkdir -pv /dev/shm/build
$ ln -sfv /dev/shm/build build
$ pushd build
$ wget -qO- "https://musl.libc.org/releases/musl-1.2.5.tar.gz" | tar xz
$ wget -qO- "https://www.kernel.org/pub/linux/kernel/v6.x/linux-6.14.9.tar.xz" | tar xJ
$ wget -qO- "https://github.com/llvm/llvm-project/archive/refs/tags/llvmorg-20.1.6.tar.gz" | tar xz
$ mv -v llvm-project-llvmorg-20.1.6 llvm-20.1.6
$ popdAs you can see here, I'll be doing this in a ramdisk.
If your system doesn't have enough memory or /dev/shm for some reason isn't working, you can just use a normal mkdir as well.
Next, let's prepare the new sysroot:
$ mkdir -pv sysroot
$ SYSROOT="$PWD/sysroot"
$ pushd sysroot
$ mkdir -pv {dev,proc,run,sys,tmp}
$ mkdir -pv usr/{include,lib,libexec,bin}
$ ln -sfv usr/bin bin
$ ln -sfv usr/bin sbin
$ ln -sfv usr/lib lib
$ ln -sfv usr/lib libexec
$ ln -sfv usr/lib lib64
$ ln -sfv bin usr/sbin
$ ln -sfv lib usr/lib64
$ popdIt's important that the $SYSROOT variable is set for any step of this procedure.
Note that there's no reason to create symlinks for most things like I did, I just prefer my system setup that way.
An empty sysroot isn't particularly useful and clang would fail building anything almost immediately, so let's populate it.
We're going to build musl now, but only install the headers, as we're still building with our host sysroot:
$ mkdir -pv build/musl-1.2.5/build
$ pushd build/musl-1.2.5/build
$ ../configure \
--prefix=/usr \
--target=x86_64-dog-linux-musl \
CC=clang AR=llvm-ar RANLIB=llvm-ranlib
$ make -j32
$ DESTDIR="$SYSROOT" make install-headers
$ popd
$ rm -rfv build/musl-1.2.5/buildLet's take a look at the configure flags.
Setting --prefix is required for the headers to be installed to the right folders later.
It's important to specify --target with the target of our new sysroot,
as some headers may be different depending on the architecture.
Finally, due to --target being specified, musl will try to find cross-compiler tools, which don't exist,
so we override them with CC, AR and RANLIB.
Make sure to replace -j32 with something lower if your system doesn't handle 32-parallel build processes well.
Finally we use install-headers combined with DESTDIR to install the headers to our sysroot.
Next, let's install the linux headers.
$ pushd build/linux-6.14.9
$ make headers -j32
$ find usr/include -type f ! -name '*.h' -delete
$ cp -rv usr/include "$SYSROOT/usr"
$ popd
$ rm -rfv build/linux-6.14.9There is a make target called headers_install,
but some linux headers are already present so we need to manually install all the headers with find and cp.
We can actually delete the linux-6.14.9 directory entirely here, we don't need it anymore.
Now we can build compiler-rt, which replaces gcc-libs and is essential to compile anything else:
$ pushd build/llvm-20.1.6
$ cmake -S compiler-rt -B build-compiler-rt -G "Ninja" \
-DCMAKE_INSTALL_PREFIX="$SYSROOT/usr" \
-DCOMPILER_RT_BUILD_BUILTINS=ON \
-DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
-DCOMPILER_RT_BUILD_MEMPROF=OFF \
-DCOMPILER_RT_BUILD_PROFILE=OFF \
-DCOMPILER_RT_BUILD_SANITIZERS=OFF \
-DCOMPILER_RT_BUILD_XRAY=OFF \
-DCOMPILER_RT_BUILD_CTX_PROFILE=OFF \
-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON \
-DCMAKE_ASM_COMPILER_TARGET="x86_64-dog-linux-musl" \
-DCMAKE_C_COMPILER_TARGET="x86_64-dog-linux-musl" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSROOT="$SYSROOT"
$ cmake --build build-compiler-rt
$ cmake --install build-compiler-rt
$ rm -rfv build-compiler-rt
$ popdLet's take a look at these cmake flags.
First of all, we force the compiler to be clang and clang++ using CMAKE_C(XX)_COMPILER.
Again, we set the compiler target using CMAKE_(ASM,C)_COMPILER_TARGET. We can't set the CXX target yet, because we don't have a c++ standard library in our sysroot, compilation will fail otherwise.
We let cmake know that we want to install to our sysroot via CMAKE_INSTALL_PREFIX
and also that we want a Release build using CMAKE_BUILD_TYPE. Now comes the interesting part.
We override the sysroot of this build using CMAKE_SYSROOT,
letting cmake know that we are still in a linux environment using CMAKE_SYSTEM_NAME.
This overrides where the compilers, linkers, etc. look for libraries and headers to our sysroot.
Finally, we only want a tiny usable compiler-rt and don't rely on any fuzzing or special targets,
so we disable all of this using COMPILER_RT_BUILD_XXX and COMPILER_RT_DEFAULT_TARGET_ONLY.
Now that the low-level stuff is out of the way, we can compile musl properly:
$ mkdir -pv build/musl-1.2.5/build
$ pushd build/musl-1.2.5/build
$ ../configure \
--prefix=/usr \
--target=x86_64-dog-linux-musl \
CC=clang \
AR=llvm-ar RANLIB=llvm-ranlib \
LIBCC="$SYSROOT/lib/linux/libclang_rt.builtins-x86_64.a" \
CFLAGS="--sysroot=$SYSROOT -fuse-ld=lld"
$ make -j32
$ DESTDIR="$SYSROOT" make install
$ popd
$ rm -rfv build/musl-1.2.5In addition to the previous flags, we override LIBCC with compiler-rt,
append our sysroot to CFLAGS via --sysroot and force musl to link with lld using -fuse-ld.
After this is fully installed, we can also delete the entire musl folder.
Next up, libcxx and it's dependencies libcxxabi and libunwind:
$ pushd build/llvm-20.1.6
$ cmake -S runtimes -B build-runtimes -G "Ninja" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="$SYSROOT/usr" \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_SYSTEM_NAME=Linux \
-DLLVM_ENABLE_RUNTIMES="libunwind;libcxxabi;libcxx" \
-DLIBUNWIND_USE_COMPILER_RT=true \
-DLIBCXX_USE_COMPILER_RT=true \
-DLIBCXX_HAS_MUSL_LIBC=On \
-DLIBCXXABI_USE_COMPILER_RT=true \
-DCMAKE_SYSROOT="$SYSROOT" \
-DCMAKE_C_COMPILER_TARGET=x86_64-dog-linux-musl \
-DCMAKE_CXX_COMPILER_TARGET=x86_64-dog-linux-musl \
-DLLVM_TARGETS_TO_BUILD=X86 \
-DCMAKE_C_FLAGS="-rtlib=compiler-rt -Qunused-arguments -Wl,--dynamic-linker=/lib/ld-musl-x86_64.so.1 -fuse-ld=lld" \
-DCMAKE_CXX_FLAGS="-rtlib=compiler-rt -Qunused-arguments -Wl,--dynamic-linker=/lib/ld-musl-x86_64.so.1 -fuse-ld=lld" \
-DLLVM_ENABLE_ZSTD=false \
-DLLVM_ENABLE_ZLIB=false \
-DLIBCXX_HAS_ATOMIC_LIB=false \
-DLIBCXXABI_HAS_CXA_THREAD_ATEXIT_IMPL=false \
-DCMAKE_C_COMPILER_WORKS=true \
-DCMAKE_CXX_COMPILER_WORKS=true
$ cmake --build build-runtimes
$ cmake --install build-runtimes
$ rm -rfv build-runtimes
$ popdThis is where the flags become a bit wild. First of all, we enable the runtimes we want to build using LLVM_ENABLE_RUNTIMES.
Then we require all parts of the build to use compiler_rt using XXX_USE_COMPILER_RT and musl using LIBCXX_HAS_MUSL_LIBC.
We need to disable libatomic, zstd and zlib manually, as the buildsystem doesn't really respect the sysroot override by itself,
so we add LLVM_ENABLE_(ZSTD,ZLIB) and LIBCXX_HAS_ATOMIC_LIB.
Then we also need to fix a small linking issue,
no idea what causes it but setting LIBCXXABI_HAS_CXA_THREAD_ATEXIT_IMPL to false fixes it.
Onto the CFLAGS, we specify -rtlib=compiler-rt for compiling with compiler-rt,
as well as the linker flag -Wl,--dynamic-linker to override the glibc linker/loader with the musl one.
We also add -fuse-ld=lld to override the linker.
Now onto the reaaallly weird part.
We specify CMAKE_(C,CXX)_COMPILER_WORKS to skip the test, where it checks if the c compiler does the c compiling, because
the test is ran without the sysroot, but with the musl override, so it fails because I don't have musl on my main system.
We also specify -Qunused-arguments, because when compiling (not linking), the -rtlib=compiler-rt flag is unused,
which doesn't just lead to console spam,
but also leads to multiple vital configure tests failing and turning off parts of libcxx silently.
Still with me? We can now finally compile clang and lld fully:
$ pushd build/llvm-20.1.6
$ cmake -S llvm -B build-llvm -G "Ninja" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="$SYSROOT/usr" \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSROOT="$SYSROOT" \
-DLLVM_ENABLE_PROJECTS="clang;lld" \
-DCMAKE_C_COMPILER_TARGET=x86_64-dog-linux-musl \
-DCMAKE_CXX_COMPILER_TARGET=x86_64-dog-linux-musl \
-DLLVM_TARGETS_TO_BUILD=X86 \
-DCMAKE_C_FLAGS="-rtlib=compiler-rt -Qunused-arguments -Wl,--dynamic-linker=/lib/ld-musl-x86_64.so.1 -fuse-ld=lld" \
-DCMAKE_CXX_FLAGS="-stdlib=libc++ -rtlib=compiler-rt -Qunused-arguments -Wl,--dynamic-linker=/lib/ld-musl-x86_64.so.1 -fuse-ld=lld" \
-DLLVM_ENABLE_LIBCXX=true \
-DLLVM_ENABLE_ZSTD=false \
-DLLVM_ENABLE_ZLIB=false \
-DLLVM_USE_LINKER=lld \
-DCXX_SUPPORTS_CUSTOM_LINKER=true \
-DLLVM_ENABLE_LIBXML2=0
$ cmake --build build-llvm
$ cmake --install build-llvm
$ popd
$ rm -rfv build/llvm-20.1.6The flags here are largely the same, except we also add -stdlib=libc++ to the c++ flags,
because clang defaults to libstdc++. We also have to set LLVM_USE_LINKER AND CXX_SUPPORTS_CUSTOM_LINKER,
because of course the build system would screw things up if we didn't. And we disable libxml2 using LLVM_ENABLE_LIBXML2,
which isn't part of our sysroot so it shouldn't try to build the linker with it.
We aren't 100% finished yet, but trust me this one makes sense. Let's cleanup our build directory first though:
rm -rfv /dev/shm/build buildMuch better, usually when compiling clang and lld,
you specify LLVM_ENABLE_RUNTIMES again and it will build compiler-rt as well as libcxx again, but I skipped that.
We already compiled both, so no need to recompile it. We just have to nudge compiler-rt into the right place:
$ mkdir -pv sysroot/lib/clang/20/lib
$ mv sysroot/lib/linux sysroot/lib/clang/20/lib/x86_64-unknown-linux-gnu
$ pushd sysroot/lib/clang/20/lib/x86_64-unknown-linux-gnu
$ mv clang_rt.crtbegin-x86_64.o crtbeginS.o
$ mv clang_rt.crtend-x86_64.o crtendS.o
$ mv libclang_rt.builtins-x86_64.a libclang_rt.builtins.a
$ popdNow we're done, let's test if it works!
Grab a shell, I'll just use bash from here (be careful when downloading blobs!). Make sure your shell is statically compiled, or compile it yourself with the sysroot flags you learned here.
Earlier I created the run directory in my sysroot. This is just so I can use arch-chroot sysroot without failing.
Chroot into your sysroot and test your toolchain:
$ echo -e "#include <iostream>\n\nint main() {\n std::cout << \"Hello, world"'!'"\" << '\\\\n';\n return 0;\n}\n" > main.cpp
$ clang++ -stdlib=libc++ -rtlib=compiler-rt -fuse-ld=lld -Wl,-dynamic-linker=/lib/ld-musl-x86_64.so.1 main.cpp
$ ./a.outThis should print Hello, world! into your console.
As you can see in the clang++ call, we have to specify -stdlib, -rtlib, -fuse-ld and -Wl,-dynamic-linker every time.
I suggest you write a neat little wrapper (and make sure to update the musl.clang wrapper already installed in /bin!),
to make this process less painful.
Finally, I just wanna say: I'm not a compiler dev, nor someone who writes gist a lot. This was a fun little side project, a learning opportunity and I didn't wanna keep my findings to myself.
Thanks for the guide! I've been tweaking this over the past few days in an attempt to make it work with LLVM 21 for Maple Linux and I've been running into all kinds of weird issues. Currently, it appears to be using headers from my Devuan system, which is preventing it from building the demangle library at this point, but I'll keep digging around to see what I can find. Feel free to take stuff from my bootstrap scripts if you'd like, but I honestly don't understand what I'm doing when it comes to bootstrapping a system, so take things with a grain of salt.