Build Google gn build tool standalone

build-gn.sh

#!/usr/bin/env sh

set -euv

# Updated on April 2019 to reflect changes in GN
# See:
# https://github.com/ninja-build/ninja
# https://gn.googlesource.com/gn/

# We need a recent git
export PATH=/depot/git-2.8.3/bin:$PATH

# Directory where all the stuff will be built
rm -rf build_tools
mkdir build_tools
cd build_tools

# We install all to $HOME/bin
mkdir -p $HOME/bin

# Build ninja
git clone https://github.com/ninja-build/ninja.git
cd ninja
./configure.py --bootstrap
cp -f ninja $HOME/bin/
cd ..

# Setup tools for GN
export PATH=/depot/binutils-2.30/bin:/depot/gcc-6.2.0/bin:$HOME/bin:$PATH
export CC=gcc
export CXX=g++
export LDFLAGS=-lrt

# Build GN
git clone https://gn.googlesource.com/gn
cd gn
python build/gen.py
ninja -C out
# out/gn_unittests
cp -f out/gn $HOME/bin

Fails with the same error as when I try to bootstrap it manually, without the script:

ninja: Entering directory `/tmp/tmpOgDNqW'
[359/359] LINK gn
FAILED: gn 
c++ -pthread -o gn -Wl,--start-group tools/gn/gn_main.o libevent.a base.a xdg_user_dirs.a gn_lib.a dynamic_annotations.a -Wl,--end-group -lrt -latomic
base/metrics/field_trial.o: In function `base::FieldTrialList::CreateTrialsFromDescriptor(int) [clone .part.224]':
field_trial.cc:(.text+0x5757): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle(base::FileDescriptor const&)'
field_trial.cc:(.text+0x5764): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle(base::SharedMemoryHandle const&)'
base/trace_event/trace_event_memory_overhead.o: In function `base::trace_event::TraceEventMemoryOverhead::AddString(std::string const&)':
trace_event_memory_overhead.cc:(.text+0x228): undefined reference to `unsigned long base::trace_event::EstimateMemoryUsage<char, std::char_traits<char>, std::allocator<char> >(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::SharedMemory()':
shared_memory_posix.cc:(.text+0x5): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle()'
shared_memory_posix.cc:(.text+0xe): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle()'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::SharedMemory(base::SharedMemoryHandle const&, bool)':
shared_memory_posix.cc:(.text+0x3c): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle(base::SharedMemoryHandle const&)'
shared_memory_posix.cc:(.text+0x45): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle()'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::CloseHandle(base::SharedMemoryHandle const&)':
shared_memory_posix.cc:(.text+0x8c): undefined reference to `base::SharedMemoryHandle::IsValid() const'
shared_memory_posix.cc:(.text+0x98): undefined reference to `base::SharedMemoryHandle::Close() const'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::DuplicateHandle(base::SharedMemoryHandle const&)':
shared_memory_posix.cc:(.text+0xf5): undefined reference to `base::SharedMemoryHandle::Duplicate() const'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::GetSizeFromSharedMemoryHandle(base::SharedMemoryHandle const&, unsigned long*)':
shared_memory_posix.cc:(.text+0x11c): undefined reference to `base::SharedMemoryHandle::GetHandle() const'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::MapAt(long, unsigned long)':
shared_memory_posix.cc:(.text+0x177): undefined reference to `base::SharedMemoryHandle::IsValid() const'
shared_memory_posix.cc:(.text+0x1ac): undefined reference to `base::SharedMemoryHandle::GetHandle() const'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::handle() const':
shared_memory_posix.cc:(.text+0x2a5): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle(base::SharedMemoryHandle const&)'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::TakeHandle()':
shared_memory_posix.cc:(.text+0x2bd): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle(base::SharedMemoryHandle const&)'
shared_memory_posix.cc:(.text+0x2ca): undefined reference to `base::SharedMemoryHandle::SetOwnershipPassesToIPC(bool)'
shared_memory_posix.cc:(.text+0x2d2): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle()'
shared_memory_posix.cc:(.text+0x2dd): undefined reference to `base::SharedMemoryHandle::operator=(base::SharedMemoryHandle const&)'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::Close()':
shared_memory_posix.cc:(.text+0x309): undefined reference to `base::SharedMemoryHandle::IsValid() const'
shared_memory_posix.cc:(.text+0x319): undefined reference to `base::SharedMemoryHandle::IsValid() const'
shared_memory_posix.cc:(.text+0x325): undefined reference to `base::SharedMemoryHandle::Close() const'
shared_memory_posix.cc:(.text+0x32d): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle()'
shared_memory_posix.cc:(.text+0x338): undefined reference to `base::SharedMemoryHandle::operator=(base::SharedMemoryHandle const&)'
shared_memory_posix.cc:(.text+0x34c): undefined reference to `base::SharedMemoryHandle::Close() const'
shared_memory_posix.cc:(.text+0x354): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle()'
shared_memory_posix.cc:(.text+0x35f): undefined reference to `base::SharedMemoryHandle::operator=(base::SharedMemoryHandle const&)'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::Create(base::SharedMemoryCreateOptions const&)':
shared_memory_posix.cc:(.text+0x5f8): undefined reference to `base::SharedMemoryHandle::IsValid() const'
shared_memory_posix.cc:(.text+0x975): undefined reference to `base::SharedMemoryHandle::ImportHandle(int)'
shared_memory_posix.cc:(.text+0x985): undefined reference to `base::SharedMemoryHandle::operator=(base::SharedMemoryHandle const&)'
shared_memory_posix.cc:(.text+0x996): undefined reference to `base::SharedMemoryHandle::ImportHandle(int)'
shared_memory_posix.cc:(.text+0x9a8): undefined reference to `base::SharedMemoryHandle::operator=(base::SharedMemoryHandle const&)'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::Open(std::string const&, bool)':
shared_memory_posix.cc:(.text+0xeb7): undefined reference to `base::SharedMemoryHandle::ImportHandle(int)'
shared_memory_posix.cc:(.text+0xec4): undefined reference to `base::SharedMemoryHandle::operator=(base::SharedMemoryHandle const&)'
shared_memory_posix.cc:(.text+0xed2): undefined reference to `base::SharedMemoryHandle::ImportHandle(int)'
shared_memory_posix.cc:(.text+0xee0): undefined reference to `base::SharedMemoryHandle::operator=(base::SharedMemoryHandle const&)'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::GetReadOnlyHandle()':
shared_memory_posix.cc:(.text+0xf74): undefined reference to `base::SharedMemoryHandle::IsValid() const'
shared_memory_posix.cc:(.text+0xf83): undefined reference to `base::SharedMemoryHandle::Duplicate() const'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::GetUniqueId(std::pair<unsigned long, unsigned long>*) const':
shared_memory_posix.cc:(.text+0xfec): undefined reference to `base::SharedMemoryHandle::SharedMemoryHandle(base::SharedMemoryHandle const&)'
shared_memory_posix.cc:(.text+0xff4): undefined reference to `base::SharedMemoryHandle::GetHandle() const'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::IsHandleValid(base::SharedMemoryHandle const&)':
shared_memory_posix.cc:(.text+0x71): undefined reference to `base::SharedMemoryHandle::IsValid() const'
base/memory/shared_memory_posix.o: In function `base::SharedMemory::GetFdFromSharedMemoryHandle(base::SharedMemoryHandle const&)':
shared_memory_posix.cc:(.text+0x101): undefined reference to `base::SharedMemoryHandle::GetHandle() const'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Command '['ninja', '-C', '/tmp/tmpOgDNqW', 'gn']' returned non-zero exit status 1

@vi
You need to modify the gn-standalone/tools/gn/bootstrap/bootstrap.py a little bit by adding the ommitted cpp files in around the line 400 of bootstrap.py

Hi,

I have also created mirror of the code for your convenience. There is a patch folder that contains fixes to the build.
Link follows at https://github.com/DanCraft99/gn-standalone-mirror.

Regards,
Dan

my latest working version: https://github.com/nodenative/nodenative/blob/master/build/gn-standalone.sh

To build the latest changes always is tricky and a often is broken. e.g. https://bugs.chromium.org/p/chromium/issues/detail?id=725439#c16

FWIW I converted this to Dockerfile syntax in a build stage and it worked well for me:

RUN mkdir -p /opt/local/gn-standalone
WORKDIR /opt/local/gn-standalone

RUN git clone --depth 1 https://chromium.googlesource.com/chromium/src/base
RUN git clone --depth 1 https://chromium.googlesource.com/chromium/src/build
RUN git clone --depth 1 https://chromium.googlesource.com/chromium/src/build/config

RUN mkdir -p tools
RUN git clone --depth 1 https://chromium.googlesource.com/chromium/src/tools/gn tools/gn

RUN mkdir -p testing
RUN git clone --depth 1 https://chromium.googlesource.com/chromium/testing/gtest testing/gtest

RUN mkdir -p third_party/libevent
ADD https://chromium.googlesource.com/chromium/chromium/+archive/master/third_party/libevent.tar.gz third_party/libevent

WORKDIR /opt/local/gn-standalone/tools/gn
RUN ./bootstrap/bootstrap.py -s

And if you don't like ADD and you already have curl then you could use this snippet instead:

RUN mkdir -p third_party/libevent
WORKDIR /opt/local/gn-standalone/third_party/libevent
RUN curl -L https://chromium.googlesource.com/chromium/chromium/+archive/master/third_party/libevent.tar.gz | tar xz

damn, you could just easly download the whole chromium tar ball here , no need to mess with complicated setup like this. Just be sure to pick the lastest update . In linux with custom clang svn build at chromium_src/tools/gn/bootstrap.py remove -latomic ldflags and run it (bootstrap.py) with python2. Hopefully will be useful small tips.

Fails with:

# Build
cd tools/gn
./bootstrap/bootstrap.py -s
Building gn manually in a temporary directory for bootstrapping...
ninja: Entering directory `/tmp/tmpsvuDoi'
ninja: error: '/srv/glaubitz/tmp/gn-standalone/buildtools/third_party/libc++/trunk/src/algorithm.cpp', needed by '/srv/glaubitz/tmp/gn-standalone/buildtools/third_party/libc++/trunk/src/algorithm.o', missing and no known rule to make it
Command '['ninja', '-C', '/tmp/tmpsvuDoi', '-w', 'dupbuild=err', 'gn']' returned non-zero exit status 1

Debian unstable/sparc64.

Can confirm @glaubitz issue. Solved it by adding two new dependencies (libc++, libc++abi). Please see this PKGBUILD for a successful build recipe.

I got this error:

# Build
cd tools/gn
./bootstrap/bootstrap.py -s
Building gn manually in a temporary directory for bootstrapping...
ninja: Entering directory `/var/folders/mx/3gnl5sks49z37khmfv2l3qyr0000gn/T/tmpTMWf7W'
[11/373] CXX base/allocator/allocator_check.o
FAILED: base/allocator/allocator_check.o 
c++ -MMD -MF base/allocator/allocator_check.o.d  -I/var/folders/mx/3gnl5sks49z37khmfv2l3qyr0000gn/T/tmpTMWf7W/gen -I/Users/paulo/Developer/workspaces/cpp/mobile-pdfium/gn-standalone -DNO_TCMALLOC -D__STDC_FORMAT_MACROS -O2 -g0 -D_FILE_OFFSET_BITS=64 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -pthread -pipe -fno-exceptions -std=c++14 -Wno-c++11-narrowing -c /Users/paulo/Developer/workspaces/cpp/mobile-pdfium/gn-standalone/base/allocator/allocator_check.cc -o base/allocator/allocator_check.o
In file included from /Users/paulo/Developer/workspaces/cpp/mobile-pdfium/gn-standalone/base/allocator/allocator_check.cc:19:
/Users/paulo/Developer/workspaces/cpp/mobile-pdfium/gn-standalone/base/allocator/allocator_interception_mac.h:11:10: fatal error: 'third_party/apple_apsl/malloc.h' file not found
#include "third_party/apple_apsl/malloc.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[16/373] CXX base/allocator/allocator_extension.o
ninja: build stopped: subcommand failed.
Command '['ninja', '-C', '/var/folders/mx/3gnl5sks49z37khmfv2l3qyr0000gn/T/tmpTMWf7W', '-w', 'dupbuild=err', 'gn']' returned non-zero exit status 1

Mac OSX high sierra

gn now lives at https://gn.googlesource.com/

Just download it from: https://gn.googlesource.com/gn/

ninja: Entering directory `out'
[1/37] LINK gn
FAILED: gn
clang++ -L/usr/local/opt/libffi/lib -O3 -fdata-sections -ffunction-sections -Wl,-dead_strip -mmacosx-version-min=10.9 -pthread -o gn src/gn/gn_main.o base.a gn_lib.a
ld: warning: ignoring file base.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
ld: warning: ignoring file gn_lib.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
Undefined symbols for architecture x86_64:
"base::CommandLine::ForCurrentProcess()", referenced from:
_main in gn_main.o
"base::CommandLine::Init(int, char const* const*)", referenced from:
_main in gn_main.o
"base::CommandLine::GetArgs() const", referenced from:
_main in gn_main.o
"commands::kHelp", referenced from:
_main in gn_main.o
"MsgLoop::MsgLoop()", referenced from:
_main in gn_main.o
"commands::GetCommands()", referenced from:
_main in gn_main.o
"base::CommandLine::HasSwitch(char const*) const", referenced from:
_main in gn_main.o
"Location::Location()", referenced from:
_main in gn_main.o
"PrintShortHelp(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&)", referenced from:
_main in gn_main.o
"commands::CommandSwitches::Init(base::CommandLine const&)", referenced from:
_main in gn_main.o
"OutputString(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, TextDecoration, HtmlEscaping)", referenced from:
_main in gn_main.o
"Err::PrintToStdout() const", referenced from:
_main in gn_main.o
"switches::kVersion", referenced from:
_main in gn_main.o
"Err::Err(Location const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&)", referenced from:
_main in gn_main.o
"MsgLoop::~MsgLoop()", referenced from:
_main in gn_main.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[14/37] CXX src/gn/runtime_deps_unittest.o