WIP: Install GHC 8.0.2 starting from PKGSRC ghc 7.6.3 on SmartOS

ghc_7.10.3_settings.in.patch

--- settings.in 2017-07-23 14:25:55.733820769 +0000
+++ settings.in.patched 2017-07-23 14:26:39.156012024 +0000
@@ -1,33 +1,33 @@
 [("GCC extra via C opts", "@GccExtraViaCOpts@"),
  ("C compiler command", "@SettingsCCompilerCommand@"),
  ("C compiler flags", "@SettingsCCompilerFlags@"),
- ("C compiler link flags", "@SettingsCCompilerLinkFlags@"),
+ ("C compiler link flags", "@SettingsCCompilerLinkFlags@ -lssp"),
  ("Haskell CPP command","@SettingsHaskellCPPCommand@"),
  ("Haskell CPP flags","@SettingsHaskellCPPFlags@"),
  ("ld command", "@SettingsLdCommand@"),
- ("ld flags", "@SettingsLdFlags@"),
+ ("ld flags", "@SettingsLdFlags@ -lssp"),
  ("ld supports compact unwind", "@LdHasNoCompactUnwind@"),
  ("ld supports build-id", "@LdHasBuildId@"),
  ("ld supports filelist", "@LdHasFilelist@"),
  ("ld is GNU ld", "@LdIsGNULd@"),
- ("ar command", "@SettingsArCommand@"),
+ ("ar command", "ar"),
  ("ar flags", "@ArArgs@"),
  ("ar supports at file", "@ArSupportsAtFile@"),
  ("touch command", "@SettingsTouchCommand@"),
  ("dllwrap command", "@SettingsDllWrapCommand@"),
  ("windres command", "@SettingsWindresCommand@"),
  ("libtool command", "@SettingsLibtoolCommand@"),
- ("readelf command", "@SettingsReadElfCommand@"),
+ ("readelf command", "readelf"),
  ("perl command", "@SettingsPerlCommand@"),
  ("cross compiling", "@CrossCompiling@"),
  ("target os", "@HaskellTargetOs@"),
  ("target arch", "@HaskellTargetArch@"),
  ("target word size", "@WordSize@"),
  ("target has GNU nonexec stack", "@HaskellHaveGnuNonexecStack@"),
  ("target has .ident directive", "@HaskellHaveIdentDirective@"),
  ("target has subsections via symbols", "@HaskellHaveSubsectionsViaSymbols@"),
  ("Unregisterised", "@Unregisterised@"),
  ("LLVM llc command", "@SettingsLlcCommand@"),
  ("LLVM opt command", "@SettingsOptCommand@")
  ]

ghc_7.6.3_settings.patch

--- /opt/local/lib/ghc-7.6.3/settings   2017-07-06 13:03:27.535396964 +0000
+++ /opt/local/lib/ghc-7.6.3/settings.fixed     2017-07-06 13:03:16.515974064 +0000
@@ -1,20 +1,20 @@
 [("GCC extra via C opts", " -fwrapv"),
  ("C compiler command", "gcc"),
- ("C compiler flags", " -pipe -O2 -pipe -O2  -I/opt/local/include -I/usr/include -I/opt/local/include/ncurses -fno-stack-protector   -L/opt/local/gcc49/lib/gcc/i486-sun-solaris2.11/4.9.4 -Wl,-R/opt/local/gcc49/lib/gcc/i486-sun-solaris2.11/4.9.4  -L/opt/local/lib -L/usr/lib -Wl,-R/usr/lib -Wl,-R/opt/local/lib"),
+ ("C compiler flags", " -pipe -O2 -pipe -O2  -I/opt/local/include -I/usr/include -I/opt/local/include/ncurses -fno-stack-protector   -L/opt/local/gcc49/lib/gcc/i486-sun-solaris2.11/4.9.4 -Wl,-R/opt/local/gcc49/lib/gcc/i486-sun-solaris2.11/4.9.4  -L/opt/local/lib -L/usr/lib -Wl,-R/usr/lib -Wl,-R/opt/local/lib -lssp"),
  ("ar command", "ar"),
  ("ar flags", "q"),
  ("ar supports at file", "YES"),
  ("touch command", "touch"),
  ("dllwrap command", "/bin/false"),
  ("windres command", "/bin/false"),
  ("perl command", "/opt/local/bin/perl"),
  ("target os", "OSSolaris2"),
  ("target arch", "ArchX86"),
  ("target word size", "4"),
  ("target has GNU nonexec stack", "True"),
  ("target has .ident directive", "True"),
  ("target has subsections via symbols", "False"),
  ("LLVM llc command", "llc"),
  ("LLVM opt command", "opt")
  ]

ghc_8.0.2_settings.in.patch

--- settings.in 2017-07-23 14:22:44.329190132 +0000
+++ settings.in.patched 2017-07-23 14:23:12.682450201 +0000
@@ -1,33 +1,33 @@
 [("GCC extra via C opts", "@GccExtraViaCOpts@"),
  ("C compiler command", "@SettingsCCompilerCommand@"),
  ("C compiler flags", "@SettingsCCompilerFlags@"),
- ("C compiler link flags", "@SettingsCCompilerLinkFlags@"),
+ ("C compiler link flags", "@SettingsCCompilerLinkFlags@ -lssp"),
  ("C compiler supports -no-pie", "@SettingsCCompilerSupportsNoPie@"),
  ("Haskell CPP command","@SettingsHaskellCPPCommand@"),
  ("Haskell CPP flags","@SettingsHaskellCPPFlags@"),
  ("ld command", "@SettingsLdCommand@"),
- ("ld flags", "@SettingsLdFlags@"),
+ ("ld flags", "@SettingsLdFlags@ -lssp"),
  ("ld supports compact unwind", "@LdHasNoCompactUnwind@"),
  ("ld supports build-id", "@LdHasBuildId@"),
  ("ld supports filelist", "@LdHasFilelist@"),
  ("ld is GNU ld", "@LdIsGNULd@"),
- ("ar command", "@SettingsArCommand@"),
+ ("ar command", "ar"),
  ("ar flags", "@ArArgs@"),
  ("ar supports at file", "@ArSupportsAtFile@"),
  ("touch command", "@SettingsTouchCommand@"),
  ("dllwrap command", "@SettingsDllWrapCommand@"),
  ("windres command", "@SettingsWindresCommand@"),
  ("libtool command", "@SettingsLibtoolCommand@"),
  ("perl command", "@SettingsPerlCommand@"),
  ("cross compiling", "@CrossCompiling@"),
  ("target os", "@HaskellTargetOs@"),
  ("target arch", "@HaskellTargetArch@"),
  ("target word size", "@WordSize@"),
  ("target has GNU nonexec stack", "@HaskellHaveGnuNonexecStack@"),
  ("target has .ident directive", "@HaskellHaveIdentDirective@"),
  ("target has subsections via symbols", "@HaskellHaveSubsectionsViaSymbols@"),
  ("Unregisterised", "@Unregisterised@"),
  ("LLVM llc command", "@SettingsLlcCommand@"),
  ("LLVM opt command", "@SettingsOptCommand@")
  ]

install-ghc-builder.sh

#!/usr/bin/env bash
set -o xtrace
# use at least base/base64 14.1.0 to get the EROFS fix in GHC's included unix package
builder_user=get_from_gabor
builder_pass=get_from_gabor
builder_master=haskell.inf.elte.hu
pkgin -y update
pkgin -y full-upgrade
pkgin -y install ghc build-essential
pkgin -y install ghostscript # optional for h2ps tests
# BEGIN: HACK to work around boot GHC having been built without -lssp
curl -OL https://gist.github.com/AlainODea/ccb4f0ad7385a691b6fe/raw/2dc2142db60ea3597d2bc1d09c5b406d66cc34e3/ghc_settings.patch
patch /opt/local/lib/ghc-7.6.3/settings ghc_settings.patch
# END: HACK to work around boot GHC having been built with a custom ar
groupadd ghcbld
useradd -m -s /bin/bash -g ghcbld ghcbld
cat > /opt/local/lib/ghc-7.6.3/overridecpp.spec <<'EOF'
*cpp:
%{cpp_subtarget}
EOF
cat > /opt/local/bin/ghc-cpp <<'EOF'
#!/bin/bash
gcc -specs=/opt/local/lib/ghc-7.6.3/overridecpp.spec $@
EOF
chmod +x /opt/local/bin/ghc-cpp
su - ghcbld -c "
  # Bootstrap cabal-install (missing from PKGSRC ghc)
  cabal_install_version=1.20.0.1
  cabal_install_directory=cabal-install-${cabal_install_version}
  cabal_install_archive=${cabal_install_directory}.tar.gz

  curl -OL https://www.haskell.org/cabal/release/${cabal_install_directory}/${cabal_install_archive}
  tar xf ${cabal_install_archive}
  cd ${cabal_install_directory}
  chmod +x bootstrap.sh
  ./bootstrap.sh
  # cabal-install bootstrap complete
  cat >> .profile <<'EOF'
PATH=\$HOME/.cabal/bin:\$PATH
EOF
  ln -s /bin/bash ~/.cabal/bin/sh
  export PATH=\$HOME/.cabal/bin:\$PATH
  cabal update
  cabal install -j happy alex HsOpenSSL
  curl -OL https://downloads.haskell.org/~ghc/7.10.3/ghc-7.10.3b-src.tar.bz2
  tar xf ghc-7.10.3b-src.tar.bz2
  pushd ghc-7.10.3/utils/lndir
  gcc lndir.c -o ~/.cabal/bin/lndir
  popd
  pushd ghc-7.10.3
  cat > mk/build.mk <<'EOF'
V=1
InstallExtraPackages=YES
BeConservative=YES
EOF
  ./boot && ./configure --with-hs-cpp="ghc-cpp" --prefix /opt/local && gmake -j16
  popd
"
pushd ~ghcbld/ghc-7.10.3/
gmake install
popd
su - ghcbld -c "
  curl -OL https://downloads.haskell.org/~ghc/8.0.2/ghc-8.0.2-src.tar.xz
  tar xf ghc-8.0.2-src.tar.xz
  pushd ghc-8.0.2
  cat > mk/build.mk <<'EOF'
V=1
InstallExtraPackages=YES
BeConservative=YES
EOF
  ./boot && ./configure --with-hs-cpp="ghc-cpp" --prefix /opt/local && gmake -j16
  popd
"
pushd ~ghcbld/ghc-8.0.2/
gmake install
popd
su - ghcbld -c "
  git clone https://github.com/haskell/ghc-builder.git
  pushd ~/ghc-builder/
  cabal install common/ client/
  popd
  mkdir ~/builder
  pushd ~/builder/
  builder-client init \"$builder_user\" \"$builder_pass\" $builder_master
  mkdir certs
  curl http://haskell.inf.elte.hu/ghcBuilder/cert/root.pem > certs/root.pem
  popd
"
curl -L https://gist.github.com/AlainODea/10431222/raw/68e66c5f6516086f2630164c0db5fcd40b81f48f/ghc-builder-client.xml > /opt/local/lib/svc/manifest/ghc-builder-client.xml
svccfg import /opt/local/lib/svc/manifest/ghc-builder-client.xml
echo Run 'svcadm enable ghc-builder-client' to enable GHC builder

/bin/sh is ksh98 (missing several patches) on SmartOS. This causes issues with configure scripts generated by autotools. These configure scripts explicitly reference /bin/sh as the interpreter and the ksh98 implementation on SmartOS can generate broken config.status when ./configure is run. As a result, there is a need to hack around it.

Interactively it can be solved by setting SHELL to /bin/bash. The fact that this works suggests evil in the implementation of interpreter selection within BASH. As in, if bash is in the same directory as sh it appears to assume it is totally okay to act as the interpreter directly instead of calling the specified interpreter.

If programs run automatically as using sh from the path (as the GHC Builder client I am running is configured remotely to do) then you need an sh earlier in the PATH that is symlinked to /bin/bash. My workaround for this is to put a symlink to sh into $HOME/.cabal/bin since it is needed in the PATH and is at its head already.

Run as ghcbld instead of root. This is a good idea in general, but is actually a functional requirement since some tests will fail when run as root.

Partial update to support SmartOS base 15.4.0 or later (anything with PKGSRC 2015Q4 or later which removes haskell-platform).

This is still very much a work in progress. Dropping haskell-platform is a serious annoyance. When I get around to respinning ghc packages for PKGSRC I'm including Cabal as it's an essential piece.

Part of this is made tangly by GHC master now requiring 7.10 as a bootstrap compiler. In order to restore PKGSRC I'm going to need to install GHC 7.6.3 to build 7.10.3. I imagine 7.10.3 is the last GHC buildable with 7.6.3 so if I don't do that soon the process is going to be more tangly.

On [email protected] libssp is now dynamic or at least GHC is missing the -lssp linker flag frequently so I've had to putty in the gaps again.

Key fix step is patching ${GHC_HOME}/settings "C compiler flags" to include -lssp to ensure automatic linking of the Stack Smashing Protection library (libssp). This is important because the implicit linking that resulted from older Illumos builds left binaries open to segmentation faults when the ABI of GCC changed. The SmartOS GZ has a fixed libssp and SmartMachines - by necessity - have a libssp in sync with their GCC version.

This doesn't quite work to get to GHC 8.0.2 (latest stable release as of this writing). GHC 8.0.2 runs as does GHCi, but it can't bootstrap cabal-install-1.24.0.2.

It's a separate issue. bootstrap.sh has a bug if collect2 and ld are both present it emits a mangled LINK command with both in it separately by a newline. It looks like it is fixed in master.