Skip to content

Instantly share code, notes, and snippets.

@hypevhs

hypevhs/notes-nix-sensei.md

Last active February 28, 2022 16:03
Show Gist options
  • Save hypevhs/a4c1f11b23680c5061d4204835258fb1 to your computer and use it in GitHub Desktop.
Save hypevhs/a4c1f11b23680c5061d4204835258fb1 to your computer and use it in GitHub Desktop.
Download ZIP
Haskell sensei is broken on nix! Can we fix it?
Raw
notes-nix-sensei.md

Sensei, the haskell test runner, is marked as broken in nixpkgs. The build must be failing. But why?

Build it. It's broken, and because flake registry points to master by default, specify github branch so I don't download an extra 3 GB just getting newer coreutils. The --impure is so nix recognizes the envvar, as it's an "impure" input.

NIXPKGS_ALLOW_BROKEN=1 nix build --impure 'github:nixos/nixpkgs/nixpkgs-unstable#haskellPackages.sensei'

Fails 13 examples. All of them mention posix_spawnp. Example:

test/EventQueueSpec.hs:16:9:
1) EventQueue.processEvents, with FileEvent, returns TriggerAction
     uncaught exception: IOException of type NoSuchThing
     git: readCreateProcess: posix_spawnp: does not exist (No such file or directory)

To rerun use: --match "/EventQueue/processEvents/with FileEvent/returns TriggerAction/"

Let's try getting a dev shell.

NIXPKGS_ALLOW_BROKEN=1 nix develop --impure 'github:nixos/nixpkgs/nixpkgs-unstable#haskellPackages.sensei'

Now normally I'd consult the "generic builder" bash script, ummmm something-something stdenv, but let's see if the fact that sensei is a haskell pkg makes things different. Yes, a haskell builder: haskell-modules/generic-builder.nix#L309. But setupCompilerEnvironmentPhase is not defined. Running declare -F shows that a few other builder functions are, though.

genericBuild

This craps things out in my current directory. It also fails a completely different example than usual. It also ignores the fact that tests failed, and attempts to do subsequent phases, such as ELF patching. I think this might be the wrong builder.

Here's the sensei derivation in hackage-packages.nix:

"sensei" = callPackage
  ({ mkDerivation, ansi-terminal, base, bytestring, directory
   , filepath, fsnotify, hspec, hspec-discover, hspec-meta, hspec-wai
   , http-client, http-types, interpolate, mockery, network, process
   , silently, stm, text, time, unix, wai, warp
   }:
   mkDerivation {
     pname = "sensei";
     version = "0.6.0";
     sha256 = "19hbm83v3bn2ximfd5bqjzq1xb079ajxbw1kc8gkm9ds4mg7aw0b";
     isLibrary = false;
     isExecutable = true;
     executableHaskellDepends = [
       ansi-terminal base bytestring directory filepath fsnotify
       http-client http-types network process stm text time unix wai warp
     ];
     testHaskellDepends = [
       ansi-terminal base bytestring directory filepath fsnotify hspec
       hspec-meta hspec-wai http-client http-types interpolate mockery
       network process silently stm text time unix wai warp
     ];
     testToolDepends = [ hspec-discover ];
     description = "Automatically run Hspec tests on file modifications";
     license = lib.licenses.mit;
     hydraPlatforms = lib.platforms.none;
     broken = true;
   }) {};

This might be what was being called. Nix's default-builder.sh#L1.

echo $out
echo $prefix
# both print <PWD>/outputs/out. yuck.
echo $NIX_BUILD_TOP
# prints /tmp/nix-shell.gbccIr

In which I get distracted by current directory shenanigans

What do I call in order to make a new temp dir? Like in NIX_BUILD_TOP there. And does that same script run all the phases and is aware of all the haskell-specific stuff? ... NIX_BUILD_TOP is set by, and is cded into, by the nix CLI tool itself, before it forks off the builder. Let's try with hello:

nix develop 'github:nixos/nixpkgs/nixpkgs-unstable#hello'
cd "$NIX_BUILD_TOP"
echo $out
# still yuck. even if I `source $stdenv/setup` again.

Who sets the $out env var? Maybe nix primops.cc#L1148? Meh. I need to set it myself explicitly.

cd "$NIX_BUILD_TOP"
out="$PWD/outputs/out"
unset NIX_LDFLAGS
unset NIX_NO_SELF_RPATH
source "$stdenv/setup"

This is so hacky, fuck. It also ruins my PATH and makes any nonzero codes exit the shell. I'll cd to my own directory first and then run nix develop, ignoring the auto-created NIX_BUILD_TOP.

genericBuild
ls "$out/bin/hello"

Cool. Let's go back to haskell.

Back to the issue at hand

mkcd ~/src/dev-sensei
NIXPKGS_ALLOW_BROKEN=1 nix develop --impure 'github:nixos/nixpkgs/nixpkgs-unstable#haskellPackages.sensei'
genericBuild

It's gonna fail the wrong tests again I bet... Yep. It shat out some files to $NIX_BUILD_TOP. Not much I can do about that anymore lol. Anyway, maybe the derivation I'm supposed to be building is different. What .drv does picrossbot want?

In which I mistakenly thought the .drv was different

cd picrossbot
nix develop

It starts to build, and it doesn't print sensei's drv file until after it fails. Then, what .drv have I been building standalone?

NIXPKGS_ALLOW_BROKEN=1 nix build --impure 'github:nixos/nixpkgs/nixpkgs-unstable#haskellPackages.sensei' --dry-run
  • /nix/store/b9lb37kmwqam9nc6xv6civ5g5brwa08m-sensei-0.6.0.drv - what I build standalone
  • /nix/store/b9lb37kmwqam9nc6xv6civ5g5brwa08m-sensei-0.6.0.drv - what fails in picrossbot devShell
  • /nix/store/c59ncn961kfsvgskj5mcj1baxx13b0ml-sensei-0.6.0.drv - something else I found lol

They're the same... Ugh, maybe this is --impure's fault that despite being the same drv, they fail in different ways. First, let's see what the third one is.

nix run 'nixpkgs#nix-diff' -- /nix/store/b9lb37kmwqam9nc6xv6civ5g5brwa08m-sensei-0.6.0.drv /nix/store/c59ncn961kfsvgskj5mcj1baxx13b0ml-sensei-0.6.0.drv

These are way too different. I think there may have been a bash update of some kind. I'm ignoring this and GCing all three.

In which I get distracted by --impure

I want to try building without impure. I have two options.

  1. Create an override that un-breaks it, and build the local flake.
  2. Check out nixpkgs, manually remove broken, and build the local flake.

Trying option #1.

-    sensei
+    (sensei.overrideAttrs (oldAttrs: { broken = false; }))

This didn't work. Neither did override. Neither did any overlays. It still says to enable broken. I think it refuses to even evaluate the original drv.

Trying option #2. (Oof, it was a mistake to GC all of ghc)

cd picrossbot
nix develop
# 13 failures in building /nix/store/b9lb37kmwqam9nc6xv6civ5g5brwa08m-sensei-0.6.0.drv
mkcd ~/src/dev-sensei
nix build '/nix/store/b9lb37kmwqam9nc6xv6civ5g5brwa08m-sensei-0.6.0.drv'
# same 13 failures
nix develop !$
genericBuild
# 1 failure!?

Back to the issue at hand, again

I think it's because I have a PATH with lots of extra goodies in it. So if I do it again, but trim PATH before running the builder...

nix develop '/nix/store/b9lb37kmwqam9nc6xv6civ5g5brwa08m-sensei-0.6.0.drv'
echo "$PATH" > sourceme.txt
nvim sourceme.txt # prepend 'PATH=' and remove any path not in the /nix/store/
source sourceme.txt
genericBuild

We get the expected 13 failures. So, uh, what are the failures anyway?

echo $phases
# setupCompilerEnvironmentPhase unpackPhase patchPhase compileBuildDriverPhase configurePhase buildPhase checkPhase haddockPhase installPhase fixupPhase installCheckPhase distPhase
# for each of these, prefer the envvar (if any) over the function (if any)
eval "$setupCompilerEnvironmentPhase"
unpackPhase
cd "$sourceRoot"
echo $patches
# none.
patchPhase
eval "$compileBuildDriverPhase"
eval "$configurePhase"
eval "$buildPhase"
eval "$checkPhase"
# etc, etc

our first failure:

  test/EventQueueSpec.hs:16:9:
  1) EventQueue.processEvents, with FileEvent, returns TriggerAction
       uncaught exception: IOException of type NoSuchThing
       git: readCreateProcess: posix_spawnp: does not exist (No such file or directory)

  To rerun use: --match "/EventQueue/processEvents/with FileEvent/returns TriggerAction/"

I think it has something to do with readProcess. Hoogle says it's in System.Process.

Prelude> import System.Process
Prelude System.Process> readProcess "git" [] ""
*** Exception: git: readCreateProcess: posix_spawnp: does not exist (No such file or directory)

So there you go. Git needs to be a build input of some kind.

PATH="/nix/store/wq104960vra7gl064f1865xgiypdjc2r-git-2.33.1/:$PATH"
eval "$checkPhase"

Only one failure now.

  test/UtilSpec.hs:36:11:
  1) Util.filterGitIgnoredFiles_, when used outside of a git repository, returns all files
       expected: (Just (Cyan,"warning: not a git repository - .gitignore support not available\n"),["foo","bar"])
        but got: (Just (Red,"fatal: not a git repository (or any parent up to mount point /)\nStopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).\n"),["foo","bar"])

  To rerun use: --match "/Util/filterGitIgnoredFiles_/when used outside of a git repository/returns all files/"

This is how you rerun like it says.

./Setup test --test-options '--match "/Util/filterGitIgnoredFiles_/when used outside of a git repository/returns all files/"'

For whatever reason, running git check-ignore in /tmp/blah/ gives a different error message than in /home/me/blah/. I can edit the code to support that second error message ("or any parent up to mount point") and treat it like the original ("not a git repository").

The fixes

Here's the patch.

diff --git a/src/Util.hs b/src/Util.hs
index 0e37291..4ccb13b 100644
--- a/src/Util.hs
+++ b/src/Util.hs
@@ -59,8 +59,10 @@ gitCheckIgnore files = do
   where
     join_ = intercalate "\0"
     split = map T.unpack . T.split (== '\0') . T.pack
+    notGitWarning = Just (Cyan, "warning: not a git repository - .gitignore support not available\n")
     feedback err
-      | err == "fatal: not a git repository (or any of the parent directories): .git\n" = Just (Cyan, "warning: not a git repository - .gitignore support not available\n")
+      | err == "fatal: not a git repository (or any of the parent directories): .git\n" = notGitWarning
+      | "fatal: not a git repository (or any parent up to mount point " `isPrefixOf` err = notGitWarning
       | err == "" = Nothing
       | otherwise = Just (Red, err)

And here's the overlay.

(oself: osuper: {
  haskellPackages = osuper.haskellPackages.override {
    overrides = hself: hsuper: {
      sensei = oself.haskell.lib.overrideCabal hsuper.sensei (old: {
        patches = (old.patches or []) ++ [ ./patches/fix-git-spec.patch ];
        testHaskellDepends = old.testHaskellDepends ++ [ oself.git ];
      });
    };
  };
})

So as long as my project's flake gets this overlay as a part of importing nixpkgs, sensei will build correctly. Like this:

  outputs = { self, nixpkgs, flake-utils }:
  (
    flake-utils.lib.eachSystem [ "x86_64-linux" ] (system: let
      overlays = [
        # ... <-----
      ];
      pkgs = import nixpkgs {
        inherit system overlays; # <-----
        config.allowBroken = true;
      };
      project = returnShellEnv: (
        pkgs.haskellPackages.developPackage {
          inherit returnShellEnv;
          name = "picrossbot";
          root = ./.;
          modifier = drv: (
            pkgs.haskell.lib.addBuildTools drv (with pkgs.haskellPackages; [
              # Specify your build/dev dependencies here.
              # ...
              sensei # <-----
            ])
          );
        }
      );
      in {
        defaultPackage = project false;
        devShell = project true;
      }
    )
  );

So sensei builds correctly now. Does it WORK? It absolutely does.

How to upstream?

How can I upstream these fixes? The git dependency is bug 1, the not-a-git error message being different is bug 2.

Bug #1 - git dependency

I doubt I can add git to sensei's cabal file. It's a program that requires a binary to be present in $PATH. How can I communicate that? I found this article while not even searching for nix!:

https://input-output-hk.github.io/haskell.nix/tutorials/pkg-map/

It seems this is a redesign of what's currently in nixpkgs. It'd be nice if it was real so I could use this.

I think this is the place to put both bugfixes, actually: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-common.nix

I chose this file instead of configuration-nix.nix because these failures would also occur on a "standard, FHS-compliant non-nix system". Namely, the tmp dir being mounted differently and the git binary being missing.

Reading https://cabal.readthedocs.io/en/3.6/cabal-project.html, it seems that this is the option to add in order to ensure a binary is in the PATH. But then again, maybe not. These docs are really sparse.

This package had its test suite disabled because it required git. https://github.com/NixOS/nixpkgs/blob/4a921e5be5251c1faf37aeb4dbd7bbaee82f1dbf/pkgs/development/haskell-modules/configuration-nix.nix#L545-L546 That seems excessive.

These packages had git added to the test suite. Even though the ones I checked didn't even use it. Of course sensei uses git in the runtime as well, so is this relevant? https://github.com/NixOS/nixpkgs/blob/4a921e5be5251c1faf37aeb4dbd7bbaee82f1dbf/pkgs/development/haskell-modules/configuration-nix.nix#L803-L808

Based on the comment, this is another good example. But instead of build-time, I need test-time. https://github.com/NixOS/nixpkgs/blob/af63e81ad9cc8326d235ee9bd555b1357d8b761d/pkgs/development/haskell-modules/configuration-common.nix#L1404-L1416

Look at this, I could add pkgs.git to any of these. https://github.com/NixOS/nixpkgs/blob/4a921e5be5251c1faf37aeb4dbd7bbaee82f1dbf/pkgs/development/haskell-modules/generic-builder.nix#L550-L573

I think it comes down to a cross-compilation issue. git should be available for the targetPlatform during runtime, and for the hostPlatform during unit testing (nix build). So add nativeBuildInputs for testing and buildInputs for runtime.

"tool" vs "system" vs "framework"? "executable" vs "test"? "buildDepends" and "buildTools" and "testDepends"? for "buildInputs", "native" or "propagated" or neither?

Suppose I have testToolDepends. It will end up in:

  • collectedToolDepends
    • nativeBuildInputs
    • nativeBuildInputs for the envFunc
  • getCabalDeps

But suppose I have testFrameworkDepends:

  • otherBuildInputsSystem
    • buildInputs for the envFunc
  • otherBuildInputs
    • buildInputs
    • old getBuildInputs
  • getCabalDeps

And what about testSystemDepends:

  • otherBuildInputsSystem
    • buildInputs for the envFunc
  • otherBuildInputs
    • buildInputs
    • old getBuildInputs
  • getCabalDeps

For executables, let's see executableToolDepends:

  • collectedToolDepends
    • nativeBuildInputs
    • nativeBuildInputs for the envFunc
  • getCabalDeps

executableFrameworkDepends:

  • otherBuildInputsSystem
    • buildInputs for the envFunc
  • otherBuildInputs
    • buildInputs
    • old getBuildInputs
  • getCabalDeps

executableSystemDepends:

  • otherBuildInputsSystem
    • buildInputs for the envFunc
  • otherBuildInputs
    • buildInputs
    • old getBuildInputs
  • getCabalDeps

So it seems testSystem and testFramework are the same. I think framework is a macOS term. Tool means nativeBuildInputs and system means buildInputs. So I should add git to testToolDepends and executableSystemDepends.

Bug #2 - GIT_DISCOVERY_ACROSS_FILESYSTEM

Before I apply any patches, I want to figure out why the TMPDIR failure happens, and if it's not nix-specific, upstream the change to sensei itself.

Run nix develop on the unpatched sensei drv. Run the single specific test, and use strace to determine what directory is used.

nix develop '/nix/store/b9lb37kmwqam9nc6xv6civ5g5brwa08m-sensei-0.6.0.drv'
# setupCompilerEnvironmentPhase unpackPhase patchPhase compileBuildDriverPhase configurePhase buildPhase checkPhase haddockPhase installPhase fixupPhase installCheckPhase distPhase
eval "$setupCompilerEnvironmentPhase"
unpackPhase
cd "$sourceRoot"
patchPhase
eval "$compileBuildDriverPhase"
eval "$configurePhase"
eval "$buildPhase"
strace -e mkdir --output-separately ./Setup test --test-options '--match "/Util/filterGitIgnoredFiles_/when used outside of a git repository/returns all files/"'

No dice. Taking off the filter and | grep -i tmp or mockery doesn't help, either. In the end, I had two solutions:

  1. I just added xx <- readcommand pwd, printStrLn xx to the test and it gave me /tmp/nix-shell.nI7JcD/mockery-4d5dbda8f8cd9dfd.
  2. I used strace -f -e mkdir -e signal=none -e quiet=exit,attach ... >/dev/null and it gave me similar results. The fact that the process fork'd was key.

So there you go. This would have broken on any linux system (because of tmpfs), let alone arch, let alone archnix. The author was likely using macOS which doesn't use any special FS for /tmp. So I want to upstream this to sensei.

Upstreaming my fixes

PR created for bug #2. Now we wait. hspec/sensei#53

For bug #1, I was looking for a cabal field for non-haskell binary requirements that cabal2nix understands so that I might PR sensei, but when I searched github for readProcess (lang = Haskell) I found nothing. Even if the field exists, nobody uses it. So upstreaming this likely means changing configuration-common.nix to explicitly require pkgs.git for test and execution.

Summary

The sensei build is broken (and is marked as such) on nixpkgs because of two different bugs. The first is that the sensei specs require the git binary to be on the PATH (an envvar that nix is very explicit in controlling), and cabal does not have any field to signal its non-haskell requirements, esp. in a way that cabal2nix would pick it up. The second bug is that git, when searching for the .git folder and then not finding one, shows different error messages depending on whether it hits a filesystem boundary or /. These bugs were causing the spec to fail, and probably the executable to malfunction.

Currently, I have an overlay in my project that fixes both issues. I'm trying to upstream my fixes. I have a PR open against sensei itself fixing the second bug, and once that merges, I'll bump sensei's version within nixpkgs while fixing the first bug by adding sensei to the configuration-common.nix overlay such that it requires pkgs.git. Finally, I will figure out how to mark sensei as no longer broken, unless hydra or a maintainer does this routinely.

Other Links

Feedback on Sensei PR

Maintainer's concerns:

  • Move feedback var to top level and rename it to gitCheckIgnoreFeedback
  • Add a test that passes the exact, complete error message to gitCheckIgnoreFeedback

Method A: cabal2nix

I need to test/run/develop sensei in my normal source working directory, rather than a directory created via a nixpkgs derivation. So I will add flake.nix to sensei. How can I do that? I looked at running hackage2nix in nixpkgs, but the README for that project says there's no reason to run hackage2nix directly. I'll try cabal2nix.

cd src/sensei
nix shell 'nixpkgs#haskellPackages.cabal2nix-unstable'
cabal2nix . > sensei.nix
exit

Any different than the derivation in hackage-packages.nix? Barely. But rather than call cabal2nix directly, I'll do what I did in my haskell project...

Method B: developPackage

Just use pkgs.haskellPackages.developPackage { root = ./.; } in a flake output.

git add ./flake.nix # no commit yet
nix develop

It almost seems to work until it doesn't.

Method C: practical template

https://serokell.io/blog/practical-nix-flakes#packaging-existing-applications

rm flake.nix flake.lock
nix flake init -t 'github:serokell/templates#haskell-cabal2nix'
# edit packageName, add nixpkgs-unstable tracking to nixpkgs input
nix develop
./Setup.lhs configure
# Configuring sensei-0.6.1...
# Setup.lhs: Encountered missing or private dependencies:
# ansi-terminal -any,
# fsnotify -any,
# http-client >=0.5.0,
# http-types -any,
# network -any,
# wai -any,
# warp -any

Ehhh

Method B again

Note: Need to check out submodules.

./Setup.lhs configure
./Setup.lhs build
./Setup.lhs test
# Setup.lhs: No test suites enabled. Did you remember to configure with
# '--enable-tests'?
./Setup.lhs configure --enable-tests
# Setup.lhs: Encountered missing or private dependencies:
# hspec >=2.9.0

Distraction: doCheck

What, where is hspec? Is it missing? (Narrator: no it wasn't)

The haskell generic-builder.nix uses doCheck as an argument on whether or not to enable tests.

nix develop --profile /tmp/sensei-build-env .
exit
jq '.variables.out.value' /tmp/sensei-build-env
# /nix/store/ch8hyqry0hvyz57x8z9xhbi31wmfngil-ghc-shell-for-sensei-0.6.1-env
nix develop /nix/store/ch8hyqry0hvyz57x8z9xhbi31wmfngil-ghc-shell-for-sensei-0.6.1-env
# ????
exit
nix build
# error: builder for '/nix/store/vwbwq5k75ly7x43bfibybcssj9gpaw26-sensei-0.6.1.drv' failed with exit code 1;
nix show-derivation /nix/store/vwbwq5k75ly7x43bfibybcssj9gpaw26-sensei-0.6.1.drv
# ...
# "inputDrvs": {
#   "/nix/store/0znzkxa0hvsssav62vk54589xlcmwswk-cabal2nix-sensei.drv"
nix build '/nix/store/0znzkxa0hvsssav62vk54589xlcmwswk-cabal2nix-sensei.drv'
grep doCheck result/default.nix
# no results. Default is probably doCheck = true.

Hspec version problem

When I run nix develop, hspec is there, but it's too outdated. haskellPackages.hspec is 2.7.10 but sensei-0.6.1 started to require hspec-2.9.0. I was working with sensei-0.6.0 this whole time so I didn't notice.

There's a haskellPackages.hspec_2_9_4. A disappointing latest-version-only-available-as-different-package problem. Let's add it to the flake as a manual dependency. This is a hack, but since this is a nix packaging issue this is a good use case for a nix-flavored hack instead of a haskell-flavored one.

However, the build of this version of hspec is broken! Hydra has no recollection of this package. It's broken at least by itself, because haskellPackages.hadolint, which uses this version, seems to build OK. They install all three in the family all at once: hspec, hspec-core, hspec-discover. I tried doing it a dozen different ways, but was having infinite recursion problems. This is because the tests (yeah, the tests for hspec family of libraries themselves) require hspec. I just needed to dontCheck them all.

Finally, it evaluates and builds. But some tests related to hspec-meta are failing. Cannot satisfy package?

nix develop
# can skip configure and build cause dist/ is already there
./Setup.lhs test --test-options '--match "/Trigger/trigger/with an hspec-meta spec/reloads and runs spec/"'
# <command line>: cannot satisfy -package hspec-meta
# (use -v for more information)

The test code itself:

withSession ["-package hspec-meta", ...

According to local hoogle, withSession is from GHC, specifically GhcMonad.

hspec-meta has its own *2_9_3, so I overrid that. No dice. nix build is probably more reproducable, but it's currently angry:

Building test suite 'spec'
[ 7 of 19] Compiling Session          ( src/Session.hs, dist/build/spec/spec-tmp/Session.o, dist/build/spec/spec-tmp/Session.dyn_o )

src/Session.hs:30:1: error:
    Could not find module ‘Language.Haskell.GhciWrapper’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
30 | import qualified Language.Haskell.GhciWrapper as GhciWrapper
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

src/Session.hs:31:1: error:
    Could not find module ‘Language.Haskell.GhciWrapper’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
31 | import           Language.Haskell.GhciWrapper hiding (new, close)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

GhciWrapper being inside a submodule doctest/. Looking at nix log for the derivation, it only unpacks one source archive from the nix store, and builds it... somewhere. So whereas nix develop uses my working directory to build everything, nix build uses the nix store and a tmp dir. How do I fetchSubmodules? Can I pass that as an attr-arg to developPackage?

After some searching, and some dead ends involving filterSourceWith and string contexts, I discovered that flakes don't fetch submodules by default. This includes flakes in the current directory. I can't even use symlinkJoin; the submodule files are completely ignored. I'm blocked by NixOS/nix#5497.

Soooo in the meantime I will desubmodulize doctest just for testing purposes!

Now with my old PR patch I get many failed tests for many different reasons. One is a missing dependency on git. Of course, the flake nor the cabal file mentions a binary dependency yet. For git executable dependency, this nix haskell tutorial seems to recommend using the extra-libraries field. But then it directly overrides it in the nix expression that overrides cabal2nix output. So then uhhhh what's the point of the extra-libraries?

When I do it, it thinks "git" is the haskell dep by the same name, rather than the system lib.

I also tried system-build-tools in package.yaml, same result. Given that I need git to run this program and not just build or test it, I think I will override this in nixland. I still believe adding git to the test system-build-tools is a good idea, but cabal2nix misinterprets it. I can't be bothered to upstream fixes to both.

Old way:

overrides = hself: hsuper: {
  sensei = oself.haskell.lib.overrideCabal hsuper.sensei (old: {
    patches = (old.patches or []) ++ [ ./patches/fix-git-spec.patch ];
    testHaskellDepends = old.testHaskellDepends ++ [ oself.git ];
  });
};

So I just plug this overrideCabal into modifier, sans patches, changing oself to pkgs. and hsuper.sensei to drv. It works!

Git should be in testToolDepends not testHaskellDepends because it's not a haskell package, and because it needs to be available in nativeBuildInputs -- in the build host system.

So back to the hspec-meta test failing. Even though I added it to the flake as an override, it still reports "cannot satisfy". I think because hspec-meta is not even an argument in the cabal2nix so it can't be overridden. It needs to be depended on by the derivation. I think testHaskellDepends.

Now everything builds and develops. Fixing the PR now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment