Nix fetchgit false positive: asciinema cast files with nix store paths break fixed-output derivations

nix-fetchgit-false-positive-incident.md

Incident: Nix fetchgit False Positive on Store Path References

TL;DR

Adding --sha256: hashes to source-repository-package entries in cabal.project broke CI with:

error: fixed-output derivations must not reference store paths:
  '/nix/store/...-haskell-csmt-80e41c8.drv' references 1 distinct paths,
  e.g. '/nix/store/rlq03x4cwf8zn73hxaxnx0zn5q9kifls-bash-5.3p3'

Root cause: Asciinema .cast files in the upstream repo contained a hardcoded nix store path for bash that happened to match the CI builder's bash hash. Nix's store path reference scanner flagged this as a "reference" in the fixed-output derivation output and rejected the build.

Fix: Replace /nix/store/...-bash-5.3p3/bin/bash with /bin/bash in the .cast file headers.

---

Timeline

1. Added --sha256: hashes (nix32 format) to all 6 source-repository-package entries in cabal.project for supply chain security
2. CI failed with fixed-output derivations must not reference store paths
3. Local build succeeded (output was already cached in local nix store — the reference scan only runs when building, not when substituting)
4. Hypothesized nix 2.33 regression — spent time investigating nix version differences between local (2.31) and CI (2.33), even created a PR to pin nix version
5. Realized that if nix 2.33 broke pkgs.fetchgit globally, the entire nix ecosystem would be broken — this couldn't be the cause
6. Deep investigation found the actual culprit: two asciinema recording files in the haskell-csmt repository

How Nix's Reference Scanner Works

When a derivation is marked as fixed-output (content-addressed by hash), nix scans the output for references to build inputs. It does this by looking for the 32-character hash portion of any build input's store path as a raw byte pattern in the output files.

For pkgs.fetchgit:

• Build inputs include bash, git, coreutils, etc.
• Output is the cloned repository content
• If any file in the repo contains the same 32-char hash as a build input, nix treats it as a "reference"

The False Positive

The haskell-csmt repo had two asciinema .cast files:

docs/assets/asciinema/proof-ops.cast
docs/assets/asciinema/basic-ops.cast

Both contained this in their JSON header:

{
  "env": {
    "SHELL": "/nix/store/rlq03x4cwf8zn73hxaxnx0zn5q9kifls-bash-5.3p3/bin/bash",
    "TERM": "tmux-256color"
  }
}

The recordings were made inside a nix development shell, which captured the full nix store path for bash. The hash rlq03x4cwf8zn73hxaxnx0zn5q9kifls is the exact same hash as the bash derivation used by pkgs.fetchgit as its builder in CI.

Why It Worked Locally

Three possible reasons:

1. Cached output: The fetchgit output was already in the local nix store (substituted from cache). The reference scan only runs when building, not when substituting.
2. Different bash hash: Different nixpkgs versions produce different bash derivation hashes. If the local bash hash differs from the one in the .cast files, no match occurs.
3. Same nix version, different store: Build inputs in the local store may have different hashes than in CI.

Why It's Insidious

• The error is timing-dependent: it only triggers when the builder's bash hash matches the hash in the .cast files
• It masquerades as a nix version issue: the error message gives no hint that the problem is in the repo content
• It works on some machines but not others: depending on cached outputs and bash versions
• The .cast files are documentation assets that nobody thinks to inspect for build issues

Lessons Learned

1. Never commit nix store paths to version-controlled files. When recording terminal sessions, generating test data, or creating config files inside a nix shell, sanitize store paths before committing.
2. When fixed-output derivations must not reference store paths appears, the fix is almost always in the content of the fetched repo, not in the nix version or the build configuration.
3. Grep for nix/store in your repos before adding --sha256: hashes:

   git clone --depth 1 <url> /tmp/check && grep -r 'nix/store' /tmp/check

References

• NixOS/nix #11673 — Help debug "illegal path references in fixed-output derivation"
• NixOS/nix PR #12356 — Improved error message
• Phip1611 Blog — Fixing illegal path references in fixed-output derivation
• haskell-csmt PR #55 — The fix