Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save sini/dd522e09e793cf7591e11e9e7bc231cd to your computer and use it in GitHub Desktop.

Select an option

Save sini/dd522e09e793cf7591e11e9e7bc231cd to your computer and use it in GitHub Desktop.
flake-aspects Library Extraction (Final) — Den Aspect Type System Spec

Aspects Library Extraction (Final)

Extract den's aspect type system into a standalone library — a successor to flake-aspects with nested aspect support, built-in identity, unified module key typing, and schema extensibility. Developed inside den at nix/lib/aspects2/, later extracted to its own repo with flake-aspects' git history preserved.

Companion specs:

  • den-schema (github:denful/den-schema) — standalone schema library providing sidecar extraction, strict mode, identity hashing, methods, and instance registries. The aspects library adopts den-schema's sidecar extraction pattern; the two libraries are independent but composable.
  • HOAG Pipeline Architecture — the demand-driven attribute grammar evaluator that consumes flake-aspects as its content data model. flake-aspects provides three native aspect shapes; the HOAG pipeline adds a fourth (conditional) via the schema extension mechanism. flake-aspects' resolve output is wrapped by the pipeline to attach per-element identity for dedup.
  • scope-engine — the generic HOAG evaluator that uses flake-aspects' identity.key as the dedup key in the resolved-aspects attribute.

Revision history:

  • v1 (2026-05-09): Two-branch freeform dispatch (aspectSubmodule + deferredModule)
  • v2 (2026-05-10): Three-branch dispatch, adds parametricType. Theoretical — written before implementation.
  • revised (2026-05-10): Post-implementation. Incorporates learnings from fixing the __contentValues wrapper leak in den's current architecture. Corrects v2's assumptions about listOf merge behavior and documents the multi-path include dedup problem.
  • revised+schema (2026-05-11): Incorporates design improvements from den-schema implementation. Replaces schemaExtensions module-list with sidecar extraction pattern. Replaces either + addCheck with custom mkOptionType for mixed-def merge correctness. Adds !isModuleFn guard to coercedAspectType. Adds lazy-eval constraint documentation.
  • final (2026-05-19): HOAG pipeline integration. Clarifies conditional aspects as den extension (not native fourth shape). Documents resolve identity wrapping for HOAG dedup. Adds meta.guard to den schema modules appendix. Updates migration path for HOAG pipeline.

Goals

  1. Standalone library — usable without den, drop-in upgrade from flake-aspects
  2. Built-in identity — aspects are addressable by key (aspect-chain + name), computed from the type system
  3. Nested aspectsfoo.bar.baz can be an aspect, not just a class key
  4. Unified module keys — no class/quirk distinction in the type system; all freeform keys are module content with identity and provenance
  5. Schema extensibility — den adds policies, constraints, collision policy, etc. via a schema extension point
  6. flake-parts integration — ships flakeModule.nix for flake.aspectsflake.modules wiring

Non-Goals

  • Pipeline execution (scope graph evaluation, context injection) — that's scope-engine + den-pipeline (HOAG spec)
  • Policy dispatch, enrichment convergence — den-specific attributes in the HOAG evaluator
  • Pipe/quirk routing — den-pipeline concern (modeled as import edges in the HOAG spec)
  • Constraint system (exclude, substitute, filterBy) — den schema extension via sidecars
  • Class/quirk registry dispatch — downstream classification via schema-provided registries
  • Conditional aspects (meta.guard) — den extension via schemaModules, not a native library shape. The HOAG pipeline adds guard evaluation as a den-specific attribute (guard-set). flake-aspects provides the metaType extension point; Den populates it.

Approach

Hybrid: flake-aspects' file layout and API shape, rewritten type system informed by both codebases. Drops flake-aspects' simpler-than-needed freeform type and den's aspectContentType/__contentValues wrapper complexity. Retains __fn/__args parametric wrappers at freeform positions only (necessary to break the coercion recursion — see "Three-branch freeform dispatch"). Clean-room types.nix with the right semantics from the start.

File Layout

nix/lib/aspects2/
  types.nix         — aspectSubmodule, aspectType, coercedAspectType, aspectsType
  resolve.nix       — recursive include tree expansion
  identity.nix      — aspectPath, pathKey, key, structural keys
  default.nix       — 2-level transpose with emit customization
  aspects.nix       — wires transpose + resolve
  forward.nix       — cross-class forwarding
  new.nix           — low-level aspect scope factory
  new-scope.nix     — named scope creator
  flakeModule.nix   — flake-parts integration
  lib.nix           — public API surface

Developed at nix/lib/aspects2/ during the dev cycle. Den's current nix/lib/aspects/ stays untouched until the pipeline migrates. When extracted, aspects2/ becomes the new repo's root.

Type System (types.nix)

Core types

aspectSubmodule(config) — the core aspect type.

  • freeformType: lazyAttrsOf aspectType — all non-structural keys are module content
  • Structural options:
    • name: str (default: attr name)
    • description: str (default: "Aspect ${name}") — carried forward from flake-aspects
    • meta: metaType — identity and provenance
    • includes: listOf coercedAspectType — composition references (functions coerced to { includes = [fn]; })
    • provides: submodule with freeformType = lazyAttrsOf coercedAspectType — sub-aspects
    • _: alias for provides (via mkAliasOptionModule)
    • __functor: functorType — default provider function (see below). Makes aspects callable.
    • resolve: internal read-only — { class, aspect-chain? } → module. Triggers recursive resolution for a specific class.
    • modules: internal read-only — lazyAttrsOf module. Lazily resolved per-class modules: aspect.modules.nixos.
  • config._module.args.aspect = config — self-reference, so module functions can access the aspect via { aspect, ... }:
  • Provides forwarding: provides children are shallow-merged onto the aspect root post-merge, so aspect.docker resolves to aspect.provides.docker. Forwarded keys are tagged (__providesForwarded) so classification skips them.
  • Sidecar extraction: metadata channels (e.g. policies, excludes) extracted from defs before submodule merge, exposed on result
  • Schema modules: imports ++ schemaModules (list of NixOS modules declaring typed options, e.g. class keys)

functorType — custom merge type for the __functor option, carried forward from flake-aspects.

  • Preserves __functionArgs through module system merges (critical for parametric providers where users inspect lib.functionArgs on provides children)
  • Last-def semantics: when multiple modules define __functor on the same aspect, the last definition wins
  • Wraps non-function results so the functor always returns a callable chain
functorType = lib.types.mkOptionType {
  name = "aspectFunctor";
  description = "aspect functor function";
  check = lib.isFunction;
  merge = _loc: defs:
    let lastDef = lib.last defs;
    in {
      __functionArgs = lib.functionArgs lastDef.value;
      __functor = _: callerArgs:
        let result = lastDef.value callerArgs;
        in if builtins.isFunction result then result else _: result;
    };
};

The default __functor value is configurable via defaultFunctor parameter on mkAspectType (default: identity provider that returns the aspect itself when called). This allows consumers to customize how aspects behave when called as functions — flake-aspects uses this for context-aware resolution; den overrides it for resolveAspectWith.

aspectsType — top-level aspects container.

  • freeformType: lazyAttrsOf (aspectType cnf) — all keys are aspects
  • config._module.args.aspects = config — injects aspects so definition functions can use { aspects, ... }: to reference sibling aspects
  • Used for flake.aspects (flake-parts), ${name}.aspects (evalModules), and den.aspects
aspectsType = cnf: lib.types.submodule ({ config, ... }: {
  freeformType = lib.types.lazyAttrsOf (aspectType cnf);
  config._module.args.aspects = config;
});

The provides submodule also injects config._module.args.aspects = config for the same reason — provides children may need to reference siblings via { aspects, ... }:.

metaType(config) — identity and provenance, built into every aspect.

  • aspect-chain: listOf str — parent chain from root to this aspect (set by type nesting)
  • name: str — computed display name from aspect-chain ++ [name]
  • file: str — definition site
  • loc: listOf str — option location path

Design motivation: isolated responsibility

Den's current type system uses aspectContentType — a universal wrapper that wraps ALL freeform values (functions, attrsets, whatever) into { __contentValues = [...]; __provider = path; }. It doesn't classify. It wraps uniformly and defers classification to the pipeline, which uses contentUtil (unwrapping), key-classification (class vs quirk vs nested), and wrapClassModule + bind/compile-parametric (function arg handling). Three components share the classification concern. This caused a class of bugs (see "Lessons from the __contentValues wrapper leak") where wrappers leaked through merge boundaries and buried content.

The new library isolates responsibility:

  • Type system: classifies values (aspect, module, or parametric) and assigns identity. Bounded scope, ~15 lines of merge logic in types.nix.
  • Pipeline/resolve: receives pre-classified values with identity. No unwrapping, no contentUtil mediation layer, no __contentValues.

The type system's internal complexity (custom mkOptionType with per-def classification, submoduleWith with explicit description, sidecar extraction) is acceptable because it's contained. The overall system is simpler because no component half-owns the classification concern.

Three-branch freeform dispatch

Three-level dispatch using a custom mkOptionType with explicit merge logic. At freeform key positions: attrsets become aspectSubmodule (nested aspect), module functions become deferredModule (terminal), and parametric functions become __fn/__args wrappers (deferred for scope resolution). At aspect positions (root, provides, includes), functions are coerced to { includes = [fn]; } via coercedTo.

Why not lib.types.either: nixpkgs either uses checkDefsForError which validates ALL defs against t1.check. Mixed-type multi-def merges (fn + attrset at the same key) fail: the function def fails isAttrs → t1 rejected → t2 also rejects the attrset defs → falls through to mergeOneOption → error. A custom mkOptionType with explicit per-def classification in its merge function handles mixed defs correctly — matching den's current providerType.merge dispatch pattern (mergeMixed, mergeFunctions).

Freeform type (inside aspects):

mkAspectType = { aspectChain ? [], sidecars ? {}, schemaModules ? [], defaultFunctor ? null }:
  let
    sidecarKeys = builtins.attrNames sidecars;

    aspectSubmodule = lib.types.submoduleWith {
      description = "aspect";  # explicit — prevents forcing docsEval, avoids recursion
      modules = [({ name, config, ... }: {
        freeformType = lib.types.lazyAttrsOf aspectType;
        imports = [
          (lib.mkAliasOptionModule [ "_" ] [ "provides" ])
        ] ++ schemaModules;
        config._module.args.aspect = config;  # self-reference

        options.name = lib.mkOption { type = lib.types.str; default = name; };
        options.description = lib.mkOption { type = lib.types.str; default = "Aspect ${name}"; };
        options.meta = ...;
        options.includes = ...;
        options.provides = lib.mkOption {
          default = {};
          type = lib.types.submodule ({ config, ... }: {
            freeformType = lib.types.lazyAttrsOf (coercedAspectType ...);
            config._module.args.aspects = config;  # sibling reference in provides
          });
        };
        options.__functor = lib.mkOption {
          internal = true; visible = false; readOnly = true;
          type = functorType;
          default = defaultFunctor or (aspect: { class ? null, aspect-chain ? [], ... }:
            if true then aspect else class aspect-chain);
        };
        options.resolve = lib.mkOption {
          internal = true; visible = false; readOnly = true;
          type = lib.types.raw;
          apply = _: { class, aspect-chain ? [] }:
            resolve class aspect-chain (config { inherit class aspect-chain; });
        };
        options.modules = lib.mkOption {
          internal = true; visible = false; readOnly = true;
          type = lib.types.raw;
          apply = _: lib.mapAttrs (class: _: config.resolve { inherit class; }) config;
        };
      })];
    };

    # mergeAspect wraps aspectSubmodule.merge with provides forwarding.
    # ALL paths that produce aspects must route through this — not raw
    # aspectSubmodule.merge. See Problem 8 (c9a0dce3).
    mergeAspect = loc: defs:
      let
        merged = aspectSubmodule.merge loc defs;
        providesChildren = builtins.removeAttrs (merged.provides or {}) ["_module"];
      in
      providesChildren // merged // {
        __functor = merged.__functor or (self: resolve self);
        __providesForwarded = builtins.attrNames providesChildren;
      };

    # Three-branch dispatch:
    #   1. attrsets → mergeAspect (nested aspect with identity + provides forwarding)
    #   2. module functions (config/options/lib args) → deferredModule (terminal)
    #   3. parametric functions (host/user/etc args) → __fn/__args wrapper
    aspectType = lib.types.mkOptionType {
      name = "aspectType";
      description = "aspect, module function, or parametric function";
      check = v: builtins.isAttrs v || builtins.isFunction v;
      merge = loc: defs:
        let
          # Classify each def
          attrDefs = builtins.filter (d: builtins.isAttrs d.value) defs;
          moduleDefs = builtins.filter (d: isModuleFn d.value) defs;
          parametricDefs = builtins.filter (d:
            builtins.isFunction d.value && !isModuleFn d.value
          ) defs;

          hasAttrs = attrDefs != [];
          hasModules = moduleDefs != [];
          hasParametric = parametricDefs != [];
        in
        # Empty defs — should not occur (module system guarantees ≥1 def
        # at merge time), but guard against it rather than falling through
        # to the mixed-def path with an empty list.
        if defs == [] then
          throw "aspectType.merge: no definitions at ${lib.showOption loc}"
        # All attrsets → mergeAspect (provides forwarding)
        else if hasAttrs && !hasModules && !hasParametric then
          mergeAspect loc defs
        # All module functions → deferredModule (terminal, no provides)
        else if hasModules && !hasAttrs && !hasParametric then
          lib.types.deferredModule.merge loc defs
        # All parametric → parametricType (terminal, no provides)
        else if hasParametric && !hasAttrs && !hasModules then
          parametricType.merge loc defs
        # Mixed: coerce functions to { includes = [fn]; } and merge through mergeAspect
        else
          mergeAspect loc (map (d:
            if builtins.isAttrs d.value then d
            else d // { value = { includes = [ d.value ]; }; }
          ) defs);
    };
  in aspectType;

Mixed-def merge: When attrsets and functions appear at the same freeform key (e.g. two files both define aspect.editor), functions are coerced to { includes = [fn]; } and all defs merge through aspectSubmodule. The functions land in the declared includes option — same invariant as the parametricType multi-def path. This matches den's current mergeMixed logic in providerType.merge.

Module function detection:

# A function is a module function if it accepts any NixOS module system arg.
# This replaces isSubmoduleFn (canTake.upTo {lib,config,options}) which is
# too narrow — it rejects { config, pkgs }: because pkgs isn't in the set.
# The broader check handles _module.args-injected parameters correctly.
isModuleFn = fn:
  builtins.isFunction fn
  && let args = builtins.functionArgs fn;
  in args ? config || args ? options || args == {};

args == {} catches { ... }: functions (no named args, only varargs) — these are module functions that rely entirely on _module.args. Parametric functions always have at least one named non-module arg.

Parametric wrapper type:

parametricType = lib.types.mkOptionType {
  name = "parametric";
  description = "parametric function awaiting scope context";
  check = v: builtins.isFunction v && !isModuleFn v;
  merge = loc: defs:
    if builtins.length defs == 1 then
      let
        fn = (builtins.head defs).value;
        nameFromLoc = lib.last loc;
      in {
        name = nameFromLoc;
        meta.aspect-chain = aspectChain;
        __fn = fn;
        __args = builtins.functionArgs fn;
        __functor = self: self.__fn;
      }
    else
      # Multiple parametric defs at the same key: coerce each to
      # { includes = [fn]; } and merge through aspectSubmodule.
      aspectSubmodule.merge loc (map (d: d // {
        value = { includes = [ d.value ]; };
      }) defs);
};

No recursion: { __fn, __args, __functor, name, meta } has no includes key → never re-enters coercedAspectType. Single-def path avoids aspectSubmodule evaluation entirely (same OOM optimization as den's current providerType). Multi-def path coerces to includes, which IS safe through aspectSubmodule because the functions land in the declared includes option, not in a freeform key.

Multi-def path invariant: The coercion { includes = [ d.value ]; } produces an attrset that aspectType.merge classifies as an attrset def → routed to aspectSubmodule.merge. Inside aspectSubmodule, includes is a declared option (listOf coercedAspectType), so the function lands there — NOT in the freeform type. If this invariant breaks (e.g. includes is absent from the submodule options), the function would land in freeform → re-enter parametricType.check → fail !isModuleFn → classified as deferredModule → wrong classification. Implementation must assert that aspectSubmodule's options.includes is defined before parametricType references aspectSubmodule.merge. Since both are defined in the same let block, this is structural — but test coverage of the multi-def path is critical (see Testing Strategy).

Lazy-eval constraint: parametricType and aspectType reference aspectSubmodule.merge in their merge function bodies (lambdas). Both are defined in the same let block as aspectSubmodule. This works because Nix evaluates let bindings lazily — the reference is only forced when the merge function is called, not when the type is constructed. Do not reference aspectSubmodule in check functions or at the let binding level — this would force evaluation during type construction, creating infinite recursion between aspectSubmodule (which references aspectType via its freeform type) and aspectType (which references aspectSubmodule.merge).

Root/provides type (aspect positions):

coercedAspectType = lib.types.coercedTo
  (lib.types.addCheck lib.types.raw
    (v: builtins.isFunction v && builtins.functionArgs v != {} && !isModuleFn v))
  (fn: { includes = [ fn ]; })
  aspectType;

Why !isModuleFn v: Without this guard, module functions with named args (e.g. { lib, ... }: { nixos = ...; }) at root/provides positions pass the functionArgs != {} check and get coerced to { includes = [fn]; }. The function then lands in the includes list typed listOf coercedAspectType. When listOf.merge processes elements, coercedAspectType.merge coerces fn again → { includes = [fn]; } → nested submodule with same function in includes → infinite recursion when resolve walks the include tree. The !isModuleFn v guard matches den's current !isSubmoduleFn v check on coercedProviderType (types.nix:514-515). Module functions at root/provides pass through to aspectType directly, where the custom merge classifies them as module defs → deferredModule.merge (terminal).

Why no infinite recursion: deferredModule and parametricType are both terminal base cases — their merge outputs are not attrsets that re-enter aspectType. lazyAttrsOf defers per-key — only accessed keys recurse. Recursion depth = data nesting depth, which is bounded by the user's actual aspect tree. The explicit description parameter on submoduleWith prevents the type construction from forcing evalModules eagerly (without it, submoduleWith forces documentation evaluation which would recursively instantiate the freeform type). The mixed-def merge path routes through aspectSubmodule with functions wrapped as { includes = [fn]; }includes is a declared option, so the functions enter the listOf coercedAspectType type, not the freeform type.

Why the unified "everything coerces" model doesn't work: coercing ALL functions to { includes = [fn]; } turns them into attrsets, which trigger aspectSubmodule merge, which creates a new freeform with aspectType, which tries to coerce again — unbounded recursion at merge time, not construction time. lazyAttrsOf laziness is per-key but within a single key the merge is eager. The parametricType avoids this because its output { __fn, __args, name, meta } has no includes — the function is stored as __fn, not wrapped in an includes list.

isModuleFn vs isSubmoduleFn

Den's current isSubmoduleFn uses canTake.upTo { lib = true; config = true; options = true; } — requiring ALL named args to be in { lib, config, options }. This rejects { config, pkgs, ... }: because pkgs is not in the set. The check was designed for a world where module functions only take the three standard args; _module.args-injected parameters (like pkgs, self', inputs') break it.

Confirmed by production testing (2026-05-10): Attempting to coerce functions in aspectContentType using !isSubmoduleFn incorrectly coerced { config, pkgs }: { ... } at class key positions (e.g. bspwm.homeManager), producing The option 'includes' does not exist errors. The broader isModuleFn check correctly classifies these.

The new isModuleFn check: "does the function accept config or options, or have no named args?"

Function isSubmoduleFn (old) isModuleFn (new) Correct?
{ config, lib, ... }: yes yes module yes
{ config, pkgs, ... }: no (pkgs) yes module yes (new is correct)
{ pkgs, ... }: no (pkgs) no parametric see below
{ host, user }: no no parametric yes
{ host, ... }: no no parametric yes
{ ... }: no yes module (args == {}) yes

{ pkgs, ... }: edge case: Technically a module function (pkgs from _module.args), but isModuleFn classifies it as parametric. In practice harmless: if at a freeform position, it becomes a __fn/__args wrapper. The pipeline defers it (no pkgs scope handler), logs a warning. Better than silently burying the function. Can be broadened to args ? config || args ? options || args ? lib || args ? pkgs if needed, but config alone covers >99% of real module functions.

Implementation decision: use the broader check. With schema-declared class keys (see below), class key functions bypass freeform entirely — { pkgs, ... }: at a declared nixos option goes through deferredModule regardless of isModuleFn. The only functions reaching freeform dispatch are genuinely non-class. Adding args ? lib || args ? pkgs to the check costs nothing and eliminates the edge case for any undeclared module functions that slip through. Final check:

isModuleFn = fn:
  builtins.isFunction fn
  && let args = builtins.functionArgs fn;
  in args ? config || args ? options || args ? lib || args ? pkgs || args == {};

This is safe because no parametric scope handler uses lib or pkgs as parameter names — those are unambiguously module system args.

Discrimination rules

Position Value Becomes Identity
Root (aspects.*) Function { includes = [fn]; } via coercedTo Yes
Root (aspects.*) Attrset aspectSubmodule Yes
provides.* Function { includes = [fn]; } via coercedTo Yes
provides.* Attrset aspectSubmodule Yes
Freeform key Module fn (config/options args) deferredModule (terminal) Via _file
Freeform key Parametric fn (other named args) { __fn, __args } wrapper Via meta.aspect-chain + name
Freeform key Attrset aspectSubmodule (nested) Via meta.aspect-chain
Freeform key { ... }: (no named args) deferredModule (terminal) Via _file
Mixed defs (same key) fn + attrset fns coerced to { includes = [fn]; }, all merge through aspectSubmodule Yes

Lessons from the __contentValues wrapper leak (2026-05-10)

Debugging a real user config (den-configs/dotfiles) revealed three interacting problems when moving aspects from provides to direct freeform children. These findings shaped the three-branch dispatch design and validated assumptions the v1/v2 specs made theoretically.

Problem 1: Wrapper promotion buries content

Setup: ns.apps.helix = { host, user }: { homeManager = ...; } (parametric function at freeform key).

aspectContentType wraps the function as { __contentValues = [{value = fn; ...}]; __provider = [...]; }. When used as an include element (via <ns/apps/helix> or bare ns.apps.helix), the wrapper enters providerType.merge. The wrapper is an attrset → dispatched to baseType.mergeaspectSubmodule.merge → promoted to a full aspect. The function is buried as a freeform key named __contentValues. The pipeline never reaches it for parametric resolution.

Key insight: providerType.merge is called on include elements because the includes option's type is listOf (providerType typeCfg). The NixOS module system's listOf implementation calls the element type's merge for each individual element — not just for combining multiple definitions of the same option. This is the mechanism that triggers promotion.

Den interim fix: Normalize content wrappers at the start of providerType.merge — detect wrappers with parametric functions (non-empty args, no config/options) and extract the raw function before dispatch. The existing merge branches then handle it correctly (single bare parametric fn → __fn/__args wrapper).

Library fix: Eliminate __contentValues entirely. parametricType produces { __fn, __args, name, meta, __functor } — a value that survives listOf element-level merge because __functor makes it callable, triggering coercedAspectType coercion to { includes = [wrapper]; } rather than aspectSubmodule promotion.

Problem 2: Non-parametric content duplicates via __contentValues classification

Setup: ns.overlays.openrazer.nixos = { nixpkgs.overlays = [...]; } (non-parametric attrset at freeform key).

The content wrapper has nixos = {...} as a forwarded attr AND __contentValues = [{value = {nixos = {...}}; ...}]. When promoted to a full aspect via providerType.merge, both nixos (forwarded) and __contentValues (freeform key) are present. The pipeline classifies __contentValues as an unregistered class key (it's not in structuralKeysSet). Class emission processes its content — which contains {nixos = {...}} — emitting the overlay a second time. Result: patch applied twice.

Den interim fix: Add __contentValues and __provider to structuralKeysSet so they're never classified as class content.

Library fix: No __contentValues keys exist. Attrsets at freeform positions go through aspectSubmodule which produces proper aspects with declared structural keys. No wrapper metadata leaks into classification.

Problem 3: Inner wrappers in includes lists bypass normalization

Setup: Polybar module includes razermon, which includes the overlay. Two include paths reach the overlay: polybar→razermon→overlay and direct razermon→overlay.

The razermon content wrapper's includes = [openrazer-wrapper] is a forwarded attr — the openrazer wrapper passes through listOf concatenation WITHOUT going through providerType.merge (because listOf doesn't call elemType.merge per-element when the list comes from a single definition — only when multiple definitions are merged).

Correction from v2: v2 stated "listOf elemType calls elemType.merge for each list element." This is partially wrong. listOf calls elemType.merge when MERGING multiple definitions of the same list option (each element gets its own merge call in the concatenated result). But when a list comes from a single definition (e.g. forwarded attr), the elements pass through as-is. Inner wrappers in forwarded includes lists are never normalized by providerType.merge.

Den interim fix: Handle inner content wrappers in wrapChild (pipeline's include normalization). When a child has __contentValues but no name, inject identity from __provider and extract parametric functions into includes.

Library fix: No content wrappers exist to leak. Inner includes go through coercedAspectType coercion at definition time (when the aspect submodule processes its includes option), not at pipeline time.

Problem 4: Freeform keys lose parent identity

Setup: gloom.overlays.openrazer — the __provider is ["gloom", "openrazer"], missing the intermediate overlays segment.

The aspectSubmodule freeform type uses the parent's typeCfg without extending providerPrefix with the aspect's own name. The provides submodule DOES extend: providerPrefix ++ [config.name]. Freeform keys should do the same — openrazer on the overlays aspect is logically at gloom/overlays/openrazer, not gloom/openrazer.

Den fix: Thread providerPrefix ++ [config.name] through the freeform type in aspectSubmodule, matching the provides submodule pattern.

Library design: Already specified in v1/v2 — aspectChain ++ [name] threading for freeform. Confirmed correct by the fix.

Problem 5: Multi-path include dedup requires stable identity

Setup: Overlay included from two paths — polybar→razermon→overlay and direct razermon→overlay. Without stable identity, the same content gets different anonymous names and dedup fails.

With provides, the overlay had identity gloom/overlays/openrazer (from the providerType merge) regardless of which path included it. Without provides, the content wrapper had no name, getting anonymous identities like razermon/<anon>:0/<anon>:0 from one path and gloom/overlays/openrazer from another. Different identities → dedup doesn't fire → overlay applied twice.

Key requirement for the library: Identity must derive from the DEFINITION site (where the aspect is declared), not the INCLUDE site (where it's referenced). The type system must assign identity at definition time so all include paths see the same key.

Library design: parametricType sets name from lib.last loc and meta.aspect-chain from the parent chain — both derived from the definition position. aspectSubmodule does the same via metaType. Identity is stable regardless of include path.

Problem 6: Provides children must be accessible at the aspect root

Setup: den.aspects.virtualization.provides.docker.nixos = { ... }. User code references den.aspects.virtualization.docker (without provides or _).

PR #504 removed _.docker accessor usage, assuming provides children would be directly accessible at the root. But no forwarding mechanism existed — aspect.docker raised attribute 'docker' missing.

Why submodule-level forwarding fails: Setting config = lib.mapAttrs ... config.provides in the aspectSubmodule module causes infinite recursion — config.provides depends on the submodule config which now includes the forwarding.

Why naive // forwarding causes double-processing: providesChildren // merged makes provides children appear as regular attribute keys. The pipeline's classifyKeys iterates builtins.attrNames aspect and classifies them as nested or class keys — processing them alongside includes that already reference the same provides children.

Den fix: Forward in mergeWithAspectMeta (post-merge, no circularity). Tag forwarded keys via __providesForwarded so classifyKeys skips them. Forwarded keys are accessible for user code (constraints, references) but invisible to the pipeline.

Library design: Same pattern — post-merge forwarding with classification exclusion. The _ alias remains for explicit access; root-level access is the preferred ergonomic form. The new library's structuralKeysSet includes __providesForwarded (or equivalent marker) to prevent classification of forwarded keys.

Problem 7: Bracket-resolved forwarded attrs lack identity

Setup: <gloom/apps/polybar/razermon> resolves through resolveWithProvidesFallback → reaches the razermon forwarded attr on polybar. The forwarded attr is a bare attrset (from providesChildren // merged) — no __contentValues, no __provider, no __fn.

When this bare attrset enters providerType.merge as an include element, isContentWrapper doesn't match (no __contentValues). The attrset goes through baseType.mergeaspectSubmodule.merge with an anonymous name from the include index. When the same content is reached via a different path (e.g. host-aspects re-resolution), it gets a different anonymous name → dedup fails → module emitted twice.

Den fix (#507): tagProvider in den-brackets.nix injects __provider = path (the full bracket path) onto bare attrset results. isContentWrapper in providerType.merge broadened to detect __provider (not just __contentValues) so the normalization path injects name + meta.provider for stable identity.

Library design: In the new library, forwarded provides children are proper aspects (from aspectSubmodule or coercedAspectType), not bare attrsets. They carry meta.aspect-chain and name from definition time. No post-resolution tagging needed because identity is structural, not wrapper-based.

Problem 8: __functor attrsets bypass provides forwarding in single-def fast path (c9a0dce3)

Setup: den.batteries.import-tree is an attrset with __functor and provides. At a freeform key, it's the sole definition.

Bug 1 — missing provides forwarding: lib.isFunction returns true for { __functor = ... } attrsets, routing them through mergeFunctions' single-def fast path. That fast path returned the raw attrset without provides forwarding or _ alias. aspect._.host and aspect.host both failed.

Bug 2 — parametric class arg only fires once: import-tree used __fn = { class, ... }: ... expecting per-class invocation. The pipeline's bind handler calls __fn once (with the scope's class), not once per registered class. import-tree only resolved homeManager content, missing nixos entirely.

Den fix (c9a0dce3):

  1. Forward provides children and add _ alias in the single-def isAttrs fast path, matching mergeWithAspectMeta behavior.
  2. Rewrite import-tree to eagerly scan _<class> directories via builtins.readDir at definition time, producing a plain attrset with per-class keys. No parametric wrapper needed.

Library design implications:

  1. __functor attrsets must go through provides forwarding. The custom aspectType.merge classifies { __functor } attrsets as attrDefs (builtins.isAttrs = true, builtins.isFunction = false), routing them to aspectSubmodule.mergemergeAspect which includes provides forwarding. This is correct. However, any optimization fast paths that skip mergeAspect (e.g. single-def shortcuts) must still apply provides forwarding. The library's mergeAspect wrapper is the only place that does forwarding — all paths must route through it.

  2. Parametric wrappers are single-invocation. The pipeline's bind handler resolves __fn once with scope context. Parametric functions that need per-class or per-entity invocation must restructure to produce plain attrsets at definition time, using builtins.readDir or similar eager evaluation. The library should document this constraint: parametricType wrappers are resolved exactly once; multi-invocation patterns should use eager attrset construction instead.

Problem 9: Multi-def nested aspect keys lose definitions via last-win overwrite (b7ac9930, #521)

Setup: Two modules both define den.aspects.igloo.base.nixos:

# Module A
den.aspects.igloo.base.nixos.environment.variables.FROM_A = "yes";
# Module B
den.aspects.igloo.base.nixos.environment.variables.FROM_B = "yes";

aspectContentType wraps each module's definition into a __contentValues entry. When compile-static processes the nested key base, unwrapContentValuesRaw extracts only one value — the other definition is silently dropped. Result: only FROM_B (or FROM_A) survives, not both.

Root cause: unwrapContentValuesRaw assumes each freeform key has a single effective value. Nested aspect keys are freeform keys that can have multiple definitions (from different modules), each wrapped independently into __contentValues. The unwrap function doesn't merge them.

Den fix (b7ac9930): mergeContentValuesPerKey — instead of taking one value, iterate all __contentValues entries, collect per-sub-key definitions across entries, and re-wrap multi-def sub-keys as new __contentValues lists for downstream processing. Only used when builtins.length attrVals > 1.

Library design: The three-branch dispatch eliminates __contentValues entirely. Multiple definitions at the same freeform key go through aspectType.merge, which classifies all defs and routes them to mergeAspect (for attrsets) or deferredModule.merge (for modules). aspectSubmodule.merge naturally merges multiple attrset defs via the NixOS module system — both definitions' nixos values land in the same declared or freeform option and merge through its type. No per-key unwrap step, no content loss.

Problem 10: Included nested aspects auto-walk leaks sub-aspects (b7ac9930, #521)

Setup: den.aspects.apps.dev-tools has a nested sub-aspect dev-tools.foo. Including dev-tools should only include dev-tools's own class content (nixos), not auto-walk and emit dev-tools.foo.

den.aspects.apps.dev-tools.nixos.environment.variables.DEV_TOOLS = "yes";
den.aspects.apps.dev-tools.foo.nixos.environment.variables.FOO = "yes";
den.aspects.igloo.includes = [ den.aspects.apps.dev-tools ];
# Expected: igloo gets DEV_TOOLS but NOT FOO
# Actual (before fix): igloo gets both DEV_TOOLS and FOO

Root cause: compile-static auto-walks ALL nestedKeys on every aspect it processes, including aspects that arrive via includes. When dev-tools is included in igloo, the pipeline processes it and auto-walks its nested keys — finding foo and emitting it as if it were part of the inclusion. But foo is an independent sub-aspect that should only be included when explicitly referenced.

Den fix (b7ac9930): Skip auto-walk of nested keys when the tagged aspect has __contentValues at the top level. Aspects from includes arrive as __contentValues wrappers — their nested sub-keys are independent aspects, not locally-defined nested content. Full aspects from aspectSubmodule (locally-defined) do NOT have __contentValues at the top level, so their nested keys auto-walk normally.

Library design: This problem requires a design decision about include semantics:

Include = class content only. When an aspect is included, only its class-level content (declared class keys and deferredModule values) should be emitted. Nested sub-aspects are independent aspects with their own identity — they should only be emitted when explicitly included or when they are part of the including aspect's own definition. The library's resolve.nix already handles this correctly: it extracts provided.${class} (class content) and recursively resolves provided.includes (explicit dependencies). It does NOT auto-walk freeform keys looking for more aspects to include. The auto-walk is a pipeline concern (compile-static in den), not a resolve concern.

Library constraint: resolve must never recursively descend into freeform keys of included aspects. Only includes (explicit) and the target class key are extracted. Nested sub-aspects at freeform positions are only reachable when the pipeline processes them at their definition site, not when they're included from another site. This is already the behavior of both flake-aspects' resolve and the spec's resolve — no change needed. The bug was entirely in den's compile-static pipeline handler, not in the type system or resolve algorithm. Documenting this invariant prevents future pipeline implementations from making the same mistake.

Problem 11: List-valued structural keys (includes/excludes) lost in content wrapper shallow merge (82f080c9, #524)

Setup: Multiple modules each add children to the same aspect's includes list:

# Module 2: { gloom.apps.polybar.includes = [ coreuse ]; }
# Module 3: { gloom.apps.polybar.includes = [ wifi ]; }

When igloo includes gloom.apps.polybar, only the last module's includes list survives — coreuse is silently dropped.

Root cause (two interacting bugs):

  1. aspectContentType.merge shallow-merges forwarded attrs with foldl' //. Each module's polybar definition becomes a __contentValues entry. The forwarded attr merge (foldl' (a: b: a // b) {} attrVals) is last-win per key. Module 2's includes = [coreuse] is overwritten by Module 3's includes = [wifi].

  2. mergeContentValuesPerKey (from Problem 9 fix) wraps list keys as __contentValues instead of concatenating. When the per-key reconstruction encounters multiple definitions of includes, it creates { __contentValues = defsForKey; __provider = ...; } — an attrset where a list was expected. The pipeline crashes with expected a list but found a set.

Interaction with Problem 10: Before #521, auto-walking masked the incomplete includes by resolving nested keys regardless of what was in includes. After #521 correctly suppressed auto-walking for content wrappers, the incomplete includes became the only resolution path and children from earlier modules were silently dropped.

Den fix (82f080c9):

  1. In aspectContentType.merge: collect ALL list-valued keys across definitions and concatenate them (lib.concatMap), then overlay mergedLists onto the shallow-merged forwarded before attaching __contentValues/__provider.
  2. In mergeContentValuesPerKey: when all definitions for a sub-key are lists, concatenate instead of wrapping as __contentValues.

Library design: Both bugs stem from aspectContentType's uniform wrapping that doesn't understand structural keys. The three-branch dispatch eliminates this:

  • includes is a declared option (listOf coercedAspectType) on aspectSubmodule. The NixOS module system's listOf merge naturally concatenates lists from multiple definitions. No manual list collection.
  • excludes (schema extension sidecar) is extracted with merge strategy acc ++ val — also concatenation.
  • No __contentValues wrapper, no forwarded attr shallow merge, no per-key reconstruction. Multiple modules defining aspect.polybar.includes each contribute to the listOf option through normal module merge.

This is the strongest validation of the three-branch dispatch design: an entire class of bugs around list-valued structural keys (Problems 9, 10, 11 form a cascade) is eliminated by having aspectSubmodule understand its own structure instead of wrapping everything uniformly and reconstructing it downstream.

Summary: what the library eliminates

Mechanism Purpose Eliminated by
aspectContentType Uniform wrapping of freeform values Three-branch dispatch classifies at type level
__contentValues Provenance tracking deferredModule._file + metaType.aspect-chain
__provider Provider chain tracking metaType.aspect-chain
contentUtil unwrapping Extract content from wrappers No wrappers to unwrap
providerType.merge normalization Fix wrapper leak for parametric fns parametricType produces correct output natively
wrapChild content detection Fix inner wrapper leak in includes No wrappers in includes — coercedAspectType coerces at definition time
structuralKeysSet for __contentValues/__provider Prevent wrapper metadata classification No wrapper metadata exists
isSubmoduleFn Distinguish module from parametric fns isModuleFn — broader, handles _module.args
__providesForwarded + classification skip Prevent forwarded provides from double-processing Same pattern — post-merge forwarding with classification exclusion
tagProvider in bracket resolution Inject identity onto bare forwarded attrs Not needed — forwarded provides are proper aspects with structural identity
structuralKeysSet manual extension Prevent schema extension keys from classification Sidecar extraction strips keys before merge; sidecarKeys list filters post-merge overlay
unwrapContentValuesRaw single-value extraction Extract one value from __contentValues aspectSubmodule.merge naturally merges multi-def attrsets via module system
__contentValues top-level detection for auto-walk gating Prevent included aspects from leaking sub-aspects resolve only extracts includes + target class — no auto-walk of freeform keys
foldl' // shallow merge of forwarded attrs Combine content wrapper attrs across defs aspectSubmodule declared options merge via module system (listOf concatenates)
mergeContentValuesPerKey list-key special-casing Fix list keys wrapped as attrsets No content wrappers — includes is a declared listOf option, merges natively

Identity (identity.nix)

Built into the type system, not a pipeline concern. Computed from the aspect's position in the declaration tree. In the HOAG pipeline, identity.key is the dedup key for the resolved-aspects attribute — two aspects with the same key are the same aspect regardless of how many include paths reach them.

flake-aspects' identity implements the Palmer et al. (2024) intensional function pattern, formalized and owned by den-schema's identity primitives. The key function below is equivalent to den-schema.identity.mkIdentity { name = pathKey (aspectPath a); } — the aspect's definition-site path is the program point, and the identity hash enables conservative equality (same key = same aspect, proven sound by Palmer's Theorem 5.12).

Parametric wrappers ({ __fn, __args }) are intensional functions in Palmer's sense: name + meta.aspect-chain is the program point, __args is the inspectable closure, __fn is the function. These can optionally be constructed via den-schema.identity.mkIntensional for hash-based dedup.

aspectPath = a: (a.meta.aspect-chain or []) ++ [ (a.name or "<anon>") ];

pathKey = path: lib.concatStringsSep "/" path;

key = a: pathKey (aspectPath a);

isMeaningfulName = name:
  name != "<anon>"
  && name != "<function body>"
  && !(lib.hasPrefix "[definition " name);

structuralKeysSet = lib.genAttrs [
  "name" "description" "meta" "includes" "provides"
  "__functor" "__functionArgs" "__fn" "__args" "_module" "_"
  "__providesForwarded"
] (_: true);

Sidecar keys and classification: Sidecar fields (e.g. policies, excludes) are extracted from defs before submodule merge but overlaid onto the merged result post-merge. They appear in builtins.attrNames aspect. Rather than adding each sidecar to structuralKeysSet, classifyKeys receives the sidecarKeys list (known at type construction time) and filters them alongside structural keys:

classifyKeys = { structuralKeysSet, sidecarKeys, ... }: targetClass: aspect:
  let
    sidecarSet = lib.genAttrs sidecarKeys (_: true);
    forwardedSet = lib.genAttrs (aspect.__providesForwarded or []) (_: true);
    allKeys = builtins.filter (k:
      !(structuralKeysSet ? ${k}) && !(sidecarSet ? ${k}) && !(forwardedSet ? ${k})
    ) (builtins.attrNames aspect);
  in ...;

This makes sidecarKeys the single source of truth — no manual structuralKeysSet extension needed when adding new sidecars.

Identity flows from the type system: metaType has aspect-chain set automatically by aspectType/coercedAspectType/parametricType when nesting. Each nesting level passes aspectChain = parentChain ++ [parentName] to child type configuration. Both freeform keys AND provides children receive the extended chain — critical for correct identity (see Problem 4).

The aspect's key is derived from meta.aspect-chain ++ [name]. This is stable across include paths because both components are set at definition time, not include time.

Note: __fn and __args are in structuralKeysSet because parametricType outputs include them — they must not be classified as class or nested keys.

Identity threading through freeform nesting

aspectSubmodule = lib.types.submoduleWith {
  description = "aspect";
  modules = [({ name, config, ... }: {
    # Schema modules and sidecars are inherited by all descendant aspects.
    # A nested aspect `editor.vim` accepts the same class keys (nixos,
    # homeManager) and sidecars (policies, excludes) as its parent.
    # Consumers who need per-level schemas should use separate mkAspectType
    # calls with different parameters.
    freeformType = lib.types.lazyAttrsOf (mkAspectType {
      aspectChain = aspectChain ++ [ name ];  # thread chain into children
      inherit sidecars schemaModules defaultFunctor;  # propagate to nested aspects
    });
    options.provides = lib.mkOption {
      type = lib.types.lazyAttrsOf (mkCoercedAspectType {
        aspectChain = aspectChain ++ [ name ];  # same chain threading as freeform
      });
    };
  })];
};

Freeform and provides use identical chain threading. This was confirmed correct by the den fix — the original code only threaded through provides, causing freeform children to have truncated identity paths.

Provides-to-root forwarding

Provides children must be accessible at the aspect root (aspect.dockeraspect.provides.docker). This is implemented post-merge via mergeAspect (defined in the three-branch dispatch section) to avoid infinite recursion. mergeAspect wraps aspectSubmodule.merge with provides forwarding and __providesForwarded tagging. All aspectType.merge paths that produce aspects route through mergeAspect — never through raw aspectSubmodule.merge.

Why post-merge: Setting config = ... config.provides inside the submodule causes infinite recursion — config.provides depends on the submodule config which includes the forwarding module.

Shadowing semantics: providesChildren // merged means direct freeform keys and declared options (from merged) take priority over same-named provides children. This is intentional — if an aspect declares both provides.foo and a direct foo freeform key, the direct key wins at the root level. The provides child remains accessible via aspect.provides.foo or aspect._.foo. This matches the principle that explicit definitions shadow implicit forwarding.

Why __providesForwarded: Without it, forwarded provides children appear as regular keys to classifyKeys. The pipeline classifies them as nested or class keys and processes them alongside includes that already reference the same provides children — causing double emission (see Problem 6). The classification function filters out forwarded keys:

classifyKeys = targetClass: aspect:
  let
    forwardedSet = lib.genAttrs (aspect.__providesForwarded or []) (_: true);
    allKeys = builtins.filter (k:
      !(structuralKeysSet ? ${k}) && !(forwardedSet ? ${k})
    ) (builtins.attrNames aspect);
  in ...;

Provenance

Module content (deferredModule path), parametric content (parametricType path), and aspect content (aspectSubmodule path) carry provenance through different mechanisms.

For module content (deferredModule path):

  1. Source file + option path: deferredModule sets _file including the full option path: "my-config.nix, via option aspects.vim.nixos".
  2. Function args: The original function is preserved inside the deferredModule wrapper with its functionArgs intact.
  3. Parent aspect identity: meta.aspect-chain + meta.name on the containing aspect.

For parametric content (parametricType path):

Identity and provenance are on the wrapper directly:

# aspects.apps.helix = { host, user }: { homeManager = ...; };
# becomes:
{
  name = "helix";
  meta.aspect-chain = ["apps"];
  __fn = «lambda»;
  __args = { host = false; user = false; };
  __functor = self: self.__fn;
}

The pipeline's bind handler resolves __fn with scope context. The result is a plain attrset that goes through normal aspect classification.

Single-invocation constraint: The pipeline resolves each __fn exactly once per scope. Parametric functions that need per-class invocation (e.g. scanning _<class> directories for each registered class) cannot rely on being called multiple times. Instead, they must eagerly produce a plain attrset with per-class keys at definition time — e.g. via builtins.readDir to discover classes, then lib.mapAttrs' to build the keyed result. This is a fundamental constraint of the wrapper model: parametricType defers application, it does not multiply it. See Problem 8 (import-tree regression) for the concrete case that established this rule.

Unresolvable parametric wrappers: If __args contains parameter names with no matching scope handler (e.g. { self', inputs', ... }: misclassified as parametric because self' and inputs' aren't in the isModuleFn check), the pipeline's bind handler cannot resolve the function. The pipeline must emit a diagnostic warning including the wrapper's identity (name, meta.aspect-chain) and the unresolvable parameter names, then skip the wrapper. This is preferable to silent failure — the user sees which aspect was skipped and which parameters were unexpected, enabling them to either add a scope handler or restructure the function. With schema-declared class keys, this scenario is rare: module functions at declared class positions bypass freeform dispatch entirely.

For nested aspect content (aspectSubmodule path):

Identity is directly on the aspect via meta:

# aspects.editor.foo.lsp has:
meta.aspect-chain = ["editor" "foo"]
meta.name = "lsp"
# key = "editor/foo/lsp"

No custom aspectContentType, no __contentValues wrapping. The NixOS module system's deferredModule + the library's metaType + parametricType provide complete provenance.

Why registry-based dispatch fails in the current type system

The current aspectKeyType comment says "the else branch switches to providerType" after provides removal. This was attempted during the PR #501 investigation and cannot work in the current architecture:

  1. Circular dependency: aspectKeyType.merge runs during den.aspects evaluation. If the merge dispatches on den.classes (registry lookup), and den.classes is populated by aspect evaluation, the lookup triggers infinite recursion.

  2. providerType at freeform positions triggers _ alias errors: providerTypeaspectTypeaspectSubmodule imports mkAliasOptionModule ["_"] ["provides"]. Simple freeform values produce The option '_' does not exist.

  3. __contentValues wrappers leak through providerType.merge: Content wrappers used as include elements get promoted to full aspects with the original content buried as freeform keys. (See Problem 1.)

The new library avoids all three:

  • No registry lookup (structural dispatch)
  • No providerType/_ alias (uses submoduleWith with explicit description)
  • No __contentValues wrappers (three-branch dispatch classifies natively)

Resolve (resolve.nix)

Recursive include tree expansion — the core algorithm from flake-aspects, evolved for nested aspects and identity.

resolve = class: aspect-chain: aspect:
  let
    # lib.isFunction (not builtins.isFunction) — catches __functor attrsets.
    # Aspects with __functor must be CALLED to produce their provided content,
    # not just treated as plain attrsets. This matches flake-aspects' behavior.
    provided = if lib.isFunction aspect
      then aspect { inherit class aspect-chain; }
      else aspect;

    classContent = provided.${class} or null;
    classImports =
      if classContent == null then []
      else if builtins.isAttrs classContent && classContent ? includes then
        (resolve class (aspect-chain ++ [provided.name or "<anon>"]) classContent).imports
      else
        [ classContent ];

    # Parametric wrappers (__fn/__args) are passed through as-is —
    # the pipeline's bind handler resolves them with scope context.
    childChain = aspect-chain ++ [provided.name or "<anon>"];

    # lib.isFunction (not builtins.isFunction) is used throughout because
    # __functor attrsets must be detected as callable. builtins.isFunction
    # returns false for { __functor = ... } attrsets.
    includeImports = map (inc:
      if builtins.isAttrs inc && inc ? __fn && inc ? __args then
        inc  # parametric wrapper — pipeline resolves via bind
      else if lib.isFunction inc then
        # Catches plain functions AND __functor attrsets.
        # __functor attrsets (aspects, batteries) are called via their functor
        # to produce provided content. Plain functions are called with
        # { class, aspect-chain } if they accept those args.
        let args = builtins.functionArgs inc;  # {} for __functor attrsets
        in if builtins.isAttrs inc then
          resolve class childChain inc  # __functor attrset — resolve calls it
        else if args ? class && args ? aspect-chain then
          resolve class childChain inc  # aspect-defining function
        else
          { ${class} = inc; }  # bare module function → wrap as class content
      else if builtins.isAttrs inc then
        resolve class childChain inc  # plain attrset (no __functor)
      else
        resolve class childChain inc  # fallback
    ) (provided.includes or []);
  in {
    imports = lib.flatten (classImports ++ includeImports);
  };

What resolve does NOT do (left to the pipeline):

  • Dedup — produces the raw expanded tree. Dedup by identity is downstream.
  • Policy dispatch — no effects, no enrichment, no resolve.to.
  • Context application — parametric wrappers stay as wrappers.
  • Constraints — no exclude/substitute filtering. Schema extension concern.

How the HOAG pipeline consumes resolve

resolve returns { imports = [ <module> ... ]; } — a flat list without per-element identity. The HOAG pipeline's resolved-aspects attribute needs identity on each element for dedup and exclude filtering. Den's wiring layer wraps resolve to attach identity:

# Den's expandIncludes — wraps flake-aspects resolve with identity
expandIncludes = aspect:
  let resolved = flake-aspects.resolve targetClass [] aspect;
  in map (m: {
    identity = flake-aspects.identity.key aspect;
    module = m;
  }) resolved.imports;

This is the caller's responsibility, not resolve's. resolve is a generic include-tree walker that should not assume consumers need identity on every element. The wrapping preserves resolve's generality while giving the HOAG evaluator the identity it needs for dedup (a: a.identity).

Parametric resolution: Parametric wrappers ({ __fn, __args }) pass through resolve unchanged. In the HOAG pipeline, they are resolved in a distinct resolved-content attribute that applies scope context (enriched-context) to __fn before classification. This happens between resolved-aspects and class-modules — see HOAG spec for the full attribute chain.

Schema extension point

The aspects library supports two orthogonal extension mechanisms: sidecar extraction (for aspect-level metadata that doesn't belong in the module system) and schema modules (for declaring typed options on aspects). This is informed by den-schema's sidecar pattern (mkSchemaEntryType), where extension data is extracted from definition values before deferredModule.merge — ensuring metadata never leaks into freeform keys.

Sidecar extraction

Sidecars are declared at type construction time with defaults and merge strategies. The aspectType custom merge extracts sidecar fields from each def before routing to aspectSubmodule.merge, then exposes them as attributes on the merged result.

mkAspectType = {
  aspectChain ? [],
  sidecars ? {},       # { policies = { default = []; }; excludes = { default = []; }; }
  schemaModules ? [],  # NixOS modules declaring typed options on aspects
}:
  let
    # Infer merge strategy from default type (matches den-schema pattern)
    inferMerge = name: sidecar:
      if sidecar ? merge then sidecar.merge
      else if builtins.isList sidecar.default then (acc: val: acc ++ val)
      else if builtins.isAttrs sidecar.default then (acc: val: acc // val)
      else throw "sidecar '${name}': no merge strategy — provide explicit merge";

    sidecarKeys = builtins.attrNames sidecars;

    aspectSubmodule = lib.types.submoduleWith {
      description = "aspect";
      modules = [({ name, config, ... }: {
        freeformType = lib.types.lazyAttrsOf aspectType;
        imports = [
          (lib.mkAliasOptionModule [ "_" ] [ "provides" ])
        ] ++ schemaModules;
        options.meta = ...;
        options.includes = ...;
        options.provides = ...;
      })];
    };

    aspectType = lib.types.mkOptionType {
      name = "aspectType";
      description = "aspect, module function, or parametric function";
      check = v: builtins.isAttrs v || builtins.isFunction v;
      merge = loc: defs:
        let
          # Extract sidecars from attrset defs before dispatch
          extractedSidecars = lib.mapAttrs (name: sidecar:
            let merge = inferMerge name sidecar;
            in lib.foldl' (acc: d:
              if builtins.isAttrs d.value && d.value ? ${name}
              then merge acc d.value.${name}
              else acc
            ) sidecar.default defs
          ) sidecars;

          # Strip sidecar keys from defs before submodule merge
          strippedDefs = map (d:
            if builtins.isAttrs d.value && lib.any (k: d.value ? ${k}) sidecarKeys
            then d // { value = builtins.removeAttrs d.value sidecarKeys; }
            else d
          ) defs;

          # Three-branch dispatch on stripped defs (same as before)
          ...

          # Attach sidecars to merged result
          result = mergedAspect // extractedSidecars;
        in result;
    };
  in aspectType;

Sidecar constraint: unconditional declarations only. Sidecar extraction runs on raw defs BEFORE evalModules. mkIf, mkMerge, and mkOverride wrappers are opaque thunks at this stage — the builtins.isAttrs d.value && d.value ? ${name} check cannot see through them. A definition like mkIf condition { policies = [...]; nixos = ...; } will have its policies silently missed because d.value is the mkIf wrapper, not the inner attrset. Sidecars must be declared at the top level of definition values, not inside NixOS module combinators. This matches den-schema's constraint (sidecars must be inline attrsets, not path modules). Conditional sidecar values should use lib.optional/lib.optionalAttrs at the sidecar value level: policies = lib.optional condition myPolicy; — the outer attrset is plain, the sidecar value itself is conditional.

__functor attrsets and sidecar interaction: An attrset with __functor (e.g. { __functor = ...; policies = [...]; provides = ...; }) passes the builtins.isAttrs check, so its policies sidecar is extracted and stripped before dispatch. The stripped attrset (without policies) then routes to mergeAspect via the all-attrsets path. This is correct — the __functor is preserved, provides forwarding applies, and the sidecar is accumulated separately. However, consumers should be aware that sidecar keys are removed from the value before it enters the module system.

Why sidecars, not module options: Aspect-level metadata like policies, excludes, and constraints are pipeline concerns, not module system concerns. They don't need mkOption type checking, defaults, or module merging — they need simple accumulation (list ++, attrset //). Declaring them as NixOS module options would:

  • Add them to builtins.attrNames aspect → require structuralKeysSet entries
  • Create module system overhead for data that's just collected and passed to the pipeline
  • Make the structuralKeysSet extensibility problem worse (every new extension adds keys)

Sidecar extraction avoids all three: sidecar keys are stripped from defs before they reach aspectSubmodule, so they never appear as freeform keys and never need structuralKeysSet entries. They appear only on the merged result, where classifyKeys ignores them because they're not in builtins.attrNames of the submodule output — they're overlaid post-merge.

Interaction with structuralKeysSet: Sidecars are attached to the merged result via // overlay. Since classifyKeys operates on builtins.attrNames aspect, sidecar keys ARE visible there. Two options:

  1. Add sidecar keys to structuralKeysSet — requires the library to accept extraStructuralKeys (already specified).
  2. Have classifyKeys use a sidecar-aware filter — sidecarKeys is known at type construction time and can be threaded to classification.

Option 2 is cleaner: the sidecar declaration is the single source of truth for which keys are metadata vs content.

Schema modules (declared class keys)

Schema modules are NixOS modules imported into aspectSubmodule that declare typed options. The primary use is declaring class keys as explicit deferredModule options, bypassing freeform dispatch entirely.

# Passed via schemaModules parameter:
{ lib, ... }: {
  options.nixos = lib.mkOption { type = lib.types.deferredModule; default = {}; };
  options.homeManager = lib.mkOption { type = lib.types.deferredModule; default = {}; };
}

Schema-declared class keys bypass aspectType. Values at declared options go through the option's own type, not the freeform aspectType. This means:

  • Declared class keys are explicitly typed deferredModule — no dispatch needed.
  • Only truly undeclared freeform keys go through aspectType — nested aspects, parametric functions, or untyped module content.
  • Strict mode (via den-schema's mkStrictModule pattern) errors on undeclared freeform keys entirely.

This reduces aspectType dispatch surface significantly.

Strict mode

The library can optionally import den-schema's mkStrictModule (or an equivalent) to reject undeclared freeform keys:

# den-schema pattern, directly reusable:
mkStrictModule = kind: { ... }: {
  _module.freeformType = lib.mkDefault (lib.mkOptionType {
    name = "strict";
    merge = path: _: throw ''
      STRICT MODE: "${lib.last path}" is not declared on ${kind}.
    '';
  });
};

When strict mode is active, the aspectType freeform type is overridden — undeclared keys throw at merge time rather than entering three-branch dispatch. This is injected via schemaModules, controlled by the consumer (den enables it via den.schema.aspect).

Pre-migration step: declare class keys in den's current architecture

This should land BEFORE the library migration. The schema-declared class key pattern doesn't require the new library — it can be adopted incrementally in the current architecture to eliminate the hasRecognizedSubKeys heuristic and isModuleFn/isSubmoduleFn ambiguity. Landing it first shrinks the freeform dispatch surface so the library migration only needs to handle genuinely ambiguous keys (nested aspects, parametric functions, quirk/pipe data), reducing risk.

Current state: den.schema.aspect is {}. All keys (nixos, homeManager, helix, monitoring) go through the freeform type → aspectKeyTypeaspectContentType. Classification relies on den.classes registry lookup + hasRecognizedSubKeys heuristic at runtime. This produces false positives (data keys like git.user matching den.classes.user) and can't be used in aspectKeyType.merge due to circular dependency with den.classes.

Incremental fix: Each class battery module already knows its key name. Have it declare the key as a schema option:

# modules/batteries/os-class.nix (or similar):
{
  den.schema.aspect.options.nixos = lib.mkOption {
    type = lib.types.deferredModule;
    default = {};
  };
  # ... rest of the battery
}

# modules/batteries/home-manager.nix:
{
  den.schema.aspect.options.homeManager = lib.mkOption {
    type = lib.types.deferredModule;
    default = {};
  };
}

Effect: aspect.nixos = { config, pkgs, ... }: { ... } goes through the declared option's deferredModule type — always terminal, no dispatch needed. aspect.helix = { host, user }: { ... } is NOT a declared option → goes through the freeform type → three-branch dispatch or aspectContentType (current).

What this eliminates:

  • hasRecognizedSubKeys heuristic — declared class keys never reach freeform classification
  • looksLikeClassContent guard — no false positives because class keys aren't freeform
  • The isModuleFn check at freeform positions — the only functions reaching freeform are parametric (class key functions go through deferredModule)
  • den.classes circular dependency in aspectKeyType.merge — class keys bypass freeform entirely

What remains freeform: Nested aspects (attrsets), parametric functions, quirk/pipe data. These are unambiguously non-class. The three-branch dispatch (attrset→aspect, parametric→wrapper) handles them without heuristics.

Migration path: This can land as a series of small PRs — one per class battery — with no breaking changes. Each PR adds an options.{className} declaration to den.schema.aspect. Existing code continues to work because declared options and freeform keys merge compatibly. The hasRecognizedSubKeys heuristic can be removed once all class keys are declared.

Interaction with provides forwarding: Schema-declared class keys on the aspect are declared options. Provides children forwarded via __providesForwarded are attribute-level overlays (post-merge //). These don't conflict — declared options are evaluated by the module system, forwarded provides are attribute access shortcuts. A provides child with class keys (e.g. provides.docker.nixos = { ... }) has its nixos go through deferredModule on the provides child's own submodule, not on the parent.

Transpose, Forward, Scope Factories

Carried forward from flake-aspects with minimal changes (~200 lines total).

  • default.nix — 2-level transpose. Unchanged.
  • aspects.nix — wires transpose + resolve. Unchanged.
  • forward.nix — cross-class forwarding. Unchanged.
  • new.nix — low-level scope factory. Unchanged.
  • new-scope.nix — named scope creator. Unchanged.
  • flakeModule.nix — flake-parts integration. Unchanged.

Public API (lib.nix)

{
  # Public API — stable contract
  mkAspectType{ aspectChain?, sidecars?, schemaModules?, defaultFunctor? }aspectType
  aspectsTypecnfsubmodule type (top-level container with _module.args.aspects)
  isModuleFnfnbool (module function classifier)
  identity{ aspectPath, pathKey, key, isMeaningfulName, structuralKeysSet }
  resolveclassaspect-chainaspect{ imports }
  transpose{ emit? }attrstransposed
  aspectsaspects-attrset{ transposed }
  forward{ each, fromClass, intoClass, intoPath, fromAspect }aspect
  newcallback-based scope factory
  new-scopenamed scope creator

  # Internal — accessible for testing and advanced use, not API contract
  _internal{ aspectSubmodule, coercedAspectType, parametricType, mergeAspect, functorType }
}

Testing Strategy

The library ships with its own tests:

  1. Type system tests — aspect creation, nested aspects, provenance tracking, identity computation, sidecar extraction and stripping, schema module injection, parametric function dispatch at freeform positions, strict mode rejection of undeclared keys
  2. Resolve tests — include tree expansion, parametric aspects, nested resolution, diamond dedup verification, parametric wrapper passthrough
  3. Transpose/forward tests — transposition and cross-class forwarding
  4. flake-parts integration testsflake.aspectsflake.modules wiring
  5. Regression tests — critical scenarios from the __contentValues wrapper leak:
    • Parametric function at freeform position, accessed via bracket AND bare attribute
    • Non-parametric freeform child included from multiple paths (dedup)
    • isModuleFn classification: { config, pkgs }: vs { host, user }: vs { ... }: vs { pkgs, ... }:
    • Identity stability across include paths (same content, two include chains)
    • Overlay duplication: same nixpkgs.overlays module must not appear twice
    • Multi-def parametric merge: two parametric functions at the same freeform key must coerce to { includes = [fn]; } and merge through aspectSubmodule — verify both functions land in the declared includes option, not in freeform, and produce a single aspect with two includes
    • __functor attrset at freeform key: verify builtins.isAttrs classification (not builtins.isFunction), provides forwarding via mergeAspect, _ alias, sidecar extraction from __functor attrsets with sidecar keys
    • Multi-def nested aspect key merge: two modules defining the same freeform key (e.g. aspect.base.nixos) must merge both definitions, not last-win overwrite
    • Include scoping: including an aspect must only emit its class content and explicit includes, NOT auto-include its nested sub-aspects at freeform positions
    • Multi-module includes merge: three modules each adding to aspect.polybar.includes must all contribute — list concatenation, not last-win overwrite

Design Opportunities (standalone library)

These are enhancements beyond correctness — things that would make this a great library for external consumers.

Composable aspect plugins

Two independent consumers extending sidecars currently requires manual sidecars attrset merging. A plugin pattern would be cleaner:

mkAspectType {
  plugins = [
    { sidecars.policies = { default = []; }; schemaModules = [ policyModule ]; }
    { sidecars.constraints = { default = []; }; schemaModules = [ constraintModule ]; }
  ];
}

mkAspectType composes plugins internally — merging sidecars, concatenating schemaModules. Each plugin is a self-contained extension contribution. This is the natural composition point for independent consumers (den's pipeline, third-party integrations).

Consumer test helpers

Expose test utilities matching den-schema's _internal pattern:

  • classifyValue — returns the dispatch branch ("aspect", "module", "parametric") for a value without merging. Useful for testing that user-defined functions are classified correctly.
  • mkTestAspect — creates a minimal aspect with specified class keys for property testing.
  • assertIdentity — checks identity computation: assertIdentity expected aspect → throws if key aspect != expected.

Documentation generation

Analogous to den-schema's renderDocs:

  • renderAspectDocs aspects — renders the aspect tree structure with identity paths, per-aspect class keys and their types, sidecar declarations and merge strategies.
  • _meta introspection on the aspects option — aspectNames, aspectMeta providing per-aspect information without forcing full evaluation. Follows den-schema's kindNames/kindMeta pattern.

inferMerge shared utility

The inferMerge function is identical between den-schema (entry-type.nix) and this library. Factor it into a shared utility — either a micro-library both depend on, or the aspects library imports it from den-schema. Otherwise the implementations will drift.

Error diagnostics

The check function (builtins.isAttrs v || builtins.isFunction v) produces generic nixpkgs error messages when a non-attrset, non-function value appears at a freeform position (e.g. a string). Consider wrapping with builtins.addErrorContext or a custom error in the else throw path of aspectType.merge to explain what was expected and why. The isModuleFn misclassification case (parametric wrapper with no scope handler) should produce diagnostics at the library level, not just the pipeline level — consider an optional onUnresolved callback parameter on mkAspectType.

Migration Path

For flake-aspects users

Near-drop-in upgrade:

  • Same file names, same API shape
  • new-scope, forward, resolve, transpose — same signatures
  • flakeModule.nix — same wiring
  • _module.args.aspect and _module.args.aspects — preserved

Breaking changes:

  • aspect-chain in metaType — additive, may affect attrset shape checks
  • Freeform key type is aspectType with three-branch dispatch — module functions are terminal, parametric functions are wrappers

For den (HOAG pipeline migration)

  1. Pre-migration: declare class keys — land schema-declared class keys (one PR per battery) on the current architecture. Remove hasRecognizedSubKeys once all class keys are declared. Independently valuable, de-risks step 4.
  2. Add den-schema identity primitivesmkIdentity, mkIntensional in identity.nix. ~18 lines. See den-schema integration spec.
  3. Design buildScopeGraphs bridge — specify how den-schema entity registries map to scope-engine algebraic graphs. This determines the shape of baseNodes and must be designed before scope-engine, since scope-engine's API must accommodate the bridge's needs (e.g., types parameter, synthesized node imports).
  4. Build scope-engine — the generic HOAG/RAG evaluator (~200-350 lines). Independent of den. See scope-engine spec.
  5. Develop flake-aspects at nix/lib/aspects2/ with independent tests. Wire den's schema extensions via sidecars (policies, excludes, meta.guard) and schemaModules (class key declarations).
  6. Implement buildScopeGraphs bridge — Den's wiring layer constructs algebraic scope graphs from den-schema entity registries. See den-schema integration spec.
  7. Implement den-specific attributes — the HOAG pipeline attribute definitions (resolved-aspects, resolved-content, guard-set, class-modules, received-pipes, nixos-modules, etc.). These consume flake-aspects' resolve (wrapped with identity) and scope-engine's eval.
  8. Migrate — replace the 37-handler chain with HOAG attribute evaluation. Eliminate wrapChild, providerType.merge, contentUtil, assemble-pipes.nix. With class keys already declared (step 1), only genuinely ambiguous freeform keys need the three-branch dispatch.
  9. Remove nix/lib/aspects/types.nix (454 lines) → aspects2/types.nix (~270 lines)
  10. Remove nix/lib/aspects/fx/content-util.nix — no __contentValues
  11. Extract to own repo with flake-aspects git history

Naming changes

den current new library rationale
typeCfg.providerPrefix typeCfg.aspectChain Break conflation with provides option
meta.provider meta.aspect-chain Matches flake-aspects' parameter name
aspectContentType wrapper eliminated Three-branch dispatch classifies at type level
isSubmoduleFn isModuleFn Broader: args ? config || args ? options || args == {}
providerType normalization eliminated parametricType produces correct output natively

Estimated size

File Lines (est.) Delta from flake-aspects
types.nix ~320 +150 (metaType, mkOptionType dispatch, mergeAspect, parametricType, coercedAspectType, isModuleFn, sidecar extraction, structuralKeysSet)
resolve.nix ~55 +19 (nested recursion, chain threading, parametric passthrough)
identity.nix ~40 New file
default.nix ~46 Unchanged
aspects.nix ~21 Unchanged
forward.nix ~72 Unchanged
new.nix ~19 Unchanged
new-scope.nix ~18 Unchanged
flakeModule.nix ~16 Unchanged
lib.nix ~25 +identity, parametricType export
Total ~632 ~50% growth over flake-aspects

Appendix: Den integration surface

This section documents exactly what den needs to wire into the standalone library via sidecars, schema modules, and classification hooks. The library itself ships none of this — it provides the extension points; den populates them.

Sidecars

Three sidecar channels, extracted from aspect defs before aspectSubmodule.merge:

Sidecar Default Merge Carried data
policies {} a // b Named policy functions: { policyName = { __isPolicy = true; name; fn; }; }
excludes [] a ++ b Constraint objects: exclude, substitute, filterBy from fx/constraints.nix
classes {} a // b Per-aspect class schema declarations, folded into den.classes registry

These replace the current declared options on aspectSubmodule (types.nix lines 489, 499, 518). As sidecars they never enter the module system, never appear as freeform keys, and never need structuralKeysSet entries.

Schema modules

One module declaring class keys as explicit deferredModule options, bypassing freeform dispatch:

# Built dynamically from battery registrations:
{ lib, ... }: {
  options.nixos = lib.mkOption { type = lib.types.deferredModule; default = {}; };
  options.homeManager = lib.mkOption { type = lib.types.deferredModule; default = {}; };
  options.darwin = lib.mkOption { type = lib.types.deferredModule; default = {}; };
  # one per den.classes entry — each battery declares its own
}

One module extending meta with den-specific sub-options (including guard for conditional aspects — the fourth aspect shape, added by Den, not by flake-aspects natively):

{ lib, ... }: {
  options.meta.guard = lib.mkOption {
    type = lib.types.nullOr lib.types.raw;
    default = null;
    description = ''
      Guard predicate for conditional aspects. Receives { pathSet, hasAspect }
      and returns bool. Aspect content is deferred until the guard passes.
      Used by the HOAG pipeline's guard-set attribute for monotone fixed-point
      evaluation. This is a den extension — flake-aspects has no awareness of guards.
    '';
  };
  options.meta.handleWith = lib.mkOption {
    type = lib.types.raw;
    default = null;
    description = "Override resolution handlers for subtree";
  };
  options.meta.collisionPolicy = lib.mkOption {
    type = lib.types.enum [ "error" "class-wins" "den-wins" ];
    default = "error";
    description = "Flat-form class module arg overlap policy";
  };
}

defaultFunctor override

Den overrides the default __functor with resolveAspectWith — the pipeline-aware resolution function that injects __scopeHandlers and walks the scope chain:

mkAspectType {
  defaultFunctor = resolveAspectWith;
}

Pipeline-internal attributes

Set by den's pipeline handlers during tree walk, not by user defs or the type system. These need classification filtering so classifyKeys skips them:

Key Set by Purpose
__scopeHandlers resolveAspectWith, bind, entity resolution Scope-provided entity values (host, user) for parametric resolution
__ctxId children, aspect handlers Identity disambiguation across entity contexts
__entityKind Policy dispatch, entity resolution Schema entity kind for scoped policy dispatch
__parametricResolvedArgs tagParametricResult Tracks resolved parametric args for context-dependency detection

Passed to classifyKeys via extraStructuralKeys (not sidecars — these are pipeline state, not user-declared metadata):

classifyKeys {
  structuralKeysSet = library.identity.structuralKeysSet;
  sidecarKeys = ["policies" "excludes" "classes"];
  extraStructuralKeys = ["__scopeHandlers" "__ctxId" "__entityKind" "__parametricResolvedArgs"];
}

den.quirks (pipe registry)

Not an aspect-type extension. Separate registry (lazyAttrsOf pipeSchemaType) passed to classifyKeys as pipeRegistry — a classification concern, not a type concern:

classifyKeys {
  pipeRegistry = den.quirks or {};
  # ... other params
}

den.batteries type

Same mkAspectType as aspects, rooted at a different chain:

options.den.batteries = lib.mkOption {
  type = lib.types.submodule {
    freeformType = lib.types.attrsOf (mkAspectType {
      aspectChain = ["den" "batteries"];
      inherit sidecars schemaModules defaultFunctor;
    });
  };
};
# den.provides and den._ are aliases for den.batteries

The full wiring call

denAspectType = mkAspectType {
  aspectChain = [];  # root; per-namespace prefixes set at namespace level
  sidecars = {
    policies = { default = {}; };
    excludes = { default = []; };
    classes  = { default = {}; };
  };
  schemaModules = [
    classKeyDeclarations  # nixos, homeManager, darwin as deferredModule
    metaExtensions        # handleWith, collisionPolicy
  ];
  defaultFunctor = resolveAspectWith;
};

What den no longer needs

Eliminated Lines Replacement
aspectContentType + __contentValues wrapping ~30 Three-branch dispatch classifies at type level
contentUtil (4 unwrap functions) ~79 No wrappers to unwrap
providerType.merge normalization (6-branch dispatch) ~150 mergeAspect + parametricType produce correct output natively
wrapChild content detection ~40 coercedAspectType coerces at definition time
hasRecognizedSubKeys + looksLikeClassContent heuristics ~20 Schema-declared class keys bypass freeform
isSubmoduleFn / canTake.upTo ~10 isModuleFn — broader check
tagProvider in bracket resolution ~10 Structural identity from metaType
mergeContentValuesPerKey + list-key special-casing ~50 listOf on declared options merges natively
__contentValues top-level detection for auto-walk gating ~5 resolve only extracts includes + class key
Total eliminated ~394
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment