I wanted to write a module that generates multiple systemd services and timers to scrub some zfs pools at certain intervals. The default scrub config does not support individual scrub intervals for each pool.
I want the config to look like this:
{
services.zfs-auto-scrub = {
tank = "Sat *-*-* 00:00:00";
backups = "Sat *-*-* 06:00:00";
};
}So let's define the basic structure of our module:
{ config, lib, pkgs, ... }:
with lib;
let
cfg = config.services.zfs-auto-scrub;
in
{
options.services.zfs-auto-scrub = mkOption {
description = "Set of pools to scrub, to when to scrub them";
type = types.attrsOf types.str;
default = {};
};
confg = {}; # TODO: implement
}Side note: I don't bother with an enable option for my own modules, because I comment out the import in my main config to disable a module, but feel free to add it if you're following along.
So far pretty normal, let's use mapAttrs' to generate the unit and timer for each pool:
{
# ...
config.systemd.services = (
mapAttrs'
(name: interval: nameValuePair "zfs-scrub-${name}" {
description = "ZFS scrub for pool ${name}";
after = [ "zfs-import.target" ];
serviceConfig = {
Type = "oneshot";
};
script = "${config.boot.zfs.package}/bin/zpool scrub ${name}";
})
cfg
);
config.systemd.timers = (
mapAttrs'
(name: interval: nameValuePair "zfs-scrub-${name}" {
wantedBy = [ "timers.target" ];
after = [ "multi-user.target" ];
timerConfig = {
OnCalendar = interval;
Persistent = "yes";
};
})
cfg
);
}Well, that's not so bad for this simple example, but I'm sure you can see how repetitive it gets to have to mapAttrs' for every key that you want to generate.
Enter mkMerge, it takes a list of options definitions and merges them. So we should be able to generate the units and timers individually and merge them into one at the top, right?
{
# ...
config = mkMerge (mapAttrsToList (
name: interval: {
systemd.services."zfs-scrub-${name}" = {
description = "ZFS scrub for pool ${name}";
after = [ "zfs-import.target" ];
serviceConfig = {
Type = "oneshot";
};
script = "${config.boot.zfs.package}/bin/zpool scrub ${name}";
};
systemd.timers."zfs-scrub-${name}" = {
wantedBy = [ "timers.target" ];
after = [ "multi-user.target" ];
timerConfig = {
OnCalendar = interval;
Persistent = "yes";
};
};
}
) cfg);
}Right?
building Nix...
error: infinite recursion encountered, at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:131:21
(use '--show-trace' to show detailed location information)
Guess not.
There is a quick workaround for this case, since we're only generating systemd.* options we can put our merge at the config.systemd level:
{
# ...
config.systemd = mkMerge (mapAttrsToList (
name: interval: {
services."zfs-scrub-${name}" = {
# ...
};
timers."zfs-scrub-${name}" = {
# ...
};
}
) cfg);
}(repeated options omitted for brevity)
While this works, it's really only fine for simple modules that generate options under one top level key. For example let's say we wanted to generate some users as well (doesn't fit with the example, but bare with me). If we add another option like config.users = mkMerge ...? Then we're back to square one.
What if we were to put the mkMerge's one level lower? Essentially we would want to turn a list of options like:
[ { a = 1; } { a = 2; b = 3; } ]into
{ a = mkMerge [ 1 2 ]; b = mkMerge [ 3 ]; }(imagine the integers as real options).
It seems like a complicated problem, but there's a function in lib that solves the whole thing for us, foldAttrs. The example looks exactly like our problem!
lib.attrsets.foldAttrs
(n: a: [n] ++ a) []
[
{ a = 2; b = 7; }
{ a = 3; }
{ b = 6; }
]
=> { a = [ 2 3 ]; b = [ 7 6 ]; }So all we need to do is wrap foldAttrs in a mapAttrs so we can put each list through mkMerge:
{
mkMergeTopLevel = attrs: (
mapAttrs (k: v: mkMerge v) (foldAttrs (n: a: [n] ++ a) [] attrs)
);
}Let's slap that into our module:
let
# ...
mkMergeTopLevel = attrs: (
mapAttrs (k: v: mkMerge v) (foldAttrs (n: a: [n] ++ a) [] attrs)
);
in
{
# ...
config = mkMergeTopLevel (mapAttrsToList (
name: interval: {
systemd.services."zfs-scrub-${name}" = {
# ...
};
systemd.timers."zfs-scrub-${name}" = {
# ...
};
}
) cfg);
}And...
building Nix...
error: infinite recursion encountered
(i cri)
What gives? if we take one key from the output of our function and assign it to the module, then it works fine:
{
config.systemd = (mkMergeTopLevel (...).systemd);
}So clearly what we're doing is legal, so lets explicitly pull out the option(s) that we want using getAttrs:
let
mkMergeTopLevel = names: attrs: getAttrs names (
mapAttrs (k: v: mkMerge v) (foldAttrs (n: a: [n] ++ a) [] attrs)
);
in
{
# ...
config = mkMergeTopLevel ["systemd"] (...);
}And it works! I don't really get exactly why explicitly pulling out the options that we want avoids the infinite recursion.
Obviously for this trivial example, putting the merge at the config.systemd level makes more sense, but for a more complex module it definitely helps with readability.
Something else to note is that if we wanted to define service options then we would get a recursion error, the solution in that case is to move our module's options to another top level key that we're not going to use in the config section (for example, options.custom.services.zfs-auto-scrub).
Life saver! I've been confused by this infinite recursion problem for a long time.
I think the key point here is thegetAttrscall to explicitly extract second level attribute sets.Anyways, great post! Thanks a lot!
Update: Stroke through my nonsense. I was too excited to notice the conclusion from the post.