Skip to content

Instantly share code, notes, and snippets.

@toraritte
Created October 3, 2021 15:30
Show Gist options
  • Save toraritte/78b9646f2c1d7005751ac420780a6ba3 to your computer and use it in GitHub Desktop.
Save toraritte/78b9646f2c1d7005751ac420780a6ba3 to your computer and use it in GitHub Desktop.

Had this Stackoverflow question with little activity, but have no clue why I didn't ask it here in the first place...

I think I found the best definition in the flox documentation:

Within channels, jobsets allow flox to build packages against multiple versions of dependencies simultaneously.

Here's the flox announcement on NixOS discourse, and a description of what it is.

If I understand it correctly,

  • a package's Nix expression specifies the build attributes (source location, branch to use, etc.) and prescribes how to build it

  • the jobset specifies the versions of the dependencies the project should be built against

where package ≠ project (i.e., a project comprises many packages, and jobsets are uniformly applied to all of them).

Note to self: jobset specification applies to every package in a channel.

Meaning that

  • stable jobsets take as inputs the latest stable releases of all the dependencies of the channel's packages (e.g., from their release branch)

  • staging / unstable jobsets take the latest revision of a development branch (e.g., master / main) of the dependencies (a.k.a., bleeding edge)

and so on.

To belabor the point, this would allow building combinations (or testing whether they would build) such as

+---------+---------+------------+
|\  jobset|         |            |
| \_____  | STABLE  |  STAGING   |
|       \ |         |            |
|source  \|         |            |
+---------+---------+------------+
| main    |   1     |     3      |
| HEAD    |         |            |
+---------+---------+------------+
| release |   2     |     4      |
| HEAD    |         |            |
+---------+---------+------------+

edit: Am I confusing jobsets with channels? From the Differences between Nix channels thread:

All the “unstable” channels are a snapshot of master branch at a certain point of time, all the “stable” channels are a snapshot of release-*.* branch.


Some lingering questions if the above is correct:

  • What does a jobset specification look like? How does it enforce the same level of the dependencies? Do each dependencies have to have the same branch-naming system in place to be able to target them?

For completeness' sake, here are the other definitions I've found:

  • Eelco Dolstra, Eelco Visser: "Hydra: A Declarative Approach to Continuous Integration"

    a specification of the location of a Nix expression, along with possible values for the function arguments of the jobs defined by the Nix expression.

  • NixOS Wiki: "Hydra"

    Job Set

    A list of jobs which will be run. Often a Jobset fits to a certain branch (master, staging, stable). A jobset is defined by its inputs and will trigger if these inputs change, e.g. like a new commit onto a branch is added. Job sets may depend on each other

  • Hydra User's Guide

    3.2 Job Sets

    A project can consist of multiple job sets (hereafter jobsets), separate tasks that can be built separately, but may depend on each other (without cyclic dependencies, of course).


Scratchpad

In the context of hydra.nixos.org1, these statements seem to be true:

[1]: This distinction is important because the terms below refer to slightly different things in the context of the flox build farm, Storehouse (and their relationship with channels also differs).

  • a project is a git (most often a GitHub) repo

  • a jobset is indeed a set of jobs, but most importantly, it is a unit of control applying constraints on the input dependencies (mostly by specifying the git branch of the input git repo - pushing a commit will trigger an evaluation).

    A jobset is purely a Hydra configuration construct: it is created on the Hydra side (clicking through the UI or adding a config file I presume), and it does not correspond to a Nix expression (neither file nor set of statements).

  • jobs are declared in the repo's release.nix2 or automatically determined from the flake.nix3, if present. Many times they are the Cartesian product of concrete build expression(s) and the list of supported architectures (e.g., x86_64-linux) but jobs can really be anything: a built release artifact, tests, etc.; for the best(?) example, see footnote [2] below.

    [2]: cf. disnix:disnixos-trunk's jobs & svanderburg/disnixos:release.nix's jobs attribute [3]: cf. dwarffs:master's jobs & edolstra/dwarffs:flake.nix

Not sure where channels and Hydra meet yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment