Incremental builds are not one mechanism. In practice there are at least five reuse layers: up-to-date skipping, persistent task/action caching, in-process bundler reuse, page/route-level static-output reuse, and request-time regeneration. Systems that work well distinguish those layers explicitly instead of expecting one cache to solve every performance problem.1234567
The strongest common rule across build systems is that reuse must be a function of declared inputs, outputs, configuration, and runtime context. Gradle, MSBuild, Bazel, Pants, Nx, and Turborepo all make correctness depend on explicit dependency modeling, deterministic work, and conservative invalidation.13891011
For static site generators, the hardest problems are hidden dependencies, global fan-out from shared data, path-sensitive caches, and the temptation to blur build-time incrementality with runtime regeneration. Gatsby, Jekyll, Eleventy, Next.js ISR, and Netlify DPR each expose a different slice of that problem space.121314156167
The safest implementation strategy for a static generator is conservative static-output incrementality first: immutable build-time snapshots, a strong build fingerprint, page-level dependency snapshots, atomic state writes, explicit CI cache policy, and good debug tooling. Runtime freshness features such as ISR or DPR should be treated as a separate architecture, not phase one of the same feature.13810116167121517
- Up-to-date skipping: A task or target is skipped when its declared inputs and outputs have not changed. Gradle's
UP-TO-DATEbehavior and MSBuild'sInputs/Outputsmodel are the classic examples, and both effectively warn that poor input/output declarations either break correctness or force excess rebuilding.13 - Persistent local/remote task caching: The system hashes task inputs and restores outputs from local or remote storage. Bazel remote caching, Gradle build cache, Nx, Turborepo, and Pants all operate in this layer.2891011
- In-process bundler reuse: The tool reuses analysis state from a previous build running in the same long-lived process. Rollup's
cacheis the previous bundle'sbundle.cache, and esbuild'scontextmakes later rebuilds incremental inside one process.45 - Page/route-level static-output reuse: The generator rerenders only affected pages or restores previously generated files for unchanged routes. Gatsby incremental builds, Eleventy
--incremental, and Jekyll--incrementallive here, although all three document important limits.12131415 - Request-time regeneration: Pages are refreshed or first-rendered on demand after deployment. Next.js ISR and Netlify DPR are useful, but they solve runtime freshness and deploy-scale problems, not the same problem as “make
builditself skip unchanged work”.6167
That distinction matters because a static generator usually wants layers 1-4 first, and only later layer 5 if product requirements truly demand it.6167121415
An incremental static-generation pipeline should be layered so correctness is checked at each boundary:
Sources / content / config / env / public assets / remote data
|
v
Snapshot all build inputs
(content digests, config/env/version state)
|
v
Capture page/route dependency relationships
(templates, layouts, collections, queries, assets, paths)
|
v
Invalidation / reuse planner
- Did the global fingerprint change?
- Is full output reuse safe?
- Which pages/routes actually changed?
|
+---------+---------+
| |
v v
restore unchanged outputs render changed pages
| |
+---------+---------+
v
bundle/finalize outputs
|
v
persist outputs + state atomically
|
v
local cache / remote CI cache / build artifacts
Request-time regeneration (ISR/DPR) is a separate runtime path.
This architecture matches the official guidance from task-oriented systems that require explicit inputs and outputs, and it also reflects the documented limitations of current SSG incremental modes: if dependencies or outputs are not modeled well enough, correctness degrades quickly.138912131415
| System | What it demonstrates | Main caution |
|---|---|---|
| Gradle / MSBuild | Explicit inputs/outputs and partial rebuilds are the foundation of correctness.13 | Coarse or incomplete input/output models either miss reuse or produce wrong reuse.13 |
| Bazel / Pants | Hermetic process/action modeling makes remote cache reuse safe at scale.89 | Shared caches only help when the work is reproducible and fully declared.89 |
| Nx / Turborepo | Hash-based task replay works well when runtime context, config, env, and outputs are part of the fingerprint.1011 | Non-deterministic tasks and missing outputs make cache hits misleading or useless.1011 |
| Rollup / esbuild / webpack | Bundler-level reuse can dramatically reduce repeated work.4518 | Bundler caches are not a full site-level incremental model and can be path-sensitive.4518 |
| Gatsby / Jekyll / Eleventy | Page-level incrementality is possible, but dependency visibility limits dominate the design.12131415 | Hidden reads, shared data, includes, collections, and cold-start gaps are the hard part.131415 |
| Next.js ISR / Netlify DPR | Runtime regeneration is a real answer for large path sets and freshness requirements.6167 | It introduces distributed cache and runtime-coordination problems that build-time incrementality does not.6167 |
Gradle's docs are direct: incremental build only works if task properties that affect outputs are declared as inputs, tasks have outputs, and nondeterministic tasks do not opt in. MSBuild says the same thing in a different vocabulary: targets build incrementally only when Inputs and Outputs are specified well enough, and partial incremental work is much better when outputs map directly to inputs via transforms.13
The same principle appears in hash-based systems. Bazel actions are described by explicit inputs, output names, command line, and environment variables. Nx hashes source files, dependency files, external dependency versions, global configuration, runtime values such as the Node version, and command flags. Turborepo similarly fingerprints task inputs and outputs and misses cache when global or package-level inputs change.81011
The practical takeaway is simple: you do not add incremental builds by adding a cache directory. You add them by building a precise model of what makes an output valid.1381011
Timestamp-based models can work, but they age badly in distributed workflows. Go's build cache is a strong example of the alternative: since Go 1.10, out-of-date detection is based on source content, build flags, and build metadata rather than modification times, and the cache is explicitly intended to help when switching between branches or copies of the same source tree.19
For a static generator, the analogous move is to fingerprint input contents and dependency relationships instead of leaning primarily on mtimes. That is much more robust across CI restores, branch switches, copied workspaces, and generated inputs.181119
Rollup's cache option is the prior bundle's bundle.cache, intended to speed watch-mode rebuilds by reanalyzing only changed modules. Esbuild's incremental behavior is also tied to a long-lived context, where subsequent rebuilds reuse prior work in the same process.45
Those are useful optimizations, but they are session-local. They help repeated builds in the same process, not cold starts on CI, cache restores in fresh worktrees, or page-level static-output reuse across separate build invocations.45
Webpack's filesystem cache is a step closer to persisted reuse, but its docs explicitly note that CI should run the job in the same absolute path when sharing the filesystem cache because the cache files store absolute paths. That is a good reminder that bundler caches and generator-level reuse should be treated as separate layers.18
Bazel's remote cache is only safe when builds are reproducible; it stores action-cache metadata and content-addressed outputs so machines can reuse one another's work. Pants says its engine caches processes precisely based on inputs and sandboxes execution to keep cache keys accurate. Turborepo states the rule directly: cached tasks must be deterministic, and logs are artifacts, so whatever you print can be replayed remotely too.8910
That rule carries straight into polyglot static-generation pipelines. If the site build depends on hidden reads, current time, non-fingerprinted environment values, or mutable remote state, then shared caches will only magnify the problem.89101113
Cargo's docs explain that incremental compilation stores extra state for reuse, but the Cargo CI discussion captures the real trade-off: on blank CI builds, the bookkeeping cost often does not amortize because the next run starts from another blank cache. Turborepo makes the same economic point in task-cache terms: caching can be slower than recomputing for tiny tasks, very large artifacts, or tools with their own internal caches.202110
That means the right design is not just “fast locally.” Measure warm local rebuilds, cold CI builds, warm CI builds with restored state, and branch-switch workflows separately, because those workloads reward different layers of reuse.10202119
GitHub Actions caches are immutable, searched via exact keys plus prefix-style restore keys, and scoped so that current-branch, base-branch, and default-branch behavior matters. GitHub also warns not to store sensitive data in caches because users with read access can recover cache contents through pull requests. Azure DevOps draws the same line between immutable pipeline caches and artifacts that are required for downstream correctness, and documents branch-scoped read/write rules explicitly.2223
That means a generator's incremental-build documentation should include cache-key policy as part of the feature, not as an afterthought.2223
Gatsby's OSS incremental builds regenerate only the subset of HTML files whose inputs changed, provided the previous .cache and public directories are preserved. The tracked inputs include the page template, page-query result, static queries used by the page template, and frontend source code including browser and SSR entrypoints.12
Gatsby's debugging guide also exposes the most important SSG failure modes. Shared static queries can fan out widely, causing many pages to rebuild, and using current date/time or buildTime can trigger rebuilds you did not expect. If Gatsby detects arbitrary filesystem reads in gatsby-ssr, it disables incremental builds because it cannot safely model those dependencies.13
Jekyll says its incremental regeneration is still experimental. It tracks file modification times and only a narrow set of inter-document dependencies such as includes and layouts. The docs explicitly admit that plain references to other documents, such as iterating over site.posts, are not detected as dependencies.14
Eleventy documents a more capable local-development story: changed templates, layouts, some dependency-mapped templates, collections, and passthrough copy rules can participate in --incremental. But the same page also lists missing pieces: no cold-start incremental persistence yet, no build-server/CI incremental mode yet, and incomplete dependency mapping for several template/data features.15
The shared lesson is that SSG incrementality is limited first by dependency visibility, not by how many cache directories you create.1415
Next.js ISR updates static content without rebuilding the entire site by serving cached prerendered pages, revalidating them in the background, or regenerating them on demand. The docs also make clear that it is runtime-dependent: ISR is not supported for static export, default cache storage is per-instance filesystem storage, and multi-instance deployments need shared cache coordination or a custom cache handler to avoid inconsistent behavior.616
Netlify's DPR proposal addresses another runtime-scale problem: for very large sites, defer rendering of long-tail URLs until first request, then persist them until the next deploy to preserve the Jamstack's atomic-deploy mental model.7
Both are useful patterns, but they are not a substitute for a correct and efficient build pipeline. Treat them as a separate runtime capability layer.6167
Nx includes runtime values such as the Node version in its computation hash, and Turborepo includes configuration, lockfile state, environment declarations, and behavior-changing flags in its hash inputs. Cargo's own defaults differ between dev and release, which is another reminder that optimization profile and runtime context change the economics of incremental work.111020
For a static generator, cache keys should include generator version, build mode, runtime mode, dependency lockfiles, relevant environment inputs, and any platform/runtime boundary that can change emitted artifacts.101120
Webpack's filesystem cache is path-sensitive enough that the docs tell CI to run in the same absolute path when sharing that cache, and GitHub's cache docs also make cross-OS reuse opt-in via enableCrossOsArchive rather than the default. That is a strong signal that path normalization and platform normalization belong in your design from the beginning.1822
The build-system versions of this problem are familiar: if the same logical input is represented by different absolute paths, cache reuse becomes either fragile or incorrect. Prefer root-relative or content-addressed identifiers whenever possible above the bundler/toolchain layer.81118
Gradle's cacheable tasks span JVM languages, native compilation, tests, and code quality tasks. Bazel actions are language-agnostic so long as inputs, outputs, and commands are declared. Pants achieves cross-language reuse by putting invalidation, concurrency, caching, and remote execution in the engine rather than in each individual rule author's local policy.289
The design implication for a static generator is to let lower-level caches do their job, but keep generator-level correctness at the route/page/data/output layer where your users actually reason about change.28945
- Do not conflate build-time incrementality with runtime regeneration. Next.js ISR and Netlify DPR solve freshness and large-path deployment problems, but they add runtime cache-coordination concerns that are absent from pure build-time reuse.6167
- Do not permit hidden dependencies in incremental mode. Gatsby disables incremental builds when it sees arbitrary filesystem reads in SSR; Jekyll misses common cross-document fan-out; Eleventy still documents several dependency-mapping gaps.131415
- Do not ignore shared-data fan-out. Gatsby's shared static queries, Jekyll's
site.postsexample, and Eleventy's collections notes all show how one “small” content change can invalidate far more pages than expected.131415 - Do not rely primarily on timestamps. Content digests and normalized input models survive CI restores, branch switching, and copied workspaces better than mtimes.1319
- Do not over-cache. Turborepo explicitly warns that very fast tasks, huge artifacts, or tools with their own caching can be slower to cache than to recompute; Azure DevOps makes the same general point about restore/save cost versus regeneration cost.1023
- Do not assume a shared cache is correct just because it exists. Bazel, Pants, and Buck2 all imply or state that correctness depends on declared inputs, previous-state discipline, and explicit cleanup or sandboxing rules.8917
- Do not treat CI caches as mutable state or artifact transport. GitHub and Azure both document immutable caches and distinguish them from required artifacts.2223
- Do not ignore path sensitivity and platform boundaries. Webpack's filesystem cache and GitHub's cross-OS cache flag both show that reuse assumptions can quietly become platform-specific.1822
- Do not let nondeterministic values escape the fingerprint. Date/time inputs, mutable environment values, and undeclared flags are a direct path to false cache hits or unexpectedly broad invalidations.131011
- Do not ship the feature without explanations for misses and rerenders. Gatsby's diff-based debugging guidance and Turborepo's run summaries are a strong sign that observability is a product requirement, not a nice-to-have.1310
- Page-input tracking can deliver real wins. Gatsby's own Shopify-style example demonstrates the value of rebuilding only affected pages when the dependency model is good enough and prior outputs are preserved.12
- Scope control is a success pattern, not a weakness. Eleventy limits
--incrementalto local-development use cases today and documents its roadmap for cold-start and build-server support instead of overselling the feature.15 - Deploy-scale runtime deferral is worth treating separately. Netlify's DPR proposal came out of teams deploying hundreds of thousands of pages and is explicitly framed as a way to preserve atomic deploys while cutting front-loaded rendering cost.7
- Shared deterministic caches compound across teams. Bazel remote caching, Pants remote execution/caching, Nx replay, and Turborepo remote caching all show that correctness-first reuse has organizational payoff beyond a single developer machine.891011
- Content-aware local caching improves branch-switch and repeated-build workflows. Go's build cache is a good reminder that not all incremental value comes from CI; switching between copies or branches is a major developer-experience win too.19
The first version should focus on static-output reuse, not runtime regeneration. The SSG evidence shows that dependency visibility is already hard enough without also taking on multi-instance runtime cache coherence.6167121415
Fingerprint at least the generator version, build mode, runtime mode, normalized config, dependency lockfiles, relevant environment inputs, declared plugins/integrations, and output configuration. That follows the same core principle as Gradle, MSBuild, Nx, and Turborepo: if any input that affects output changes, reuse must stop.131011
Do not let render-time code read “live” remote data or undeclared files behind the back of the planner. Snapshot data up front, digest it, and treat it as part of the build input set, because every successful incremental system depends on explicit and hashable inputs.18101117
At minimum, model each page or route in terms of template/layout dependencies, queries or collections used, shared data sources, generated output paths, and emitted asset references. Gatsby's tracked inputs and the documented gaps in Jekyll and Eleventy together show what happens when this model is strong versus weak.12131415
Use bundler caches where available, but do not make them the only correctness mechanism. Rollup, esbuild, and webpack all prove that bundler reuse is useful, but it operates at a different layer from route/page/output reuse.4518
Buck2's incremental-actions design is explicit that previous outputs and previous-state metadata need disciplined handling, including manual cleanup and updating state only after successful output mutation. That same discipline applies to any site-level incremental state file or output manifest.17
Document branch-scoped cache keys, conservative restore-key usage, platform/runtime boundaries in keys, path expectations for filesystem caches, and the difference between optional caches and required artifacts. The GitHub and Azure docs make clear that this behavior is not incidental infrastructure detail; it is part of the feature's correctness envelope.182223
Ship a forced clean rebuild mode, a “why was this rerendered?” explanation path, and summary/debug output for cache misses and reuse decisions. Gatsby's debugging docs and Turborepo's run-summary model both point in this direction.1310
If product needs eventually require ISR- or DPR-like behavior, build that as a separate runtime capability with its own cache handler, invalidation API, and multi-instance coordination story. The evidence does not support combining that with the first version of build-time incrementality.6167
High confidence
- The core architectural principles: explicit inputs/outputs, deterministic work, conservative invalidation, and the separation of build-time reuse from runtime regeneration are strongly supported across official documentation for Gradle, MSBuild, Bazel, Pants, Nx, Turborepo, Next.js, Gatsby, Jekyll, and Eleventy.138910116121415
- The most durable pitfalls are also well-supported: hidden dependencies, path sensitivity, branch/instance cache scope, and over-caching are directly documented by the cited systems.181314152223
Moderate confidence
- The exact cost/benefit crossover between cold CI, warm CI, and warm local workflows is workload-specific. The cited systems clearly document that the economics differ, but none provides a universal threshold for when a specific static-generation task should or should not be cached.10202123
- The exact page-level dependency model needed for any given generator depends on its templating and data model. Gatsby, Jekyll, and Eleventy show the problem shape clearly, but not one universal schema.12131415
Important inference I am making
- The best near-term strategy for a static generator is conservative static-output incrementality first, with runtime regeneration later if needed. That is an inference from the cross-ecosystem evidence, not a quote from a single source, but it is strongly supported by the documented success patterns and failure modes.616712141517
Footnotes
-
Gradle incremental build docs: https://docs.gradle.org/current/userguide/incremental_build.html#sec:how_does_it_work. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14
-
Gradle build cache docs: https://docs.gradle.org/current/userguide/build_cache.html. ↩ ↩2 ↩3 ↩4
-
MSBuild incremental build docs: https://learn.microsoft.com/en-us/visualstudio/msbuild/incremental-builds?view=vs-2022 and https://learn.microsoft.com/en-us/visualstudio/msbuild/how-to-build-incrementally?view=vs-2022. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12
-
Rollup
cacheoption docs: https://rollupjs.org/configuration-options/#cache. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 -
esbuild incremental/context docs: https://esbuild.github.io/api/#incremental. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8
-
Next.js ISR docs: https://nextjs.org/docs/app/guides/incremental-static-regeneration. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14
-
Netlify DPR announcement: https://www.netlify.com/blog/2021/04/14/distributed-persistent-rendering-a-new-jamstack-approach-for-faster-builds/. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14
-
Bazel remote caching docs: https://bazel.build/remote/caching?hl=en. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14 ↩15 ↩16 ↩17 ↩18
-
Pants engine/introduction docs: https://www.pantsbuild.org/stable/docs/introduction/how-does-pants-work. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12
-
Turborepo caching docs: https://turborepo.dev/docs/crafting-your-repository/caching. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14 ↩15 ↩16 ↩17 ↩18 ↩19 ↩20 ↩21 ↩22
-
Nx caching docs: https://nx.dev/docs/concepts/how-caching-works. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14 ↩15 ↩16 ↩17
-
Gatsby v3 incremental builds release notes: https://www.gatsbyjs.com/docs/reference/release-notes/v3.0/#incremental-builds-beta. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13
-
Gatsby debugging incremental builds guide: https://www.gatsbyjs.com/docs/debugging-incremental-builds/. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14 ↩15
-
Jekyll incremental regeneration docs: https://jekyllrb.com/docs/configuration/incremental-regeneration/. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14 ↩15 ↩16
-
Eleventy incremental build docs: https://www.11ty.dev/docs/usage/incremental/. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12 ↩13 ↩14 ↩15 ↩16 ↩17 ↩18
-
Next.js self-hosting cache/ISR docs: https://nextjs.org/docs/app/guides/self-hosting#caching-and-isr. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11 ↩12
-
Buck2 incremental actions design doc: https://github.com/facebook/buck2/blob/main/docs/rule_authors/incremental_actions.md. ↩ ↩2 ↩3 ↩4 ↩5
-
webpack cache docs: https://webpack.js.org/configuration/cache/. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9
-
Go 1.10 build cache notes: https://go.dev/doc/go1.10#build. ↩ ↩2 ↩3 ↩4 ↩5
-
Cargo profiles/incremental docs: https://doc.rust-lang.org/cargo/reference/profiles.html#incremental. ↩ ↩2 ↩3 ↩4 ↩5
-
Cargo CI discussion: https://github.com/rust-lang/cargo/issues/11853. ↩ ↩2 ↩3
-
GitHub Actions dependency caching docs: https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7
-
Azure DevOps pipeline caching docs: https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7