Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save aaronschachter/5b0ae70e73091f97233f477661705034 to your computer and use it in GitHub Desktop.

Select an option

Save aaronschachter/5b0ae70e73091f97233f477661705034 to your computer and use it in GitHub Desktop.

CMS Component Usages Orphan Handling - Notes

Roope's RFC: Slack thread | Confluence design note | PR #112752

Context

cms_component_usages is a PG table that indexes which components are used in which RTF field of which CMS item (CMSAUTH-5260). Collection schemas live in Mongo, usages in PG, so they can drift. The most common reads are cross-collection ("where is component X used?"), so read-time filtering via per-result PG-to-Mongo liveness checks is too expensive.

The open decision

On the delete path, a genuine PG failure is currently swallowed (log-and-continue). The Mongo collection gets deleted anyway, and the PG rows (usage rows + item table) are orphaned permanently, with no self-healing path.

Roope proposes "authoritative table + reconciliation" and asks whether we agree, or if there's a cheaper read-side angle.

My take

Roope's framing is right. Points to make in the thread:

  1. Delete-path: propagate, don't swallow. A silent orphan means both usage rows AND the item table leak. Since this table now drives component governance ("is it safe to delete this component?"), silently wrong answers are a real product risk. Fail and retry is better than silent drift.

  2. Reconciliation is the backstop, not the primary fix. Fix the delete path first. The sweep covers edge cases (rare delete failures, the "never-edited-again collection" update-path gap), but shouldn't be the reason to keep the swallow.

  3. The cheap collection-existence read filter (step 4) is worth doing. The "where is X used" panel likely already knows the live collection set. Dropping rows whose collection_id doesn't exist is basically free (in-memory set intersection). Defense-in-depth for the window between orphan creation and sweep execution.

  4. Flag for reconciliation sweep: it needs to query Mongo for the live collection/field set. Batch and rate-limit per database so it doesn't spike Mongo read load across the fleet. The prune logic it reuses is scoped to a single collection, but the sweep iterates across all collections in a database.

Roope's proposed sequence (agreed)

  1. Observability (metrics/counters on swallowed failures)
  2. Reliable delete-path cleanup (propagate instead of swallow)
  3. Periodic reconciliation sweep (reuses existing prune logic)
  4. Optional cheap collection-existence read filter
  5. Internal remediation endpoint (admin panel button, last)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment