Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save aaronschachter/181e79b8bb452f7684770a229506e007 to your computer and use it in GitHub Desktop.

Select an option

Save aaronschachter/181e79b8bb452f7684770a229506e007 to your computer and use it in GitHub Desktop.

CMSEXT-2034: Sitemap silently drops draft-changes CMS items on PG-backed sites

Bug Summary

getItemsInCollections in entrypoints/server/lib/sitemapUtils.ts builds a draftChangesPublishedItemsQuery to fetch published slugs for draft-changes items. That query is missing name in its returnFields. On PG-backed collections, the CMS storage layer's CMSItem.get name accessor calls intoNonNull, which throws BadDatabaseStateError: Expected a non-nullable string when name is undefined. The error is caught silently and logged as logger.warn, causing all draft-changes items for that collection to be dropped from the sitemap.

Fix: Add {field: {slug: 'name'}, include: true} to the live query's returnFields (PR #111007).

Impact

Metric Value
Warning logs (last 7 days) ~300,000
Unique sites affected (last 7 days) ~19,850
Unique sites affected per weekday ~8,000
Service webflow-batch-publication

Every warning means draft-changes CMS items were silently excluded from the published sitemap. Mongo-backed collections are unaffected (Mongo returns all fields by default).

Daily Breakdown

Date Occurrences Unique Sites
Jun 2 (partial) 2,386 670
Jun 3 57,913 8,191
Jun 4 54,663 7,696
Jun 5 47,619 6,897
Jun 6 (Sat) 14,108 1,699
Jun 7 (Sun) 13,485 1,750
Jun 8 53,907 7,969
Jun 9 55,772 8,189

Weekday volume is steady at 50-58K/day. Weekend dips to ~14K (fewer publishes). The pattern is stable, not growing.

Duration

Log retention is 7 days, so we can only see back to June 2. The bug has likely been present since PG-backed collections started hitting this code path. Zero occurrences were found in any earlier retained logs, confirming June 2 22:11 UTC is the start of the retention window, not necessarily the onset.

Distribution

The issue is broadly distributed across sites, not concentrated:

Site ID Occurrences (7d)
64b773371a8be924f500533b 5,064
65e877bc56ac4224ea0ba62c 4,706
63ff9bdfd376ec6456c210b5 3,359
66a59d3d4524a3df29729282 2,399
642c269c448892e2ab74e4ff 2,016
... ~19,840 more sites ...

Top 10 sites account for ~22K of the ~300K occurrences. The long tail confirms this affects any PG-backed site with draft-changes CMS items.

Stack Trace

[sitemap] Failed to get published items for draft changes items
AppError [BadDatabaseStateError]: Internal data is in an unexpected state: Expected a non-nullable string
    at intoNonNull (CMSItem.ts:298:11)
    at CMSItem.get name (CMSItem.ts:116:12)
    at getItemsInCollections (sitemapUtils.ts:537:33)
    at async buildCmsItemsUrlBlocks (sitemapUtils.ts:291:27)
    at async Object.generateSitemap (sitemapUtils.ts:123:5)
    at async <anonymous> (sitePublish.ts:309:25)

Datadog Links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment