MVP-912: Directed Graph backend design scoping (Bookmarks + People Who Liked)

20260815-mvp-912-directed-graph-backend-for-bookmarks-and-people-who-liked.md

MVP-912 — Directed Graph Backend for Bookmarks and People Who Liked (Design Only)

Jira: MVP-912 (In Progress) — blocks MVP-913 (FE: Bookmarks off EVIL Modal onto DG) and MVP-914 (FE: People Who Liked onto DG, stacked not replace). Both blocked tickets are Backlog, frontend-only, out of scope here. Status: Design only — not implemented. Written to scope the architecture before committing engineering time; see "Open Decision" at the end.

Related: MVP-870 Tree backend spec, MVP-870 phase 2 grid spec, Confluence Directed Graphs (Trees) — alternative to Algo (draft, architecture vision only — not a spec, see "What the Confluence doc actually says" below). Also related, added 2026-08-16: MVP-893 ("Algo upgrades for 'soon'," Review) and MPD-311 ("SGC-PubMed. New infographic 'Dig Deeper'," Idea/Parking lot) — see §3b.

1. The ask, and why it doesn't fit the existing Tree system

MVP-912's description: "a shared Directed Graph backend so that Bookmarks and People Who Liked can both be served graph-shaped data (current node plus adjacent nodes) instead of running through EVIL Modals... shared infrastructure, not a Bookmarks-only build." Scope also names future work: Bookmarks category-organization, sharing/export/import, DD Pro vs DD Consumer bookmark types.

The repo already has a Directed Graph system — app/trees/ (MVP-870) — and Harold's only comment on the ticket says exactly that: "Directed Graph (Tree) should be used for both Bookmark fixes and Circles, and many others in the future." The natural first instinct is "extend Trees." A full read of app/trees/models.py, service.py, router.py, admin_router.py, and sheet_import.py shows why that's not a drop-in fit:

• Every Tree row is global. Tree.created_by_user_id is provenance (who authored it), not a viewer scope — once status='live', every user sees the identical structure. There is no profile_id anywhere in trees/tree_nodes/tree_node_edges.
• Every write path assumes a human authors the graph ahead of time — the admin API (admin_router.py, gated by require_tree_manager) or a bulk .ods/CSV import (sheet_import.py, driven by scripts/import_tree_sheet.py). Nothing in app/trees/ programmatically builds a tree from runtime data.
• Zero per-user state exists anywhere in the service layer. nodes_page() (the closest thing to "serve a graph") does offset-cursor pagination over a fixed node list; the client computes left/right/up/down adjacency itself from (column_index, position). There's no "give me this user's current position" call, because there's no such thing as a user-specific position in a structure everyone shares identically.
• "People Who Liked This Also Liked" doesn't exist in this codebase at all — not as a broken feature, not as a spec, not as a query. Grepped exhaustively (app/, specs/, git history for "912", "EVIL Modal"). It's a genuinely new feature; whatever reliability problems the ticket's "old original request" section describes live in a different repo (frontend) or predate this codebase's history.

Both target features are inherently personal or computed, not curated: a Bookmarks graph is one specific user's saved posts in an order only they control; a "People Who Liked" graph is computed per-post from reaction data. Neither is "a human authors a fixed structure ahead of time," which is the one invariant every trees/tree_nodes row and every admin/import write path assumes. Bolting a profile_id onto Tree to cover this would touch the schema every existing Circle/pharma/daily-dash/influencer tree and the sheet-importer depend on, for a use case that isn't actually shared-graph-shaped in the first place.

2. What the Confluence doc actually says (and doesn't)

The linked Confluence page is a draft architecture-vision doc (Gemini-assisted product brainstorming, not reviewed/approved), useful for the response shape it argues for but not a concrete spec. Its actual recommendation: "the backend doesn't just send Post 5, it sends Post 5 AND the metadata/media URLs for the adjacent nodes" — i.e. a server response that resolves neighbor nodes inline, so the frontend can prefetch without a second round trip. That's a different contract than the existing Tree system's nodes_page(), which returns a flat page and lets the client resolve adjacency from grid position plus explicit nav overrides. The doc also discusses a "spine and ribs" pattern for graphs at 500+ nodes (mostly linear with occasional branches) — directly applicable to a Bookmarks list (linear, occasionally organized into categories) and not at all applicable to "People Who Liked" (small, fixed-size, no branching).

Implication for this spec: even setting aside the ownership/curation mismatch above, MVP-912's own reference doc argues for a node+resolved-neighbors response shape, not a re-use of TreeNodesPageOut's page-plus-nav-override shape. The two features likely want a similar looking but distinct contract from Trees, not the literal same endpoint.

3. Recommended shape (design only — not implemented)

Two new, independent, read-only surfaces. Neither touches trees/tree_nodes/tree_node_edges. Both return a small, Tree-flavored response (title, description/subtitle, an ordered list of lightweight post nodes with prev/next resolved inline) so the frontend can reuse visual/gesture components built for Trees without the backend pretending these are Tree rows.

3a. Bookmarks-as-DG

• Reuses bookmarks (app/db/models.py) as-is for "which posts are bookmarked" — no schema change needed for the MVP-913 ask (linear swipe through your own bookmarks, no modal).
• New endpoint, e.g. GET /v1/bookmarks/graph?cursor=&limit= — same auth as today's GET /v1/bookmarks (@require_auth() + profile-ownership check, app/sherpahealthy/bookmarks.py), same ORDER BY bookmarks.created_at DESC, but response shape resolves prev_post_id/next_post_id inline per item instead of leaving pagination purely cursor-based — matching the Confluence doc's "send the neighbor, not just the page" argument.
• Category-organization (explicitly named as future scope in the ticket) needs a real schema addition: a bookmark_categories table (id, profile_id, name, position) and either a category_id column on bookmarks or a bookmark_category_items join table if a bookmark can belong to more than one category (the ticket's own example — "some in 2 categories" — says it must be many-to-many). Not designed further here; flag as its own follow-up spec once MVP-913 needs it.
• Sharing/export/import, DD Pro vs DD Consumer bookmark types: both explicitly named as future scope in the ticket description, neither has enough detail yet to design (no UX reference, no existing DD Pro/Consumer distinction on Bookmark today). Flag as open, not designed here.

3b. "People Who Liked This Also Liked" — computed graph, with a pinned prefix

New requirement, added 2026-08-16 from MVP-893's latest comment (Harold, 2026-08-15) and MPD-311. MVP-893 shipped a mechanism to pin an SGC-PubMed post's "sibling" (Type 1 ↔ Type 2 — the same abstract rendered as two different infographic styles) so the sibling surfaces on the next fresh feed load after a more_like_this swipe. Harold's verdict after prod/iOS testing: "It may actually be working, but I now realize that 'next fresh feed load' is not helpful to the user. We should abandon this approach and move to the DG/circle approach: put 'the other type' as the first post in People Who Liked This Liked That." MPD-311 (Idea, Parking lot) spells out the target ordering precisely: Type 1 → Type 2 → Type 3 → Type 4 → all other People Who Liked, with "the existing post Type... removed from the list" (i.e. don't show the anchor post's own type again). Only Type 1/Type 2 exist today; Type 3 (longer PubMed-abstract-only infographic, 12th-grade–college reading level) and Type 4 (needs the full publication, not just the abstract) are explicitly deferred — "No spec until 2027-2028" for Type 4, Type 3 gated on Tranche 2/3 funding. Only Type 1↔Type 2 pinning is in scope for MVP-912/914; Type 3/4 are out of scope.

This changes the query from a single computed rank to pinned prefix + organic ranking:

1. Pinned prefix (0 or 1 item today): if the anchor post is an SGC-PubMed Type 1 or Type 2 post, resolve its sibling and place it first, unconditionally, regardless of its reaction-overlap rank. Pairing already exists and needs no new column: MVP-893's own sibling-pin mechanism resolves it via find_diagram_siblings() (app/sherpahealthy/posts/feed_adapters.py:528) — a plain slug-pattern match (<base>-diagram-N, siblings share everything before -diagram-), not a stored FK. Reuse this function directly rather than re-deriving the relationship. Not applicable to non-PubMed posts (most of the catalog, no -diagram-N suffix) — prefix is empty for those, falls straight through to organic ranking.
2. Organic ranking (everything else): the reaction-overlap query described below, excluding the pinned sibling from its own results so it doesn't appear twice.

Original organic-ranking design, unchanged: self-join post_reactions (app/db/models.py:1852, kind ∈ like|dislike|love|sad|angry|surprised — scope to kind='like' to match the feature's literal name; broadening to a positive-reaction set is a config decision, not an architecture one): find profiles who reacted like to the anchor post, find other posts those same profiles reacted like to, rank by frequency (how many shared likers), tie-break by recency.

• New endpoint, e.g. GET /v1/posts/{post_id}/also-liked/graph?limit=. No new tables required for a first cut — this is a read-time aggregate query, not a materialized structure. Cost/latency of that live aggregate at scale is unverified (no reaction-volume figures gathered for this spec) — worth a quick EXPLAIN against prod's actual post_reactions row count before committing to "always compute live" vs. a periodic materialized-view refresh, the same way related_posts already has both a live (pgvector) and precomputed path (specs/20260619-pgvector-semantic-search-and-related-posts-plan.md).
• MVP-914 asks for "stacked, not replace" navigation (clicking a related post stacks on top of what you're viewing, doesn't lose your place) — that's a frontend state-management concern the backend response shape doesn't need to solve; the API just needs to return enough of each related post to render a stacked card (same ShortPostModel-shaped fields the existing /v1/full_posts/{slug}/related endpoint already returns).
• MVP-893 follow-up: once this endpoint exists and the sibling-pin moves here, the more_like_this-triggered "pin sibling for next fresh feed load" mechanism in MVP-893 becomes redundant and should be removed (not just left as dead code) — that cleanup belongs to whichever ticket implements this section, flag it there rather than doing it speculatively here.

3c. Search Results as DG — added 2026-08-28, per Harold's ticket-description edit

Design only, not implemented. Harold edited MVP-912's own description to read "a shared Directed Graph backend so that Bookmarks, Search Results, and People Who Liked can both be served graph-shaped data" — Search Results wasn't in the description when §3a/§3b were designed and built. This section scopes it the same way those two were scoped before any code was written.

Existing surfaces to wrap (app/sherpahealthy/search.py, all today return PaginatedList[ShortPostModel], all public/no-auth):

Endpoint	Ranking	Cursor shape
GET /v1/search/	Keyword-first hybrid: weighted full-text (title/tags/source/content), semantic (pgvector) fill on page 1 when POST_EMBEDDINGS_ENABLED	Opaque encoded exclude-list (cursor) for hybrid scroll pages 2+; created_at (since) for the ILIKE-only fallback path
GET /v1/search/interests	Tag-name match count desc	Compound: (match_count, created_at, id) via since=<post_id>, reconstructed server-side each call
GET /v1/search/symptoms	Same shape as interests (same tag system, different label)	Same compound cursor
GET /v1/search/tags	Tag-name match count desc, simpler ordering	since=<post_id>, plain id < filter

Harold's own phrasing ("the new Search results") most likely means the primary keyword bar (GET /v1/search/) — that's this section's v1 target. The other three share enough shape that the same wrapper trivially extends to them as a fast-follow (see "Recommended shape" below) — not required for v1, flagged so it isn't forgotten.

Key design difference from §3a/§3b: no cross-page boundary lookahead. §3a's /bookmarks/graph fetches one extra row on each side of the requested page so every node's prev_post_id/ next_post_id resolves even at a page edge — cheap there because bookmarks have one stable, replayable total order (ORDER BY created_at DESC, id DESC). Search doesn't have that:

• The hybrid path's page-1 semantic fill is not a deterministic re-runnable ORDER BY — re-querying "the row just before the requested window" means re-running the whole hybrid computation (full-text rank + a live Bedrock/pgvector call) for a throwaway row. Doubles the most expensive part of the request for zero user-visible benefit.
• The interests/symptoms compound cursor is already a HAVING-reconstructed comparison per call; deriving "one row earlier" is more of the same expensive machinery for a value nobody asked for (a user scrolling forward through search results doesn't need to know what came before the page they're on — they can scroll up).

Recommendation: resolve prev_post_id/next_post_id only within the already-fetched page (first node's prev_post_id=None, last node's next_post_id=None) and keep page-to-page continuation exactly as it works today, via the existing next cursor at the envelope level (not per-node). This still satisfies the actual ask — "current node plus adjacent nodes" for anything the user has already loaded, prefetchable without a second round trip — without inventing a new, expensive boundary computation that search's ranking model can't support as cheaply as bookmarks' can.

New endpoint, not a response-shape change to the existing four: GET /v1/search/graph — identical query params to GET /v1/search/ (query, since, cursor, limit), calls the exact same underlying search logic (_hybrid_search, _build_search_filters, etc. — zero query-logic duplication) and wraps the resulting List[ShortPostModel] + next-cursor into the shared graph envelope (§4). Additive, matching §3a's pattern: /v1/search/ stays untouched for existing consumers (this is a live, publicly-used endpoint — a breaking response-shape change isn't safe), /v1/search/graph is new.

Analytics: the existing four endpoints all call analytics.capture(..., event="post_searched", ...). The graph variant should do the same, not skip it — same event, mode property distinguishes it (e.g. "hybrid_graph").

Non-goals for v1:

• /interests/graph, /symptoms/graph, /tags/graph — same wrapper, trivial fast-follow, not built until asked for.
• Any change to /v1/search/'s existing response shape or behavior.
• Cross-page prev/next (see above) — deliberately deferred, not a gap to fill later without a concrete ask; today's per-node prev/next covers everything actually prefetchable in one response.

4. What's genuinely shared between the two — now three — surfaces

All three surfaces want the same response envelope shape (ordered nodes, each carrying enough post data to render + resolved prev/next), so that's worth factoring into one shared Pydantic schema/serializer (e.g. app/services/dg_response.py or similar) even though the three queries feeding it are unrelated (one reads bookmarks, one computes from post_reactions, one wraps the existing search functions). That shared-serialization layer is the actual "shared Directed Graph backend" the ticket asks for — not a shared table, a shared response contract multiple different data sources feed into. §3a and §3b already independently defined their own node schemas (BookmarkGraphNodeOut, AlsoLikedNodeOut) before this was written down as a principle — worth consolidating into one generic node/envelope pair when §3c is implemented, rather than adding a third bespoke pair.

Open Decision

This spec stops at "here's the shape, here's why Trees don't fit, here's the two new surfaces" — deliberately not implemented yet (per direction: scope only, hold the code). Before implementation starts, worth confirming:

1. RESOLVED 2026-08-24, per direction — see Update below. V1 is a linear bookmark list only: no categories, no sharing. Both stay explicitly deferred past v1.
2. Is kind='like' the right scope for "People Who Liked," or should it include love too?
3. Live-aggregate vs. precomputed for the "also liked" query — needs a real post_reactions row count from prod to decide, not a guess.
4. Confirmed, not open: the Type 1↔Type 2 pinned prefix (§3b) is a direct, explicit ask from Harold's 2026-08-15 comment + MPD-311 — not a guess on my part. Still open: whether MVP-913/914 want this shipped in the same PR as the base "People Who Liked" endpoint, or as a fast-follow once the base query is live. Also confirm whether removing MVP-893's now-redundant FEED_SIBLING_INSERT_ENABLED pin-on-next-load mechanism happens in the same change or waits until the DG replacement is confirmed working (safer: leave the flag in place, off, until the new path is validated in prod).
5. Confirm with Matheus/Harold whether the frontend genuinely wants to reuse Tree-rendering components for these two screens (which would argue for keeping the response shape closer to TreeNodeOut) or is building bespoke UI for each (which frees the backend to use whatever shape is most natural for each query).

Update 2026-08-23: ticket comment thread since this spec was gisted

Read the full comment thread on MVP-912 (5 comments). Net effect: confirms this spec's core call, doesn't change it — but sharpens two of the "Open Decision" items above with concrete detail from Harold that's worth capturing before implementation starts.

Wajahat (2026-08-22), directly endorsing §1's recommendation: "Agreed with Aldrin here. Our current infrastructure does not support what these new features need, and making it fit would mean changing the existing Tree structure that other features already depend on. Better to keep Trees as they are and build these separately." This is exactly this spec's §1 conclusion (don't extend app/trees/, build two new read-only surfaces) — a second independent read landing on the same architecture call, not new information, but worth recording as confirmed-not-contested.

Harold (2026-08-22), a longer clarifying comment, walking through both features by example. Two things it adds, both already-deferred items in this spec rather than new requirements:

1. "People Who Liked" example ("Post 11001 includes People Who Liked 6540, 12543, 250202, 38000... programmatically constructed") is a concrete worked instance of exactly §3b's reaction-overlap design — a post, and a small ordered list of computed related posts. Confirms the shape, doesn't change it. His framing ("why is this not a straightforward instance of a DG, programmatically determined") reads as a terminology gap, not a design disagreement: he's describing a DG-shaped API contract (nodes + resolved adjacency), which this spec already commits to (§2, §4) — the thing it argues against is storing this in the same tables as curated Trees, which is a different axis he isn't actually asking for.
2. Bookmarks example, with a named category and sharing: "If the User starts to look at Bookmarks and opens 'People', the API would pre-fetch all 5 of the Posts... If the User shares their People Bookmarks, when User2 opens 'People', it would pre-fetch the 5 posts." This is the first concrete illustration of §3a's two explicitly-deferred items — bookmark categories (a category literally named "People") and sharing (User2 viewing User1's shared category). Both were flagged in §3a as "future scope... not designed further here" and in Open Decision #1 as possibly deferrable past v1. Harold's example treats a named category + sharing as central to how he pictures the feature working, not a someday-extra — this shifts Open Decision #1's answer: worth confirming directly with Harold whether v1 genuinely ships without categories/sharing (linear bookmark list only) or whether a first cut needs at least one named category + a share mechanism to feel like what he's describing. Still not designed here (schema for bookmark_categories + sharing is real, undesigned work — see §3a) — flagging the priority signal, not doing the design, per the standing "hold the code" instruction.

No change to §3a/§3b's actual recommended shape. No new architectural surface introduced. The "nearest 20 nodes" pre-fetch-on-view behavior Harold describes for Circles/DG navigation is the existing Tree/Circle grid system's own prefetch contract (MVP-870 phase-2 grid spec), not new scope for this ticket — already consistent with §2's "resolve neighbors inline" argument.

Update 2026-08-24: Open Decision #1 resolved — V1 ships without categories/sharing

Per direction: V1 = linear bookmark list only. No bookmark_categories table, no category-membership join, no sharing mechanism — none of it in the first cut. This directly resolves the question the 2026-08-23 update raised (Harold's "People" category + sharing example suggested those might be closer to core than "eventually").

Practical effect on §3a: the endpoint design as already written (GET /v1/bookmarks/graph, reusing the existing bookmarks table as-is, prev_post_id/next_post_id resolved inline) is now the complete v1 scope, not a placeholder pending a categories/sharing decision. Nothing else in this spec changes — §3a's "Category-organization" and "Sharing/export/import" bullets stay exactly as written (explicitly deferred, undesigned), now with a firm decision behind the deferral instead of an open question.

With this resolved, all five Open Decision items are closed or intentionally left as implementation-time calls (Comments 2/3/5 don't block starting; Comment 4's PR-sequencing question is an implementation-time call, not a design blocker). This spec is no longer blocked on product input — the remaining step is an implementation PR, not more scoping, whenever that's greenlit.

Update 2026-08-26: both surfaces implemented

§3a shipped (PR #150, merged to develop/master): GET /v1/bookmarks/graph exactly as designed above — reuses bookmarks as-is, prev_post_id/next_post_id resolved inline via a one-row lookahead/lookbehind window fetch around the requested page.

§3b shipped: GET /v1/posts/{post_id}/also-liked/graph — pinned Type 1↔Type 2 prefix via find_diagram_siblings() + organic reaction-overlap ranking (shared kind='like' reactors, tie-broken by recency), per Open Decision items #2/#4's defaults (this spec's own recommendation of scoping to kind='like', pin shipped in the same endpoint rather than a fast-follow). Live-aggregate, no materialized view (Open Decision #3) — see below for why that's fine for now.

Important finding, worth flagging explicitly: before building §3b, checked post_reactions row counts directly against dev/staging/prod. All three are zero rows — no one has reacted to a post in any environment yet. This makes Open Decision #3 (live vs. precomputed) moot for the moment (0 rows is fast either way) but means the organic-ranking half of /also-liked/graph returns nothing today for any post; only the Type 1↔Type 2 pinned-sibling prefix (independent of reaction data, pure slug matching) currently populates results. Flagged to the user before building; decision was to ship the full endpoint anyway — it's correct infra ready for MVP-914's frontend today, and the pinned-prefix half is useful immediately for the PubMed catalog. Revisit the live-vs-precomputed question once real reaction volume exists.

MVP-893's FEED_SIBLING_INSERT_ENABLED pin-on-next-load mechanism (superseded by this endpoint per §3b's original note) has not been removed as part of this change — left in place per the spec's own stated caution ("safer: leave the flag in place, off, until the new path is validated in prod").

Update 2026-08-28: §3c added — Search Results, scoping only

Harold edited MVP-912's own ticket description to add Search Results to the original ask ("Bookmarks, Search Results, and People Who Liked"), then separately asked (ticket follow-up questions, same day) what's implemented, where to see examples, what happened to the "PWL" section at the bottom of regular posts, and flagged Search-as-DG as high priority. §3c above answers the design half of that: how Search Results would fit this same pattern, why its prev/next scope is narrower than §3a/§3b's (page-local only, no cross-page boundary fetch), and why it should be a new /v1/search/graph endpoint rather than a breaking change to the four existing (live, publicly consumed) search endpoints.

Status: design only, matching §3a/§3b's own precedent before they were built. No code, no PR, no migration — this section exists so implementation can start from an agreed shape rather than a guess, the same reasoning that applied when §3a/§3b were first written up. Reply to Harold's questions (posted on the ticket) pointed here for the Search-as-DG scoping and confirmed the other four answers (what's live, where to see it, the "PWL"/"Related posts" clarification, and the post_reactions-is-empty caveat) against the actual shipped code rather than the original design draft.