detailed.md

Let me check what logs are available on the server.

Let me pull comprehensive data in parallel.

Let me also see the earlier UI activity and any pre-16:00 API activity by this user.

Let me also check the solver errors around the Create Schedule runs.

I have enough now. Let me put together the final table. Quick sanity check on the start-of-day baseline first.

Here's the detailed timeline. All times are 2026-05-17. The user is user_3Dg5l4JWmt6gY3yrHGcaaqlGK5M on program 222 (Kentucky Final). EDT = UTC − 4.

Starting state (carried over from 2026-05-16)

Main schedule was 12947 "Med Peds UTD" (550 overrides), accepted by the user at 22:06 UTC the previous day.

Morning glance

UTC	EDT	Action	Result	Notes
10:59:41	06:59	UI: GET /222/schedule	—	Browser nav, settles in for the day
10:59:44	06:59	API: GET /programs/222/schedule	200	Schedule payload fetched

Then 5h45m idle.

Session 1 — Save Freezes burst (no Create Schedule yet)

UTC	EDT	Action	Schedule written	Δ ovr (running)	Notes
16:42:11	12:42	UI: /222/schedule re-opened	—	—	Back in the app
16:42:30	12:42	API: GET /programs/222/schedule	(main = 12947)	—
16:45:25	12:45	PUT /overrides/222	→ 12948	+1 (551)	Save Freezes #1
16:47:57	12:47	PUT /overrides/222	→ 12949	+1 (552)	Save Freezes #2
16:49:20	12:49	PUT /overrides/222	→ 12950	+1 (553)	Save Freezes #3
16:51:37	12:51	PUT /overrides/222	→ 12951	+2 (555)	Save Freezes #4
16:58:34	12:58	PUT /overrides/222	→ 12952	−6 (549)	Save Freezes #5 — bulk unfreeze of 6 entries
17:23:20	13:23	PUT /overrides/222	→ 12953	+1 (550)	Save Freezes #6

Session 2 — A 499 + the "phantom" schedule

UTC	EDT	Action	Schedule written	Δ ovr	Notes
17:27:06	13:27	PUT /overrides/222 → 499	→ 12954	+4 (554)	Client closed connection before nginx responded; server still committed. Toast on the client probably never fired.
17:27:09	13:27	API: GET /schedules/12954 (×3)	—	—	SSE picked up the orphan, client fetched it
17:27:16	13:27	PUT /overrides/222	→ 12955	0 (554)	User retried (10s after the 499) — same overrides re-saved
17:30:00	13:30	PUT /overrides/222	→ 12956	+1 (555)
17:31:20	13:31	PUT /overrides/222	→ 12957	+1 (556)
17:51:51	13:51	PUT /overrides/222	→ 12958	+1 / −20 (537)	Bulk wipe of all overrides for Cole Porter F (10) + Matthew Hall W (10); +1 for Nicole Oliashirazi F → Float (UL) wk 49. Probably a right-click "unfreeze resident".
17:54:38	13:54	PUT /overrides/222	→ 12959	+1 (538)
17:56:12	13:56	PUT /overrides/222	→ 12960	+1 (539)

Session 3 — First Create Schedule pair

UTC	EDT	Action	Schedule written	Δ ovr	Notes
17:59:04	13:59	POST /programs/222/schedule	202 (queued)	—	Create Schedule #1 (reference = 12960)
17:59:09	13:59	Solver log: update_schedule_from_existing warnings	—	—	"Newborn Nursery (PGY1) cannot staff PGY 2", "Newborn Nursery (UL) cannot staff PGY 4", "Wards (PGY1) cannot staff PGY 2", "Advocacy (UL) cannot staff PGY 1", etc. — reference schedule contains earlier force-placements
17:59:10	13:59	Solver log: Overrides caused the following issues that were allowed to proceed	—	—	Override clean-up step let them through
18:02:33	14:02	UI: hover-prefetch nav	—	—
18:03:00	14:03	API: GET /programs/222/schedule	(main still 12960)	—	First solver hasn't finished
18:03:10	14:03	POST /programs/222/schedule	202 (queued)	—	Create Schedule #2 — user clicked it again 4 min into the first run
18:03:14	14:03	Solver log: same warnings for run #2	—	—
18:08:11	14:08	Solver done → 12961 written	parent = 12960	+38 / −8 (569)	Run #1 result. Named "Float Fully staffed" later by user.
18:08:15	14:08	SSE new_schedule for 12961	—	—	Client stages it
18:09:39	14:09:39	PATCH /programs/222/schedule = 12961	main = 12961	—	User accepted 12961 from diff view. acceptStagedSchedule clears pendingOverrides in localStorage.
18:09:45	14:09	UI: /222/schedule-history	—	—
18:09:59	14:09	UI: back to /222/schedule	—	—
18:10:00	14:10	UI: /222/lab	—	—
18:11:07	14:11	UI: back to /222/schedule	—	—
18:12:21	14:12	Solver done → 12962 written	parent = 12961	0 (569)	Run #2 result — never accepted. Same overrides as 12961; orphaned.
18:12:24	14:12	SSE new_schedule for 12962	—	—	Client stages it but user doesn't accept

Frustration window (after looking at 12961)

UTC	EDT	Action	Notes
18:17	14:17	📧 Email from user	"The program continues to unfreeze saved freezes and does not make overrides when the schedule is created… placed dozens of overrides, saved freezes, then created the schedule. There are dozens of overrides that it did not save and I'm attempting them again."

Session 4 — Final attempt (the one the second email is about)

UTC	EDT	Action	Schedule written	Δ ovr	Notes
18:24:08	14:24	PUT /overrides/222	→ 12963 (parent = 12961)	0 (569)	Save Freezes — nothing new since 12961
18:24:51	14:24	PUT /overrides/222	→ 12964	+1 (570)	+ Madeleine Marks F → Float (UL) wk 21
18:27:05	14:27	PUT /overrides/222	→ 12965	+1 (571)	+ Ketty Liang W → Float (UL) wk 41
18:30:41	14:30	PUT /overrides/222	→ 12966	+1 (572)	+ Peyton Roach M GHT → Float (UL) wk 5
18:34:35	14:34	PUT /overrides/222	→ 12967	0 (572)	Save Freezes #5 — nothing new
18:34:38	14:34	POST /programs/222/schedule	202 (queued)	—	Create Schedule #3 (reference = 12967)
18:34:42	14:34	Solver log: same warnings, plus added Child Neurology / Do Not Schedule entries	—	—
18:43:15	14:43	Solver done → 12968 written	parent = 12967	+1 (573)	+ Brianna Singer F → Newborn Nursery (PGY1) wk 37 — the only surviving drag override
18:43:18	14:43	SSE new_schedule for 12968	—	—
18:43:42	14:43:42	PATCH /programs/222/schedule = 12968	main = 12968	—	User accepted
18:43:47	14:43	UI: hover-prefetch	—	—
18:46:47	14:46:47	📧 Email from user	"I attempted to override the categorical, TB, and CN PGY-1s only newborn nursery where I wanted them and it again did not place the overrides when creating a new schedule."

What the data says, vs what the user said

The user's stated intent in the second email was to override categorical, TB, and CN PGY-1s onto Newborn Nursery (PGY1). The PGY-1s in those tracks number 18 residents (11 categorical, 3 TB, 4 child-neurology). The actual database delta from when they accepted 12961 to when they accepted 12968 contains one such override: Brianna Singer F (categorical) → Newborn Nursery (PGY1) wk 37. The other three new overrides are Float (UL) freezes.

So the user is correct that drag overrides they staged did not survive to the new schedule. The pattern — five Save Freezes back-to-back inside 10 minutes, each emitting a main_schedule_changed SSE event — is exactly the race window for the preserveOverridesAcrossNextSwapRef bug: the first SSE consumes the flag, every subsequent one leaves the override-context anchored to a stale schedule id, and the next state transition wipes pending overrides from localStorage (and the visual overlay along with them). One drag (Brianna) happened to be staged late enough in the sequence to survive into the Create Schedule POST body; the rest were dropped before the POST fired.

The two PUT-returns-499 / +0 no-op saves (12955, 12963, 12967) and the bulk −20 wipe at 12958 are also consistent with the client being confused about its own state — repeating requests, re-saving the same payload, and occasionally nuking whole residents by mistake.

report.md

Let me check the schedule.tsx for how drag-and-drop overrides are added to the UI:

Investigation summary

User: user_3Dg5l4JWmt6gY3yrHGcaaqlGK5M operating on program 222 (Kentucky Final) today (2026-05-17 UTC).

Today's endpoint timeline

13× PUT /overrides/222 (Save Freezes) interleaved with 3× POST /programs/222/schedule (Create Schedule):

Time UTC	Endpoint	Schedule	Δ overrides vs parent
16:45–17:31	8× Save Freezes	12948–12957	+1 / +1 / +1 / +2 / -6 / +1 / +4 / 0 / +1 / +1
17:51–17:56	3× Save Freezes	12958–12960	-19 / +1 / +1
17:59:04	Create Schedule	→ 12961 "Float Fully staffed"	+38 / -8
18:03:10	Create Schedule	→ 12962	0 (same overrides)
18:09:39	Set main schedule	(to 12961)	—
18:24–18:34	5× Save Freezes	12963–12967	0 / +1 / +1 / +1 / 0
18:34:38	Create Schedule	→ 12968	+1
18:43:42	Set main schedule	(to 12968)	—

Between 12961 (the "Float Fully staffed" baseline) and the user's complaint at 18:46, only 4 net overrides were added: Brianna Singer F → Newborn Nursery (PGY1) wk 37, plus 3 Float (UL) placements. The user reports they made "dozens" of changes.

The bug

web/app/[program]/schedule/page.tsx:889-919 — saveFreezes() deliberately filters out drag-and-drop overrides:

const freezes = currentOverrides.filter(
  (rotationId, key) => baseSchedule.get(key) === rotationId,
);

This sends only entries where the override rotation matches the base schedule's rotation (true "freezes"). Any cell the user dragged to a different rotation is silently excluded, but the toast still says "Freezes saved successfully!". The drag overrides survive locally via pendingOverrideAddsKey in localStorage (rehydrated by the useEffect at page.tsx:291-382), so they reappear in the UI — making it look like they're saved even though the server hasn't seen them.

createSchedule() (page.tsx:825-887) does include drag overrides, so they finally hit the server on Create Schedule. But if the user reloads, switches schedules, or hits a context-mismatch path (page.tsx:340-349), the localStorage-only drag overrides are dropped.

Solver-side noise

update_schedule_from_existing (chiefly/server/run.py:82-88) logs many ERROR Error updating schedule: the rotation X provided cannot staff PGY Y messages during today's Create Schedule runs — e.g. Newborn Nursery (PGY1) cannot staff PGY 2, Newborn Nursery (UL) cannot staff PGY 4. These come from the reference schedule containing override-forced placements that violate rotations_can_staff (id 14393 staffs PGY 1 only; id 14418 staffs PGY 2 med-peds or PGY 3 categorical only). The code swallows these and leaves the week blank rather than placing the rotation — which contributes to the "unfreeze" behavior the user is complaining about.

Actual outcome in schedule 12968

All 26 PGY-1s ARE on Newborn Nursery (PGY1) somewhere in the schedule (solver-placed for most; override-pinned for med-peds PGY-1s, plus Brianna Singer F categorical and Clark Bailey T tb). So the rotation IS being scheduled — but the user wanted to pin specific PGY-1s to specific weeks and only 2 of those pins (Brianna, Clark) survived to schedule 12968.

Recommendation

The most actionable fix is saveFreezes() — either send drag overrides too, or rename the button / show a clearer toast so users don't believe their drag overrides are persisted when only true freezes are.

response.md

Let me trace the specific scenario Daniel described — staged overrides → Save Freezes → Create Schedule.

Let me check the server-side flow more carefully — Daniel suspects the local overrides may not be passed due to schedule ID changes.

Let me run a targeted check on the user's actual data to verify Daniel's hypothesis with their newborn nursery overrides today.

You're right — let me trace the localStorage context dance to see where it can break.

Daniel — I agree your hypothesis is the most likely culprit. Here's what I traced:

Where the mismatch can happen

The override-context dance in page.tsx:322-350 only stays consistent if exactly one main_schedule_changed event arrives between user actions and preserveOverridesAcrossNextSwapRef is consumed correctly. The ref is a single boolean — first SSE consumes it, any subsequent main_schedule_changed (without preserve) leaves the context pointing at the previous schedule id while state.schedule.id advances. On the next render the rehydrator sees storedContext !== currentScheduleId, calls clearPendingOverrides (page.tsx:345), and fires schedule:accepted which setPersistedOverlays([]) in schedule.tsx.

Concrete paths into that state:

1. Two quick Save Freezes in a row. First SSE writes context = NEW_ID_1, consumes ref. Second SSE arrives, ref is false, context stays at NEW_ID_1 while currentScheduleId becomes NEW_ID_2 → mismatch → drag overrides wiped. The user's 17:27:08 → 17:27:16 saves today (8 seconds apart) are well inside the SSE round-trip window for this.

2. Save Freezes → schedule-history rewind. schedule-history/page.tsx:291 calls setMainSchedule which fires main_schedule_changed without preserveRef being set → mismatch → wipe.

3. Save Freezes → accept staged Create Schedule. acceptStagedSchedule calls clearPendingOverrides itself (page.tsx:597) before the PATCH-triggered SSE arrives, so that one's not a bug per se — but it still nulls the context, so any drag override the user did between accepting and the SSE arriving will be cleared on the next state transition.

Why "they see them on their client"

persistedOverlays (the blue spans rendered in schedule.tsx) is initialized once at mount with the context check (schedule.tsx:214-227). After mount it's only mutated by setPersistedOverlays calls and the schedule:accepted handler. If the handler-dispatch race loses a tick — or if the listener missed an event for any reason — the blue overlay stays visible while state.schedule.overrides no longer has the entry. Next Create Schedule call reads state.schedule.overrides, sends without the drag.

On the UKY data specifically

I can't see a smoking gun in 12961→12968 — only 4 new overrides were ever committed to the server in that span (Brianna Singer F + 3 Float UL placements), which is consistent with the user staging more locally than they were aware made it. The schedule chain also branched (12963's parent is 12961 not 12962, because the user PATCH'd back to 12961 at 18:09:39), which is the kind of state-jumping that exercises the race above.

Suggested fix direction

Replace the single preserveOverridesAcrossNextSwapRef boolean with a per-schedule-id queue, or — simpler — anchor the context to state.schedule.id directly inside the rehydrator's "match" branch instead of relying on the SSE handler to write it. That way every SSE that doesn't explicitly clear overrides re-anchors the context, and the only way to lose them is acceptStagedSchedule or clearPendingOverrides explicitly.