On unified-memory systems (e.g. DGX Spark, UMA) or when repeatedly swapping models (7B ↔ 120B), Linux page cache can be reported as "used" memory by PyTorch/Ray/vLLM and similar runtimes. That causes false OOM or "free memory on device is less than desired" on startup even though the memory is reclaimable. This spec captures the mitigation as a first-class feature: drop page cache before model launch in the deploy flow so the runtime sees accurate free memory.
Objective: As an operator swapping or starting the 120B (or 7B) model on gx10, I want the deploy scripts to drop Linux page cache before launching the model, so that runtimes do not see stale cache as consumed memory and trigger false OOM.
- 1.1 WHEN the swap script (e.g.
swap-llm.sh) is about to start a 120B service THEN the system SHALL runsyncand then drop page cache (echo 3 > /proc/sys/vm/drop_caches) on the local host before starting the systemd unit. - 1.2 IF the drop-cache step fails (e.g. no sudo, permission denied) THEN the system SHALL log a warning and SHALL continue with the model start (non-fatal).
- 1.3 WHEN the easy button or deploy flow starts the 120B backend THEN the system SHALL invoke the same drop-cache step before starting the backend.
- 1.4 WHERE drop cache is performed THE implementation SHALL be documented and SHALL reference the footgun doc (MODEL_DEPLOY_MEMORY_PAGE_CACHE.md).
Objective: As a maintainer, I want a single reusable way to drop page cache so that swap, easy.sh, and future deploy paths can call it without duplicating logic.
- 2.1 WHEN a deploy path needs to drop page cache THEN it SHALL call a shared script or function that runs sync + drop_caches.
- 2.2 IF the environment or platform does not support dropping cache (e.g. read-only /proc) THEN the helper SHALL exit cleanly and SHALL not fail the deploy.
- 2.3 The helper SHALL be configurable or skippable (e.g. env var) so operators can disable it if not needed on a given host.