Production AI Automation Notes #16: Agent memory that degrades instead of failing — keyword fallback when the embedding provider dies ("Premature close", APIConnectionError)

README.md

Agent memory that degrades instead of failing: keyword fallback when the embedding provider dies ("Premature close", APIConnectionError)

Retrieval with a single vector path goes to zero recall the moment the embedder is unreachable. Two branches plus reciprocal rank fusion keeps it answering.

Last tested: August 2026. See Changelog at the bottom.

If this saves you an outage, follow @renezander030 — production notes on agent memory, retrieval and approval gates.

Reference implementation: github.com/renezander030/agentic-task-system

TL;DR cheat sheet

Symptom	Cause	Fix
APIConnectionError: Connection error on every embed call	Embedding provider unreachable, single retrieval path	Add a keyword branch that needs no network
invalid response body ... Premature close	Legacy HTTP stack with no happy-eyeballs, host has no IPv6 route	Fix the client, but also stop letting one client take recall to zero
Agent answers "I don't know" during an embedder outage	Vector branch threw, retrieval returned []	Catch per branch, return what the other branch found
Recall silently worse, no error anywhere	One of N sources failed, results still returned	Per-source status, surface degraded: true
Ranking looks random after adding a second branch	Concatenating two result lists	Reciprocal rank fusion, not concat

The rule of thumb

Three questions decide whether your memory layer survives an embedder outage:

1. Can any branch of retrieval answer with zero network calls? If no, provider downtime equals total recall loss.
2. Does a failing branch throw, or degrade? If retrieval is one await, one failure is total failure.
3. Can the agent tell it is running degraded? If not, you get quietly worse answers, which is worse than an error.

Threshold worth remembering: an embedding outage should cost you ranking quality, never recall itself.

Recommended setup

One retrieval function, two independent branches, each individually wrapped, merged by rank:

// retrieval.js — the whole pattern in one function
async function retrieve(query, { sources, k = 20 }) {
  const status = {};

  // Branch A: vector. Needs the embedding provider. May fail.
  const vector = (async () => {
    const vec = await embed(query);              // network
    return await vectorSearch(vec, { k });
  })().then(
    r => { status.vector = 'ok'; return r; },
    e => { status.vector = `failed: ${e.code || e.message}`; return []; }
  );

  // Branch B: keyword. Pure CPU, no network, cannot fail for provider reasons.
  const keyword = (async () => {
    return await ftsSearch(query, { k });        // SQLite FTS5, BM25, whatever
  })().then(
    r => { status.keyword = 'ok'; return r; },
    e => { status.keyword = `failed: ${e.message}`; return []; }
  );

  const [v, kw] = await Promise.all([vector, keyword]);
  const fused = rrf([v, kw], { k: 60 });

  return {
    results: fused.slice(0, k),
    status,
    degraded: Object.values(status).some(s => s !== 'ok'),
  };
}

The important part is not the vector search. It is that .then(ok, fail) is applied per branch, so one branch failing yields [] from that branch, not a rejected promise from the whole function.

Reciprocal rank fusion, the merge step

Do not concatenate. Two branches score on incompatible scales (cosine similarity vs BM25), so concatenation destroys ordering. RRF merges on rank, which is scale free:

function rrf(lists, { k = 60 } = {}) {
  const scores = new Map();
  for (const list of lists) {
    list.forEach((item, i) => {
      const id = item.id;
      const prev = scores.get(id) || { item, score: 0 };
      prev.score += 1 / (k + i + 1);   // rank, not similarity
      scores.set(id, prev);
    });
  }
  return [...scores.values()]
    .sort((a, b) => b.score - a.score)
    .map(e => e.item);
}

k = 60 is the value from the original TREC work. It damps the top of each list so a single branch cannot dominate. When the vector branch is down, RRF over one list is just that list, ordered, which is exactly the behaviour you want.

Common failure error strings

APIConnectionError: Connection error.

The embedding call never completed. On its own this tells you nothing about why. Check the cause.

invalid response body while trying to fetch https://<provider>/v1/embeddings: Premature close

The classic one, and the reason this pattern exists. A legacy node-fetch / http.Agent HTTP stack does not implement happy-eyeballs (autoSelectFamily). If DNS returns an AAAA record but the host has no IPv6 route, which is common in containers and datacenters, the client attempts IPv6 and has no IPv4 fallback. The connection dies mid-response.

Fix the client (a stack built on undici / native fetch races v4 and v6 and falls back). But note the blast radius: with a single retrieval path this was 100% of recall gone, from a networking detail two dependencies down.

fetch failed with cause: ECONNREFUSED 127.0.0.1:11434

Local embedding server not running. Same class, different cause. Ollama stopped, container restarted, port moved.

model "nomic-embed-text" not found, try pulling it first

The model was never pulled into the running container. Recall silently collapses to whatever the keyword branch finds, which is why degraded in the response matters.

Debug flow

If recall dropped, walk this in order:

1. Does the query return anything at all? If yes, jump to step 4. If nothing, retrieval is failing whole, not degrading.
2. Is the failure in a branch or in the function? Log status. If status is missing entirely, your try/catch is around the whole function and one branch is taking down the other. That is the bug.
3. Can the keyword branch answer alone? Kill the network and run the same query. If it returns nothing, you do not have a fallback, you have a second copy of the same dependency.
4. Is degraded true? If true with status.vector failed, ranking is BM25-only. Expect worse ordering, not worse recall.
5. Are all sources reporting? With multiple upstreams, one expired token means recall from N-1 sources with no error anywhere. Per-source status catches this.
6. Did ranking get worse right after adding a branch? You are concatenating instead of fusing. See the RRF section.

Per-source status, and why a boolean is not enough

Once memory spans more than one upstream, "is retrieval up" has no single answer:

{
  "degraded": true,
  "sources": {
    "github":   { "state": "ok",      "results": 12, "ms": 340 },
    "notion":   { "state": "ok",      "results": 8,  "ms": 512 },
    "ticktick": { "state": "auth",    "results": 0,  "error": "401 token expired" },
    "vector":   { "state": "failed",  "results": 0,  "error": "ECONNREFUSED" }
  }
}

An agent that sees this can say "I searched two of four sources" instead of confidently answering from a partial corpus. That distinction is the whole point: a quiet 60% recall is more dangerous than a loud 0%, because nothing looks wrong.

Why not just retry?

Approach	Handles provider down	Handles slow provider	Cost	Recall during outage
Retry with backoff	No, retries also fail	Adds latency	Free	0%
Cache embeddings	Only for repeat queries	Yes	Memory	~0% on new queries
Second embedding provider	Yes	Yes	Second vendor, second bill, drift between vector spaces	100%
Keyword branch + RRF	Yes	Yes, branch returns independently	CPU only	100% recall, reduced ranking

Retry is orthogonal and still worth having. It just does not help with the failure mode where the provider is unreachable from this host, which is the one that produces Premature close.

Setups I would avoid

• One try/catch around all of retrieval. Guarantees that any single branch failure is a total failure. This is the single most common version of this bug.
• Returning [] on failure with no status. The caller cannot distinguish "no matches" from "retrieval broken", and neither can the agent.
• Concatenating branch results. Two scoring scales, one list, meaningless order.
• A keyword branch that hits the same network service as the vector branch. Not a fallback. Test it with the network off.
• Treating the fallback as untested. If the keyword path only runs during incidents, it only breaks during incidents.

Smoke test

Prove the fallback works by breaking the provider on purpose:

# 1. Baseline, both branches healthy
node -e "import('./retrieval.js').then(m=>m.retrieve('auth token refresh').then(r=>
  console.log(JSON.stringify({n:r.results.length,degraded:r.degraded,status:r.status},null,2))))"

{
  "n": 20,
  "degraded": false,
  "status": { "vector": "ok", "keyword": "ok" }
}

# 2. Point the embedder at a closed port and run the identical query
EMBED_URL=http://127.0.0.1:9 node -e "import('./retrieval.js').then(m=>m.retrieve('auth token refresh').then(r=>
  console.log(JSON.stringify({n:r.results.length,degraded:r.degraded,status:r.status},null,2))))"

{
  "n": 14,
  "degraded": true,
  "status": { "vector": "failed: ECONNREFUSED", "keyword": "ok" }
}

Pass criteria: second run returns a non-empty results, degraded: true, and a status.vector that names the cause. If run 2 throws or returns zero results, the fallback is decorative.

Wire that into CI as a regression test. It needs no API key, since the point is that the provider is down.

Steal-able config

Retrieval settings worth making explicit rather than hardcoding:

retrieval:
  k: 20
  rrf_k: 60                  # TREC default, damps top-of-list dominance
  branches:
    vector:
      enabled: true
      provider: ollama
      model: nomic-embed-text
      url: http://127.0.0.1:11434
      timeout_ms: 3000       # fail fast, the other branch is already running
      required: false        # false = its failure degrades, never fails the call
    keyword:
      enabled: true
      engine: fts5
      required: true         # the branch that must always be able to answer
  report_status: true        # emit per-source state to the caller

The required flag is the whole design encoded as config: exactly one branch is required, and it is the one with no network dependency.

When you don't need this

• Single-source retrieval where the source is the vector DB, and losing it means you have no data anyway.
• Batch pipelines that can fail and be re-run. Degradation matters for interactive agents, where the alternative to a partial answer is a confidently wrong one.
• Corpora small enough to fit in the prompt. Do not build retrieval you do not need.

---

Series

This is Production AI Automation Notes #16. Related entries:

• #3: Claude Code persistent memory between sessions (Agentic Task System) — the layer this pattern lives in: your task app as agent memory, hybrid RRF retrieval
• #1: Agent Approval Gates — proposed actions, schema validation, audit log
• #11: Pipeline fixture testing — zero-API-call deterministic CI, same instinct as the smoke test above

Reference implementation: agentic-task-system (Node, MIT). Follow @renezander030 for new entries.

Sources

• Reciprocal rank fusion, k = 60: Cormack, Clarke, Buettcher, Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods (SIGIR 2009)
• Node autoSelectFamily / happy-eyeballs behaviour: Node.js net.connect docs
• Failure signature and reproduction table: mem0ai/mem0#5794 (2026-06-23)

Reader contributions

Comment with your setup and what broke:

• Embedding provider and model (hosted or local, which quant)
• Keyword engine (FTS5, BM25, Postgres tsvector, something else)
• Number of upstream sources
• What your retrieval did the last time the embedder went down, and whether you found out from a log or from a bad answer

Changelog

2026-08-02

• Initial publication.
• Deliberate gate skips: hardware matrix (not hardware-bound), companion-repo creation (reference implementation already exists at agentic-task-system, so it is linked rather than scaffolded).