A frame to help you reason through the assignment — concepts, diagrams, and the API map. It deliberately does not contain the answers or filled-in activity code. Instead it gives you the questions to ask yourself and the method to get there. The work — and the learning — is in running the cells, reading the decision tables / cache timings / the A2A trace, and writing your own conclusions.
Session shape: THREE independent parts — two notebooks + a runnable
a2a/mini-project. The notebooks are fully worked (no blanks to fill): your job is to run them, keep the outputs that explain your work, and understand why. Do any subset — one part is a legitimate submission. There is no set order.Notebooks:
01_Cat_Health_Agent_Guardrails.ipynb,02_Cat_Health_Agent_Caching.ipynb; mini-project:a2a/. Safety framing that recurs everywhere: educational cat-health assistant, not a veterinary tool.
| You want to… | Reach for | One-liner |
|---|---|---|
| Refuse an emergency before the model runs | deterministic input rail | run_input_rails() → escalate short-circuits to EMERGENCY_MESSAGE |
| Block a known jailbreak cheaply | regex injection rail | INJECTION_PATTERNS → block (high-precision, low-recall; a first pass) |
| Redact PII without punishing the user | redact_pii() |
re.subn → [REDACTED_EMAIL]/[REDACTED_PHONE], action = rewrite not block |
| Judge off-topic when regex can't | model-based rail | check_topic() → guard_llm.with_structured_output(TopicVerdict) |
| Fix/replace a bad draft reply | output rail | run_output_rails(draft) → replace (medical authority) or repair (append disclaimer) |
| Wire rails into the agent loop | LangChain middleware | @before_model(can_jump_to=["end"]) + @after_model; replace messages by id |
| Skip paying for an identical prompt | exact-match cache | set_llm_cache(InMemoryCache()) — keyed by exact prompt string + params |
| Serve paraphrases from cache | semantic cache | SemanticCache(embed, threshold=0.90); lookup() on max cosine ≥ threshold |
| Reuse a deterministic embedding | content-hash cache | CachingEmbedder keys by hashlib.sha256(text).hexdigest() |
| Stop re-hitting a slow backend | tool-result cache + TTL | TOOL_CACHE + TOOL_TTL_SECONDS = 300.0 (the staleness "honesty knob") |
| Measure provider-side prompt caching | raw OpenAI() client |
usage.prompt_tokens_details.cached_tokens (LangChain hides it) |
| Advertise an agent to other agents | A2A agent card | GET /.well-known/agent-card.json → SPECIALIST_CARD (résumé, no internals) |
| Send work to another agent | JSON-RPC message/send |
send_message(base_url, text) over HTTP POST |
| Let one agent delegate to another | card-injected system prompt | front_desk.py injects fetched name/description; model decides per question |
| Test A2A with no API key | protocol smoke test | smoke_test.py — stub agent, card + round-trip + -32600/-32601/-32602 |
Anchor: Production hardening is three cheap disciplines layered on a working agent — guardrails decide what is safe, caches decide what is repeatable, A2A decides what is delegable — and they interlock: the rails decide what is safe to cache, and both travel across the A2A wire.
flowchart TB
subgraph G["① Guardrails (NB1) — funnel: cheap+certain first"]
U[user input] --> IR["input rails (free regex):\nemergency→escalate · injection→block · PII→rewrite"]
IR -->|survives| TG["model rail: check_topic → TopicVerdict"]
TG -->|on-topic| A[agent + tools]
A --> OR["output rails: medical-authority→replace · disclaimer→repair"]
OR --> R[reply]
end
subgraph C["② Caching (NB2) — don't pay twice"]
EX[exact-match] --- SEM["semantic (⚠ threshold cannot save you)"] --- TOOL[embedding/tool + TTL] --- PP[provider prefix cache]
end
subgraph P["③ A2A (a2a/) — reach other agents"]
FD[front desk] -->|"reads card, message/send"| SP[remote specialist]
SP -.opaque: model/tools/prompt hidden.-> FD
end
IR -. "rails decide what is safe to cache" .-> SEM
R -. "guard both directions of the wire" .-> FD
ASCII fallback:
① GUARDRAILS user ─► input rails (free regex: emergency/injection/PII) ─► model topic rail ─► agent ─► output rails ─► reply
│ escalate/block short-circuit: model never called (0 tokens)
② CACHING exact-match ─ semantic(⚠) ─ embedding/tool(+TTL) ─ provider prefix cache ["dress for it": stable first, variable last]
③ A2A front desk ──reads agent card, message/send──► remote specialist (specialist's model/tools/prompt stay opaque)
interlock: rails decide what is safe to cache · guardrails + caching both apply across the A2A wire
Why this shape? All three parts wrap the same working agent from earlier sessions — nothing new is being built, it is being made production-safe. The recurring discipline is run the cheapest, most certain thing first (free regex before a paid model rail; an exact-match cache before a semantic one) and know the failure mode (regex misses paraphrases; a semantic cache can serve a poisoning answer to a dinner question). The parts interlock rather than stack: the emergency rail is also the cache's safety gate, and your rails still matter on your side of an A2A wire because the remote agent is opaque.
cd 12_Production_Agent_Patterns && uv sync # requires-python >=3.13
export OPENAI_API_KEY="your-key" # notebooks + a2a specialist
# optional: export LANGSMITH_TRACING=true LANGSMITH_API_KEY="your-key"
# a2a: terminal 1 → `cd a2a && uv run python server.py` (serves http://127.0.0.1:9999)| Component | Role |
|---|---|
main chat model (AIM_CHAT_MODEL, default gpt-5.4-mini) |
powers the agent's answers |
| guard model | the model-based topical rail — usually small; one classification job |
text-embedding-3-small |
semantic-cache + embedding-cache vectors |
raw openai OpenAI() client |
Task-6 only — exposes usage.prompt_tokens_details LangChain hides |
A2A server (uvicorn, port 9999) |
hosts the specialist behind card discovery + message/send |
Concept before code: a system prompt only asks the model to behave; a guardrail is code that runs outside the model and doesn't trust it. Three seams (input / policy-during-loop / output) × two kinds (deterministic = free/exact/brittle; model-based = paid/general/fallible). Order cheapest-first.
# input rails: fixed precedence emergency → injection → PII → allow
dec = run_input_rails(text) # → RailDecision(action ∈ allow|block|escalate|rewrite)
# model rail (only for inputs that survived the free rails)
verdict = check_topic(text) # → TopicVerdict(on_topic, category, reason) — structured, not prose
# output rails: some violations repair, some must replace
final, actions = run_output_rails(draft)
# wire both into the agent
guarded_agent = create_agent(model=llm, tools=[care_guide_lookup],
system_prompt=AGENT_SYSTEM_PROMPT, middleware=[input_rails_middleware, output_rails_middleware])Docs: LangChain agent middleware (@before_model / @after_model, can_jump_to).
Concept before code: anything deterministic and repeated is a caching candidate — but a cache must never turn a look-alike into a wrong answer. Four layers, increasing subtlety:
set_llm_cache(InMemoryCache()) # exact-match: key = exact prompt string + params
cache = SemanticCache(embed=embeddings.embed_query, threshold=0.90) # paraphrase hits… and danger
key = hashlib.sha256(text.encode()).hexdigest() # embedding cache
TOOL_TTL_SECONDS = 300.0 # tool cache: TTL = the staleness you accept
usage.prompt_tokens_details.cached_tokens # provider prefix cache: stable content firstThe crux (spend real thinking time here): similarity("cat ate chocolate", "cat ate chicken") is
HIGH — one word apart as sentences — but one is dinner and one is a poisoning emergency.
Embeddings measure how alike sentences look, not how much the difference matters. Docs: LangChain
caching; OpenAI prompt caching.
Concept before code: MCP is agent → tool ("call this function"); A2A is agent → agent
("consult this colleague"). A tool executes; an agent reasons, and the caller sees none of its
internals (opacity). A2A standardizes only the conversation: the card (discovery) + messages.
GET /.well-known/agent-card.json → the card: name, description, skills, capabilities (NO model/tools/framework)
POST / {jsonrpc, id, method:"message/send", params:{message:{role, parts:[{kind:"text",text}]}}}
errors: -32600 bad envelope · -32601 unknown method · -32602 no text parts (structured, never a stack trace)
Front desk delegates by injecting the fetched card's description into its prompt (publish a better
card → better routing, no client change); the model decides per question. Trace shows [A2A] lines on
delegated questions, none on locally-answered ones. Docs: a2a-protocol.org; JSON-RPC 2.0 spec.
This section gives you the questions to ask yourself and what to inspect. The answers are in the outputs you generate by running the cells and the mini-project — read them, then write your own conclusion. Keep the outputs that explain your work (decision tables, timings, the A2A trace).
Run the notebook and read each Task's printed output before moving on. Ask yourself:
- Layering & order. There are three places to intervene and two kinds of rail. Why run the free regex rails before the model-based one? What does the model rail cost that the regex rails don't?
- Input rails. Watch the decision table: which action fires for an emergency, for an injection
attempt, for a message containing an email/phone? Why is PII
rewrite/redact rather thanblock? What is the honest thing regex cannot do — and which later Task exists to cover that gap? - Model rail. Inspect a
TopicVerdict— it's a structured object, not prose. Why give the guard its own single-job prompt instead of adding "also refuse off-topic" to the main agent prompt? - Output rails. Two of the demo drafts are handled differently: one is replaced wholesale, one is repaired (something appended). Which is which, and why can't the dangerous one just be patched? What decides repair-vs-replace?
- Middleware. In the five guarded runs, notice which inputs the model never sees. What makes a blocked/escalated request cost zero model tokens? How does a rewrite actually take effect? Do the rails replace the system prompt, or back it up?
Keep as evidence: the Task-3 decision table, the topic verdicts, the output-rail runs, and the five guarded agent runs.
- Baseline (Task 2). Ask the same question twice with no cache. Is the second ask any cheaper? Are the two answers even identical? What two problems does that motivate a cache to solve?
- Exact-match (Task 3). Compare cold vs warm timing — then re-ask with one extra character. Why does one character send you back to full price? Who does an exact-match cache actually help — humans, or programs?
- Semantic — and its danger (Task 4). This is the most important thing in the notebook. A
paraphrase hits the cache — good. Then look at
similarity("…ate chocolate…", "…ate chicken…"). It's high. One of those is a poisoning emergency. What would a cache hit do here? Now the trap to avoid: can you fix this just by tuning the threshold? Think about what embeddings actually measure. What would make a high-stakes query safe — where should it be decided (hint: look back at Part 1)? What else bounds a production semantic cache (time, size, per whom)? - Embedding & tool caches (Task 5). After two tool runs, read the
slow lookups so far:counter, not just wall-clock. How many times did the slow backend actually run? What does the TTL value say you're willing to serve — and where would 5-minute-old data be unacceptable? - Provider prefix cache (Task 6). Read
served from prompt cacheon the second call. Why does putting the stable content first and the variable question last matter? What happens to caching if you put a timestamp at the top of the system prompt? (A first-call reading of 0 is normal — wait a moment and re-run.)
Keep as evidence: the cold/warm timings, the paraphrase HIT, the chocolate-vs-chicken similarity score, the tool-cache counter, and the prompt-cache token counts.
Start the server (uv run python server.py), then explore. Ask yourself:
- MCP vs A2A. You used MCP in Session 8 to reach tools. How is reaching another agent different — who does the thinking, and what can the caller see of the other side?
- The card.
curlthe/.well-known/agent-card.json. What's in it — and, more tellingly, what is not (does it reveal the model? the tools? the framework?)? Why does the client route on the card and not the code? - Transport & errors. Change the request
methodtotasks/get. What comes back — a stack trace, or a structured error object? Why does a remote caller need the latter? - Sync vs Task. This specialist answers immediately. When would a server instead hand back a Task you poll for artifacts — and is answering synchronously still spec-legal?
- Delegation (
front_desk.py). Run the demo and read the trace: the clinic-hours question shows no[A2A]lines; the health question shows them. What makes the front desk decide to delegate — a hardcoded rule, or something it read from the fetched card and reasoned about? - The interlock. Where do your Part-1 guardrails apply once you're talking to an agent you don't control — one direction, or both? And what new way can a cached A2A answer go stale that a local cache can't?
Keep as evidence: the front_desk.py delegation trace, the agent-card JSON, and/or the
smoke_test.py pass (smoke_test.py needs no API key — a good first check).