Skip to content

Instantly share code, notes, and snippets.

@donbr
Last active July 8, 2026 03:45
Show Gist options
  • Select an option

  • Save donbr/be86b6da2cff3d9fbae0de3f8c5fa985 to your computer and use it in GitHub Desktop.

Select an option

Save donbr/be86b6da2cff3d9fbae0de3f8c5fa985 to your computer and use it in GitHub Desktop.
Session 9 Cheat Sheet — Agent Servers

Session 9 Cheat Sheet — Agent Servers

A frame to help you reason through the assignment — concepts, diagrams, and the API map. It deliberately does not contain the answers or filled-in activity code. Instead it gives you the questions to ask yourself and the method to get there. The work — and the learning — is in packaging the agent, inspecting a Studio/LangSmith trace, shipping it, and writing your own conclusions.

Source repo: 09_Agent_Servers/ (a packaged LangGraph agent + a Next.js frontend/). No notebook — Q1/Q2 are answered in README.md. Corpus: data/cat_health_guidelines.pdf (a feline-health PDF the RAG tool indexes in-memory; benign).

Quick Reference

You want to… Reach for One-liner
Tell LangGraph what graphs exist langgraph.json graphs "simple_agent": "app.graphs.simple_agent:graph" (id → module:attribute)
Export a runnable graph a module-level graph graph = create_agent(model=..., tools=..., system_prompt=...)
Run the agent API locally langgraph dev serves http://localhost:2024 + opens LangGraph Studio
Self-host a prod container langgraph up local Docker; you own scaling/uptime/auth
Deploy to LangSmith cloud langgraph deploy managed build+host (needs LangSmith Plus) → Deployment URL
Talk to the agent in Python langgraph_sdk.get_client client.runs.stream(None, "agent", input={...}, stream_mode="updates")
Stream the agent into a UI @langchain/react useStream useStream({ apiUrl: "/api", assistantId: "simple_agent" })
Hide the API key from the browser langgraph-nextjs-api-passthrough initApiPassthrough({ apiUrl, apiKey, runtime: "edge" }) in route.ts
Mark a var public in Next.js NEXT_PUBLIC_ prefix only NEXT_PUBLIC_* reaches the browser; everything else is server-only
Deploy the UI vercel / vercel --prod set LANGGRAPH_API_URL, LANGSMITH_API_KEY, NEXT_PUBLIC_API_URL
See what the agent actually did LangSmith traces / Studio every run is traced; Studio steps through nodes + tool calls

Anchor: LangSmith hosts the agent as an API; Vercel hosts the UI and the server-side proxy that injects the secret — the browser only ever talks to same-origin /api/*.

The Big Picture

flowchart LR
  User[User in browser] --> Vercel[Next.js UI on Vercel]
  Vercel -->|"/api/* passthrough (injects key server-side)"| LS[LangSmith Agent API]
  LS --> Agent[Your LangGraph graph]
  Agent --> Tools[Tavily + Arxiv + RAG tool]
  LS --> Traces[LangSmith tracing]
  subgraph local[Local dev]
    Dev["langgraph dev :2024"] --> Studio[LangGraph Studio]
  end
Loading

ASCII fallback:

browser ─► Vercel UI ─► /api proxy (adds LANGSMITH_API_KEY server-side) ─► LangSmith Agent API ─► graph ─► tools
                                                                              └─► LangSmith traces
  local:  langgraph dev :2024 ─► LangGraph Studio (debug/step/fork)

Why this shape? A deployed agent is a stateful API (threads / runs / assistants + streaming), not a website. The two deployment targets map to two concerns: LangSmith runs + observes the agent; Vercel serves the UI and keeps the secret on the trusted server tier. The same compiled graph runs in Studio (debug) and in production (hosted) — only the environment differs.

Setup & roles

uv sync                 # agent deps (Python 3.13)
cp .env.example .env    # fill OPENAI_API_KEY, TAVILY_API_KEY, LANGSMITH_API_KEY
cd frontend && npm install   # the frontend ships complete — no create-next-app needed
Component Role
app/models.py model factory get_chat_model() (default gpt-5.4-mini, temperature=0)
app/tools.py tool belt: TavilySearch, ArxivQueryRun, retrieve_information (RAG)
app/rag.py loads data/*.pdf → split 750/0 → text-embedding-3-small → in-memory Qdrant
app/graphs/simple_agent.py create_agent(...) → exports compiled graph
langgraph.json manifest: graphs + assistants the server exposes
frontend/app/api/[...path]/route.ts secure passthrough — injects the key server-side
frontend/components/chat.tsx useStream chat UI pointed at /api

Core concepts

1. Packaging — the manifest is the contract

A notebook agent becomes a server by exporting a compiled graph and registering it. langgraph.json maps a graph_id to "module:attribute" and declares named assistants. dependencies: ["."] installs the local package; env: ".env" loads keys. Docs: https://docs.langchain.com/langgraph-platform/

"graphs": { "simple_agent": "app.graphs.simple_agent:graph" }

2. Local server + LangGraph Studio

langgraph dev serves the Agent Server API at :2024 with hot reload and opens Studio. Studio = debugging: visualize topology, step through runs, inspect tool calls, fork threads, switch assistants. Not a production UI. Docs: https://docs.langchain.com/langgraph-platform/langgraph-studio

3. The SDK — same events as Studio

The production integration path. A passing local smoke test proves the manifest, graph import, and streaming all work — the README gate before deploying.

from langgraph_sdk import get_client
client = get_client(url="http://localhost:2024")
for chunk in client.runs.stream(None, "agent", input={"messages": [...]}, stream_mode="updates"):
    ...

4. Deploy the agent (LangSmith vs self-host)

langgraph deploy = managed LangSmith cloud build+host (needs Plus) → a https://<...>.us.langgraph.app Deployment URL + optional auto-update-on-push. langgraph up = self-hosted Docker container. Either way you get the same API surface (threads/runs/assistants) plus tracing. Docs: https://docs.langchain.com/langsmith/deployments. Docker/self-host instructions: langgraph up (Docker required, production-like local) — https://docs.langchain.com/langsmith/local-dev-testing#langgraph-up; standalone Agent Servers via Docker / Docker Compose / Kubernetes — https://docs.langchain.com/langsmith/deploy-standalone-server.

4½. Do you need LangSmith Plus? (check the cost before you deploy)

Cloud deploy (langgraph deploy or GitHub→UI) requires Plus — $39/seat/mo 1. Free without Plus: everything local (langgraph dev + Studio + SDK, all of Part 1), tracing (5k traces/mo on the Developer plan), and self-hosting via langgraph up (Docker). Check your plan: LangSmith → Settings → Billing and Usage (smith.langchain.com/settings/payments) — a personal org is Developer (free, no cloud deploy); a team/shared org is Plus. An "Upgrade to Plus" banner means you're not on it. Upgrade is self-serve there (add a card); Enterprise is contact-sales. Plus includes 1 free Dev deployment, unlimited runs — enough for this assignment; beyond it, ~$0.005/run plus uptime (⚠️ a Production deployment left running accrues cost). No-Plus path to the deliverable: self-host with langgraph up, or record the Loom against the local Studio demo (the README accepts "Studio debugging" as one of its two demo options). Cost table: pricing 2.

Plan Price Free traces/mo Cloud deploy?
Developer $0/seat, 1 seat 5,000
Plus $39/seat/mo 10,000 ✅ 1 free Dev deployment
Enterprise custom custom ✅ + self-hosted/hybrid

5. The agent itself

create_agent wires a ReAct-style agent: the model decides when to call a tool, the tool runs, control returns until the model gives a final answer. Tools: Tavily (web), Arxiv (papers), retrieve_information (RAG over the cat-health PDF).

6. Frontend streaming — useStream

A "use client" component subscribes to the agent's event stream and renders messages as they arrive. Point it at the same-origin /api proxy, not the LangSmith URL directly. Docs: https://www.npmjs.com/package/@langchain/react

const { messages, submit, isLoading } = useStream({ apiUrl: "/api", assistantId: "simple_agent" });

7. The secure passthrough — the key never reaches the browser

Anything in client code is readable by the user. The passthrough route runs server-side and injects LANGSMITH_API_KEY into the upstream request; the browser only sees /api/*. Only NEXT_PUBLIC_* vars are public. Docs: https://www.npmjs.com/package/langgraph-nextjs-api-passthrough

export const { GET, POST, ... } = initApiPassthrough({
  apiUrl: process.env.LANGGRAPH_API_URL, apiKey: process.env.LANGSMITH_API_KEY, runtime: "edge" });

8. Deploy the UI (Vercel)

Set Root Directory to frontend, add LANGGRAPH_API_URL + LANGSMITH_API_KEY (server-only) + NEXT_PUBLIC_API_URL, then vercel --prod. Verify end-to-end: the live site streams, tool calls fire, traces appear in LangSmith. Docs: https://vercel.com/docs/frameworks/nextjs


Questions — reason it through yourself (no answers here)

Q1 — Why does LangSmith deploy your agent as an API backend only, and why still a separate Vercel frontend?

Method to get there — ask yourself:

  • When you hit your deployed agent, what comes back — HTML you could render in a browser, or JSON/streamed events? What does that tell you about whether LangSmith can be your website?
  • List what the agent API exposes (threads? runs? assistants?). Is any of that a user interface?
  • If you skipped Vercel and pointed the browser directly at the LangSmith URL, what would the user need in the request — and what does §7 say about putting that in the browser?
  • Frame your answer as "two concerns, two hosts." Name each host's job in one line.

Q2 — Why should the LangSmith API key live in a Next.js API route (server-side), not the browser?

Method to get there — ask yourself:

  • Open devtools on any site → Network + Sources. What can you read? Could a secret compiled into client JS stay hidden?
  • In Next.js, which env vars reach the browser? What does the NEXT_PUBLIC_ prefix do, and why are LANGGRAPH_API_URL / LANGSMITH_API_KEY deliberately not prefixed while NEXT_PUBLIC_API_URL is?
  • Trace one request: browser → /api/* → ? Where does initApiPassthrough add the key — on the user's machine or on Vercel's server?
  • What could someone do with your key if they lifted it from the bundle? Let that consequence anchor your "why."

Activities — what to build + how to inspect it (no solution code here)

Activity 1 — Build a Helpfulness Loop (agent_with_helpfulness)

Deliverable: a new graph that, after the agent answers, has a judge model decide whether the response was helpful and loops back for another attempt if not — with a safe loop limit — registered in langgraph.json, deployed, with traces compared for a passing vs failing query. Method to get there — ask yourself:

  • The provided agent uses create_agent. Can you insert a step that runs after the final answer with it? If not, what lower-level construct lets you own the nodes and edges (and a cycle)?
  • What nodes do you need — who answers, who runs tools, who judges? Draw the edges: when does the judge send control back to the agent vs to the end?
  • How will the judge "decide"? What does it compare (the answer against…?), and how do you keep its output easy to branch on?
  • What would happen if the judge kept saying "not helpful"? Design the loop limit first — what will you count, and where do you force the end?
  • In Studio/LangSmith, inspect both runs: does the passing query end in one pass? Does the failing one show the retry edge firing? Where do you see the judge call?
  • Reflect (the README asks): is the loop's logic any different in Studio vs production — or is it the same graph in a different environment?

Advanced Activity — Auth & custom routes (optional)

Deliverable: describe (optionally implement) how to add authentication so each user only sees their own threads. Method to get there — ask yourself:

  • Hiding the chat page behind a login in React — does that actually stop someone from reading another user's threads through the API? Where must the real check live?
  • What would the server need to know on each request to scope threads to one user, and what could you stamp on a thread when it's created so reads can be filtered?
  • Where should the user's token be attached — the same place §7 puts the API key, or the browser? Why?
  • Look at the README's lsd-custom-route-react-ui reference for the pattern.

References

Grounded against the LangChain docs and the LangChain pricing page (verify — plans and prices change; figures current ~mid-2026).

LangSmith Plus / deployment / cost (§4½):

Docker / self-host deployment (langgraph up, standalone container):

Packaging, server, SDK, frontend:

Footnotes

  1. Deploy to Cloud — "Agent deployments running on Cloud require a Plus plan or above"https://docs.langchain.com/langsmith/deploy-to-cloud-overview

  2. LangSmith pricing (Developer $0 / Plus $39-seat / Enterprise custom; free-trace tiers; deployment run + uptime rates) — https://www.langchain.com/pricing

Session 9 — Agent Servers: Learning Journey (Student Version)

A step-by-step build log for the Session 9 assignment — what to build, the docs that unblock each step, and the gotchas to watch for. Use it as a companion while you package the agent, run it locally, and ship the frontend.

This is the student version. The engineering lessons and pitfalls are kept, but the answers to the graded Questions & Activities are intentionally removed — those are yours to reason out (see the prompts near the end and answer them in README.md). No API keys or secrets appear here; anything key-shaped is a <placeholder>.

A reference environment: WSL2 (Linux), Python 3.13 via uv, Node 20.x, Docker running, with your three API keys in a local .env (git-ignored). Provider = OpenAI + Tavily (session 10's Fireworks variant is a sibling, structural reference only).

Legend: ✅ verifiable locally · 📄 needs a paid account (documented, not required to run) · ⚠️ gotcha


Grounding sources (prefer the LangChain / platform docs over guessing)

Topic Source Why it matters
langgraph.json structure & fields LangChain docs — Application structure (/langsmith/application-structure) The real manifest keys: dependencies, graphs, env, python_version.
Local server LangChain docs — Run a local server (/oss/python/langgraph/local-server) langgraph dev needs langgraph-cli[inmem]; needs a LangSmith API key.
Assistants vs graphs LangChain docs — Default assistants (/langsmith/assistants), Configuration (/langsmith/configuration-cloud) The run identifier is the graph ID (key under graphs); each graph gets a default assistant. Drives the assistantId gotcha below.

(Frontend / deploy sources are listed in their own Parts further down.)


Part 1 — Package the agent & run it locally ✅

What the package looks like

The notebook agent becomes an importable package. Layout:

09_Agent_Servers/
├── langgraph.json          # manifest: 2 graphs + env + python_version
├── pyproject.toml          # uv project, langgraph-cli[inmem] for `langgraph dev`
├── .env / .env.example     # OPENAI/TAVILY/LANGSMITH keys (.env git-ignored)
├── main.py                 # in-process smoke test (graph.invoke, no server)
├── smoke_test_sdk.py       # SDK client smoke test (streams via a running server)
├── app/
│   ├── __init__.py         # loads .env at import time
│   ├── state.py            # MessagesState
│   ├── models.py           # get_chat_model() → ChatOpenAI (gpt-4.1-mini)
│   ├── tools.py            # Tavily + Arxiv + RAG tool
│   ├── rag.py              # lazy, cached in-memory Qdrant RAG over the PDF
│   └── graphs/
│       ├── simple_agent.py            # ReAct loop; exports `graph`
│       └── agent_with_helpfulness.py  # adds a helpfulness retry loop; exports `graph`
└── data/cat-health-guide.pdf

Each graph file exports a module-level graph = build_graph().compile() — that is the object langgraph.json points at (app.graphs.simple_agent:graph). Forgetting the :graph suffix or not compiling at import time is the most common packaging mistake.

Steps to run (all verifiable locally)

  1. uv sync — installs langgraph, langgraph-api, langgraph-cli, langchain, langchain-openai, langchain-qdrant, etc.

  2. uv run python main.py — in-process invoke. The agent calls retrieve_information, the RAG pipeline loads + embeds the PDF and answers from context. ✅

  3. uv run langgraph dev --no-browser --port 2024 — server up in ~1s, both graphs imported. Health check GET /ok → 200. ✅

  4. SDK smoke test — smoke_test_sdk.py (langgraph_sdk.get_sync_client, client.runs.stream(...)) streams updates events through agent → action → agent. ✅

  5. Same client against agent_with_helpfulness: agent → action → agent → helpfulness:HELPFULNESS:Y. ✅

    main.py = in-process (graph.invoke) for a dependency check without a server; smoke_test_sdk.py = real client hitting the running server over HTTP (the README's Part 1 step 4). Run: uv run python smoke_test_sdk.py [graph_id] ["question"].

Docs that unblock Part 1

  • Run a local server (/oss/python/langgraph/local-server): the CLI extra langgraph-cli[inmem] is what provides langgraph dev; without it the command is missing. A LangSmith API key is required even for purely local dev (the in-mem server phones home to submit metadata — see log line POST /v1/metadata/submit 204).
  • Application structure (/langsmith/application-structure): real langgraph.json keys.

⚠️ Gotchas (Part 1)

  1. assistantId: "agent" does not exist on the local server. If your langgraph.json includes an assistants block mapping a friendly name "agent"simple_agent, and you then stream against "agent", langgraph dev fails:

    UnprocessableEntityError: Invalid assistant: 'agent'. Must be either:
      - A valid assistant UUID, or
      - One of the registered graphs: simple_agent, agent_with_helpfulness
    

    Why: assistants is not a documented langgraph.json field. Per the Default assistants docs, each entry under graphs automatically gets a default assistant whose id is the graph key. POST /assistants/search on the local server returns exactly two, named simple_agent and agent_with_helpfulness — the assistants block is silently ignored. Use the graph id (simple_agent) as the assistantId in the SDK and in the frontend's useStream, or create a real named assistant via the API/UI and pass its UUID.

  2. Provider-specific tool-call shims. Some open-source models (e.g. gpt-oss via Fireworks, in session 10) emit malformed tool calls that need post-processing. OpenAI's native tool calling is well-formed, so no such shim is needed here — but keep it in mind if you swap the provider.

  3. RAG build is lazy + cached on purpose. The PDF load → chunk → embed → Qdrant build is wrapped in functools.lru_cache, so it runs once on the first tool call rather than at import. This keeps langgraph dev startup fast; otherwise the server blocks on embedding the whole PDF before it can serve. Trade-off: the first retrieve_information call is slow.

  4. langgraph dev is dev-only / in-memory. The startup banner literally says "For production use, please use LangSmith Deployment." State lives in memory and is lost on restart — fine for Studio debugging, not for serving traffic. That's the whole reason Parts 2–4 exist.


Part 2 — Deploy the agent on LangSmith 📄 (needs a paid account)

Cloud deployment needs a LangSmith account on a paid plan (Plus / the README's "~$40/mo"), plus Docker for the CLI path. There are three real paths (the README lists two):

Do you actually need LangSmith Plus? (requirement · how to check · sign up · cost)

Is it required? For Cloud agent deployments — langgraph deploy (Path A) or the GitHub → LangSmith UI (Path B) — yes, Plus plan or above. LangChain docs, Deploy to Cloud: "Agent deployments running on Cloud require a Plus plan or above." What does not need Plus:

  • Part 1 entirelylanggraph dev + Studio + the SDK smoke test run on the free Developer plan.
  • Tracing / observability — free tier (5k traces/mo).
  • Self-hosting the container yourself (Path C, langgraph up / standalone Docker) — needs a LangSmith API key but not the Plus cloud plan. (The full self-hosted platform is Enterprise.)

How to check which plan you're on: LangSmith → Settings → Billing and Usage / Plans and Billing (smith.langchain.com/settings/payments). Personal orgs are Developer; team/shared orgs are Plus. A "Upgrade to Plus" banner means you're not on it yet.

How to sign up / upgrade (self-serve): create a new team/shared organization (new orgs are required to be on Plus), or click Upgrade to Plus in Settings → Billing (add a card + business info). Enterprise = contact sales.

Cost (from langchain.com/pricing, ~mid-2026 — prices change, verify on the page):

Plan Price Free traces/mo Agent deployments?
Developer $0/seat · 1 seat · then pay-as-you-go 5,000 ❌ no deployment access
Plus $39/seat/mo · then PAYG 10,000 1 free Dev deployment, unlimited runs
Enterprise custom custom ✅ + self-hosted / hybrid

For this assignment the $39/seat Plus plan's free Dev deployment (unlimited runs) is enough. Beyond it: ~$0.005/run + uptime $0.0036/min (Production) or $0.0007/min (Development) — ⚠️ a left-running Production deployment quietly accrues cost.

No-Plus alternatives that still satisfy the deliverable: self-host with langgraph up (Path C), or record the Loom against Studio + the local server (Part 1) — the README accepts "LangGraph Studio debugging your agent" as one of the two demo options.

Path A — langgraph deploy CLI (one step, beta)

uv tool install langgraph-cli          # global CLI (separate from the project venv)
# add LANGSMITH_API_KEY=<your-langsmith-api-key> to your .env
uv run langgraph deploy                # or: langgraph deploy --name cat-health --deployment-type prod
  • Beta + needs Docker running. On Apple Silicon it also needs Docker Buildx to cross-compile to linux/amd64.
  • Creates a dev deployment named after the project directory by default.
  • Re-run langgraph deploy to update an existing deployment in place (finds it by name).
  • Manage with langgraph deploy list, langgraph deploy logs, langgraph deploy delete <ID>.

Path B — LangSmith UI from GitHub (this is where "auto-update on push" lives)

  1. Push the agent repo to GitHub.
  2. LangSmith → Deployments+ New DeploymentImport from GitHub.
    • A GitHub org owner must authorize the hosted-langserve app once per workspace.
  3. Pick repo, Git branch, and the path to langgraph.json (here: langgraph.json at root).
  4. Tick "Automatically update deployment on push to branch" — this is the README's "auto-update on push." Choose Development vs Production deployment type.
  5. Add env vars; mark OPENAI_API_KEY / TAVILY_API_KEY as secrets. A LangSmith tracing project is auto-created with the deployment's name.

You can also click Deploy straight from Studio while langgraph dev is running.

Path C — Self-hosted with Docker (langgraph up)

uv run langgraph up        # production-like: API + PostgreSQL + Redis, on port 8123
curl http://localhost:8123/ok   # -> {"ok":true}

Heavier than dev (real Postgres persistence, Redis pub/sub for streaming). langgraph build -t my-image builds just the image; langgraph dockerfile emits a Dockerfile to customize. You own scaling/uptime/auth.

What you get / what to copy down

A hosted API with the standard threads / runs / assistants routes. Copy two things for the frontend: the Deployment URL (https://<name>.us.langgraph.app) and a LangSmith API key. The agent runs behind that API; LangSmith traces every run.

Docs that ground Part 2

  • LangGraph CLI reference (/langsmith/cli): the dev / build / deploy / dockerfile / up command table.
  • Deploy to Cloud (/langsmith/deploy-to-cloud) and Deployment quickstart (/langsmith/deployment-quickstart): the UI-vs-CLI steps, hosted-langserve OAuth, and the auto-update-on-push checkbox.
  • Local dev & testing (/langsmith/local-dev-testing): the dev vs up comparison (Docker, ports 2024 vs 8123, in-mem vs Postgres).

⚠️ Gotchas (Part 2)

  1. langgraph deploy is beta and Docker-bound. It is not a pure-cloud build; it builds a Docker image locally first. No Docker (or no Buildx on Apple Silicon) → it fails.
  2. langgraph deploy ≠ free. Cloud deployments need a paid LangSmith workspace; the free tier covers langgraph dev and tracing only.
  3. Secrets belong in the deployment, not the repo. Set OPENAI_API_KEY etc. as LangSmith secrets (or Path-C .env), never committed. The project .gitignore already blocks .env.

Part 3 — Build a website that uses the agent ✅ (verifiable in a real browser)

Scaffold a Next.js app, add the secure passthrough route, build a useStream chat UI, and test it against your local agent — full path:

Browser (localhost:PORT)  ->  /api/* proxy (Next route, edge)  ->  agent (localhost:2024)
                                                               ->  retrieve_information (RAG over PDF)

When it works, the agent log shows POST /api/threads then POST /api/threads/{id}/runs/stream 200"Background run succeeded" graph_id=simple_agent, and the UI renders the human turn, the streamed tool result, and the final answer.

What to build

  • frontend/create-next-app (App Router, TS, Tailwind). Expect Next 16 / React 19.
  • frontend/app/api/[...path]/route.tsinitApiPassthrough({ apiUrl, apiKey, runtime: "edge" }).
  • frontend/app/page.tsx"use client" chat UI calling useStream({ apiUrl, assistantId: "simple_agent" }).
  • frontend/.env.localLANGGRAPH_API_URL=http://localhost:2024 (no key needed locally).

Docs that unblock Part 3

  • useStream setup (/oss/javascript/langgraph/frontend/...): hook shape { messages, submit, isLoading }, submit({ messages: [{ type: "human", content }] }). Crucially, the docs' own examples use assistantId: "simple_agent" — the graph id.
  • Next.js 16 bundled docs (node_modules/next/dist/docs/): the scaffold ships an AGENTS.md telling you to read these first. Turbopack is now default, Node 20.9+ required, and runtime: "edge" route handlers still work (only incompatible with Cache Components, which we don't use).
  • npm view to see the versions you actually got: @langchain/react, langgraph-nextjs-api-passthrough, @langchain/langgraph-sdk.

⚠️ Gotchas (Part 3) — this is where the real time goes

  1. The README's useStream import (@langchain/react) can silently hang. With the latest @langchain/react, useStream(...).submit(...) renders the optimistic human message but never fires the run — its promise hangs, no network request. That package ships a "v2-native stream runtime" that doesn't complete the handshake through the langgraph-nextjs-api-passthrough proxy. Fix: import the canonical, stable hook:

    // instead of:                              // use:
    import { useStream } from "@langchain/react";
    import { useStream } from "@langchain/langgraph-sdk/react";

    Same { messages, submit, isLoading } API. (Diagnose this kind of hang by adding onError/onCreated logging and watching the browser console + network tab.)

  2. The SDK client needs an ABSOLUTE apiUrl — a relative "/api" throws "Invalid URL". After switching hooks, submit may throw TypeError: Failed to construct 'URL': Invalid URL from ThreadsClient.create. The SDK builds requests with new URL(), which rejects a relative path. Fix — derive an absolute, same-origin URL at runtime so it still works on any deploy domain:

    const apiUrl =
      process.env.NEXT_PUBLIC_API_URL ??
      (typeof window !== "undefined" ? `${window.location.origin}/api` : "/api");
  3. assistantId must be the graph id simple_agent, not "agent" (same root cause as Part 1 gotcha #1). assistantId: "agent" 422s.

  4. The passthrough package warns it's legacy. At build/run it prints: "This is no longer the recommended way of handling authentication… implement custom authentication and routes in your LangGraph deployment." It still works, but the forward-looking pattern is custom auth/routes on the deployment (see the Advanced Activity). Silence with initApiPassthrough({ ..., disableWarningLog: true }).

  5. Fast Refresh hook-order error is a red herring. Hot-swapping a hook import mid-session can trigger "change in the order of Hooks" — it clears on a full page reload and isn't a code bug.

  6. Port coupling. NEXT_PUBLIC_API_URL=http://localhost:3000/api breaks if next dev picks another port (3000 is often already taken). Omitting it and falling back to a same-origin absolute URL (gotcha #2) is more robust.


Part 4 — Deploy the frontend on Vercel 📄 (needs an account)

The frontend is a standard Next.js app, so the Vercel path is the conventional one:

  1. Push frontend/ to GitHub (same repo as the agent or a separate one).

  2. Import at vercel.com/new. If the app isn't at the repo root, set Root Directory = frontend. Vercel auto-detects Next.js (build next build).

  3. Set Environment Variables (Project → Settings → Environment Variables):

    LANGGRAPH_API_URL = https://<your-deployment>.us.langgraph.app   # server-side
    LANGSMITH_API_KEY = <your-langsmith-api-key>                     # server-side, injected by the proxy
    # NEXT_PUBLIC_API_URL — optional; omit to use the same-origin "/api" fallback (recommended)
    

    LANGGRAPH_API_URL / LANGSMITH_API_KEY have no NEXT_PUBLIC_ prefix on purpose, so they stay server-only and never reach the browser bundle.

  4. Deploy, then verify end-to-end on the *.vercel.app URL: the UI streams, tool calls run against the deployed agent, and traces appear in LangSmith per run.

⚠️ Gotchas (Part 4)

  1. Only NEXT_PUBLIC_-prefixed env vars reach the browser. Keep the LangSmith key un-prefixed; the passthrough route reads it server-side. (Reasoning through why is the crux of Question #2.)
  2. Edge runtime + your deployment region. The route is runtime: "edge"; calls fan out from Vercel's edge to your LangGraph deployment region — fine, just adds a hop. Switch the route to runtime: "nodejs" if you ever need Node-only APIs.
  3. Redeploys on push. Connect the repo so Vercel redeploys the frontend on push, mirroring the agent's auto-update-on-push — keep both in sync.

Questions & Activities — reason it through yourself (no answers here)

These are the graded deliverables. Answer Q1/Q2 in README.md and demo the activities in your Loom. The prompts below are a method to get there, not the answer.

Question #1 — Why does LangSmith deploy your agent as an API backend only, and why still a separate Vercel frontend?

Ask yourself:

  • When you hit your deployed agent URL, what comes back — renderable HTML, or JSON/streamed events? What does that tell you about whether LangSmith can be your website?
  • List what the agent API exposes (threads? runs? assistants?). Is any of that a user interface?
  • If you pointed the browser straight at the LangSmith URL, what would the request need — and what does Part 3 / the secret discussion say about putting that in the browser?
  • Frame your answer as "two concerns, two hosts" and name each host's job in one line.

Question #2 — Why should the LangSmith API key live in a Next.js API route (server-side), not the browser?

Ask yourself:

  • Open devtools → Network + Sources on any site. What can you read? Could a secret compiled into client JS ever stay hidden?
  • In Next.js, which env vars reach the browser? What does NEXT_PUBLIC_ do, and why are LANGGRAPH_API_URL / LANGSMITH_API_KEY deliberately not prefixed while NEXT_PUBLIC_API_URL is?
  • Trace one request: browser → /api/* → ? Where does initApiPassthrough add the key — on the user's machine or on the server?
  • What could someone do with your key if they lifted it from the bundle? Let that consequence anchor your "why."

Activity 1 — Helpfulness loop in production

The repo ships agent_with_helpfulness (the second graph in langgraph.json). Deploy it and compare LangSmith traces for a query that passes vs. one that fails the helpfulness check. Ask yourself:

  • Trace the nodes: who answers, who runs tools, who judges? When does the judge send control back to the agent vs. to the end?
  • Run a clear query vs. a vague/under-specified one — which one loops? Where in the Studio/LangSmith trace do you see the judge decision and the retry edge firing?
  • What stops the loop from running forever? Find the safety limit in the code — what does it count, and where does it force the end?
  • The README asks: is the loop's logic any different in Studio vs. production, or is it the same graph in a different runtime? What actually differs (persistence, concurrency, how iterations show up in the trace tree)?

Advanced Activity — Auth & custom routes (optional)

Deliverable: describe (optionally implement) how to make each user see only their own threads. Ask yourself:

  • Hiding the chat page behind a login in React — does that actually stop someone from reading another user's threads through the API directly? Where must the real check live?
  • What would the server need on each request to scope threads to one user, and what could you stamp on a thread when it's created so reads can be filtered?
  • Where should the user's token be attached — the same server-side place the API key is injected, or the browser? Why?
  • Study the README's lsd-custom-route-react-ui reference for the custom-auth / custom-route pattern (langgraph_sdk.Auth, the auth key in langgraph.json).

Additional issues & environment notes (things that cost time)

Build / tooling

  1. Port 3000 already in use → EADDRINUSE. npm run dev can fail to bind 3000 if something else holds it; run on another port (--port 3030). This is exactly why a same-origin apiUrl (Part 3 gotcha #2/#6) matters — hardcoding localhost:3000 would break the browser calls.
  2. Restarting next dev via a broad pkill -f next can cascade and take down the wrapper process. Prefer killing by exact port/PID, or start fresh on a new port.
  3. create-next-app --no-turbopack is effectively ignored on Next 16 — Turbopack is the default bundler now. Nothing breaks; just don't expect Webpack.
  4. npm install may report moderate-severity advisories from transitive deps — normal for a fresh scaffold. Don't run npm audit fix --force blind; it can introduce breaking bumps.
  5. Versions come in ahead of the README's assumptions (Next 16 / React 19, @langchain/react v1.x). The Part 3 useStream hang is a direct consequence — pin versions if you need reproducibility.

Static type-checker (Pyright) diagnostics — all benign

None are runtime errors:

  • reportMissingImports on app.*, langchain_tavily, etc. — appear only before uv sync / before the editor selects the .venv interpreter. Gone after install.
  • DirectoryLoader(loader_cls=PyMuPDFLoader) "not assignable" — a known Pyright false positive; PyMuPDFLoader is a valid loader and runs fine.
  • _RAGState partial-dict return "not assignable" — LangGraph nodes legitimately return a partial state dict; the reducer merges it. Runtime is correct.
  • reportUnsupportedDunderAll in __init__.py — cosmetic; submodules import fine by path.

Streaming output detail (not a bug)

In the raw SSE from runs/stream, early messages chunks show invalid_tool_calls with partial fragments. That's just the tool-call arguments being streamed token-by-token and assembled — the final updates event shows the complete, valid tool call. Don't mistake mid-stream partial tool-call chunks for errors.

LangSmith phone-home during local dev

langgraph dev (in-mem) still calls https://api.smith.langchain.com/v1/metadata/submit (a 204 No Content in the log) and requires a LANGSMITH_API_KEY even though state is purely local. Expected behavior — it submits graph metadata, not your data.


Final status

Part Status How to prove it
1. Package + langgraph dev + SDK ✅ verifiable server up on :2024, both graphs, SDK stream, helpfulness loop
2. Deploy on LangSmith 📄 needs paid account grounded CLI/UI/self-host steps
3. Next.js frontend + proxy + useStream ✅ verifiable browser run, full stream through /api
4. Deploy on Vercel 📄 needs account grounded steps + env wiring
Q&A + Activities 🧠 your turn answer in README.md; demo in the Loom

Top takeaways

  1. The run identifier is the graph id, end to end. langgraph.json's assistants block is not materialized locally; use simple_agent in the SDK and useStream, not "agent".
  2. Match the frontend libs to the proxy. Use @langchain/langgraph-sdk/react's useStream (not the newer @langchain/react) with langgraph-nextjs-api-passthrough, and give the SDK an absolute apiUrl.
  3. The secret stays server-side. The passthrough route is the security boundary — though LangChain now nudges you toward custom auth on the deployment itself.
"""SDK smoke test against a running agent server (README Part 1, step 4).
Streams a run through the LangGraph SDK the same way a production client would —
this is the server-based counterpart to ``main.py`` (which invokes the graph
in-process). Start the server first:
uv run langgraph dev # serves http://localhost:2024
then run this against it:
uv run python smoke_test_sdk.py
uv run python smoke_test_sdk.py agent_with_helpfulness "What vaccines does a kitten need?"
Usage:
python smoke_test_sdk.py [ASSISTANT_ID] [QUESTION]
Notes:
* ASSISTANT_ID is the GRAPH ID from langgraph.json — ``simple_agent`` (default)
or ``agent_with_helpfulness``. It is NOT ``"agent"``: the ``assistants`` block
in langgraph.json is not materialized on the local server, so ``"agent"`` 422s
with "Invalid assistant". (See LEARNING_JOURNEY.md, Part 1 gotcha #1.)
* Uses ``get_sync_client`` so the README's plain ``for chunk in ...`` loop works
without asyncio boilerplate.
"""
from __future__ import annotations
import os
import sys
from langgraph_sdk import get_sync_client
URL = os.environ.get("AGENT_URL", "http://localhost:2024")
def main() -> None:
assistant_id = sys.argv[1] if len(sys.argv) > 1 else "simple_agent"
question = (
sys.argv[2]
if len(sys.argv) > 2
else "How often should I deworm my cat?"
)
client = get_sync_client(url=URL)
print(f"→ {URL} assistant={assistant_id!r}")
print(f"? {question}\n")
for chunk in client.runs.stream(
None, # threadless run
assistant_id,
input={"messages": [{"role": "human", "content": question}]},
stream_mode="updates",
):
if not isinstance(chunk.data, dict):
continue
for node, payload in chunk.data.items():
messages = payload.get("messages") if isinstance(payload, dict) else None
if not messages:
continue
last = messages[-1]
content = last.get("content") if isinstance(last, dict) else ""
tool_calls = last.get("tool_calls") if isinstance(last, dict) else None
if tool_calls:
names = ", ".join(tc.get("name", "?") for tc in tool_calls)
print(f"[{node}] → tool call: {names}")
elif content:
print(f"[{node}] {str(content)[:400]}")
if __name__ == "__main__":
main()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment