Created/edited by GitHub Copilot with human review/feedback by Avi Levin.

Copilot canvas apps: the missing mental model

This guide is for the moment after you have read GitHub's Working with canvas extensions in the GitHub Copilot app, seen a create-canvas demo, and want to build something real with it.

The demo makes the feature look simple: Copilot creates an extension, opens a panel, and shows an HTML app. That is real. The confusing part starts when you expect the app to remember things, react to agent changes, or ask the agent to continue work.

The short version:

A canvas is the runtime contract for opening a panel and routing actions. A canvas app is the extension-owned web app you build on top of that contract.

The quick model

If you only remember one page, remember this:

The host/runtime opens canvas panels and routes declared capabilities/actions.
The extension process owns the app logic, HTTP server, state, and action handlers.
The iframe is just browser JavaScript loaded from the URL returned by open().
instanceId means which panel is open, not which data is being edited.
Durable state needs its own identity: statePath, documentId, repo path, database key, or service record.
The agent does not see the iframe DOM. It reads state through capabilities/actions or durable files.
Live refresh is an app design choice. The extension can use refresh, polling, SSE, file watching, or an external subscription.

instanceId = the visible panel
statePath / documentId = the durable data

Names you may see in the GitHub docs

The public docs use product-facing names. This guide sometimes uses lower-level implementation names.

GitHub docs term	Implementation term here	Meaning
Canvas extension	Extension directory / `extension.mjs`	The code package that contributes the canvas.
Capability	Canvas action / declared action	An agent-callable operation such as `get_board`, `add_card`, or `move_card`.
UI controls	Iframe buttons/forms	Human-facing controls that usually call the extension's HTTP routes.
Right side panel	Canvas instance	One open visible panel, commonly tracked with `instanceId`.
Shared state / artifacts	Durable state identity	The persisted data the canvas edits: artifact file, `statePath`, repo file, database row, or service record.
Project scope / user scope	Extension installation scope	Where the extension code lives. This is separate from where each canvas's data lives.

What the demo actually made

A typical create-canvas scaffold gives you this shape:

extension.mjs
  imports joinSession and createCanvas
  starts a loopback HTTP server
  calls joinSession(...)
  declares createCanvas(...)
  implements open(ctx)
  returns a URL for the iframe
  declares capabilities/actions

In a scaffolded extension, @github/copilot-sdk/extension is the SDK import the scaffold uses for extension wiring. You normally get this from the generated extension; if you are hand-writing an extension, make sure the package is available through the scaffolded package.json or your extension's dependencies. The exact generated file and helper names may change; treat this code as the current shape to orient yourself, not as a replacement for the generated extension.

import { createServer } from "node:http";
import { createCanvas, joinSession } from "@github/copilot-sdk/extension";

const servers = new Map();

async function startServer(instanceId, input) {
  let entry = servers.get(instanceId);
  if (entry) return entry.url;

  const server = createServer((req, res) => {
    // Serve iframe HTML, static assets, and whatever app routes you choose.
    res.writeHead(200, { "content-type": "text/html; charset=utf-8" });
    res.end("<!doctype html><h1>My board</h1>");
  });

  await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
  const { port } = server.address();
  const url = `http://127.0.0.1:${port}/`;
  servers.set(instanceId, { server, url, input });
  return url;
}

const session = await joinSession({
  canvases: [
    createCanvas({
      id: "my-board",
      actions: [
        {
          name: "get_board",
          description: "Return the current board state.",
          handler: async (ctx) => loadBoard(ctx.input?.statePath),
        },
      ],
      open: async (ctx) => ({
        title: "My board",
        url: await startServer(ctx.instanceId, ctx.input),
      }),
    }),
  ],
});

ctx.input is the object passed when the canvas is opened. For a stateful app, it often carries the durable state identity:

{ "statePath": "<session-or-artifact-path>/my-board.json" }

Action handlers also receive a ctx with their own input; use the same input shape when the action needs to operate on the same durable state.

The session returned by joinSession(...) is also the object the extension can capture for session-level calls. For example, a button route can later re-enter chat with:

await session.send({
  prompt: `Read the latest board via get_board. State file: ${statePath}`,
});

The scaffold/server pattern binds to 127.0.0.1 on a random available port and returns that URL from open(ctx). For serious apps, avoid treating that local HTTP endpoint as a public API: keep it loopback-only, keep routes narrow, and pass a per-open token or other unguessable value if you are worried about other local pages posting to it.

That proves the plumbing works. The host can discover the canvas, ask the extension to open it, and load the returned URL in a panel.

It does not automatically solve the app design:

where durable state belongs
how two open panels share the same data
how an open iframe stays current after state changes
how iframe actions and agent actions avoid drifting apart
how a button in the iframe hands work back to the agent

Those choices belong to your extension.

The three boxes

Every canvas app has three runtime contexts:

flowchart LR
    subgraph Host["Host / runtime"]
        Agent["agent / model loop"]
        Tools["generic canvas tools"]
    end

    subgraph Extension["Extension process"]
        Open["open(ctx)"]
        Actions["canvas actions"]
        Server["loopback HTTP server"]
        State["state helpers"]
    end

    subgraph Iframe["Browser iframe"]
        UI["HTML / CSS / JS"]
    end

    Agent --> Tools
    Tools -- "open canvas" --> Open
    Tools -- "invoke action" --> Actions
    Open -- "returns URL" --> Tools
    Tools -- "loads URL" --> UI
    UI -- "fetch(...)" --> Server
    Server --> State
    Actions --> State

The iframe boundary is the important one. The iframe has no SDK session object, no file-system access, and no direct invoke_canvas_action. It talks to the extension over ordinary browser APIs, usually fetch().

The iframe is a real web app

The canvas surface is not a special Copilot UI component. It is an HTML page loaded from the URL returned by open(ctx).

For a throwaway demo, inline HTML in extension.mjs is fine. Once the surface has real behavior, split it out:

extension.mjs   # backend wiring, capabilities/actions, HTTP routes, state
index.html      # browser UI
assets/...      # optional CSS, JS, images, bundled libraries

Inside the iframe, use normal browser tools:

HTML for structure
CSS for layout and app theme tokens
JavaScript for interaction
fetch(...) for commands back to the extension
EventSource(...) or another subscription mechanism when open panels should update live

Use a small reactive layer if it helps. Alpine is enough for many canvas apps; React is fine if the surface really needs it. Avoid building a full SPA unless the workflow actually needs one.

The browser constraints still matter: no Node APIs, no file system, no SDK session, and no direct CopilotSession. The extension backend owns those.

Keeping open views current

Start simple: refresh/reopen is enough for demos.

When the canvas starts feeling like an app, add a publish path:

state-changing action
  -> save durable state
  -> publish state or delta
  -> iframe applies update

SSE is a good default because it is simple browser technology. The route name is your choice; /events is just a common convention:

const events = new EventSource("/events");
events.addEventListener("state", (event) => render(JSON.parse(event.data)));
events.addEventListener("delta", (event) => applyDelta(JSON.parse(event.data)));

Choose the transport that fits the app. If state changes outside the iframe, the iframe needs a way to hear about it.

How messages move

There are four useful paths:

Agent -> extension: the agent calls a declared capability/action with invoke_canvas_action.
Iframe -> extension: the iframe calls an app-defined loopback HTTP route, often something like POST /api/action.
Extension -> iframe: the extension refreshes, pushes, or publishes state changes to open views.
Extension -> agent: the extension can use the CopilotSession captured from joinSession(...) to send a new prompt into the session.

The healthiest pattern is to make the agent path and iframe path converge:

flowchart LR
    Agent["agent action"] --> Router["routeAction(...) app helper"]
    Iframe["iframe HTTP command"] --> Router
    Router --> Domain["shared app logic"]
    Domain --> Store["durable state"]
    Domain --> Notify["optional UI update"]

routeAction(...) is not an SDK primitive; it is a useful app helper. If the HTTP route and the canvas action each implement their own version of the same operation, they will eventually disagree.

Where state lives

The panel is not the data.

Use instanceId to reopen, focus, reload, or close the visible panel. Use a separate durable identity for the content the panel is showing.

Good durable identities include:

a session/artifact file path for one-conversation artifacts
a user-global file for personal dashboards or preferences
a repo path for project-owned documents
a database key or service record for shared state

The GitHub docs mention optional JSON artifacts, for example files under an artifacts directory. Treat that as one concrete storage choice. The broader rule is to choose a durable owner that matches the data's lifetime.

Memory still has a role. Extension module memory is useful for open servers, subscriber lists, locks, and short-lived caches. Iframe memory is useful for local UI state. Neither should be the only copy of state that the user expects to keep.

A minimal app pattern

Once the demo needs to persist state, a minimal real app usually looks like this:

extension.mjs
  resolve durable identity from open(ctx).input
  load or create persisted state
  start/reuse a loopback server
  declare capabilities/actions for agent-side operations
  expose HTTP routes for iframe-side operations

iframe HTML/JS
  render current state
  call fetch(...) for edits and buttons
  refresh or subscribe for updates

For a throwaway demo, rendering from memory may be fine. For a workflow the user will rely on, decide the durable state identity early.

A robust app pattern

The minimal pattern becomes robust when you add a few design rules.

One action catalog. Define the app's operations once, then let both callers use them:

agent -> invoke_canvas_action -> routeAction
iframe -> app HTTP route      -> routeAction

One write-and-publish path. If a change should be visible in an open iframe, the state-changing write should also publish an update. The transport could be SSE, polling, refresh, or something else. The important point is that persistence and notification are deliberately connected.

Conflict semantics. If the agent and the iframe can both write, think about stale writes. The simplest version may just re-read before writing. A stronger version uses item revisions, document revisions, locks, or atomic writes.

One source of truth. Avoid parallel sources of truth, such as keeping a SQL table and a canvas state file in sync manually. Fallback modes are fine; simultaneous truth owners are not.

Bounded recovery. If the UI or extension gets stale, preserve durable state and retry deliberately. Do not build a workflow that depends on endless reload/reopen loops.

You do not need all of this on day one. But if the app becomes part of a workflow, these are the pieces that keep it from feeling flaky.

Worked example: a PR review board

The example behind this guide is a PR review board canvas.

The app helps review pull request comments. The agent gathers comments from a PR and writes rows into a board. The user triages those rows in the iframe: suggested fixes, confirmed work, completed items, notes, and draft replies. Then the agent executes the confirmed work.

In that app:

canvasId is pr-review-board
extensionId is user:pr-review-board
instanceId is usually pr-review-<PR_ID>
durable state is a JSON file in the session workspace
the iframe sends commands with POST /api/action
the agent reads and writes through capabilities/actions
open iframes stay current through pushed updates

The extension code can be global while each board's data stays session-specific:

Extension code:
  global/user extension

Session board data:
  <session.workspacePath>\files\pr-review-board-<PR_ID>.json

That is the general lesson: code can be shared, but state belongs to the thing that owns the data.

Example: Take confirmed handoff

The PR board has a Take confirmed button. The user clicks it when the board contains the rows they want the agent to execute.

You might initially consider dumping all confirmed rows into chat as JSON. That works for a tiny board, but it makes the chat message a stale snapshot and can bloat the conversation.

The better pattern is to send routing information and let the agent pull canonical state:

sequenceDiagram
    participant UI as iframe button
    participant Ext as extension backend
    participant Chat as Copilot session
    participant Agent as agent turn
    participant Canvas as canvas action

    UI->>Ext: POST /api/action send_confirmed
    Ext->>Chat: session.send({ prompt })
    Chat->>Agent: new turn
    Agent->>Canvas: invoke_canvas_action("take_confirmed")
    Canvas-->>Agent: latest confirmed rows

Here session is the CopilotSession object returned by joinSession(...), and prompt should name the state identity and the canvas instance to read from.

The principle is:

When a canvas re-enters chat, send enough information for the agent to find and read canonical state. Do not smuggle the canonical state itself through chat unless it is intentionally tiny and ephemeral.

The full app loop

Once the pieces are in place, the useful mental model is two input adapters, one state layer, one update path, and an optional session handoff.

flowchart TB
    subgraph Host["Host / runtime"]
        Agent["agent / model loop"]
        Tools["generic canvas tools"]
    end

    subgraph Extension["Extension process"]
        CanvasActions["declared capabilities/actions"]
        Http["loopback HTTP server"]
        Router["routeAction(...) app helper"]
        State["state helpers"]
        Store["state file / artifact / service"]
        Events["event subscribers"]
        Session["captured CopilotSession"]
    end

    subgraph Iframe["Browser iframe"]
        UI["HTML UI"]
        Fetch["fetch(app route)"]
        EventStream["EventSource(app route)"]
    end

    Agent --> Tools
    Tools --> CanvasActions
    CanvasActions --> Router

    UI --> Fetch
    Fetch --> Http
    Http --> Router

    Router --> State
    State --> Store
    State --> Events
    Events --> EventStream
    EventStream --> UI

    Http --> Session
    Session --> Agent

Checklist for generated canvas apps

Use this when a generated canvas "works" and you need to decide whether it is ready to rely on.

Can you name the durable state identity?
Is durable state keyed by something other than instanceId?
Is iframe state treated as disposable?
Do agent actions and iframe HTTP routes converge on the same logic?
Do state-changing writes update open views somehow?
Can the agent read canonical state through a declared canvas action?
Does app -> agent handoff send routing information instead of large stale snapshots?
Are conflicting writes detected or deliberately allowed?
Is there exactly one source of truth for the current workflow?
Is recovery bounded instead of reload/reopen forever?
Can you validate behavior through capabilities/actions, not by assuming the agent sees the iframe DOM?

Advanced notes

You can stop at the checklist for most first-pass canvas apps. These topics matter when the canvas becomes a serious workflow surface.

Alternate sync producers. File watching, WebSockets, polling, and external subscriptions can all feed the same write-and-publish path. Use them when the durable state can change outside normal iframe or agent actions.

Concurrency. If two actors can edit the same state, decide what stale write means. The PR board uses item revisions for row edits, board revisions for structural edits, and serialized mutating writes per durable state file.

Fallback backends. A workflow may have a canvas backend and a SQL/chat fallback, but only one should own truth in a session. Parallel trackers create reconciliation problems.

Recovery. Reloading an extension is sometimes necessary, but it should not become the workflow. Preserve the durable state, retry once, and ask the user rather than looping.

Cross-session state. Session files are not the only option. A canvas can be user-global, repo-backed, document-backed, or service-backed if that matches the data's lifetime.

What to remember

The runtime opens panels and routes declared actions. The extension owns the app.

If you build only for the demo, the iframe can feel like the app. If you build for real use, the durable state and action layer are the app; the iframe is one view of it, and the agent is another caller into it.

Arithmomaniac/copilot-canvas-app-vibe-doc.md

Select an option

No results found