Skip to content

Instantly share code, notes, and snippets.

@igalshilman
Last active May 16, 2026 12:43
Show Gist options
  • Select an option

  • Save igalshilman/89c3ea9781cf0fe7d4ece809d3917d62 to your computer and use it in GitHub Desktop.

Select an option

Save igalshilman/89c3ea9781cf0fe7d4ece809d3917d62 to your computer and use it in GitHub Desktop.

restate-sdk-gen internals: fibers, the scheduler, and the stack

A walkthrough of how user generators, fibers, and the scheduler interact in @restatedev/restate-sdk-gen. Aimed at JS engineers who already know Restate and durable execution; we skip framing on journals/replay and focus on the runtime model.

Source files referenced throughout:

  • src/scheduler.ts — orchestration
  • src/fiber.ts — execution
  • src/future.ts — handles
  • src/operation.ts — descriptions
  • src/current.ts — the synchronous slot
  • src/awaitable.ts — the SDK boundary

0. The central insight

There is exactly one place in this system that awaits anything: the scheduler's main loop. Everything else — every user generator, every combinator implementation, every primitive — is synchronous from the JS engine's point of view. The pause/resume illusion that lets user code look like:

const a = yield* run(() => fetchA());
const b = yield* run(() => fetchB());
return a + b;

…is not built on await at all. It's built on generator suspension. yield returns control to the iterator's caller synchronously; the caller (the scheduler) gets a value back from iterator.next() and decides what to do.

That decision happens on the same call stack as the yield. No microtask boundary in between. No Promise.resolve().then(…). No AsyncLocalStorage propagation. The scheduler installs a pointer, calls iterator.next(), the user code runs, hits yield, and control returns up the same stack — slot still set, no context to plumb.

Everything in the rest of this document is downstream of that.


1. The cast

Four types. Three are public.

Operation<T> — a lazy description of work. Constructed by gen() from a generator factory () => Generator<…, T>, or by primitive nodes (Leaf, AwaitRace) that the scheduler dispatches on. Running the operation creates a fresh iterator each time; the operation itself is reusable.

Future<T> — an eager, memoized handle to in-flight work. Two internal backings, indistinguishable to user code:

  • journal-backed — wraps a RestatePromise from the SDK (ctx.run, ctx.sleep, ctx.awakeable, …). Resolution is observed by Restate.
  • local-backed — wraps a WaitTarget living in JS heap memory. Two kinds today: a Fiber (settled when its iterator returns) and a Channel (settled when send() is called). Future in-memory primitives slot in here without touching the scheduler.

Fiber<T> (internal) — one strand of cooperative execution. Owns the iterator, the lifecycle state, and a waiter list. Implements WaitTarget<T> so spawned fibers' Futures can be awaited.

Scheduler — owns the set of live fibers, the ready queue, the AwaitableLib, the main-loop race, and the cancellation broadcast.

classDiagram
    class Scheduler {
        +Set~Fiber~ fibers
        +Fiber[] ready
        +AwaitableLib lib
        +AbortController abortController
        +contextSlot unknown
        +run(op) Promise~T~
        +spawn(op) Future~T~
        +all/race/any/allSettled(...)
        +markReady(f)
        +markDone(f)
    }
    class Fiber {
        -Iterator it
        -FiberState state
        -Waiter[] waiters
        +advance()
        +wake(resume)
        +parkedSources()
        +awaitCompletion(waiter)
        +isDone()
        +settledValue()
    }
    class Future {
        -Backing backing
        -Symbol.iterator
    }
    class Operation {
        +Symbol.iterator
    }

    Scheduler "1" o-- "*" Fiber : owns
    Fiber ..|> WaitTarget
    Future --> Backing
    Backing <|-- JournalBacking
    Backing <|-- LocalBacking
    LocalBacking --> WaitTarget : target
    JournalBacking --> Awaitable : promise
    Future ..|> Operation : extends
Loading

2. The synchronous current-fiber slot

This is the move that lets you write:

gen(function* () {
  const x = yield* run(() => fetchA(), { name: "a" });
  return x;
});

without an ops parameter and without AsyncLocalStorage.

src/current.ts:

let CURRENT: unknown = null;

export function setCurrent(value: unknown): unknown {
  const prev = CURRENT;
  CURRENT = value;
  return prev;
}

export function clearCurrent(prev: unknown): void {
  CURRENT = prev;
}

export function getCurrent(): unknown {
  if (CURRENT === null) {
    throw new Error("…called outside an active fiber.");
  }
  return CURRENT;
}

A module-level pointer. Fiber.advance sets it on entry, clears it in finally:

// fiber.ts
advance(): void {
  if (this.state.kind !== "ready") return;
  const prevSlot = setCurrent(this.sched.contextSlot);
  try {
    // …step iterator, dispatch yields…
  } finally {
    clearCurrent(prevSlot);
  }
}

The free functions (run, sleep, all, …) read it:

// free.ts (excerpt)
export function currentOps(): RestateOperations {
  return getCurrent() as RestateOperations;
}

export const sleep = (d: Duration | number) => currentOps().sleep(d);
export const run = (action, opts) => currentOps().run(action, opts);
// …

Why this is safe

User generators yield; they don't await. iterator.next() returns synchronously. Between setCurrent and clearCurrent there is no async boundary — the entire span is sync JS, and Node never preempts sync JS. Two concurrent execute() calls in the same process interleave only at scheduler-level await boundaries (the main loop's race), never in the middle of a fiber's body.

How it fails

A free function called outside an advancing fiber — module init, inside an ops.run async closure that resolves after the fiber returned, a setTimeout callback — reads null and throws. Loud and immediate, never silent corruption. This is the deliberate trade for not paying the ALS tax.

Compare to alternatives

Approach How user gets the scheduler Cost
(ops) => parameter Plumb explicitly Every gen body takes a param
AsyncLocalStorage Implicit propagation Real perf hit; propagation gotchas
Sync slot Implicit, sync-scoped Hard error if used outside a fiber

3. A Fiber's state machine

type FiberState =
  | { kind: "ready";  resume: Settled | null }
  | { kind: "parked"; promises: PromiseSource[] }
  | { kind: "done";   settled: Settled };
                       ┌─────────────────────────┐
                       │                         │
                       ▼                         │
                   ┌───────┐  yields Leaf/   ┌───────┐
       new Fiber → │ ready │  AwaitRace      │parked │
                   │       │ ──────────────► │       │
                   └───────┘                 └───────┘
                       │                         │
                       │ iterator returns        │ fire(settled)
                       │ or throws               │ → wake() → ready
                       ▼                         │
                   ┌───────┐                     │
                   │ done  │ ◄───────────────────┘
                   └───────┘
                       │
                       ▼
                  notify waiters
                  markDone(scheduler)

ready carries the value to feed back into the generator on the next iterator.next(resume). null means "first run." parked carries a list of PromiseSources — pairs of {promise, fire} that the scheduler will race against everything else parked. done carries the final outcome.

Three transitions, each in one place in the code:

Transition Trigger Code
ready → parked yield Leaf or AwaitRace parkOnLeaf / parkOnAwaitRace
parked → ready wake(settled) Fiber.wake
any → done iterator returns or throws Fiber.finish

4. A primitive walkthrough: yield* future

Before the worked example, look at the smallest mechanism — what yielding a Future actually does.

A Future<T> is an Operation<T>. Its iterator yields one primitive: a Leaf node carrying a back-reference to itself.

// future.ts (essence)
export function makeFuture<T>(backing: Backing<T>): Future<T> {
  const future = {
    [futureBacking]: backing,
    *[Symbol.iterator]() {
      return (yield leafOp) as T;
    },
  };
  const leafOp = makePrimitive({ _tag: "Leaf", future });
  return future;
}

When user code writes yield* future, the yield* desugars to a loop that delegates each yielded value up to the outer generator. The generator function's iterator passes the Leaf node out through iterator.next() to Fiber.advance. advance dispatches on _tag:

// fiber.ts (essence)
switch (node._tag) {
  case "Leaf":      outcome = this.parkOnLeaf(node); break;
  case "AwaitRace": outcome = this.parkOnAwaitRace(node.futures); break;
}
if (outcome === null) return;           // parked, suspend
resume = outcome;                       // synchronous short-circuit

parkOnLeaf looks at the backing:

private parkOnLeaf(leaf: LeafNode<unknown>): Settled | null {
  const backing = getBacking(leaf.future);
  if (backing.kind === "journal") {
    this.state = {
      kind: "parked",
      promises: [{ promise: backing.promise, fire: (s) => this.wake(s) }],
    };
    return null;
  }
  // local-backed: ask the target if it's already done
  const settled = backing.target.awaitCompletion((s) => this.wake(s));
  if (settled !== null) return settled;   // sync short-circuit
  this.state = { kind: "parked", promises: [] };
  return null;
}

Two paths worth seeing:

  1. Journal-backed → always park. We can't peek a RestatePromise synchronously; we hand {promise, fire} to the scheduler, return null, suspend.
  2. Local-backed, target already done → short-circuit. awaitCompletion returns the settled value immediately. We return it from parkOnLeaf so advance feeds it back into the iterator on the next step of the same while(true) loop — no park, no wake.

The short-circuit is what makes yield* future cheap when the future is already resolved (re-yielding a settled future, looking up a local fiber that already returned). No promise machinery touched.


5. Worked example: the simplest non-trivial workflow

const greeter = restate.service({
  name: "greeter",
  handlers: {
    greet: async (ctx, name) =>
      execute(
        ctx,
        gen(function* () {
          const a = yield* run(() => fetchA(), { name: "a" });
          return `hi ${name}, got ${a}`;
        })
      ),
  },
});

Trace the full lifecycle from handler invocation to handler return.

Phase 1 — handler invocation

The SDK calls greet(ctx, name). The handler is async; it awaits execute(ctx, op). execute constructs a fresh scheduler, wires the contextSlot, and calls scheduler.run(op).

// restate-operations.ts
export async function execute<T>(context, op): Promise<T> {
  const sched = new SchedulerClass(defaultLib);
  sched.contextSlot = new RestateOperations(context, sched);
  return sched.run(op);
}

Stack at the moment scheduler.run is invoked:

┌──────────────────────────────┐
│ scheduler.run(op)            │   async, just entered
├──────────────────────────────┤
│ execute(ctx, op)             │   async, awaiting
├──────────────────────────────┤
│ greet handler                │   async, awaiting
├──────────────────────────────┤
│ SDK invocation dispatch      │
└──────────────────────────────┘

Phase 2 — first advance

scheduler.run does this:

async run<T>(op: Operation<T>): Promise<T> {
  const main = this.createFiber(op);    // adds to fibers + ready queue
  this.drainReady();                    // drives every ready fiber to park or done

  while (this.fibers.size > 0) {
    // …main loop…
  }
  // …return main's settled value…
}

drainReady shifts fibers off the queue and calls advance() on each, synchronously, on the run() call stack:

private drainReady(): void {
  while (this.ready.length > 0) this.ready.shift()!.advance();
}

fiber.advance sets the slot, calls iterator.next(undefined) (first call, no resume value). The user generator body begins:

function* () {
  const a = yield* run(() => fetchA(), { name: "a" });
  //              └─ free function reads currentOps() → returns Future<string>
  //              └─ yield* iterates the Future, which yields a Leaf
  return `hi ${name}, got ${a}`;
}

Inside run(…), currentOps() reads the slot, finds the RestateOperations instance, calls .run(action, opts). That method returns a journal-backed Future<string> wrapping a RestatePromise from ctx.run(...). The Future is constructed but the RestatePromise is already in flight — that's what "eager Future" means.

yield* future delegates to the Future's iterator, which yields the Leaf op. The yielded value bubbles up to iterator.next() in advance. Stack:

┌──────────────────────────────┐
│ <user gen, paused at yield*> │   not on stack — it's a suspended iterator
└──────────────────────────────┘
            ▲
            │ next() returned {done:false, value: Leaf}
            │
┌──────────────────────────────┐
│ fiber.advance                │   inspects the Leaf, calls parkOnLeaf
├──────────────────────────────┤
│ drainReady                   │
├──────────────────────────────┤
│ scheduler.run                │   sync portion, before the while loop
├──────────────────────────────┤
│ execute                      │
├──────────────────────────────┤
│ greet handler                │
└──────────────────────────────┘

parkOnLeaf sees kind: "journal", builds the PromiseSource:

this.state = {
  kind: "parked",
  promises: [{ promise: backing.promise, fire: (s) => this.wake(s) }],
};
return null;   // tells advance to stop

advance returns. drainReady has nothing left in the queue, returns. Control returns to scheduler.run's while (this.fibers.size > 0) loop.

Phase 3 — parked, awaiting

The scheduler collects parked sources from every live fiber, builds a tagged race, and awaits the lib:

while (this.fibers.size > 0) {
  const items: PromiseSource[] = [];
  for (const f of this.fibers) {
    for (const src of f.parkedSources()) items.push(src);
  }

  const tagged = items.map(({ promise }, i) =>
    promise.map((v, e) => e !== undefined
      ? { i, ok: false, e }
      : { i, ok: true, v })
  );

  raceWinner = await this.lib.race(tagged);
  // …dispatch winner, drainReady, continue…
}

This await is the only suspension point in the whole system. Stack unwinds back to the SDK / event loop. Heap state:

Heap:
  Scheduler {
    fibers: { mainFiber },
    ready:  [],
    abortController: AbortController { signal: <not aborted> },
  }
  mainFiber: Fiber {
    state: { kind: "parked",
             promises: [{
               promise: RestatePromise(run "a"),
               fire: (s) => this.wake(s),
             }] }
    it: <iterator suspended at yield* future>
  }

  (running inside Restate runtime)
  RestatePromise(run "a")  ◄── eventually resolves with "hello from A"
sequenceDiagram
    actor SDK
    participant H as handler
    participant E as execute
    participant S as scheduler.run
    participant DR as drainReady
    participant F as Fiber.advance
    participant IT as user iterator

    SDK->>H: invoke
    H->>E: execute(ctx, op)
    E->>S: scheduler.run(op)
    S->>S: createFiber(op)
    S->>DR: drainReady()
    DR->>F: advance()
    F->>F: setCurrent(ops)
    F->>IT: it.next(undefined)
    IT-->>F: {done:false, value: Leaf(future)}
    F->>F: parkOnLeaf → state=parked
    F-->>DR: return
    DR-->>S: return (queue empty)
    S->>S: await lib.race(tagged)
    Note over S: suspend
Loading

Phase 4 — resolution and wake

The Restate runtime resolves the RestatePromise. The lib's race settles. scheduler.run resumes as a microtask. The race winner has { i: 0, ok: true, v: "hello from A" }. The scheduler peels off the index and fires the corresponding source:

items[i]!.fire(settled);
this.drainReady();

fire is the closure (s) => this.wake(s) from when the source was registered. wake transitions the fiber to ready and pushes it onto the scheduler's ready queue:

wake(resume: Settled | null): void {
  if (this.state.kind === "done") return;
  this.state = { kind: "ready", resume };
  this.sched.markReady(this);
}

Then drainReady runs again, shifts the main fiber, calls advance. Inside advance's while(true) loop:

let resume: Settled | null = this.state.resume;   // = { ok: true, v: "hello from A" }
while (true) {
  let next: IteratorResult<unknown, unknown>;
  try {
    next = stepIterator(this.it, resume);   // it.next("hello from A")
  } catch (e) {
    this.finish({ ok: false, e });
    return;
  }
  if (next.done) {
    this.finish({ ok: true, v: next.value });
    return;
  }
  // …
}

stepIterator calls it.next("hello from A"). The generator resumes inside yield* future (the inner iterator), which returns the value, which becomes the value of yield* run(…) in user code:

const a = "hello from A";     // ← that's where we are now
return `hi ${name}, got ${a}`;

The generator hits its return. it.next returns {done: true, value: "hi <name>, got hello from A"}. advance calls finish({ ok: true, v: ... }), which transitions to done and notifies waiters:

private finish(settled: Settled): void {
  this.state = { kind: "done", settled };
  const waiters = this.waiters;
  this.waiters = [];
  for (const w of waiters) w(settled);
  this.sched.markDone(this);
}

markDone removes the fiber from the live set. drainReady returns. scheduler.run's while loop sees this.fibers.size === 0, exits, reads main.settledValue(), and resolves its promise with the string.

execute's await resolves with the same value. The handler's await resolves. The SDK records the journal and returns to the caller.


6. Stack vs heap, side by side

The thing that surprises people: there is no JS call stack that holds the user generator while it's parked. The iterator is a heap object; its frames live inside it. When the fiber is parked, the JS stack above the scheduler's await is empty (modulo the SDK's own machinery).

SYNCHRONOUS PHASES (advance running):

  Stack                            Heap
  ┌────────────────────────┐       ┌──────────────────────────────┐
  │ user gen body          │       │ Fiber:                       │
  │   inside run() / yield*│       │   state: ready (advancing)   │
  ├────────────────────────┤       │   it: <currently running>    │
  │ iterator.next()        │       └──────────────────────────────┘
  ├────────────────────────┤       Scheduler:
  │ fiber.advance          │         fibers: { fiber }
  ├────────────────────────┤         ready:  []
  │ drainReady             │
  ├────────────────────────┤
  │ scheduler.run (sync)   │
  ├────────────────────────┤
  │ execute (awaiting)     │
  ├────────────────────────┤
  │ handler (awaiting)     │
  └────────────────────────┘

PARKED PHASE (scheduler awaits lib.race):

  Stack                            Heap
  (nothing scheduler-owned         ┌──────────────────────────────┐
   on the stack — only SDK         │ Fiber:                       │
   internals / event loop)         │   state: parked,             │
                                   │     promises: [{prom, fire}] │
                                   │   it: <suspended at yield*>  │
                                   └──────────────────────────────┘
                                   Scheduler:
                                     fibers: { fiber }
                                     ready:  []
                                     awaiting: lib.race(tagged)

Every cycle is stackless end-to-end. The only memory grows is the heap: the suspended iterator, the fiber's state, the parked promise sources, the scheduler's set. The stack stays bounded by the fixed depth handler → execute → scheduler.run → drainReady → advance → iterator.next → user gen → … no matter how many yields deep the user generator nests.

The next four subsections drill into why that bound holds — what keeps the stack flat, what doesn't, and where things would blow up if the design were any different.

6.1 The advance trampoline

Look at Fiber.advance again — note the while(true):

advance(): void {
  if (this.state.kind !== "ready") return;
  const prevSlot = setCurrent(this.sched.contextSlot);
  try {
    let resume: Settled | null = this.state.resume;
    while (true) {                                  // ← trampoline
      let next: IteratorResult<unknown, unknown>;
      try {
        next = stepIterator(this.it, resume);
      } catch (e) { this.finish({ ok: false, e }); return; }

      if (next.done) { this.finish({ ok: true, v: next.value }); return; }

      const op = next.value as PrimitiveOp<unknown>;
      const node = op[opTag];
      let outcome: Settled | null;
      switch (node._tag) {
        case "Leaf":      outcome = this.parkOnLeaf(node);            break;
        case "AwaitRace": outcome = this.parkOnAwaitRace(node.futures); break;
      }

      if (outcome === null) return;       // parked — suspend
      resume = outcome;                   // sync short-circuit — loop
    }
  } finally { clearCurrent(prevSlot); }
}

parkOnLeaf and parkOnAwaitRace can return one of two things:

  • null — "I parked, suspend the fiber." advance returns; the scheduler will wake the fiber later.
  • Settled — "the source was already done; here's the value to feed back in." advance loops: stores it as resume, calls stepIterator again, fires the next yield.

That while(true) is a trampoline. Without it, a synchronous short-circuit would have to recurse into advance — and a fiber that yielded N already-settled local futures in a row would grow the stack O(N) deep.

Concretely: imagine a workflow that walks 10,000 channel sends, each of which short-circuits because the receiver was waiting. With the trampoline, the stack stays at one advance frame the whole time; the inner while(true) loop iterates 10,000 times. Without it, you'd recurse 10,000 deep and blow Maximum call stack size exceeded.

The same bound applies during normal execution: every yield* future that hits a local target which is already settled (e.g. a Channel that was fired by a sibling fiber earlier in the same drain) takes one loop iteration, not a stack frame.

6.2 yield* delegation: stack depth is O(delegation depth), not O(yields)

A subtlety: yield* is generator delegation. When you write:

function* outer() {
  return yield* inner();
}
function* inner() {
  return yield* run(() => fetchA());
}

…you have three iterator objects: the outer generator, inner's iterator, and the Future's iterator (which run returns). They form a delegation chain. The question is: what does the JS call stack look like during a .next() call?

During execution — the stack does grow along the chain:

fiber.advance
  ↓ calls
stepIterator → outerIt.next(resume)
                 ↓ inside outer's body, executing yield* inner()
                 ↓ which calls
               innerIt.next(resume)
                 ↓ inside inner's body, executing yield* run(...)
                 ↓ which calls
               futureIt.next(resume)
                 ↓ futureIt's body executes `yield leafOp`
                 ↑ control returns up through every yield* boundary
fiber.advance receives { done: false, value: leafOp }

The stack depth during this .next() is proportional to the delegation depth — three frames deep in this example, more if you nest helpers. But this stack only exists for the duration of one .next() call. As soon as the leaf yield bubbles up, control returns all the way to fiber.advance, and the stack collapses back.

While suspended — the stack is gone entirely. Every iterator in the chain is paused on the heap. The fiber's state is parked. The JS stack above the scheduler's await lib.race(...) is empty.

So:

Quantity Bound
Stack depth during one .next() O(delegation depth at this yield)
Stack depth while parked 0 (above the scheduler's await)
Stack depth across N yields in a row O(delegation depth), not O(N)

The trampoline rebuilds the delegation stack on each iteration; it doesn't preserve it across iterations. That means yield-heavy workflows are bounded by their deepest helper nesting, not by their total yield count. Recursive generator helpers will still blow the stack at the recursion depth — yield* is not free.

6.3 Reentrancy through the ready queue, not the call stack

The trickiest piece of stack discipline in this system is what happens when a fiber finishes and notifies its waiters. Look at Fiber.finish:

private finish(settled: Settled): void {
  this.state = { kind: "done", settled };
  const waiters = this.waiters;
  this.waiters = [];
  for (const w of waiters) w(settled);    // ← notify each waiter
  this.sched.markDone(this);
}

Each waiter is (s) => otherFiber.wake(s). So finish is, in effect, synchronously calling otherFiber.wake(settled) for every fiber that was parked on this one. Now look at wake:

wake(resume: Settled | null): void {
  if (this.state.kind === "done") return;
  this.state = { kind: "ready", resume };
  this.sched.markReady(this);              // ← push, don't run
}

And markReady:

markReady(f: Fiber<unknown>): void {
  this.ready.push(f);
}

wake does not call advance. It pushes onto the ready queue. Control returns up through wakefinishadvance — and only when this advance returns does drainReady pick the next ready fiber and call its advance.

Why this matters: imagine a chain of 1,000 fibers where each is waiting on the previous (a long pipeline). The last one finishes; it has a waiter (the second-to-last fiber). Without the push-don't-run discipline, you'd get:

fiber[N].finish
  → waiter callback
    → fiber[N-1].wake
      → fiber[N-1].advance   ← stack grows
        → ... resumes, hits return, calls finish
          → waiter callback
            → fiber[N-2].wake
              → fiber[N-2].advance   ← stack grows again
                → ...

1,000 deep stack. Blown.

With push-don't-run:

fiber[N].finish → waiter → fiber[N-1].wake → ready.push(fiber[N-1])
fiber[N].advance returns
drainReady loop iterates → fiber[N-1].advance
fiber[N-1].finish → waiter → fiber[N-2].wake → ready.push(fiber[N-2])
fiber[N-1].advance returns
drainReady loop iterates → fiber[N-2].advance
...

Stack stays at one advance frame deep, the chain unwinds via the queue. This is the same trampoline pattern as 6.1, applied at the inter-fiber level instead of the intra-fiber one.

It also gives the system a key invariant: at any moment, at most one fiber is between advance start and advance return. Multiple fibers never run "concurrently" on the JS stack. That's what makes the synchronous current-fiber slot in current.ts safe — setCurrent and clearCurrent always nest correctly because they're always paired within one advance call.

6.4 What the stack looks like while an ops.run action executes

run(action, opts) returns a journal-backed Future. The action itself runs inside ctx.run(name, wrapped, ...) — the SDK is the one driving the closure. From the gen-SDK's perspective:

run<T>(action: RunAction<T>, opts?: RunOpts<T>): Future<T> {
  const name = resolveRunName(action, opts);
  const wrapped = wrapActionForCancellation(this.sched.abortSignal, action);
  return this.sched.makeJournalFuture(
    adapt(this.ctx.run(name, wrapped, toSdkRunOptions(opts)))
  );
}

this.ctx.run(...) returns a RestatePromise immediately. The user's action is scheduled to run inside the SDK's plumbing — not on the gen-SDK's call stack. By the time the fiber parks on this Future, the action is already in flight somewhere in the SDK's machinery.

Stack during action execution:

ACTION RUNNING (user's `async () => fetch(...)` body):

  ┌────────────────────────────────┐
  │ user action body               │   await fetch(...) etc.
  ├────────────────────────────────┤
  │ wrapActionForCancellation      │   the wrapper closure
  ├────────────────────────────────┤
  │ <SDK plumbing inside ctx.run>  │   side-effect runner, telemetry, …
  ├────────────────────────────────┤
  │ <SDK invocation loop>          │   awaiting whatever it awaits
  └────────────────────────────────┘

  Meanwhile, the gen-SDK side is purely heap:

  Scheduler:           awaiting lib.race(tagged)
  Fiber:               state: parked,
                       promises: [{ promise: <that RestatePromise>, fire }]

When the action resolves, the SDK settles the RestatePromise. The gen-SDK's main-loop race wakes; the rest of the cycle from section 5 applies. The action's stack and the gen-SDK's stack share no frames — they coexist on the heap, joined only by the RestatePromise.

This is also where the AbortSignal earns its keep. The wrapActionForCancellation wrapper captures sched.abortSignal and passes it to the user's action. If invocation cancellation arrives while the action is mid-fetch, the scheduler aborts the signal before fanning out the TerminalError to fibers, so the in-flight fetch tears down on its own stack — concurrently with the fiber's catch handler running on the gen-SDK's stack. Two separate stacks, both reacting to the same cancellation event.


7. Concurrency: spawn and two fibers interleaving

User code:

gen(function* () {
  const fA = spawn(gen(function* () {
    return yield* run(() => fetchA(), { name: "a" });
  }));
  const fB = spawn(gen(function* () {
    return yield* run(() => fetchB(), { name: "b" });
  }));
  const [a, b] = yield* all([fA, fB]);
  return a + b;
});

spawn is straightforward:

// scheduler.ts
spawn<U>(op: Operation<U>): Future<U> {
  const f = this.createFiber(op);
  return makeFuture<U>({ kind: "local", target: f });
}

It registers a new fiber (added to fibers, pushed to ready) and returns a local-backed Future whose target is the fiber itself.

all([fA, fB]) checks for the journal-backed fast path (fs.every(isJournalBacked)) — both inputs are local-backed, so it falls back to a synthesized fiber:

return this.spawn(
  gen(function* () {
    const out = new Array(fs.length);
    for (let i = 0; i < fs.length; i++) {
      out[i] = yield* fs[i]!;
    }
    return out;
  })
);

So now there are four fibers alive:

  1. Main fiber (waiting on the all-fiber's Future).
  2. fA-fiber (running run for A).
  3. fB-fiber (running run for B).
  4. all-fiber (waiting on fA, then fB).

After the first drainReady:

Heap snapshot:
  Scheduler.fibers: { main, fA, fB, allF }
  Scheduler.ready:  []
  Scheduler.await lib.race([
     <fA's parked source: RestatePromise(run "a")>,
     <fB's parked source: RestatePromise(run "b")>,
  ])

  main:  parked on local backing → target = allF, registered as waiter on allF
  allF:  parked on local backing → target = fA,  registered as waiter on fA
  fA:    parked on journal source (RestatePromise "a")
  fB:    parked on journal source (RestatePromise "b")

Two key things to notice:

  • main and allF are parked on local sources. They don't contribute to the scheduler's race promise — they have no PromiseSource to hand over. Look at the data again: when the scheduler walks f.parkedSources() to assemble the race, only fA and fB contribute.
  • main was parked by parkOnLeaf on a local-backed Future. The branch in parkOnLeaf calls awaitCompletion on the target fiber, which queues the wake callback into the target's waiter list, and sets state = { kind: "parked", promises: [] }. Empty promises array — that's how a fiber parks on a sibling.

When A's RestatePromise resolves, the race settles with i = 0, fire is the wake closure for fA. fA becomes ready, drainReady advances it, the generator returns, Fiber.finish runs:

private finish(settled: Settled): void {
  this.state = { kind: "done", settled };
  const waiters = this.waiters;
  this.waiters = [];
  for (const w of waiters) w(settled);
  this.sched.markDone(this);
}

allF was in fA.waiters. w(settled) is (s) => allF.wake(s). So allF is now ready. drainReady continues, picks up allF, advances it — yield* fs[0]! returns, the loop moves to fs[1]! (fiber for B), parkOnLeaf queues allF as a waiter on fB. allF parks again. drainReady returns. The scheduler's while loop iterates: only B's source is parked now; it awaits that single-element race. When B resolves, the chain unwinds the same way.

sequenceDiagram
    participant M as main
    participant A as allF
    participant FA as fA
    participant FB as fB
    participant S as Scheduler.run

    Note over M,FB: drainReady advances all four to parked
    M->>A: park on allF (waiter)
    A->>FA: park on fA (waiter)
    FA->>S: parked source: RP_a
    FB->>S: parked source: RP_b

    S->>S: await lib.race([RP_a, RP_b])
    Note over S: RP_a resolves first
    S->>FA: fire(ok:"A")
    FA->>FA: advance → return "A" → finish
    FA->>A: notify waiter → allF.wake(ok:"A")
    S->>A: drainReady picks up allF
    A->>A: advance → out[0]="A", yield* fs[1]
    A->>FB: park on fB (waiter)

    S->>S: await lib.race([RP_b])
    Note over S: RP_b resolves
    S->>FB: fire(ok:"B")
    FB->>FB: advance → return "B" → finish
    FB->>A: notify waiter
    S->>A: drainReady picks up allF
    A->>A: advance → out[1]="B", return ["A","B"] → finish
    A->>M: notify waiter
    S->>M: drainReady picks up main
    M->>M: advance → consume "A"+"B" → finish

    Note over S: fibers empty, run resolves
Loading

This is the model. Local-backed futures wire fibers to each other via waiter lists; journal-backed futures all feed into the same main-loop race; the scheduler is agnostic about which.


8. Combinators on a Fiber: race and select

AwaitRace is the second primitive (besides Leaf). It carries an array of Futures and gets dispatched by parkOnAwaitRace:

private parkOnAwaitRace(
  futures: ReadonlyArray<Future<unknown>>
): Settled | null {
  // 1. Sync-check: any local target already done?
  for (let i = 0; i < futures.length; i++) {
    const b = getBacking(futures[i]!);
    if (b.kind === "local" && b.target.isDone()) {
      return { ok: true, v: { index: i, settled: b.target.settledValue() } };
    }
  }

  // 2. No quick win — register on all sources with a one-shot `fire`.
  let won = false;
  const promises: PromiseSource[] = [];
  for (let i = 0; i < futures.length; i++) {
    const idx = i;
    const b = getBacking(futures[i]!);
    const fireOnce = (settled: Settled) => {
      if (won) return;       // ← the won-flag: one wake per fiber
      won = true;
      this.wake({ ok: true, v: { index: idx, settled } });
    };
    if (b.kind === "local") b.target.awaitCompletion(fireOnce);
    else promises.push({ promise: b.promise, fire: fireOnce });
  }
  this.state = { kind: "parked", promises };
  return null;
}

Two details earn their keep:

  1. The sync-check at the top means awaitRace([alreadyDone, …]) doesn't even touch the scheduler's race. The fiber rebounds in the same advance() call.

  2. The won-flag prevents double-wakes. If two of the sources settle in the same tick (e.g., the main-loop race delivers source A, and during drainReady a sibling fiber finishes and notifies the same fiber as a waiter on source B), the second fireOnce is a no-op. The wake happens once; the rest of the sources continue, their settlements harmlessly discarded.

select({ tag1: f1, tag2: f2 }) is a pure-userland generator over awaitRace:

export function* select<B>(branches: B) {
  const tags = Object.keys(branches) as Array<keyof B & string>;
  const futures = tags.map((t) => branches[t]) as Future<unknown>[];
  const result = yield* awaitRace(futures);
  const tag = tags[result.index]!;
  return { tag, future: branches[tag] };
}

No scheduler hook. It's a generator that yields a primitive (awaitRace's AwaitRace) and converts the result. This is why select doesn't need a currentOps() lookup — it's pure dispatch.


9. Cancellation: fan-out from one rejection

The main loop's race promise is the single observation point for invocation cancellation. The SDK rejects it with TerminalError. The scheduler catches the rejection, fires every parked source's callback with the same error, and lets each fiber observe the cancellation at its yield point:

try {
  raceWinner = await this.lib.race(tagged);
} catch (e) {
  if (this.lib.isCancellation(e)) {
    this.abortController.abort(e);          // ← signal aborts first
    this.abortController = new AbortController();  // fresh one for recovery
  }
  const errSettled: Settled = { ok: false, e };
  for (const it of items) it.fire(errSettled);    // ← fan-out
  this.drainReady();
  continue;                                  // back to top of while
}

fire is the same closure used for the success path; on each fiber it calls wake({ ok: false, e }). Fiber.advance runs, stepIterator sees resume.ok === false, calls it.throw(resume.e). The user's try/catch around yield* catches a TerminalError. If they throw e, the fiber finishes with rejection. If they catch and yield more work, the next yield builds a fresh race promise that isn't infected by the prior rejection — cancellation is independent per event.

Three properties that fall out cleanly:

  • Fan-out is one for-loop. Every parked source's fire callback receives the same Settled. No special "in cancellation mode" flag.
  • The signal is aborted before fan-out. So ops.run closures that captured the signal see signal.aborted === true immediately — in-flight fetch(url, { signal }) calls start cancelling during the fan-out, not after the user's catch block runs.
  • The controller is replaced, not just aborted. Aborting is one- way; if the user catches and yields cleanup work, the cleanup's ops.run closures get a fresh signal that isn't aborted. Sticky cancellation would prevent recovery from doing real work.

10. Three more worked examples

The single-step example in section 5 and the spawn/all example in section 7 cover the basic shapes. Three more examples here cover patterns that aren't obvious from those two: a journal-only race (timeout), a purely in-memory rendezvous (channel ping-pong), and a deeply-nested delegation chain (to see the stack mechanics from section 6.2 in action).

10.1 Timeout via race([work, sleep])

const result = yield* race([
  run(() => slowAPI(), { name: "api" }),
  sleep({ seconds: 5 }),
]);

This is the standard "do work, or time out at 5s." Both inputs are journal-backed Futures. Worth tracing because race does not have a journal-fast-path the way all does:

// scheduler.ts
race<const T>(futures: T): Future<...> {
  const fs = futures as ReadonlyArray<Future<unknown>>;
  // No need here to try downcasting to RestateFuture's, awaitRace will
  // anyway produce the same UnresolvedFuture tree!
  return this.spawn(
    gen<R>(function* () {
      const result = yield* awaitRace(fs);
      if (result.settled.ok) return result.settled.v as R;
      throw result.settled.e;
    })
  );
}

race always spawns a helper fiber that yields awaitRace. The comment explains why: parkOnAwaitRace already assembles every journal source into the parked-promises list, and they all end up in the main loop's lib.race regardless. The fast path would only save the cost of one extra fiber.

Sequence:

Phase 1 — drainReady on entry to execute()
  main:    yield* race([...])
            → race() calls scheduler.spawn(genHelper)
            → returns a local-backed Future<R>
            → main yields a Leaf on that Future
            → main: parked on helper (local, waiter on helper)

  helper:  begins advancing → yield* awaitRace([api, sleep])
            → parkOnAwaitRace inspects both futures
              · api: journal → push { promise: RP_api,   fire(0) }
              · sleep: journal → push { promise: RP_5s,   fire(1) }
            → state := parked, promises: [RP_api, RP_5s]

  Live fibers: { main, helper }
  parkedSources walk:
    main:    []                    (parked on local target)
    helper:  [RP_api, RP_5s]
  Main loop awaits lib.race([RP_api', RP_5s'])     (tagged with i=0, i=1)

Phase 2 — sleep wins (slowAPI took longer than 5s)
  RP_5s settles  → lib.race resolves { i: 1, ok: true, v: undefined }
  fire callback for index 1 fires once:
    won := true
    helper.wake({ ok: true, v: { index: 1, settled: { ok: true, v: undefined } } })
  drainReady picks helper:
    yield* awaitRace returns { index: 1, settled: { ok: true, v: undefined } }
    settled.ok is true → return undefined
    helper.finish({ ok: true, v: undefined })
    helper's waiters: [ (s) => main.wake(s) ]
    main.wake({ ok: true, v: undefined })
  drainReady picks main:
    yield* (the Leaf future) returns undefined
    main returns its value

Two observations:

  1. The losing race promise is not cancelled. RP_api is still in flight after the race settles. The fire callback for i=0 is now dead-on-arrival (won === true short-circuits it), but the underlying ctx.run("api", ...) is still executing on the SDK's side. It will eventually settle and be journaled, but the Future wrapping it has nobody listening. This is the "race losers keep running" property from section 9 of the design doc.

    In production, this matters: a 30-second slowAPI() that timed out at 5 seconds is still using a network connection for another 25 seconds. To cancel it eagerly, plumb the AbortSignal: run(({ signal }) => fetch(url, { signal })). The gen-SDK's signal isn't aborted on a race win (it only aborts on invocation-level cancellation), but you can build per-race cancellation patterns with a Channel<void> + select.

  2. Two journal entries. Both RP_api and RP_5s create journal entries (the SDK records the sleep timer and the run completion independently). Compare to all([f1, f2]) over journal-backed inputs, which collapses to a single RestatePromise.all entry. This is by design — race semantics don't naturally map to a single combinator entry the way all does, because the losing entries still need to be observable for replay.

10.2 Channel ping-pong (pure local-backed Futures)

const ch1 = channel<number>();
const ch2 = channel<number>();

const worker = gen(function* () {
  const x = yield* ch1.receive;
  yield* ch2.send(x * 2);
});

const main = gen(function* () {
  const f = spawn(worker);
  yield* run(() => Promise.resolve(7));   // one journal step to anchor progress
  yield* ch1.send(42);
  const reply = yield* ch2.receive;
  yield* f;
  return reply;
});

What makes this interesting: after the single run step settles, everything else happens inside one drainReady. No further main- loop awaits. The channels are purely in-memory; the scheduler never needs to ask the SDK to wait on anything.

Trace from after the run step settles:

Phase 0 — at the moment the run() journal future fires
  main:    ready (resume = { ok: true, v: 7 })
  worker:  parked on ch1 (local waiter on impl1.waiters)

  drainReady starts.

Phase 1 — main advances
  main: it.next({ ok: true, v: 7 }) → resumes after the run
        executes `yield* ch1.send(42)`
          → ch1.send(42) is gen(function*() { impl1.fire(42); })
          → yield* iterates the helper gen
          → helper gen body runs: impl1.fire(42)
            · impl1.state := { kind: "settled", value: 42 }
            · waiters: [ (s) => worker.wake(s) ]
            · for each w: w({ ok: true, v: 42 })
              → worker.wake({ ok: true, v: 42 })
                · worker.state := { kind: "ready", resume: {ok,v:42} }
                · sched.markReady(worker) → ready.push(worker)
          → helper gen returns; yield* completes (no value)
        executes `yield* ch2.receive`
          → parkOnLeaf on ch2.impl
          → impl2 isn't done → register waiter
          → main.state := parked, promises: []
        advance returns (parked)

Phase 2 — drainReady continues: worker is in ready
  worker: it.next({ ok: true, v: 42 }) → resumes after `ch1.receive`
        `const x = 42;`
        executes `yield* ch2.send(x * 2)` = `yield* ch2.send(84)`
          → ch2.send(84): impl2.fire(84)
            · impl2.state := settled with 84
            · waiters: [ (s) => main.wake(s) ]
            · w({ ok: true, v: 84 }) → main.wake → ready.push(main)
          → send returns
        worker hits return (implicit undefined)
        worker.finish({ ok: true, v: undefined })
        worker.waiters: [ (s) => main.wake(s) ]   (from main's `yield* f`?)
          ...wait, main hasn't reached `yield* f` yet.
        markDone(worker)
        advance returns

Phase 3 — drainReady continues: main is in ready
  main: it.next({ ok: true, v: 84 }) → resumes after ch2.receive
        `const reply = 84;`
        executes `yield* f`
          → parkOnLeaf on worker (local)
          → worker.isDone() === true → sync short-circuit
          → returns settled value (ok: true, v: undefined)
          → advance's while(true) loops, feeds undefined back in
        next iteration: it.next({ ok: true, v: undefined })
        `return reply;` → main.finish({ ok: true, v: 84 })

Phase 4 — drainReady exhausted, no fibers left, scheduler.run resolves.

Three things to notice:

  • The main loop's race is never entered between phase 0 and the end. All four phases run on the same drainReady call stack — one advance per fiber, alternating, driven by the ready queue. Total time spent in await: zero.

  • The yield* f at the end hits the sync short-circuit path in parkOnLeaf (section 4). worker was already done by the time main got to that yield, so awaitCompletion returns the settled value immediately; the advance trampoline (section 6.1) feeds it back without ever entering the parked state.

  • The push-don't-run discipline (section 6.3) is doing real work here. When impl1.fire(42) calls worker.wake, that synchronously pushes worker into the ready queue but does not call advance. Control returns up through firesend's generator body → yield* in main → main continues executing → main parks on ch2 → main's advance returns. Only then does drainReady's loop pick up worker. Without push-don't-run, worker's advance would run on top of main's stack, and main's parkOnLeaf for ch2 would happen after worker had already woken main again — which would be either a logic bug (waking a non-parked fiber) or a stack explosion in a longer chain.

10.3 Deeply nested delegation: the stack during one .next()

This example exists to make section 6.2's claim concrete. Five helpers, each delegating to the next:

function* level5() {
  return yield* run(() => fetchValue(), { name: "leaf" });
}
function* level4() { return (yield* level5()) + "/4"; }
function* level3() { return (yield* level4()) + "/3"; }
function* level2() { return (yield* level3()) + "/2"; }
function* level1() { return (yield* level2()) + "/1"; }

const op = gen(level1);

At the moment the run Future yields its Leaf, the stack is:

┌──────────────────────────────────────────────────────────────┐
│ fiber.advance — owns the slot, owns the trampoline loop      │
├──────────────────────────────────────────────────────────────┤
│ stepIterator                                                 │
├──────────────────────────────────────────────────────────────┤
│ outerIt.next()      (the level1 iterator)                    │
│   ┌──────────────────────────────────────────────────────┐   │
│   │ level1 body: executing `yield* level2()`             │   │
│   │   delegates to:                                      │   │
│   │     level2It.next()                                  │   │
│   │       ┌──────────────────────────────────────────┐   │   │
│   │       │ level2 body: executing `yield* level3()` │   │   │
│   │       │   delegates to:                          │   │   │
│   │       │     level3It.next()                      │   │   │
│   │       │       ┌──────────────────────────────┐   │   │   │
│   │       │       │ level3 body: yield* level4() │   │   │   │
│   │       │       │     ...                      │   │   │   │
│   │       │       │       level5 body:           │   │   │   │
│   │       │       │         yield* runFuture     │   │   │   │
│   │       │       │           futureIt.next()    │   │   │   │
│   │       │       │             `yield leafOp`   │   │   │   │
│   │       │       │           ◄── here           │   │   │   │
│   │       │       └──────────────────────────────┘   │   │   │
│   │       └──────────────────────────────────────────┘   │   │
│   └──────────────────────────────────────────────────────┘   │
└──────────────────────────────────────────────────────────────┘

Six iterator frames stacked, plus fiber.advance and stepIterator — call it ~8 frames total. At the exact moment yield leafOp executes, control returns through every yield* boundary in reverse: futureIt returns {done: false, value: leafOp}, level5It returns the same, level4It returns the same, … up to outerIt returning {done: false, value: leafOp}. Then fiber.advance receives the value.

That entire collapse happens on one .next() call, in the same synchronous span the call started in. No microtask boundary. When control returns to fiber.advance, the call stack is back to fiber.advance + stepIterator + nothing else. The six iterator objects are now on the heap, in suspended state.

When the journal future settles and the fiber wakes, stepIterator(it, resume) is called again. This time, it.next(v) walks back down through the same chain — pushing outerItlevel2Itlevel3Itlevel4Itlevel5ItfutureIt onto the stack again, resuming each at its yield*, until the deepest one returns from its yield with the resumed value. Then everything unwinds back up.

The bound: stack depth during one .next() is exactly the delegation depth at that yield site, plus the fixed scheduler preamble. It does not depend on how many yields the workflow has made in total, how many fibers are alive, or how long the workflow has been running. Across a workflow that runs 10,000 yields, each one rebuilds and tears down the same ~N-deep stack independently.

If a user writes recursive generator helpers, the stack will grow with the recursion depth on every .next(). There is no Tail Call Optimization in V8 for generators. Don't write recursive yield* patterns over unbounded input.


11. The wake-pump, summarized

The complete cycle from user yield to user resume is six steps:

  ┌─────────────────────────────────────────────────────────────┐
  │ user gen: yield* future                                     │
  │   │                                                         │
  │   ▼                                                         │
  │ Future.iterator yields Leaf node                            │
  │   │                                                         │
  │   ▼                                                         │
  │ Fiber.advance sees Leaf, calls parkOnLeaf                   │
  │   │                                                         │
  │   ▼                                                         │
  │ state := parked, promises := [{ promise, fire: wake }]      │
  │ advance returns                                             │
  └──┬──────────────────────────────────────────────────────────┘
     │ (control unwinds to Scheduler.run, then awaits lib.race)
     ▼
  ┌─────────────────────────────────────────────────────────────┐
  │ Scheduler.run: await lib.race(parked promises from all      │
  │   live fibers, tagged with indices)                         │
  │   │                                                         │
  │   ▼                                                         │
  │ SDK resolves the underlying RestatePromise                  │
  │   │                                                         │
  │   ▼                                                         │
  │ race winner: { i, ok, v }                                   │
  │   │                                                         │
  │   ▼                                                         │
  │ items[i].fire(settled) → fiber.wake → state := ready        │
  │ drainReady → fiber.advance →                                │
  │   stepIterator → it.next(value) → yield* returns to user    │
  └─────────────────────────────────────────────────────────────┘

That cycle is invariant. Local-backed Futures swap step 4 for a waiter.push on the target fiber's list, and step 5 for the target's own finish() firing the same callback. Combinators swap Leaf for AwaitRace. Cancellation swaps fire(ok) for fire(err) across every source at once. Same machine.


12. Things to take away

If you remember nothing else, take these three:

  1. User code is synchronous from the scheduler's perspective. Generators yield; they don't await. That's the prerequisite for the module-level slot, the bounded stack, and the bookkeeping simplicity. Take this away and the whole shape of the implementation changes.

  2. There is one await point in the entire system: lib.race in the main loop. Every journal-backed Future feeds into it. Every cancellation is observed at it. Every wake originates from it (or from a sibling fiber's finish, which itself was woken from it). Centralizing the await is what makes cancellation a one-line for-loop and replay-time advancement bug-free.

  3. Local-backed and journal-backed Futures look identical to user code but cost very different things. Local-backed Futures never touch the journal — the scheduler resolves them in-memory via waiter lists. Journal-backed Futures are the only ones that appear in the main race. The combinator fast path (every(isJournalBacked)) collapses N futures into one RestatePromise.all/race/… journal entry; the fallback emulates the same semantics with a synthesized fiber. Same user-visible contract, different cost. Always.

Everything else is bookkeeping around those three ideas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment