kyushu-jz-bench — engine-level benchmark: a compute-bound handler run as QuickJS-in-WASM vs jz-AOT-WASM (see kyushu#99)

README.md

kyushu-jz-bench

A small, reproducible benchmark for the question raised in kyushu#99: before evaluating any engine, what does a compute-bound handler actually cost?

It runs one handler kernel through two engines and reports size, cold start, and throughput:

• QuickJS-in-WASM — quickjs-emscripten, the same engine family Kyushu runs today via wasm-rquickjs. The handler JS is parsed and interpreted inside the WASM VM.
• jz-AOT-WASM — jz compiles the same handler JS ahead-of-time to a standalone WASM module. No interpreter.

The kernel (kernels/resize.js) is a bilinear RGBA image resize — a representative pure-compute handler. Valid jz is valid JS, so the exact same source file is what both engines run.

Run

npm install
npm run bench

Result (1080p → 320×180 thumbnail, Apple M-series, Node 25)

Workload: bilinear resize 1920×1080 → 320×180  (0.058 Mpx/op)
Cross-check: both engines produce checksum 36665856 ✓

                         QuickJS-in-WASM     jz-AOT-WASM    jz advantage
────────────────────────────────────────────────────────────────────────
module / handler size    506.7 KB engine          1.9 KB    265× smaller
cold start                      21.99 ms         0.20 ms  112.41× faster
throughput                           8/s           781/s  101.27× faster
per resize                     129.60 ms         1.28 ms        45 Mpx/s
────────────────────────────────────────────────────────────────────────

Numbers vary by machine; the shape is the point. For a tight numeric loop, an AOT-compiled WASM module runs ~100× faster than the same code interpreted, and ships as ~2 KB next to a ~500 KB engine.

Methodology

• Same source, one variable. Both engines execute kernels/resize.js verbatim. The jz path compiles it; the QuickJS path strips ESM syntax (so top-level bindings become VM globals) and interprets it. Nothing else differs.
• Correctness gate. Both engines must produce a bit-identical output checksum or the bench exits non-zero. The performance numbers only print when the two agree — so this is also a conformance check of jz against QuickJS.
• Cold start is per-worker spin-up: jz instantiates a precompiled module (Kyushu would AOT-compile once at freeze-time, not per request); QuickJS creates a context and parses the handler source.
• Throughput times steady-state resize() calls over a 1.5 s window. QuickJS calls are driven 8-per-crossing so the host↔VM boundary cost is amortized, not measured — this flatters QuickJS slightly, on purpose, to keep the comparison about execution, not marshaling.

What this does NOT show (honest scope)

This is an engine-level comparison, not a drop-in Kyushu evaluation:

• It is not a full Kyushu worker. It does not reproduce the template-freeze, routing/fetch/async glue, or Kyushu's Node.js polyfill surface. It isolates the one thing in question: the cost of running compute-bound handler code.
• jz is not a QuickJS replacement, and isn't trying to be. jz explicitly does not target async/I-O or Node polyfills. The npm/polyfill familiarity that makes Kyushu pleasant stays in the QuickJS worker. jz only earns its keep on the pure-numeric inner loop — image/audio/DSP/parsing/hashing. For I/O-bound handlers it offers nothing here.
• quickjs-emscripten ≠ wasm-rquickjs. Same engine family (QuickJS), but a different WASM build than Kyushu's. Treat the QuickJS column as representative, not as Kyushu's exact runtime.
• QuickJS can precompile to bytecode to shave parse time off cold start; the default path here parses source, which is what wasm-rquickjs does too.

The intended deployment isn't "replace the engine" — it's the split proposed in the issue: QuickJS keeps the dynamic glue, and a jz-compiled module handles the hot compute path as a peer in the same Wasmtime store.

bench.mjs

// kyushu-jz-bench — engine-level comparison for a compute-bound handler.
//
// One kernel source (kernels/resize.js), two engines:
//   • QuickJS-in-WASM  — quickjs-emscripten, the same engine family Kyushu runs
//     via wasm-rquickjs. The handler JS is parsed and interpreted inside the VM.
//   • jz-AOT-WASM      — the handler JS compiled ahead-of-time to a standalone
//     WASM module, no interpreter.
//
// Reported per engine: handler/module SIZE, COLD START (instantiate + first
// call), and steady-state THROUGHPUT (resizes/sec, megapixels/sec). A checksum
// cross-check asserts both engines compute the identical image.
//
// Honest scope: this is an ENGINE-level comparison, not a full Kyushu worker.
// It does not reproduce Kyushu's template-freeze or its Node polyfill surface —
// it isolates the one thing in question, the cost of running compute-bound
// handler code. See README.md.
//
//   npm run bench

import { readFileSync, existsSync } from 'node:fs'
import { performance } from 'node:perf_hooks'
import { fileURLToPath } from 'node:url'
import jz, { compile } from 'jz'
import { getQuickJS } from 'quickjs-emscripten'

const dir = fileURLToPath(new URL('.', import.meta.url))
// repo layout keeps kernels in kernels/; a cloned gist is flat — accept both.
const KERNEL = existsSync(dir + 'kernels/resize.js') ? dir + 'kernels/resize.js' : dir + 'resize.js'
const SRC = readFileSync(KERNEL, 'utf8')

// Workload: downscale a 1080p frame to a 320×180 thumbnail.
const SW = 1920, SH = 1080, DW = 320, DH = 180
const MPX = (DW * DH) / 1e6          // megapixels produced per resize
const BUDGET_MS = 1500               // steady-state timing window per engine

const fmt = (n, p = 2) => n.toLocaleString('en-US', { maximumFractionDigits: p, minimumFractionDigits: p })
const kb = (b) => `${fmt(b / 1024, 1)} KB`

// Run `fn` for BUDGET_MS, return { opsPerSec, msPerOp }.
const measure = (fn) => {
  let ops = 0
  const t0 = performance.now()
  do { fn(); ops++ } while (performance.now() - t0 < BUDGET_MS)
  const dt = performance.now() - t0
  return { opsPerSec: (ops / dt) * 1000, msPerOp: dt / ops }
}

// ── jz: AOT compile to WASM ───────────────────────────────────────────────
const jzWasm = compile(SRC)                 // build step — Kyushu would do this at freeze
const jzModule = await WebAssembly.compile(jzWasm)

// Cold start = bring a precompiled module up to ready-to-call. Kyushu pays the
// AOT compile once at freeze-time (jzWasm above); per worker it only instantiates.
const jzCold = (() => {
  const t0 = performance.now()
  new WebAssembly.Instance(jzModule)
  return performance.now() - t0
})()

const jzInst = jz(SRC).exports
jzInst.setup(SW, SH, DW, DH)
jzInst.fill()
jzInst.resize()
const jzCheck = jzInst.checksum() >>> 0
const jzThru = measure(() => jzInst.resize())

// ── QuickJS: parse + interpret inside WASM VM ─────────────────────────────
const QJS = await getQuickJS()
const QJS_ENGINE_BYTES = readFileSync(
  dir + 'node_modules/@jitl/quickjs-wasmfile-release-sync/dist/emscripten-module.wasm'
).byteLength
// Same source, minus ESM syntax — top-level bindings become VM globals.
const QJS_SRC = SRC.replace(/export let/g, 'let')

const evalIn = (ctx, code) => {
  const r = ctx.evalCode(code)
  if (r.error) { const e = ctx.dump(r.error); r.error.dispose(); throw new Error('QJS: ' + JSON.stringify(e)) }
  const v = ctx.dump(r.value); r.value.dispose(); return v
}

// Cold start = create a VM context and parse the handler source — the per-worker
// spin-up Kyushu pays. (QuickJS could precompile to bytecode to shave parsing;
// the default path parses source, which is what wasm-rquickjs does too.)
const qjsCold = (() => {
  const t0 = performance.now()
  const ctx = QJS.newContext()
  evalIn(ctx, QJS_SRC)
  const t = performance.now() - t0
  ctx.dispose()
  return t
})()

const qctx = QJS.newContext()
evalIn(qctx, QJS_SRC)
evalIn(qctx, `setup(${SW},${SH},${DW},${DH}); fill();`)
const qjsCheck = evalIn(qctx, `resize(); checksum() >>> 0`) >>> 0
// Drive N resizes per host→VM crossing so the boundary cost is amortized, not measured.
const qjsThru = measure(() => evalIn(qctx, `for (let i = 0; i < 8; i++) resize(); 0`))
qjsThru.opsPerSec *= 8; qjsThru.msPerOp /= 8
qctx.dispose()

// ── Correctness cross-check ───────────────────────────────────────────────
if (jzCheck !== qjsCheck) {
  console.error(`\n✗ checksum mismatch: jz=${jzCheck} quickjs=${qjsCheck}`)
  process.exit(1)
}

// ── Report ────────────────────────────────────────────────────────────────
const pad = (s, n) => String(s).padEnd(n)
const padl = (s, n) => String(s).padStart(n)
console.log()
console.log(`Workload: bilinear resize ${SW}×${SH} → ${DW}×${DH}  (${fmt(MPX, 3)} Mpx/op)`)
console.log(`Cross-check: both engines produce checksum ${jzCheck} ✓`)
console.log()
console.log(pad('', 22) + padl('QuickJS-in-WASM', 18) + padl('jz-AOT-WASM', 16) + padl('jz advantage', 16))
console.log('─'.repeat(72))
console.log(
  pad('module / handler size', 22) +
  padl(kb(QJS_ENGINE_BYTES) + ' engine', 18) +
  padl(kb(jzWasm.byteLength), 16) +
  padl(fmt(QJS_ENGINE_BYTES / jzWasm.byteLength, 0) + '× smaller', 16)
)
console.log(
  pad('cold start', 22) +
  padl(fmt(qjsCold) + ' ms', 18) +
  padl(fmt(jzCold) + ' ms', 16) +
  padl(fmt(qjsCold / jzCold) + '× faster', 16)
)
console.log(
  pad('throughput', 22) +
  padl(fmt(qjsThru.opsPerSec, 0) + '/s', 18) +
  padl(fmt(jzThru.opsPerSec, 0) + '/s', 16) +
  padl(fmt(jzThru.opsPerSec / qjsThru.opsPerSec) + '× faster', 16)
)
console.log(
  pad('per resize', 22) +
  padl(fmt(qjsThru.msPerOp) + ' ms', 18) +
  padl(fmt(jzThru.msPerOp) + ' ms', 16) +
  padl(fmt((MPX / jzThru.msPerOp) * 1000, 0) + ' Mpx/s', 16)
)
console.log('─'.repeat(72))
console.log()

package.json

{
  "name": "kyushu-jz-bench",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "description": "Engine-level benchmark: a compute-bound handler kernel run as QuickJS-in-WASM vs jz-AOT-WASM",
  "scripts": {
    "bench": "node bench.mjs"
  },
  "dependencies": {
    "jz": "latest",
    "quickjs-emscripten": "^0.31.0"
  }
}

resize.js

// Bilinear RGBA image resize — a representative compute-bound handler kernel.
//
// Valid JS == valid jz: this exact file runs unmodified inside QuickJS (the
// engine Kyushu uses today) and compiles unmodified to WASM via jz. The bench
// drives the SAME source through both, so the only variable is the engine.
//
// Buffers live module-level. `setup` allocates src+dst and hands the host a view
// over `src` to fill with the input image; `output` hands back the resized `dst`.
// Under jz these are views into linear WASM memory; under QuickJS they are plain
// typed arrays inside the VM. Either way the host never marshals per call.

let sw = 0, sh = 0, dw = 0, dh = 0
let src = new Uint8Array(0)
let dst = new Uint8Array(0)

export let setup = (sw_, sh_, dw_, dh_) => {
  sw = sw_; sh = sh_; dw = dw_; dh = dh_
  src = new Uint8Array(sw * sh * 4)
  dst = new Uint8Array(dw * dh * 4)
  return src
}

export let output = () => dst

// Deterministic gradient fill so both engines run identical input without the
// host marshaling a buffer across the boundary (which would bias the timing).
export let fill = () => {
  let i = 0, n = sw * sh
  while (i < n) {
    let x = i % sw, y = (i / sw) | 0
    let o = i * 4
    src[o] = (x ^ y) & 255
    src[o + 1] = (x * 3 + y) & 255
    src[o + 2] = (y * 5) & 255
    src[o + 3] = 255
    i++
  }
}

// Sum of dst bytes — pins the output against dead-code elimination and lets the
// bench assert jz and QuickJS produce bit-identical results.
export let checksum = () => {
  let s = 0, i = 0, n = dw * dh * 4
  while (i < n) { s = (s + dst[i]) | 0; i++ }
  return s
}

// Fixed-point bilinear sample: for each dst pixel map back into src space, take
// the 4 neighbours, blend by the fractional coordinate. Pure integer/float math
// over a flat buffer — no allocation, no I/O — exactly the shape jz targets.
export let resize = () => {
  let xr = (sw << 8) / dw      // src→dst step, .8 fixed point
  let yr = (sh << 8) / dh
  let dy = 0
  while (dy < dh) {
    let sy = (dy * yr) >> 8
    let fy = (dy * yr) & 255
    let sy1 = sy + 1 < sh ? sy + 1 : sy
    let rowT = sy * sw * 4
    let rowB = sy1 * sw * 4
    let dstRow = dy * dw * 4
    let dx = 0
    while (dx < dw) {
      let sx = (dx * xr) >> 8
      let fx = (dx * xr) & 255
      let sx1 = sx + 1 < sw ? sx + 1 : sx
      let iTL = rowT + sx * 4
      let iTR = rowT + sx1 * 4
      let iBL = rowB + sx * 4
      let iBR = rowB + sx1 * 4
      let o = dstRow + dx * 4
      let c = 0
      while (c < 4) {
        let top = src[iTL + c] * (256 - fx) + src[iTR + c] * fx
        let bot = src[iBL + c] * (256 - fx) + src[iBR + c] * fx
        dst[o + c] = (top * (256 - fy) + bot * fy) >> 16
        c++
      }
      dx++
    }
    dy++
  }
}