Tree Sitter WASM benchmarking

README.md

It works, but wasm isn't free. I noticed that zed has some funny-looking code to carefully reuse TS objects, so I did some benchmarks to make sure that this API design doesn't put us into a corner. Benchmarks were run on my Macbook only. It turns out that what Zed is doing saves ~20% of the wasm language load time (25ms->20ms), entirely because instantiating a TSWasmStore takes ~5ms.

Loading a language

grammar	native ms	wasm ms	native RAM MB	wasm RAM MB
javascript	0.1	24.0	0.2	0.9
rust	0.1	20.7	0.0	2.2
python	0.1	20.6	0.0	1.0
typescript	0.1	23.4	0.0	2.0
cpp	0.1	69.9	0.0	13.6

• Timing a dlopen is difficult to measure precisely, in part because the OS shares loaded .dylib files between processes.
• The first time macOS loads a .dylib it validates its signature, which takes ~200ms. The validation is cached for the lifetime of the .dylib file.

Parsing a file

file	KiB	native ms	wasm ms	diff
jquery.js	242	18.8	57.7	3.1x
parser.ts	368	17.6	67.5	3.8x
ast.rs	65	4.4	12.6	2.7x
python3.8_grammar.py	49	5.2	14.3	2.8x

These files can be found in test/fixtures/grammars/$LANG/examples/.

Note: there was no measurable peak or final RAM usage difference between native and wasm parsers.

Why so slow?

AI analysis:

1. Guest → host, once per character. This is the killer. Inside the wasm lexer, advance, mark_end, get_column, eof are host imports, not wasm functions. Every advance(lexer) traps back out to callback__lexer_advance on the host, which runs the native advance and copies lookahead back into linear memory. tree-sitter's lexer walks the source codepoint by codepoint, so a parse of N source bytes does roughly N guest→host trampoline crossings, each tens of ns of register save/restore + VM-context + memcpy.
2. Host → guest, once per token attempt. The host parser calls ts_wasm_store_call_lex_main → ts_wasm_store__call → wasmtime_func_call_unchecked. Each call also does a wasmtime_table_get funcref lookup and a memcpy of the lexer state prefix into linear memory before and out of it after.