Giving RISC-V 1024 registers for zkVMs

zkVMs like OpenVM pick RISC-V because it's simple. It's a clean ISA, well-understood, with mature toolchains. But RISC-V was designed for hardware: 32 general-purpose registers is a reasonable number when spills go to cache and cost you a few cycles.

In a zkVM, memory is expensive. Every load and store has to be proven, and memory consistency constraints dominate the proof cost of compute-heavy programs. That 32-register limit, harmless on silicon, becomes a bottleneck.

At Powdr we already built crush, which compiles from WASM to a custom ISA with infinite registers and zero spills. That's the clean solution. But I got curious about a different question: what if we just took plain old RISC-V and gave it a bigger register file? LLVM IR already uses infinite virtual registers internally. The register allocator's job is to map them to a finite physical register set. If we give it 1024 registers instead of 32, it should trivially succeed without spills for most real-world functions.

So I asked Claude to implement this in LLVM and wire it up to OpenVM. The experiment ran end-to-end: real ELFs, real STARK proofs, real metrics.

What spills look like on RISC-V

The benchmark uses tiny_sha3 (Saarinen's FIPS 202 reference implementation in C, commit dcbb319, byte-identical to the upstream source — we didn't touch it) plus a tiny freestanding driver that runs 1000 SHA3-256 iterations on a buffer starting as 32 zero bytes. Same source for both variants, same clang -O3 frontend, same LLVM IR — only the llc -mattr flag changes between them.

Here's the opening of sha3_keccakf (the keccak permutation) compiled to standard RISC-V:

sha3_keccakf:
    addi sp, sp, -272
    sw   ra,  268(sp)          # callee-save spill
    sw   s0,  264(sp)
    sw   s1,  260(sp)
    sw   s2,  256(sp)
    sw   s3,  252(sp)
    sw   s4,  248(sp)
    sw   s5,  244(sp)
    sw   s6,  240(sp)
    sw   s7,  236(sp)
    sw   s8,  232(sp)
    sw   s9,  228(sp)
    sw   s10, 224(sp)
    sw   s11, 220(sp)          # 13 callee-save spills
    li   t3, 0
    lw   t2, 0(a0)
    lw   t4, 4(a0)
    lui  a1, %hi(.L__const.sha3_keccakf.keccakf_rotc)
    addi a1, a1, %lo(.L__const.sha3_keccakf.keccakf_rotc)
    addi a2, a1, 4
    sw   a2, 12(sp)            # immediately spill a constant pointer
    addi a1, a1, 96
    sw   a1, 8(sp)             # spill another
    ...

The function opens with 13 callee-save spills, then starts loading the keccak state and constant pointers and immediately spilling them, because it's already out of registers. The keccak state is 25 × 64-bit lanes (50 × 32-bit on RV32). The function has to juggle all of them plus temporaries and constant pointers into 32 architectural registers.

Counting across the whole sha3_keccakf:

784 instructions total
111 stack stores + 111 stack loads = 222 memory operations
272 bytes of stack frame

Every one of those 222 memory ops is pure overhead in a zkVM — they exist only because we ran out of registers.

The same code on RISC-V with 1024 registers

I asked Claude to add a +xregs1024 subtarget feature to LLVM's RISC-V backend: 1024 GPRs, a 64-bit instruction encoding (to fit 10-bit register fields), and a calling convention where everything except ra is caller-saved. Same tiny_sha3 C source, same LLVM IR, same -O3 — just -mattr=+xregs1024 added to the llc invocation.

Here's the same sha3_keccakf prologue:

sha3_keccakf:                  # no stack frame at all
    li   a1, 0
    lw   t1, 0(a0)
    lw   t2, 4(a0)
    lui  a6, %hi(.L__const.sha3_keccakf.keccakf_rotc)
    addi a6, a6, %lo(.L__const.sha3_keccakf.keccakf_rotc)
    lui  a2, %hi(.L__const.sha3_keccakf.keccakf_piln+4)
    addi a2, a2, %lo(.L__const.sha3_keccakf.keccakf_piln+4)
    li   a3, 64
    li   a4, 32
    addi a5, a6, 4
    addi a6, a6, 96
    lui  a7, %hi(.L__const.sha3_keccakf.keccakf_rndc)
    addi a7, a7, %lo(.L__const.sha3_keccakf.keccakf_rndc)
    li   t0, 24
    j    .LBB0_2
    ...

No prologue. No spills. All the constants live in registers, the loop counter lives in a register, and inside the round body the 50 state words sit in x32–x80.

The full numbers for this function:

561 instructions (down from 784 — a 28% reduction)
0 stack stores, 0 stack loads (down from 222 — 100% elimination)
0 bytes of stack frame
80 distinct registers used — 25 standard plus ~55 extended (x32–x91), which is about what you'd expect: 50 words of keccak state plus temporaries across the theta/rho/pi/chi/iota steps.

So sha3_keccakf needs about 80 registers to run without spilling, and when you give the register allocator that many to work with, it finds them. The standard 32-register RISC-V forces it to shuffle values back and forth through memory for the entire function.

Running it in OpenVM

The interesting part is what happens when a zkVM tries to prove this. Since the wider registers don't fit in the standard 32-bit RISC-V instruction format, the binary uses a 64-bit encoding for every instruction (the RISC-V spec actually reserves encoding space for exactly this).

I had to modify OpenVM to accept the new binary format: a new transpiler extension that decodes the 64-bit instructions, widened register byte offsets internally (a u8 → u16 cleanup across the circuit code), and an expanded register file. The instruction set itself didn't change — same opcodes, same semantics, just wider register identifiers.

Then we fed both binaries through OpenVM and generated actual STARK proofs for a benchmark that runs SHA3-256 1000 times in a row over a 32-byte buffer that starts as all zeros. The final hash is 52cf48e88ce4dea40f272b6aaf083675ade26504a0129f51ec30204a2fdb1c5b — both the baseline and extended ELFs produce exactly this, matching Python's hashlib.sha3_256.

(For the OpenVM guest specifically, I'm using freestanding C and linking directly with ld.lld; wiring the full Rust + OpenVM toolchain through our modified backend needs a custom rustc sysroot, which is future work.)

The proof metrics

Aggregated across all AIRs, from the raw metrics JSON of the two runs (1000 SHA3-256 iterations, 4 proof segments, actual STARK proofs generated end-to-end):

	Baseline (32 regs)	Extended (1024 regs)	Change
Executed instructions	35,896,008	31,067,004	−13.5%
`total_cells` (allocated trace)	4,049,774,760	3,853,796,520	−4.8%
`total_cells_used` (filled trace)	3,275,912,933	2,829,414,701	−13.6%
`main_cells_used`	1,426,439,001	1,229,953,329	−13.8%
Total proof time	130.8s	127.7s	−2.4%

~14% fewer trace cells actually used in the proof on a workload that's already one of the best cases for the standard RISC-V backend, because keccak has such a large live set. The total_cells allocated number drops less because of FRI padding to powers of two — actual prover work scales with the filled cells. Proof wall-clock barely moves because a lot of it is FRI/commitment overhead that doesn't shrink with the trace.

The real savings show up in fewer memory operations proved: every spill/reload in the baseline is a load and a store that have to be accounted for in the memory argument, and those are gone.

It's a real win. But it's not a huge win.

Where the spills are (and aren't)

The assembly-level data showed 222 memory ops eliminated inside sha3_keccakf alone. The end-to-end reduction is ~14%. Why not more?

Because this experiment solves the easy half of the problem: spills inside basic blocks. A function with plenty of live values and nowhere to put them. Give the register allocator a bigger register file and it stops juggling.

What it doesn't solve is the other half: spills across function calls. Even with 1024 caller-saved registers, when one function calls another, the caller has to save any live value it wants to see after the call returns. The callee is free to clobber everything. So if there are values live across a call, they land on the stack — in particular, ra always does, because every call overwrites it.

This is where crush is structurally different. In crush, each function gets its own disjoint slice of the infinite register space via a frame pointer — caller and callee never share registers, so there's nothing to save and nothing to restore. It's a true frame separation, not a convention on top of a shared register file.

RISC-V, like every low-level ISA, has a single flat register file shared across every stack frame. The calling convention is a cooperative agreement about who saves what, but it's always a convention on top of a shared resource. You can make all registers caller-saved (which we did) or all callee-saved, or split them — but you can't escape the fact that a call boundary is a place where live values have to move somewhere if the two functions want to use the same registers. And the register allocator can't coordinate across function boundaries without whole-program analysis.

Frame-based register partitioning, like crush does, needs to be designed into the ISA from the start. You need some equivalent of a frame pointer that shifts the register window, or literally infinite registers with SSA-style naming. Retrofitting it onto RISC-V through the existing calling convention mechanism doesn't work, because the mechanism itself is what leaks spills across call sites.

So: the experiment succeeded at what it set out to do. Register pressure inside functions goes to zero. That's a meaningful improvement on real workloads (~14% fewer trace cells on keccak, and more on compute-heavier workloads). But the calling convention problem remains, and for that you need a different kind of ISA.

leonardoalt/xregs1024_blog.md

Select an option

No results found