EVM Interpreter Overhead vs Direct Compilation: RISC-V Instruction-Level Analysis

interpreter_overhead_analysis.md

EVM Interpreter Overhead vs Direct Compilation: Instruction-Level Analysis

The interpreter loop (per opcode)

From revm-interpreter source (interpreter.rs:282), every single EVM opcode goes through this path:

fn step(&mut self, instruction_table: &InstructionTable, host: &mut H) {
    let opcode = self.bytecode.opcode();        // 1. Fetch opcode byte
    self.bytecode.relative_jump(1);              // 2. Advance PC
    let instruction = instruction_table[opcode]; // 3. Table lookup (fn ptr + static_gas)
    if self.gas.record_cost_unsafe(instruction.static_gas()) {  // 4. Gas check
        return self.halt_oog();
    }
    instruction.execute(context);                // 5. Indirect call to handler
}

Compiled to rv32im, this is approximately:

Step	RISC-V instructions	Description
Opcode fetch	2	Load byte from bytecode pointer
PC advance	1	Increment bytecode pointer
Table lookup	3	Compute table[opcode*entry_size], load fn_ptr + static_gas
Gas check	3	Compare remaining >= cost, subtract, branch if OOG
Loop check	2	Check is_not_end(), branch back to top
Indirect call	2	jalr to handler function pointer + return
Total per-opcode dispatch	~13

This overhead is paid for every single EVM opcode, regardless of how trivial it is.

Case study: a hot EVM basic block

From the block 24171377 trace, one of the hottest blocks is at contract 0xc2b7...4De4, pc 2728, executed 255 times:

PUSH32 SWAP1 DUP2 DUP6 PUSH1 MULMOD SWAP5 DUP3 ...

Let's take a simpler, representative basic block for analysis. Consider this 5-opcode EVM basic block:

JUMPDEST DUP1 ADD SWAP1 JUMP

This is a common loop iteration pattern: duplicate top of stack, add it to the next value, swap, and jump back. Let's trace the exact RISC-V cost.

Interpreter cost (rv32im): JUMPDEST DUP1 ADD SWAP1 JUMP

Per-opcode: dispatch overhead (×5 opcodes = 65 instructions)

Each opcode pays ~13 instructions for the interpreter loop. For 5 opcodes: ~65 instructions.

Per-opcode: handler bodies

Measured from the actual compiled rv32im ELF:

JUMPDEST (4 bytes, 1 instruction):

  ret                               ; no-op, just returns

Cost: 1 instruction (plus 13 dispatch = 14 total)

DUP1 (116 bytes, 29 instructions):

  lw    a1, 336(a0)           ; load stack size
  addi  a2, a1, -9            ; check: size >= 10? (underflow+overflow guard)
  li    a3, 1014              ; max stack depth check
  bltu  a3, a2, halt          ; overflow check
  lw    a2, 332(a0)           ; load stack base pointer
  addi  a3, a1, 1             ; new stack size
  slli  a1, a1, 5             ; offset = size * 32
  add   a1, a2, a1            ; pointer to new top
  ; Copy 32 bytes (8 × lw+sw pairs for U256):
  lw    a2, -288(a1)          ; load limb 0 from source
  ; ... 7 more lw/sw pairs ...
  sw    a6, 28(a1)            ; store limb 7
  sw    a3, 336(a0)           ; update stack size
  ret

Cost: 29 instructions (stack underflow check + overflow check + 16 lw/sw for U256 copy + bookkeeping)

ADD (312 bytes, 78 instructions):

  lw    a1, 336(a0)           ; stack size
  li    a2, 2                 ; need >= 2
  bltu  a1, a2, halt          ; underflow check
  ; Pop 2 U256 (16 lw):
  lw    a2, 332(a0)           ; stack base
  addi  a3, a1, -1            ; new size
  slli  a1, a1, 5
  sw    a3, 336(a0)           ; update stack size
  add   a0, a1, a2
  ; Load 16 limbs (2 x 8 limbs):
  lw    a3, -32(a0)           ; B.limb0
  ; ... 14 more lw ...
  lw    t3, -52(a0)           ; A.limb7
  ; 8-limb addition with carry chain (~40 instructions):
  add   a1, a3, a1            ; limb0
  sltu  t1, a1, a3            ; carry
  ; ... carry propagation through all 8 limbs ...
  ; Store result (8 sw):
  sw    a1, -64(a0)           ; result.limb0
  ; ... 7 more sw ...
  ret

Cost: 78 instructions (underflow check + 16 loads + ~40 add/carry + 8 stores + bookkeeping)

SWAP1 — not found as a separate symbol (likely inlined or in a swap generic). Estimated from pattern:

  ; Load stack size, compute addresses (4 insns)
  ; Swap 32 bytes between stack[-32] and stack[-64] (16 lw + 16 sw = 32 insns)
  ; Total: ~36 instructions

Cost: ~36 instructions

JUMP (from the control module, estimated ~20 instructions):

  ; Pop target PC from stack (load U256, take low word)
  ; Validate jump destination (check jumpdest table)
  ; Set new PC
  ; Total: ~20 instructions

Cost: ~20 instructions

Total interpreter cost for the basic block

Opcode	Handler body	Dispatch overhead	Total
JUMPDEST	1	13	14
DUP1	29	13	42
ADD	78	13	91
SWAP1	~36	13	~49
JUMP	~20	13	~33
Total	~164	65	~229

Compiled cost (rv64): same basic block

With direct compilation (Solidity unchecked { } → resolc → rv64), the EVM sequence DUP1 ADD duplicates the top value and adds it to itself, i.e., x = x * 2.

  ;; x = x + x (256-bit doubling)
  add     a1, a0, a0            ; limb0 = limb0 + limb0
  sltu    a5, a1, a0            ; carry0
  add     a2, a2, a2            ; limb1 = limb1 + limb1
  add     a2, a2, a5            ; + carry0
  sltu    a5, a2, a5            ; carry1
  add     a3, a3, a3            ; limb2
  add     a3, a3, a5            ; + carry1
  sltu    a5, a3, a5            ; carry2
  add     a4, a4, a4            ; limb3
  add     a4, a4, a5            ; + carry2

Plus loop branch: 1-2 instructions (bne or j).

But this understates the real cost. See the corrections below.

What can and cannot be eliminated

1. Dispatch (65 insns, 28%) — largely eliminable

Opcode fetch, table lookup, and indirect call are fully eliminated by compilation. The control flow is baked into the compiled code.

Gas metering is more nuanced. For a zkEVM that must produce identical results to an execution client, we need to track gas. However:

• Gas for a basic block can be precomputed at compile time (sum of static gas costs for all opcodes in the block) and checked once at the block entry, instead of per-opcode. This turns 5 gas checks into 1.
• Some opcodes have dynamic gas (SLOAD, CALL, memory expansion). These still need runtime gas checks.
• For a pure-computation basic block like ours (no memory/storage), a single gas check at entry suffices.

Realistic saving: ~50 of the 65 dispatch instructions (keep ~15 for a single batched gas check + loop control).

2. Stack checks (43 insns, 19%) — mostly required

The interpreter checks stack underflow/overflow on every opcode. In a compiled version:

For Solidity-compiled code: The Solidity compiler generates well-formed EVM bytecode where stack discipline is guaranteed by construction. A compiler that knows this can prove stack safety statically and skip runtime checks entirely.

For arbitrary EVM bytecode: We must match EVM semantics exactly. If the bytecode would underflow, we must revert. However:

• The net stack effect of a basic block is known at compile time (e.g., our block: JUMPDEST +0, DUP1 +1, ADD -1, SWAP1 +0, JUMP -1 = net -1). A single check at entry ("stack has at least N items and at most 1024-M") replaces per-opcode checks.
• The EVM spec guarantees stack depth <= 1024. This can be checked once per basic block entry.

Realistic saving: ~35 of the 43 instructions (replace 5 per-opcode checks with 1 basic-block-entry check, ~8 instructions).

3. EVM stack I/O (72 insns, 31%) — reducible but not eliminable

This is the most misunderstood part. The interpreter loads each U256 operand from the EVM stack (a memory array) before every operation and stores results back after. In a compiled version:

The stack doesn't disappear. Values still need to live somewhere. On rv32, a single U256 is 8 registers — with 15 usable registers, you can hold fewer than 2 U256 values simultaneously. On rv64, it's 4 registers per U256, so ~3-4 live values before spilling. The resolc MUL showed this: even on rv64, the compiler emitted 50 ld/sd spills for a single multiply with overflow checking.

What the compiler CAN do:

• Reuse values across consecutive opcodes. The interpreter's DUP1; ADD sequence loads the top-of-stack for DUP1, stores the copy, then loads both copies again for ADD. A compiler keeps the value in registers between the two operations, saving one round-trip (~8 lw + 8 sw = 16 insns on rv32).
• Schedule spills optimally. Instead of a mandatory load/store for every opcode, the compiler uses register allocation to minimize total memory traffic across the entire basic block.
• Avoid copies for DUP/SWAP. These are pure stack shuffles that a compiler represents as register renaming — no memory traffic needed unless the register file is exhausted.

What the compiler CANNOT eliminate:

• Values that are live across basic block boundaries must be in memory (spilled to a stack frame, analogous to the EVM stack).
• With >3-4 live U256 values, the compiler spills to memory just like the interpreter — just more efficiently.

Realistic saving on rv32: ~40 of the 72 instructions (eliminate the DUP1 round-trip and optimize ADD's operand fetch, but ADD still needs to load at least one operand from memory and store the result).

Realistic saving on rv64: ~20 of the ~36 instructions (same logic, halved by 64-bit loads).

4. Actual computation (49 insns, 22%) — irreducible on same ISA

The 256-bit add with carry chain is the same work regardless of interpreter or compiler. On rv32: ~40 insns for 8-limb add. On rv64: ~10 insns for 4-limb add. This is purely a function of the ISA word size.

Revised comparison

Category	Interpreter (rv32)	Compiled (rv32, est.)	Compiled (rv64, est.)
Dispatch	65	~15 (batched gas check)	~15
Stack checks	43	~8 (one check at BB entry)	~8
Stack I/O	72	~32 (optimized spills)	~16
Computation	49	49 (same ISA)	~10
Total	229	~104	~49
Speedup vs rv32 interpreter	1x	~2.2x	~4.7x

Where the savings come from

  Interpreter rv32                     Compiled rv32 (est.)
  229 instructions                     ~104 instructions (~2.2x)
  ┌──────────────────────────┐         ┌──────────────────────────┐
  │ Dispatch     65 (28%)    │    →    │ Batched gas   15 (14%)   │
  │ Stack checks 43 (19%)    │    →    │ BB-entry chk   8  (8%)   │
  │ Stack I/O    72 (31%)    │    →    │ Opt. spills   32 (31%)   │
  │ Computation  49 (22%)    │    →    │ Computation   49 (47%)   │
  └──────────────────────────┘         └──────────────────────────┘

  Interpreter rv32                     Compiled rv64 (est.)
  229 instructions                     ~49 instructions (~4.7x)
  ┌──────────────────────────┐         ┌──────────────────────────┐
  │ Dispatch     65 (28%)    │    →    │ Batched gas   15 (31%)   │
  │ Stack checks 43 (19%)    │    →    │ BB-entry chk   8 (16%)   │
  │ Stack I/O    72 (31%)    │    →    │ Opt. spills   16 (33%)   │
  │ Computation  49 (22%)    │    →    │ Computation   10 (20%)   │
  └──────────────────────────┘         └──────────────────────────┘

Key takeaways

1. The 19x number from the previous analysis was wrong. It assumed zero overhead for the compiled case, which isn't realistic. A more honest estimate is ~2.2x on the same ISA (rv32) or ~4.7x with the rv64 ISA advantage.

2. The biggest single saving is dispatch elimination (~50 insns saved), specifically removing per-opcode gas checks, table lookups, and indirect calls. Batching gas to once-per-basic-block is the key optimization.

3. Stack I/O savings are real but modest. The compiler can reuse values across consecutive opcodes and avoid redundant copies for DUP/SWAP, but it still spills to memory when register pressure is high. On rv32 with U256, register pressure is always high.

4. Stack checks can be batched but not eliminated for arbitrary EVM bytecode. One check per basic block entry replaces N per-opcode checks.

5. The ISA matters more than the compilation strategy. Going from rv32 to rv64 gives ~2x improvement within the interpreter (4 limbs vs 8 limbs). Direct compilation on the same ISA gives ~2x improvement from reduced overhead. Combined (rv32 interpreter → rv64 compiled): ~4.7x.

6. This analysis is for pure-computation basic blocks. For blocks with SLOAD/SSTORE/CALL, the opcode handler cost dominates (1000+ instructions for SLOAD with cold access), and the interpreter dispatch overhead becomes proportionally negligible.