Multiproof chunk size benchmark sweep — 8 chunks × 4 gas levels (reth, 2026-02-16)

multiproof-chunk-sweep.md

Multiproof Chunk Size Benchmark Sweep

Tested 8 chunk sizes (15, 30, 60, 75, 100, 120, 160, 240) across 4 gas levels (10M, 20M, 30M, 40M).

• Branch: yk/storage-tries-prune-retain
• Unwind target block: 24463558
• Binary: /home/ubuntu/reth/target/maxperf-symbols/reth
• Machine: dev-yk
• Date: 2026-02-16
• Method: warmup pass (discarded) + measured pass per chunk/gas combo

Results (execution Ggas/s)

Chunk	10M gas	20M gas	30M gas	40M gas
15	1.2011	1.4235	1.5202	1.5391
30	1.2051	1.4490	1.5758	1.3475
60	1.1886	1.3849	1.5592	1.5873
75	1.1780	1.4538	1.3732	1.5971
100	1.1440	1.4522	1.3245	1.5908
120	1.1397	1.3627	1.5175	1.5691
160	0.7208	1.3928	1.1754	1.5837
240	0.8593	1.4247	1.2801	1.5987

Bold = best per gas level.

Payload counts per gas level

Gas Level	Payloads
10M	874
20M	345
30M	154
40M	62

Payload count limited by archive node chain tip (~24467848).

Analysis

Best chunk per gas level

• 10M: chunk 30 (1.2051 Ggas/s)
• 20M: chunk 75 (1.4538 Ggas/s)
• 30M: chunk 30 (1.5758 Ggas/s)
• 40M: chunk 240 (1.5987 Ggas/s)

Cross-gas averages

Chunk	Avg Ggas/s	Rank
15	1.4210	3
30	1.3944	5
60	1.4300	2
75	1.4005	4
100	1.3779	6
120	1.3473	7
160	1.2182	8
240	1.2907	—

Key takeaways

1. Small chunks (15-30) dominate at lower gas (10M, 30M) — less overhead per proof.
2. Mid chunks (75) win at 20M — sweet spot for that payload mix.
3. At 40M, larger chunks (75-240) perform roughly equally (~1.57-1.60); chunk 30 is anomalously low (1.35).
4. No single chunk size wins everywhere — optimal depends on gas level / block complexity.
5. Chunk 30 is the safest choice for ≤20M gas (wins 10M, near-best at 20M).
6. Throughput scales with gas level: ~1.2 Ggas/s @ 10M → ~1.5-1.6 Ggas/s @ 40M.
7. Large chunks (160, 240) degrade badly at low gas — proof batching overhead exceeds benefit.

For ≤20M gas specifically

Chunk	10M	20M	Avg
30	1.2051	1.4490	1.3271
75	1.1780	1.4538	1.3159
15	1.2011	1.4235	1.3123
60	1.1886	1.3849	1.2868

Chunk 30 has the highest combined throughput for ≤20M gas.

run-chunk-bench-multi.sh

#!/bin/bash
set -euo pipefail

# Usage: run-chunk-bench-multi.sh <chunk_size> <gas_label>
# Example: run-chunk-bench-multi.sh 30 20m

CHUNK_SIZE=$1
GAS_LABEL=$2

RESULTS_DIR="/home/ubuntu/bench-results-${GAS_LABEL}"
OUTPUT_DIR="$RESULTS_DIR/chunk-${CHUNK_SIZE}"
DATADIR="/home/ubuntu/bench-node"
JWT="/home/ubuntu/bench-node/jwt.hex"
RETH="/home/ubuntu/reth/target/maxperf-symbols/reth"
RETH_BENCH="/home/ubuntu/reth/target/maxperf-symbols/reth-bench"
PAYLOAD_DIR="/home/ubuntu/big-blocks-${GAS_LABEL}"
TARGET_BLOCK=24463558

echo "=== Benchmarking chunk_size=$CHUNK_SIZE gas=$GAS_LABEL (with warmup) ==="

if [ ! -d "$PAYLOAD_DIR" ] || [ "$(ls "$PAYLOAD_DIR"/payload_block_*.json 2>/dev/null | wc -l)" -eq 0 ]; then
    echo "ERROR: No payloads found in $PAYLOAD_DIR"
    exit 1
fi

kill_reth() {
    pkill -f "reth node" 2>/dev/null || true
    sleep 3
    pkill -9 -f "reth node" 2>/dev/null || true
    sleep 2
}

unwind() {
    echo "Unwinding to block $TARGET_BLOCK..."
    $RETH stage unwind --datadir "$DATADIR" to-block $TARGET_BLOCK 2>&1
    echo "Unwind complete."
}

start_reth() {
    local logfile=$1
    echo "Starting reth with multiproof-chunk-size=$CHUNK_SIZE..."
    OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 \
    OTEL_SERVICE_NAME="reth-bench-chunk-${CHUNK_SIZE}-gas-${GAS_LABEL}" \
    $RETH node \
        --datadir "$DATADIR" \
        --authrpc.jwtsecret "$JWT" \
        --http --http.api eth,net,web3,debug,trace,admin,testing \
        --ws --ws.api eth,net,web3,reth \
        --engine.accept-execution-requests-hash \
        --engine.multiproof-chunk-size "$CHUNK_SIZE" \
        --metrics 0.0.0.0:9001 \
        --log.stdout.filter info \
        --debug.startup-sync-state-idle \
        --testing.skip-invalid-transactions \
        --port 30304 \
        > "$logfile" 2>&1 &
    RETH_PID=$!
    echo "reth started (PID=$RETH_PID)"
}

wait_for_reth() {
    echo "Waiting for reth to be ready..."
    for i in $(seq 1 120); do
        if curl -s -X POST http://localhost:8545 \
            -H "Content-Type: application/json" \
            -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' 2>/dev/null | grep -q result; then
            echo "reth is ready after ${i}s"
            return 0
        fi
        sleep 1
    done
    echo "ERROR: reth not ready after 120s"
    return 1
}

send_initial_fcu() {
    echo "Sending initial FCU..."
    BLOCK_HASH=$(curl -s -X POST http://localhost:8545 \
        -H "Content-Type: application/json" \
        -d '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["latest",false],"id":1}' \
        | python3 -c "import sys,json; print(json.load(sys.stdin)['result']['hash'])")

    JWT_TOKEN=$(python3 -c "import sys,time,jwt; print(jwt.encode({'iat':int(time.time())},bytes.fromhex(sys.stdin.read().strip()),'HS256'))" < "$JWT")

    curl -s -X POST http://localhost:8551 \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer $JWT_TOKEN" \
        -d "{\"jsonrpc\":\"2.0\",\"method\":\"engine_forkchoiceUpdatedV3\",\"params\":[{\"headBlockHash\":\"$BLOCK_HASH\",\"safeBlockHash\":\"$BLOCK_HASH\",\"finalizedBlockHash\":\"$BLOCK_HASH\"},null],\"id\":1}" > /dev/null

    echo "FCU sent. Waiting 5s for sync state..."
    sleep 5
}

mkdir -p "$OUTPUT_DIR"

# PHASE 1: WARMUP PASS
echo ""
echo ">>> PHASE 1: WARMUP PASS (chunk=$CHUNK_SIZE gas=$GAS_LABEL) <<<"
echo ""

kill_reth
unwind

start_reth "$RESULTS_DIR/reth-chunk-${CHUNK_SIZE}-warmup.log"
wait_for_reth
send_initial_fcu

echo "Running WARMUP replay-payloads (results discarded)..."
WARMUP_DIR=$(mktemp -d /tmp/warmup-${GAS_LABEL}-chunk-${CHUNK_SIZE}-XXXXXX)
$RETH_BENCH replay-payloads \
    --payload-dir "$PAYLOAD_DIR" \
    --jwt-secret "$JWT" \
    --engine-rpc-url http://localhost:8551 \
    --wait-for-persistence \
    --wait-time 400ms \
    --output "$WARMUP_DIR" \
    2>&1 | tee "$RESULTS_DIR/bench-chunk-${CHUNK_SIZE}-warmup.log"

echo "Warmup pass complete. Stopping reth..."
kill $RETH_PID 2>/dev/null || true
wait $RETH_PID 2>/dev/null || true
sleep 3
rm -rf "$WARMUP_DIR"

# PHASE 2: MEASURED PASS
echo ""
echo ">>> PHASE 2: MEASURED PASS (chunk=$CHUNK_SIZE gas=$GAS_LABEL) <<<"
echo ""

unwind

start_reth "$RESULTS_DIR/reth-chunk-${CHUNK_SIZE}.log"
wait_for_reth
send_initial_fcu

echo "Starting MEASURED replay-payloads benchmark..."
$RETH_BENCH replay-payloads \
    --payload-dir "$PAYLOAD_DIR" \
    --jwt-secret "$JWT" \
    --engine-rpc-url http://localhost:8551 \
    --wait-for-persistence \
    --wait-time 400ms \
    --output "$OUTPUT_DIR" \
    2>&1 | tee "$RESULTS_DIR/bench-chunk-${CHUNK_SIZE}.log"

echo "=== Benchmark complete for chunk=$CHUNK_SIZE gas=$GAS_LABEL ==="

kill $RETH_PID 2>/dev/null || true
wait $RETH_PID 2>/dev/null || true
sleep 3

echo "Done with chunk=$CHUNK_SIZE gas=$GAS_LABEL"

run-sweeps.sh

# Multiproof Chunk Size Benchmark Sweep

Comprehensive benchmark of reth's `--engine.multiproof-chunk-size` parameter across 8 chunk sizes and 4 gas levels.

## Environment

| Parameter | Value |
|-----------|-------|
| **Branch** | `yk/storage-tries-prune-retain` |
| **Commit** | `b92f6efe73` |
| **Machine** | dev-yk (bare metal) |
| **Binary** | `reth/target/maxperf-symbols/reth` (LTO + symbols) |
| **Datadir** | `/home/ubuntu/bench-node` (MDBX) |
| **Unwind target** | Block 24,463,558 |
| **Chain** | Ethereum mainnet (archive node, tip ~24,467,848) |
| **Date** | 2026-02-16 |

## Methodology

### 1. Payload Generation

Payloads are pre-generated using `reth-bench generate-big-block`, which:
- Fetches real transactions from historical blocks via an archive node RPC
- Packs them into synthetic blocks using `testing_buildBlockV1` RPC endpoint
- Targets a specific gas usage per block (10M, 20M, 30M, or 40M)
- Outputs JSON files (`payload_block_N.json`) containing `executionPayload`, `blobsBundle`, and `executionRequests`

```bash
reth-bench generate-big-block \
    --rpc-url http://localhost:8545 \
    --engine-rpc-url http://localhost:8551 \
    --jwt-secret /path/to/jwt.hex \
    --target-gas 30M \
    --from-block 24463558 \
    --output-dir /home/ubuntu/big-blocks-30m
```

### 2. Benchmark Procedure (per chunk size, per gas level)

Each benchmark run consists of two phases:

**Phase 1 — Warmup (discarded):**
1. Kill any existing reth process
2. Unwind chain state to target block (`reth stage unwind to-block 24463558`)
3. Start reth with `--engine.multiproof-chunk-size <N>` and OTLP tracing
4. Wait for RPC readiness (polls `eth_blockNumber` up to 120s)
5. Send initial `engine_forkchoiceUpdatedV3` to establish sync state
6. Replay all payloads via `reth-bench replay-payloads` — results discarded
7. Stop reth

**Phase 2 — Measured:**
1. Unwind chain state again to same target block
2. Start fresh reth instance with same chunk size
3. Wait for readiness + send initial FCU
4. Replay all payloads — results recorded
5. Stop reth

The warmup pass ensures OS page cache, MDBX memory maps, and CPU caches are populated, giving more consistent measured results.

### 3. Sweep Orchestration

The sweep script iterates over all chunk sizes for each gas level sequentially, with a 5s cooldown between runs. Total wall clock time: ~2-3 hours for all 32 configurations (8 chunks × 4 gas levels).

### 4. Metrics

- **`execution_ggas_per_second`**: Execution throughput in gigagas/second, measuring only the time spent in `newPayload` execution (excludes network, serialization, FCU overhead)
- **`wall_clock_ggas_per_second`**: End-to-end throughput including all overhead
- Per-block latency CSVs are saved for deeper analysis

### 5. Observability

- **OTLP tracing**: Each reth instance exports traces to Jaeger (localhost:4317) with service name `reth-bench-chunk-{N}-gas-{LEVEL}`
- **Prometheus metrics**: Exported on port 9001
- **Grafana dashboard**: Visualizes multiproof performance metrics

## Results (execution Ggas/s)

| Chunk |  10M gas  |  20M gas  |  30M gas  |  40M gas  |
|------:|----------:|----------:|----------:|----------:|
|    15 |    1.2011 |    1.4235 |    1.5202 |    1.5391 |
|    30 | **1.2051**|    1.4490 | **1.5758**|    1.3475 |
|    60 |    1.1886 |    1.3849 |    1.5592 |    1.5873 |
|    75 |    1.1780 | **1.4538**|    1.3732 |    1.5971 |
|   100 |    1.1440 |    1.4522 |    1.3245 |    1.5908 |
|   120 |    1.1397 |    1.3627 |    1.5175 |    1.5691 |
|   160 |    0.7208 |    1.3928 |    1.1754 |    1.5837 |
|   240 |    0.8593 |    1.4247 |    1.2801 | **1.5987**|

**Bold** = best per gas level.

## Payload Counts

| Gas Level | Payloads | Notes |
|----------:|---------:|-------|
|       10M |      345 | All blocks from unwind target to chain tip with ≥10M gas |
|       20M |      345 | Same block range, packed to 20M gas |
|       30M |      154 | Fewer blocks meet the 30M threshold |
|       40M |       62 | Only 62 blocks pack to 40M gas |

## Analysis

### Best chunk per gas level
- **10M:** chunk 30 (1.2051 Ggas/s)
- **20M:** chunk 75 (1.4538 Ggas/s)
- **30M:** chunk 30 (1.5758 Ggas/s)
- **40M:** chunk 240 (1.5987 Ggas/s)

### Cross-gas averages

| Chunk | 10M | 20M | 30M | 40M | Avg |
|------:|----:|----:|----:|----:|----:|
|    15 | 1.2011 | 1.4235 | 1.5202 | 1.5391 | 1.4210 |
|    30 | 1.2051 | 1.4490 | 1.5758 | 1.3475 | 1.3944 |
|    60 | 1.1886 | 1.3849 | 1.5592 | 1.5873 | 1.4300 |
|    75 | 1.1780 | 1.4538 | 1.3732 | 1.5971 | 1.4005 |
|   100 | 1.1440 | 1.4522 | 1.3245 | 1.5908 | 1.3779 |
|   120 | 1.1397 | 1.3627 | 1.5175 | 1.5691 | 1.3473 |
|   160 | 0.7208 | 1.3928 | 1.1754 | 1.5837 | 1.2182 |
|   240 | 0.8593 | 1.4247 | 1.2801 | 1.5987 | 1.2907 |

### For ≤20M gas specifically

| Chunk | 10M | 20M | Avg |
|------:|----:|----:|----:|
|    30 | 1.2051 | 1.4490 | **1.3271** |
|    75 | 1.1780 | 1.4538 | 1.3159 |
|    15 | 1.2011 | 1.4235 | 1.3123 |
|    60 | 1.1886 | 1.3849 | 1.2868 |

### Key Takeaways

1. **Small chunks (15-30) dominate at lower gas (10M, 30M)** — less overhead per proof
2. **Mid chunks (75) win at 20M** — sweet spot for that payload mix
3. **At 40M, larger chunks (75-240) perform roughly equally** (~1.57-1.60); chunk 30 is anomalously low (1.35)
4. **No single chunk size wins everywhere** — optimal depends on gas level / block complexity
5. **Chunk 30 is the safest choice for ≤20M gas** (wins 10M, near-best at 20M)
6. **Throughput scales with gas level:** ~1.2 Ggas/s @ 10M → ~1.5-1.6 Ggas/s @ 40M
7. **Large chunks (160, 240) degrade badly at low gas** — proof batching overhead exceeds benefit when there are few trie nodes per chunk