Skip to content

Instantly share code, notes, and snippets.

@jsign

jsign/cmp

Last active July 31, 2025 20:38
Show Gist options
  • Save jsign/0bfb5fb0196ccbaef74691e6fbf8c3c0 to your computer and use it in GitHub Desktop.
Save jsign/0bfb5fb0196ccbaef74691e6fbf8c3c0 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
cmp
Loading baseline metrics from: /home/ignacio/code/zkevm-benchmark-workload/zkevm-metrics
Found subfolders with metrics: ['sp1-v5.1.0']
Loaded 100 baseline files
Loading optimized metrics from: /home/ignacio/code/zkevm-benchmark-workload/local-optimized-zkevm-metrics
Found subfolders with metrics: ['sp1-v5.1.0']
Loaded 100 optimized files
Calculating speedups...
Found 100 common files with 8 regions
Regions found: block_execution, decode_headers, post_state_compute, read_input, recover_signers, validation, verify_witness, total_num_cycles
================================================================================
SPEEDUP COMPARISON TABLE
================================================================================
File block_execution decode_headers post_state_computeread_input recover_signers validation verify_witness total_num_cycles
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sp1-v5.1.0/rpc_block_22974575 0.72x 1.00x 4.84x 1.00x 1.00x 1.69x 3.15x 1.67x
sp1-v5.1.0/rpc_block_22974576 0.66x 1.00x 4.89x 1.00x 1.00x 1.83x 2.97x 1.80x
sp1-v5.1.0/rpc_block_22974577 0.74x 1.00x 4.84x 1.00x 1.00x 1.66x 3.01x 1.64x
sp1-v5.1.0/rpc_block_22974578 0.79x 1.00x 4.91x 1.00x 1.00x 1.60x 3.34x 1.59x
sp1-v5.1.0/rpc_block_22974579 0.77x 1.00x 4.81x 1.00x 1.00x 1.63x 2.81x 1.60x
sp1-v5.1.0/rpc_block_22974580 0.86x 1.00x 4.72x 1.00x 1.00x 1.45x 3.02x 1.44x
sp1-v5.1.0/rpc_block_22974581 0.75x 1.00x 4.84x 1.00x 1.00x 1.73x 2.96x 1.71x
sp1-v5.1.0/rpc_block_22974582 0.81x 1.00x 4.74x 1.00x 1.00x 1.55x 3.12x 1.54x
sp1-v5.1.0/rpc_block_22974583 0.81x 1.00x 4.70x 1.00x 1.00x 1.56x 3.02x 1.55x
sp1-v5.1.0/rpc_block_22974584 0.79x 1.00x 4.77x 1.00x 1.00x 1.58x 3.13x 1.57x
sp1-v5.1.0/rpc_block_22974585 0.78x 1.00x 4.71x 1.00x 1.00x 1.61x 2.92x 1.60x
sp1-v5.1.0/rpc_block_22974586 0.77x 1.00x 4.74x 1.00x 1.00x 1.65x 3.02x 1.62x
sp1-v5.1.0/rpc_block_22974587 0.76x 1.00x 4.73x 1.00x 1.00x 1.61x 3.22x 1.59x
sp1-v5.1.0/rpc_block_22974588 0.74x 1.00x 4.80x 1.00x 1.00x 1.71x 3.12x 1.68x
sp1-v5.1.0/rpc_block_22974589 0.78x 1.00x 4.77x 1.00x 1.00x 1.62x 3.24x 1.60x
sp1-v5.1.0/rpc_block_22974590 0.80x 1.00x 4.80x 1.00x 1.00x 1.60x 2.88x 1.58x
sp1-v5.1.0/rpc_block_22974591 0.79x 1.00x 4.77x 1.00x 1.00x 1.59x 3.20x 1.57x
sp1-v5.1.0/rpc_block_22974592 0.80x 1.00x 4.82x 1.00x 1.00x 1.59x 2.97x 1.57x
sp1-v5.1.0/rpc_block_22974593 0.76x 1.00x 4.88x 1.00x 1.00x 1.67x 3.06x 1.66x
sp1-v5.1.0/rpc_block_22974594 0.80x 1.00x 4.78x 1.00x 1.00x 1.56x 3.09x 1.54x
sp1-v5.1.0/rpc_block_22974595 0.79x 1.00x 4.80x 1.00x 1.00x 1.60x 3.10x 1.58x
sp1-v5.1.0/rpc_block_22974596 0.81x 1.00x 4.76x 1.00x 1.00x 1.54x 3.06x 1.53x
sp1-v5.1.0/rpc_block_22974597 0.79x 1.00x 4.74x 1.00x 1.00x 1.57x 3.15x 1.56x
sp1-v5.1.0/rpc_block_22974598 0.82x 1.00x 4.76x 1.00x 1.00x 1.57x 2.97x 1.55x
sp1-v5.1.0/rpc_block_22974599 0.78x 1.00x 4.89x 1.00x 1.00x 1.64x 3.05x 1.62x
sp1-v5.1.0/rpc_block_22974600 0.80x 1.00x 4.77x 1.00x 1.00x 1.58x 3.19x 1.56x
sp1-v5.1.0/rpc_block_22974601 0.79x 1.00x 4.82x 1.00x 1.00x 1.54x 3.31x 1.53x
sp1-v5.1.0/rpc_block_22974602 0.84x 1.00x 4.79x 1.00x 1.00x 1.50x 3.02x 1.48x
sp1-v5.1.0/rpc_block_22974603 0.78x 1.00x 4.84x 1.00x 1.00x 1.61x 2.99x 1.59x
sp1-v5.1.0/rpc_block_22974604 0.78x 1.00x 4.76x 1.00x 1.00x 1.62x 3.10x 1.60x
sp1-v5.1.0/rpc_block_22974605 0.74x 1.00x 4.84x 1.00x 1.00x 1.66x 3.05x 1.64x
sp1-v5.1.0/rpc_block_22974606 0.78x 1.00x 4.80x 1.00x 1.00x 1.61x 3.03x 1.59x
sp1-v5.1.0/rpc_block_22974607 0.77x 1.00x 4.81x 1.00x 1.00x 1.64x 3.16x 1.62x
sp1-v5.1.0/rpc_block_22974608 0.77x 1.00x 4.91x 1.00x 1.00x 1.60x 3.22x 1.58x
sp1-v5.1.0/rpc_block_22974609 0.78x 1.00x 4.78x 1.00x 1.00x 1.60x 3.28x 1.58x
sp1-v5.1.0/rpc_block_22974610 0.76x 1.00x 4.83x 1.00x 1.00x 1.61x 3.23x 1.59x
sp1-v5.1.0/rpc_block_22974611 0.78x 1.00x 4.84x 1.00x 1.00x 1.66x 2.90x 1.64x
sp1-v5.1.0/rpc_block_22974612 0.77x 1.00x 4.90x 1.00x 1.00x 1.62x 3.19x 1.61x
sp1-v5.1.0/rpc_block_22974613 0.74x 1.00x 4.82x 1.00x 1.00x 1.69x 3.08x 1.66x
sp1-v5.1.0/rpc_block_22974614 0.77x 1.00x 4.82x 1.00x 1.00x 1.65x 3.03x 1.63x
sp1-v5.1.0/rpc_block_22974615 0.77x 1.00x 4.85x 1.00x 1.00x 1.63x 3.17x 1.62x
sp1-v5.1.0/rpc_block_22974616 0.74x 1.00x 4.89x 1.00x 1.00x 1.75x 2.82x 1.72x
sp1-v5.1.0/rpc_block_22974617 0.76x 1.00x 4.78x 1.00x 1.00x 1.63x 3.39x 1.61x
sp1-v5.1.0/rpc_block_22974618 0.77x 1.00x 4.83x 1.00x 1.00x 1.60x 3.08x 1.58x
sp1-v5.1.0/rpc_block_22974619 0.81x 1.00x 4.82x 1.00x 1.00x 1.55x 3.08x 1.54x
sp1-v5.1.0/rpc_block_22974620 0.78x 1.00x 4.83x 1.00x 1.00x 1.58x 2.91x 1.56x
sp1-v5.1.0/rpc_block_22974621 0.71x 1.00x 4.89x 1.00x 1.00x 1.69x 3.10x 1.67x
sp1-v5.1.0/rpc_block_22974622 0.77x 1.00x 4.88x 1.00x 1.00x 1.65x 2.81x 1.63x
sp1-v5.1.0/rpc_block_22974623 0.82x 1.00x 4.76x 1.00x 1.00x 1.54x 3.13x 1.53x
sp1-v5.1.0/rpc_block_22974624 0.75x 1.00x 4.91x 1.00x 1.00x 1.69x 3.10x 1.66x
sp1-v5.1.0/rpc_block_22974625 0.75x 1.00x 4.82x 1.00x 1.00x 1.62x 3.18x 1.60x
sp1-v5.1.0/rpc_block_22974626 0.77x 1.00x 4.82x 1.00x 1.00x 1.60x 3.13x 1.58x
sp1-v5.1.0/rpc_block_22974627 0.80x 1.00x 4.74x 1.00x 1.00x 1.59x 3.05x 1.57x
sp1-v5.1.0/rpc_block_22974628 0.77x 1.00x 4.84x 1.00x 1.00x 1.62x 3.08x 1.60x
sp1-v5.1.0/rpc_block_22974629 0.77x 1.00x 4.85x 1.00x 1.00x 1.59x 3.19x 1.58x
sp1-v5.1.0/rpc_block_22974630 0.77x 1.00x 4.83x 1.00x 1.00x 1.65x 3.17x 1.62x
sp1-v5.1.0/rpc_block_22974631 0.75x 1.00x 4.82x 1.00x 1.00x 1.67x 3.13x 1.65x
sp1-v5.1.0/rpc_block_22974632 0.75x 1.00x 4.83x 1.00x 1.00x 1.64x 3.19x 1.62x
sp1-v5.1.0/rpc_block_22974633 0.74x 1.00x 4.82x 1.00x 1.00x 1.68x 3.07x 1.66x
sp1-v5.1.0/rpc_block_22974634 0.77x 1.00x 4.87x 1.00x 1.00x 1.67x 3.06x 1.64x
sp1-v5.1.0/rpc_block_22974635 0.76x 1.00x 4.82x 1.00x 1.00x 1.66x 2.97x 1.63x
sp1-v5.1.0/rpc_block_22974636 0.76x 1.00x 4.82x 1.00x 1.00x 1.64x 2.99x 1.61x
sp1-v5.1.0/rpc_block_22974637 0.79x 1.00x 4.85x 1.00x 1.00x 1.58x 3.11x 1.57x
sp1-v5.1.0/rpc_block_22974638 0.79x 1.00x 5.05x 1.00x 1.00x 1.68x 3.06x 1.66x
sp1-v5.1.0/rpc_block_22974639 0.78x 1.00x 4.80x 1.00x 1.00x 1.62x 3.22x 1.60x
sp1-v5.1.0/rpc_block_22974640 0.87x 1.00x 4.84x 1.00x 1.00x 1.40x 3.14x 1.39x
sp1-v5.1.0/rpc_block_22974641 0.76x 1.00x 4.84x 1.00x 1.00x 1.63x 2.97x 1.61x
sp1-v5.1.0/rpc_block_22974642 0.69x 1.00x 4.93x 1.00x 1.00x 1.79x 2.91x 1.76x
sp1-v5.1.0/rpc_block_22974643 0.74x 1.00x 4.85x 1.00x 1.00x 1.67x 3.04x 1.66x
sp1-v5.1.0/rpc_block_22974644 0.73x 1.00x 4.84x 1.00x 1.00x 1.67x 3.08x 1.65x
sp1-v5.1.0/rpc_block_22974645 0.72x 1.00x 4.85x 1.00x 1.00x 1.73x 3.06x 1.70x
sp1-v5.1.0/rpc_block_22974646 0.72x 1.00x 4.84x 1.00x 1.00x 1.69x 3.25x 1.66x
sp1-v5.1.0/rpc_block_22974647 0.78x 1.00x 4.85x 1.00x 1.00x 1.62x 3.07x 1.60x
sp1-v5.1.0/rpc_block_22974648 0.76x 1.00x 4.83x 1.00x 1.00x 1.64x 3.12x 1.61x
sp1-v5.1.0/rpc_block_22974649 0.74x 1.00x 4.90x 1.00x 1.00x 1.65x 3.11x 1.63x
sp1-v5.1.0/rpc_block_22974650 0.74x 1.00x 4.87x 1.00x 1.00x 1.68x 3.09x 1.66x
sp1-v5.1.0/rpc_block_22974651 0.74x 1.00x 4.86x 1.00x 1.00x 1.67x 3.03x 1.64x
sp1-v5.1.0/rpc_block_22974652 0.77x 1.00x 4.89x 1.00x 1.00x 1.64x 2.91x 1.62x
sp1-v5.1.0/rpc_block_22974653 0.69x 1.00x 4.98x 1.00x 1.00x 1.78x 3.09x 1.76x
sp1-v5.1.0/rpc_block_22974654 0.78x 1.00x 4.81x 1.00x 1.00x 1.59x 3.40x 1.57x
sp1-v5.1.0/rpc_block_22974655 0.76x 1.00x 4.84x 1.00x 1.00x 1.64x 3.08x 1.62x
sp1-v5.1.0/rpc_block_22974656 0.75x 1.00x 4.86x 1.00x 1.00x 1.65x 3.15x 1.63x
sp1-v5.1.0/rpc_block_22974657 0.71x 1.00x 4.82x 1.00x 1.00x 1.75x 3.24x 1.73x
sp1-v5.1.0/rpc_block_22974658 0.73x 1.00x 4.84x 1.00x 1.00x 1.69x 3.16x 1.66x
sp1-v5.1.0/rpc_block_22974659 0.73x 1.00x 4.83x 1.00x 1.00x 1.71x 3.08x 1.68x
sp1-v5.1.0/rpc_block_22974660 0.73x 1.00x 4.86x 1.00x 1.00x 1.67x 3.15x 1.65x
sp1-v5.1.0/rpc_block_22974661 0.74x 1.00x 4.91x 1.00x 1.00x 1.68x 3.09x 1.65x
sp1-v5.1.0/rpc_block_22974662 0.73x 1.00x 4.78x 1.00x 1.00x 1.70x 3.10x 1.68x
sp1-v5.1.0/rpc_block_22974663 0.79x 1.00x 4.92x 1.00x 1.00x 1.57x 3.08x 1.56x
sp1-v5.1.0/rpc_block_22974664 0.67x 1.00x 4.84x 1.00x 1.00x 1.69x 3.66x 1.66x
sp1-v5.1.0/rpc_block_22974665 0.75x 1.00x 4.76x 1.00x 1.00x 1.66x 3.05x 1.64x
sp1-v5.1.0/rpc_block_22974666 0.80x 1.00x 4.99x 1.00x 1.00x 1.56x 3.41x 1.55x
sp1-v5.1.0/rpc_block_22974667 0.74x 1.00x 4.83x 1.00x 1.00x 1.64x 3.16x 1.61x
sp1-v5.1.0/rpc_block_22974668 0.75x 1.00x 4.86x 1.00x 1.00x 1.66x 3.01x 1.64x
sp1-v5.1.0/rpc_block_22974669 0.72x 1.00x 4.83x 1.00x 1.00x 1.69x 3.15x 1.67x
sp1-v5.1.0/rpc_block_22974670 0.77x 1.00x 4.80x 1.00x 1.00x 1.60x 3.14x 1.58x
sp1-v5.1.0/rpc_block_22974671 0.75x 1.00x 4.86x 1.00x 1.00x 1.64x 3.11x 1.62x
sp1-v5.1.0/rpc_block_22974672 0.74x 1.00x 4.82x 1.00x 1.00x 1.68x 3.15x 1.66x
sp1-v5.1.0/rpc_block_22974673 0.63x 1.00x 4.94x 1.00x 1.00x 1.92x 2.80x 1.88x
sp1-v5.1.0/rpc_block_22974674 0.77x 1.00x 4.82x 1.00x 1.00x 1.61x 3.22x 1.59x
================================================================================
ANALYSIS
================================================================================
OVERALL SUMMARY:
Note: Speedup < 1.0x means the optimized version is slower (regression)
Speedup > 1.0x means the optimized version is faster (improvement)
BLOCK EXECUTION:
Average speedup: 0.76x (-23.7%)
Min speedup: 0.63x
Max speedup: 0.87x
Top 3 best speedups:
1. sp1-v5.1.0/rpc_block_22974640: 0.87x
2. sp1-v5.1.0/rpc_block_22974580: 0.86x
3. sp1-v5.1.0/rpc_block_22974602: 0.84x
Top 3 worst speedups:
1. sp1-v5.1.0/rpc_block_22974664: 0.67x
2. sp1-v5.1.0/rpc_block_22974576: 0.66x
3. sp1-v5.1.0/rpc_block_22974673: 0.63x
DECODE HEADERS:
Average speedup: 1.00x (+0.0%)
Min speedup: 1.00x
Max speedup: 1.00x
Top 3 best speedups:
1. sp1-v5.1.0/rpc_block_22974635: 1.00x
2. sp1-v5.1.0/rpc_block_22974628: 1.00x
3. sp1-v5.1.0/rpc_block_22974597: 1.00x
Top 3 worst speedups:
1. sp1-v5.1.0/rpc_block_22974577: 1.00x
2. sp1-v5.1.0/rpc_block_22974631: 1.00x
3. sp1-v5.1.0/rpc_block_22974609: 1.00x
POST STATE COMPUTE:
Average speedup: 4.83x (+383.1%)
Min speedup: 4.70x
Max speedup: 5.05x
Top 3 best speedups:
1. sp1-v5.1.0/rpc_block_22974638: 5.05x
2. sp1-v5.1.0/rpc_block_22974666: 4.99x
3. sp1-v5.1.0/rpc_block_22974653: 4.98x
Top 3 worst speedups:
1. sp1-v5.1.0/rpc_block_22974580: 4.72x
2. sp1-v5.1.0/rpc_block_22974585: 4.71x
3. sp1-v5.1.0/rpc_block_22974583: 4.70x
READ INPUT:
Average speedup: 1.00x (+0.0%)
Min speedup: 1.00x
Max speedup: 1.00x
Top 3 best speedups:
1. sp1-v5.1.0/rpc_block_22974639: 1.00x
2. sp1-v5.1.0/rpc_block_22974609: 1.00x
3. sp1-v5.1.0/rpc_block_22974674: 1.00x
Top 3 worst speedups:
1. sp1-v5.1.0/rpc_block_22974599: 1.00x
2. sp1-v5.1.0/rpc_block_22974653: 1.00x
3. sp1-v5.1.0/rpc_block_22974673: 1.00x
RECOVER SIGNERS:
Average speedup: 1.00x (+0.0%)
Min speedup: 1.00x
Max speedup: 1.00x
Top 3 best speedups:
1. sp1-v5.1.0/rpc_block_22974655: 1.00x
2. sp1-v5.1.0/rpc_block_22974587: 1.00x
3. sp1-v5.1.0/rpc_block_22974607: 1.00x
Top 3 worst speedups:
1. sp1-v5.1.0/rpc_block_22974654: 1.00x
2. sp1-v5.1.0/rpc_block_22974669: 1.00x
3. sp1-v5.1.0/rpc_block_22974657: 1.00x
VALIDATION:
Average speedup: 1.64x (+63.6%)
Min speedup: 1.40x
Max speedup: 1.92x
Top 3 best speedups:
1. sp1-v5.1.0/rpc_block_22974673: 1.92x
2. sp1-v5.1.0/rpc_block_22974576: 1.83x
3. sp1-v5.1.0/rpc_block_22974642: 1.79x
Top 3 worst speedups:
1. sp1-v5.1.0/rpc_block_22974602: 1.50x
2. sp1-v5.1.0/rpc_block_22974580: 1.45x
3. sp1-v5.1.0/rpc_block_22974640: 1.40x
VERIFY WITNESS:
Average speedup: 3.10x (+209.8%)
Min speedup: 2.80x
Max speedup: 3.66x
Top 3 best speedups:
1. sp1-v5.1.0/rpc_block_22974664: 3.66x
2. sp1-v5.1.0/rpc_block_22974666: 3.41x
3. sp1-v5.1.0/rpc_block_22974654: 3.40x
Top 3 worst speedups:
1. sp1-v5.1.0/rpc_block_22974622: 2.81x
2. sp1-v5.1.0/rpc_block_22974579: 2.81x
3. sp1-v5.1.0/rpc_block_22974673: 2.80x
TOTAL NUM CYCLES:
Average speedup: 1.62x (+61.7%)
Min speedup: 1.39x
Max speedup: 1.88x
Top 3 best speedups:
1. sp1-v5.1.0/rpc_block_22974673: 1.88x
2. sp1-v5.1.0/rpc_block_22974576: 1.80x
3. sp1-v5.1.0/rpc_block_22974642: 1.76x
Top 3 worst speedups:
1. sp1-v5.1.0/rpc_block_22974602: 1.48x
2. sp1-v5.1.0/rpc_block_22974580: 1.44x
3. sp1-v5.1.0/rpc_block_22974640: 1.39x
================================================================================
KEY FINDINGS
================================================================================
Regions ranked by average speedup:
1. Post State Compute: 4.83x (+383.1%) - IMPROVEMENT
2. Verify Witness: 3.10x (+209.8%) - IMPROVEMENT
3. Validation: 1.64x (+63.6%) - IMPROVEMENT
4. Recover Signers: 1.00x (+0.0%) - NO CHANGE
5. Read Input: 1.00x (+0.0%) - NO CHANGE
6. Decode Headers: 1.00x (+0.0%) - NO CHANGE
7. Block Execution: 0.76x (-23.7%) - REGRESSION
Total files analyzed: 100
🎯 OVERALL PERFORMANCE (Total Num Cycles): 1.62x (+61.7%) - IMPROVEMENT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment