YK yongkangc

RocksDB Multi-Database Consistency Architecture Analysis

Problem Statement

When reth uses multiple storage backends (RocksDB for history indices + MDBX for changesets), a commit order race condition causes execution divergence during block validation.

The bug manifests as "nonce 917 too high, expected 0" because:

RocksDB history indices commit first, saying "account X was modified at block N"
MDBX changesets haven't committed yet
Read path follows RocksDB's pointer to a non-existent changeset

RocksDB Benchmark Methodology & Bug Analysis

Executive Summary

Bug Found: RocksDB pending batch writes are not visible to concurrent history index reads, causing EVM execution divergence during engine_newPayload processing.

The Bug

Root Cause

RocksDB vs MDBX Benchmark Status

Previous Results (INVALID)

The initial benchmarks compared two MDBX datadirs (not RocksDB vs MDBX):

The "RocksDB" datadir was synced before storage settings were enabled
Both used MDBX for all data

Current Status

Probability Matrix Strategy - Steps 3 & 4: Signal Detection

Step 3: Query Probability Matrix

At any moment, you have:

Current market state (crypto, price, time remaining)
Probability matrix with historical win rates

Your goal: Look up the historical data for this exact situation.

	[2m2026-01-20T13:13:15.734451Z[0m [32m INFO[0m Running benchmark using data from RPC URL: https://eth.drpc.org
	[2m2026-01-20T13:13:15.772700Z[0m [32m INFO[0m Connecting to Engine RPC at http://localhost:8551/ for replay
	[2m2026-01-20T13:13:16.002256Z[0m [32m INFO[0m [3mcombined_result[0m[2m=[0mBlock 24262068 processed at 0.9734 Ggas/s, used 20307563 total gas. Combined: 0.9143 Ggas/s. fcu: 1.348709ms, newPayload: 20.862554ms
	[2m2026-01-20T13:13:16.112979Z[0m [32m INFO[0m [3mcombined_result[0m[2m=[0mBlock 24262069 processed at 1.0036 Ggas/s, used 19394546 total gas. Combined: 0.9371 Ggas/s. fcu: 1.37066ms, newPayload: 19.32462ms
	[2m2026-01-20T13:13:16.210811Z[0m [32m INFO[0m [3mcombined_result[0m[2m=[0mBlock 24262070 processed at 0.9430 Ggas/s, used 20225346 total gas. Combined: 0.8675 Ggas/s. fcu: 1.8666ms, newPayload: 21.448162ms
	[2m2026-01-20T13:13:16.333041Z[0m [32m INFO[0m [3mcombined_result[0m[2m=[0mBlock 24262071 processed at 1.6818 Ggas/s, used 49123279 total gas

	Finished `bench` profile [optimized + debuginfo] target(s) in 0.25s
	Running benches/merge_strategies.rs (target/release/deps/merge_strategies-5c4e859c5b1f0919)
	Gnuplot not found, using plotters backend
	Benchmarking HashedPostState_merge/extend_ref_old/n=50_k=3
	Benchmarking HashedPostState_merge/extend_ref_old/n=50_k=3: Warming up for 3.0000 s
	Benchmarking HashedPostState_merge/extend_ref_old/n=50_k=3: Collecting 100 samples in estimated 5.0073 s (1.3M iterations)
	Benchmarking HashedPostState_merge/extend_ref_old/n=50_k=3: Analyzing
	HashedPostState_merge/extend_ref_old/n=50_k=3
	time: [3.6096 µs 3.6130 µs 3.6167 µs]
	change: [-2.3141% -2.1418% -1.9619%] (p = 0.00 < 0.05)

	{
	"baseline": {},
	"feature": {},
	"comparison": {}
	}

	{
	"baseline": {},
	"feature": {},
	"comparison": {}
	}

	{
	"baseline": {},
	"feature": {},
	"comparison": {}
	}

	{
	"baseline": {},
	"feature": {},
	"comparison": {}
	}