dangell7

  title: NuDB — bloom-filter negative lookups + mmap key reads
  description: Perf work on the vendored NuDB store — lock-free bloom filter built from key-file buckets on open() to short-circuit fetch/insert misses, hash-first pool find, and POSIX mmap read path for bucket fetches.
  author: Denis Angell (@dangell7)
  status: Active
  created: 2026-08-04
  updated: 2026-08-04
  repos:
    - xrplf/NuDB @ master

dangell7

plan-14 — SHAMap tree-cache RAM scaling (the cache OOM design fix)

Companion code branch: xrpld → dangell7/tree-cache-ram-scaling (off develop). Surfaced by the 25M-account capacity soak (see soak-issues-to-fix.md #5/#6).

Problem

[node_size] (tiny…huge) picks a fixed entry-count target for the SHAMap tree-node cache (Config.cpp TreeCacheSize = 262 144 … 8 388 608 entries). That number has no relationship to the machine's RAM and no memory bound:

dangell7

Plan 3 — Batch Signature Verification: Design (corrected after code + crypto research)

Workspace: /Users/infinityworks/projects/xrplf/xrpld-batch-sign, branch dangell7/batch-sig-verify, off develop @ 2f3558c61.

Status: Design / decision point. No code written yet — research invalidated two premises of the original todo.md Plan 3 and the scope must be re-decided on the corrected facts.

---

The two findings that change everything

dangell7

Plan 8 — Top-of-book cache (best-price pointer)

Goal: for every active orderbook (and AMM-augmented book), maintain a sticky pointer to the keylet of the best-quality directory page. The path engine reads the top of book by dereferencing one pointer instead of issuing a SHAMap succ() from the book's base keylet.

Scope: a single, narrow optimization at the orderbook semantic layer. Composes with Plan 6 (flat state lookup) — they sit at different layers and stack. No protocol change, no amendment, no consensus impact.

---

1. The problem

dangell7

Plan 6 — Flat keylet-indexed state lookup (read-side)

Goal: materialize XRPL state as a flat keylet→SLE map. Apply does dual-write (SHAMap stays authoritative, flat map indexes reads). Reads go to the flat map only — no fallback path. This is the pure NoSQL 2-writes-for-1-read pattern. Direct TPS lift is moderate-to-large; indirect value is the larger story — this unlocks downstream consumers (indexers, light clients, hooks, AMM bots, sidecar services) that currently can't query XRPL state at scale.

Why this is Plan 6: the SHAMap is a NoSQL substrate (keylet→SLE Merkle trie) currently being used in a SQL access pattern (descend the index on every read). Plan 6 finishes what the substrate was designed for: materialize the read view. Plan 5 (incremental hashing) becomes mostly obsolete once Plan 7 (deferred SHAMap, the structural follow-on to this plan) lands.

---

1. The problem

dangell7

Plan 1 — Account-sharded parallel transaction application

Goal: remove the master-mutex serial-apply ceiling. Target: 5–10× sustained TPS lift (from ~159 TPS payment baseline to 800–1500+ on commodity multi-core hardware, scaling further with cores).

Headline approach: Solana-style static conflict-graph scheduling, not Block-STM. XRPL transactions already carry a small, statically-determinable access set per tx type; we extract it, build a conflict graph, dispatch independent groups to a worker pool, and merge per-account writes at the end of the round. No speculation, no abort cost.

---

VERIFICATION STATUS

dangell7

6-Month Roadmap — Performance Network & Staging/Canary Network

Companion to: takeover-network-and-capacity-testing.md Author: Denis Angell, Director of Technology Window: Jul – Dec 2026 Note: This is the time-bound execution plan. The proposal itself is deliberately timeline-free; scheduling lives here.

---

Workstreams

dangell7

Validation relay loss on mainnet — one-ledger cohort drops at the s1/s2 vantage

Status: investigated (2026-08-02), monitor cross-check shipped (xrplf-release-notifier, uncommitted), root-cause candidates ranked from code; perfnet repro pending Origin: observatory LOW_QUORUM alert, ledger 106013031 ("25/35 UNL validations, need 28"). Investigation proved the network was healthy and the alert was a vantage artifact — which surfaced the real finding: Ripple's public s1/s2 clusters intermittently lose a cohort of validators' validations for exactly one ledger. Relevant to the capacity campaign because relay redundancy, not consensus, was the layer that degraded under load.

dangell7

Design: generational (N-segment) NodeStore for online_delete

Concrete implementation design for the fix in online-delete-generational-gc.md. Target: Transia-RnD/rippled, branch dangell7/online-delete-generational-gc (own work off develop).

Problem restated (one paragraph)

Today DatabaseRotating keeps two append-only backends (writable + archive). Every deleteInterval_ ledgers, SHAMapStoreImp::run() walks the entire current state (visitNodes(copyNode)) and re-stores every live node into a fresh writable backend, then drops the old archive. Because the writable is recreated each rotation, the whole live set is re-written

dangell7

Weakness #0 — consensus liveness under job-queue saturation

Status: investigated (2026-08-01), fix PR in progress Origin: the "XRPL falls over at 20–25M accounts" stress campaign. On a real 5-validator perf net under sustained live account creation, consensus lost liveness — LoadMonitor:WRN Job: AcceptLedger run: 19s, validators fell behind, dropped quorum, and the network deep-forked with no self-heal (only a genesis reset recovered it). Masked operationally by resizing validators 8→16 vCPU; the protocol weaknesses remain.

Three read-only investigations (code refs against Transia develop). Weakness #0 is not one bug