l0rinc

Benchmarking meeting notes

Add a macro benchmark suite with longer-running operations, exercising a different set of functionality that is performance-critical.

• One of them could be the speed of an assumeUTXO load-and-dump (this would check block read/write and (de)serialization, but only part of validation).
• Another one could be an actual IBD for 840-880k blocks (small part of IBD, but should produce a representative comparison quickly).
• Another one would be a reindex-chainstate up to e.g. 880k blocks (more realistic, but takes long and doesn't involve reading/writing blocks).
• Lastly, several full IBDs to make sure the previous quick checks are representative (likely only required for big changes). We don't have to run all of these for every PR; it could be tied to labels added to PRs where the author/reviewers think they're relevant (e.g., not for doc typo fixes). This would require integration with CoreCheck.

Other suggested macro benchmarks by the participants:

• Compact block reconstruction

l0rinc

#!/bin/bash

# Set up the working directory and output folder
DATADIR="/mnt/bitcoin-data"
OUTPUT_DIR="$DATADIR/stale_blocks"
LOG_FILE="$OUTPUT_DIR/stale_blocks.csv"

mkdir -p "$OUTPUT_DIR"
while read -r bhash; do
    # Fetch block information (height and hash) and skip if unavailable

l0rinc

This guide helps you compare the performance of two different versions of Bitcoin Core during the initial block download (IBD) process. You will run two builds and share the results.

Background and Purpose

I am working on improving the performance of the first phase of Bitcoin Core, specifically the process of downloading and validating the blockchain on a fresh machine. I have proposed several modifications that, based on my own measurements, result in measurable speed improvements. However, to ensure broader acceptance, I need help reproducing these results by having others run similar tests.

This is a technical process (though not overly complex, in my opinion), requiring the source code to be compiled. The goal is to validate these improvements on multiple setups and environments, which will help accelerate their adoption.

Requirements:

l0rinc

Ez az útmutató segít abban, hogy összehasonlítsd a Bitcoin Core két különböző verziójának teljesítményét az első blokklánc-letöltési (IBD) folyamat során. Két buildet fogsz futtatni, és az eredményeket megoszthatod.

Kell hozzá:

• GitHub fiók (az eredmények publikálásához).
• Linux alapú rendszer (például Ubuntu/Debian).
• Legalább 1 TB szabad hely a blokklánc adataihoz (ideiglenes, utána törölhető).

Először telepítsd a szükséges csomagokat (Debian/Ubuntu rendszeren). Csak megbízható forrásból telepíts!

l0rinc

#!/usr/bin/env python3

import tarfile
import io
from io import BytesIO

def convert_to_binary():
    # Raw histogram data as (size, count) pairs
    histogram = [
        (1, 47584838861),

l0rinc

Subject: [PATCH] Exclude `incomin&g` from lint-spelling

Before:
 % test/lint/lint-spelling.py
src/qt/forms/optionsdialog.ui:345: incomin ==> incoming

After:
 % test/lint/lint-spelling.py

l0rinc

Subject: [PATCH] TODO
---
Index: src/test/util_tests.cpp
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/util_tests.cpp b/src/test/util_tests.cpp
--- a/src/test/util_tests.cpp	(revision 8b13872aa2960e66e0bf8542fcb5b8dda68c5d8c)
+++ b/src/test/util_tests.cpp	(date 1724237518191)

l0rinc

See related discussions:
* https://github.com/bitcoin/bitcoin/pull/28280#discussion_r1676800505
* https://github.com/bitcoin/bitcoin/pull/28280#discussion_r1676801434
* https://github.com/bitcoin/bitcoin/pull/17487#discussion_r402037193

```diff
---
diff --git a/src/test/coins_tests.cpp b/src/test/coins_tests.cpp
--- a/src/test/coins_tests.cpp	(revision 9f3bf64d29c497fb90406974548a717ba5918b25)
+++ b/src/test/coins_tests.cpp	(date 1723646168861)

l0rinc

// Copyright (c) 2011-2022 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.

#include <test/data/base58_encode_decode.json.h>

#include <base58.h>
#include <test/util/json.h>
#include <test/util/random.h>
#include <test/util/setup_common.h>

l0rinc

// Copyright (c) 2024 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or https://www.opensource.org/licenses/mit-license.php.

#include <bench/bench.h>
#include <chainparams.h>
#include <wallet/coincontrol.h>
#include <consensus/merkle.h>
#include <kernel/chain.h>
#include <node/context.h>