Felföldi Zsolt zsfelfoldi

Log index search implementation overview

The purpose of the log index search mechanism is to return a set of logs that might potentially match a given pattern. The main search function GetPotentialMatches receives the following parameters:

block range
a set of addresses one of which has to match the log address (an empty set is interpreted as a wild card)
a set of topics for each of at most 4 topic positions, one of which has to match the topic at the given position (an empty set is interpreted as a wild card)

GetPotentialMatches constructs a pattern matcher based on the given addresses and topics and translates the block range into log value index range. Then it runs the pattern matcher on the filter maps covering the searched range, running multiple workers in parallel, each processing one epoch at a time. After evaluating the matcher it looks up the actual logs at the resulting potentially matching log value indices. Parallel worker results are collected and returned in chronological order.

FilterMaps data structure explanation

FilterMaps is a search structure that maps the log events of an Ethereum blockchain in a way that allows the implementation of pattern matchers with reasonable amount of data access. It is suitable both for the acceleration of local lookups and for generating reasonably sized trustless proofs for the complete set of results for a certain match pattern and block range. The latter assumes that the structure is tree hashed and the receiver of the proof knows the root hash of this tree. Note that the tree hashing scheme and the proof format are outside the scope of this document.

Linear log value index space

Log values are defined as the SHA2 hashes of log addresses and topics. FilterMaps maps all log values onto a linear index space as shown below:

Block number	Transaction index	Log index	Log event	Log value indices

Log indexer implementation overview

The purpose of the log indexer is to maintain the log index data structure and keep it consistent with the canonical chain. This structure has four components:

information about the currently indexed range
actual filter map data for a consecutive series of maps
pointers from each indexed block number to the first log value index it occupies in the linear index space
pointers from each rendered map to the number of the block that occupies the last log value index of the map

Last block of map pointers and the corresponding reverse direction block to log value index pointers are also stored for the last map of each epoch before the indexed range. During initialization these pointers are added based on a set of hardcoded checkpoints for the given chain (which is a workaround solution until EIP-7745 is passed and these pointers become provable from the latest head). Later if an indexed section gets unindexed then these pointers are retained. Indexing can be started

eip: 7745 title: Light client and DHT friendly log index structure description: An efficient, light client and DHT friendly replacement for block header bloom filters author: Zsolt Felföldi (@zsfelfoldi) discussions-to: https://ethereum-magicians.org/t/eip-7745-two-dimensional-log-filter-data-structure/20580 status: Draft type: Standards Track category: Core created: 2024-07-17

Mapping ethereum chain/state access interfaces to trusted and trustless API calls

ethclient (trusted)

Package ethclient implements interface calls by directly calling the respective RPC API call. It requrired a trusted API endpoint because it does not verify the received results.

BlockByHash(ctx context.Context, blockHash common.Hash) (*types.Block, error)
    eth_getBlockByHash(blockHash, fullTxs=true)
BlockByNumber(ctx context.Context, blockNumber *big.Int) (*types.Block, error)

Beacon light sync top level overview

This document gives a top level view of the blsync PR in order to help the review/merge process. It describes the most important mechanisms and data structures and how they are used in this PR (sometimes also how they are going to be used later).

The current PR implements an MVP feature called blsync that can light sync the beacon chain from a beacon node that supports the light_client REST API namespace (Lodestar or Nimbus) and drive an EL node through the engine API. Its components are sometimes more general purpose though as they are also intended to be part of the new full featured PoS capable Geth light mode. Note that a significant part of this PR (more specifically light.LightChain, merkle.MultiProof, sync.HeaderSync, sync.StateSync) are only used here in order to get the finalized block hash out of the beacon state. It is a possible option to strip down the PR even further by removing this featu

eip	title	description	author	status	type	category	created
<to be assigned>	Beacon State API endpoint	A proposal for a general purpose beacon state access endpoint in the `light_client` API	Zsolt Felföldi (@zsfelfoldi) <[email protected]>	Draft	Standards Track	Interface	2022-11-04

Beacon node light client API proposals

Zsolt Felfoldi [email protected]

The "complete beacon light client service dataset"

is a lot smaller than a full CL node database but can do more
can prove

Notes on implementing uTP over the Geth DiscV5 implementation

The uTP-over-DiscV5 document specifies that all uTP traffic is encapsulated into TALKREQ packets while TALKRESP is not used. The problem is that the current DiscV5 implementation kind of insists on doing TALKREQ / TALKRESP packet exchanges and expects a response to every request. See the existing interface here:

type TalkRequestHandler func(enode.ID, *net.UDPAddr, []byte) []byte
func (t *UDPv5) RegisterTalkHandler(protocol string, handler TalkRequestHandler) {
func (t *UDPv5) TalkRequest(n *enode.Node, protocol string, request []byte) ([]byte, error) {

	Instructions for testing LES/5 (proof-of-stake light client) on the Kiln testnet

	- build the geth branch:
	git clone https://github.com/zsfelfoldi/go-ethereum.git -b les5
	- build the latest master version of lodestar:
	git clone https://github.com/ChainSafe/lodestar.git
	yarn && yarn build
	- build instructions:
	https://github.com/ChainSafe/lodestar/
	- pull the merge testnet configs repo: