Bolt Explorer Schema

explorer-schema.md

Context

We need to redesign the data indexer with the following in mind:

• Correct indexing for measuring network health
• Correct indexing for attributing faults (liveness, safety, timeliness)
• Correct indexing for finding bugs (local & remote error responses, tracking confirmations)
• Enough metadata for all of the above

With this in mind, this is what the Clickhouse tables will look like:

raw_transaction_requests

• Description: tracks all of the individual raw transactions that were seen in inclusion requests
• Schema:

tx_hash	request_id (FK)	transaction
String	UUID	String

inclusion_requests

• Description: inclusion request events. The transactions inside of the requests can be found in the raw_transaction_requests through the matching request_id.
• Schema:

request_id	timestamp	target_slot	signer	operator	operator_endpoint	validator_pubkey
UUID	UInt128	UInt64	Nullable(String)	String	Nullable(String)	String

• Notes
  • Every request that passes initial checks MUST be logged
  • If an error occurs, the error will be logged in the inclusion_errors table
  • If a commitment was made, the corresponding commitment will be logged in the inclusion_commitments table
  • If operator_endpoint is unknown (firewall delegation), set to NULL
  • signer is the signer of the inclusion request, not the transaction sender

inclusion_commitments

• Description: inclusion commitments made by proposers. Commitments can be matched with requests through request_id.
• Schema:

request_id	timestamp	slot	operator	signature
UUID	UInt128	UInt64	String	String

inclusion_confirmations

• Description: tracks inclusion confirmations. If there's a confirmation, there can be no fault for the same request.
• Schema:

request_id	slot
UUID	UInt64

• Notes:
  • A row MUST only be added when all of the transactions in the inclusion request were included. Partial inclusion is considered a fault.

inclusion_faults

• Description: tracks inclusion faults. If there's a fault, there can be no confirmation for the same request.
• Schema:

request_id	timestamp	slot	type
UUID	UInt128	UInt64	Enum('liveness', 'safety', 'timeliness')

• Notes
  • type = liveness, safety or timeliness fault
  • timeliness fault = inclusion request was included, but commitment was never made (once we consider deadlines, also include these faults)

user_equivocations

• Description: tracks user equivocations, where a user invalides their own commitment by getting a transaction included with the same nonce before the target slot.
• Schema:

committed_tx_hash	invalidated_id	tx_hash	transaction
String	UUID	String	String

inclusion_errors

• Description: tracks inclusion errors. These can come either from local simulation at the RPC or from the sidecar. If there's an error for this request, there can never be a commitment too.
• Schema:

request_id	timestamp	error	error_source
UUID	UInt128	String	Enum('local', 'remote')