gavinandresen

Optimizing New Block Messages

Motivation

The network message used to announce the discovery of a new block is inefficient, and that inefficiency translates indirectly into higher transaction fees for users. The dominating cost for a miner to include an extra transaction in their block is the incremental "orphan cost", which is dependent on how quickly the block is propagated across the peer-to-peer network.

gavinandresen

Network/miner fee policy migration plan

Introduction

Changing the fees required for transactions to be relayed across the network and/or accepted by miners into their blocks requires care; policy mis-matches can result in users' transactions never confirming, either because they never reach a miner (they are not relayed) or because they are rejected by miners.

I have been working on simplifying the transaction fee rules for the 0.9 release; this document is intended to capture all of the details needed to make that transition as smoothly as possible.

The general idea is to make sure the transaction-generating code (the wallet code) is conservative, and generates transactions compatible with the previous version's rules. The relay and mining code is changed more aggressively, to anticipate future releases.

gavinandresen

Design sketch: "reject" p2p message

Motivation

Mike Hearn has been lobbying for an "error" message in the Bitcoin p2p protocol for years (at least since the "ban peers if they send us garbage" denial-of-service mitigation code was pull-requested). This came up again with my proposed "smartfee" changes, which would drop low-priority or low-fee transactions.

In short, giving peers feedback about why their blocks or transactions are dropped or

gavinandresen

The reference implementation of Bitcoin (bitcoind/Bitcoin-Qt) has ad-hoc code for dealing with transaction priorities and fees. In particular, there are several 'magic' constants that are chosen by the core developers, setting policies for minimum transaction fees, minimum transaction priorities and minimum output sizes.

This document describes the algorithms used by my 'smartfee' branch to replace most of those arbitrary constants with values that will rise and fall based on transaction volume and miners' willingness to include transactions in their blocks.

Fee/Priority estimation

The reference implementation needs to know two things to provide a good user experience:

1. If a transaction has priority X, is it likely to be included in the next N blocks?
2. If a transaction has a fee of Y, is it likely to be included in the next N blocks?

gavinandresen

Thumbnail sketch: Two-factor (Yubikey or Google Authenticator) protected wallets

Hardware:

Computer. Shared-secret authenticator (Yubikey/Google Authenticator/etc). Server (possibly shared with millions of other users).

Motivating use case setup:

User creates a split (2-of-2 multisig) wallet on the computer and server. Keys must be securely backed up to protect against loss. GUI to be determined, but there will be some type of "Use Authenticator" checkbox specified at setup.

gavinandresen

Everybody count to ten and take a deep breath in.... hold... out....

Ok.

If you have ever been involved with an organization that has more than three members, you should know that there will always be conflicts between members. Always.

Since the Foundation has corporate members, and since corporations use the legal system to help resolve their disputes (much better than using guns, by the way) it isn't surprising that one member of the Foundation is suing another.

The fact that they both happen to be on the Foundation Board is, in my view, not terribly relevant.

gavinandresen

Transaction fee handling is quickly becoming an issue, due to several factors:

1. Bitcoin prices rising.
2. Transaction volume increasing, and competition for space in blocks beginning.
3. People with too much cash on their hands deciding to bloat the blockchain with data.

I propose the following band-aid, quick fix for the 0.8.2 release:

Modify the definition of CTransaction::IsStandard() as follows:

gavinandresen

What we know:

• the fork was caused by a block that needed over 10,000 BDB locks to be confirmed
• A conservative estimate of the number of locks needed to confirm a block is: (#unique txids referenced)*2.03 + 100, where unique txids referenced is number of transactions created in the block plus the number of previous transactions referenced by those transaction's inputs
• A majority of hashing power was running version 0.8, and most miners are motivated to upgrade
• BDB lock settings can be configured at runtime, by putting a file called DB_CONFIG in the data directory. DB_CONFIG overrides compiled-in settings.
• Chain re-orgs can also trip the "too many locks" problem, depending on how many unique txids are involved. Since transactions tend to overlap between chains during a re-org, this is not as large a problem as one might think.
• We would have a problem even if the network was all 0.7 nodes, because 0.7 could theoretically "self-fork" : the exact number of locks taken depends on the blkindex.d

gavinandresen

Motivation

This gist tries to come up with a back-of-the-envelope estimate for how much it costs a miner to create larger, rather than smaller, blocks. An economically rational miner will only include transactions with enough fees to cover the cost to the miner of processing the transaction.

I hope this analysis helps solo miners and mining pool operators to reason about their transaction fee policies, and helps them estimate their 'charity cost' if they decide to include fee-free transactions in the blocks they create to promote more use of Bitcoin.

Uncertain numbers in this document are rounded to one digit of precision; this is meant to be an order-of-magnitude estimate; when I say that 100K of free transactions costs about $1, you should assume that the real number might be anywhere between 10 cents and 10 dollars.

Executive summary

gavinandresen

Core development priorities

Advice for contributors

The over-arching core development priority is to avoid really big, disastrous mistakes.

There are billions of dollars at stake, and the core developers are acutely aware of how much responsibility that puts on them to try to ensure the core system is as reliable and stable as