Skip to content

Instantly share code, notes, and snippets.

@jamesob
Created January 24, 2020 20:14
Show Gist options
  • Save jamesob/c0853318e2f5b9672b0c1b95332cf630 to your computer and use it in GitHub Desktop.
Save jamesob/c0853318e2f5b9672b0c1b95332cf630 to your computer and use it in GitHub Desktop.
diff --git a/_posts/2020-01-29-#17487.md b/_posts/2020-01-29-#17487.md
index b59763d..17fafb8 100644
--- a/_posts/2020-01-29-#17487.md
+++ b/_posts/2020-01-29-#17487.md
@@ -13,16 +13,13 @@ status: upcoming
The UTXO cache is a critical datastructure in Bitcoin for both correctness and
performance. It's responsible for maintaining a view of the spendable coins
-based upon the transactions in blocks. The UTXO cache is just a denormalization
-(i.e. a copy of) of the information found in blocks, but it is often consulted
-exclusively for various things for performance reasons. For example while
-validating the spends in an incoming block, we consult the UTXO cache (instead
-of digging the coins out of the blockchain) because we want this process to be
-as fast as possible.
+based upon the transactions in blocks. It is often a major bottleneck during
+block validation, and incorrect behavior is almost certain to lead to consensus
+failure.
Because the UTXO set is accessed so frequently, we would like its contents
available as quickly as possible. This amounts to having as much of the set
-in-memory as possible. The issue is that (at the moment) the in-memory
+in memory as possible. The issue is that (at the moment) the in-memory
representation of the UTXO set is more than 8GB, and obviously not all hosts
running Bitcoin have that much memory.
@@ -35,15 +32,15 @@ logic](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L2198-L
At that point, we completely empty the UTXO cache by writing it to disk by
calling
-[`CCoinsViewCache::Flush()`](https://github.com/bitcoin/bitcoin/blob/master/src/coins.cpp#L205-L210).
-In master, this is the only way of reconciling the state of the in-memory cache
+[`CCoinsViewCache::Flush()`](https://github.com/bitcoin/bitcoin/blob/28fbe68fdcac2a06f359b1e48555a3d23015c2b7/src/coins.cpp#L205-L210).
+In master, this is the only way of reconciling the state of the cache
with the leveldb store on disk, even though sometimes we Flush() not because we
have exceeded our dbcache, but to ensure durability. For example, we
[periodically
-flush](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L2275-L2276)
+flush](https://github.com/bitcoin/bitcoin/blob/28fbe68fdcac2a06f359b1e48555a3d23015c2b7/src/validation.cpp#L2275-L2276)
the coins cache to avoid having to replay blocks if we shutdown improperly.
-Once we flush the in-memory cache, we are forced to read from and write to disk
+Once we flush the cache, we are forced to read from and write to disk
for all UTXO operations, which can be notably slower depending on the
underlying disk. For this reason, separating the emptying of the cache
from the writing to disk might allow us to ensure durability without losing the
@@ -67,13 +64,6 @@ shown](https://github.com/bitcoin/bitcoin/pull/17487#issuecomment-561741590)
that some platforms benefit significantly from maintaining the contents of the
cache after writing them to disk.
-### Consensus bug
-
-One interesting dimension of this PR is that in a previous version of it, I had
-actually proposed a consensus bug. I wish I could pass this off as a test of my
-fellow reviewers, but unfortunately it was just a plain mistake. You should use
-the review of this PR to explore how that bug happened.
-
## Questions
- Did you review the PR? [Concept ACK, approach ACK, ACK \<commit\>, or
@@ -95,7 +85,7 @@ the review of this PR to explore how that bug happened.
- Why do we go to the trouble of maintaining these flags?
- What happens when `CCoinsViewCache::Flush()` is called? How do the coins
- in-memory make their way to the disk?
+ in memory make their way to the disk?
- When an unspent coin is flushed to disk and then spent in the next block, we
will have done (at least) two writes and a read for that coin. If a coin is
@@ -107,6 +97,9 @@ the review of this PR to explore how that bug happened.
- Are you able to describe how
[`ccoins_cache_simulation_test`](https://github.com/bitcoin/bitcoin/blob/b12c4b092de159a38342fb7069cd1903bf5ce680/src/test/coins_tests.cpp#L102-L111) works?
+ - This test has a lot randomness. How do we ensure full coverage?
+ - How does this test compare to [the one I've
+ added](https://github.com/bitcoin/bitcoin/pull/17487/commits/eebaca7620bbd0af0ec385c6c7d47b2b4b524d55)?
- Do you understand the
[benchmarks](https://github.com/bitcoin/bitcoin/pull/17487#issuecomment-557226923)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment