Skip to content

Instantly share code, notes, and snippets.

View tbg's full-sized avatar

Tobias Grieger tbg

View GitHub Profile
$ benchstat benchdiff/50c064e/artifacts/out.2022-12-21T13_05_49Z benchdiff/52671d3/artifacts/out.2022-12-21T13_05_49Z
name old time/op new time/op delta
ReplicaProposal/bytes=256_B,withFollower=false-24 108µs ± 4% 109µs ± 5% ~ (p=0.286 n=30+30)
ReplicaProposal/bytes=256_B,withFollower=true-24 139µs ± 4% 140µs ± 6% ~ (p=0.273 n=30+30)
ReplicaProposal/bytes=512_B,withFollower=false-24 112µs ± 5% 113µs ± 5% ~ (p=0.217 n=30+27)
ReplicaProposal/bytes=512_B,withFollower=true-24 146µs ± 4% 145µs ± 3% ~ (p=0.255 n=30+27)
ReplicaProposal/bytes=1.0_KiB,withFollower=false-24 123µs ± 7% 124µs ± 6% ~ (p=0.227 n=29+29)
ReplicaProposal/bytes=1.0_KiB,withFollower=true-24 156µs ± 3% 157µs ± 4% ~ (p=0.362 n=29+30)
ReplicaProposal/bytes=256_KiB,withFollower=false-24 2.48ms ± 7% 2.56ms ± 7% +3.39% (p=0.002 n=30+29)
ReplicaProposal/bytes=256_KiB,withFollower=true
@tbg
tbg / writebatch-memcpy.md
Last active December 14, 2022 20:22
WriteBatch memcpy

WriteBatch gets created internally in a storage.Batch // instance 1

https://github.com/cockroachdb/cockroach/blob/f089b43803b1d59d4752238e5b4bb37cfe0fbf8d/pkg/kv/kvserver/replica_proposal.go#L713-L717

The impl actually copies, womp womp (the batch reuses the original memory but still): // instance 2

https://github.com/cockroachdb/cockroach/blob/5bafe901cd6fd4ceee00ee922e7f13ed95c4134b/pkg/storage/pebble_batch.go#L567-L576

Aside: If we kept the batch around with the proposal, we could conceivably use its backing memory. We could also apply the batch directly, possibly saving overhead on the leaseholder.

Advent of protobuf hackery
You have this proto message.
message RequestHeader {
// some fields
}
You want to add a field to it that is "there" only during crdb_test.
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
AVG_DURATION_SEC TEST_NAME
88.719246032000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestReplicateQueueSwapVotersWithNonVoters
67.954845606000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestReplicateQueueRebalanceMultiStore
59.736863905000 github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptstorage: TestStorage
52.481829026000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestMergeQueueSeesNonVoters
49.185198413000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestReplicateQueueDecommissioningNonVoters
48.835447316000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestResetQuorum
42.682879684000 github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptstorage: TestStorage/Protect_-_unlimited_bytes
41.607149644000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestReplicateQueueRebalanceMultiStore/simple
41.130417495000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestStoreRangeMergeTimestampCache
AVG_DURATION_SEC TEST_NAME
88.719246032000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestReplicateQueueSwapVotersWithNonVoters
67.954845606000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestReplicateQueueRebalanceMultiStore
59.736863905000 github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptstorage: TestStorage
52.481829026000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestMergeQueueSeesNonVoters
49.185198413000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestReplicateQueueDecommissioningNonVoters
48.835447316000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestResetQuorum
42.682879684000 github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptstorage: TestStorage/Protect_-_unlimited_bytes
41.607149644000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestReplicateQueueRebalanceMultiStore/simple
41.130417495000 github.com/cockroachdb/cockroach/pkg/kv/kvserver: TestStoreRangeMergeTimestampCache
// When the server shuts down there may be proposals inflight. They may have
// a trace span attached to them, so close those out properly. This is mostly
// to appease the span leak check in TestCluster.
//
// TODO(during review): this doesn't actually fix it, when you make the skip
// in TestReplicaRemovalClosesProposalQuota unconditional we still leak the
// same spans within ~a minute.
stopper.AddCloser(stop.CloserFn(func() {
s.VisitReplicas(func(repl *Replica) (wantMore bool) {
// NB: we may not need raftMu but doesn't hurt either.
<pkg>: <short description - lowercase, no final period>
<what was there before: Previously, ...>
<why it needed to change: This was inadequate because ...>
<what you did about it: To address this, this patch ...>
# Note: to disable this commit template, run: git config --global --add cockroachdb.disable-commit-template true
Release justification:
#Release note: None
--- FAIL: TestMVCCStatsRandomized/userspace (0.04s)
--- FAIL: TestMVCCStatsRandomized/userspace/inline=false (0.04s)
mvcc_stats_test.go:1845: seed: 5946155527612219895
mvcc_stats_test.go:1649: 1.524851328,3 Del txn=<none>
mvcc_stats_test.go:1649: 4.943406208,8 DelRange txn=<none>
mvccRangeDel={/Meta1/-foo\x00}, returnKeys=false, max=0
stats delta: {"GCBytesAge":-100,"RangeKeyCount":1,"RangeKeyBytes":18,"RangeValCount":1,"RangeValBytes":7}
absolute: {"GCBytesAge":-123,"KeyBytes":16,"KeyCount":1,"ValBytes":7,"ValCount":1,"RangeKeyCount":1,"RangeKeyBytes":18,"RangeValCount":1,"RangeValBytes":7}
mvcc_stats_test.go:1649: 5.265516864,8 GC txn=<none>
2.191204235,0
#!/usr/bin/env bash
set -euo pipefail
echo $#
if [ $# -lt 5 ]; then
echo "Usage: $0 bench.txt someBenchVar A B C"
echo " ^ "
echo " | "
echo " old "
exit 1