Skip to content

Instantly share code, notes, and snippets.

@tbg
Created July 9, 2015 22:40
Show Gist options
  • Save tbg/a9908067dae02160bd76 to your computer and use it in GitHub Desktop.
Save tbg/a9908067dae02160bd76 to your computer and use it in GitHub Desktop.
cockroach put acceptance test: trace of a range split txn which gets retried needlessly (because rpc/send abandons the first request)
td06877be roach0.local:8080 17:37:16.838148 899.612522ms node server/node.go:467
td06877be roach0.local:8080 17:37:16.838168 899.587718ms ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:16.838177 33.422µs ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:16.838218 899.533937ms ··raft storage/range.go:725
td06877be roach0.local:8080 17:37:16.844148 893.210049ms ···applying EndTransaction storage/range.go:804
td06877be roach0.local:8080 17:37:17.339896 400.998514ms node server/node.go:467
td06877be roach0.local:8080 17:37:17.339919 400.947752ms ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:17.339930 398.41169ms ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:17.738349 2.513129ms ··raft storage/range.go:725
td06877be roach0.local:8080 17:37:17.740726 118.904µs ···applying EndTransaction storage/range.go:804
td06877be roach0.local:8080 17:37:17.740889 0 ·error: *proto.RangeKeyMismatchError server/node.go:473
td06877be roach0.local:8080 17:37:18.350626 2.409219541s node server/node.go:467
td06877be roach0.local:8080 17:37:18.350648 2.409184988s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:18.350658 2.408820072s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:20.759842 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:19.772546 1.231515936s node server/node.go:467
td06877be roach0.local:8080 17:37:19.772567 1.231478879s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:19.772576 1.231091879s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:21.004058 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:20.275520 881.810761ms node server/node.go:467
td06877be roach0.local:8080 17:37:20.275539 881.770251ms ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:20.275545 881.484215ms ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:21.157325 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:22.164463 3.934830369s node server/node.go:467
td06877be roach0.local:8080 17:37:22.164478 3.934720117s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:22.164487 3.928876742s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:26.093375 5.805118ms ··raft storage/range.go:725
td06877be roach0.local:8080 17:37:26.097753 177.282µs ···applying EndTransaction storage/range.go:804
td06877be roach0.local:8080 17:37:26.099287 0 ·error: *proto.TransactionStatusError server/node.go:473
td06877be roach0.local:8080 17:37:22.272428 3.903805072s node server/node.go:467
td06877be roach0.local:8080 17:37:22.272465 3.903761419s ·executing InternalHeartbeatTxn storage/store.go:1261
td06877be roach0.local:8080 17:37:22.272475 3.898960923s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:26.171448 4.760793ms ··raft storage/range.go:725
td06877be roach0.local:8080 17:37:26.174739 138.102µs ···applying InternalHeartbeatTxn storage/range.go:804
td06877be roach0.local:8080 17:37:22.667341 7.666994435s node server/node.go:467
td06877be roach0.local:8080 17:37:22.667355 7.666957358s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:22.667362 7.665425058s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:30.334330 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:22.774812 7.609996979s node server/node.go:467
td06877be roach0.local:8080 17:37:22.774833 7.609914959s ·executing InternalHeartbeatTxn storage/store.go:1261
td06877be roach0.local:8080 17:37:22.774844 7.608005871s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:30.384759 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:23.714963 7.297201513s node server/node.go:467
td06877be roach0.local:8080 17:37:23.714982 7.297148278s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:23.714991 7.296082s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:31.012141 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:23.777771 7.313810727s node server/node.go:467
td06877be roach0.local:8080 17:37:23.777796 7.313762817s ·executing InternalHeartbeatTxn storage/store.go:1261
td06877be roach0.local:8080 17:37:23.777808 7.312247631s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:31.091577 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:16.817632 14.598692109s sending Batch kv/txn_coord_sender.go:351
td06877be roach0.local:8080 17:37:16.817685 30.989µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:16.817723 4.917898ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:16.817759 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:16.822688 949.94µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:16.823648 4.997539ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:16.823689 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:16.828723 5.734µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:16.828735 9.115572ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:16.828801 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:16.837903 8.275µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:16.837917 501.357366ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:16.837956 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.338329 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:17.338438 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.339409 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:17.339447 55.33µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.339509 401.886694ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.339547 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.741470 0 ·reply error: *proto.RangeKeyMismatchError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.741905 1.730493ms ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.743664 1.491113ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.743707 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.745244 0 ·reply error: *proto.RangeKeyMismatchError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.745589 8.263421ms ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.753860 1.635533ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.754111 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.755751 0 ·reply error: *proto.RangeKeyMismatchError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.756345 3.521791ms ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.759874 1.607626ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.759909 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.761541 0 ·reply error: *proto.RangeKeyMismatchError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.761940 4.051009ms ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.766079 1.754523ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.766128 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.767969 0 ·reply error: *proto.RangeKeyMismatchError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.769011 7.088µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.769024 2.311934ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.769061 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.772221 0 ·reply error: *proto.RangeNotFoundError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.772256 32.957316ms ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.805221 2.282807ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.805257 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.808743 0 ·reply error: *proto.RangeKeyMismatchError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.808788 29.950619ms ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.838747 1.728136ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.838783 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.840536 0 ·reply error: *proto.RangeKeyMismatchError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.840845 5.900193ms ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.846752 966.125µs ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.846785 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:17.847850 0 ·reply error: *proto.RangeKeyMismatchError kv/dist_sender.go:660
td06877be roach0.local:8080 17:37:17.848389 6.519µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:17.848400 1.92361812s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:17.848435 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:18.349794 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:18.350081 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:18.850587 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:18.850671 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:19.772107 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:19.772146 7.581µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:19.772159 502.553711ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:19.772191 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:20.272736 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:20.272820 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:20.274891 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:20.274938 8.887µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:20.274952 885.990561ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:20.274987 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:20.775750 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:20.785937 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:21.161338 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:21.161370 7.568µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:21.161381 501.94324ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:21.161410 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:21.662888 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:21.662960 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:21.663513 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:21.663610 9.102µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:21.663625 1.002430075s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:21.663806 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:22.164095 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:22.164192 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:22.664540 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:22.664607 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:22.666116 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:22.666165 23.828µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:22.666196 1.048073317s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:22.666231 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:23.166554 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:23.166629 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:23.670442 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:23.670490 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:23.714306 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:23.714336 5.937µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:23.714347 1.0036429s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:23.714798 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:24.215232 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:24.215296 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:24.716717 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:24.716766 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:24.718083 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:24.718116 6.144µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:24.718127 1.002730026s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:24.718150 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:25.218257 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:25.218354 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:25.719529 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:25.719592 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:25.721022 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:25.721052 6.743µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:25.721064 1.003601392s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:25.721092 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:26.221990 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:26.222051 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:26.723272 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:26.723346 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:26.725073 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:26.725112 7.027µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:26.725124 1.003154325s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:26.725156 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:27.225357 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:27.225436 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:27.726932 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:27.726980 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:27.728315 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:27.728336 5.086µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:27.728345 1.003637801s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:27.728367 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:28.228675 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:28.228778 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:28.729718 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:28.729824 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:28.732242 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:28.732347 7.369µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:28.732368 2.683647979s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:28.732389 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:29.232744 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:29.232816 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:29.733345 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:29.733412 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:31.416185 0 ·coordinator stops kv/txn_coord_sender.go:144
td06877be roach0.local:8080 17:37:31.416256 0 ·reply error: *proto.TransactionStatusError kv/txn_coord_sender.go:354
td06877be roach0.local:8080 17:37:24.718634 6.893901592s node server/node.go:467
td06877be roach0.local:8080 17:37:24.718651 6.893861717s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:24.718659 6.892721177s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:31.612531 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:24.780394 6.866054618s node server/node.go:467
td06877be roach0.local:8080 17:37:24.780417 6.8660099s ·executing InternalHeartbeatTxn storage/store.go:1261
td06877be roach0.local:8080 17:37:24.780429 6.865147042s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:31.646443 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:21.771636 10.156712003s heartbeat kv/txn_coord_sender.go:763
td06877be roach0.local:8080 17:37:21.771651 20.498µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:21.771686 1.002468822s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:21.771723 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:22.272094 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:22.272228 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:22.773103 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:22.773148 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:22.774297 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:22.774336 8.007µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:22.774349 1.002713539s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:22.774457 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:23.274684 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:23.274765 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:23.775298 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:23.775360 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:23.777148 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:23.777198 9.834µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:23.777214 1.002712596s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:23.777257 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:24.277535 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:24.277579 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:24.777904 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:24.777969 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:24.780024 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:24.780062 10.577µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:24.780078 1.002303248s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:24.780120 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:25.281176 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:25.281222 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:25.781424 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:25.781478 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:25.782461 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:25.782487 12.579µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:25.782504 1.006136318s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:25.782535 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:26.286166 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:26.287492 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:26.787736 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:26.787782 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:26.788708 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:26.788735 8.193µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:26.788764 1.001491372s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:26.788801 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:27.289183 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:27.289237 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:27.789491 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:27.789542 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:27.790325 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:27.790352 7.686µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:27.790363 1.001666291s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:27.790393 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:28.290704 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:28.290754 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:28.790921 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:28.790975 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:28.792265 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:28.792303 67.979µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:28.792377 3.135966109s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:28.792413 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:29.292685 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:29.292759 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:29.793297 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:29.793355 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:25.721396 6.281793679s node server/node.go:467
td06877be roach0.local:8080 17:37:25.721408 6.281766497s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:25.721414 6.281031845s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.003186 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:25.782726 6.293443652s node server/node.go:467
td06877be roach0.local:8080 17:37:25.782743 6.293401156s ·executing InternalHeartbeatTxn storage/store.go:1261
td06877be roach0.local:8080 17:37:25.782751 6.292415245s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.076163 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:26.725627 5.652611487s node server/node.go:467
td06877be roach0.local:8080 17:37:26.725647 5.652567782s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:26.725657 5.651828146s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.378232 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:26.789009 5.638433445s node server/node.go:467
td06877be roach0.local:8080 17:37:26.789026 5.638394599s ·executing InternalHeartbeatTxn storage/store.go:1261
td06877be roach0.local:8080 17:37:26.789036 5.637902634s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.427437 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:27.728496 4.906003508s node server/node.go:467
td06877be roach0.local:8080 17:37:27.728510 4.905976546s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:27.728516 4.905770882s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.634496 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:27.790623 4.869968538s node server/node.go:467
td06877be roach0.local:8080 17:37:27.790639 4.869938733s ·executing InternalHeartbeatTxn storage/store.go:1261
td06877be roach0.local:8080 17:37:27.790648 4.869756724s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.660587 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:28.733105 4.066901502s node server/node.go:467
td06877be roach0.local:8080 17:37:28.733121 4.06686691s ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:28.733130 4.066631584s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.800001 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:28.792758 4.017206239s node server/node.go:467
td06877be roach0.local:8080 17:37:28.792778 4.01716468s ·executing InternalHeartbeatTxn storage/store.go:1261
td06877be roach0.local:8080 17:37:28.792789 4.016845897s ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.809959 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:31.920353 904.269776ms node server/node.go:467
td06877be roach0.local:8080 17:37:31.920376 904.227089ms ·executing EndTransaction storage/store.go:1261
td06877be roach0.local:8080 17:37:31.920379 904.174296ms ··command queue storage/range.go:683
td06877be roach0.local:8080 17:37:32.824618 0 ·error: *proto.NotLeaderError server/node.go:473
td06877be roach0.local:8080 17:37:31.418476 1.87472299s sending EndTransaction kv/txn_coord_sender.go:351
td06877be roach0.local:8080 17:37:31.418489 6.825µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:31.418501 1.406566339s ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:31.418534 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:31.919721 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:31.919776 0 ··sending to roach0.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:32.420570 0 ··timeout, trying next peer rpc/send.go:219
td06877be roach0.local:8080 17:37:32.420617 0 ··sending to roach1.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:32.825180 0 ·reply error: *proto.NotLeaderError kv/dist_sender.go:670
td06877be roach0.local:8080 17:37:32.825253 8.47µs ·meta descriptor lookup kv/dist_sender.go:617
td06877be roach0.local:8080 17:37:32.825267 467.677266ms ·sending RPC kv/dist_sender.go:528
td06877be roach0.local:8080 17:37:32.825302 0 ··sending to roach2.local:8080 rpc/send.go:179
td06877be roach0.local:8080 17:37:33.293191 0 ·reply error: *proto.TransactionStatusError kv/txn_coord_sender.go:354
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment