Created
June 19, 2019 12:33
-
-
Save tbg/bb50553e5ac3bf7d0b5e125e3042871d to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| === RUN Example_node | |
| W190617 21:07:48.618723 1 server/status/runtime.go:310 [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006" | |
| W190617 21:07:49.275779 94259 storage/store.go:1532 [n1,s1,r6/1:/Table/{SystemCon…-11}] could not gossip system config: [NotLeaseHolderError] r6: replica (n1,s1):1 not lease holder; lease holder unknown | |
| W190617 21:07:49.335282 94259 storage/store.go:1532 [n1,s1,r6/1:/Table/{SystemCon…-11}] could not gossip system config: [NotLeaseHolderError] r6: replica (n1,s1):1 not lease holder; lease holder unknown | |
| W190617 21:07:49.431817 94259 storage/store.go:1532 [n1,s1,r6/1:/Table/{SystemCon…-11}] could not gossip system config: [NotLeaseHolderError] r6: replica (n1,s1):1 not lease holder; lease holder unknown | |
| W190617 21:07:51.624489 94218 storage/store.go:3740 [n1,s1,r12/1:/Table/1{6-7}] handle raft ready: 0.6s [processed=1] | |
| W190617 21:08:21.631685 94074 storage/closedts/provider/provider.go:152 [ct-closer] unable to move closed timestamp forward: not live | |
| W190617 21:08:47.098354 94447 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| E190617 21:08:53.646008 94185 storage/consistency_queue.go:135 [n1,consistencyChecker,s1,r12/1:/Table/1{6-7}] computing own checksum: operation "collect checksum" timed out after 15s | |
| W190617 21:08:53.689975 94190 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:09:14.658551 94385 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:09:18.801632 94454 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:09:28.964022 94265 storage/store_rebalancer.go:221 [n1,s1,store-rebalancer] StorePool missing descriptor for local store | |
| W190617 21:09:28.606526 94399 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:09:55.637074 94126 storage/replica_write.go:206 [n1,s1,r19/1:/Table/2{3-4}] have been waiting 62.18s for proposing command RequestLease [/Table/23,/Min). | |
| This range is likely unavailable. | |
| Please submit this message at | |
| https://github.com/cockroachdb/cockroach/issues/new/choose | |
| along with | |
| https://yourhost:8080/#/reports/range/19 | |
| and the following Raft status: {"id":"1","term":6,"vote":"1","commit":13,"lead":"1","raftState":"StateLeader","applied":10,"progress":{"1":{"match":13,"next":14,"state":"ProgressStateReplicate"}},"leadtransferee":"0"} | |
| W190617 21:10:05.295099 94476 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:10:05.630350 94258 storage/replica_range_lease.go:1032 [n1,s1,r1/1:/{Min-System/NodeL…}] have been waiting 1m0s attempting to acquire lease | |
| W190617 21:10:07.781049 94074 storage/closedts/provider/provider.go:152 [ct-closer] unable to move closed timestamp forward: not live | |
| E190617 21:10:08.177496 94185 storage/queue.go:1035 [n1,consistencyChecker,s1,r12/1:/Table/1{6-7}] operation "consistencyChecker queue process replica 12" timed out after 1m0s | |
| W190617 21:10:21.737439 94265 storage/store_rebalancer.go:221 [n1,s1,store-rebalancer] StorePool missing descriptor for local store | |
| W190617 21:10:21.634154 94485 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:10:36.475281 94295 storage/node_liveness.go:561 [n1,liveness-hb] slow heartbeat took 161.0s | |
| W190617 21:10:53.770062 94295 storage/node_liveness.go:486 [n1,liveness-hb] failed node liveness heartbeat: operation "node liveness heartbeat" timed out after 4.5s | |
| W190617 21:10:53.569695 94074 storage/closedts/provider/provider.go:152 [ct-closer] unable to move closed timestamp forward: not live | |
| W190617 21:10:56.693457 94260 storage/replica_range_lease.go:1032 [n1,s1,r2/1:/System/NodeLiveness{-Max}] have been waiting 1m0s attempting to acquire lease | |
| W190617 21:10:56.693503 94259 storage/replica_range_lease.go:1032 [n1,s1,r6/1:/Table/{SystemCon…-11}] have been waiting 1m0s attempting to acquire lease | |
| W190617 21:11:03.178362 94265 storage/store_rebalancer.go:221 [n1,s1,store-rebalancer] StorePool missing descriptor for local store | |
| W190617 21:11:03.655019 94295 storage/node_liveness.go:561 [n1,liveness-hb] slow heartbeat took 1.2s | |
| W190617 21:11:15.966503 94295 storage/node_liveness.go:486 [n1,liveness-hb] failed node liveness heartbeat: operation "node liveness heartbeat" timed out after 4.5s | |
| W190617 21:11:33.165875 94547 storage/node_liveness.go:561 [n1,s1,r4/1:/System{/tsd-tse}] slow heartbeat took 34.9s | |
| W190617 21:11:34.110564 94295 storage/node_liveness.go:561 [n1,liveness-hb] slow heartbeat took 4.6s | |
| E190617 21:11:56.007438 94547 storage/replica_range_lease.go:341 [n1,s1,r4/1:/System{/tsd-tse}] context canceled | |
| W190617 21:12:07.902785 94295 storage/node_liveness.go:486 [n1,liveness-hb] failed node liveness heartbeat: operation "node liveness heartbeat" timed out after 4.5s | |
| W190617 21:12:09.853200 94519 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:12:11.432228 94473 storage/replica_write.go:206 [n1,s1,r1/1:/{Min-System/NodeL…}] have been waiting 60.41s for proposing command RequestLease [/Min,/Min). | |
| This range is likely unavailable. | |
| Please submit this message at | |
| https://github.com/cockroachdb/cockroach/issues/new/choose | |
| along with | |
| https://yourhost:8080/#/reports/range/1 | |
| and the following Raft status: {"id":"1","term":6,"vote":"1","commit":14,"lead":"1","raftState":"StateLeader","applied":13,"progress":{"1":{"match":14,"next":15,"state":"ProgressStateReplicate"}},"leadtransferee":"0"} | |
| W190617 21:12:11.335753 94432 storage/replica_write.go:206 [n1,s1,r2/1:/System/NodeLiveness{-Max}] have been waiting 60.70s for proposing command RequestLease [/System/NodeLiveness,/Min). | |
| This range is likely unavailable. | |
| Please submit this message at | |
| https://github.com/cockroachdb/cockroach/issues/new/choose | |
| along with | |
| https://yourhost:8080/#/reports/range/2 | |
| and the following Raft status: {"id":"1","term":6,"vote":"1","commit":15,"lead":"1","raftState":"StateLeader","applied":14,"progress":{"1":{"match":15,"next":16,"state":"ProgressStateReplicate"}},"leadtransferee":"0"} | |
| W190617 21:12:17.578932 94265 storage/store_rebalancer.go:221 [n1,s1,store-rebalancer] StorePool missing descriptor for local store | |
| W190617 21:12:53.714648 94492 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:12:55.881096 94567 server/server_engine_health.go:76 [n1] disk stall detected: unable to write to <no-attributes>=<in-mem> within 10s | |
| W190617 21:12:57.783989 94074 storage/closedts/provider/provider.go:152 [ct-closer] unable to move closed timestamp forward: not live | |
| W190617 21:12:58.029655 94295 storage/node_liveness.go:561 [n1,liveness-hb] slow heartbeat took 2.3s | |
| W190617 21:13:02.204518 94338 storage/replica_range_lease.go:1032 [n1,s1,r3/1:/System/{NodeLive…-tsd}] have been waiting 1m0s attempting to acquire lease | |
| W190617 21:13:03.305346 94295 storage/node_liveness.go:486 [n1,liveness-hb] failed node liveness heartbeat: operation "node liveness heartbeat" timed out after 4.5s | |
| W190617 21:13:07.915591 94233 storage/store.go:3740 [n1,s1,r16/1:/Table/2{0-1}] handle raft ready: 113.8s [processed=2] | |
| W190617 21:13:07.916055 94235 storage/store.go:3740 [n1,s1,r5/1:/{Systemtse-Table/System…}] handle raft ready: 106.7s [processed=2] | |
| W190617 21:13:07.989858 94234 storage/store.go:3740 [n1,s1,r2/1:/System/NodeLiveness{-Max}] handle raft ready: 117.4s [processed=1] | |
| W190617 21:13:07.990188 94232 storage/store.go:3740 [n1,s1,r1/1:/{Min-System/NodeL…}] handle raft ready: 125.7s [processed=1] | |
| W190617 21:13:08.425797 94236 storage/store.go:3740 [n1,s1,r9/1:/Table/1{3-4}] handle raft ready: 1.3s [processed=2] | |
| W190617 21:13:09.150066 94295 storage/node_liveness.go:561 [n1,liveness-hb] slow heartbeat took 4.8s | |
| W190617 21:13:09.150731 94295 storage/node_liveness.go:486 [n1,liveness-hb] failed node liveness heartbeat: operation "node liveness heartbeat" timed out after 4.5s | |
| W190617 21:13:09.251843 94231 storage/store.go:3740 [n1,s1,r19/1:/Table/2{3-4}] handle raft ready: 312.6s [processed=3] | |
| W190617 21:13:09.589774 94253 storage/store.go:3740 [n1,s1,r2/1:/System/NodeLiveness{-Max}] handle raft ready: 0.9s [processed=1] | |
| W190617 21:13:10.357751 94214 storage/store.go:3740 [n1,s1,r2/1:/System/NodeLiveness{-Max}] handle raft ready: 0.6s [processed=1] | |
| W190617 21:13:10.709048 94386 storage/node_liveness.go:561 [n1,s1,r6/1:/Table/{SystemCon…-11}] slow heartbeat took 183.8s | |
| W190617 21:13:10.709547 94480 storage/node_liveness.go:561 [n1,s1,r3/1:/System/{NodeLive…-tsd}] slow heartbeat took 138.9s | |
| W190617 21:13:10.709840 94295 storage/node_liveness.go:561 [n1,liveness-hb] slow heartbeat took 1.6s | |
| W190617 21:13:10.710006 94522 storage/node_liveness.go:561 [n1,s1,r5/1:/{Systemtse-Table/System…}] slow heartbeat took 1.5s | |
| W190617 21:13:10.709579 94505 storage/node_liveness.go:561 [n1,s1,r4/1:/System{/tsd-tse}] slow heartbeat took 14.0s | |
| W190617 21:13:10.710239 94554 storage/node_liveness.go:561 [n1,s1,r19/1:/Table/2{3-4}] slow heartbeat took 1.1s | |
| E190617 21:13:10.710465 94522 storage/replica_range_lease.go:341 [n1,s1,r5/1:/{Systemtse-Table/System…}] heartbeat failed on epoch increment | |
| E190617 21:13:10.710263 94480 storage/replica_range_lease.go:341 [n1,s1,r3/1:/System/{NodeLive…-tsd}] heartbeat failed on epoch increment | |
| E190617 21:13:10.710780 94554 storage/replica_range_lease.go:341 [n1,s1,r19/1:/Table/2{3-4}] heartbeat failed on epoch increment | |
| E190617 21:13:10.710578 94505 storage/replica_range_lease.go:341 [n1,s1,r4/1:/System{/tsd-tse}] heartbeat failed on epoch increment | |
| F190617 21:13:12.064398 94338 sqlmigrations/migrations.go:458 [n1] not enough time left on migration lease, terminating for safety | |
| goroutine 94338 [running]: | |
| github.com/cockroachdb/cockroach/pkg/util/log.getStacks(0x7587401, 0xc000396360, 0x0, 0xc0018c1838) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1018 +0xbf | |
| github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).outputLogEntry(0x6bbb740, 0xc000000004, 0x660f9e1, 0x1b, 0x1ca, 0xc00f276280, 0x44) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:876 +0xb68 | |
| github.com/cockroachdb/cockroach/pkg/util/log.addStructured(0x4b460a0, 0xc00c764480, 0xc000000004, 0x2, 0x0, 0x0, 0xc0018c1ee8, 0x1, 0x1) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/structured.go:68 +0x291 | |
| github.com/cockroachdb/cockroach/pkg/util/log.logDepth(0x4b460a0, 0xc00c764480, 0x1, 0xc000000004, 0x0, 0x0, 0xc0018c1ee8, 0x1, 0x1) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:69 +0x9a | |
| github.com/cockroachdb/cockroach/pkg/util/log.Fatal(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:189 | |
| github.com/cockroachdb/cockroach/pkg/sqlmigrations.(*Manager).EnsureMigrations.func2(0x4b460a0, 0xc00c764480) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sqlmigrations/migrations.go:458 +0x374 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc00c764480, 0xc0086f5f80, 0x29, 0x0, 0x0, 0xc001aa5e20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:323 +0xf4 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:318 +0x14b | |
| goroutine 1 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).execInternal(0xc00e8f03c0, 0x4b460a0, 0xc00578d1d0, 0x3f755a4, 0x1f, 0x0, 0x1, 0x3f3542c, 0x4, 0xc00578cff0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:567 +0xc5d | |
| github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).Exec(0xc00e8f03c0, 0x4b460a0, 0xc00c764480, 0x3f755a4, 0x1f, 0x0, 0x3fca97c, 0x38, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:302 +0x13b | |
| github.com/cockroachdb/cockroach/pkg/sqlmigrations.runStmtAsRootWithRetry(0x4b460a0, 0xc00c764480, 0x4b46320, 0xc010a9d680, 0xc00e8f03c0, 0x3f755a4, 0x1f, 0x3fca97c, 0x38, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sqlmigrations/migrations.go:565 +0x30b | |
| github.com/cockroachdb/cockroach/pkg/sqlmigrations.optInToDiagnosticsStatReporting(0x4b460a0, 0xc00c764480, 0x4b46320, 0xc010a9d680, 0xc00e8f03c0, 0x0, 0x1) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sqlmigrations/migrations.go:588 +0xf9 | |
| github.com/cockroachdb/cockroach/pkg/sqlmigrations.(*Manager).EnsureMigrations(0xc002473680, 0x4b460a0, 0xc00c764480, 0xc00e184300, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sqlmigrations/migrations.go:498 +0xb28 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start(0xc00df6d000, 0x4b46020, 0xc000052118, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1653 +0x4d8f | |
| github.com/cockroachdb/cockroach/pkg/server.(*TestServer).Start(0xc006aaa1c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/testserver.go:350 +0x2ea | |
| github.com/cockroachdb/cockroach/pkg/testutils/serverutils.StartServerRaw(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/testutils/serverutils/test_server_shim.go:210 +0x160 | |
| github.com/cockroachdb/cockroach/pkg/cli.newCLITest(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/cli/cli_test.go:132 +0x38d | |
| github.com/cockroachdb/cockroach/pkg/cli.Example_node() | |
| /go/src/github.com/cockroachdb/cockroach/pkg/cli/cli_test.go:1890 +0xa5 | |
| testing.runExample(0x3f446fd, 0xc, 0x40a1dd8, 0x3ff6e81, 0x6f, 0x0, 0x1) | |
| /usr/local/go/src/testing/example.go:121 +0x2a5 | |
| testing.runExamples(0xc001e1dec0, 0x6945e40, 0x11, 0x11, 0x101) | |
| /usr/local/go/src/testing/example.go:45 +0x214 | |
| testing.(*M).Run(0xc0007a6f00, 0x0) | |
| /usr/local/go/src/testing/testing.go:1073 +0x3a0 | |
| github.com/cockroachdb/cockroach/pkg/cli_test.TestMain(0xc0007a6f00) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/cli/main_test.go:36 +0xa6 | |
| main.main() | |
| _testmain.go:156 +0x223 | |
| goroutine 5 [syscall, 11 minutes]: | |
| os/signal.signal_recv(0x946031) | |
| /usr/local/go/src/runtime/sigqueue.go:139 +0x9c | |
| os/signal.loop() | |
| /usr/local/go/src/os/signal/signal_unix.go:23 +0x30 | |
| created by os/signal.init.0 | |
| /usr/local/go/src/os/signal/signal_unix.go:29 +0x4f | |
| goroutine 7 [semacquire]: | |
| sync.runtime_SemacquireMutex(0x6bbb798, 0x1100000000) | |
| /usr/local/go/src/runtime/sema.go:71 +0x3d | |
| sync.(*Mutex).Lock(0x6bbb794) | |
| /usr/local/go/src/sync/mutex.go:134 +0x1e2 | |
| github.com/cockroachdb/cockroach/pkg/util/syncutil.(*Mutex).Lock(0x6bbb794) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/syncutil/mutex_sync_race.go:31 +0x3b | |
| github.com/cockroachdb/cockroach/pkg/util/log.flushDaemon() | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1229 +0x17b | |
| created by github.com/cockroachdb/cockroach/pkg/util/log.init.0 | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:627 +0x1e4 | |
| goroutine 8 [chan receive, 11 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/util/log.signalFlusher() | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:634 +0x14b | |
| created by github.com/cockroachdb/cockroach/pkg/util/log.init.0 | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:628 +0x1fc | |
| goroutine 94000 [select]: | |
| github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc00e607400) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:122 +0x1fe | |
| created by github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.newCCBalancerWrapper | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:113 +0x28d | |
| goroutine 94159 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux.muxListener.Accept(...) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:184 | |
| crypto/tls.(*listener).Accept(0xc0093d8440, 0x6938401, 0xc009862210, 0xc001405e10, 0xa35f9d) | |
| /usr/local/go/src/crypto/tls/tls.go:62 +0x55 | |
| net/http.(*Server).Serve(0xc004669c70, 0x4b1ef60, 0xc0093d8440, 0x0, 0x0) | |
| /usr/local/go/src/net/http/server.go:2859 +0x356 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func5(0x4b460a0, 0xc009862180) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1205 +0x93 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448f60, 0xc00d9a01e0, 0xc0093d8460) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94278 [select]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc002474060) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:756 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be36230, 0xc00d9a01e0, 0xc00cf21880) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94114 [select]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).executeWriteBatch(0xc008222680, 0x4b460a0, 0xc00caa7260, 0x15a918c78c581091, 0x0, 0x100000001, 0x1, 0x0, 0x3, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_write.go:182 +0xf4d | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).sendWithRangeID(0xc008222680, 0x4b460a0, 0xc00caa7260, 0x3, 0x15a918c78c581091, 0x0, 0x100000001, 0x1, 0x0, 0x3, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica.go:541 +0x9dd | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).Send(0xc008222680, 0x4b460a0, 0xc00caa71d0, 0x15a918c78c581091, 0x0, 0x100000001, 0x1, 0x0, 0x3, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica.go:489 +0xee | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).Send(0xc001b73500, 0x4b460a0, 0xc00caa70b0, 0x15a918c78c581091, 0x0, 0x100000001, 0x1, 0x0, 0x3, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:2974 +0x7da | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Stores).Send(0xc0097ecf20, 0x4b460a0, 0xc00caa70b0, 0x0, 0x0, 0x100000001, 0x1, 0x0, 0x3, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/stores.go:183 +0x11f | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal.func1(0x4b460a0, 0xc00caa6fc0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:917 +0x270 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc00d9a01e0, 0x4b460a0, 0xc00caa6fc0, 0x3f4e14b, 0x10, 0xc000c3e698, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:302 +0xf9 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal(0xc00d566900, 0x4b460a0, 0xc00caa6fc0, 0xc008c5aa20, 0xc00caa6fc0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:904 +0x266 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch(0xc00d566900, 0x4b460a0, 0xc00caa6f90, 0xc008c5aa20, 0xa36fbc, 0xc00c482e10, 0x947acc) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:945 +0xb5 | |
| github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.Batch(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:497 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*grpcTransport).sendBatch(0xc00c3e8570, 0x4b460a0, 0xc00caa6f90, 0x1, 0x4b13820, 0xc00c448a00, 0x0, 0x0, 0x100000001, 0x1, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/transport.go:196 +0x1d7 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*grpcTransport).SendNext(0xc00c3e8570, 0x4b460a0, 0xc00c482e10, 0x0, 0x0, 0x100000001, 0x1, 0x0, 0x3, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/transport.go:165 +0x383 | |
| github.com/cockroachdb/cockroach/pkg/kv.raceTransport.SendNext(0x4b50ce0, 0xc00c3e8570, 0x4b460a0, 0xc00c482e10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/transport_race.go:80 +0x483 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendToReplicas(0xc0027c7e60, 0x4b460a0, 0xc00c482e10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:1800 +0x652 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendRPC(0xc0027c7e60, 0x4b460a0, 0xc00c482e10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:428 +0x377 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendSingleRange(0xc0027c7e60, 0x4b460a0, 0xc00c482e10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:505 +0x29d | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendPartialBatch(0xc0027c7e60, 0x4b460a0, 0xc00c482e10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:1413 +0x415 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).divideAndSendBatchToRanges(0xc0027c7e60, 0x4b460a0, 0xc00c482e10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:1065 +0x1b17 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).Send(0xc0027c7e60, 0x4b460a0, 0xc0024758f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:717 +0x13c4 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*CrossRangeTxnWrapperSender).Send(0xc010a9d6e0, 0x4b460a0, 0xc0024758f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:219 +0x1d2 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).sendUsingSender(0xc010a9d680, 0x4b460a0, 0xc0024758f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:663 +0x1ae | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).send(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:645 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.sendAndFill(0x4b460a0, 0xc0024758f0, 0xc010cf7510, 0xc00378ab00, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:588 +0x18e | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Run(0xc010a9d680, 0x4b460a0, 0xc0024758f0, 0xc00378ab00, 0xc00eeae600, 0x8f0901) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:611 +0xd2 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).PutInline(0xc010a9d680, 0x4b460a0, 0xc0024758f0, 0x3d93700, 0xc009383de0, 0x3e25120, 0xc00eeae600, 0x0, 0xc010cf7648) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:350 +0xee | |
| github.com/cockroachdb/cockroach/pkg/server/status.(*MetricsRecorder).WriteNodeStatus(0xc0074dbc80, 0x4b460a0, 0xc0024758f0, 0xc010a9d680, 0x1, 0x3f34574, 0x3, 0xc0089803b0, 0xf, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/status/recorder.go:501 +0x191 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).writeNodeStatus.func1(0x4b460a0, 0xc0024758f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:829 +0x395 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTask(0xc00d9a01e0, 0x4b460a0, 0xc0024758f0, 0x3f66985, 0x1a, 0xc00a4fee30, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:285 +0xf9 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).writeNodeStatus(0xc00d566900, 0x4b460a0, 0xc0024758f0, 0x4a817c800, 0x1, 0x30) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:800 +0xda | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).startWriteNodeStatus.func1(0x4b460a0, 0xc0024758f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:786 +0x1d4 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00ae814e0, 0xc00d9a01e0, 0xc00c6ea160) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94213 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d4) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723650) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723650) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715140, 0xc00d9a01e0, 0xc005715130) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94237 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000ea) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfce0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfce0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057154b0, 0xc00d9a01e0, 0xc0057154a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94073 [semacquire, 5 minutes]: | |
| sync.runtime_SemacquireMutex(0xc0089803c4, 0x900000000) | |
| /usr/local/go/src/runtime/sema.go:71 +0x3d | |
| sync.(*Mutex).Lock(0xc0089803c0) | |
| /usr/local/go/src/sync/mutex.go:134 +0x1e2 | |
| sync.(*Once).Do(0xc0089803c0, 0xc003e86740) | |
| /usr/local/go/src/sync/once.go:40 +0x61 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func15(0x4b460a0, 0xc005723260) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1490 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714300, 0xc00d9a01e0, 0xc00ce8cde0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94242 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xed) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723890) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723890) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715560, 0xc00d9a01e0, 0xc005715550) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94207 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000ce) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057235c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057235c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715030, 0xc00d9a01e0, 0xc005715020) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94154 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/util/hlc.(*Clock).StartMonitoringForwardClockJumps.func1(0x40a80b8, 0xc00e184370, 0xc00d8de460, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/hlc/hlc.go:183 +0x17c | |
| created by github.com/cockroachdb/cockroach/pkg/util/hlc.(*Clock).StartMonitoringForwardClockJumps | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/hlc/hlc.go:176 +0x14a | |
| goroutine 94294 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*Server).PeriodicallyClearStmtStats.func1(0x4b460a0, 0xc00afbef60) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:680 +0x31d | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160190, 0xc00d9a01e0, 0xc00a77ee20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94200 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c8) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf470) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf470) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714f10, 0xc00d9a01e0, 0xc005714f00) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94246 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f6) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723920) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723920) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057155e0, 0xc00d9a01e0, 0xc0057155d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94262 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).startLeaseRenewer.func1(0x4b460a0, 0xc00afbe3c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1593 +0x3ed | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057157f0, 0xc00d9a01e0, 0xc0057157e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94074 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/storage/closedts.AsCloseFn.func1(0x15a918c1ed43c9de, 0x0, 0x1, 0x1, 0x0, 0x0, 0xc0133c3dd8) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/closedts/closedts.go:170 +0xf5 | |
| github.com/cockroachdb/cockroach/pkg/storage/closedts/provider.(*Provider).runCloser(0xc005062580, 0x4b460a0, 0xc0057232c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/closedts/provider/provider.go:160 +0x8cf | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714430, 0xc00d9a01e0, 0xc005714400) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94529 [runnable]: | |
| sync.(*RWMutex).RLock(0xc00b3c2218) | |
| /usr/local/go/src/sync/rwmutex.go:43 +0xa3 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).IsInitialized(0xc00b3c2000, 0xc00e085300) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_init.go:250 +0x67 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).maybeAdd(0xc0014a66e0, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba5ebc36, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:570 +0x108 | |
| github.com/cockroachdb/cockroach/pkg/storage.baseQueueHelper.MaybeAdd(0xc0014a66e0, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba5ebc36, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:495 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).MaybeAddAsync.func1(0x4b460a0, 0xc002474030, 0x4b13da0, 0xc0014a66e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:536 +0x8a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).Async.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:525 +0x66 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc002474030, 0xc005f70b20, 0x14, 0xc007f53b60, 0x0, 0x0, 0xc008e0d3e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:383 +0x11e | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:377 +0x2a9 | |
| goroutine 94167 [select]: | |
| github.com/cockroachdb/cockroach/pkg/server.startAssertEngineHealth.func1(0x4b460a0, 0xc0065ac720) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server_engine_health.go:46 +0x22e | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c449510, 0xc00d9a01e0, 0xc0065a77d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94170 [runnable]: | |
| strings.(*Builder).WriteString(...) | |
| /usr/local/go/src/strings/builder.go:122 | |
| strings.Join(0xc00f475c40, 0x2, 0x2, 0x3f33957, 0x1, 0x0, 0x0) | |
| /usr/local/go/src/strings/strings.go:441 +0x45f | |
| github.com/cockroachdb/cockroach/pkg/gossip.MakeKey(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/gossip/keys.go:102 | |
| github.com/cockroachdb/cockroach/pkg/gossip.MakeGossipClientsKey(0xc000000001, 0x28, 0x6) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/gossip/keys.go:140 +0xb5 | |
| github.com/cockroachdb/cockroach/pkg/gossip.(*Gossip).updateClients(0xc00add96c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/gossip/gossip.go:898 +0x364 | |
| github.com/cockroachdb/cockroach/pkg/gossip.(*Gossip).manage.func1(0x4b460a0, 0xc0065aca50) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/gossip/gossip.go:1392 +0x74e | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c4495b0, 0xc00d9a01e0, 0xc00c4495a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94076 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher.(*RequestBatcher).run(0xc00365c2a0, 0x4b46020, 0xc000052118) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher/batcher.go:408 +0x40a | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d9a01e0, 0x4b46020, 0xc000052118, 0xc0073f6e10, 0x22, 0x0, 0x0, 0xc0057145e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:323 +0xf4 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:318 +0x14b | |
| goroutine 94164 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*ccResolverWrapper).watcher(0xc0065a7740) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/resolver_conn_wrapper.go:109 +0x221 | |
| created by github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*ccResolverWrapper).start | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/resolver_conn_wrapper.go:95 +0x4d | |
| goroutine 94258 [runnable]: | |
| github.com/cockroachdb/cockroach/vendor/github.com/google/btree.(*node).iterate(0xc0044b2800, 0xffffffffffffffff, 0x4ae3520, 0xc009069080, 0x0, 0x0, 0x1, 0xc011a41b50, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/github.com/google/btree/btree.go:502 +0xd31 | |
| github.com/cockroachdb/cockroach/vendor/github.com/google/btree.(*BTree).DescendLessOrEqual(...) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/github.com/google/btree/btree.go:795 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).LookupReplica(0xc001b73500, 0x6961d20, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1973 +0x375 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).startGossip.func4(0x4b460a0, 0xc005723b00) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1529 +0x10d | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715790, 0xc00d9a01e0, 0xc00e2acaa0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94240 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000cc) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723860) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723860) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715520, 0xc00d9a01e0, 0xc005715510) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94148 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc0065ac150) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448500, 0xc00d9a01e0, 0xc00c4484f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94156 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func2(0x4b460a0, 0xc0065ac390) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1174 +0x98 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448f30, 0xc00d9a01e0, 0xc0093d81a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94257 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).Start.func2(0x4b460a0, 0xc0065ad950) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1403 +0x15a | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715780, 0xc00d9a01e0, 0xc00a77e360) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94530 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/util/syncutil.(*Mutex).Unlock(0xc0014a7188) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/syncutil/mutex_sync_race.go:36 +0x69 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).maybeAdd(0xc0014a7080, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba6117e5, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:564 +0xde | |
| github.com/cockroachdb/cockroach/pkg/storage.baseQueueHelper.MaybeAdd(0xc0014a7080, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba6117e5, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:495 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).MaybeAddAsync.func1(0x4b460a0, 0xc002474030, 0x4b13da0, 0xc0014a7080) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:536 +0x8a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).Async.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:525 +0x66 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc002474030, 0xc005f70b40, 0x12, 0xc007f53ce0, 0x0, 0x0, 0xc008e0d400) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:383 +0x11e | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:377 +0x2a9 | |
| goroutine 94259 [select]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).startGossip.func4(0x4b460a0, 0xc0065ad980) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1544 +0x43d | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057157a0, 0xc00d9a01e0, 0xc00e2acaf0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94131 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/rpc.NewContext.func1(0x4b460a0, 0xc009862150) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:446 +0x69 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00897f360, 0xc00d9a01e0, 0xc002a33420) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94253 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000cb) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cff80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cff80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057156c0, 0xc00d9a01e0, 0xc0057156b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94238 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f0) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723830) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723830) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057154d0, 0xc00d9a01e0, 0xc0057154c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94143 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc008b954a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448460, 0xc00d9a01e0, 0xc00c448450) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94270 [select]: | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).startGossip.func1(0x4b460a0, 0xc00427f050) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:683 +0x39c | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715f90, 0xc00d9a01e0, 0xc00a77ec40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94275 [select]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc00afbe420) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:756 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be36200, 0xc00d9a01e0, 0xc00cf21820) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94339 [select]: | |
| github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).Start.func1(0x4b460a0, 0xc009863770) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:297 +0x185 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00ae814f0, 0xc00d9a01e0, 0xc00281fd10) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94244 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xf3) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057238c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057238c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057155a0, 0xc00d9a01e0, 0xc005715590) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94078 [semacquire]: | |
| sync.runtime_SemacquireMutex(0x6bbb798, 0x900000000) | |
| /usr/local/go/src/runtime/sema.go:71 +0x3d | |
| sync.(*Mutex).Lock(0x6bbb794) | |
| /usr/local/go/src/sync/mutex.go:134 +0x1e2 | |
| github.com/cockroachdb/cockroach/pkg/util/syncutil.(*Mutex).Lock(0x6bbb794) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/syncutil/mutex_sync_race.go:31 +0x3b | |
| github.com/cockroachdb/cockroach/pkg/util/log.VDepth(0x1, 0x1, 0x100000000011ef3) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1477 +0x12f | |
| github.com/cockroachdb/cockroach/pkg/util/log.V(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:210 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).traceEntries(0xc008222c00, 0xc00d8273b0, 0x1, 0x1, 0x3f8a0e8, 0x26) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft.go:224 +0x45 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).handleRaftReadyRaftMuLocked(0xc008222c00, 0x4b460a0, 0xc00ca4fb60, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:777 +0x98e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).handleRaftReady(0xc008222c00, 0x4b460a0, 0xc00ca4fb60, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:506 +0x16b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).processReady(0xc001b73500, 0x4b460a0, 0xc0087cf350, 0x4) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:3729 +0x172 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf350) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:212 +0x33f | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf350) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714da0, 0xc00d9a01e0, 0xc005714d90) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94133 [sync.Cond.Wait, 5 minutes]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0056ba618, 0x0) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0056ba608) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage/engine.(*RocksDB).syncLoop(0xc0056ba4e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:677 +0x81 | |
| created by github.com/cockroachdb/cockroach/pkg/storage/engine.(*RocksDB).open | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:664 +0x4e7 | |
| goroutine 94255 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*NodeLiveness).GetIsLiveMap(0xc008b5b7a0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/node_liveness.go:679 +0x2ea | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).raftTickLoop(0xc001b73500, 0x4b460a0, 0xc0087cffb0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:3828 +0x4c1 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715700, 0xc00d9a01e0, 0xc0057156f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94243 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000ec) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfd70) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfd70) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715580, 0xc00d9a01e0, 0xc005715570) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94162 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func9(0x4b460a0, 0xc009862300) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1299 +0xbf | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c449330, 0xc00d9a01e0, 0xc0065a7680) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94151 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).gcDaemon(0xc00e8e4c80, 0x4b45fe0, 0xc00c12c9c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1288 +0x13d | |
| created by github.com/cockroachdb/cockroach/pkg/util/log.NewSecondaryLogger | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/secondary_log.go:81 +0x4c8 | |
| goroutine 94081 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000be) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723470) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723470) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714e10, 0xc00d9a01e0, 0xc005714e00) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94218 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000da) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf920) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf920) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057151f0, 0xc00d9a01e0, 0xc0057151e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94080 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/storage/stateloader.StateLoader.SetHardState(0xc00c5017c0, 0x3, 0x20, 0x4b460a0, 0xc00caa7bf0, 0x4b9dd40, 0xc002df7330, 0x6, 0x1, 0x15, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/stateloader/stateloader.go:582 +0x433 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).handleRaftReadyRaftMuLocked(0xc008222680, 0x4b460a0, 0xc00caa7bf0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:700 +0x2578 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).handleRaftReady(0xc008222680, 0x4b460a0, 0xc00caa7bf0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:506 +0x16b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).processReady(0xc001b73500, 0x4b460a0, 0xc0087cf380, 0x3) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:3729 +0x172 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:212 +0x33f | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714df0, 0xc00d9a01e0, 0xc005714dd0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94134 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc00d103ec0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448310, 0xc00d9a01e0, 0xc00c448300) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94226 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e5) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057236e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057236e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715310, 0xc00d9a01e0, 0xc005715300) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94202 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c5) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf4a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf4a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714f60, 0xc00d9a01e0, 0xc005714f50) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94166 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0056bb7f8, 0x5a) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0056bb7e8) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage/engine.(*RocksDB).syncLoop(0xc0056bb6c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:677 +0x81 | |
| created by github.com/cockroachdb/cockroach/pkg/storage/engine.(*RocksDB).open | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:664 +0x4e7 | |
| goroutine 94260 [select]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).startGossip.func4(0x4b460a0, 0xc00281f7a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1544 +0x43d | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057157b0, 0xc00d9a01e0, 0xc00e2acbe0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94223 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e1) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057236b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057236b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057152b0, 0xc00d9a01e0, 0xc0057152a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94225 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000df) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc000cd5f50) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc000cd5f50) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057152f0, 0xc00d9a01e0, 0xc0057152e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94250 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f4) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057239b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057239b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715660, 0xc00d9a01e0, 0xc005715650) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94132 [select]: | |
| github.com/cockroachdb/cockroach/pkg/gossip.newInfoStore.func1(0x4b460a0, 0xc008b95380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/gossip/infostore.go:192 +0x225 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00897f420, 0xc00d9a01e0, 0xc00897f410) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94211 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d7) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723620) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723620) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715100, 0xc00d9a01e0, 0xc0057150f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94168 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/gossip.(*server).start.func3(0x4b460a0, 0xc009862540) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/gossip/server.go:385 +0x86 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c449570, 0xc00d9a01e0, 0xc0093d9700) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94644 [runnable]: | |
| sync.(*RWMutex).RLock(0xc00b3c2218) | |
| /usr/local/go/src/sync/rwmutex.go:43 +0xa3 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).IsInitialized(0xc00b3c2000, 0xc007fa0000) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_init.go:250 +0x67 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).maybeAdd(0xc0014a74a0, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba656999, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:570 +0x108 | |
| github.com/cockroachdb/cockroach/pkg/storage.baseQueueHelper.MaybeAdd(0xc0014a74a0, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba656999, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:495 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).MaybeAddAsync.func1(0x4b460a0, 0xc002474030, 0x4b13da0, 0xc0014a74a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:536 +0x8a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).Async.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:525 +0x66 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc002474030, 0xc005f70b80, 0x1d, 0xc003b02000, 0x0, 0x0, 0xc008e0d440) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:383 +0x11e | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:377 +0x2a9 | |
| goroutine 94252 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xfa) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057239e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057239e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057156a0, 0xc00d9a01e0, 0xc005715690) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94205 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d1) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723590) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723590) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714fd0, 0xc00d9a01e0, 0xc005714fc0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94263 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b25d0, 0xc) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b25c0) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage/closedts/provider.(*Provider).Subscribe(0xc005062580, 0x4b460a0, 0xc00be75d40, 0xc00c965920) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/closedts/provider/provider.go:292 +0x6db | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).startClosedTimestampRangefeedSubscriber.func1(0x4b460a0, 0xc00be75d40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1614 +0xb2 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc00be75d40, 0xc0073f7ef0, 0x25, 0x0, 0x0, 0xc00a77e380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:323 +0xf4 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:318 +0x14b | |
| goroutine 94220 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d8) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf980) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf980) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715250, 0xc00d9a01e0, 0xc005715240) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94175 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc00427e000) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:756 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be361c0, 0xc00d9a01e0, 0xc00cf217a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94345 [chan receive]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).execInternal(0xc00e8f03c0, 0x4b460a0, 0xc00d8af530, 0x3f4a328, 0xe, 0xc00af3a3f0, 0x1, 0x3f3542c, 0x4, 0xc007d1baa0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:567 +0xc5d | |
| github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).Exec(0xc00e8f03c0, 0x4b460a0, 0xc005aca270, 0x3f4a328, 0xe, 0xc00af3a3f0, 0x3ff22ea, 0x60, 0xc000e11ef8, 0x3, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:302 +0x13b | |
| github.com/cockroachdb/cockroach/pkg/sql.(*setClusterSettingNode).startExec.func1(0x4b460a0, 0xc005aca270, 0xc00af3a3f0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/set_cluster_setting.go:147 +0x750 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Txn.func1(0x4b460a0, 0xc005aca270, 0xc00af3a3f0, 0x352929742c334bbd, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:626 +0x5f | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*Txn).exec(0xc00af3a3f0, 0x4b460a0, 0xc005aca270, 0xc00cb96360, 0x1, 0xc00af3a3f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/txn.go:688 +0xea | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Txn(0xc010a9d680, 0x4b460a0, 0xc005aca270, 0xc013245e30, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:625 +0x162 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*setClusterSettingNode).startExec(0xc0099ddb40, 0x4b460a0, 0xc005aca270, 0xc00d7386c0, 0xc005a246d8, 0x9, 0xc000064aa0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/set_cluster_setting.go:108 +0x31a | |
| github.com/cockroachdb/cockroach/pkg/sql.startExec.func2(0x3f55673, 0x13, 0x4b47d20, 0xc0099ddb40, 0x28, 0xc000f40600) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan.go:486 +0x71 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal.func1(0xc008b83740, 0x3f55673, 0x13, 0x4b47d20, 0xc0099ddb40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:144 +0x9a | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal(0xc008b83740, 0x4b47d20, 0xc0099ddb40, 0x3f55673, 0x13) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:626 +0x562 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visit(0xc008b83740, 0x4b47d20, 0xc0099ddb40, 0x968053, 0x6bbb7b8) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:111 +0xfa | |
| github.com/cockroachdb/cockroach/pkg/sql.walkPlan(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:75 | |
| github.com/cockroachdb/cockroach/pkg/sql.startExec(0x4b460a0, 0xc005aca270, 0xc00d7386c0, 0xc005a246d8, 0x4b47d20, 0xc0099ddb40, 0x3d95300, 0x3e8ca60) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan.go:489 +0x2ec | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Start(0xc00d830000, 0x4b460a0, 0xc005aca270, 0xc00e0f88c0, 0xc00e0f88a8) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan_node_to_row_source.go:119 +0x1c9 | |
| github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*ProcessorBase).Run(0xc00d830000, 0x4b460a0, 0xc005aca270) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/processors.go:793 +0x8b | |
| github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*Flow).Run(0xc00e0f8780, 0x4b460a0, 0xc005aca270, 0x40a2b90, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/flow.go:624 +0x404 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run(0xc00932bcc0, 0xc00a750120, 0xc00ccee480, 0xc00e679f08, 0xc002d14000, 0xc005a247b0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:251 +0xcb7 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun(0xc00932bcc0, 0x4b460a0, 0xc00b551980, 0xc005a247b0, 0xc00a750120, 0xc00ccee480, 0x4b47d20, 0xc0099ddb40, 0xc002d14000) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:839 +0x25f | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine(0xc005a24380, 0x4b460a0, 0xc00b551980, 0xc005a246d8, 0x0, 0x7fdab173f1e8, 0xc00dfe11a0, 0xc00e67a200, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:903 +0x5d3 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine(0xc005a24380, 0x4b460a0, 0xc00b551980, 0xc005a246d8, 0x7fdab173f1e8, 0xc00dfe11a0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:738 +0xab0 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState(0xc005a24380, 0x4b460a0, 0xc00b551980, 0x4b4b020, 0xc00c6ebec0, 0x3fca97c, 0x38, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:417 +0xb0e | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt(0xc005a24380, 0x4b460a0, 0xc00b551980, 0x4b4b020, 0xc00c6ebec0, 0x3fca97c, 0x38, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:98 +0x6d7 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc005a24380, 0x4b45fe0, 0xc0093a4580, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1205 +0x23ee | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc005a24380, 0x4b460a0, 0xc00578d1d0, 0xc00d9a1900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1141 +0x3e8 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).initConnEx.func1(0xc005a24380, 0x4b460a0, 0xc00578d1d0, 0xc00e8f03c0, 0xc00c6ebf00, 0x0, 0xc0061d89e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:203 +0x8f | |
| created by github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).initConnEx | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:202 +0xa2e | |
| goroutine 94196 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c2) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf3e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf3e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714e70, 0xc00d9a01e0, 0xc005714e60) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94231 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000ca) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfc50) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfc50) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057153e0, 0xc00d9a01e0, 0xc0057153d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94214 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000eb) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf830) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf830) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715160, 0xc00d9a01e0, 0xc005715150) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94201 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000cd) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723530) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723530) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714f40, 0xc00d9a01e0, 0xc005714f30) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94172 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage/closedts/provider.(*Provider).Notify.func1(0x4b460a0, 0xc005723380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/closedts/provider/provider.go:219 +0x1ae | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be360d0, 0xc00d9a01e0, 0xc00cf21500) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94526 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/util/encoding.EncodeUvarintAscending(0x0, 0x0, 0x0, 0x12, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/encoding/encoding.go:329 +0xd86 | |
| github.com/cockroachdb/cockroach/pkg/keys.MakeTablePrefix(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/keys/keys.go:672 | |
| github.com/cockroachdb/cockroach/pkg/config.(*SystemConfig).ComputeSplitKey.func1(0x1800000011, 0xc000000031, 0x18, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/config/system.go:437 +0xdb | |
| github.com/cockroachdb/cockroach/pkg/config.(*SystemConfig).ComputeSplitKey(0xc00e085320, 0xc00c416118, 0x1, 0x8, 0xc00c416120, 0x1, 0x8, 0xc0013f7e60, 0x1, 0x1) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/config/system.go:479 +0x594 | |
| github.com/cockroachdb/cockroach/pkg/config.(*SystemConfig).NeedsSplit(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/config/system.go:497 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).requiresSplit(0xc0014a6160, 0xc00e085320, 0x4b86ec0, 0xc00b3c2000, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:616 +0x12a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).maybeAdd(0xc0014a6160, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba585e6f, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:578 +0x7d2 | |
| github.com/cockroachdb/cockroach/pkg/storage.baseQueueHelper.MaybeAdd(0xc0014a6160, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba585e6f, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:495 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).MaybeAddAsync.func1(0x4b460a0, 0xc002474030, 0x4b13da0, 0xc0014a6160) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:536 +0x8a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).Async.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:525 +0x66 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc002474030, 0xc0061d9460, 0x10, 0xc0018a3500, 0x0, 0x0, 0xc008e0d380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:383 +0x11e | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:377 +0x2a9 | |
| goroutine 94241 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000ef) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfd40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfd40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715540, 0xc00d9a01e0, 0xc005715530) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94001 [select]: | |
| github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*addrConn).createTransport(0xc000280840, 0x9, 0x0, 0xbf3a1e80175b936a, 0xa058acfd02, 0x6bb8920, 0xbf3a1e80175b936a, 0xa058acfd02, 0x6bb8920, 0xc00d4ba780, ...) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1361 +0x12fa | |
| github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc000280840, 0x38, 0xc00af850c0) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1251 +0x763 | |
| github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*addrConn).transportMonitor(0xc000280840) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1436 +0x621 | |
| github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*addrConn).connect.func1(0xc000280840) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:960 +0x275 | |
| created by github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*addrConn).connect | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:951 +0x191 | |
| goroutine 94198 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c1) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf410) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf410) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714ec0, 0xc00d9a01e0, 0xc005714ea0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94142 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc0065ac090) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448440, 0xc00d9a01e0, 0xc00c448430) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94265 [select]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*StoreRebalancer).Start.func1(0x4b460a0, 0xc0093e13b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_rebalancer.go:184 +0x1fa | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715810, 0xc00d9a01e0, 0xc00a77e3e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94157 [IO wait, 5 minutes]: | |
| internal/poll.runtime_pollWait(0x7fdab1752450, 0x72, 0x72) | |
| /usr/local/go/src/runtime/netpoll.go:182 +0x56 | |
| internal/poll.(*pollDesc).wait(0xc008906818, 0x72, 0x0, 0x0, 0x3f3a30a) | |
| /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe5 | |
| internal/poll.(*pollDesc).waitRead(...) | |
| /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 | |
| internal/poll.(*FD).Accept(0xc008906800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /usr/local/go/src/internal/poll/fd_unix.go:384 +0x294 | |
| net.(*netFD).accept(0xc008906800, 0x10, 0x100000000203002, 0x0) | |
| /usr/local/go/src/net/fd_unix.go:238 +0x56 | |
| net.(*TCPListener).accept(0xc00c621268, 0x0, 0xc00bb4b610, 0xc008b955f0) | |
| /usr/local/go/src/net/tcpsock_posix.go:139 +0x50 | |
| net.(*TCPListener).Accept(0xc00c621268, 0x40a5530, 0xc00c12cbc0, 0xc00bb4b610, 0xc011405140) | |
| /usr/local/go/src/net/tcpsock.go:260 +0x51 | |
| github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux.(*cMux).Serve(0xc00c12cbc0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:124 +0xcc | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func3(0x4b460a0, 0xc008b955f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1186 +0x47 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448f40, 0xc00d9a01e0, 0xc0093d8340) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94050 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/security.(*CertificateManager).RegisterSignalHandler.func1(0xc00d9a01e0, 0xc0099e1e40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/security/certificate_manager.go:177 +0x17e | |
| created by github.com/cockroachdb/cockroach/pkg/security.(*CertificateManager).RegisterSignalHandler | |
| /go/src/github.com/cockroachdb/cockroach/pkg/security/certificate_manager.go:174 +0x57 | |
| goroutine 94233 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f7) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfc80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfc80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715430, 0xc00d9a01e0, 0xc005715420) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94137 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc008b953e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448390, 0xc00d9a01e0, 0xc00c448360) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94234 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f2) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057237d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057237d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715450, 0xc00d9a01e0, 0xc005715440) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94340 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).Start.func2(0x4b460a0, 0xc0098637d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:308 +0x155 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00ae81500, 0xc00d9a01e0, 0xc00c6ea180) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94140 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc00d103f80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448400, 0xc00d9a01e0, 0xc00c4483e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94178 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc00427e090) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:756 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be361f0, 0xc00d9a01e0, 0xc00cf21800) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94209 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d3) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057235f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057235f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057150c0, 0xc00d9a01e0, 0xc0057150b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94187 [chan send]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).addToPurgatoryLocked.func2.1(0xc0014a6420, 0x4b460a0, 0xc00b872e10, 0xc00d9a01e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:1098 +0x7f | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).addToPurgatoryLocked.func2(0x4b460a0, 0xc00b872e10) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:1130 +0x65d | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00cb7a410, 0xc00d9a01e0, 0xc00c871880) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94024 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/kv.GRPCTransportFactory.func1(0x4b46020, 0xc000052140) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/transport_race.go:139 +0x451 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d9a01e0, 0x4b46020, 0xc000052140, 0xc0053b45e0, 0x17, 0x0, 0x0, 0xc0062c2110) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:323 +0xf4 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:318 +0x14b | |
| goroutine 94249 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f9) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfef0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfef0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715640, 0xc00d9a01e0, 0xc005715630) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94138 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc00d103f20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c4483b0, 0xc00d9a01e0, 0xc00c4483a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94079 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c0) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723440) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723440) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714dc0, 0xc00d9a01e0, 0xc005714db0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94274 [select]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).executeWriteBatch(0xc008222c00, 0x4b460a0, 0xc00cb8ade0, 0x15a918c7803b969d, 0x0, 0x100000001, 0x1, 0x0, 0x4, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_write.go:182 +0xf4d | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).sendWithRangeID(0xc008222c00, 0x4b460a0, 0xc00cb8ade0, 0x4, 0x15a918c7803b969d, 0x0, 0x100000001, 0x1, 0x0, 0x4, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica.go:541 +0x9dd | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).Send(0xc008222c00, 0x4b460a0, 0xc00cb8ad80, 0x15a918c7803b969d, 0x0, 0x100000001, 0x1, 0x0, 0x4, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica.go:489 +0xee | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).Send(0xc001b73500, 0x4b460a0, 0xc00cb8acf0, 0x15a918c7803b969d, 0x0, 0x100000001, 0x1, 0x0, 0x4, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:2974 +0x7da | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Stores).Send(0xc0097ecf20, 0x4b460a0, 0xc00cb8acf0, 0x0, 0x0, 0x100000001, 0x1, 0x0, 0x4, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/stores.go:183 +0x11f | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal.func1(0x4b460a0, 0xc00cb8ac90, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:917 +0x270 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc00d9a01e0, 0x4b460a0, 0xc00cb8ac90, 0x3f4e14b, 0x10, 0xc0078da8e0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:302 +0xf9 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal(0xc00d566900, 0x4b460a0, 0xc00cb8ac90, 0xc00651a240, 0xc00cb8ac90, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:904 +0x266 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch(0xc00d566900, 0x4b460a0, 0xc00cb8ac60, 0xc00651a240, 0xa36fbc, 0xc00caa7200, 0x947acc) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:945 +0xb5 | |
| github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.Batch(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:497 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*grpcTransport).sendBatch(0xc005acae40, 0x4b460a0, 0xc00cb8ac60, 0x1, 0x4b13820, 0xc00c448a00, 0x0, 0x0, 0x100000001, 0x1, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/transport.go:196 +0x1d7 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*grpcTransport).SendNext(0xc005acae40, 0x4b460a0, 0xc00caa7200, 0x0, 0x0, 0x100000001, 0x1, 0x0, 0x4, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/transport.go:165 +0x383 | |
| github.com/cockroachdb/cockroach/pkg/kv.raceTransport.SendNext(0x4b50ce0, 0xc005acae40, 0x4b460a0, 0xc00caa7200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x4, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/transport_race.go:80 +0x483 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendToReplicas(0xc0027c7e60, 0x4b460a0, 0xc00caa7200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x4, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:1800 +0x652 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendRPC(0xc0027c7e60, 0x4b460a0, 0xc00caa7200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x4, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:428 +0x377 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendSingleRange(0xc0027c7e60, 0x4b460a0, 0xc00caa7200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:505 +0x29d | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendPartialBatch(0xc0027c7e60, 0x4b460a0, 0xc00caa7200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:1413 +0x415 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).divideAndSendBatchToRanges(0xc0027c7e60, 0x4b460a0, 0xc00caa7200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:1065 +0x1b17 | |
| github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).Send(0xc0027c7e60, 0x4b460a0, 0xc00cc9e090, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:717 +0x13c4 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*CrossRangeTxnWrapperSender).Send(0xc010a9d6e0, 0x4b460a0, 0xc00cc9e090, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:219 +0x1d2 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).sendUsingSender(0xc010a9d680, 0x4b460a0, 0xc00cc9e090, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:663 +0x1ae | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).send(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:645 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.sendAndFill(0x4b460a0, 0xc00cc9e090, 0xc0078dd758, 0xc00b3c4c00, 0x1, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:588 +0x18e | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Run(0xc010a9d680, 0x4b460a0, 0xc00cc9e090, 0xc00b3c4c00, 0x2d, 0xc00d26f3e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:611 +0xd2 | |
| github.com/cockroachdb/cockroach/pkg/ts.(*DB).storeKvs(0xc0071547e0, 0x4b460a0, 0xc00cc9e090, 0xc000c9a000, 0x287, 0x4cc, 0xc00d26f3e0, 0x32) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/ts/db.go:309 +0x2e0 | |
| github.com/cockroachdb/cockroach/pkg/ts.(*DB).tryStoreData(0xc0071547e0, 0x4b460a0, 0xc00cc9e090, 0x1, 0xc014124000, 0x287, 0x287, 0xc00d2ee8c0, 0x3f3b7d9) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/ts/db.go:247 +0x3b4 | |
| github.com/cockroachdb/cockroach/pkg/ts.(*DB).StoreData(0xc0071547e0, 0x4b460a0, 0xc00cc9e090, 0x1, 0xc014124000, 0x287, 0x287, 0x4b95540, 0xc007f9c000) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/ts/db.go:212 +0x297 | |
| github.com/cockroachdb/cockroach/pkg/ts.(*poller).poll.func1(0x4b460a0, 0xc000cdb8f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/ts/db.go:195 +0x1ac | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTask(0xc00d9a01e0, 0x4b460a0, 0xc000cdb8f0, 0x3f4c487, 0xf, 0xc010ea5e88, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:285 +0xf9 | |
| github.com/cockroachdb/cockroach/pkg/ts.(*poller).poll(0xc00e0841e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/ts/db.go:186 +0x1c5 | |
| github.com/cockroachdb/cockroach/pkg/ts.(*poller).start.func1(0x4b460a0, 0xc0065adc20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/ts/db.go:170 +0xc1 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160120, 0xc00d9a01e0, 0xc00d160110) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94203 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c7) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723560) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723560) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714f90, 0xc00d9a01e0, 0xc005714f70) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94176 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc00427e060) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:756 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be361d0, 0xc00d9a01e0, 0xc00cf217c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94251 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xfb) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cff20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cff20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715680, 0xc00d9a01e0, 0xc005715670) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94643 [runnable]: | |
| sync.(*RWMutex).RLock(0xc00b3c2218) | |
| /usr/local/go/src/sync/rwmutex.go:43 +0xa3 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).IsInitialized(0xc00b3c2000, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_init.go:250 +0x67 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).maybeAdd(0xc0014a7340, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba63731e, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:570 +0x108 | |
| github.com/cockroachdb/cockroach/pkg/storage.baseQueueHelper.MaybeAdd(0xc0014a7340, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba63731e, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:495 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).MaybeAddAsync.func1(0x4b460a0, 0xc002474030, 0x4b13da0, 0xc0014a7340) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:536 +0x8a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).Async.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:525 +0x66 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc002474030, 0xc005f70b60, 0x17, 0xc007f53e00, 0x0, 0x0, 0xc008e0d420) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:383 +0x11e | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:377 +0x2a9 | |
| goroutine 94082 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000cf) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf3b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf3b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714e30, 0xc00d9a01e0, 0xc005714e20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94197 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c4) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057234d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057234d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714e90, 0xc00d9a01e0, 0xc005714e80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94527 [runnable]: | |
| sync.(*RWMutex).RLock(0xc00b3c2218) | |
| /usr/local/go/src/sync/rwmutex.go:43 +0xa3 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Replica).IsInitialized(0xc00b3c2000, 0xc00e085300) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_init.go:250 +0x67 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).maybeAdd(0xc0014a62c0, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba5a84a1, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:570 +0x108 | |
| github.com/cockroachdb/cockroach/pkg/storage.baseQueueHelper.MaybeAdd(0xc0014a62c0, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba5a84a1, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:495 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).MaybeAddAsync.func1(0x4b460a0, 0xc002474030, 0x4b13da0, 0xc0014a62c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:536 +0x8a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).Async.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:525 +0x66 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc002474030, 0xc0061d9470, 0x10, 0xc007f535c0, 0x0, 0x0, 0xc008e0d3a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:383 +0x11e | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:377 +0x2a9 | |
| goroutine 94341 [chan receive, 3 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).execInternal(0xc00e8f03c0, 0x4b460a0, 0xc00c9967b0, 0x3f3e9fc, 0x9, 0x0, 0x1, 0x3f3542c, 0x4, 0xc00ca0c4e0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:567 +0xc5d | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).queryInternal(0xc00e8f03c0, 0x4b460a0, 0xc0098637a0, 0x3f3e9fc, 0x9, 0x0, 0x1, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:253 +0x140 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).Query(0xc00e8f03c0, 0x4b460a0, 0xc0098637a0, 0x3f3e9fc, 0x9, 0x0, 0x3ff334c, 0x64, 0xc00ad7f8a0, 0x2, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:224 +0x13b | |
| github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).maybeAdoptJob(0xc00d9a0aa0, 0x4b460a0, 0xc0098637a0, 0x4b10760, 0xc008b5b7a0, 0xc000060058, 0x30) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:607 +0x168 | |
| github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).Start.func3(0x4b460a0, 0xc0098637a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:324 +0x1c0 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00ae81510, 0xc00d9a01e0, 0xc00281fd40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94215 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d9) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf890) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf890) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715180, 0xc00d9a01e0, 0xc005715170) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94208 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d6) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf7a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf7a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715050, 0xc00d9a01e0, 0xc005715040) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94145 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc008b95500) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c4484a0, 0xc00d9a01e0, 0xc00c448490) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 93998 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/util/netutil.MakeServer.func2(0x4b460a0, 0xc0065ac540) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:97 +0x7e | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00af85000, 0xc00d9a01e0, 0xc00bae84a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94161 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux.muxListener.Accept(...) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:184 | |
| github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc.(*Server).Serve(0xc00a092000, 0x4b33760, 0xc0028fdfa0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/server.go:544 +0x2e5 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func7(0x4b460a0, 0xc0098622a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1221 +0xa7 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448f80, 0xc00d9a01e0, 0xc0093d8480) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94216 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000dd) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf8c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf8c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057151a0, 0xc00d9a01e0, 0xc005715190) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94146 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc0065ac120) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c4484c0, 0xc00d9a01e0, 0xc00c4484b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94525 [runnable]: | |
| sort.Search(0x26, 0xc001467a08, 0x0) | |
| /usr/local/go/src/sort/search.go:59 +0xab | |
| github.com/cockroachdb/cockroach/pkg/config.(*SystemConfig).GetLargestObjectID(0xc00e085320, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/config/system.go:189 +0x174 | |
| github.com/cockroachdb/cockroach/pkg/config.(*SystemConfig).ComputeSplitKey(0xc00e085320, 0xc00c416118, 0x1, 0x8, 0xc00c416120, 0x1, 0x8, 0xc0013f7e60, 0x1, 0x1) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/config/system.go:486 +0x2a1 | |
| github.com/cockroachdb/cockroach/pkg/config.(*SystemConfig).NeedsSplit(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/config/system.go:497 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).requiresSplit(0xc0014a6000, 0xc00e085320, 0x4b86ec0, 0xc00b3c2000, 0xc001013c50) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:616 +0x12a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).maybeAdd(0xc0014a6000, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba55cf0f, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:578 +0x7d2 | |
| github.com/cockroachdb/cockroach/pkg/storage.baseQueueHelper.MaybeAdd(0xc0014a6000, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba55cf0f, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:495 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).MaybeAddAsync.func1(0x4b460a0, 0xc002474030, 0x4b13da0, 0xc0014a6000) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:536 +0x8a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).Async.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:525 +0x66 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc002474030, 0xc0061d9450, 0xd, 0xc0018a30e0, 0x0, 0x0, 0xc008e0d360) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:383 +0x11e | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:377 +0x2a9 | |
| goroutine 94245 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000ee) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfda0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfda0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057155c0, 0xc00d9a01e0, 0xc0057155b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94269 [select]: | |
| github.com/cockroachdb/cockroach/pkg/server.(*Node).startComputePeriodicMetrics.func1(0x4b460a0, 0xc00afbecf0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:718 +0x17b | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715f10, 0xc00d9a01e0, 0xc00a77ec00) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94049 [IO wait, 5 minutes]: | |
| internal/poll.runtime_pollWait(0x7fdab9c9eb68, 0x72, 0x4ae4800) | |
| /usr/local/go/src/runtime/netpoll.go:182 +0x56 | |
| internal/poll.(*pollDesc).wait(0xc00ec2ff98, 0x72, 0x8001, 0x8000, 0xffffffffffffffff) | |
| /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe5 | |
| internal/poll.(*pollDesc).waitRead(...) | |
| /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 | |
| internal/poll.(*FD).Read(0xc00ec2ff80, 0xc00305a000, 0x8000, 0x8000, 0x0, 0x0, 0x0) | |
| /usr/local/go/src/internal/poll/fd_unix.go:169 +0x221 | |
| os.(*File).read(...) | |
| /usr/local/go/src/os/file_unix.go:263 | |
| os.(*File).Read(0xc00c620d88, 0xc00305a000, 0x8000, 0x8000, 0x0, 0x0, 0xc00aa8f060) | |
| /usr/local/go/src/os/file.go:108 +0xa8 | |
| io.copyBuffer(0x4ae28e0, 0xc00aa8f060, 0x4ae2740, 0xc00c620d88, 0xc00305a000, 0x8000, 0x8000, 0xc000a29f27, 0x1, 0x13d9fd3) | |
| /usr/local/go/src/io/io.go:402 +0x144 | |
| io.Copy(...) | |
| /usr/local/go/src/io/io.go:364 | |
| testing.runExample.func1(0xc00c620d88, 0xc00d97d3e0) | |
| /usr/local/go/src/testing/example.go:75 +0xce | |
| created by testing.runExample | |
| /usr/local/go/src/testing/example.go:73 +0x1c7 | |
| goroutine 94232 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e8) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057237a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057237a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715400, 0xc00d9a01e0, 0xc0057153f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94229 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e9) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfa70) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfa70) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057153a0, 0xc00d9a01e0, 0xc005715380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94155 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/util/netutil.MakeServer.func2(0x4b460a0, 0xc008b955c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:97 +0x7e | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c4489d0, 0xc00d9a01e0, 0xc0028fdf40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94228 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e3) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723740) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723740) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715370, 0xc00d9a01e0, 0xc005715360) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94171 [IO wait, 5 minutes]: | |
| internal/poll.runtime_pollWait(0x7fdab9c9e348, 0x72, 0x72) | |
| /usr/local/go/src/runtime/netpoll.go:182 +0x56 | |
| internal/poll.(*pollDesc).wait(0xc008906798, 0x72, 0x0, 0x0, 0x3f3a30a) | |
| /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe5 | |
| internal/poll.(*pollDesc).waitRead(...) | |
| /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 | |
| internal/poll.(*FD).Accept(0xc008906780, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /usr/local/go/src/internal/poll/fd_unix.go:384 +0x294 | |
| net.(*netFD).accept(0xc008906780, 0xc00e492a01, 0x0, 0x0) | |
| /usr/local/go/src/net/fd_unix.go:238 +0x56 | |
| net.(*TCPListener).accept(0xc00c621250, 0x965b15, 0x9690e5, 0xc003691e40) | |
| /usr/local/go/src/net/tcpsock_posix.go:139 +0x50 | |
| net.(*TCPListener).Accept(0xc00c621250, 0x40a5538, 0xc00c12cb80, 0x4b7b840, 0xc00e492ad0) | |
| /usr/local/go/src/net/tcpsock.go:260 +0x51 | |
| github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux.(*cMux).Serve(0xc00c12cb80, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:124 +0xcc | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func13.1() | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1433 +0x47 | |
| sync.(*Once).Do(0xc0089803c0, 0xc003833f40) | |
| /usr/local/go/src/sync/once.go:44 +0xdf | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func13(0x4b460a0, 0xc002a05f80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1432 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c4495c0, 0xc00d9a01e0, 0xc0093d9720) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94217 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000dc) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf8f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf8f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057151d0, 0xc00d9a01e0, 0xc0057151c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94135 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc008b953b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448330, 0xc00d9a01e0, 0xc00c448320) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94293 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*Server).Start.func1(0x4b460a0, 0xc002474090) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:323 +0x197 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160180, 0xc00d9a01e0, 0xc00a77ee00) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94169 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/gossip.(*Gossip).bootstrap.func1(0x4b460a0, 0xc0065ac960) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/gossip/gossip.go:1348 +0x5af | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c449590, 0xc00d9a01e0, 0xc00c449580) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94174 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc00427e030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:756 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be361b0, 0xc00d9a01e0, 0xc00cf21780) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94222 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e2) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf9b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf9b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715290, 0xc00d9a01e0, 0xc005715280) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94097 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/sql/sqlbase.(*PrivilegeDescriptor).Unmarshal(0xc0090690a0, 0xc0005de109, 0x15, 0x1a2e, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlbase/privilege.pb.go:304 +0xc45 | |
| github.com/cockroachdb/cockroach/pkg/sql/sqlbase.(*TableDescriptor).Unmarshal(0xc006526e00, 0xc0005de05e, 0x11a, 0x1ad9, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlbase/structured.pb.go:6740 +0x23a1 | |
| github.com/cockroachdb/cockroach/pkg/sql/sqlbase.(*Descriptor).Unmarshal(0xc00b2d9c20, 0xc0005de05b, 0x11d, 0x1adc, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlbase/structured.pb.go:8487 +0x2b7 | |
| github.com/cockroachdb/cockroach/pkg/util/protoutil.Unmarshal(0xc0005de05b, 0x11d, 0x1adc, 0x4b69300, 0xc00b2d9c20, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/protoutil/marshal.go:87 +0x75 | |
| github.com/cockroachdb/cockroach/pkg/roachpb.Value.GetProto(0xc0005de056, 0x122, 0x1ae1, 0x15a9187c86aa2b4f, 0x0, 0x0, 0x4b69300, 0xc00b2d9c20, 0x0, 0xc00b2d9c20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/roachpb/data.go:553 +0x140 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*KeyValue).ValueProto(0xc000921df8, 0x4b69300, 0xc00b2d9c20, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:113 +0x109 | |
| github.com/cockroachdb/cockroach/pkg/sql.GetAllDescriptors(0x4b460a0, 0xc00b9220c0, 0xc00c5de090, 0x7fdab244bf60, 0xc00ea8fe80, 0x8f24a5, 0x7fdab244bf60, 0x2) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/descriptor.go:213 +0x314 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*TableCollection).getAllDescriptors(0xc001b09840, 0x4b460a0, 0xc00b9220c0, 0xc00c5de090, 0x10, 0xc000000007, 0xc0002b1588, 0x7fdab9df2d01, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/table.go:590 +0x114 | |
| github.com/cockroachdb/cockroach/pkg/sql.glob..func52(0x4b460a0, 0xc00b9220c0, 0xc001b09958, 0xc00c3e8690, 0xc0089940f0, 0xc0089940a0, 0x2c60001) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/crdb_internal.go:218 +0xb9 | |
| github.com/cockroachdb/cockroach/pkg/sql.virtualDefEntry.getPlanInfo.func1(0x4b460a0, 0xc00b9220c0, 0xc001b09958, 0x3f38fa9, 0x6, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:271 +0x6ae | |
| github.com/cockroachdb/cockroach/pkg/sql.(*execFactory).ConstructVirtualScan.func1(0x4b460a0, 0xc00b9220c0, 0xc001b09958, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt_exec_factory.go:137 +0xa8 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*delayedNode).startExec(0xc0080eac80, 0x4b460a0, 0xc00b9220c0, 0xc00b2b4d80, 0xc001b09958, 0x9, 0xc000064aa0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/delayed.go:60 +0xa8 | |
| github.com/cockroachdb/cockroach/pkg/sql.startExec.func2(0x3f48094, 0xd, 0x4b472a0, 0xc0080eac80, 0x28, 0xc0046f0180) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan.go:486 +0x71 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal.func1(0xc001f24180, 0x3f48094, 0xd, 0x4b472a0, 0xc0080eac80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:144 +0x9a | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal(0xc001f24180, 0x4b472a0, 0xc0080eac80, 0x3f48094, 0xd) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:626 +0x562 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visit(0xc001f24180, 0x4b472a0, 0xc0080eac80, 0x968053, 0x6bbb7b8) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:111 +0xfa | |
| github.com/cockroachdb/cockroach/pkg/sql.walkPlan(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:75 | |
| github.com/cockroachdb/cockroach/pkg/sql.startExec(0x4b460a0, 0xc00b9220c0, 0xc00b2b4d80, 0xc001b09958, 0x4b472a0, 0xc0080eac80, 0x3d95300, 0x3e8ca60) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan.go:489 +0x2ec | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Start(0xc00d830500, 0x4b460a0, 0xc00b9220c0, 0xc00e0f8140, 0xc00e0f8128) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan_node_to_row_source.go:119 +0x1c9 | |
| github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*ProcessorBase).Run(0xc00d830500, 0x4b460a0, 0xc00b9220c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/processors.go:793 +0x8b | |
| github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*Flow).Run(0xc00e0f8000, 0x4b460a0, 0xc00b9220c0, 0x40a2b90, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/flow.go:624 +0x404 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run(0xc00932bcc0, 0xc0040642a0, 0xc00c5de090, 0xc00ea91f08, 0xc002d142c0, 0xc001b09a30, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:251 +0xcb7 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun(0xc00932bcc0, 0x4b460a0, 0xc00719fbc0, 0xc001b09a30, 0xc0040642a0, 0xc00c5de090, 0x4b47aa0, 0xc00ad82000, 0xc002d142c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:839 +0x25f | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine(0xc001b09600, 0x4b460a0, 0xc00719fbc0, 0xc001b09958, 0x3, 0x7fdab173f1e8, 0xc00e085ce0, 0xc00ea92200, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:903 +0x5d3 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine(0xc001b09600, 0x4b460a0, 0xc00719fbc0, 0xc001b09958, 0x7fdab173f1e8, 0xc00e085ce0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:738 +0xab0 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState(0xc001b09600, 0x4b460a0, 0xc00719fbc0, 0x4b4afa0, 0xc00ca84b80, 0xc00cd21501, 0x74, 0x0, 0x1, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:417 +0xb0e | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt(0xc001b09600, 0x4b460a0, 0xc00719fbc0, 0x4b4afa0, 0xc00ca84b80, 0xc00cd21501, 0x74, 0x0, 0x1, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:98 +0x6d7 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc001b09600, 0x4b45fe0, 0xc00ca11ec0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1205 +0x23ee | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc001b09600, 0x4b460a0, 0xc00cc77920, 0xc00d9a1900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1141 +0x3e8 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).initConnEx.func1(0xc001b09600, 0x4b460a0, 0xc00cc77920, 0xc00e8f03c0, 0xc00ad7e620, 0x0, 0xc00650b050) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:203 +0x8f | |
| created by github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).initConnEx | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:202 +0xa2e | |
| goroutine 94247 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f8) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfdd0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfdd0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715600, 0xc00d9a01e0, 0xc0057155f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94144 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc0065ac0f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448480, 0xc00d9a01e0, 0xc00c448470) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94158 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux.muxListener.Accept(...) | |
| /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:184 | |
| net/http.(*Server).Serve(0xc008ba68f0, 0x4b33760, 0xc0093d8280, 0x0, 0x0) | |
| /usr/local/go/src/net/http/server.go:2859 +0x356 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func4(0x4b460a0, 0xc0065ac3c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1198 +0x156 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448f50, 0xc00d9a01e0, 0xc0065a7380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94152 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*LeaseManager).RefreshLeases.func1(0x4b460a0, 0xc0065ac1e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/lease.go:1655 +0x4cd | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448920, 0xc00d9a01e0, 0xc0065a70e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94177 [chan send]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc00afbe3f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:775 +0x2a5 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be361e0, 0xc00d9a01e0, 0xc00cf217e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94199 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c6) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723500) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723500) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714ef0, 0xc00d9a01e0, 0xc005714ed0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94160 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func6(0x4b460a0, 0xc008b95620) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1209 +0xc1 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448f70, 0xc00d9a01e0, 0xc00c12cc40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94256 [select]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).coalescedHeartbeatsLoop(0xc001b73500, 0x4b460a0, 0xc005723ad0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:3879 +0x1ad | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715720, 0xc00d9a01e0, 0xc005715710) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94291 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*SchemaChangeManager).Start.func1(0x4b460a0, 0xc00afbede0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/schema_changer.go:1761 +0xbdc | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160130, 0xc00d9a01e0, 0xc00a77ede0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94272 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).refreshSettings.func2(0x4b460a0, 0xc0065adad0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/settingsworker.go:112 +0x190 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160070, 0xc00d9a01e0, 0xc00a77ece0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94149 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc008b95560) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448520, 0xc00d9a01e0, 0xc00c448510) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94295 [runnable]: | |
| time.AfterFunc(0xfadf2722, 0xc00afe8d20, 0xc0018f3390) | |
| /usr/local/go/src/time/sleep.go:158 +0x57 | |
| context.WithDeadline(0x4b460a0, 0xc000cdbbf0, 0xbf3a1e7c5f8c1691, 0x9ce2cbaa15, 0x6bb8920, 0x0, 0x0, 0x0) | |
| /usr/local/go/src/context/context.go:401 +0x4e4 | |
| context.WithTimeout(0x4b460a0, 0xc000cdbbf0, 0x10c388d00, 0xc000f9feb8, 0xc001fc2720, 0xc004752c88) | |
| /usr/local/go/src/context/context.go:451 +0x79 | |
| github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout(0x4b460a0, 0xc000cdbbf0, 0x3f5e9e6, 0x17, 0x10c388d00, 0xc000f9fe70, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:125 +0x74 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*NodeLiveness).StartHeartbeat.func1(0x4b460a0, 0xc000cdbad0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/node_liveness.go:466 +0x4e6 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160260, 0xc00d9a01e0, 0xc00e2ad5e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94210 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d2) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf7d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf7d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057150e0, 0xc00d9a01e0, 0xc0057150d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94219 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000de) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf950) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf950) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715210, 0xc00d9a01e0, 0xc005715200) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94239 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f1) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfd10) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfd10) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057154f0, 0xc00d9a01e0, 0xc0057154e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94528 [runnable]: | |
| sync.(*RWMutex).RUnlock(0xc00add97b8) | |
| /usr/local/go/src/sync/rwmutex.go:62 +0xee | |
| github.com/cockroachdb/cockroach/pkg/gossip.(*Gossip).GetSystemConfig(0xc00add96c0, 0xc00e085320) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/gossip/gossip.go:1140 +0xa1 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).maybeAdd(0xc0014a6420, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba5c9833, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:553 +0x89d | |
| github.com/cockroachdb/cockroach/pkg/storage.baseQueueHelper.MaybeAdd(0xc0014a6420, 0x4b460a0, 0xc002474030, 0x4b86ec0, 0xc00b3c2000, 0x15a918c7ba5c9833, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:495 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).MaybeAddAsync.func1(0x4b460a0, 0xc002474030, 0x4b13da0, 0xc0014a6420) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:536 +0x8a | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).Async.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:525 +0x66 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc002474030, 0xc005f70b00, 0x14, 0xc007f536e0, 0x0, 0x0, 0xc008e0d3c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:383 +0x11e | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunLimitedAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:377 +0x2a9 | |
| goroutine 94342 [chan receive, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).execInternal(0xc00e8f03c0, 0x4b460a0, 0xc00cc77920, 0x3f41459, 0xa, 0x0, 0x1, 0x3f3542c, 0x4, 0xc00cc778c0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:567 +0xc5d | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).queryInternal(0xc00e8f03c0, 0x4b460a0, 0xc00389bce0, 0x3f41459, 0xa, 0x0, 0x1, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:253 +0x140 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).Query(0xc00e8f03c0, 0x4b460a0, 0xc00389bce0, 0x3f41459, 0xa, 0x0, 0xc00cd21500, 0x76, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:224 +0x13b | |
| github.com/cockroachdb/cockroach/pkg/sql/stats.(*Refresher).ensureAllTables(0xc00d574550, 0x4b460a0, 0xc00389bce0, 0xc003eb0c80, 0x3b9aca00) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/stats/automatic_stats.go:319 +0x1d5 | |
| github.com/cockroachdb/cockroach/pkg/sql/stats.(*Refresher).Start.func1(0x4b460a0, 0xc00389bce0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/stats/automatic_stats.go:249 +0x85b | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00ae81520, 0xc00d9a01e0, 0xc00c6ea1a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94273 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).startSampleEnvironment.func1(0x4b460a0, 0xc00afbedb0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1963 +0x743 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160100, 0xc00d9a01e0, 0xc000cdb8c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94136 [select, 5 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc00d103ef0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448350, 0xc00d9a01e0, 0xc00c448340) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94224 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000db) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf9e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf9e0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057152d0, 0xc00d9a01e0, 0xc0057152c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94557 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/util/tracing.(*span).Finish(0xc007b36540) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/tracing/tracer_span.go:302 +0xdf | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processReplica(0xc0014a6420, 0x4b460a0, 0xc00cc9f500, 0x4b86ec0, 0xc00b3c3080, 0x4ae0f40, 0xc00cb58ba0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:860 +0x313 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1.2(0x4b460a0, 0xc00ca0d2f0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:787 +0xfa | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d9a01e0, 0x4b460a0, 0xc00ca0d2f0, 0xc00b401080, 0x2d, 0x0, 0x0, 0xc00ca0d320) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:323 +0xf4 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:318 +0x14b | |
| goroutine 94487 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/sql/sqlbase.(*TableDescriptor).Unmarshal(0xc0033f5180, 0xc00031c3df, 0x1ba, 0x1ba, 0xc000453500, 0xc00378b130) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlbase/structured.pb.go:6687 +0x2f6e | |
| github.com/cockroachdb/cockroach/pkg/sql/sqlbase.(*Descriptor).Unmarshal(0xc00118e460, 0xc00031c3dc, 0x1bd, 0x1bd, 0x20, 0xc013244c00) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlbase/structured.pb.go:8487 +0x2b7 | |
| github.com/cockroachdb/cockroach/pkg/util/protoutil.Unmarshal(0xc00031c3dc, 0x1bd, 0x1bd, 0x4b69300, 0xc00118e460, 0xc00ca58780, 0xc00902c860) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/protoutil/marshal.go:87 +0x75 | |
| github.com/cockroachdb/cockroach/pkg/roachpb.Value.GetProto(0xc00031c3d7, 0x1c2, 0x1c2, 0x15a9187c86aa2b4f, 0x0, 0x0, 0x4b69300, 0xc00118e460, 0x0, 0xc00378b4d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/roachpb/data.go:553 +0x140 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*KeyValue).ValueProto(0xc000e25f40, 0x4b69300, 0xc00118e460, 0x3d93700, 0xc00118e480) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:113 +0x109 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*Txn).GetProto(0xc00651a000, 0x4b460a0, 0xc013244c00, 0x3d93700, 0xc00118e480, 0x4b69300, 0xc00118e460, 0x1, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/txn.go:323 +0x126 | |
| github.com/cockroachdb/cockroach/pkg/sql.getDescriptorByID(0x4b460a0, 0xc013244c00, 0xc00651a000, 0xf, 0x4b953c0, 0xc000e267a8, 0x1, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/descriptor.go:169 +0x1dc | |
| github.com/cockroachdb/cockroach/pkg/sql.UncachedPhysicalAccessor.GetObjectDesc(0x4b460a0, 0xc013244c00, 0xc00651a000, 0xc001b0b9e8, 0xc004000000, 0xc00e678b58, 0x1989755, 0xc00e678b58, 0x8f31e8) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/physical_schema_accessors.go:162 +0x444 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*TableCollection).getTableVersion.func1(0xc0088e09e8, 0x3, 0xc001b0b9e8) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/table.go:254 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/sql.(*TableCollection).getTableVersion(0xc001b0b340, 0x4b460a0, 0xc013244c00, 0xc00651a000, 0xc001b0b9e8, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/table.go:262 +0xf83 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*CachedPhysicalAccessor).GetObjectDesc(0xc008a05040, 0x4b460a0, 0xc013244c00, 0xc00651a000, 0xc001b0b9e8, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/physical_schema_accessors.go:235 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/sql.(*LogicalSchemaAccessor).GetObjectDesc(0xc0093460c0, 0x4b460a0, 0xc013244c00, 0xc00651a000, 0xc001b0b9e8, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/logical_schema_accessors.go:95 +0x837 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planner).LookupObject(0xc001b0b458, 0x4b460a0, 0xc013244c00, 0x0, 0x3ff3376, 0x6, 0x3f38c07, 0x6, 0x3ff337d, 0x4, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/resolver.go:285 +0x2ab | |
| github.com/cockroachdb/cockroach/pkg/sql/sem/tree.(*TableName).ResolveExisting(0xc001b0bc00, 0x4b460a0, 0xc013244c00, 0x7fdab82d4930, 0xc001b0b458, 0x0, 0x3f38fa9, 0x6, 0xc00cb977c0, 0x1, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sem/tree/name_resolution.go:298 +0x90f | |
| github.com/cockroachdb/cockroach/pkg/sql.resolveExistingObjectImpl(0x4b460a0, 0xc013244c00, 0x4b7f2c0, 0xc001b0b458, 0xc001b0bc00, 0xc00e670001, 0x0, 0x7fdab2434b00, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/resolver.go:137 +0x172 | |
| github.com/cockroachdb/cockroach/pkg/sql.ResolveExistingObject(0x4b460a0, 0xc013244c00, 0x4b7f2c0, 0xc001b0b458, 0xc001b0bc00, 0x3b92201, 0x0, 0x89, 0x7fda00000064, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/resolver.go:101 +0x92 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*optCatalog).ResolveDataSource(0xc001b0bbe8, 0x4b460a0, 0xc013244c00, 0xc009fc0000, 0xc0080ea800, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt_catalog.go:169 +0x147 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).resolveDataSource(0xc007f9c700, 0xc0080ea800, 0x5, 0xc00e679f98, 0xa2bd36, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/util.go:444 +0xf3 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildDataSource(0xc007f9c700, 0x4b103a0, 0xc0080ea800, 0x0, 0xc00a79a900, 0xc00e67a1d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/select.go:89 +0x688 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildDataSource(0xc007f9c700, 0x4b0fda0, 0xc00d4cd2c0, 0x0, 0xc00a79a900, 0xc00b9ae325) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/select.go:56 +0x3ec | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildFromTablesRightDeep(0xc007f9c700, 0xc0008a5b40, 0x1, 0x1, 0xc00a79a900, 0xc0018f3c80) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/select.go:833 +0xa2 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildFromTables(0xc007f9c700, 0xc0008a5b40, 0x1, 0x1, 0xc00a79a900, 0xc00e67a500) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/select.go:810 +0x191 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildFrom(0xc007f9c700, 0xc00cb8a5a0, 0xc00a79a900, 0x1) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/select.go:734 +0xcb | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildSelectClause(0xc007f9c700, 0xc00c93a360, 0xc00ab98e98, 0x1, 0x1, 0x0, 0x0, 0x0, 0xc00a79a900, 0xc00e67a728) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/select.go:661 +0x89 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildSelect(0xc007f9c700, 0xc0099c8ec0, 0x0, 0x0, 0x0, 0xc00a79a900, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/select.go:615 +0x47f | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildStmt(0xc007f9c700, 0x4b4afa0, 0xc0099c8ec0, 0x0, 0x0, 0x0, 0xc00a79a900, 0xc007f9c700) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:215 +0x367 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).Build(0xc007f9c700, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:156 +0x251 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*optPlanningCtx).buildReusableMemo(0xc001b0bbe0, 0x4b460a0, 0xc013244c00, 0xc00ab949a0, 0x61, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:287 +0x394 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planner).prepareUsingOptimizer(0xc001b0b458, 0x4b460a0, 0xc013244c00, 0x1, 0x2, 0x3f64114) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:88 +0x1cd | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).populatePrepared(0xc001b0b100, 0x4b460a0, 0xc013244c00, 0xc00651a000, 0xc00a3ec8e0, 0x2, 0x2, 0xc001b0b458, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:211 +0x28a | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).prepare(0xc001b0b100, 0x4b460a0, 0xc013244c00, 0x4b4afa0, 0xc0099c8ec0, 0x3ff334c, 0x64, 0x2, 0x1, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:159 +0x67e | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).addPreparedStmt(0xc001b0b100, 0x4b460a0, 0xc013244c00, 0x0, 0x0, 0x4b4afa0, 0xc0099c8ec0, 0x3ff334c, 0x64, 0x2, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:98 +0x14c | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execPrepare(0xc001b0b100, 0x4b460a0, 0xc013244c00, 0x0, 0x0, 0x4b4afa0, 0xc0099c8ec0, 0x3ff334c, 0x64, 0x2, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:52 +0x1bf | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc001b0b100, 0x4b460a0, 0xc00c9967b0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1272 +0x2dd7 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc001b0b100, 0x4b460a0, 0xc00c9967b0, 0xc00d9a1900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1141 +0x3e8 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).initConnEx.func1(0xc001b0b100, 0x4b460a0, 0xc00c9967b0, 0xc00e8f03c0, 0xc009383620, 0x0, 0xc0073b0330) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:203 +0x8f | |
| created by github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).initConnEx | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:202 +0xa2e | |
| goroutine 94235 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000bf) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfcb0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfcb0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715470, 0xc00d9a01e0, 0xc005715460) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94212 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d0) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf800) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf800) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715120, 0xc00d9a01e0, 0xc005715110) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94248 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e6) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723950) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723950) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715620, 0xc00d9a01e0, 0xc005715610) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94279 [semacquire]: | |
| sync.runtime_SemacquireMutex(0x6bbb798, 0x1900000000) | |
| /usr/local/go/src/runtime/sema.go:71 +0x3d | |
| sync.(*Mutex).Lock(0x6bbb794) | |
| /usr/local/go/src/sync/mutex.go:134 +0x1e2 | |
| github.com/cockroachdb/cockroach/pkg/util/syncutil.(*Mutex).Lock(0x6bbb794) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/syncutil/mutex_sync_race.go:31 +0x3b | |
| github.com/cockroachdb/cockroach/pkg/util/log.VDepth(0xc000000006, 0x1, 0x3) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1477 +0x12f | |
| github.com/cockroachdb/cockroach/pkg/util/log.V(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:210 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*replicaScanner).waitAndProcess(0xc00c8428f0, 0x4b460a0, 0xc002474030, 0xc00d9a01e0, 0xe1432c3, 0xed499f825, 0x0, 0xc00b3c3b80, 0x358ec60) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scanner.go:221 +0x2bb | |
| github.com/cockroachdb/cockroach/pkg/storage.(*replicaScanner).scanLoop.func1.1(0xc00b3c3b80, 0x78) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scanner.go:281 +0x13c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*storeReplicaVisitor).Visit(0xc00be758f0, 0xc00c12d2c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:356 +0x321 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*replicaScanner).scanLoop.func1(0x4b460a0, 0xc002474030) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scanner.go:279 +0x38b | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be36240, 0xc00d9a01e0, 0xc00cf218a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94227 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e7) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cfa10) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cfa10) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715350, 0xc00d9a01e0, 0xc005715330) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94150 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).gcDaemon(0xc00e8e4b40, 0x4b45fe0, 0xc00c12c9c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1288 +0x13d | |
| created by github.com/cockroachdb/cockroach/pkg/util/log.NewSecondaryLogger | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/log/secondary_log.go:81 +0x4c8 | |
| goroutine 94266 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage/compactor.(*Compactor).Start.func1.1(0x4b460a0, 0xc0098625d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/compactor/compactor.go:125 +0x269 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715820, 0xc00d9a01e0, 0xc00a77e400) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94141 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc008b95470) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448420, 0xc00d9a01e0, 0xc00c448410) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94075 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher.(*RequestBatcher).run(0xc00365c1c0, 0x4b46020, 0xc000052118) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher/batcher.go:408 +0x40a | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d9a01e0, 0x4b46020, 0xc000052118, 0xc0073f6db0, 0x22, 0x0, 0x0, 0xc0057145d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:323 +0xf4 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:318 +0x14b | |
| goroutine 94276 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc00afbe450) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:756 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be36210, 0xc00d9a01e0, 0xc00cf21840) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94254 [semacquire, 6 minutes]: | |
| sync.runtime_Semacquire(0xc0056b8110) | |
| /usr/local/go/src/runtime/sema.go:56 +0x39 | |
| sync.(*WaitGroup).Wait(0xc0056b8108) | |
| /usr/local/go/src/sync/waitgroup.go:130 +0xb2 | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Wait(...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:169 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057156e0, 0xc00d9a01e0, 0xc0057156d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94271 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/server.periodicallyPersistHLCUpperBound(0xc00e184370, 0xc00903a7e0, 0xc00d160050, 0x40a80b8, 0xc00d97d4a0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:990 +0x1f8 | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).startPersistingHLCUpperBound.func2(0x4b460a0, 0xc00afbed20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1064 +0xde | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160060, 0xc00d9a01e0, 0xc000cdb860) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94195 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c3) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0057234a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0057234a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714e50, 0xc00d9a01e0, 0xc005714e40) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94236 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000f5) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723800) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723800) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715490, 0xc00d9a01e0, 0xc005715480) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94077 [chan receive, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func1(0x4b460a0, 0xc005723410) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:153 +0x64 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714d80, 0xc00d9a01e0, 0xc00a77e340) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94277 [select, 3 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x4b460a0, 0xc00afbe480) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:756 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00be36220, 0xc00d9a01e0, 0xc00cf21860) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94204 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000d5) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf740) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf740) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005714fb0, 0xc00d9a01e0, 0xc005714fa0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94153 [select, 3 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*LeaseManager).PeriodicallyRefreshSomeLeases.func1(0x4b460a0, 0xc008b95590) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/lease.go:1730 +0x327 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c448940, 0xc00d9a01e0, 0xc00c448930) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94459 [runnable]: | |
| github.com/cockroachdb/cockroach/pkg/sql/sqlbase.(*TableDescriptor).Unmarshal(0xc001ad8000, 0xc0006e401f, 0x170, 0x170, 0x203002, 0xc00b3c5230) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlbase/structured.pb.go:6738 +0x241a | |
| github.com/cockroachdb/cockroach/pkg/sql/sqlbase.(*Descriptor).Unmarshal(0xc00b2d9780, 0xc0006e401c, 0x173, 0x173, 0x20, 0xc0088e08a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlbase/structured.pb.go:8487 +0x2b7 | |
| github.com/cockroachdb/cockroach/pkg/util/protoutil.Unmarshal(0xc0006e401c, 0x173, 0x173, 0x4b69300, 0xc00b2d9780, 0xc00ca0dd10, 0xc00650b1a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/protoutil/marshal.go:87 +0x75 | |
| github.com/cockroachdb/cockroach/pkg/roachpb.Value.GetProto(0xc0006e4017, 0x178, 0x178, 0x15a9187c86aa2b4f, 0x0, 0x0, 0x4b69300, 0xc00b2d9780, 0x0, 0xc00b3c55d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/roachpb/data.go:553 +0x140 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*KeyValue).ValueProto(0xc0015f2590, 0x4b69300, 0xc00b2d9780, 0x3d93700, 0xc00b2d97a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:113 +0x109 | |
| github.com/cockroachdb/cockroach/pkg/internal/client.(*Txn).GetProto(0xc008c5a750, 0x4b460a0, 0xc0088e08a0, 0x3d93700, 0xc00b2d97a0, 0x4b69300, 0xc00b2d9780, 0x1, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/internal/client/txn.go:323 +0x126 | |
| github.com/cockroachdb/cockroach/pkg/sql.getDescriptorByID(0x4b460a0, 0xc0088e08a0, 0xc008c5a750, 0x6, 0x4b953c0, 0xc0015f2df8, 0x1, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/descriptor.go:169 +0x1dc | |
| github.com/cockroachdb/cockroach/pkg/sql.UncachedPhysicalAccessor.GetObjectDesc(0x4b460a0, 0xc0088e08a0, 0xc008c5a750, 0xc00ef09ee8, 0xc006000000, 0xc010cf91a8, 0x1989755, 0xc010cf91a8, 0x8f31e8) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/physical_schema_accessors.go:162 +0x444 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*TableCollection).getTableVersion.func1(0xc00ca0d408, 0x3, 0xc00ef09ee8) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/table.go:254 +0x7b | |
| github.com/cockroachdb/cockroach/pkg/sql.(*TableCollection).getTableVersion(0xc00ef09840, 0x4b460a0, 0xc0088e08a0, 0xc008c5a750, 0xc00ef09ee8, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/table.go:262 +0xf83 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*CachedPhysicalAccessor).GetObjectDesc(0xc005ce8f00, 0x4b460a0, 0xc0088e08a0, 0xc008c5a750, 0xc00ef09ee8, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/physical_schema_accessors.go:235 +0x1ef | |
| github.com/cockroachdb/cockroach/pkg/sql.(*LogicalSchemaAccessor).GetObjectDesc(0xc005ce8f40, 0x4b460a0, 0xc0088e08a0, 0xc008c5a750, 0xc00ef09ee8, 0xc007000000, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/logical_schema_accessors.go:95 +0x837 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planner).LookupObject(0xc00ef09958, 0x4b460a0, 0xc0088e08a0, 0x8f0f00, 0x3ff22f6, 0x6, 0x3f38c07, 0x6, 0x3ff22fd, 0x8, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/resolver.go:285 +0x2ab | |
| github.com/cockroachdb/cockroach/pkg/sql/sem/tree.(*TableName).ResolveExisting(0xc00ef0a100, 0x4b460a0, 0xc0088e08a0, 0x7fdab82d4930, 0xc00ef09958, 0x3b92200, 0x3f38fa9, 0x6, 0xc00b5c6520, 0x1, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/sem/tree/name_resolution.go:298 +0x90f | |
| github.com/cockroachdb/cockroach/pkg/sql.resolveExistingObjectImpl(0x4b460a0, 0xc0088e08a0, 0x4b7f2c0, 0xc00ef09958, 0xc00ef0a100, 0x1, 0x0, 0x0, 0x0, 0x2, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/resolver.go:137 +0x172 | |
| github.com/cockroachdb/cockroach/pkg/sql.ResolveExistingObject(0x4b460a0, 0xc0088e08a0, 0x4b7f2c0, 0xc00ef09958, 0xc00ef0a100, 0x1, 0x0, 0x0, 0x0, 0xc007c1d068) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/resolver.go:101 +0x92 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*optCatalog).ResolveDataSource(0xc00ef0a0e8, 0x4b460a0, 0xc0088e08a0, 0x0, 0xc0080eb080, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt_catalog.go:169 +0x147 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).resolveDataSource(0xc00ac06620, 0xc0080eb080, 0x6, 0xc00afe9150, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/util.go:444 +0xf3 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).resolveTable(0xc00ac06620, 0xc0080eb080, 0xc000000006, 0x0, 0xc010cfa6a8, 0x0, 0x0, 0x0, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/util.go:430 +0x90 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildInsert(0xc00ac06620, 0xc002a52870, 0xc00a46e000, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/insert.go:168 +0xe7 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildStmt(0xc00ac06620, 0x4b4a9a0, 0xc002a52870, 0x0, 0x0, 0x0, 0xc00a46e000, 0xc00ac06620) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:209 +0x6f0 | |
| github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).Build(0xc00ac06620, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:156 +0x251 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*optPlanningCtx).buildReusableMemo(0xc00ef0a0e0, 0x4b460a0, 0xc0088e08a0, 0xc0134613e0, 0x5f, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:287 +0x394 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*planner).prepareUsingOptimizer(0xc00ef09958, 0x4b460a0, 0xc0088e08a0, 0x1, 0x2, 0x3f64114) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:88 +0x1cd | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).populatePrepared(0xc00ef09600, 0x4b460a0, 0xc0088e08a0, 0xc008c5a750, 0xc00a2f24e0, 0x3, 0x3, 0xc00ef09958, 0x0, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:211 +0x28a | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).prepare(0xc00ef09600, 0x4b460a0, 0xc0088e08a0, 0x4b4a9a0, 0xc002a52870, 0x3ff22ea, 0x60, 0x3, 0x1, 0x0, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:159 +0x67e | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).addPreparedStmt(0xc00ef09600, 0x4b460a0, 0xc0088e08a0, 0x0, 0x0, 0x4b4a9a0, 0xc002a52870, 0x3ff22ea, 0x60, 0x3, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:98 +0x14c | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execPrepare(0xc00ef09600, 0x4b460a0, 0xc0088e08a0, 0x0, 0x0, 0x4b4a9a0, 0xc002a52870, 0x3ff22ea, 0x60, 0x3, ...) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:52 +0x1bf | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc00ef09600, 0x4b45fe0, 0xc0085c4100, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1272 +0x2dd7 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc00ef09600, 0x4b460a0, 0xc00d8af530, 0xc00d9a1900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1141 +0x3e8 | |
| github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).initConnEx.func1(0xc00ef09600, 0x4b460a0, 0xc00d8af530, 0xc00e8f03c0, 0xc005ce8e00, 0xc00af3a3f0, 0xc009e02090) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:203 +0x8f | |
| created by github.com/cockroachdb/cockroach/pkg/sql.(*internalExecutorImpl).initConnEx | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:202 +0xa2e | |
| goroutine 94221 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e0) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723680) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723680) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715270, 0xc00d9a01e0, 0xc005715260) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94206 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000c9) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc0087cf770) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc0087cf770) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715000, 0xc00d9a01e0, 0xc005714fe0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94264 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/storage.(*Store).startClosedTimestampRangefeedSubscriber.func2(0x4b460a0, 0xc0093e1380) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1622 +0x3c8 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc005715800, 0xc00d9a01e0, 0xc00a77e3a0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94230 [sync.Cond.Wait]: | |
| runtime.goparkunlock(...) | |
| /usr/local/go/src/runtime/proc.go:307 | |
| sync.runtime_notifyListWait(0xc0044b2690, 0xc0000000e4) | |
| /usr/local/go/src/runtime/sema.go:510 +0xf9 | |
| sync.(*Cond).Wait(0xc0044b2680) | |
| /usr/local/go/src/sync/cond.go:56 +0x8e | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc0056b8090, 0x4b460a0, 0xc005723770) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:194 +0x9c | |
| github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x4b460a0, 0xc005723770) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:163 +0x56 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc0057153c0, 0xc00d9a01e0, 0xc0057153b0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94165 [chan receive, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func12(0x4b460a0, 0xc009862510) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1329 +0x98 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c449420, 0xc00d9a01e0, 0xc0093d8760) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94147 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc008b95530) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c4484e0, 0xc00d9a01e0, 0xc00c4484d0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94292 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*flowScheduler).Start.func1(0x4b460a0, 0xc009862a20) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/flow_scheduler.go:149 +0x1d6 | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d160170, 0xc00d9a01e0, 0xc00d160160) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| goroutine 94139 [select, 6 minutes]: | |
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x4b460a0, 0xc008b95410) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:94 +0x1be | |
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00c4483d0, 0xc00d9a01e0, 0xc00c4483c0) | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:198 +0x160 | |
| created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker | |
| /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:191 +0xc4 | |
| **************************************************************************** | |
| This node experienced a fatal error (printed above), and as a result the | |
| process is terminating. | |
| Fatal errors can occur due to faulty hardware (disks, memory, clocks) or a | |
| problem in CockroachDB. With your help, the support team at Cockroach Labs | |
| will try to determine the root cause, recommend next steps, and we can | |
| improve CockroachDB based on your report. | |
| Please submit a crash report by following the instructions here: | |
| https://github.com/cockroachdb/cockroach/issues/new/choose | |
| If you would rather not post publicly, please contact us directly at: | |
| [email protected] | |
| The Cockroach Labs team appreciates your feedback. | |
| FAIL github.com/cockroachdb/cockroach/pkg/cli 704.674s |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment