Skip to content

Instantly share code, notes, and snippets.

@imjching

imjching/gist:dec1c61c505f59bb875bdf6d6c77829e

Created February 23, 2022 00:50
Show Gist options
  • Save imjching/dec1c61c505f59bb875bdf6d6c77829e to your computer and use it in GitHub Desktop.
Save imjching/dec1c61c505f59bb875bdf6d6c77829e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
[jay@archome] cockroach (master)$ go test -test.v -exec 'stress -p=1' -v ./pkg/ccl/sqlproxyccl/ -run TestDirectoryConnect -timeout 0
0 runs so far, 0 failures, over 5s
1 runs so far, 0 failures, over 10s
1 runs so far, 0 failures, over 15s
2 runs so far, 0 failures, over 20s
3 runs so far, 0 failures, over 25s
3 runs so far, 0 failures, over 30s
4 runs so far, 0 failures, over 35s
4 runs so far, 0 failures, over 40s
5 runs so far, 0 failures, over 45s
6 runs so far, 0 failures, over 50s
6 runs so far, 0 failures, over 55s
7 runs so far, 0 failures, over 1m0s
7 runs so far, 0 failures, over 1m5s
8 runs so far, 0 failures, over 1m10s
9 runs so far, 0 failures, over 1m15s
9 runs so far, 0 failures, over 1m20s
10 runs so far, 0 failures, over 1m25s
10 runs so far, 0 failures, over 1m30s
11 runs so far, 0 failures, over 1m35s
12 runs so far, 0 failures, over 1m40s
12 runs so far, 0 failures, over 1m45s
13 runs so far, 0 failures, over 1m50s
14 runs so far, 0 failures, over 1m55s
14 runs so far, 0 failures, over 2m0s
15 runs so far, 0 failures, over 2m5s
15 runs so far, 0 failures, over 2m10s
16 runs so far, 0 failures, over 2m15s
17 runs so far, 0 failures, over 2m20s
17 runs so far, 0 failures, over 2m25s
18 runs so far, 0 failures, over 2m30s
18 runs so far, 0 failures, over 2m35s
19 runs so far, 0 failures, over 2m40s
20 runs so far, 0 failures, over 2m45s
20 runs so far, 0 failures, over 2m50s
21 runs so far, 0 failures, over 2m55s
21 runs so far, 0 failures, over 3m0s
22 runs so far, 0 failures, over 3m5s
23 runs so far, 0 failures, over 3m10s
23 runs so far, 0 failures, over 3m15s
24 runs so far, 0 failures, over 3m20s
25 runs so far, 0 failures, over 3m25s
25 runs so far, 0 failures, over 3m30s
26 runs so far, 0 failures, over 3m35s
26 runs so far, 0 failures, over 3m40s
27 runs so far, 0 failures, over 3m45s
28 runs so far, 0 failures, over 3m50s
28 runs so far, 0 failures, over 3m55s
29 runs so far, 0 failures, over 4m0s
29 runs so far, 0 failures, over 4m5s
30 runs so far, 0 failures, over 4m10s
31 runs so far, 0 failures, over 4m15s
31 runs so far, 0 failures, over 4m20s
32 runs so far, 0 failures, over 4m25s
33 runs so far, 0 failures, over 4m30s
33 runs so far, 0 failures, over 4m35s
34 runs so far, 0 failures, over 4m40s
34 runs so far, 0 failures, over 4m45s
35 runs so far, 0 failures, over 4m50s
36 runs so far, 0 failures, over 4m55s
36 runs so far, 0 failures, over 5m0s
37 runs so far, 0 failures, over 5m5s
37 runs so far, 0 failures, over 5m10s
38 runs so far, 0 failures, over 5m15s
39 runs so far, 0 failures, over 5m20s
39 runs so far, 0 failures, over 5m25s
40 runs so far, 0 failures, over 5m30s
40 runs so far, 0 failures, over 5m35s
41 runs so far, 0 failures, over 5m40s
42 runs so far, 0 failures, over 5m45s
42 runs so far, 0 failures, over 5m50s
43 runs so far, 0 failures, over 5m55s
43 runs so far, 0 failures, over 6m0s
44 runs so far, 0 failures, over 6m5s
45 runs so far, 0 failures, over 6m10s
45 runs so far, 0 failures, over 6m15s
46 runs so far, 0 failures, over 6m20s
47 runs so far, 0 failures, over 6m25s
47 runs so far, 0 failures, over 6m30s
48 runs so far, 0 failures, over 6m35s
48 runs so far, 0 failures, over 6m40s
49 runs so far, 0 failures, over 6m45s
50 runs so far, 0 failures, over 6m50s
50 runs so far, 0 failures, over 6m55s
51 runs so far, 0 failures, over 7m0s
51 runs so far, 0 failures, over 7m5s
52 runs so far, 0 failures, over 7m10s
53 runs so far, 0 failures, over 7m15s
53 runs so far, 0 failures, over 7m20s
54 runs so far, 0 failures, over 7m25s
55 runs so far, 0 failures, over 7m30s
55 runs so far, 0 failures, over 7m35s
56 runs so far, 0 failures, over 7m40s
^C56 runs completed, 0 failures, over 7m43s
context canceled
FAIL
FAIL github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl 463.322s
@imjching
Copy link
Author 

[jay@archome] cockroach (master)$ echo $PATH
/home/jay/.nvm/versions/node/v12.22.7/bin:/home/jay/.go/1.17.3/bin:/home/jay/workspace/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
[jay@archome] cockroach (master)$ ./dev doctor --debug
DEBUG: getenv DEV_NO_REMOTE_CACHE
DEBUG: doctor: running xcode check
DEBUG: doctor: running cmake check
DEBUG: cmake --version
DEBUG: doctor: running submodules check
DEBUG: bazel build //build/bazelutil:test_stamping
DEBUG: bazel info bazel-bin --color=no
DEBUG: cat /home/jay/.cache/bazel/_bazel_jay/ad30d6d5ac3ec70478dfe5900736ea48/execroot/cockroach/bazel-out/k8-fastbuild/bin/build/bazelutil/test_stamping.txt
DEBUG: doctor: setting up cache
Configuring cache...
DEBUG: bazel run @com_github_buchgr_bazel_remote//:bazel-remote --run_under=//build/bazelutil/whereis
ERROR: exit status 1

@imjching
Copy link
Author 

./dev test ./pkg/ccl/sqlproxyccl --cpus 30 -f=TestDirectoryConnect -v --stress --test-args='-test.timeout=10s'
$ bazel test --local_cpu_resources=30 --test_sharding_strategy=disabled pkg/ccl/sqlproxyccl:all --test_env=GOTRACEBACK=all --test_timeout=86400 --run_under '@com_github_cockroachdb_stress//:stress -bazel -shardable-artifacts '"'"'XML_OUTPUT_FILE=/home/jay/workspace/src/github.com/cockroachdb/cockroach/bin/dev-versions/dev.17 merge-test-xmls'"'"' -p=30' --test_filter=TestDirectoryConnect --test_sharding_strategy=disabled --test_arg -test.v --test_arg -test.outputdir=/home/jay/workspace/src/github.com/cockroachdb/cockroach --sandbox_writable_path=/home/jay/workspace/src/github.com/cockroachdb/cockroach --test_arg -test.timeout=10s --test_output streamed
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 5f9698ce-d22f-4cd5-9718-5a70059f139d
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
INFO: Analyzed 3 targets (1251 packages loaded, 20818 targets configured).
INFO: Found 2 targets and 1 test target...
WARNING: Remote Cache: Connection refused: /127.0.0.1:9867
0 runs so far, 0 failures, over 5s
30 runs so far, 0 failures, over 10s
30 runs so far, 0 failures, over 15s
60 runs so far, 0 failures, over 20s
76 runs so far, 0 failures, over 25s
90 runs so far, 0 failures, over 30s
120 runs so far, 0 failures, over 35s
120 runs so far, 0 failures, over 40s
150 runs so far, 0 failures, over 45s
179 runs so far, 0 failures, over 50s
180 runs so far, 0 failures, over 55s
210 runs so far, 0 failures, over 1m0s
211 runs so far, 0 failures, over 1m5s
240 runs so far, 0 failures, over 1m10s
270 runs so far, 0 failures, over 1m15s
270 runs so far, 0 failures, over 1m20s
300 runs so far, 0 failures, over 1m25s
310 runs so far, 0 failures, over 1m30s
330 runs so far, 0 failures, over 1m35s
360 runs so far, 0 failures, over 1m40s
360 runs so far, 0 failures, over 1m45s
390 runs so far, 0 failures, over 1m50s
412 runs so far, 0 failures, over 1m55s
420 runs so far, 0 failures, over 2m0s

initialized metamorphic constant "span-reuse-rate" with value 79
initialized metamorphic constant "coldata-batch-size" with value 3989
initialized metamorphic constant "addsst-rewrite-concurrency" with value 7
initialized metamorphic constant "kv.refresh_range.time_bound_iterators.enabled" with value false
initialized metamorphic constant "raft-log-truncation-clearrange-threshold" with value 129597
initialized metamorphic constant "kv.rangefeed.catchup_scan_iterator_optimization.enabled" with value false
initialized metamorphic constant "kv-batch-size" with value 1
initialized metamorphic constant "datum-row-converter-batch-size" with value 1
initialized metamorphic constant "row-container-rows-per-chunk-shift" with value 1
initialized metamorphic constant "inverted-joiner-batch-size" with value 1
initialized metamorphic constant "zig-zag-joiner-batch-size" with value 1
initialized metamorphic constant "spilling-queue-initial-len" with value 15
initialized metamorphic constant "merge-joiner-groups-buffer" with value 9
initialized metamorphic constant "ColIndexJoin-batch-size" with value 3187065
initialized metamorphic constant "max-batch-byte-size" with value 29862101
initialized metamorphic constant "async-IE-result-channel-buffer-size" with value 25
I220223 01:49:29.110408 1 (gostd) rand.go:147  [-] 1  random seed: 7902158050586388781
=== RUN   TestDirectoryConnect
    test_log_scope.go:79: test logs captured to: /home/jay/.cache/bazel/_bazel_jay/ad30d6d5ac3ec70478dfe5900736ea48/sandbox/linux-sandbox/1/execroot/cockroach/_tmp/a455c59caed0cf9d0a0298ad88bfe7a2/logTestDirectoryConnect1906532003
    test_log_scope.go:80: use -show-logs to present logs inline
=== RUN   TestDirectoryConnect/fallback_when_tenant_not_found
=== RUN   TestDirectoryConnect/fail_to_connect_to_backend
=== RUN   TestDirectoryConnect/successful_connection
=== RUN   TestDirectoryConnect/successful_connection_after_restart
=== RUN   TestDirectoryConnect/drain_connection
    proxy_handler_test.go:727: 
        	Error Trace:	proxy_handler_test.go:1090
        	Error:      	Received unexpected error:
        	            	while receiving startup message
        	            	(1) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.newErrorf
        	            	  | 	github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/error.go:100
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.glob..func4
        	            	  | 	github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/frontend_admitter.go:35
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.(*proxyHandler).handle
        	            	  | 	github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/proxy_handler.go:206
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.(*Server).Serve.func2
        	            	  | 	github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/server.go:185
        	            	  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
        	            	  | 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:494
        	            	  | runtime.goexit
        	            	  | 	GOROOT/src/runtime/asm_amd64.s:1581
        	            	Wraps: (2) while receiving startup message
        	            	Error types: (1) *withstack.withStack (2) *errutil.leafError
        	Test:       	TestDirectoryConnect/drain_connection
=== CONT  TestDirectoryConnect
    proxy_handler_test.go:753: -- test log scope end --
test logs left over in: /home/jay/.cache/bazel/_bazel_jay/ad30d6d5ac3ec70478dfe5900736ea48/sandbox/linux-sandbox/1/execroot/cockroach/_tmp/a455c59caed0cf9d0a0298ad88bfe7a2/logTestDirectoryConnect1906532003
--- FAIL: TestDirectoryConnect (8.13s)
    --- PASS: TestDirectoryConnect/fallback_when_tenant_not_found (0.00s)
    --- PASS: TestDirectoryConnect/fail_to_connect_to_backend (0.23s)
    --- PASS: TestDirectoryConnect/successful_connection (0.00s)
    --- PASS: TestDirectoryConnect/successful_connection_after_restart (2.57s)
    --- FAIL: TestDirectoryConnect/drain_connection (2.01s)
FAIL
I220223 01:49:37.250692 1 (gostd) testmain.go:154  [-] 1  Test //pkg/ccl/sqlproxyccl:sqlproxyccl_test exited with error code 1


ERROR: exit status 1

446 runs completed, 1 failures, over 2m4s
context canceled
FAIL
FAIL: //pkg/ccl/sqlproxyccl:sqlproxyccl_test (see /home/jay/.cache/bazel/_bazel_jay/ad30d6d5ac3ec70478dfe5900736ea48/execroot/cockroach/bazel-out/k8-fastbuild/testlogs/pkg/ccl/sqlproxyccl/sqlproxyccl_test/test.log)
INFO: Elapsed time: 129.758s, Critical Path: 125.63s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed, 1 test FAILED, 2 total actions
//pkg/ccl/sqlproxyccl:sqlproxyccl_test                                   FAILED in 125.1s
  /home/jay/.cache/bazel/_bazel_jay/ad30d6d5ac3ec70478dfe5900736ea48/execroot/cockroach/bazel-out/k8-fastbuild/testlogs/pkg/ccl/sqlproxyccl/sqlproxyccl_test/test.log

INFO: Build completed, 1 test FAILED, 2 total actions
ERROR: exit status 3

@imjching
Copy link
Author 

   proxy_handler_test.go:727: 
        	Error Trace:	proxy_handler_test.go:1091
        	Error:      	Received unexpected error:
        	            	while receiving startup message: invalid length of startup packet: 1347569948
        	            	(1) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.newErrorf
        	            	  | 	github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/error.go:100
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.glob..func4
        	            	  | 	github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/frontend_admitter.go:35
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.(*proxyHandler).handle
        	            	  | 	github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/proxy_handler.go:206
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.(*Server).Serve.func2
        	            	  | 	github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/server.go:185
        	            	  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
        	            	  | 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:494
        	            	  | runtime.goexit
        	            	  | 	GOROOT/src/runtime/asm_amd64.s:1581
        	            	Wraps: (2) secondary error attachment
        	            	  | invalid length of startup packet: 1347569948
        	            	  | (1) invalid length of startup packet: 1347569948
        	            	  | Error types: (1) *errors.errorString
        	            	Wraps: (3) while receiving startup message: invalid length of startup packet: 1347569948
        	            	Error types: (1) *withstack.withStack (2) *secondary.withSecondaryError (3) *errutil.leafError
        	Test:       	TestDirectoryConnect/drain_connection
=== CONT  TestDirectoryConnect
    proxy_handler_test.go:753: -- test log scope end --
test logs left over in: /home/jay/.cache/bazel/_bazel_jay/ad30d6d5ac3ec70478dfe5900736ea48/sandbox/linux-sandbox/12/execroot/cockroach/_tmp/a455c59caed0cf9d0a0298ad88bfe7a2/logTestDirectoryConnect78286009

@imjching
Copy link
Author 

80 82 73 32 42 32 72 84 84 80 47 50 46 48

PRI * HTTP/2.0

@imjching
Copy link
Author 

80 82 73 32 42 32 72 84 84 80 47 50 46 48 13 10 13 10 83 77 13 10 13 10 0 0 0 4 0

PRI * HTTP/2.0

SM



postgres://root:[email protected]:36547/?sslmode=disable&options=--cluster=tenant-cluster-28
postgres://root:[email protected]:37441/?sslmode=disable&options=--cluster=tenant-cluster-28

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment