Last active
August 20, 2019 18:19
-
-
Save mrballcb/629cf122869baf1f02ae838534db9cc5 to your computer and use it in GitHub Desktop.
RabbitMQ Federation Instability
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Cluster description: | |
Three 2 node RMQ-3.7.14 clusters, running in Kube, all inbound ports are exposed | |
via an internal ELB. | |
All users, permissions, policies, parameters (upstreams, sets) are defined in a | |
definitions.json file read when each cluster node starts. | |
The data volumes are EBS volumes, so is persistent across restarts. | |
Cluster locations: | |
us-east-1 (10.9.73.0/24) | |
eu-central-1 (10.10.17.0/24) | |
us-west-1 (10.11.12.0/24) | |
Each cluster is federated to the other 2 (forms a bi-directional triangle formation), and | |
is connecting to the others through their ELB. Federation docs refer to it as a "small | |
complete graph" and it looks like this: | |
https://www.rabbitmq.com/img/federation03.png | |
Each cluster has: | |
2 federation-upstreams in a federation-upstream-set A | |
2 federation-upstreams in a federation-upstream-set B | |
2 policies (1 for each federation-upstream-set, applied by a regex against exchange names) | |
Problem description: | |
Create an exchange, policy applies federation to it, and brings up federation | |
to the other two clusters. Is stable for a while (a day, a week sometimes). | |
Both us-east-1 and us-west-1 seem to be stable federated to each other long | |
term. | |
The eu-central-1 seems to be stable federated *to* us-east-1 and us-west-1. | |
Main Problem: | |
When it finally starts messing up, the federation link *to* eu-central-1 from | |
both US regions is constantly erroring and reconnecting, about every 45 seconds. | |
I don't have a successful way to recover other than completely deleting the | |
exchange from all clusters, then let our custom application recreate the | |
exchange. | |
IPs: | |
us-east-1 ELB rmq-fed-debug-us-east-1.dev.example.net: | |
10.9.73.4 | |
10.9.73.114 | |
10.9.73.211 | |
us-east-1 cluster members: | |
rmq-debug-rabbitmq-ha-0 10.9.73.23 | |
rmq-debug-rabbitmq-ha-1 10.9.73.113 | |
eu-central-1 ELB rmq-fed-debug-eu-central-1.dev.example.net: | |
10.10.17.48 | |
10.10.17.110 | |
10.10.17.163 | |
eu-central-1 cluster members: | |
rmq-debug-rabbitmq-ha-0 10.10.17.122 | |
rmq-debug-rabbitmq-ha-1 10.10.17.169 | |
us-west-1 ELB rmq-fed-debug-us-west-1.dev.example.net: | |
10.11.61.57 | |
10.11.61.88 | |
us-west-1 cluster members: | |
rmq-debug-rabbitmq-ha-0 10.11.61.24 | |
rmq-debug-rabbitmq-ha-1 10.11.61.99 | |
Non Debug logs for the recipient (10.10.17.122) of this federation attempt follow. | |
This rmq node is in eu-central-1. | |
The complaint seems to be spelled out in line 7, line 17, and line 33. | |
The exchange exists so I'm not quite sure what error it's describing. | |
It seems like merely it's complaining that the federation link is down, which is | |
accurate. Is there something else non-obvious? | |
1 2019-08-17 02:13:52.456 [info] <0.11875.1031> accepting AMQP connection <0.11875.1031> (10.10.17.122:29584 -> 100.96.12.4:5672) | |
2 2019-08-17 02:13:52.642 [info] <0.11875.1031> connection <0.11875.1031> (10.10.17.122:29584 -> 100.96.12.4:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
3 2019-08-17 02:13:54.126 [info] <0.12298.1031> accepting AMQP connection <0.12298.1031> (100.96.2.0:53420 -> 100.96.12.4:5672) | |
4 2019-08-17 02:14:23.508 [info] <0.12277.1031> accepting AMQP connection <0.12277.1031> (100.96.2.0:53470 -> 100.96.12.4:5672) | |
5 2019-08-17 02:14:23.695 [info] <0.12277.1031> connection <0.12277.1031> (100.96.2.0:53470 -> 100.96.12.4:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
6 2019-08-17 02:14:23.881 [error] <0.11731.1031> Channel error on connection <0.12277.1031> (100.96.2.0:53470 -> 100.96.12.4:5672, vhost: '/', user: 'FEDUSER'), channel 1: | |
7 operation exchange.declare caused a channel exception not_found: no exchange 'federation: bob-request-test-1 -> rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local B' in vhost '/' | |
8 2019-08-17 02:14:24.066 [info] <0.12277.1031> closing AMQP connection <0.12277.1031> (100.96.2.0:53470 -> 100.96.12.4:5672, vhost: '/', user: 'FEDUSER') | |
9 2019-08-17 02:14:24.732 [info] <0.11720.1031> closing AMQP connection <0.11720.1031> (10.10.17.122:29580 -> 100.96.12.4:5672 - Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited), vhost: '/', user: 'FEDUSER') | |
10 2019-08-17 02:14:25.433 [info] <0.11868.1031> closing AMQP connection <0.11868.1031> (10.10.17.122:29582 -> 100.96.12.4:5672 - Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited), vhost: '/', user: 'FEDUSER') | |
11 2019-08-17 02:14:30.613 [info] <0.12132.1031> accepting AMQP connection <0.12132.1031> (100.96.4.0:53264 -> 100.96.12.4:5672) | |
12 2019-08-17 02:14:30.771 [info] <0.12132.1031> Connection <0.12132.1031> (100.96.4.0:53264 -> 100.96.12.4:5672) has a client-provided name: Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited) | |
13 2019-08-17 02:14:30.928 [info] <0.12132.1031> connection <0.12132.1031> (100.96.4.0:53264 -> 100.96.12.4:5672 - Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited)): user 'FEDUSER' authenticated and granted access to vhost '/' | |
14 2019-08-17 02:15:02.246 [info] <0.12094.1031> accepting AMQP connection <0.12094.1031> (100.96.4.0:53294 -> 100.96.12.4:5672) | |
15 2019-08-17 02:15:02.433 [info] <0.12094.1031> connection <0.12094.1031> (100.96.4.0:53294 -> 100.96.12.4:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
16 2019-08-17 02:15:02.620 [error] <0.11832.1031> Channel error on connection <0.12094.1031> (100.96.4.0:53294 -> 100.96.12.4:5672, vhost: '/', user: 'FEDUSER'), channel 1: | |
17 operation exchange.declare caused a channel exception not_found: no exchange 'federation: bob-request-test-1 -> rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local A' in vhost '/' | |
18 2019-08-17 02:15:02.806 [info] <0.12094.1031> closing AMQP connection <0.12094.1031> (100.96.4.0:53294 -> 100.96.12.4:5672, vhost: '/', user: 'FEDUSER') | |
19 2019-08-17 02:15:04.184 [info] <0.12132.1031> closing AMQP connection <0.12132.1031> (100.96.4.0:53264 -> 100.96.12.4:5672 - Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited), vhost: '/', user: 'FEDUSER') | |
20 2019-08-17 02:15:09.381 [info] <0.12462.1031> accepting AMQP connection <0.12462.1031> (10.10.17.122:29672 -> 100.96.12.4:5672) | |
21 2019-08-17 02:15:09.538 [info] <0.12462.1031> Connection <0.12462.1031> (10.10.17.122:29672 -> 100.96.12.4:5672) has a client-provided name: Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited) | |
22 2019-08-17 02:15:09.694 [info] <0.12462.1031> connection <0.12462.1031> (10.10.17.122:29672 -> 100.96.12.4:5672 - Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited)): user 'FEDUSER' authenticated and granted access to vhost '/' | |
23 2019-08-17 02:15:11.589 [info] <0.12744.1031> accepting AMQP connection <0.12744.1031> (100.96.2.0:53534 -> 100.96.12.4:5672) | |
24 2019-08-17 02:15:11.904 [info] <0.12744.1031> connection <0.12744.1031> (100.96.2.0:53534 -> 100.96.12.4:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
25 2019-08-17 02:15:12.536 [info] <0.12744.1031> closing AMQP connection <0.12744.1031> (100.96.2.0:53534 -> 100.96.12.4:5672, vhost: '/', user: 'FEDUSER') | |
26 2019-08-17 02:15:42.929 [info] <0.12462.1031> closing AMQP connection <0.12462.1031> (10.10.17.122:29672 -> 100.96.12.4:5672 - Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited), vhost: '/', user: 'FEDUSER') | |
27 2019-08-17 02:15:48.109 [info] <0.12313.1031> accepting AMQP connection <0.12313.1031> (10.10.17.122:29710 -> 100.96.12.4:5672) | |
28 2019-08-17 02:15:48.266 [info] <0.12313.1031> Connection <0.12313.1031> (10.10.17.122:29710 -> 100.96.12.4:5672) has a client-provided name: Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited) | |
29 2019-08-17 02:15:48.423 [info] <0.12313.1031> connection <0.12313.1031> (10.10.17.122:29710 -> 100.96.12.4:5672 - Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited)): user 'FEDUSER' authenticated and granted access to vhost '/' | |
30 2019-08-17 02:16:19.770 [info] <0.12663.1031> accepting AMQP connection <0.12663.1031> (100.96.4.0:53396 -> 100.96.12.4:5672) | |
31 2019-08-17 02:16:19.958 [info] <0.12663.1031> connection <0.12663.1031> (100.96.4.0:53396 -> 100.96.12.4:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
32 2019-08-17 02:16:20.146 [error] <0.12901.1031> Channel error on connection <0.12663.1031> (100.96.4.0:53396 -> 100.96.12.4:5672, vhost: '/', user: 'FEDUSER'), channel 1: | |
33 operation exchange.declare caused a channel exception not_found: no exchange 'federation: bob-request-test-1 -> rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local A' in vhost '/' | |
34 2019-08-17 02:16:20.333 [info] <0.12663.1031> closing AMQP connection <0.12663.1031> (100.96.4.0:53396 -> 100.96.12.4:5672, vhost: '/', user: 'FEDUSER') | |
35 2019-08-17 02:16:21.702 [info] <0.12313.1031> closing AMQP connection <0.12313.1031> (10.10.17.122:29710 -> 100.96.12.4:5672 - Federation link (upstream: bridge_solicited_stream-eu-central-1, policy: federate-bob-solicited), vhost: '/', user: 'FEDUSER') | |
Debug level logs for the federation initiator in us-east-1: | |
1 2019-08-17 01:47:22.721 [debug] <0.418.0> Supervisor {<0.418.0>,rabbit_federation_link_sup} started rabbit_federation_exchange_link:start_link({{upstream,[<<"amqp://FEDUSER:[email protected]....">>],...},...}) at pid <0.4106.1411> | |
2 2019-08-17 01:47:22.722 [debug] <0.3984.1411> Supervisor {<0.3984.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.4270.1411> | |
3 2019-08-17 01:47:22.722 [debug] <0.3984.1411> Supervisor {<0.3984.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.4270.1411>, {amqp_params_direct,none,none,<<"/">>,'rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local',...}) at pid <0.3682.1411> | |
4 2019-08-17 01:47:22.723 [debug] <0.4270.1411> Supervisor {<0.4270.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(direct, <0.3682.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.3...">>) at pid <0.3170.1411> | |
5 2019-08-17 01:47:22.723 [debug] <0.4270.1411> Supervisor {<0.4270.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.3682.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.3...">>, <0.3170.1411>) at pid <0.4123.1411> | |
6 2019-08-17 01:47:22.723 [debug] <0.4270.1411> Supervisor {<0.4270.1411>,amqp_connection_type_sup} started rabbit_queue_collector:start_link(<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.3...">>) at pid <0.4069.1411> | |
7 2019-08-17 01:47:22.723 [debug] <0.4449.1411> Supervisor {<0.4449.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.3124.1411> | |
8 2019-08-17 01:47:22.723 [debug] <0.4449.1411> Supervisor {<0.4449.1411>,amqp_channel_sup} started amqp_channel:start_link(direct, <0.3682.1411>, 1, <0.3124.1411>, {<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.4044.1411> | |
9 2019-08-17 01:47:22.724 [debug] <0.2746.1411> Supervisor {<0.2746.1411>,rabbit_channel_sup} started rabbit_limiter:start_link({<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.3174.1411> | |
10 2019-08-17 01:47:22.724 [debug] <0.2746.1411> Supervisor {<0.2746.1411>,rabbit_channel_sup} started rabbit_channel:start_link(1, <0.4044.1411>, <0.4044.1411>, <0.3682.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.3...">>, rabbit_framing_amqp_0_9_1, {user,<<"none">>,[],[{rabbit_auth_backend_dummy,none}]}, <<"/">>, [{<<"publisher_confirms">>,bool,true},{<<"exchange_exchange_bindings">>,bool,true},{<<"basic.nack">>,...},...], <0.4069.1411>, <0.3174.1411>) at pid <0.4262.1411> | |
11 2019-08-17 01:47:22.724 [debug] <0.3513.1411> Supervisor {<0.3513.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.3512.1411> | |
12 2019-08-17 01:47:22.724 [debug] <0.3513.1411> Supervisor {<0.3513.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.3512.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-eu-ce...",...}) at pid <0.3678.1411> | |
13 2019-08-17 01:47:22.900 [debug] <0.3512.1411> Supervisor {<0.3512.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.3678.1411>, <<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>) at pid <0.3985.1411> | |
14 2019-08-17 01:47:22.900 [debug] <0.3512.1411> Supervisor {<0.3512.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.3678.1411>, <<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>, <0.3985.1411>) at pid <0.4134.1411> | |
15 2019-08-17 01:47:22.900 [debug] <0.3512.1411> Supervisor {<0.3512.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943639>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.3678.1411>, <<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>) at pid <0.3662.1411> | |
16 2019-08-17 01:47:22.900 [debug] <0.3512.1411> Supervisor {<0.3512.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943639>, <0.3678.1411>, <0.4134.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>) at pid <0.3623.1411> | |
17 2019-08-17 01:47:23.213 [debug] <0.3512.1411> Supervisor {<0.3512.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943639>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>}) at pid <0.3400.1411> | |
18 2019-08-17 01:47:23.213 [debug] <0.3512.1411> Supervisor {<0.3512.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943639>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>}) at pid <0.5040.1411> | |
19 2019-08-17 01:47:23.370 [debug] <0.3683.1411> Supervisor {<0.3683.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,1}) at pid <0.3729.1411> | |
20 2019-08-17 01:47:23.370 [debug] <0.3683.1411> Supervisor {<0.3683.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3678.1411>, 1, <0.3729.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,1}) at pid <0.4260.1411> | |
21 2019-08-17 01:47:23.370 [debug] <0.3683.1411> Supervisor {<0.3683.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943639>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.4260.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,1}) at pid <0.4436.1411> | |
22 2019-08-17 01:47:23.996 [debug] <0.3490.1411> Supervisor {<0.3490.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3445.1411> | |
23 2019-08-17 01:47:23.996 [debug] <0.3490.1411> Supervisor {<0.3490.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3678.1411>, 2, <0.3445.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3658.1411> | |
24 2019-08-17 01:47:23.996 [debug] <0.3490.1411> Supervisor {<0.3490.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943639>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.3658.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3600.1411> | |
25 2019-08-17 01:47:24.464 [debug] <0.3515.1411> Supervisor {<0.3515.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.1785.1411> | |
26 2019-08-17 01:47:24.465 [debug] <0.3515.1411> Supervisor {<0.3515.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3678.1411>, 2, <0.1785.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.4150.1411> | |
27 2019-08-17 01:47:24.465 [debug] <0.3515.1411> Supervisor {<0.3515.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943639>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4150.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3632.1411> | |
28 2019-08-17 01:47:24.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
29 2019-08-17 01:47:24.672 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
30 2019-08-17 01:47:24.940 [debug] <0.4866.1411> Supervisor {<0.4866.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.3734.1411> | |
31 2019-08-17 01:47:24.940 [debug] <0.4866.1411> Supervisor {<0.4866.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.3734.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-eu-ce...",...}) at pid <0.3810.1411> | |
32 2019-08-17 01:47:25.107 [debug] <0.3734.1411> Supervisor {<0.3734.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.3810.1411>, <<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>) at pid <0.3327.1411> | |
33 2019-08-17 01:47:25.108 [debug] <0.3734.1411> Supervisor {<0.3734.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.3810.1411>, <<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>, <0.3327.1411>) at pid <0.3319.1411> | |
34 2019-08-17 01:47:25.108 [debug] <0.3734.1411> Supervisor {<0.3734.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943777>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.3810.1411>, <<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>) at pid <0.2007.1411> | |
35 2019-08-17 01:47:25.108 [debug] <0.3734.1411> Supervisor {<0.3734.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943777>, <0.3810.1411>, <0.3319.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>) at pid <0.4274.1411> | |
36 2019-08-17 01:47:25.198 [debug] <0.4695.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
37 2019-08-17 01:47:25.290 [debug] <0.3693.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
38 2019-08-17 01:47:25.422 [debug] <0.3734.1411> Supervisor {<0.3734.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943777>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>}) at pid <0.3500.1411> | |
39 2019-08-17 01:47:25.422 [debug] <0.3734.1411> Supervisor {<0.3734.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943777>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>}) at pid <0.2476.1411> | |
40 2019-08-17 01:47:25.578 [debug] <0.4252.1411> Supervisor {<0.4252.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>,1}) at pid <0.4284.1411> | |
41 2019-08-17 01:47:25.578 [debug] <0.4252.1411> Supervisor {<0.4252.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3810.1411>, 1, <0.4284.1411>, {<<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>,1}) at pid <0.3360.1411> | |
42 2019-08-17 01:47:25.578 [debug] <0.4252.1411> Supervisor {<0.4252.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943777>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.3360.1411>, {<<"client 100.96.3.38:35231 -> 10.10.17.48:5672">>,1}) at pid <0.2975.1411> | |
43 2019-08-17 01:47:26.220 [debug] <0.4636.1411> Supervisor {<0.4636.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3057.1411> | |
44 2019-08-17 01:47:26.220 [debug] <0.4636.1411> Supervisor {<0.4636.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3678.1411>, 2, <0.3057.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3294.1411> | |
45 2019-08-17 01:47:26.220 [debug] <0.4636.1411> Supervisor {<0.4636.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943639>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.3294.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3345.1411> | |
46 2019-08-17 01:47:26.600 [debug] <0.4181.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
47 2019-08-17 01:47:26.701 [info] <0.4106.1411> Federation exchange 'bob-request-test-1' in vhost '/' connected to exchange 'bob-request-test-1' in vhost '/' on amqp://rmq-fed-debug-eu-central-1.dev.example.net | |
48 2019-08-17 01:47:28.771 [debug] <0.3813.1411> Supervisor {<0.3813.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.4497.1411> | |
49 2019-08-17 01:47:28.771 [debug] <0.3813.1411> Supervisor {<0.3813.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.4497.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-us-ea...",...}) at pid <0.4364.1411> | |
50 2019-08-17 01:47:28.864 [debug] <0.4497.1411> Supervisor {<0.4497.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.4364.1411>, <<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>) at pid <0.3939.1411> | |
51 2019-08-17 01:47:28.864 [debug] <0.4497.1411> Supervisor {<0.4497.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.4364.1411>, <<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>, <0.3939.1411>) at pid <0.3529.1411> | |
52 2019-08-17 01:47:28.864 [debug] <0.4497.1411> Supervisor {<0.4497.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943775>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.4364.1411>, <<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>) at pid <0.2943.1411> | |
53 2019-08-17 01:47:28.864 [debug] <0.4497.1411> Supervisor {<0.4497.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943775>, <0.4364.1411>, <0.3529.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>) at pid <0.3543.1411> | |
54 2019-08-17 01:47:29.005 [debug] <0.4497.1411> Supervisor {<0.4497.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943775>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>}) at pid <0.3201.1411> | |
55 2019-08-17 01:47:29.005 [debug] <0.4497.1411> Supervisor {<0.4497.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943775>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>}) at pid <0.5023.1411> | |
56 2019-08-17 01:47:29.075 [debug] <0.4570.1411> Supervisor {<0.4570.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>,1}) at pid <0.3605.1411> | |
57 2019-08-17 01:47:29.076 [debug] <0.4570.1411> Supervisor {<0.4570.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.4364.1411>, 1, <0.3605.1411>, {<<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>,1}) at pid <0.3413.1411> | |
58 2019-08-17 01:47:29.076 [debug] <0.4570.1411> Supervisor {<0.4570.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943775>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.3413.1411>, {<<"client 100.96.3.38:36709 -> 10.9.73.4:5672">>,1}) at pid <0.4291.1411> | |
59 2019-08-17 01:47:30.196 [debug] <0.3868.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
60 2019-08-17 01:47:30.291 [debug] <0.4279.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
61 2019-08-17 01:47:31.601 [debug] <0.3828.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
62 2019-08-17 01:47:31.808 [debug] <0.4621.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
63 2019-08-17 01:47:34.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
64 2019-08-17 01:47:34.673 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
65 2019-08-17 01:47:35.193 [debug] <0.4892.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
66 2019-08-17 01:47:35.288 [debug] <0.4780.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
67 2019-08-17 01:47:36.600 [debug] <0.3408.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
68 2019-08-17 01:47:40.198 [debug] <0.3086.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
69 2019-08-17 01:47:40.293 [debug] <0.2401.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
70 2019-08-17 01:47:41.600 [debug] <0.4346.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
71 2019-08-17 01:47:41.808 [debug] <0.4292.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
72 2019-08-17 01:47:43.831 [debug] <0.3577.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
73 2019-08-17 01:47:43.834 [debug] <0.4087.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
74 2019-08-17 01:47:43.843 [debug] <0.2333.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
75 2019-08-17 01:47:43.854 [debug] <0.4373.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
76 2019-08-17 01:47:44.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
77 2019-08-17 01:47:44.674 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
78 2019-08-17 01:47:45.196 [debug] <0.4858.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
79 2019-08-17 01:47:45.298 [debug] <0.3975.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
80 2019-08-17 01:47:46.599 [debug] <0.4066.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
81 2019-08-17 01:47:50.191 [debug] <0.4130.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
82 2019-08-17 01:47:50.288 [debug] <0.4900.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
83 2019-08-17 01:47:51.600 [debug] <0.4064.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
84 2019-08-17 01:47:51.811 [debug] <0.4782.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
85 2019-08-17 01:47:54.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
86 2019-08-17 01:47:54.672 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
87 2019-08-17 01:47:55.196 [debug] <0.4030.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
88 2019-08-17 01:47:55.291 [debug] <0.3714.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
89 2019-08-17 01:47:55.378 [info] <0.4106.1411> Federation exchange 'bob-request-test-1' in vhost '/' received 'basic.cancel' | |
90 2019-08-17 01:47:55.378 [debug] <0.2465.1411> Supervisor {<0.2465.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.4390.1411> | |
91 2019-08-17 01:47:55.378 [debug] <0.2465.1411> Supervisor {<0.2465.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3678.1411>, 2, <0.4390.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.4545.1411> | |
92 2019-08-17 01:47:55.378 [debug] <0.2465.1411> Supervisor {<0.2465.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943639>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4545.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3926.1411> | |
93 2019-08-17 01:47:55.847 [debug] <0.4086.1411> Supervisor {<0.4086.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.4410.1411> | |
94 2019-08-17 01:47:55.847 [debug] <0.4086.1411> Supervisor {<0.4086.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3678.1411>, 2, <0.4410.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.4906.1411> | |
95 2019-08-17 01:47:55.847 [debug] <0.4086.1411> Supervisor {<0.4086.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943639>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4906.1411>, {<<"client 100.96.3.38:43295 -> 10.10.17.110:5672">>,2}) at pid <0.3248.1411> | |
96 2019-08-17 01:47:56.478 [error] <0.418.0> Supervisor {<0.418.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqp://FEDUSER:[email protected]">>], | |
97 <<"bob-request-test-1">>,<<"bob-request-test-1">>,1000,1,5,10000, | |
98 10000,false,'on-publish',none, | |
99 <<"bridge_solicited_stream-eu-central-1">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqp://FEDUSER:[email protected]....">>],...},...}) at <0.4106.1411> exit with reason {shutdown,restart} in context child_terminated | |
100 2019-08-17 01:47:56.601 [debug] <0.3099.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
101 2019-08-17 01:47:59.238 [debug] <0.3564.1411> Supervisor {<0.3564.1411>,rabbit_connection_sup} started rabbit_connection_helper_sup:start_link() at pid <0.4259.1411> | |
102 2019-08-17 01:47:59.238 [debug] <0.3564.1411> Supervisor {<0.3564.1411>,rabbit_connection_sup} started rabbit_reader:start_link(<0.4259.1411>, {acceptor,{0,0,0,0,0,0,0,0},5672}) at pid <0.3758.1411> | |
103 2019-08-17 01:47:59.238 [info] <0.3758.1411> accepting AMQP connection <0.3758.1411> (100.96.7.0:35134 -> 100.96.3.38:5672) | |
104 2019-08-17 01:47:59.287 [debug] <0.4261.1411> Supervisor {<0.4261.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.5150.1411> | |
105 2019-08-17 01:47:59.287 [debug] <0.4261.1411> Supervisor {<0.4261.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.5150.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-us-ea...",...}) at pid <0.2968.1411> | |
106 2019-08-17 01:47:59.308 [debug] <0.3758.1411> User 'FEDUSER' authenticated successfully by backend rabbit_auth_backend_internal | |
107 2019-08-17 01:47:59.373 [debug] <0.5150.1411> Supervisor {<0.5150.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.2968.1411>, <<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>) at pid <0.4020.1411> | |
108 2019-08-17 01:47:59.373 [debug] <0.5150.1411> Supervisor {<0.5150.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.2968.1411>, <<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>, <0.4020.1411>) at pid <0.4675.1411> | |
109 2019-08-17 01:47:59.373 [debug] <0.5150.1411> Supervisor {<0.5150.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943847>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.2968.1411>, <<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>) at pid <0.4037.1411> | |
110 2019-08-17 01:47:59.373 [debug] <0.5150.1411> Supervisor {<0.5150.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943847>, <0.2968.1411>, <0.4675.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>) at pid <0.3567.1411> | |
111 2019-08-17 01:47:59.378 [debug] <0.4259.1411> Supervisor {<0.4259.1411>,rabbit_connection_helper_sup} started rabbit_queue_collector:start_link(<<"100.96.7.0:35134 -> 100.96.3.38:5672">>) at pid <0.3853.1411> | |
112 2019-08-17 01:47:59.378 [debug] <0.4259.1411> Supervisor {<0.4259.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943842>, 10, #Fun<rabbit_reader.34.21003307>, {heartbeat_sender,<<"100.96.7.0:35134 -> 100.96.3.38:5672">>}) at pid <0.4297.1411> | |
113 2019-08-17 01:47:59.378 [debug] <0.4259.1411> Supervisor {<0.4259.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943842>, 10, #Fun<rabbit_reader.35.21003307>, {heartbeat_receiver,<<"100.96.7.0:35134 -> 100.96.3.38:5672">>}) at pid <0.3128.1411> | |
114 2019-08-17 01:47:59.378 [debug] <0.4259.1411> Supervisor {<0.4259.1411>,rabbit_connection_helper_sup} started rabbit_channel_sup_sup:start_link() at pid <0.3289.1411> | |
115 2019-08-17 01:47:59.378 [info] <0.3758.1411> connection <0.3758.1411> (100.96.7.0:35134 -> 100.96.3.38:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
116 2019-08-17 01:47:59.447 [debug] <0.4072.1411> Supervisor {<0.4072.1411>,rabbit_channel_sup} started rabbit_writer:start_link(#Port<0.12943842>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.3758.1411>, {<<"100.96.7.0:35134 -> 100.96.3.38:5672">>,1}, true) at pid <0.4229.1411> | |
117 2019-08-17 01:47:59.447 [debug] <0.4072.1411> Supervisor {<0.4072.1411>,rabbit_channel_sup} started rabbit_limiter:start_link({<<"100.96.7.0:35134 -> 100.96.3.38:5672">>,1}) at pid <0.4586.1411> | |
118 2019-08-17 01:47:59.447 [debug] <0.4072.1411> Supervisor {<0.4072.1411>,rabbit_channel_sup} started rabbit_channel:start_link(1, <0.3758.1411>, <0.4229.1411>, <0.3758.1411>, <<"100.96.7.0:35134 -> 100.96.3.38:5672">>, rabbit_framing_amqp_0_9_1, {user,<<FEDUSER>>,[administrator],[{rabbit_auth_backend_internal,none}]}, <<"/">>, [{<<"publisher_confirms">>,bool,true},{<<"exchange_exchange_bindings">>,bool,true},{<<"basic.nack">>,...},...], <0.3853.1411>, <0.4586.1411>) at pid <0.4227.1411> | |
119 2019-08-17 01:47:59.514 [debug] <0.5150.1411> Supervisor {<0.5150.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943847>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>}) at pid <0.3643.1411> | |
120 2019-08-17 01:47:59.514 [debug] <0.5150.1411> Supervisor {<0.5150.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943847>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>}) at pid <0.4029.1411> | |
121 2019-08-17 01:47:59.584 [debug] <0.3197.1411> Supervisor {<0.3197.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>,1}) at pid <0.3279.1411> | |
122 2019-08-17 01:47:59.584 [debug] <0.3197.1411> Supervisor {<0.3197.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.2968.1411>, 1, <0.3279.1411>, {<<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>,1}) at pid <0.4846.1411> | |
123 2019-08-17 01:47:59.585 [debug] <0.3197.1411> Supervisor {<0.3197.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943847>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.4846.1411>, {<<"client 100.96.3.38:50577 -> 10.9.73.114:5672">>,1}) at pid <0.4482.1411> | |
124 2019-08-17 01:47:59.655 [info] <0.3758.1411> closing AMQP connection <0.3758.1411> (100.96.7.0:35134 -> 100.96.3.38:5672, vhost: '/', user: 'FEDUSER') | |
125 2019-08-17 01:48:00.196 [debug] <0.4785.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
126 2019-08-17 01:48:00.291 [debug] <0.4319.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
127 2019-08-17 01:48:01.486 [debug] <0.418.0> Supervisor {<0.418.0>,rabbit_federation_link_sup} started rabbit_federation_exchange_link:start_link({{upstream,[<<"amqp://FEDUSER:[email protected]....">>],...},...}) at pid <0.3816.1411> | |
128 2019-08-17 01:48:01.486 [debug] <0.4451.1411> Supervisor {<0.4451.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.4351.1411> | |
129 2019-08-17 01:48:01.486 [debug] <0.4451.1411> Supervisor {<0.4451.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.4351.1411>, {amqp_params_direct,none,none,<<"/">>,'rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local',...}) at pid <0.4159.1411> | |
130 2019-08-17 01:48:01.487 [debug] <0.4351.1411> Supervisor {<0.4351.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(direct, <0.4159.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.4...">>) at pid <0.4421.1411> | |
131 2019-08-17 01:48:01.487 [debug] <0.4351.1411> Supervisor {<0.4351.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.4159.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.4...">>, <0.4421.1411>) at pid <0.4005.1411> | |
132 2019-08-17 01:48:01.487 [debug] <0.4351.1411> Supervisor {<0.4351.1411>,amqp_connection_type_sup} started rabbit_queue_collector:start_link(<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.4...">>) at pid <0.4116.1411> | |
133 2019-08-17 01:48:01.487 [debug] <0.4316.1411> Supervisor {<0.4316.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.4415.1411> | |
134 2019-08-17 01:48:01.487 [debug] <0.4316.1411> Supervisor {<0.4316.1411>,amqp_channel_sup} started amqp_channel:start_link(direct, <0.4159.1411>, 1, <0.4415.1411>, {<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.5214.1411> | |
135 2019-08-17 01:48:01.487 [debug] <0.4305.1411> Supervisor {<0.4305.1411>,rabbit_channel_sup} started rabbit_limiter:start_link({<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.3036.1411> | |
136 2019-08-17 01:48:01.487 [debug] <0.4305.1411> Supervisor {<0.4305.1411>,rabbit_channel_sup} started rabbit_channel:start_link(1, <0.5214.1411>, <0.5214.1411>, <0.4159.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.4...">>, rabbit_framing_amqp_0_9_1, {user,<<"none">>,[],[{rabbit_auth_backend_dummy,none}]}, <<"/">>, [{<<"publisher_confirms">>,bool,true},{<<"exchange_exchange_bindings">>,bool,true},{<<"basic.nack">>,...},...], <0.4116.1411>, <0.3036.1411>) at pid <0.3629.1411> | |
137 2019-08-17 01:48:01.488 [debug] <0.4371.1411> Supervisor {<0.4371.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.3477.1411> | |
138 2019-08-17 01:48:01.488 [debug] <0.4371.1411> Supervisor {<0.4371.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.3477.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-eu-ce...",...}) at pid <0.3666.1411> | |
139 2019-08-17 01:48:01.600 [debug] <0.4021.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
140 2019-08-17 01:48:01.658 [debug] <0.3477.1411> Supervisor {<0.3477.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.3666.1411>, <<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>) at pid <0.4100.1411> | |
141 2019-08-17 01:48:01.658 [debug] <0.3477.1411> Supervisor {<0.3477.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.3666.1411>, <<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>, <0.4100.1411>) at pid <0.3797.1411> | |
142 2019-08-17 01:48:01.658 [debug] <0.3477.1411> Supervisor {<0.3477.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943772>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.3666.1411>, <<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>) at pid <0.2958.1411> | |
143 2019-08-17 01:48:01.658 [debug] <0.3477.1411> Supervisor {<0.3477.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943772>, <0.3666.1411>, <0.3797.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>) at pid <0.2761.1411> | |
144 2019-08-17 01:48:01.810 [debug] <0.3242.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
145 2019-08-17 01:48:01.972 [debug] <0.3477.1411> Supervisor {<0.3477.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943772>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>}) at pid <0.2760.1411> | |
146 2019-08-17 01:48:01.973 [debug] <0.3477.1411> Supervisor {<0.3477.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943772>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>}) at pid <0.3870.1411> | |
147 2019-08-17 01:48:02.130 [debug] <0.3018.1411> Supervisor {<0.3018.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,1}) at pid <0.4337.1411> | |
148 2019-08-17 01:48:02.130 [debug] <0.3018.1411> Supervisor {<0.3018.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3666.1411>, 1, <0.4337.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,1}) at pid <0.4666.1411> | |
149 2019-08-17 01:48:02.130 [debug] <0.3018.1411> Supervisor {<0.3018.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943772>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.4666.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,1}) at pid <0.4107.1411> | |
150 2019-08-17 01:48:02.758 [debug] <0.3732.1411> Supervisor {<0.3732.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4868.1411> | |
151 2019-08-17 01:48:02.758 [debug] <0.3732.1411> Supervisor {<0.3732.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3666.1411>, 2, <0.4868.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.3956.1411> | |
152 2019-08-17 01:48:02.758 [debug] <0.3732.1411> Supervisor {<0.3732.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943772>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.3956.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.5159.1411> | |
153 2019-08-17 01:48:03.227 [debug] <0.4394.1411> Supervisor {<0.4394.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4773.1411> | |
154 2019-08-17 01:48:03.227 [debug] <0.4394.1411> Supervisor {<0.4394.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3666.1411>, 2, <0.4773.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4650.1411> | |
155 2019-08-17 01:48:03.227 [debug] <0.4394.1411> Supervisor {<0.4394.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943772>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4650.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4910.1411> | |
156 2019-08-17 01:48:03.705 [debug] <0.4617.1411> Supervisor {<0.4617.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.4608.1411> | |
157 2019-08-17 01:48:03.705 [debug] <0.4617.1411> Supervisor {<0.4617.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.4608.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-eu-ce...",...}) at pid <0.3573.1411> | |
158 2019-08-17 01:48:03.875 [debug] <0.4608.1411> Supervisor {<0.4608.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.3573.1411>, <<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>) at pid <0.3463.1411> | |
159 2019-08-17 01:48:03.875 [debug] <0.4608.1411> Supervisor {<0.4608.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.3573.1411>, <<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>, <0.3463.1411>) at pid <0.4579.1411> | |
160 2019-08-17 01:48:03.875 [debug] <0.4608.1411> Supervisor {<0.4608.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943854>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.3573.1411>, <<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>) at pid <0.3892.1411> | |
161 2019-08-17 01:48:03.875 [debug] <0.4608.1411> Supervisor {<0.4608.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943854>, <0.3573.1411>, <0.4579.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>) at pid <0.4088.1411> | |
162 2019-08-17 01:48:04.189 [debug] <0.4608.1411> Supervisor {<0.4608.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943854>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>}) at pid <0.4542.1411> | |
163 2019-08-17 01:48:04.189 [debug] <0.4608.1411> Supervisor {<0.4608.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943854>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>}) at pid <0.3366.1411> | |
164 2019-08-17 01:48:04.346 [debug] <0.4187.1411> Supervisor {<0.4187.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>,1}) at pid <0.4714.1411> | |
165 2019-08-17 01:48:04.346 [debug] <0.4187.1411> Supervisor {<0.4187.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3573.1411>, 1, <0.4714.1411>, {<<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>,1}) at pid <0.5204.1411> | |
166 2019-08-17 01:48:04.346 [debug] <0.4187.1411> Supervisor {<0.4187.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943854>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.5204.1411>, {<<"client 100.96.3.38:59965 -> 10.10.17.110:5672">>,1}) at pid <0.4537.1411> | |
167 2019-08-17 01:48:04.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
168 2019-08-17 01:48:04.674 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
169 2019-08-17 01:48:04.992 [debug] <0.4914.1411> Supervisor {<0.4914.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4520.1411> | |
170 2019-08-17 01:48:04.993 [debug] <0.4914.1411> Supervisor {<0.4914.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3666.1411>, 2, <0.4520.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4353.1411> | |
171 2019-08-17 01:48:04.993 [debug] <0.4914.1411> Supervisor {<0.4914.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943772>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4353.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4084.1411> | |
172 2019-08-17 01:48:05.193 [debug] <0.3814.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
173 2019-08-17 01:48:05.286 [debug] <0.4248.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
174 2019-08-17 01:48:05.482 [info] <0.3816.1411> Federation exchange 'bob-request-test-1' in vhost '/' connected to exchange 'bob-request-test-1' in vhost '/' on amqp://rmq-fed-debug-eu-central-1.dev.example.net | |
175 2019-08-17 01:48:06.600 [debug] <0.3556.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
176 2019-08-17 01:48:10.196 [debug] <0.3845.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
177 2019-08-17 01:48:10.291 [debug] <0.4706.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
178 2019-08-17 01:48:11.604 [debug] <0.4673.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
179 2019-08-17 01:48:11.810 [debug] <0.4737.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
180 2019-08-17 01:48:13.831 [debug] <0.2693.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
181 2019-08-17 01:48:13.834 [debug] <0.5322.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
182 2019-08-17 01:48:13.842 [debug] <0.3917.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
183 2019-08-17 01:48:13.853 [debug] <0.3596.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
184 2019-08-17 01:48:14.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
185 2019-08-17 01:48:14.672 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
186 2019-08-17 01:48:15.191 [debug] <0.5012.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
187 2019-08-17 01:48:15.286 [debug] <0.3847.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
188 2019-08-17 01:48:16.599 [debug] <0.4395.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
189 2019-08-17 01:48:18.892 [debug] <0.4212.1411> Supervisor {<0.4212.1411>,rabbit_connection_sup} started rabbit_connection_helper_sup:start_link() at pid <0.3832.1411> | |
190 2019-08-17 01:48:18.892 [debug] <0.4212.1411> Supervisor {<0.4212.1411>,rabbit_connection_sup} started rabbit_reader:start_link(<0.3832.1411>, {acceptor,{0,0,0,0,0,0,0,0},5672}) at pid <0.4777.1411> | |
191 2019-08-17 01:48:18.892 [info] <0.4777.1411> accepting AMQP connection <0.4777.1411> (10.11.61.24:62978 -> 100.96.3.38:5672) | |
192 2019-08-17 01:48:19.049 [debug] <0.4777.1411> User 'FEDUSER' authenticated successfully by backend rabbit_auth_backend_internal | |
193 2019-08-17 01:48:19.204 [debug] <0.3832.1411> Supervisor {<0.3832.1411>,rabbit_connection_helper_sup} started rabbit_queue_collector:start_link(<<"10.11.61.24:62978 -> 100.96.3.38:5672">>) at pid <0.4849.1411> | |
194 2019-08-17 01:48:19.205 [debug] <0.3832.1411> Supervisor {<0.3832.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943887>, 10, #Fun<rabbit_reader.34.21003307>, {heartbeat_sender,<<"10.11.61.24:62978 -> 100.96.3.38:5672">>}) at pid <0.3481.1411> | |
195 2019-08-17 01:48:19.205 [debug] <0.3832.1411> Supervisor {<0.3832.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943887>, 10, #Fun<rabbit_reader.35.21003307>, {heartbeat_receiver,<<"10.11.61.24:62978 -> 100.96.3.38:5672">>}) at pid <0.5452.1411> | |
196 2019-08-17 01:48:19.205 [info] <0.4777.1411> connection <0.4777.1411> (10.11.61.24:62978 -> 100.96.3.38:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
197 2019-08-17 01:48:19.205 [debug] <0.3832.1411> Supervisor {<0.3832.1411>,rabbit_connection_helper_sup} started rabbit_channel_sup_sup:start_link() at pid <0.4210.1411> | |
198 2019-08-17 01:48:19.361 [debug] <0.3450.1411> Supervisor {<0.3450.1411>,rabbit_channel_sup} started rabbit_writer:start_link(#Port<0.12943887>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.4777.1411>, {<<"10.11.61.24:62978 -> 100.96.3.38:5672">>,1}, true) at pid <0.4356.1411> | |
199 2019-08-17 01:48:19.361 [debug] <0.3450.1411> Supervisor {<0.3450.1411>,rabbit_channel_sup} started rabbit_limiter:start_link({<<"10.11.61.24:62978 -> 100.96.3.38:5672">>,1}) at pid <0.4616.1411> | |
200 2019-08-17 01:48:19.361 [debug] <0.3450.1411> Supervisor {<0.3450.1411>,rabbit_channel_sup} started rabbit_channel:start_link(1, <0.4777.1411>, <0.4356.1411>, <0.4777.1411>, <<"10.11.61.24:62978 -> 100.96.3.38:5672">>, rabbit_framing_amqp_0_9_1, {user,<<FEDUSER>>,[administrator],[{rabbit_auth_backend_internal,none}]}, <<"/">>, [{<<"publisher_confirms">>,bool,true},{<<"exchange_exchange_bindings">>,bool,true},{<<"basic.nack">>,...},...], <0.4849.1411>, <0.4616.1411>) at pid <0.4877.1411> | |
201 2019-08-17 01:48:19.828 [info] <0.4777.1411> closing AMQP connection <0.4777.1411> (10.11.61.24:62978 -> 100.96.3.38:5672, vhost: '/', user: 'FEDUSER') | |
202 2019-08-17 01:48:20.194 [debug] <0.3234.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
203 2019-08-17 01:48:20.291 [debug] <0.2720.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
204 2019-08-17 01:48:21.600 [debug] <0.3863.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
205 2019-08-17 01:48:21.810 [debug] <0.5926.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
206 2019-08-17 01:48:24.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
207 2019-08-17 01:48:24.673 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
208 2019-08-17 01:48:25.194 [debug] <0.5017.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
209 2019-08-17 01:48:25.283 [debug] <0.3453.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
210 2019-08-17 01:48:26.599 [debug] <0.3412.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
211 2019-08-17 01:48:29.796 [debug] <0.3990.1411> Supervisor {<0.3990.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.4885.1411> | |
212 2019-08-17 01:48:29.796 [debug] <0.3990.1411> Supervisor {<0.3990.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.4885.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-us-ea...",...}) at pid <0.4442.1411> | |
213 2019-08-17 01:48:29.815 [debug] <0.4833.1411> Supervisor {<0.4833.1411>,rabbit_connection_sup} started rabbit_connection_helper_sup:start_link() at pid <0.5215.1411> | |
214 2019-08-17 01:48:29.815 [debug] <0.4833.1411> Supervisor {<0.4833.1411>,rabbit_connection_sup} started rabbit_reader:start_link(<0.5215.1411>, {acceptor,{0,0,0,0,0,0,0,0},5672}) at pid <0.3598.1411> | |
215 2019-08-17 01:48:29.816 [info] <0.3598.1411> accepting AMQP connection <0.3598.1411> (100.96.11.0:32368 -> 100.96.3.38:5672) | |
216 2019-08-17 01:48:29.885 [debug] <0.4885.1411> Supervisor {<0.4885.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.4442.1411>, <<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>) at pid <0.4641.1411> | |
217 2019-08-17 01:48:29.886 [debug] <0.4885.1411> Supervisor {<0.4885.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.4442.1411>, <<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>, <0.4641.1411>) at pid <0.4184.1411> | |
218 2019-08-17 01:48:29.886 [debug] <0.4885.1411> Supervisor {<0.4885.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943912>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.4442.1411>, <<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>) at pid <0.5446.1411> | |
219 2019-08-17 01:48:29.886 [debug] <0.4885.1411> Supervisor {<0.4885.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943912>, <0.4442.1411>, <0.4184.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>) at pid <0.4789.1411> | |
220 2019-08-17 01:48:29.886 [debug] <0.3598.1411> User 'FEDUSER' authenticated successfully by backend rabbit_auth_backend_internal | |
221 2019-08-17 01:48:29.956 [debug] <0.5215.1411> Supervisor {<0.5215.1411>,rabbit_connection_helper_sup} started rabbit_queue_collector:start_link(<<"100.96.11.0:32368 -> 100.96.3.38:5672">>) at pid <0.3630.1411> | |
222 2019-08-17 01:48:29.956 [debug] <0.5215.1411> Supervisor {<0.5215.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943903>, 10, #Fun<rabbit_reader.34.21003307>, {heartbeat_sender,<<"100.96.11.0:32368 -> 100.96.3.38:5672">>}) at pid <0.3530.1411> | |
223 2019-08-17 01:48:29.957 [debug] <0.5215.1411> Supervisor {<0.5215.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943903>, 10, #Fun<rabbit_reader.35.21003307>, {heartbeat_receiver,<<"100.96.11.0:32368 -> 100.96.3.38:5672">>}) at pid <0.3830.1411> | |
224 2019-08-17 01:48:29.957 [debug] <0.5215.1411> Supervisor {<0.5215.1411>,rabbit_connection_helper_sup} started rabbit_channel_sup_sup:start_link() at pid <0.4412.1411> | |
225 2019-08-17 01:48:29.957 [info] <0.3598.1411> connection <0.3598.1411> (100.96.11.0:32368 -> 100.96.3.38:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
226 2019-08-17 01:48:30.027 [debug] <0.4885.1411> Supervisor {<0.4885.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943912>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>}) at pid <0.4046.1411> | |
227 2019-08-17 01:48:30.027 [debug] <0.4885.1411> Supervisor {<0.4885.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943912>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>}) at pid <0.3235.1411> | |
228 2019-08-17 01:48:30.027 [debug] <0.3707.1411> Supervisor {<0.3707.1411>,rabbit_channel_sup} started rabbit_writer:start_link(#Port<0.12943903>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.3598.1411>, {<<"100.96.11.0:32368 -> 100.96.3.38:5672">>,1}, true) at pid <0.3574.1411> | |
229 2019-08-17 01:48:30.027 [debug] <0.3707.1411> Supervisor {<0.3707.1411>,rabbit_channel_sup} started rabbit_limiter:start_link({<<"100.96.11.0:32368 -> 100.96.3.38:5672">>,1}) at pid <0.4138.1411> | |
230 2019-08-17 01:48:30.027 [debug] <0.3707.1411> Supervisor {<0.3707.1411>,rabbit_channel_sup} started rabbit_channel:start_link(1, <0.3598.1411>, <0.3574.1411>, <0.3598.1411>, <<"100.96.11.0:32368 -> 100.96.3.38:5672">>, rabbit_framing_amqp_0_9_1, {user,<<FEDUSER>>,[administrator],[{rabbit_auth_backend_internal,none}]}, <<"/">>, [{<<"publisher_confirms">>,bool,true},{<<"exchange_exchange_bindings">>,bool,true},{<<"basic.nack">>,...},...], <0.3630.1411>, <0.4138.1411>) at pid <0.3866.1411> | |
231 2019-08-17 01:48:30.097 [debug] <0.3665.1411> Supervisor {<0.3665.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>,1}) at pid <0.4375.1411> | |
232 2019-08-17 01:48:30.097 [debug] <0.3665.1411> Supervisor {<0.3665.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.4442.1411>, 1, <0.4375.1411>, {<<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>,1}) at pid <0.3143.1411> | |
233 2019-08-17 01:48:30.097 [debug] <0.3665.1411> Supervisor {<0.3665.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943912>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.3143.1411>, {<<"client 100.96.3.38:56227 -> 10.9.73.114:5672">>,1}) at pid <0.2865.1411> | |
234 2019-08-17 01:48:30.194 [debug] <0.4404.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
235 2019-08-17 01:48:30.236 [info] <0.3598.1411> closing AMQP connection <0.3598.1411> (100.96.11.0:32368 -> 100.96.3.38:5672, vhost: '/', user: 'FEDUSER') | |
236 2019-08-17 01:48:30.324 [debug] <0.4285.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
237 2019-08-17 01:48:31.600 [debug] <0.4317.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
238 2019-08-17 01:48:31.811 [debug] <0.4135.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
239 2019-08-17 01:48:34.110 [info] <0.3816.1411> Federation exchange 'bob-request-test-1' in vhost '/' received 'basic.cancel' | |
240 2019-08-17 01:48:34.111 [debug] <0.4670.1411> Supervisor {<0.4670.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4294.1411> | |
241 2019-08-17 01:48:34.111 [debug] <0.4670.1411> Supervisor {<0.4670.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3666.1411>, 2, <0.4294.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4460.1411> | |
242 2019-08-17 01:48:34.111 [debug] <0.4670.1411> Supervisor {<0.4670.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943772>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4460.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,2}) at pid <0.4607.1411> | |
243 2019-08-17 01:48:34.581 [debug] <0.3762.1411> Supervisor {<0.3762.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,3}) at pid <0.5058.1411> | |
244 2019-08-17 01:48:34.581 [debug] <0.3762.1411> Supervisor {<0.3762.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3666.1411>, 3, <0.5058.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,3}) at pid <0.4120.1411> | |
245 2019-08-17 01:48:34.581 [debug] <0.3762.1411> Supervisor {<0.3762.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943772>, 3, 131072, rabbit_framing_amqp_0_9_1, <0.4120.1411>, {<<"client 100.96.3.38:60487 -> 10.10.17.48:5672">>,3}) at pid <0.4588.1411> | |
246 2019-08-17 01:48:34.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
247 2019-08-17 01:48:34.672 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
248 2019-08-17 01:48:35.194 [debug] <0.4063.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
249 2019-08-17 01:48:35.214 [error] <0.418.0> Supervisor {<0.418.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqp://FEDUSER:[email protected]">>], | |
250 <<"bob-request-test-1">>,<<"bob-request-test-1">>,1000,1,5,10000, | |
251 10000,false,'on-publish',none, | |
252 <<"bridge_solicited_stream-eu-central-1">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqp://FEDUSER:[email protected]....">>],...},...}) at <0.3816.1411> exit with reason {shutdown,restart} in context child_terminated | |
253 2019-08-17 01:48:35.289 [debug] <0.3399.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
254 2019-08-17 01:48:36.600 [debug] <0.3664.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
255 2019-08-17 01:48:40.194 [debug] <0.4808.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
256 2019-08-17 01:48:40.220 [debug] <0.418.0> Supervisor {<0.418.0>,rabbit_federation_link_sup} started rabbit_federation_exchange_link:start_link({{upstream,[<<"amqp://FEDUSER:[email protected]....">>],...},...}) at pid <0.4176.1411> | |
257 2019-08-17 01:48:40.221 [debug] <0.3606.1411> Supervisor {<0.3606.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.5475.1411> | |
258 2019-08-17 01:48:40.221 [debug] <0.3606.1411> Supervisor {<0.3606.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.5475.1411>, {amqp_params_direct,none,none,<<"/">>,'rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local',...}) at pid <0.4236.1411> | |
259 2019-08-17 01:48:40.222 [debug] <0.5475.1411> Supervisor {<0.5475.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(direct, <0.4236.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.4...">>) at pid <0.5184.1411> | |
260 2019-08-17 01:48:40.222 [debug] <0.5475.1411> Supervisor {<0.5475.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.4236.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.4...">>, <0.5184.1411>) at pid <0.3565.1411> | |
261 2019-08-17 01:48:40.222 [debug] <0.5475.1411> Supervisor {<0.5475.1411>,amqp_connection_type_sup} started rabbit_queue_collector:start_link(<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.4...">>) at pid <0.4869.1411> | |
262 2019-08-17 01:48:40.222 [debug] <0.4776.1411> Supervisor {<0.4776.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.4749.1411> | |
263 2019-08-17 01:48:40.222 [debug] <0.4776.1411> Supervisor {<0.4776.1411>,amqp_channel_sup} started amqp_channel:start_link(direct, <0.4236.1411>, 1, <0.4749.1411>, {<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.3922.1411> | |
264 2019-08-17 01:48:40.222 [debug] <0.4073.1411> Supervisor {<0.4073.1411>,rabbit_channel_sup} started rabbit_limiter:start_link({<<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2...">>,...}) at pid <0.4809.1411> | |
265 2019-08-17 01:48:40.222 [debug] <0.4073.1411> Supervisor {<0.4073.1411>,rabbit_channel_sup} started rabbit_channel:start_link(1, <0.3922.1411>, <0.3922.1411>, <0.4236.1411>, <<"<rabbit@rmq-debug-rabbitmq-ha-0.rmq-debug-rabbitmq-ha-discovery.rmq-debug.svc.cluster.local.2.4...">>, rabbit_framing_amqp_0_9_1, {user,<<"none">>,[],[{rabbit_auth_backend_dummy,none}]}, <<"/">>, [{<<"publisher_confirms">>,bool,true},{<<"exchange_exchange_bindings">>,bool,true},{<<"basic.nack">>,...},...], <0.4869.1411>, <0.4809.1411>) at pid <0.4571.1411> | |
266 2019-08-17 01:48:40.222 [debug] <0.5296.1411> Supervisor {<0.5296.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.5149.1411> | |
267 2019-08-17 01:48:40.223 [debug] <0.5296.1411> Supervisor {<0.5296.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.5149.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-eu-ce...",...}) at pid <0.5237.1411> | |
268 2019-08-17 01:48:40.291 [debug] <0.4082.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
269 2019-08-17 01:48:40.414 [debug] <0.5149.1411> Supervisor {<0.5149.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.5237.1411>, <<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>) at pid <0.4521.1411> | |
270 2019-08-17 01:48:40.414 [debug] <0.5149.1411> Supervisor {<0.5149.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.5237.1411>, <<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>, <0.4521.1411>) at pid <0.4376.1411> | |
271 2019-08-17 01:48:40.415 [debug] <0.5149.1411> Supervisor {<0.5149.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943867>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.5237.1411>, <<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>) at pid <0.4553.1411> | |
272 2019-08-17 01:48:40.415 [debug] <0.5149.1411> Supervisor {<0.5149.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943867>, <0.5237.1411>, <0.4376.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>) at pid <0.3897.1411> | |
273 2019-08-17 01:48:40.729 [debug] <0.5149.1411> Supervisor {<0.5149.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943867>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>}) at pid <0.4572.1411> | |
274 2019-08-17 01:48:40.729 [debug] <0.5149.1411> Supervisor {<0.5149.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943867>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>}) at pid <0.4111.1411> | |
275 2019-08-17 01:48:40.886 [debug] <0.4464.1411> Supervisor {<0.4464.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,1}) at pid <0.3802.1411> | |
276 2019-08-17 01:48:40.886 [debug] <0.4464.1411> Supervisor {<0.4464.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.5237.1411>, 1, <0.3802.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,1}) at pid <0.4224.1411> | |
277 2019-08-17 01:48:40.886 [debug] <0.4464.1411> Supervisor {<0.4464.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943867>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.4224.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,1}) at pid <0.4522.1411> | |
278 2019-08-17 01:48:41.514 [debug] <0.3772.1411> Supervisor {<0.3772.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4504.1411> | |
279 2019-08-17 01:48:41.514 [debug] <0.3772.1411> Supervisor {<0.3772.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.5237.1411>, 2, <0.4504.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4561.1411> | |
280 2019-08-17 01:48:41.514 [debug] <0.3772.1411> Supervisor {<0.3772.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943867>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4561.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4817.1411> | |
281 2019-08-17 01:48:41.600 [debug] <0.4872.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
282 2019-08-17 01:48:41.810 [debug] <0.4539.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
283 2019-08-17 01:48:41.984 [debug] <0.4873.1411> Supervisor {<0.4873.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4190.1411> | |
284 2019-08-17 01:48:41.985 [debug] <0.4873.1411> Supervisor {<0.4873.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.5237.1411>, 2, <0.4190.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.3902.1411> | |
285 2019-08-17 01:48:41.985 [debug] <0.4873.1411> Supervisor {<0.4873.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943867>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.3902.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4448.1411> | |
286 2019-08-17 01:48:42.464 [debug] <0.5370.1411> Supervisor {<0.5370.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.4904.1411> | |
287 2019-08-17 01:48:42.464 [debug] <0.5370.1411> Supervisor {<0.5370.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.4904.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-eu-ce...",...}) at pid <0.4660.1411> | |
288 2019-08-17 01:48:42.636 [debug] <0.4904.1411> Supervisor {<0.4904.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.4660.1411>, <<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>) at pid <0.5148.1411> | |
289 2019-08-17 01:48:42.637 [debug] <0.4904.1411> Supervisor {<0.4904.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.4660.1411>, <<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>, <0.5148.1411>) at pid <0.4430.1411> | |
290 2019-08-17 01:48:42.637 [debug] <0.4904.1411> Supervisor {<0.4904.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943880>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.4660.1411>, <<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>) at pid <0.4903.1411> | |
291 2019-08-17 01:48:42.637 [debug] <0.4904.1411> Supervisor {<0.4904.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943880>, <0.4660.1411>, <0.4430.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>) at pid <0.5778.1411> | |
292 2019-08-17 01:48:42.951 [debug] <0.4904.1411> Supervisor {<0.4904.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943880>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>}) at pid <0.4792.1411> | |
293 2019-08-17 01:48:42.952 [debug] <0.4904.1411> Supervisor {<0.4904.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943880>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>}) at pid <0.4936.1411> | |
294 2019-08-17 01:48:43.108 [debug] <0.3401.1411> Supervisor {<0.3401.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>,1}) at pid <0.3391.1411> | |
295 2019-08-17 01:48:43.108 [debug] <0.3401.1411> Supervisor {<0.3401.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.4660.1411>, 1, <0.3391.1411>, {<<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>,1}) at pid <0.3272.1411> | |
296 2019-08-17 01:48:43.108 [debug] <0.3401.1411> Supervisor {<0.3401.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943880>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.3272.1411>, {<<"client 100.96.3.38:56453 -> 10.10.17.48:5672">>,1}) at pid <0.4055.1411> | |
297 2019-08-17 01:48:43.751 [debug] <0.5067.1411> Supervisor {<0.5067.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4847.1411> | |
298 2019-08-17 01:48:43.751 [debug] <0.5067.1411> Supervisor {<0.5067.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.5237.1411>, 2, <0.4847.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4140.1411> | |
299 2019-08-17 01:48:43.752 [debug] <0.5067.1411> Supervisor {<0.5067.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943867>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4140.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.3624.1411> | |
300 2019-08-17 01:48:43.832 [debug] <0.3548.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
301 2019-08-17 01:48:43.835 [debug] <0.5193.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
302 2019-08-17 01:48:43.843 [debug] <0.4239.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
303 2019-08-17 01:48:43.855 [debug] <0.4444.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
304 2019-08-17 01:48:44.236 [info] <0.4176.1411> Federation exchange 'bob-request-test-1' in vhost '/' connected to exchange 'bob-request-test-1' in vhost '/' on amqp://rmq-fed-debug-eu-central-1.dev.example.net | |
305 2019-08-17 01:48:44.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
306 2019-08-17 01:48:44.672 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
307 2019-08-17 01:48:45.194 [debug] <0.4296.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
308 2019-08-17 01:48:45.289 [debug] <0.3953.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
309 2019-08-17 01:48:46.600 [debug] <0.3246.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
310 2019-08-17 01:48:50.004 [debug] <0.3995.1411> Supervisor {<0.3995.1411>,rabbit_connection_sup} started rabbit_connection_helper_sup:start_link() at pid <0.3675.1411> | |
311 2019-08-17 01:48:50.004 [debug] <0.3995.1411> Supervisor {<0.3995.1411>,rabbit_connection_sup} started rabbit_reader:start_link(<0.3675.1411>, {acceptor,{0,0,0,0,0,0,0,0},5672}) at pid <0.5007.1411> | |
312 2019-08-17 01:48:50.004 [info] <0.5007.1411> accepting AMQP connection <0.5007.1411> (100.96.9.0:39196 -> 100.96.3.38:5672) | |
313 2019-08-17 01:48:50.169 [debug] <0.5007.1411> User 'FEDUSER' authenticated successfully by backend rabbit_auth_backend_internal | |
314 2019-08-17 01:48:50.194 [debug] <0.3886.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
315 2019-08-17 01:48:50.289 [debug] <0.4177.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
316 2019-08-17 01:48:50.326 [debug] <0.3675.1411> Supervisor {<0.3675.1411>,rabbit_connection_helper_sup} started rabbit_queue_collector:start_link(<<"100.96.9.0:39196 -> 100.96.3.38:5672">>) at pid <0.4071.1411> | |
317 2019-08-17 01:48:50.326 [debug] <0.3675.1411> Supervisor {<0.3675.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943942>, 10, #Fun<rabbit_reader.34.21003307>, {heartbeat_sender,<<"100.96.9.0:39196 -> 100.96.3.38:5672">>}) at pid <0.3580.1411> | |
318 2019-08-17 01:48:50.327 [debug] <0.3675.1411> Supervisor {<0.3675.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943942>, 10, #Fun<rabbit_reader.35.21003307>, {heartbeat_receiver,<<"100.96.9.0:39196 -> 100.96.3.38:5672">>}) at pid <0.5110.1411> | |
319 2019-08-17 01:48:50.327 [debug] <0.3675.1411> Supervisor {<0.3675.1411>,rabbit_connection_helper_sup} started rabbit_channel_sup_sup:start_link() at pid <0.4972.1411> | |
320 2019-08-17 01:48:50.327 [info] <0.5007.1411> connection <0.5007.1411> (100.96.9.0:39196 -> 100.96.3.38:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
321 2019-08-17 01:48:50.484 [debug] <0.5174.1411> Supervisor {<0.5174.1411>,rabbit_channel_sup} started rabbit_writer:start_link(#Port<0.12943942>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.5007.1411>, {<<"100.96.9.0:39196 -> 100.96.3.38:5672">>,1}, true) at pid <0.4829.1411> | |
322 2019-08-17 01:48:50.484 [debug] <0.5174.1411> Supervisor {<0.5174.1411>,rabbit_channel_sup} started rabbit_limiter:start_link({<<"100.96.9.0:39196 -> 100.96.3.38:5672">>,1}) at pid <0.4619.1411> | |
323 2019-08-17 01:48:50.484 [debug] <0.5174.1411> Supervisor {<0.5174.1411>,rabbit_channel_sup} started rabbit_channel:start_link(1, <0.5007.1411>, <0.4829.1411>, <0.5007.1411>, <<"100.96.9.0:39196 -> 100.96.3.38:5672">>, rabbit_framing_amqp_0_9_1, {user,<<FEDUSER>>,[administrator],[{rabbit_auth_backend_internal,none}]}, <<"/">>, [{<<"publisher_confirms">>,bool,true},{<<"exchange_exchange_bindings">>,bool,true},{<<"basic.nack">>,...},...], <0.4071.1411>, <0.4619.1411>) at pid <0.4445.1411> | |
324 2019-08-17 01:48:50.954 [info] <0.5007.1411> closing AMQP connection <0.5007.1411> (100.96.9.0:39196 -> 100.96.3.38:5672, vhost: '/', user: 'FEDUSER') | |
325 2019-08-17 01:48:51.601 [debug] <0.4358.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
326 2019-08-17 01:48:51.809 [debug] <0.4597.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
327 2019-08-17 01:48:54.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
328 2019-08-17 01:48:54.673 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
329 2019-08-17 01:48:55.194 [debug] <0.4338.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
330 2019-08-17 01:48:55.289 [debug] <0.4354.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
331 2019-08-17 01:48:56.601 [debug] <0.2663.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
332 2019-08-17 01:49:00.197 [debug] <0.4281.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
333 2019-08-17 01:49:00.291 [debug] <0.3763.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
334 2019-08-17 01:49:00.325 [debug] <0.3782.1411> Supervisor {<0.3782.1411>,amqp_connection_sup} started amqp_connection_type_sup:start_link() at pid <0.3801.1411> | |
335 2019-08-17 01:49:00.325 [debug] <0.3782.1411> Supervisor {<0.3782.1411>,amqp_connection_sup} started amqp_gen_connection:start_link(<0.3801.1411>, {amqp_params_network,<<FEDUSER>>,<<"FEDPASS">>,<<"/">>,"rmq-fed-debug-us-ea...",...}) at pid <0.3423.1411> | |
336 2019-08-17 01:49:00.337 [debug] <0.4169.1411> Supervisor {<0.4169.1411>,rabbit_connection_sup} started rabbit_connection_helper_sup:start_link() at pid <0.4853.1411> | |
337 2019-08-17 01:49:00.337 [debug] <0.4169.1411> Supervisor {<0.4169.1411>,rabbit_connection_sup} started rabbit_reader:start_link(<0.4853.1411>, {acceptor,{0,0,0,0,0,0,0,0},5672}) at pid <0.4995.1411> | |
338 2019-08-17 01:49:00.337 [info] <0.4995.1411> accepting AMQP connection <0.4995.1411> (100.96.7.0:35226 -> 100.96.3.38:5672) | |
339 2019-08-17 01:49:00.406 [debug] <0.3801.1411> Supervisor {<0.3801.1411>,amqp_connection_type_sup} started amqp_channel_sup_sup:start_link(network, <0.3423.1411>, <<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>) at pid <0.3976.1411> | |
340 2019-08-17 01:49:00.406 [debug] <0.3801.1411> Supervisor {<0.3801.1411>,amqp_connection_type_sup} started amqp_channels_manager:start_link(<0.3423.1411>, <<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>, <0.3976.1411>) at pid <0.3728.1411> | |
341 2019-08-17 01:49:00.407 [debug] <0.3801.1411> Supervisor {<0.3801.1411>,amqp_connection_type_sup} started rabbit_writer:start_link(#Port<0.12943904>, 0, 4096, rabbit_framing_amqp_0_9_1, <0.3423.1411>, <<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>) at pid <0.5243.1411> | |
342 2019-08-17 01:49:00.407 [debug] <0.3801.1411> Supervisor {<0.3801.1411>,amqp_connection_type_sup} started amqp_main_reader:start_link(#Port<0.12943904>, <0.3423.1411>, <0.3728.1411>, {method,rabbit_framing_amqp_0_9_1}, <<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>) at pid <0.4765.1411> | |
343 2019-08-17 01:49:00.408 [debug] <0.4995.1411> User 'FEDUSER' authenticated successfully by backend rabbit_auth_backend_internal | |
344 2019-08-17 01:49:00.478 [debug] <0.4853.1411> Supervisor {<0.4853.1411>,rabbit_connection_helper_sup} started rabbit_queue_collector:start_link(<<"100.96.7.0:35226 -> 100.96.3.38:5672">>) at pid <0.4488.1411> | |
345 2019-08-17 01:49:00.479 [debug] <0.4853.1411> Supervisor {<0.4853.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943894>, 10, #Fun<rabbit_reader.34.21003307>, {heartbeat_sender,<<"100.96.7.0:35226 -> 100.96.3.38:5672">>}) at pid <0.4113.1411> | |
346 2019-08-17 01:49:00.479 [debug] <0.4853.1411> Supervisor {<0.4853.1411>,rabbit_connection_helper_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943894>, 10, #Fun<rabbit_reader.35.21003307>, {heartbeat_receiver,<<"100.96.7.0:35226 -> 100.96.3.38:5672">>}) at pid <0.3889.1411> | |
347 2019-08-17 01:49:00.479 [debug] <0.4853.1411> Supervisor {<0.4853.1411>,rabbit_connection_helper_sup} started rabbit_channel_sup_sup:start_link() at pid <0.4587.1411> | |
348 2019-08-17 01:49:00.479 [info] <0.4995.1411> connection <0.4995.1411> (100.96.7.0:35226 -> 100.96.3.38:5672): user 'FEDUSER' authenticated and granted access to vhost '/' | |
349 2019-08-17 01:49:00.549 [debug] <0.5024.1411> Supervisor {<0.5024.1411>,rabbit_channel_sup} started rabbit_writer:start_link(#Port<0.12943894>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.4995.1411>, {<<"100.96.7.0:35226 -> 100.96.3.38:5672">>,1}, true) at pid <0.4745.1411> | |
350 2019-08-17 01:49:00.549 [debug] <0.3801.1411> Supervisor {<0.3801.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_sender(#Port<0.12943904>, 10, #Fun<amqp_network_connection.2.82346747>, {heartbeat_sender,<<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>}) at pid <0.3895.1411> | |
351 2019-08-17 01:49:00.550 [debug] <0.5024.1411> Supervisor {<0.5024.1411>,rabbit_channel_sup} started rabbit_limiter:start_link({<<"100.96.7.0:35226 -> 100.96.3.38:5672">>,1}) at pid <0.5076.1411> | |
352 2019-08-17 01:49:00.550 [debug] <0.3801.1411> Supervisor {<0.3801.1411>,amqp_connection_type_sup} started rabbit_heartbeat:start_heartbeat_receiver(#Port<0.12943904>, 10, #Fun<amqp_network_connection.3.82346747>, {heartbeat_receiver,<<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>}) at pid <0.3909.1411> | |
353 2019-08-17 01:49:00.550 [debug] <0.5024.1411> Supervisor {<0.5024.1411>,rabbit_channel_sup} started rabbit_channel:start_link(1, <0.4995.1411>, <0.4745.1411>, <0.4995.1411>, <<"100.96.7.0:35226 -> 100.96.3.38:5672">>, rabbit_framing_amqp_0_9_1, {user,<<FEDUSER>>,[administrator],[{rabbit_auth_backend_internal,none}]}, <<"/">>, [{<<"publisher_confirms">>,bool,true},{<<"exchange_exchange_bindings">>,bool,true},{<<"basic.nack">>,...},...], <0.4488.1411>, <0.5076.1411>) at pid <0.4417.1411> | |
354 2019-08-17 01:49:00.620 [debug] <0.4647.1411> Supervisor {<0.4647.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>,1}) at pid <0.3587.1411> | |
355 2019-08-17 01:49:00.620 [debug] <0.4647.1411> Supervisor {<0.4647.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.3423.1411>, 1, <0.3587.1411>, {<<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>,1}) at pid <0.4357.1411> | |
356 2019-08-17 01:49:00.620 [debug] <0.4647.1411> Supervisor {<0.4647.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943904>, 1, 131072, rabbit_framing_amqp_0_9_1, <0.4357.1411>, {<<"client 100.96.3.38:56273 -> 10.9.73.114:5672">>,1}) at pid <0.5144.1411> | |
357 2019-08-17 01:49:00.760 [info] <0.4995.1411> closing AMQP connection <0.4995.1411> (100.96.7.0:35226 -> 100.96.3.38:5672, vhost: '/', user: 'FEDUSER') | |
358 2019-08-17 01:49:01.598 [debug] <0.4455.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
359 2019-08-17 01:49:01.809 [debug] <0.4070.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
360 2019-08-17 01:49:04.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
361 2019-08-17 01:49:04.673 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
362 2019-08-17 01:49:05.194 [debug] <0.3999.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
363 2019-08-17 01:49:05.289 [debug] <0.4342.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
364 2019-08-17 01:49:06.601 [debug] <0.5319.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
365 2019-08-17 01:49:10.189 [debug] <0.5032.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
366 2019-08-17 01:49:10.287 [debug] <0.4688.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
367 2019-08-17 01:49:11.601 [debug] <0.2765.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
368 2019-08-17 01:49:11.808 [debug] <0.4705.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
369 2019-08-17 01:49:12.853 [info] <0.4176.1411> Federation exchange 'bob-request-test-1' in vhost '/' received 'basic.cancel' | |
370 2019-08-17 01:49:12.854 [debug] <0.4325.1411> Supervisor {<0.4325.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4383.1411> | |
371 2019-08-17 01:49:12.854 [debug] <0.4325.1411> Supervisor {<0.4325.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.5237.1411>, 2, <0.4383.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4876.1411> | |
372 2019-08-17 01:49:12.854 [debug] <0.4325.1411> Supervisor {<0.4325.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943867>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4876.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.3820.1411> | |
373 2019-08-17 01:49:13.324 [debug] <0.5378.1411> Supervisor {<0.5378.1411>,amqp_channel_sup} started amqp_gen_consumer:start_link(amqp_selective_consumer, [], {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4519.1411> | |
374 2019-08-17 01:49:13.324 [debug] <0.5378.1411> Supervisor {<0.5378.1411>,amqp_channel_sup} started amqp_channel:start_link(network, <0.5237.1411>, 2, <0.4519.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.4828.1411> | |
375 2019-08-17 01:49:13.324 [debug] <0.5378.1411> Supervisor {<0.5378.1411>,amqp_channel_sup} started rabbit_writer:start_link(#Port<0.12943867>, 2, 131072, rabbit_framing_amqp_0_9_1, <0.4828.1411>, {<<"client 100.96.3.38:40745 -> 10.10.17.110:5672">>,2}) at pid <0.5442.1411> | |
376 2019-08-17 01:49:13.833 [debug] <0.5911.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
377 2019-08-17 01:49:13.837 [debug] <0.5653.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
378 2019-08-17 01:49:13.845 [debug] <0.4912.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
379 2019-08-17 01:49:13.856 [debug] <0.5037.1411> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal | |
380 2019-08-17 01:49:13.958 [error] <0.418.0> Supervisor {<0.418.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqp://FEDUSER:[email protected]">>], | |
381 <<"bob-request-test-1">>,<<"bob-request-test-1">>,1000,1,5,10000, | |
382 10000,false,'on-publish',none, | |
383 <<"bridge_solicited_stream-eu-central-1">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqp://FEDUSER:[email protected]....">>],...},...}) at <0.4176.1411> exit with reason {shutdown,restart} in context child_terminated | |
384 2019-08-17 01:49:14.671 [debug] <0.523.0> Peer discovery: checking for partitioned nodes to clean up. | |
385 2019-08-17 01:49:14.672 [debug] <0.523.0> Peer discovery: all known cluster nodes are up. | |
386 2019-08-17 01:49:15.192 [debug] <0.4854.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
387 2019-08-17 01:49:15.284 [debug] <0.4002.1411> User 'sysdig' authenticated successfully by backend rabbit_auth_backend_internal | |
388 2019-08-17 01:49:16.601 [debug] <0.3388.1411> User 'management' authenticated successfully by backend rabbit_auth_backend_internal | |
389 ^C |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment