Skip to content

Instantly share code, notes, and snippets.

@dav-rob
Created May 27, 2011 16:46
Show Gist options
  • Save dav-rob/995655 to your computer and use it in GitHub Desktop.
Save dav-rob/995655 to your computer and use it in GitHub Desktop.
[2011-05-27 06:55:40,003][INFO ][node ] [Lila Cheney] {elasticsearch/0.15.2}[25473]: initializing ...
[2011-05-27 06:55:40,052][INFO ][plugins ] [Lila Cheney] loaded []
[2011-05-27 06:55:47,500][DEBUG][discovery.zen.ping.unicast] [Lila Cheney] using initial hosts [166.15.121.116:9300, 166.15.121.116:9301, 169.49.110.161:9300, 169.49.110.161:9301, 169.49.110.161:9302, 169.49.110.161:9303, 169.49.110.161:9304, 169.49.110.161:9305, 169.49.110.161:9306, 169.49.110.161:9307, 169.49.110.161:9308, 169.49.110.161:9309, 169.49.110.161:9310]
[2011-05-27 06:55:47,522][DEBUG][discovery.zen ] [Lila Cheney] using initial_ping_timeout [3s]
[2011-05-27 06:55:47,528][DEBUG][discovery.zen.fd ] [Lila Cheney] [master] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2011-05-27 06:55:47,541][DEBUG][discovery.zen.fd ] [Lila Cheney] [node ] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2011-05-27 06:55:49,302][INFO ][node ] [Lila Cheney] {elasticsearch/0.15.2}[25473]: initialized
[2011-05-27 06:55:49,304][INFO ][node ] [Lila Cheney] {elasticsearch/0.15.2}[25473]: starting ...
[2011-05-27 06:55:49,770][INFO ][transport ] [Lila Cheney] bound_address {inet[/169.49.110.161:9300]}, publish_address {inet[/169.49.110.161:9300]}
[2011-05-27 06:55:49,827][TRACE][discovery ] [Lila Cheney] waiting for 30s for the initial state to be set by the discovery
[2011-05-27 06:55:50,023][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_1#][inet[/166.15.121.116:9300]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/166.15.121.116:9300]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
[2011-05-27 06:55:50,126][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_2#][inet[/166.15.121.116:9301]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/166.15.121.116:9301]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
[2011-05-27 06:55:50,180][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] connecting to [Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]], disconnect[false]
[2011-05-27 06:55:50,295][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_4#][inet[/169.49.110.161:9301]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9301]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 13 more
[2011-05-27 06:55:50,313][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_5#][inet[/169.49.110.161:9302]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9302]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 13 more
[2011-05-27 06:55:50,332][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_6#][inet[/169.49.110.161:9303]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9303]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 13 more
[2011-05-27 06:55:50,340][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] received response from [Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]]: [ping_response{target [[Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]]], master [null], cluster_name[CMTQA1]}, ping_response{target [[Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]]], master [null], cluster_name[CMTQA1]}]
[2011-05-27 06:55:50,349][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_7#][inet[/169.49.110.161:9304]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9304]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 13 more
[2011-05-27 06:55:53,747][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_8#][inet[/169.49.110.161:9305]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9305]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
[2011-05-27 06:55:53,768][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_9#][inet[/169.49.110.161:9306]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9306]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 13 more
[2011-05-27 06:55:53,784][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_10#][inet[/169.49.110.161:9307]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9307]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 13 more
[2011-05-27 06:55:53,799][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_11#][inet[/169.49.110.161:9308]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9308]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 13 more
[2011-05-27 06:55:57,197][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_12#][inet[/169.49.110.161:9309]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9309]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
[2011-05-27 06:55:57,218][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_13#][inet[/169.49.110.161:9310]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9310]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:154)
at org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:129)
at org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:111)
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:483)
at org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:242)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:62)
at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:227)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 13 more
[2011-05-27 06:56:00,320][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_1#][inet[/166.15.121.116:9300]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/166.15.121.116:9300]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
[2011-05-27 06:56:00,411][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_2#][inet[/166.15.121.116:9301]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/166.15.121.116:9301]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
[2011-05-27 06:56:00,414][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] connecting to [Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]], disconnect[false]
[2011-05-27 06:56:00,419][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] received response from [Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]]: [ping_response{target [[Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]]], master [null], cluster_name[CMTQA1]}, ping_response{target [[Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]]], master [null], cluster_name[CMTQA1]}]
[2011-05-27 06:56:00,428][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_4#][inet[/169.49.110.161:9301]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9301]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 8 more
[2011-05-27 06:56:00,442][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_5#][inet[/169.49.110.161:9302]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9302]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 8 more
[2011-05-27 06:56:00,497][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_6#][inet[/169.49.110.161:9303]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9303]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 8 more
[2011-05-27 06:56:00,510][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_7#][inet[/169.49.110.161:9304]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9304]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 8 more
[2011-05-27 06:56:03,896][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_8#][inet[/169.49.110.161:9305]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9305]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
[2011-05-27 06:56:03,913][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_9#][inet[/169.49.110.161:9306]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9306]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 8 more
[2011-05-27 06:56:03,927][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_10#][inet[/169.49.110.161:9307]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9307]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 8 more
[2011-05-27 06:56:03,940][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_11#][inet[/169.49.110.161:9308]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9308]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 8 more
[2011-05-27 06:56:07,336][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_12#][inet[/169.49.110.161:9309]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9309]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
[2011-05-27 06:56:07,354][TRACE][discovery.zen.ping.unicast] [Lila Cheney] [1] failed to connect to [#zen_unicast_13#][inet[/169.49.110.161:9310]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/169.49.110.161:9310]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:512)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:473)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:126)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:189)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$000(UnicastZenPing.java:59)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:157)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:555)
at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:541)
at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:218)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:499)
... 8 more
[2011-05-27 06:56:07,358][DEBUG][discovery.zen ] [Lila Cheney] ping responses: {none}
[2011-05-27 06:56:07,372][INFO ][cluster.service ] [Lila Cheney] new_master [Lila Cheney][O_p4H9a4RGy15DnKQAAIew][inet[/169.49.110.161:9300]], reason: zen-disco-join (elected_as_master)
[2011-05-27 06:56:07,385][TRACE][discovery ] [Lila Cheney] initial state set from discovery
[2011-05-27 06:56:07,385][INFO ][discovery ] [Lila Cheney] CMTQA1/O_p4H9a4RGy15DnKQAAIew
[2011-05-27 06:56:07,487][INFO ][http ] [Lila Cheney] bound_address {inet[/169.49.110.161:9200]}, publish_address {inet[/169.49.110.161:9200]}
[2011-05-27 06:56:07,490][INFO ][node ] [Lila Cheney] {elasticsearch/0.15.2}[25473]: started
[2011-05-27 07:03:02,430][INFO ][cluster.service ] [Lila Cheney] added {[Colossus][YIQOzlTzTwidgSoaOteHXQ][inet[/169.49.110.161:9301]]{client=true, data=false},}, reason: zen-disco-receive(join from node[[Colossus][YIQOzlTzTwidgSoaOteHXQ][inet[/169.49.110.161:9301]]{client=true, data=false}])
[2011-05-27 07:03:02,624][DEBUG][discovery.zen.publish ] [Lila Cheney] failed to send cluster state to [[Colossus][YIQOzlTzTwidgSoaOteHXQ][inet[/169.49.110.161:9301]]{client=true, data=false}], should be detected as failed soon...
org.elasticsearch.transport.RemoteTransportException: [Colossus][inet[/169.49.110.161:9301]][discovery/zen/publish]
Caused by: java.io.EOFException
at org.elasticsearch.common.io.stream.LZFStreamInput.readByte(LZFStreamInput.java:123)
at org.elasticsearch.common.io.stream.HandlesStreamInput.readByte(HandlesStreamInput.java:80)
at org.elasticsearch.common.io.stream.StreamInput.readInt(StreamInput.java:64)
at org.elasticsearch.common.io.stream.StreamInput.readLong(StreamInput.java:95)
at org.elasticsearch.cluster.ClusterState$Builder.readFrom(ClusterState.java:245)
at org.elasticsearch.discovery.zen.publish.PublishClusterStateAction$PublishClusterStateRequestHandler.messageReceived(PublishClusterStateAction.java:127)
at org.elasticsearch.discovery.zen.publish.PublishClusterStateAction$PublishClusterStateRequestHandler.messageReceived(PublishClusterStateAction.java:117)
at org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:183)
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:85)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:302)
at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:51)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:540)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:274)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:261)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:349)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:280)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:200)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
[2011-05-27 07:03:48,323][TRACE][discovery.zen.fd ] [Lila Cheney] [node ] [[Colossus][YIQOzlTzTwidgSoaOteHXQ][inet[/169.49.110.161:9301]]{client=true, data=false}] transport disconnected (with verified connect)
[2011-05-27 07:03:48,333][INFO ][cluster.service ] [Lila Cheney] removed {[Colossus][YIQOzlTzTwidgSoaOteHXQ][inet[/169.49.110.161:9301]]{client=true, data=false},}, reason: zen-disco-node_failed([Colossus][YIQOzlTzTwidgSoaOteHXQ][inet[/169.49.110.161:9301]]{client=true, data=false}), reason transport disconnected (with verified connect)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment