-
-
Save bcg/1993328 to your computer and use it in GitHub Desktop.
Riak all_nodes_down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ curl -v -d 'this is a test' -H "Content-Type: text/plain" http://10.32.36.92:8098/riak/test | |
* About to connect() to 10.32.36.92 port 8098 (#0) | |
* Trying 10.32.36.92... connected | |
* Connected to 10.32.36.92 (10.32.36.92) port 8098 (#0) | |
> POST /riak/test HTTP/1.1 | |
> User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15 | |
> Host: 10.32.36.92:8098 | |
> Accept: */* | |
> Content-Type: text/plain | |
> Content-Length: 14 | |
> | |
< HTTP/1.1 500 Internal Server Error | |
< Vary: Accept-Encoding | |
< Server: MochiWeb/1.1 WebMachine/1.9.0 (someone had painted it blue) | |
< Location: /riak/test/Mobuxmt6vIikc40fFnwatTLSS2q | |
< Date: Wed, 07 Mar 2012 14:05:04 GMT | |
< Content-Type: text/plain | |
< Content-Length: 22 | |
< | |
Error: | |
all_nodes_down | |
* Connection #0 to host 10.32.36.92 left intact | |
* Closing connection #0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
([email protected])2> riak_core_vnode_manager:force_handoffs(). | |
ok | |
([email protected])3> 08:59:55.728 [error] gen_server riak_kv_vnode_master terminated with reason: no match of right hand value {error,{{badmatch,{error,enoent}},[{riak_kv_vnode,update_vnode_status,2},{riak_kv_vnode,init,1},{riak_core_vnode,init,1},{gen_fsm,init_it,6},{proc_lib,init_p_do_apply,3}]}} in riak_core_vnode_master:get_vnode/2 | |
08:59:55.730 [error] CRASH REPORT Process <0.8550.6> with 0 neighbours crashed with reason: no match of right hand value {error,enoent} in riak_kv_vnode:update_vnode_status/2 | |
08:59:55.732 [error] CRASH REPORT Process riak_kv_vnode_master with 0 neighbours crashed with reason: no match of right hand value {error,{{badmatch,{error,enoent}},[{riak_kv_vnode,update_vnode_status,2},{riak_kv_vnode,init,1},{riak_core_vnode,init,1},{gen_fsm,init_it,6},{proc_lib,init_p_do_apply,3}]}} in riak_core_vnode_master:get_vnode/2 | |
08:59:55.734 [error] Supervisor riak_core_vnode_sup had child undefined started with {riak_core_vnode,start_link,undefined} at <0.8550.6> exit with reason no match of right hand value {error,enoent} in riak_kv_vnode:update_vnode_status/2 in context child_terminated | |
08:59:55.736 [error] Supervisor riak_kv_sup had child riak_kv_vnode_master started with riak_core_vnode_master:start_link(riak_kv_vnode, riak_kv_legacy_vnode) at <0.8543.6> exit with reason no match of right hand value {error,{{badmatch,{error,enoent}},[{riak_kv_vnode,update_vnode_status,2},{riak_kv_vnode,init,1},{riak_core_vnode,init,1},{gen_fsm,init_it,6},{proc_lib,init_p_do_apply,3}]}} in riak_core_vnode_master:get_vnode/2 in context child_terminated |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
([email protected])1> riak_core_ring_manager:force_update(). | |
ok | |
([email protected])2> 08:57:01.132 [error] CRASH REPORT Process <0.8493.6> with 0 neighbours crashed with reason: no match of right hand value {error,enoent} in riak_kv_vnode:update_vnode_status/2 | |
08:57:01.133 [error] Supervisor riak_core_vnode_sup had child undefined started with {riak_core_vnode,start_link,undefined} at <0.8493.6> exit with reason no match of right hand value {error,enoent} in riak_kv_vnode:update_vnode_status/2 in context child_terminated | |
08:57:01.134 [error] gen_server riak_kv_vnode_master terminated with reason: no match of right hand value {error,{{badmatch,{error,enoent}},[{riak_kv_vnode,update_vnode_status,2},{riak_kv_vnode,init,1},{riak_core_vnode,init,1},{gen_fsm,init_it,6},{proc_lib,init_p_do_apply,3}]}} in riak_core_vnode_master:get_vnode/2 | |
08:57:01.136 [error] CRASH REPORT Process riak_kv_vnode_master with 0 neighbours crashed with reason: no match of right hand value {error,{{badmatch,{error,enoent}},[{riak_kv_vnode,update_vnode_status,2},{riak_kv_vnode,init,1},{riak_core_vnode,init,1},{gen_fsm,init_it,6},{proc_lib,init_p_do_apply,3}]}} in riak_core_vnode_master:get_vnode/2 | |
... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ sudo riak-admin member_status | |
Attempting to restart script through sudo -u riak | |
================================= Membership ================================== | |
Status Ring Pending Node | |
------------------------------------------------------------------------------- | |
valid 1.6% 50.0% '[email protected]' | |
valid 0.0% 0.0% '[email protected]' | |
valid 0.0% 0.0% '[email protected]' | |
valid 0.0% 0.0% '[email protected]' | |
valid 98.4% 50.0% '[email protected]' | |
valid 0.0% 0.0% '[email protected]' | |
------------------------------------------------------------------------------- | |
Valid:6 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ sudo riak-admin ring_status | |
Attempting to restart script through sudo -u riak | |
================================== Claimant =================================== | |
Claimant: '[email protected]' | |
Status: up | |
Ring Ready: true | |
============================== Ownership Handoff ============================== | |
Owner: [email protected] | |
Next Owner: [email protected] | |
Index: 0 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 22835963083295358096932575511191922182123945984 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 91343852333181432387730302044767688728495783936 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 114179815416476790484662877555959610910619729920 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 182687704666362864775460604089535377456991567872 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 205523667749658222872393179600727299639115513856 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 274031556999544297163190906134303066185487351808 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 296867520082839655260123481645494988367611297792 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 388211372416021087647853783690262677096107081728 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 456719261665907161938651510223838443642478919680 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 479555224749202520035584085735030365824602865664 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 548063113999088594326381812268606132370974703616 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 570899077082383952423314387779798054553098649600 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 639406966332270026714112114313373821099470487552 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 662242929415565384811044689824565743281594433536 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 730750818665451459101842416358141509827966271488 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 753586781748746817198774991869333432010090217472 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 822094670998632891489572718402909198556462055424 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 844930634081928249586505293914101120738586001408 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 913438523331814323877303020447676887284957839360 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 936274486415109681974235595958868809467081785344 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1004782375664995756265033322492444576013453623296 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1027618338748291114361965898003636498195577569280 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1096126227998177188652763624537212264741949407232 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1118962191081472546749696200048404186924073353216 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1187470080331358621040493926581979953470445191168 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1210306043414653979137426502093171875652569137152 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1278813932664540053428224228626747642198940975104 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1301649895747835411525156804137939564381064921088 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1370157784997721485815954530671515330927436759040 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
Index: 1392993748081016843912887106182707253109560705024 | |
Waiting on: [riak_kv_vnode] | |
Complete: [riak_pipe_vnode] | |
------------------------------------------------------------------------------- | |
============================== Unreachable Nodes ============================== | |
All nodes are up and reachable |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment