Skip to content

Instantly share code, notes, and snippets.

@johnl
Last active December 25, 2015 17:59
Show Gist options
  • Save johnl/7016698 to your computer and use it in GitHub Desktop.
Save johnl/7016698 to your computer and use it in GitHub Desktop.
coreos etcd cluster bootstrap failure
-- Logs begin at Wed 2013-10-16 23:08:37 UTC, end at Wed 2013-10-16 23:12:45 UTC. --
Oct 16 23:09:05 srv-5ai7t systemd[1]: Starting etcd...
Oct 16 23:09:05 srv-5ai7t systemd[1]: Started etcd.
Oct 16 23:09:05 srv-5ai7t etcd-bootstrap[649]: trying 10.242.48.122:7001
Oct 16 23:09:05 srv-5ai7t etcd-bootstrap[649]: + exec /usr/bin/etcd -d /var/lib/etcd -f -cl 0.0.0.0 -n 10.241.16.190 -c 10.241.16.190:4001 -s 10.241.16.190:7001 -sl 0.0.0.0 -C 10.242.48.122:7001
Oct 16 23:09:05 srv-5ai7t etcd-bootstrap[649]: [etcd] Oct 16 23:09:05.307 INFO | Wrote node configuration to '/var/lib/etcd/info'
Oct 16 23:09:05 srv-5ai7t etcd[649]: Wrote node configuration to '/var/lib/etcd/info'
Oct 16 23:09:05 srv-5ai7t etcd-bootstrap[649]: [etcd] Oct 16 23:09:05.312 INFO | etcd server [name 10.241.16.190, listen on 0.0.0.0:4001, advertised url http://10.241.16.190:4001]
Oct 16 23:09:05 srv-5ai7t etcd[649]: etcd server [name 10.241.16.190, listen on 0.0.0.0:4001, advertised url http://10.241.16.190:4001]
Oct 16 23:09:05 srv-5ai7t etcd-bootstrap[649]: [etcd] Oct 16 23:09:05.312 INFO | raft server [name 10.241.16.190, listen on 0.0.0.0:7001, advertised url http://10.241.16.190:7001]
Oct 16 23:09:05 srv-5ai7t etcd[649]: raft server [name 10.241.16.190, listen on 0.0.0.0:7001, advertised url http://10.241.16.190:7001]
cat /var/lib/etcd/conf
{"commitIndex":4,"peers":[{"name":"10.242.48.122","connectionString":""},{"name":"10.241.24.130","connectionString":""}]}
pgrep -lf etcd
649 /usr/bin/etcd -d /var/lib/etcd -f -cl 0.0.0.0 -n 10.241.16.190 -c 10.241.16.190:4001 -s 10.241.16.190:7001 -sl 0.0.0.0 -C 10.242.48.122:7001
curl -L http://127.0.0.1:4001/v1/keys/message -d value="Hello"
{"action":"SET","key":"/message","prevValue":"Hello","value":"Hello","index":10}
cat /run/etcd/bootstrap.config
10.242.48.122
10.241.16.190
-- Logs begin at Wed 2013-10-16 23:08:43 UTC, end at Wed 2013-10-16 23:16:04 UTC. --
Oct 16 23:09:15 srv-al5ca systemd[1]: Starting etcd...
Oct 16 23:09:15 srv-al5ca systemd[1]: Started etcd.
Oct 16 23:09:15 srv-al5ca etcd-bootstrap[699]: trying 10.242.48.122:7001,10.241.16.190:7001
Oct 16 23:09:15 srv-al5ca etcd-bootstrap[699]: + exec /usr/bin/etcd -d /var/lib/etcd -f -cl 0.0.0.0 -n 10.241.24.130 -c 10.241.24.130:4001 -s 10.241.24.130:7001 -sl 0.0.0.0 -C 10.242.48.122:7001,10.2
Oct 16 23:09:15 srv-al5ca etcd-bootstrap[699]: [etcd] Oct 16 23:09:15.474 INFO | Wrote node configuration to '/var/lib/etcd/info'
Oct 16 23:09:15 srv-al5ca etcd[699]: Wrote node configuration to '/var/lib/etcd/info'
Oct 16 23:09:15 srv-al5ca etcd-bootstrap[699]: [etcd] Oct 16 23:09:15.479 INFO | etcd server [name 10.241.24.130, listen on 0.0.0.0:4001, advertised url http://10.241.24.130:4001]
Oct 16 23:09:15 srv-al5ca etcd-bootstrap[699]: [etcd] Oct 16 23:09:15.479 INFO | raft server [name 10.241.24.130, listen on 0.0.0.0:7001, advertised url http://10.241.24.130:7001]
Oct 16 23:09:15 srv-al5ca etcd[699]: etcd server [name 10.241.24.130, listen on 0.0.0.0:4001, advertised url http://10.241.24.130:4001]
Oct 16 23:09:15 srv-al5ca etcd[699]: raft server [name 10.241.24.130, listen on 0.0.0.0:7001, advertised url http://10.241.24.130:7001]
cat /var/lib/etcd/conf
cat: /var/lib/etcd/conf: No such file or directory
pgrep -lf etcd
699 /usr/bin/etcd -d /var/lib/etcd -f -cl 0.0.0.0 -n 10.241.24.130 -c 10.241.24.130:4001 -s 10.241.24.130:7001 -sl 0.0.0.0 -C 10.242.48.122:7001,10.241.16.190:7001
curl -L http://127.0.0.1:4001/v1/keys/message -d value="Hello"
{"errorCode":300,"message":"Raft Internal Error"}
cat /run/etcd/bootstrap.config
10.242.48.122
10.241.16.190
10.241.24.130
-- Logs begin at Wed 2013-10-16 23:08:34 UTC, end at Wed 2013-10-16 23:12:56 UTC. --
Oct 16 23:08:42 srv-b8nz7 systemd[1]: Starting etcd...
Oct 16 23:08:42 srv-b8nz7 systemd[1]: Started etcd.
Oct 16 23:08:42 srv-b8nz7 etcd-bootstrap[634]: becoming master...
Oct 16 23:08:43 srv-b8nz7 etcd-bootstrap[634]: [etcd] Oct 16 23:08:43.175 INFO | Wrote node configuration to '/var/lib/etcd/info'
Oct 16 23:08:43 srv-b8nz7 etcd[634]: Wrote node configuration to '/var/lib/etcd/info'
Oct 16 23:08:43 srv-b8nz7 etcd-bootstrap[634]: [etcd] Oct 16 23:08:43.177 INFO | etcd server [name 10.242.48.122, listen on 0.0.0.0:4001, advertised url http://10.242.48.122:4001]
Oct 16 23:08:43 srv-b8nz7 etcd[634]: etcd server [name 10.242.48.122, listen on 0.0.0.0:4001, advertised url http://10.242.48.122:4001]
Oct 16 23:08:43 srv-b8nz7 etcd[634]: raft server [name 10.242.48.122, listen on 0.0.0.0:7001, advertised url http://10.242.48.122:7001]
Oct 16 23:08:43 srv-b8nz7 etcd-bootstrap[634]: [etcd] Oct 16 23:08:43.178 INFO | raft server [name 10.242.48.122, listen on 0.0.0.0:7001, advertised url http://10.242.48.122:7001]
cat /var/lib/etcd/conf
{"commitIndex":4,"peers":[{"name":"10.241.16.190","connectionString":""},{"name":"10.241.24.130","connectionString":""}]}
pgrep -lf etcd
634 /usr/bin/etcd -d /var/lib/etcd -f -cl 0.0.0.0 -n 10.242.48.122 -c 10.242.48.122:4001 -s 10.242.48.122:7001 -sl 0.0.0.0
curl -L http://127.0.0.1:4001/v1/keys/message -d value="Hello"
{"action":"SET","key":"/message","prevValue":"Hello","value":"Hello","index":9}
cat /run/etcd/bootstrap.config
10.242.48.122
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment