Recovering a node from a backup using https://github.com/coreos/etcd/pull/5397

node-recovery-from-backup.md

Prerequisites

• etcdctl from etcd-io/etcd#5397
• a running etcd cluster
• broken member is removed

Recovery Procedure

Start with a running cluster (a 1-node cluster is enough), e.g.:

$ etcd --name 0 --listen-peer-urls=http://localhost:2380 --listen-client-urls=http://localhost:2379 -advertise-client-urls=http://localhost:2379 --initial-cluster-state=new

Make a (live-) backup of that node:

$ etcdctl backup --backup-dir backup.etcd --data-dir 0.etcd --keep-cluster-id

The flag --keep-cluster-id is only available with the mentioned PR.

Add the new node (remove an old member with the same address before if necessary):

$ etcdctl member add 1 http://localhost:12380

Copy the node-id and use it to restore the data from the backup snapshot:

$ etcdctl backup --backup-dir 1.etcd --data-dir backup.etcd --keep-cluster-id --node-id 368ff4e7245ee2e3

Start up the new node:

$ etcd --name 1 --initial-cluster=0=http://localhost:7001,0=http://localhost:02380,1=http://localhost:12380 --initial-advertise-peer-urls=http://localhost:12380 --listen-peer-urls=http://localhost:12380 --listen-client-urls=http://localhost:12379 -advertise-client-urls=http://localhost:12379 --initial-cluster-state=existing

The node should join the cluster and eventually the cluster should be healthy with two nodes:

$ etcdctl cluster-health

The new node should show the data from the time after the backup has been made and before the new node joined.

$ etcdctl --peers http://localhost:12379 --no-sync ls