Skip to content

Instantly share code, notes, and snippets.

@micahlee
Created October 18, 2019 21:04
Show Gist options
  • Save micahlee/33d02ad7c7aa459fe61a47b6a8f67307 to your computer and use it in GitHub Desktop.
Save micahlee/33d02ad7c7aa459fe61a47b6a8f67307 to your computer and use it in GitHub Desktop.
ETCD Heartbeat Update Procedure

ETCD Heartbeat Update Procedure

Stop Cluster Service

  1. On each Standby:

    sv stop cluster
    
  2. On the Master:

    sv stop cluster
    

Stop ETCD

  1. On each Standby:

    sv stop etcd
    
  2. On the Master:

    sv stop etcd
    

Modify ETCD Config

  1. On the Master and each Standby:

    Open the etcd config file:

    nano /etc/etcd/etcd.config
    

    Modify the two existing lines to your new values (in milliseconds):

    ETCD_HEARTBEAT_INTERVAL=4000
    ETCD_ELECTION_TIMEOUT=20000
    

    Add a new line:

    ETCD_SNAPSHOT_COUNT=5000
    

Restart ETCD

  1. On the Master:

    sv start etcd
    
  2. On each Standby:

    sv start etcd
    
  3. Verify cluster health on the Master:

    If etcd is not healthy before restarting the DAP cluster service, then the master will failover immediately upon starting the cluster service.

    $ etcdctl cluster-health
    ...
    cluster is healthy
    

Restart Cluster Service

  1. On the Master:

    sv start cluster
    
  2. On each Standby:

    sv start cluster
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment