Manual Method for Running ETCD Version 3.4.9 Based in an OpenShift 4.6 Environment

ocp46-etcd-manual-runing.md

- Cluster Node #1

$ etcd \
--data-dir '/var/lib/etcd' \
--wal-dir '/var/lib/etcd/member/wal' \
--initial-advertise-peer-urls=https://192.168.0.81:2380 \
--cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master01.ocp46.local.crt \
--key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master01.ocp46.local.key \
--trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \
--client-cert-auth=true \
--peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-peer/etcd-peer-master01.ocp46.local.crt \
--peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-peer/etcd-peer-master01.ocp46.local.key \
--peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \
--peer-client-cert-auth=true \
--advertise-client-urls=https://192.168.0.81:2379 \
--listen-client-urls=https://0.0.0.0:2379 \
--listen-peer-urls=https://0.0.0.0:2380 \
--listen-metrics-urls=https://0.0.0.0:9978 \
--cipher-suites 'TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384'

- Cluster Node #2

$ etcd \
--data-dir '/var/lib/etcd' \
--wal-dir '/var/lib/etcd/member/wal' \
--initial-advertise-peer-urls=https://192.168.0.82:2380 \
--cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master02.ocp46.local.crt \
--key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master02.ocp46.local.key \
--trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \
--client-cert-auth=true \
--peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-peer/etcd-peer-master02.ocp46.local.crt \
--peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-peer/etcd-peer-master02.ocp46.local.key \
--peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \
--peer-client-cert-auth=true \
--advertise-client-urls=https://192.168.0.82:2379 \
--listen-client-urls=https://0.0.0.0:2379 \
--listen-peer-urls=https://0.0.0.0:2380 \
--listen-metrics-urls=https://0.0.0.0:9978 \
--cipher-suites 'TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384'

- Cluster Node #3

$ etcd \
--data-dir '/var/lib/etcd' \
--wal-dir '/var/lib/etcd/member/wal' \
--initial-advertise-peer-urls=https://192.168.0.83:2380 \
--cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master03.ocp46.local.crt \
--key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master03.ocp46.local.key \
--trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \
--client-cert-auth=true \
--peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-peer/etcd-peer-master03.ocp46.local.crt \
--peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-peer/etcd-peer-master03.ocp46.local.key \
--peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \
--peer-client-cert-auth=true \
--advertise-client-urls=https://192.168.0.83:2379 \
--listen-client-urls=https://0.0.0.0:2379 \
--listen-peer-urls=https://0.0.0.0:2380 \
--listen-metrics-urls=https://0.0.0.0:9978 \
--cipher-suites 'TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384'

- Checking the Status Information of ETCD Endpoints

$ etcdctl endpoint status -w table \
--cacert="/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" \
--cert="/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master01.ocp46.local.crt" \
--key="/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master01.ocp46.local.key" \
--endpoints=[https://192.168.0.81:2379,https://192.168.0.82:2379,https://192.168.0.83:2379]
+---------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|         ENDPOINT          |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+---------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://192.168.0.81:2379 | a02e4fe919db844a |   3.4.9 |  125 MB |     false |      false |      3621 |  616079037 |          616079037 |        |
+---------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

ETCD Snapshot Backup

$ etcdctl snapshot save master01-ocp46-local-etcd-snapshot.db \
--cacert="/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" \
--cert="/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master01.ocp46.local.crt" \
--key="/etc/kubernetes/static-pod-certs/secrets/etcd-all-serving/etcd-serving-master01.ocp46.local.key"
$ ls 
master01-ocp46-local-etcd-snapshot.db

To be continued...