Using Ubuntu with kubeadm, kubelet, and kubectl v1.9.5 installed (specifically the libvirt vagrant box heptio/quickstart-ubuntu)
-
install v1.9.7
sudo apt-get install -y curl software-properties-common sudo apt-add-repository ppa:lihiwish/etcd3-xenial curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - sudo apt-get update sudo apt-get install -y etcd kubelet=1.9.7-00 kubeadm=1.9.7-00 kubectl=1.9.7-00 cat <<EOF > kubeadm-conf.yaml apiVersion: kubeadm.k8s.io/v1alpha1 kind: MasterConfiguration etcd: endpoints: - http://127.0.0.1:2379 EOF sudo kubeadm init --config kubeadm-conf.yaml mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config export kubever=$(kubectl version | base64 | tr -d '\n') kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever" kubectl taint nodes --all node-role.kubernetes.io/master-
-
upgrade to v1.10.2
sudo apt install kubeadm=1.10.2-00 sudo kubeadm upgrade plan sudo kubeadm upgrade apply v1.10.2 sudo apt install kubectl=1.10.2-00 kubelet=1.10.2-00 sudo systemctl restart kubelet
-
Current Result:
vagrant@ubuntu:~$ sudo kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade] Making sure the cluster is healthy: [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' could not read manifests from: /etc/kubernetes/manifests, error: failed to check if etcd pod implements TLS: failed to read manifest for "/etc/kubernetes/manifests/etcd.yaml": open /etc/kubernetes/manifests/etcd.yaml: no such file or directory
-
Using kubeadm from https://github.com/detiber/kubernetes/tree/external_etcd_1_10
-
Plan
vagrant@ubuntu:~$ sudo ./kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade] Making sure the cluster is healthy: [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [upgrade/plan] computing upgrade possibilities [upgrade] Fetching available versions to upgrade to [upgrade/versions] Cluster version: v1.9.7 [upgrade/versions] kubeadm version: v1.10.3-beta.0.11+97c21a54b50f66 [upgrade/versions] Latest stable version: v1.10.2 [upgrade/versions] Latest version in the v1.9 series: v1.9.7 External components that should be upgraded manually before you upgrade the control plane with 'kubeadm upgrade apply': COMPONENT CURRENT AVAILABLE Etcd 3.1.0 3.1.12 Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply': COMPONENT CURRENT AVAILABLE Kubelet 1 x v1.9.7 v1.10.2 Upgrade to the latest stable version: COMPONENT CURRENT AVAILABLE API Server v1.9.7 v1.10.2 Controller Manager v1.9.7 v1.10.2 Scheduler v1.9.7 v1.10.2 Kube Proxy v1.9.7 v1.10.2 Kube DNS 1.14.8 1.14.8 You can now apply the upgrade by executing the following command: kubeadm upgrade apply v1.10.2 Note: Before you can perform this upgrade, you have to update kubeadm to v1.10.2. _____________________________________________________________________
-
Apply
vagrant@ubuntu:~$ sudo ./kubeadm upgrade apply v1.10.2 [preflight] Running pre-flight checks. [upgrade] Making sure the cluster is healthy: [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [upgrade/version] You have chosen to change the cluster version to "v1.10.2" [upgrade/versions] Cluster version: v1.9.7 [upgrade/versions] kubeadm version: v1.10.3-beta.0.11+97c21a54b50f66 [upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y [upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler] [upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.10.2"... Static pod: kube-apiserver-ubuntu hash: 85853dacfc28ca73f8ecafc777840ce3 Static pod: kube-controller-manager-ubuntu hash: 86d3edb6f8c3c83c3039223896aa3ce1 Static pod: kube-scheduler-ubuntu hash: 14166f4128d60694f8e58670f97b1763 [upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests350538632" [controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests350538632/kube-apiserver.yaml" [controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests350538632/kube-controller-manager.yaml" [controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests350538632/kube-scheduler.yaml" [certificates] Generated etcd/ca certificate and key. [certificates] Generated apiserver-etcd-client certificate and key. [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests807271751/kube-apiserver.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component Static pod: kube-apiserver-ubuntu hash: 85853dacfc28ca73f8ecafc777840ce3 Static pod: kube-apiserver-ubuntu hash: 85853dacfc28ca73f8ecafc777840ce3 Static pod: kube-apiserver-ubuntu hash: 85853dacfc28ca73f8ecafc777840ce3 Static pod: kube-apiserver-ubuntu hash: 9456f3d1476d0fa3adc7a17b08cdea57 [apiclient] Found 1 Pods for label selector component=kube-apiserver [upgrade/staticpods] Component "kube-apiserver" upgraded successfully! [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests807271751/kube-controller-manager.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component Static pod: kube-controller-manager-ubuntu hash: 86d3edb6f8c3c83c3039223896aa3ce1 Static pod: kube-controller-manager-ubuntu hash: 9403ba0735cfafa48d7924f8165f263d [apiclient] Found 1 Pods for label selector component=kube-controller-manager [upgrade/staticpods] Component "kube-controller-manager" upgraded successfully! [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests807271751/kube-scheduler.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component Static pod: kube-scheduler-ubuntu hash: 14166f4128d60694f8e58670f97b1763 Static pod: kube-scheduler-ubuntu hash: 454b0ba64d68fe4ae13a2d1ca9625979 [apiclient] Found 1 Pods for label selector component=kube-scheduler [upgrade/staticpods] Component "kube-scheduler" upgraded successfully! [uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [addons] Applied essential addon: kube-dns [addons] Applied essential addon: kube-proxy [upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.10.2". Enjoy! [upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets in turn.
-
-
Using Ubuntu with kubeadm, kubelet, and kubectl v1.9.5 installed (specifically the libvirt vagrant box heptio/quickstart-ubuntu)
-
install v1.9.7
sudo apt install curl software-properties-common sudo add-apt-repository ppa:gophers/archive sudo apt-add-repository ppa:lihiwish/etcd3-xenial curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - sudo apt update sudo apt install golang-1.10-go etcd kubelet=1.9.7-00 kubeadm=1.9.7-00 kubectl=1.9.7-00 export PATH=/usr/lib/go-1.10/bin:${PATH}:~/go/bin go get -u github.com/cloudflare/cfssl/cmd/... # Create the etcd CA mkdir etcd-ca pushd etcd-ca cfssl print-defaults csr > ca-csr.json cat << EOF > ca-config.json { "signing": { "default": { "expiry": "43800h" }, "profiles": { "server": { "expiry": "43800h", "usages": [ "signing", "key encipherment", "server auth" ] }, "client": { "expiry": "43800h", "usages": [ "signing", "key encipherment", "client auth" ] }, "peer": { "expiry": "43800h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca - # Create the etcd server cert cat << EOF > server.json { "CN": "$(hostname -f)", "hosts": [ "localhost", "127.0.0.1" ], "key": { "algo": "ecdsa", "size": 256 } } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server server.json | cfssljson -bare server # Create the etcd peer cert cat << EOF > peer.json { "CN": "$(hostname -f)", "hosts": [ "localhost", "127.0.0.1" ], "key": { "algo": "ecdsa", "size": 256 } } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer peer.json | cfssljson -bare peer # Create the etcd client cert cat << EOF > client.json { "CN": "$(hostname -f)", "key": { "algo": "ecdsa", "size": 256 } } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client.json | cfssljson -bare client popd sudo mkdir -p /etc/etcd sudo cp etcd-ca/ca.pem /etc/etcd/ca.crt for cert in server peer; do sudo cp etcd-ca/${cert}.pem /etc/etcd/${cert}.crt sudo cp etcd-ca/${cert}-key.pem /etc/etcd/${cert}.key done sudo chown -R etcd:etcd /etc/etcd cat <<EOF | sudo tee -a /etc/default/etcd ETCD_NAME=$(hostname) ETCD_CERT_FILE=/etc/etcd/server.crt ETCD_KEY_FILE=/etc/etcd/server.key ETCD_CLIENT_CERT_AUTH=true ETCD_TRUSTED_CA_FILE=/etc/etcd/ca.crt ETCD_PEER_CLIENT_CERT_AUTH=true ETCD_PEER_CERT_FILE=/etc/etcd/peer.crt ETCD_PEER_KEY_FILE=/etc/etcd/peer.key ETCD_PEER_TRUSTED_CA_FILE=/etc/etcd/ca.crt ETCD_LISTEN_PEER_URLS=https://localhost:2380 ETCD_LISTEN_CLIENT_URLS=https://localhost:2379 ETCD_INITIAL_ADVERTISE_PEER_URLS=https://localhost:2380 ETCD_INITIAL_CLUSTER="$(hostname)=https://localhost:2380" ETCD_INITIAL_CLUSTER_STATE=new ETCD_ADVERTISE_CLIENT_URLS=https://localhost:2379 EOF sudo systemctl stop etcd sudo rm -rf /var/lib/etcd/default sudo systemctl start etcd sudo cp etcd-ca/ca.pem /etc/kubernetes/etcd-ca.crt sudo cp etcd-ca/client.pem /etc/kubernetes/etcd-client.crt sudo cp etcd-ca/client-key.pem /etc/kubernetes/etcd-client.key cat <<EOF > kubeadm-conf.yaml apiVersion: kubeadm.k8s.io/v1alpha1 kind: MasterConfiguration etcd: endpoints: - https://127.0.0.1:2379 caFile: /etc/kubernetes/etcd-ca.crt certFile: /etc/kubernetes/etcd-client.crt keyFile: /etc/kubernetes/etcd-client.key EOF sudo kubeadm init --config kubeadm-conf.yaml mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config export kubever=$(kubectl version | base64 | tr -d '\n') kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever" kubectl taint nodes --all node-role.kubernetes.io/master-
- upgrade to v1.10.2
sudo apt install kubeadm=1.10.2-00 sudo kubeadm upgrade plan sudo kubeadm upgrade apply v1.10.2 sudo apt install kubectl=1.10.2-00 kubelet=1.10.2-00 sudo systemctl restart kubelet
-
Current Result:
vagrant@ubuntu:~$ sudo kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade] Making sure the cluster is healthy: [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' could not read manifests from: /etc/kubernetes/manifests, error: failed to check if etcd pod implements TLS: failed to read manifest for "/etc/kubernetes/manifests/etcd.yaml": open /etc/kubernetes/manifests/etcd.yaml: no such file or directory
-
Using kubeadm from https://github.com/detiber/kubernetes/tree/external_etcd_1_10
-
Plan
vagrant@ubuntu:~$ sudo ./kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade] Making sure the cluster is healthy: [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [upgrade/plan] computing upgrade possibilities [upgrade] Fetching available versions to upgrade to [upgrade/versions] Cluster version: v1.9.7 [upgrade/versions] kubeadm version: v1.10.3-beta.0.11+97c21a54b50f66 [upgrade/versions] Latest stable version: v1.10.2 [upgrade/versions] Latest version in the v1.9 series: v1.9.7 External components that should be upgraded manually before you upgrade the control plane with 'kubeadm upgrade apply': COMPONENT CURRENT AVAILABLE Etcd 3.1.0 3.1.12 Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply': COMPONENT CURRENT AVAILABLE Kubelet 1 x v1.9.7 v1.10.2 Upgrade to the latest stable version: COMPONENT CURRENT AVAILABLE API Server v1.9.7 v1.10.2 Controller Manager v1.9.7 v1.10.2 Scheduler v1.9.7 v1.10.2 Kube Proxy v1.9.7 v1.10.2 Kube DNS 1.14.8 1.14.8 You can now apply the upgrade by executing the following command: kubeadm upgrade apply v1.10.2 Note: Before you can perform this upgrade, you have to update kubeadm to v1.10.2. _____________________________________________________________________
-
Apply
vagrant@ubuntu:~$ sudo ./kubeadm upgrade apply v1.10.2 [preflight] Running pre-flight checks. [upgrade] Making sure the cluster is healthy: [upgrade/config] Making sure the configuration is correct: [upgrade/config] Reading configuration from the cluster... [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [upgrade/version] You have chosen to change the cluster version to "v1.10.2" [upgrade/versions] Cluster version: v1.9.7 [upgrade/versions] kubeadm version: v1.10.3-beta.0.11+97c21a54b50f66 [upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y [upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler] [upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.10.2"... Static pod: kube-apiserver-ubuntu hash: 761b9375c09ca68e1bfde114f3755875 Static pod: kube-controller-manager-ubuntu hash: 24627a5bd0bd470320381e4d437c0bab Static pod: kube-scheduler-ubuntu hash: 96944ce896a1ba4844bab386f40c0acc [upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests949774848" [controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests949774848/kube-apiserver.yaml" [controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests949774848/kube-controller-manager.yaml" [controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests949774848/kube-scheduler.yaml" [certificates] Generated etcd/ca certificate and key. [certificates] Generated apiserver-etcd-client certificate and key. [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests279255903/kube-apiserver.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component Static pod: kube-apiserver-ubuntu hash: 761b9375c09ca68e1bfde114f3755875 Static pod: kube-apiserver-ubuntu hash: b2cdff92d7243dfd2f56894b0c6bf531 [apiclient] Found 1 Pods for label selector component=kube-apiserver [upgrade/staticpods] Component "kube-apiserver" upgraded successfully! [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests279255903/kube-controller-manager.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component Static pod: kube-controller-manager-ubuntu hash: 24627a5bd0bd470320381e4d437c0bab Static pod: kube-controller-manager-ubuntu hash: e5bd07bb06a751c539f2b83fd0391e19 [apiclient] Found 1 Pods for label selector component=kube-controller-manager [upgrade/staticpods] Component "kube-controller-manager" upgraded successfully! [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests279255903/kube-scheduler.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component Static pod: kube-scheduler-ubuntu hash: 96944ce896a1ba4844bab386f40c0acc Static pod: kube-scheduler-ubuntu hash: b4e42f52dd09a5166ee4c6c6c1306a4c [apiclient] Found 1 Pods for label selector component=kube-scheduler [upgrade/staticpods] Component "kube-scheduler" upgraded successfully! [uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [addons] Applied essential addon: kube-dns [addons] Applied essential addon: kube-proxy [upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.10.2". Enjoy! [upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets in turn.
-