Created
March 22, 2019 00:38
-
-
Save magick93/814aaa5f825c5021c2d031d8b7944c28 to your computer and use it in GitHub Desktop.
Unable to install okd 3.11 due to error with Approve node certificates when bootstrapping
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Using module file /home/user/git/sourcetopia/openshift-install/openshift/ansible/roles/lib_openshift/library/oc_csr_approve.py | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'echo ~deploy && sleep 0'"'"'' | |
| <k8s-ctl01.dev.mydomain.com> (0, '/home/deploy\n', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729 `" && echo ansible-tmp-1553214624.6-3330728421729="` echo /home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729 `" ) && sleep 0'"'"'' | |
| <k8s-ctl01.dev.mydomain.com> (0, 'ansible-tmp-1553214624.6-3330728421729=/home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729\n', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| <k8s-ctl01.dev.mydomain.com> PUT /home/user/.ansible/tmp/ansible-local-10931AT0_NQ/tmpJvNOzG TO /home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729/oc_csr_approve.py | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d '[k8s-ctl01.dev.mydomain.com]' | |
| <k8s-ctl01.dev.mydomain.com> (0, 'sftp> put /home/user/.ansible/tmp/ansible-local-10931AT0_NQ/tmpJvNOzG /home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729/oc_csr_approve.py\n', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 2\r\ndebug2: Server supports extension "[email protected]" revision 2\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug3: Sent message fd 15 T:16 I:1\r\ndebug3: SSH_FXP_REALPATH . -> /home/deploy size 0\r\ndebug3: Looking up /home/user/.ansible/tmp/ansible-local-10931AT0_NQ/tmpJvNOzG\r\ndebug3: Sent message fd 15 T:17 I:2\r\ndebug3: Received stat reply T:101 I:2\r\ndebug1: Couldn\'t stat remote file: No such file or directory\r\ndebug3: Sent message SSH2_FXP_OPEN I:3 P:/home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729/oc_csr_approve.py\r\ndebug3: Sent message SSH2_FXP_WRITE I:4 O:0 S:32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 4 32768 bytes at 0\r\ndebug3: Sent message SSH2_FXP_WRITE I:5 O:32768 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:65536 S:3446\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 5 32768 bytes at 32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 3446 bytes at 65536\r\ndebug3: Sent message SSH2_FXP_CLOSE I:4\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'chmod u+x /home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729/ /home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729/oc_csr_approve.py && sleep 0'"'"'' | |
| <k8s-ctl01.dev.mydomain.com> (0, '', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d -tt k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'sudo -H -S -p "[sudo via ansible, key=jhyurlmuibnpcshzckniaezebrggzhsn] password: " -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-jhyurlmuibnpcshzckniaezebrggzhsn; /usr/bin/python /home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729/oc_csr_approve.py'"'"'"'"'"'"'"'"' && sleep 0'"'"'' | |
| Escalation succeeded | |
| <k8s-ctl01.dev.mydomain.com> (1, '\r\n\r\n{"server_csrs": null, "exception": " File \\"/tmp/ansible_HSGjw2/ansible_module_oc_csr_approve.py\\", line 24, in <module>\\n from json.decoder import JSONDecodeError\\n", "raw_failures": [], "oc_get_nodes": {"items": [{"status": {"capacity": {"hugepages-1Gi": "0", "hugepages-2Mi": "0", "pods": "250", "cpu": "4", "memory": "16266800Ki"}, "addresses": [{"type": "InternalIP", "address": "10.4.111.101"}, {"type": "Hostname", "address": "k8s-ctl01.dev.mydomain.com"}], "nodeInfo": {"kernelVersion": "3.10.0-957.el7.x86_64", "kubeletVersion": "v1.11.0+d4cacc0", "containerRuntimeVersion": "docker://1.13.1", "machineID": "bfee8bc7a3cd4673a31ac520eb963eb2", "kubeProxyVersion": "v1.11.0+d4cacc0", "bootID": "c4c9e366-57c7-4334-933b-dab93702b583", "osImage": "CentOS Linux 7 (Core)", "architecture": "amd64", "systemUUID": "1C582A42-E6DF-7E8C-DBB8-DB6315262B0C", "operatingSystem": "linux"}, "allocatable": {"hugepages-1Gi": "0", "hugepages-2Mi": "0", "pods": "250", "cpu": "4", "memory": "16164400Ki"}, "daemonEndpoints": {"kubeletEndpoint": {"Port": 10250}}, "images": [{"sizeBytes": 1172234437, "names": ["docker.io/openshift/origin-node@sha256:fd552ffd0bdbbcf6c5c646ec7c086b2fafde0aa560711038b1e35c524d867a10", "docker.io/openshift/origin-node:v3.11.0"]}, {"sizeBytes": 1166879223, "names": ["docker.io/openshift/origin-node@sha256:d198706b58f39b87af543cf73e37cc50632a214bac107d70b6d99034982d214a"]}, {"sizeBytes": 828734067, "names": ["docker.io/openshift/origin-control-plane@sha256:508dad04d4c6c7c6ea1e50ac16a5aeffb3854d391b392d74a68c1bf767c608e0", "docker.io/openshift/origin-control-plane:v3.11", "docker.io/openshift/origin-control-plane:v3.11.0"]}, {"sizeBytes": 826232564, "names": ["docker.io/openshift/origin-control-plane@sha256:bf28e6729781f2d68c9ea121cee5c649a084ae9a4dadefded9321e2e381a2549"]}, {"sizeBytes": 383673671, "names": ["docker.io/openshift/origin-deployer@sha256:88eedcec3c877bea09653f037174f50356071e265d93889964a954a73b03aa8e", "docker.io/openshift/origin-deployer:v3.11", "docker.io/openshift/origin-deployer:v3.11.0"]}, {"sizeBytes": 261526128, "names": ["docker.io/openshift/origin-pod@sha256:91f0f130c991c2a6a9f0e9da43f7e56296bacdc46be0cd768125ed3e6f7ef09d", "docker.io/openshift/origin-pod:v3.11.0"]}, {"sizeBytes": 258456390, "names": ["docker.io/openshift/origin-pod@sha256:67b51982ad05b446e4045b7e2507768c2d25b2eb61ccadd7b6a718b1262c25b8"]}, {"sizeBytes": 37269372, "names": ["quay.io/coreos/etcd@sha256:43fbc8a457aa0cb887da63d74a48659e13947cb74b96a53ba8f47abb6172a948", "quay.io/coreos/etcd:v3.2.22"]}], "conditions": [{"status": "False", "lastTransitionTime": "2019-03-21T20:38:32Z", "reason": "KubeletHasSufficientDisk", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet has sufficient disk space available", "type": "OutOfDisk"}, {"status": "False", "lastTransitionTime": "2019-03-21T20:38:32Z", "reason": "KubeletHasSufficientMemory", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet has sufficient memory available", "type": "MemoryPressure"}, {"status": "False", "lastTransitionTime": "2019-03-21T20:38:32Z", "reason": "KubeletHasNoDiskPressure", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet has no disk pressure", "type": "DiskPressure"}, {"status": "False", "lastTransitionTime": "2019-03-21T20:38:32Z", "reason": "KubeletHasSufficientPID", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet has sufficient PID available", "type": "PIDPressure"}, {"status": "True", "lastTransitionTime": "2019-03-22T00:27:39Z", "reason": "KubeletReady", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet is posting ready status", "type": "Ready"}]}, "kind": "Node", "spec": {}, "apiVersion": "v1", "metadata": {"name": "k8s-ctl01.dev.mydomain.com", "labels": {"beta.kubernetes.io/os": "linux", "node-role.kubernetes.io/master": "true", "kubernetes.io/hostname": "k8s-ctl01.dev.mydomain.com", "node-role.kubernetes.io/compute": "true", "node-role.kubernetes.io/infra": "true", "beta.kubernetes.io/arch": "amd64"}, "namespace": "", "resourceVersion": "21959", "creationTimestamp": "2019-03-21T20:38:37Z", "annotations": {"volumes.kubernetes.io/controller-managed-attach-detach": "true", "node.openshift.io/md5sum": "a4305b3db4427b8d4bd21c1a11115c5d"}, "selfLink": "/api/v1/nodes/k8s-ctl01.dev.mydomain.com", "uid": "4dfe8828-4c19-11e9-add9-005056aa572c"}}], "kind": "List", "apiVersion": "v1", "metadata": {"selfLink": "", "resourceVersion": ""}}, "changed": false, "client_csrs": {}, "failed": true, "state": "unknown", "all_subjects_found": ["subject=/O=system:nodes/CN=system:node:k8s-ctl01.dev.mydomain.com\\n", "subject=/O=system:nodes/CN=system:node:k8s-ctl01.dev.mydomain.com\\n"], "client_approve_results": [], "rc": 0, "invocation": {"module_args": {"oc_bin": "oc", "oc_conf": "/etc/origin/master/admin.kubeconfig", "node_list": ["k8s-worker01.dev.mydomain.com", "k8s-worker02.dev.mydomain.com", "k8s-worker03.dev.mydomain.com", "k8s-ctl01.dev.mydomain.com"]}}, "unwanted_csrs": [{"status": {"conditions": [{"message": "This CSR was approved by kubectl certificate approve.", "type": "Approved", "reason": "KubectlApprove", "lastUpdateTime": "2019-03-22T00:06:42Z"}], "certificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQWFLZ0F3SUJBZ0lVQ3NzVlZMeTY4clhBcVhwbnczekEvQTg1TXRVd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRWtNQ0lHQTFVRUF3d2JiM0JsYm5Ob2FXWjBMWE5wWjI1bGNrQXhOVFV6TVRJNE56UTFNQjRYRFRFNQpNRE15TWpBd01ESXdNRm9YRFRJd01ETXlNVEF3TURJd01Gb3dSREVWTUJNR0ExVUVDaE1NYzNsemRHVnRPbTV2ClpHVnpNU3N3S1FZRFZRUURFeUp6ZVhOMFpXMDZibTlrWlRwck9ITXRZM1JzTURFdVpHVjJMblJ2Y0dsaExtNTYKTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFbmJJU2FSajZHMVpQR2RMUWlDL1lpekpxOXFWUwpBWEorSnJ6N3d1QTUveDNEZ1d5emJSQWN3bkZGbWFRUTZQZk1VWWsyY3A3MnZtS1NORVNvOUlUYWRLT0JqRENCCmlUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3d0NnWUlLd1lCQlFVSEF3RXdEQVlEVlIwVEFRSC8KQkFJd0FEQWRCZ05WSFE0RUZnUVVQTzJvQWlTaEc0bGlLZTRGSDFQTGltb0MwOHN3TlFZRFZSMFJCQzR3TElJVwphemh6TFdOMGJEQXhMbVJsZGk1MGIzQnBZUzV1ZW9JQWh3UUtCRzlsaHdTc0VRQUJod1FLZ0FBQk1BMEdDU3FHClNJYjNEUUVCQ3dVQUE0SUJBUUF3YmNITEVScHZTQkVZQUFUMlduVWlGOFppNE5rUGs5ZWRReisvcGJkeGNOeEIKUmZmSzFpRFNSYmRDNjh4ZjBHUjJxdklDWDdwMVhRUURueXJ3K2JvR2JUMUdxUVB4Z1RBTXdHVUtmcTFoMENWbQo5a0g5VE9sNTFWczRYZkhxZFZzdFA1Z3RUT0FpUFNzaWdVSmM3L3FKSHdNeHQ3aytNVmlJY1V1aUd1SjljRWFsClJ0TjZLeHBBTGx4WVA4aVZETTY4VTJISTgwalRrVFRjbW1yb2VzRnlPb1BtSlN5ZjRCNWExWWllcTB6SGNKcEwKYlRsSmpWTDl5QTRZQXdtQ0JzSmxyOCtVMlVweUhYY0hscERmV1ZvT1krRDhaQ2IzaFNRdnBlVkdJNE5OTCswOApQNVd4Ui9LK0xORElGTDVYR1ZDSG5rbm5IL041alZvNVEwUGY5bkZ5Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"}, "kind": "CertificateSigningRequest", "spec": {"username": "system:admin", "usages": ["digital signature", "key encipherment", "server auth"], "request": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQlJ6Q0I3Z0lCQURCRU1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14S3pBcEJnTlZCQU1USW5ONQpjM1JsYlRwdWIyUmxPbXM0Y3kxamRHd3dNUzVrWlhZdWRHOXdhV0V1Ym5vd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxCmhrak9QUU1CQndOQ0FBU2RzaEpwR1BvYlZrOFowdENJTDlpTE1tcjJwVklCY240bXZQdkM0RG4vSGNPQmJMTnQKRUJ6Q2NVV1pwQkRvOTh4UmlUWnludmErWXBJMFJLajBoTnAwb0Vnd1JnWUpLb1pJaHZjTkFRa09NVGt3TnpBMQpCZ05WSFJFRUxqQXNnaFpyT0hNdFkzUnNNREV1WkdWMkxuUnZjR2xoTG01NmdnQ0hCQW9FYjJXSEJLd1JBQUdICkJBcUFBQUV3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUlnVGVnSnluemkwZVl1ZHE4cU9LR3hOaXc1YlZheGkyK2sKbHV3NGVNUzVwOFFDSVFEUWRsaTY0LzJ6cldQajlQNjVYWFlSMUR0ME1pMzNERnRnQURQTHdPNnFtQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=", "groups": ["system:masters", "system:cluster-admins", "system:authenticated"]}, "apiVersion": "certificates.k8s.io/v1beta1", "metadata": {"name": "csr-bsf6g", "namespace": "", "resourceVersion": "19872", "generateName": "csr-", "creationTimestamp": "2019-03-22T00:06:21Z", "selfLink": "/apis/certificates.k8s.io/v1beta1/certificatesigningrequests/csr-bsf6g", "uid": "52f26e8c-4c36-11e9-add9-005056aa572c"}}, {"status": {"conditions": [{"message": "This CSR was approved by kubectl certificate approve.", "type": "Approved", "reason": "KubectlApprove", "lastUpdateTime": "2019-03-22T00:06:36Z"}], "certificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQWFLZ0F3SUJBZ0lVTFVQcFY4eitRRHZseDhpa3AydjBpRTA2citRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRWtNQ0lHQTFVRUF3d2JiM0JsYm5Ob2FXWjBMWE5wWjI1bGNrQXhOVFV6TVRJNE56UTFNQjRYRFRFNQpNRE15TWpBd01ESXdNRm9YRFRJd01ETXlNVEF3TURJd01Gb3dSREVWTUJNR0ExVUVDaE1NYzNsemRHVnRPbTV2ClpHVnpNU3N3S1FZRFZRUURFeUp6ZVhOMFpXMDZibTlrWlRwck9ITXRZM1JzTURFdVpHVjJMblJ2Y0dsaExtNTYKTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFanE5MUxpTFlpVzBlVFQrNU4vYVNwRUJIWFFBWgp1ZE1RQTVlbTI1Z3JvWWt0WGlycnJKN0NrcmdveGtDd09hMTBtNkMrMm5uNjU4T3ZwSjg0UHZXYkhLT0JqRENCCmlUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3d0NnWUlLd1lCQlFVSEF3RXdEQVlEVlIwVEFRSC8KQkFJd0FEQWRCZ05WSFE0RUZnUVVwRGtRRjNXS3ArdmlhSmdlVWlBbHJPRXNueWt3TlFZRFZSMFJCQzR3TElJVwphemh6TFdOMGJEQXhMbVJsZGk1MGIzQnBZUzV1ZW9JQWh3UUtCRzlsaHdTc0VRQUJod1FLZ0FBQk1BMEdDU3FHClNJYjNEUUVCQ3dVQUE0SUJBUUNDL0JJL0hKYXBhWG1kTUpsd2MvejhlMkRiazBVSWp3UlhjM0lET0FxckRoeGsKR2VOd0tNcUJJc2Fic1pWcXlzMWtqSVBaS0VFWTdwVHFhZU54blFRT2FucWZYMmNuZTV1aGEvbUo0U2NmRUorQQpQOGJaSU1GRzVIS1FPNlBPOSt5SXhKR3I4emFpM1hsd2s0YUNGZ2xCWEQ3SzRsZ0Noc0xXZjdZQWtKcmRKRGEvCkRhNmpzbFQvQithZlpKbmJZeUJSZkNqeVdkUE5lbXdsU1V2ZS9SZ3Z6eXZsMllmTFlpMWJ3bmw2RXNrTW84aWoKUVdBR0tpd3FkUnNCdGNOSStJY1JHYzFDZWY4ZFFhdGVGWFl6c2VINHdONWpwRWZoYllMOVRCbEJ1QnQwWVpZOQpJTlZxUVRGVEZQVzBJWmZQcGU4ZXhUOFR3TUJlRjBoS05uUk03blZICi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"}, "kind": "CertificateSigningRequest", "spec": {"username": "system:admin", "usages": ["digital signature", "key encipherment", "server auth"], "request": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQlNEQ0I3Z0lCQURCRU1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14S3pBcEJnTlZCQU1USW5ONQpjM1JsYlRwdWIyUmxPbXM0Y3kxamRHd3dNUzVrWlhZdWRHOXdhV0V1Ym5vd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxCmhrak9QUU1CQndOQ0FBU09yM1V1SXRpSmJSNU5QN2szOXBLa1FFZGRBQm01MHhBRGw2YmJtQ3VoaVMxZUt1dXMKbnNLU3VDakdRTEE1clhTYm9MN2FlZnJudzYra256Zys5WnNjb0Vnd1JnWUpLb1pJaHZjTkFRa09NVGt3TnpBMQpCZ05WSFJFRUxqQXNnaFpyT0hNdFkzUnNNREV1WkdWMkxuUnZjR2xoTG01NmdnQ0hCQW9FYjJXSEJLd1JBQUdICkJBcUFBQUV3Q2dZSUtvWkl6ajBFQXdJRFNRQXdSZ0loQVBFWlV4L0M0UHFncWhpKytBU3NCbWNCN3Z0SGNTcjMKUWJhL2dSS2hIUWwvQWlFQTJQbzVDK1lzQmFMcWlCMU1NcGQ4THFIS3d3ck0yL0NKczN0RHQyWDRwMkU9Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=", "groups": ["system:masters", "system:cluster-admins", "system:authenticated"]}, "apiVersion": "certificates.k8s.io/v1beta1", "metadata": {"name": "csr-v85dj", "namespace": "", "resourceVersion": "19857", "generateName": "csr-", "creationTimestamp": "2019-03-22T00:04:51Z", "selfLink": "/apis/certificates.k8s.io/v1beta1/certificatesigningrequests/csr-v85dj", "uid": "1d5cb3b4-4c36-11e9-add9-005056aa572c"}}], "server_approve_results": [], "msg": "Could not find csr for nodes: k8s-worker03.dev.mydomain.com, k8s-worker02.dev.mydomain.com, k8s-worker01.dev.mydomain.com"}\r\n', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to k8s-ctl01.dev.mydomain.com closed.\r\n') | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'rm -f -r /home/deploy/.ansible/tmp/ansible-tmp-1553214624.6-3330728421729/ > /dev/null 2>&1 && sleep 0'"'"'' | |
| <k8s-ctl01.dev.mydomain.com> (0, '', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| FAILED - RETRYING: Approve node certificates when bootstrapping (1 retries left).Result was: { | |
| "all_subjects_found": [ | |
| "subject=/O=system:nodes/CN=system:node:k8s-ctl01.dev.mydomain.com\n", | |
| "subject=/O=system:nodes/CN=system:node:k8s-ctl01.dev.mydomain.com\n" | |
| ], | |
| "attempts": 30, | |
| "changed": false, | |
| "client_approve_results": [], | |
| "client_csrs": {}, | |
| "invocation": { | |
| "module_args": { | |
| "node_list": [ | |
| "k8s-worker01.dev.mydomain.com", | |
| "k8s-worker02.dev.mydomain.com", | |
| "k8s-worker03.dev.mydomain.com", | |
| "k8s-ctl01.dev.mydomain.com" | |
| ], | |
| "oc_bin": "oc", | |
| "oc_conf": "/etc/origin/master/admin.kubeconfig" | |
| } | |
| }, | |
| "msg": "Could not find csr for nodes: k8s-worker03.dev.mydomain.com, k8s-worker02.dev.mydomain.com, k8s-worker01.dev.mydomain.com", | |
| "oc_get_nodes": { | |
| "apiVersion": "v1", | |
| "items": [ | |
| { | |
| "apiVersion": "v1", | |
| "kind": "Node", | |
| "metadata": { | |
| "annotations": { | |
| "node.openshift.io/md5sum": "a4305b3db4427b8d4bd21c1a11115c5d", | |
| "volumes.kubernetes.io/controller-managed-attach-detach": "true" | |
| }, | |
| "creationTimestamp": "2019-03-21T20:38:37Z", | |
| "labels": { | |
| "beta.kubernetes.io/arch": "amd64", | |
| "beta.kubernetes.io/os": "linux", | |
| "kubernetes.io/hostname": "k8s-ctl01.dev.mydomain.com", | |
| "node-role.kubernetes.io/compute": "true", | |
| "node-role.kubernetes.io/infra": "true", | |
| "node-role.kubernetes.io/master": "true" | |
| }, | |
| "name": "k8s-ctl01.dev.mydomain.com", | |
| "namespace": "", | |
| "resourceVersion": "21959", | |
| "selfLink": "/api/v1/nodes/k8s-ctl01.dev.mydomain.com", | |
| "uid": "4dfe8828-4c19-11e9-add9-005056aa572c" | |
| }, | |
| "spec": {}, | |
| "status": { | |
| "addresses": [ | |
| { | |
| "address": "10.4.111.101", | |
| "type": "InternalIP" | |
| }, | |
| { | |
| "address": "k8s-ctl01.dev.mydomain.com", | |
| "type": "Hostname" | |
| } | |
| ], | |
| "allocatable": { | |
| "cpu": "4", | |
| "hugepages-1Gi": "0", | |
| "hugepages-2Mi": "0", | |
| "memory": "16164400Ki", | |
| "pods": "250" | |
| }, | |
| "capacity": { | |
| "cpu": "4", | |
| "hugepages-1Gi": "0", | |
| "hugepages-2Mi": "0", | |
| "memory": "16266800Ki", | |
| "pods": "250" | |
| }, | |
| "conditions": [ | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-21T20:38:32Z", | |
| "message": "kubelet has sufficient disk space available", | |
| "reason": "KubeletHasSufficientDisk", | |
| "status": "False", | |
| "type": "OutOfDisk" | |
| }, | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-21T20:38:32Z", | |
| "message": "kubelet has sufficient memory available", | |
| "reason": "KubeletHasSufficientMemory", | |
| "status": "False", | |
| "type": "MemoryPressure" | |
| }, | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-21T20:38:32Z", | |
| "message": "kubelet has no disk pressure", | |
| "reason": "KubeletHasNoDiskPressure", | |
| "status": "False", | |
| "type": "DiskPressure" | |
| }, | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-21T20:38:32Z", | |
| "message": "kubelet has sufficient PID available", | |
| "reason": "KubeletHasSufficientPID", | |
| "status": "False", | |
| "type": "PIDPressure" | |
| }, | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-22T00:27:39Z", | |
| "message": "kubelet is posting ready status", | |
| "reason": "KubeletReady", | |
| "status": "True", | |
| "type": "Ready" | |
| } | |
| ], | |
| "daemonEndpoints": { | |
| "kubeletEndpoint": { | |
| "Port": 10250 | |
| } | |
| }, | |
| "images": [ | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-node@sha256:fd552ffd0bdbbcf6c5c646ec7c086b2fafde0aa560711038b1e35c524d867a10", | |
| "docker.io/openshift/origin-node:v3.11.0" | |
| ], | |
| "sizeBytes": 1172234437 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-node@sha256:d198706b58f39b87af543cf73e37cc50632a214bac107d70b6d99034982d214a" | |
| ], | |
| "sizeBytes": 1166879223 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-control-plane@sha256:508dad04d4c6c7c6ea1e50ac16a5aeffb3854d391b392d74a68c1bf767c608e0", | |
| "docker.io/openshift/origin-control-plane:v3.11", | |
| "docker.io/openshift/origin-control-plane:v3.11.0" | |
| ], | |
| "sizeBytes": 828734067 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-control-plane@sha256:bf28e6729781f2d68c9ea121cee5c649a084ae9a4dadefded9321e2e381a2549" | |
| ], | |
| "sizeBytes": 826232564 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-deployer@sha256:88eedcec3c877bea09653f037174f50356071e265d93889964a954a73b03aa8e", | |
| "docker.io/openshift/origin-deployer:v3.11", | |
| "docker.io/openshift/origin-deployer:v3.11.0" | |
| ], | |
| "sizeBytes": 383673671 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-pod@sha256:91f0f130c991c2a6a9f0e9da43f7e56296bacdc46be0cd768125ed3e6f7ef09d", | |
| "docker.io/openshift/origin-pod:v3.11.0" | |
| ], | |
| "sizeBytes": 261526128 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-pod@sha256:67b51982ad05b446e4045b7e2507768c2d25b2eb61ccadd7b6a718b1262c25b8" | |
| ], | |
| "sizeBytes": 258456390 | |
| }, | |
| { | |
| "names": [ | |
| "quay.io/coreos/etcd@sha256:43fbc8a457aa0cb887da63d74a48659e13947cb74b96a53ba8f47abb6172a948", | |
| "quay.io/coreos/etcd:v3.2.22" | |
| ], | |
| "sizeBytes": 37269372 | |
| } | |
| ], | |
| "nodeInfo": { | |
| "architecture": "amd64", | |
| "bootID": "c4c9e366-57c7-4334-933b-dab93702b583", | |
| "containerRuntimeVersion": "docker://1.13.1", | |
| "kernelVersion": "3.10.0-957.el7.x86_64", | |
| "kubeProxyVersion": "v1.11.0+d4cacc0", | |
| "kubeletVersion": "v1.11.0+d4cacc0", | |
| "machineID": "bfee8bc7a3cd4673a31ac520eb963eb2", | |
| "operatingSystem": "linux", | |
| "osImage": "CentOS Linux 7 (Core)", | |
| "systemUUID": "1C582A42-E6DF-7E8C-DBB8-DB6315262B0C" | |
| } | |
| } | |
| } | |
| ], | |
| "kind": "List", | |
| "metadata": { | |
| "resourceVersion": "", | |
| "selfLink": "" | |
| } | |
| }, | |
| "raw_failures": [], | |
| "rc": 0, | |
| "retries": 31, | |
| "server_approve_results": [], | |
| "server_csrs": null, | |
| "state": "unknown", | |
| "unwanted_csrs": [ | |
| { | |
| "apiVersion": "certificates.k8s.io/v1beta1", | |
| "kind": "CertificateSigningRequest", | |
| "metadata": { | |
| "creationTimestamp": "2019-03-22T00:06:21Z", | |
| "generateName": "csr-", | |
| "name": "csr-bsf6g", | |
| "namespace": "", | |
| "resourceVersion": "19872", | |
| "selfLink": "/apis/certificates.k8s.io/v1beta1/certificatesigningrequests/csr-bsf6g", | |
| "uid": "52f26e8c-4c36-11e9-add9-005056aa572c" | |
| }, | |
| "spec": { | |
| "groups": [ | |
| "system:masters", | |
| "system:cluster-admins", | |
| "system:authenticated" | |
| ], | |
| "request": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQlJ6Q0I3Z0lCQURCRU1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14S3pBcEJnTlZCQU1USW5ONQpjM1JsYlRwdWIyUmxPbXM0Y3kxamRHd3dNUzVrWlhZdWRHOXdhV0V1Ym5vd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxCmhrak9QUU1CQndOQ0FBU2RzaEpwR1BvYlZrOFowdENJTDlpTE1tcjJwVklCY240bXZQdkM0RG4vSGNPQmJMTnQKRUJ6Q2NVV1pwQkRvOTh4UmlUWnludmErWXBJMFJLajBoTnAwb0Vnd1JnWUpLb1pJaHZjTkFRa09NVGt3TnpBMQpCZ05WSFJFRUxqQXNnaFpyT0hNdFkzUnNNREV1WkdWMkxuUnZjR2xoTG01NmdnQ0hCQW9FYjJXSEJLd1JBQUdICkJBcUFBQUV3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUlnVGVnSnluemkwZVl1ZHE4cU9LR3hOaXc1YlZheGkyK2sKbHV3NGVNUzVwOFFDSVFEUWRsaTY0LzJ6cldQajlQNjVYWFlSMUR0ME1pMzNERnRnQURQTHdPNnFtQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=", | |
| "usages": [ | |
| "digital signature", | |
| "key encipherment", | |
| "server auth" | |
| ], | |
| "username": "system:admin" | |
| }, | |
| "status": { | |
| "certificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQWFLZ0F3SUJBZ0lVQ3NzVlZMeTY4clhBcVhwbnczekEvQTg1TXRVd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRWtNQ0lHQTFVRUF3d2JiM0JsYm5Ob2FXWjBMWE5wWjI1bGNrQXhOVFV6TVRJNE56UTFNQjRYRFRFNQpNRE15TWpBd01ESXdNRm9YRFRJd01ETXlNVEF3TURJd01Gb3dSREVWTUJNR0ExVUVDaE1NYzNsemRHVnRPbTV2ClpHVnpNU3N3S1FZRFZRUURFeUp6ZVhOMFpXMDZibTlrWlRwck9ITXRZM1JzTURFdVpHVjJMblJ2Y0dsaExtNTYKTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFbmJJU2FSajZHMVpQR2RMUWlDL1lpekpxOXFWUwpBWEorSnJ6N3d1QTUveDNEZ1d5emJSQWN3bkZGbWFRUTZQZk1VWWsyY3A3MnZtS1NORVNvOUlUYWRLT0JqRENCCmlUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3d0NnWUlLd1lCQlFVSEF3RXdEQVlEVlIwVEFRSC8KQkFJd0FEQWRCZ05WSFE0RUZnUVVQTzJvQWlTaEc0bGlLZTRGSDFQTGltb0MwOHN3TlFZRFZSMFJCQzR3TElJVwphemh6TFdOMGJEQXhMbVJsZGk1MGIzQnBZUzV1ZW9JQWh3UUtCRzlsaHdTc0VRQUJod1FLZ0FBQk1BMEdDU3FHClNJYjNEUUVCQ3dVQUE0SUJBUUF3YmNITEVScHZTQkVZQUFUMlduVWlGOFppNE5rUGs5ZWRReisvcGJkeGNOeEIKUmZmSzFpRFNSYmRDNjh4ZjBHUjJxdklDWDdwMVhRUURueXJ3K2JvR2JUMUdxUVB4Z1RBTXdHVUtmcTFoMENWbQo5a0g5VE9sNTFWczRYZkhxZFZzdFA1Z3RUT0FpUFNzaWdVSmM3L3FKSHdNeHQ3aytNVmlJY1V1aUd1SjljRWFsClJ0TjZLeHBBTGx4WVA4aVZETTY4VTJISTgwalRrVFRjbW1yb2VzRnlPb1BtSlN5ZjRCNWExWWllcTB6SGNKcEwKYlRsSmpWTDl5QTRZQXdtQ0JzSmxyOCtVMlVweUhYY0hscERmV1ZvT1krRDhaQ2IzaFNRdnBlVkdJNE5OTCswOApQNVd4Ui9LK0xORElGTDVYR1ZDSG5rbm5IL041alZvNVEwUGY5bkZ5Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K", | |
| "conditions": [ | |
| { | |
| "lastUpdateTime": "2019-03-22T00:06:42Z", | |
| "message": "This CSR was approved by kubectl certificate approve.", | |
| "reason": "KubectlApprove", | |
| "type": "Approved" | |
| } | |
| ] | |
| } | |
| }, | |
| { | |
| "apiVersion": "certificates.k8s.io/v1beta1", | |
| "kind": "CertificateSigningRequest", | |
| "metadata": { | |
| "creationTimestamp": "2019-03-22T00:04:51Z", | |
| "generateName": "csr-", | |
| "name": "csr-v85dj", | |
| "namespace": "", | |
| "resourceVersion": "19857", | |
| "selfLink": "/apis/certificates.k8s.io/v1beta1/certificatesigningrequests/csr-v85dj", | |
| "uid": "1d5cb3b4-4c36-11e9-add9-005056aa572c" | |
| }, | |
| "spec": { | |
| "groups": [ | |
| "system:masters", | |
| "system:cluster-admins", | |
| "system:authenticated" | |
| ], | |
| "request": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQlNEQ0I3Z0lCQURCRU1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14S3pBcEJnTlZCQU1USW5ONQpjM1JsYlRwdWIyUmxPbXM0Y3kxamRHd3dNUzVrWlhZdWRHOXdhV0V1Ym5vd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxCmhrak9QUU1CQndOQ0FBU09yM1V1SXRpSmJSNU5QN2szOXBLa1FFZGRBQm01MHhBRGw2YmJtQ3VoaVMxZUt1dXMKbnNLU3VDakdRTEE1clhTYm9MN2FlZnJudzYra256Zys5WnNjb0Vnd1JnWUpLb1pJaHZjTkFRa09NVGt3TnpBMQpCZ05WSFJFRUxqQXNnaFpyT0hNdFkzUnNNREV1WkdWMkxuUnZjR2xoTG01NmdnQ0hCQW9FYjJXSEJLd1JBQUdICkJBcUFBQUV3Q2dZSUtvWkl6ajBFQXdJRFNRQXdSZ0loQVBFWlV4L0M0UHFncWhpKytBU3NCbWNCN3Z0SGNTcjMKUWJhL2dSS2hIUWwvQWlFQTJQbzVDK1lzQmFMcWlCMU1NcGQ4THFIS3d3ck0yL0NKczN0RHQyWDRwMkU9Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=", | |
| "usages": [ | |
| "digital signature", | |
| "key encipherment", | |
| "server auth" | |
| ], | |
| "username": "system:admin" | |
| }, | |
| "status": { | |
| "certificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQWFLZ0F3SUJBZ0lVTFVQcFY4eitRRHZseDhpa3AydjBpRTA2citRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRWtNQ0lHQTFVRUF3d2JiM0JsYm5Ob2FXWjBMWE5wWjI1bGNrQXhOVFV6TVRJNE56UTFNQjRYRFRFNQpNRE15TWpBd01ESXdNRm9YRFRJd01ETXlNVEF3TURJd01Gb3dSREVWTUJNR0ExVUVDaE1NYzNsemRHVnRPbTV2ClpHVnpNU3N3S1FZRFZRUURFeUp6ZVhOMFpXMDZibTlrWlRwck9ITXRZM1JzTURFdVpHVjJMblJ2Y0dsaExtNTYKTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFanE5MUxpTFlpVzBlVFQrNU4vYVNwRUJIWFFBWgp1ZE1RQTVlbTI1Z3JvWWt0WGlycnJKN0NrcmdveGtDd09hMTBtNkMrMm5uNjU4T3ZwSjg0UHZXYkhLT0JqRENCCmlUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3d0NnWUlLd1lCQlFVSEF3RXdEQVlEVlIwVEFRSC8KQkFJd0FEQWRCZ05WSFE0RUZnUVVwRGtRRjNXS3ArdmlhSmdlVWlBbHJPRXNueWt3TlFZRFZSMFJCQzR3TElJVwphemh6TFdOMGJEQXhMbVJsZGk1MGIzQnBZUzV1ZW9JQWh3UUtCRzlsaHdTc0VRQUJod1FLZ0FBQk1BMEdDU3FHClNJYjNEUUVCQ3dVQUE0SUJBUUNDL0JJL0hKYXBhWG1kTUpsd2MvejhlMkRiazBVSWp3UlhjM0lET0FxckRoeGsKR2VOd0tNcUJJc2Fic1pWcXlzMWtqSVBaS0VFWTdwVHFhZU54blFRT2FucWZYMmNuZTV1aGEvbUo0U2NmRUorQQpQOGJaSU1GRzVIS1FPNlBPOSt5SXhKR3I4emFpM1hsd2s0YUNGZ2xCWEQ3SzRsZ0Noc0xXZjdZQWtKcmRKRGEvCkRhNmpzbFQvQithZlpKbmJZeUJSZkNqeVdkUE5lbXdsU1V2ZS9SZ3Z6eXZsMllmTFlpMWJ3bmw2RXNrTW84aWoKUVdBR0tpd3FkUnNCdGNOSStJY1JHYzFDZWY4ZFFhdGVGWFl6c2VINHdONWpwRWZoYllMOVRCbEJ1QnQwWVpZOQpJTlZxUVRGVEZQVzBJWmZQcGU4ZXhUOFR3TUJlRjBoS05uUk03blZICi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K", | |
| "conditions": [ | |
| { | |
| "lastUpdateTime": "2019-03-22T00:06:36Z", | |
| "message": "This CSR was approved by kubectl certificate approve.", | |
| "reason": "KubectlApprove", | |
| "type": "Approved" | |
| } | |
| ] | |
| } | |
| } | |
| ] | |
| } | |
| Using module file /home/user/git/sourcetopia/openshift-install/openshift/ansible/roles/lib_openshift/library/oc_csr_approve.py | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'echo ~deploy && sleep 0'"'"'' | |
| <k8s-ctl01.dev.mydomain.com> (0, '/home/deploy\n', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712 `" && echo ansible-tmp-1553214630.19-148976102485712="` echo /home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712 `" ) && sleep 0'"'"'' | |
| <k8s-ctl01.dev.mydomain.com> (0, 'ansible-tmp-1553214630.19-148976102485712=/home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712\n', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| <k8s-ctl01.dev.mydomain.com> PUT /home/user/.ansible/tmp/ansible-local-10931AT0_NQ/tmp76X_RJ TO /home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712/oc_csr_approve.py | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d '[k8s-ctl01.dev.mydomain.com]' | |
| <k8s-ctl01.dev.mydomain.com> (0, 'sftp> put /home/user/.ansible/tmp/ansible-local-10931AT0_NQ/tmp76X_RJ /home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712/oc_csr_approve.py\n', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 2\r\ndebug2: Server supports extension "[email protected]" revision 2\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug3: Sent message fd 15 T:16 I:1\r\ndebug3: SSH_FXP_REALPATH . -> /home/deploy size 0\r\ndebug3: Looking up /home/user/.ansible/tmp/ansible-local-10931AT0_NQ/tmp76X_RJ\r\ndebug3: Sent message fd 15 T:17 I:2\r\ndebug3: Received stat reply T:101 I:2\r\ndebug1: Couldn\'t stat remote file: No such file or directory\r\ndebug3: Sent message SSH2_FXP_OPEN I:3 P:/home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712/oc_csr_approve.py\r\ndebug3: Sent message SSH2_FXP_WRITE I:4 O:0 S:32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 4 32768 bytes at 0\r\ndebug3: Sent message SSH2_FXP_WRITE I:5 O:32768 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:65536 S:3446\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 5 32768 bytes at 32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 3446 bytes at 65536\r\ndebug3: Sent message SSH2_FXP_CLOSE I:4\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'chmod u+x /home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712/ /home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712/oc_csr_approve.py && sleep 0'"'"'' | |
| <k8s-ctl01.dev.mydomain.com> (0, '', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d -tt k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'sudo -H -S -p "[sudo via ansible, key=tuddxsandiilnpnlwrnfjjxpssvtizlb] password: " -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-tuddxsandiilnpnlwrnfjjxpssvtizlb; /usr/bin/python /home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712/oc_csr_approve.py'"'"'"'"'"'"'"'"' && sleep 0'"'"'' | |
| Escalation succeeded | |
| <k8s-ctl01.dev.mydomain.com> (1, '\r\n\r\n{"server_csrs": null, "exception": " File \\"/tmp/ansible_NR_QE0/ansible_module_oc_csr_approve.py\\", line 24, in <module>\\n from json.decoder import JSONDecodeError\\n", "raw_failures": [], "oc_get_nodes": {"items": [{"status": {"capacity": {"hugepages-1Gi": "0", "hugepages-2Mi": "0", "pods": "250", "cpu": "4", "memory": "16266800Ki"}, "addresses": [{"type": "InternalIP", "address": "10.4.111.101"}, {"type": "Hostname", "address": "k8s-ctl01.dev.mydomain.com"}], "nodeInfo": {"kernelVersion": "3.10.0-957.el7.x86_64", "kubeletVersion": "v1.11.0+d4cacc0", "containerRuntimeVersion": "docker://1.13.1", "machineID": "bfee8bc7a3cd4673a31ac520eb963eb2", "kubeProxyVersion": "v1.11.0+d4cacc0", "bootID": "c4c9e366-57c7-4334-933b-dab93702b583", "osImage": "CentOS Linux 7 (Core)", "architecture": "amd64", "systemUUID": "1C582A42-E6DF-7E8C-DBB8-DB6315262B0C", "operatingSystem": "linux"}, "allocatable": {"hugepages-1Gi": "0", "hugepages-2Mi": "0", "pods": "250", "cpu": "4", "memory": "16164400Ki"}, "daemonEndpoints": {"kubeletEndpoint": {"Port": 10250}}, "images": [{"sizeBytes": 1172234437, "names": ["docker.io/openshift/origin-node@sha256:fd552ffd0bdbbcf6c5c646ec7c086b2fafde0aa560711038b1e35c524d867a10", "docker.io/openshift/origin-node:v3.11.0"]}, {"sizeBytes": 1166879223, "names": ["docker.io/openshift/origin-node@sha256:d198706b58f39b87af543cf73e37cc50632a214bac107d70b6d99034982d214a"]}, {"sizeBytes": 828734067, "names": ["docker.io/openshift/origin-control-plane@sha256:508dad04d4c6c7c6ea1e50ac16a5aeffb3854d391b392d74a68c1bf767c608e0", "docker.io/openshift/origin-control-plane:v3.11", "docker.io/openshift/origin-control-plane:v3.11.0"]}, {"sizeBytes": 826232564, "names": ["docker.io/openshift/origin-control-plane@sha256:bf28e6729781f2d68c9ea121cee5c649a084ae9a4dadefded9321e2e381a2549"]}, {"sizeBytes": 383673671, "names": ["docker.io/openshift/origin-deployer@sha256:88eedcec3c877bea09653f037174f50356071e265d93889964a954a73b03aa8e", "docker.io/openshift/origin-deployer:v3.11", "docker.io/openshift/origin-deployer:v3.11.0"]}, {"sizeBytes": 261526128, "names": ["docker.io/openshift/origin-pod@sha256:91f0f130c991c2a6a9f0e9da43f7e56296bacdc46be0cd768125ed3e6f7ef09d", "docker.io/openshift/origin-pod:v3.11.0"]}, {"sizeBytes": 258456390, "names": ["docker.io/openshift/origin-pod@sha256:67b51982ad05b446e4045b7e2507768c2d25b2eb61ccadd7b6a718b1262c25b8"]}, {"sizeBytes": 37269372, "names": ["quay.io/coreos/etcd@sha256:43fbc8a457aa0cb887da63d74a48659e13947cb74b96a53ba8f47abb6172a948", "quay.io/coreos/etcd:v3.2.22"]}], "conditions": [{"status": "False", "lastTransitionTime": "2019-03-21T20:38:32Z", "reason": "KubeletHasSufficientDisk", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet has sufficient disk space available", "type": "OutOfDisk"}, {"status": "False", "lastTransitionTime": "2019-03-21T20:38:32Z", "reason": "KubeletHasSufficientMemory", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet has sufficient memory available", "type": "MemoryPressure"}, {"status": "False", "lastTransitionTime": "2019-03-21T20:38:32Z", "reason": "KubeletHasNoDiskPressure", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet has no disk pressure", "type": "DiskPressure"}, {"status": "False", "lastTransitionTime": "2019-03-21T20:38:32Z", "reason": "KubeletHasSufficientPID", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet has sufficient PID available", "type": "PIDPressure"}, {"status": "True", "lastTransitionTime": "2019-03-22T00:27:39Z", "reason": "KubeletReady", "lastHeartbeatTime": "2019-03-22T00:30:39Z", "message": "kubelet is posting ready status", "type": "Ready"}]}, "kind": "Node", "spec": {}, "apiVersion": "v1", "metadata": {"name": "k8s-ctl01.dev.mydomain.com", "labels": {"beta.kubernetes.io/os": "linux", "node-role.kubernetes.io/master": "true", "kubernetes.io/hostname": "k8s-ctl01.dev.mydomain.com", "node-role.kubernetes.io/compute": "true", "node-role.kubernetes.io/infra": "true", "beta.kubernetes.io/arch": "amd64"}, "namespace": "", "resourceVersion": "21959", "creationTimestamp": "2019-03-21T20:38:37Z", "annotations": {"volumes.kubernetes.io/controller-managed-attach-detach": "true", "node.openshift.io/md5sum": "a4305b3db4427b8d4bd21c1a11115c5d"}, "selfLink": "/api/v1/nodes/k8s-ctl01.dev.mydomain.com", "uid": "4dfe8828-4c19-11e9-add9-005056aa572c"}}], "kind": "List", "apiVersion": "v1", "metadata": {"selfLink": "", "resourceVersion": ""}}, "changed": false, "client_csrs": {}, "failed": true, "state": "unknown", "all_subjects_found": ["subject=/O=system:nodes/CN=system:node:k8s-ctl01.dev.mydomain.com\\n", "subject=/O=system:nodes/CN=system:node:k8s-ctl01.dev.mydomain.com\\n"], "client_approve_results": [], "rc": 0, "invocation": {"module_args": {"oc_bin": "oc", "oc_conf": "/etc/origin/master/admin.kubeconfig", "node_list": ["k8s-worker01.dev.mydomain.com", "k8s-worker02.dev.mydomain.com", "k8s-worker03.dev.mydomain.com", "k8s-ctl01.dev.mydomain.com"]}}, "unwanted_csrs": [{"status": {"conditions": [{"message": "This CSR was approved by kubectl certificate approve.", "type": "Approved", "reason": "KubectlApprove", "lastUpdateTime": "2019-03-22T00:06:42Z"}], "certificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQWFLZ0F3SUJBZ0lVQ3NzVlZMeTY4clhBcVhwbnczekEvQTg1TXRVd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRWtNQ0lHQTFVRUF3d2JiM0JsYm5Ob2FXWjBMWE5wWjI1bGNrQXhOVFV6TVRJNE56UTFNQjRYRFRFNQpNRE15TWpBd01ESXdNRm9YRFRJd01ETXlNVEF3TURJd01Gb3dSREVWTUJNR0ExVUVDaE1NYzNsemRHVnRPbTV2ClpHVnpNU3N3S1FZRFZRUURFeUp6ZVhOMFpXMDZibTlrWlRwck9ITXRZM1JzTURFdVpHVjJMblJ2Y0dsaExtNTYKTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFbmJJU2FSajZHMVpQR2RMUWlDL1lpekpxOXFWUwpBWEorSnJ6N3d1QTUveDNEZ1d5emJSQWN3bkZGbWFRUTZQZk1VWWsyY3A3MnZtS1NORVNvOUlUYWRLT0JqRENCCmlUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3d0NnWUlLd1lCQlFVSEF3RXdEQVlEVlIwVEFRSC8KQkFJd0FEQWRCZ05WSFE0RUZnUVVQTzJvQWlTaEc0bGlLZTRGSDFQTGltb0MwOHN3TlFZRFZSMFJCQzR3TElJVwphemh6TFdOMGJEQXhMbVJsZGk1MGIzQnBZUzV1ZW9JQWh3UUtCRzlsaHdTc0VRQUJod1FLZ0FBQk1BMEdDU3FHClNJYjNEUUVCQ3dVQUE0SUJBUUF3YmNITEVScHZTQkVZQUFUMlduVWlGOFppNE5rUGs5ZWRReisvcGJkeGNOeEIKUmZmSzFpRFNSYmRDNjh4ZjBHUjJxdklDWDdwMVhRUURueXJ3K2JvR2JUMUdxUVB4Z1RBTXdHVUtmcTFoMENWbQo5a0g5VE9sNTFWczRYZkhxZFZzdFA1Z3RUT0FpUFNzaWdVSmM3L3FKSHdNeHQ3aytNVmlJY1V1aUd1SjljRWFsClJ0TjZLeHBBTGx4WVA4aVZETTY4VTJISTgwalRrVFRjbW1yb2VzRnlPb1BtSlN5ZjRCNWExWWllcTB6SGNKcEwKYlRsSmpWTDl5QTRZQXdtQ0JzSmxyOCtVMlVweUhYY0hscERmV1ZvT1krRDhaQ2IzaFNRdnBlVkdJNE5OTCswOApQNVd4Ui9LK0xORElGTDVYR1ZDSG5rbm5IL041alZvNVEwUGY5bkZ5Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"}, "kind": "CertificateSigningRequest", "spec": {"username": "system:admin", "usages": ["digital signature", "key encipherment", "server auth"], "request": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQlJ6Q0I3Z0lCQURCRU1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14S3pBcEJnTlZCQU1USW5ONQpjM1JsYlRwdWIyUmxPbXM0Y3kxamRHd3dNUzVrWlhZdWRHOXdhV0V1Ym5vd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxCmhrak9QUU1CQndOQ0FBU2RzaEpwR1BvYlZrOFowdENJTDlpTE1tcjJwVklCY240bXZQdkM0RG4vSGNPQmJMTnQKRUJ6Q2NVV1pwQkRvOTh4UmlUWnludmErWXBJMFJLajBoTnAwb0Vnd1JnWUpLb1pJaHZjTkFRa09NVGt3TnpBMQpCZ05WSFJFRUxqQXNnaFpyT0hNdFkzUnNNREV1WkdWMkxuUnZjR2xoTG01NmdnQ0hCQW9FYjJXSEJLd1JBQUdICkJBcUFBQUV3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUlnVGVnSnluemkwZVl1ZHE4cU9LR3hOaXc1YlZheGkyK2sKbHV3NGVNUzVwOFFDSVFEUWRsaTY0LzJ6cldQajlQNjVYWFlSMUR0ME1pMzNERnRnQURQTHdPNnFtQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=", "groups": ["system:masters", "system:cluster-admins", "system:authenticated"]}, "apiVersion": "certificates.k8s.io/v1beta1", "metadata": {"name": "csr-bsf6g", "namespace": "", "resourceVersion": "19872", "generateName": "csr-", "creationTimestamp": "2019-03-22T00:06:21Z", "selfLink": "/apis/certificates.k8s.io/v1beta1/certificatesigningrequests/csr-bsf6g", "uid": "52f26e8c-4c36-11e9-add9-005056aa572c"}}, {"status": {"conditions": [{"message": "This CSR was approved by kubectl certificate approve.", "type": "Approved", "reason": "KubectlApprove", "lastUpdateTime": "2019-03-22T00:06:36Z"}], "certificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQWFLZ0F3SUJBZ0lVTFVQcFY4eitRRHZseDhpa3AydjBpRTA2citRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRWtNQ0lHQTFVRUF3d2JiM0JsYm5Ob2FXWjBMWE5wWjI1bGNrQXhOVFV6TVRJNE56UTFNQjRYRFRFNQpNRE15TWpBd01ESXdNRm9YRFRJd01ETXlNVEF3TURJd01Gb3dSREVWTUJNR0ExVUVDaE1NYzNsemRHVnRPbTV2ClpHVnpNU3N3S1FZRFZRUURFeUp6ZVhOMFpXMDZibTlrWlRwck9ITXRZM1JzTURFdVpHVjJMblJ2Y0dsaExtNTYKTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFanE5MUxpTFlpVzBlVFQrNU4vYVNwRUJIWFFBWgp1ZE1RQTVlbTI1Z3JvWWt0WGlycnJKN0NrcmdveGtDd09hMTBtNkMrMm5uNjU4T3ZwSjg0UHZXYkhLT0JqRENCCmlUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3d0NnWUlLd1lCQlFVSEF3RXdEQVlEVlIwVEFRSC8KQkFJd0FEQWRCZ05WSFE0RUZnUVVwRGtRRjNXS3ArdmlhSmdlVWlBbHJPRXNueWt3TlFZRFZSMFJCQzR3TElJVwphemh6TFdOMGJEQXhMbVJsZGk1MGIzQnBZUzV1ZW9JQWh3UUtCRzlsaHdTc0VRQUJod1FLZ0FBQk1BMEdDU3FHClNJYjNEUUVCQ3dVQUE0SUJBUUNDL0JJL0hKYXBhWG1kTUpsd2MvejhlMkRiazBVSWp3UlhjM0lET0FxckRoeGsKR2VOd0tNcUJJc2Fic1pWcXlzMWtqSVBaS0VFWTdwVHFhZU54blFRT2FucWZYMmNuZTV1aGEvbUo0U2NmRUorQQpQOGJaSU1GRzVIS1FPNlBPOSt5SXhKR3I4emFpM1hsd2s0YUNGZ2xCWEQ3SzRsZ0Noc0xXZjdZQWtKcmRKRGEvCkRhNmpzbFQvQithZlpKbmJZeUJSZkNqeVdkUE5lbXdsU1V2ZS9SZ3Z6eXZsMllmTFlpMWJ3bmw2RXNrTW84aWoKUVdBR0tpd3FkUnNCdGNOSStJY1JHYzFDZWY4ZFFhdGVGWFl6c2VINHdONWpwRWZoYllMOVRCbEJ1QnQwWVpZOQpJTlZxUVRGVEZQVzBJWmZQcGU4ZXhUOFR3TUJlRjBoS05uUk03blZICi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"}, "kind": "CertificateSigningRequest", "spec": {"username": "system:admin", "usages": ["digital signature", "key encipherment", "server auth"], "request": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQlNEQ0I3Z0lCQURCRU1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14S3pBcEJnTlZCQU1USW5ONQpjM1JsYlRwdWIyUmxPbXM0Y3kxamRHd3dNUzVrWlhZdWRHOXdhV0V1Ym5vd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxCmhrak9QUU1CQndOQ0FBU09yM1V1SXRpSmJSNU5QN2szOXBLa1FFZGRBQm01MHhBRGw2YmJtQ3VoaVMxZUt1dXMKbnNLU3VDakdRTEE1clhTYm9MN2FlZnJudzYra256Zys5WnNjb0Vnd1JnWUpLb1pJaHZjTkFRa09NVGt3TnpBMQpCZ05WSFJFRUxqQXNnaFpyT0hNdFkzUnNNREV1WkdWMkxuUnZjR2xoTG01NmdnQ0hCQW9FYjJXSEJLd1JBQUdICkJBcUFBQUV3Q2dZSUtvWkl6ajBFQXdJRFNRQXdSZ0loQVBFWlV4L0M0UHFncWhpKytBU3NCbWNCN3Z0SGNTcjMKUWJhL2dSS2hIUWwvQWlFQTJQbzVDK1lzQmFMcWlCMU1NcGQ4THFIS3d3ck0yL0NKczN0RHQyWDRwMkU9Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=", "groups": ["system:masters", "system:cluster-admins", "system:authenticated"]}, "apiVersion": "certificates.k8s.io/v1beta1", "metadata": {"name": "csr-v85dj", "namespace": "", "resourceVersion": "19857", "generateName": "csr-", "creationTimestamp": "2019-03-22T00:04:51Z", "selfLink": "/apis/certificates.k8s.io/v1beta1/certificatesigningrequests/csr-v85dj", "uid": "1d5cb3b4-4c36-11e9-add9-005056aa572c"}}], "server_approve_results": [], "msg": "Could not find csr for nodes: k8s-worker03.dev.mydomain.com, k8s-worker02.dev.mydomain.com, k8s-worker01.dev.mydomain.com"}\r\n', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to k8s-ctl01.dev.mydomain.com closed.\r\n') | |
| <k8s-ctl01.dev.mydomain.com> ESTABLISH SSH CONNECTION FOR USER: deploy | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=deploy) | |
| <k8s-ctl01.dev.mydomain.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10) | |
| <k8s-ctl01.dev.mydomain.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/user/.ansible/cp/2ea0bd867d) | |
| <k8s-ctl01.dev.mydomain.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -o ControlPath=/home/user/.ansible/cp/2ea0bd867d k8s-ctl01.dev.mydomain.com '/bin/sh -c '"'"'rm -f -r /home/deploy/.ansible/tmp/ansible-tmp-1553214630.19-148976102485712/ > /dev/null 2>&1 && sleep 0'"'"'' | |
| <k8s-ctl01.dev.mydomain.com> (0, '', 'OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 12791\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n') | |
| The full traceback is: | |
| File "/tmp/ansible_NR_QE0/ansible_module_oc_csr_approve.py", line 24, in <module> | |
| from json.decoder import JSONDecodeError | |
| fatal: [k8s-ctl01.dev.mydomain.com]: FAILED! => { | |
| "all_subjects_found": [ | |
| "subject=/O=system:nodes/CN=system:node:k8s-ctl01.dev.mydomain.com\n", | |
| "subject=/O=system:nodes/CN=system:node:k8s-ctl01.dev.mydomain.com\n" | |
| ], | |
| "attempts": 30, | |
| "changed": false, | |
| "client_approve_results": [], | |
| "client_csrs": {}, | |
| "invocation": { | |
| "module_args": { | |
| "node_list": [ | |
| "k8s-worker01.dev.mydomain.com", | |
| "k8s-worker02.dev.mydomain.com", | |
| "k8s-worker03.dev.mydomain.com", | |
| "k8s-ctl01.dev.mydomain.com" | |
| ], | |
| "oc_bin": "oc", | |
| "oc_conf": "/etc/origin/master/admin.kubeconfig" | |
| } | |
| }, | |
| "msg": "Could not find csr for nodes: k8s-worker03.dev.mydomain.com, k8s-worker02.dev.mydomain.com, k8s-worker01.dev.mydomain.com", | |
| "oc_get_nodes": { | |
| "apiVersion": "v1", | |
| "items": [ | |
| { | |
| "apiVersion": "v1", | |
| "kind": "Node", | |
| "metadata": { | |
| "annotations": { | |
| "node.openshift.io/md5sum": "a4305b3db4427b8d4bd21c1a11115c5d", | |
| "volumes.kubernetes.io/controller-managed-attach-detach": "true" | |
| }, | |
| "creationTimestamp": "2019-03-21T20:38:37Z", | |
| "labels": { | |
| "beta.kubernetes.io/arch": "amd64", | |
| "beta.kubernetes.io/os": "linux", | |
| "kubernetes.io/hostname": "k8s-ctl01.dev.mydomain.com", | |
| "node-role.kubernetes.io/compute": "true", | |
| "node-role.kubernetes.io/infra": "true", | |
| "node-role.kubernetes.io/master": "true" | |
| }, | |
| "name": "k8s-ctl01.dev.mydomain.com", | |
| "namespace": "", | |
| "resourceVersion": "21959", | |
| "selfLink": "/api/v1/nodes/k8s-ctl01.dev.mydomain.com", | |
| "uid": "4dfe8828-4c19-11e9-add9-005056aa572c" | |
| }, | |
| "spec": {}, | |
| "status": { | |
| "addresses": [ | |
| { | |
| "address": "10.4.111.101", | |
| "type": "InternalIP" | |
| }, | |
| { | |
| "address": "k8s-ctl01.dev.mydomain.com", | |
| "type": "Hostname" | |
| } | |
| ], | |
| "allocatable": { | |
| "cpu": "4", | |
| "hugepages-1Gi": "0", | |
| "hugepages-2Mi": "0", | |
| "memory": "16164400Ki", | |
| "pods": "250" | |
| }, | |
| "capacity": { | |
| "cpu": "4", | |
| "hugepages-1Gi": "0", | |
| "hugepages-2Mi": "0", | |
| "memory": "16266800Ki", | |
| "pods": "250" | |
| }, | |
| "conditions": [ | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-21T20:38:32Z", | |
| "message": "kubelet has sufficient disk space available", | |
| "reason": "KubeletHasSufficientDisk", | |
| "status": "False", | |
| "type": "OutOfDisk" | |
| }, | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-21T20:38:32Z", | |
| "message": "kubelet has sufficient memory available", | |
| "reason": "KubeletHasSufficientMemory", | |
| "status": "False", | |
| "type": "MemoryPressure" | |
| }, | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-21T20:38:32Z", | |
| "message": "kubelet has no disk pressure", | |
| "reason": "KubeletHasNoDiskPressure", | |
| "status": "False", | |
| "type": "DiskPressure" | |
| }, | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-21T20:38:32Z", | |
| "message": "kubelet has sufficient PID available", | |
| "reason": "KubeletHasSufficientPID", | |
| "status": "False", | |
| "type": "PIDPressure" | |
| }, | |
| { | |
| "lastHeartbeatTime": "2019-03-22T00:30:39Z", | |
| "lastTransitionTime": "2019-03-22T00:27:39Z", | |
| "message": "kubelet is posting ready status", | |
| "reason": "KubeletReady", | |
| "status": "True", | |
| "type": "Ready" | |
| } | |
| ], | |
| "daemonEndpoints": { | |
| "kubeletEndpoint": { | |
| "Port": 10250 | |
| } | |
| }, | |
| "images": [ | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-node@sha256:fd552ffd0bdbbcf6c5c646ec7c086b2fafde0aa560711038b1e35c524d867a10", | |
| "docker.io/openshift/origin-node:v3.11.0" | |
| ], | |
| "sizeBytes": 1172234437 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-node@sha256:d198706b58f39b87af543cf73e37cc50632a214bac107d70b6d99034982d214a" | |
| ], | |
| "sizeBytes": 1166879223 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-control-plane@sha256:508dad04d4c6c7c6ea1e50ac16a5aeffb3854d391b392d74a68c1bf767c608e0", | |
| "docker.io/openshift/origin-control-plane:v3.11", | |
| "docker.io/openshift/origin-control-plane:v3.11.0" | |
| ], | |
| "sizeBytes": 828734067 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-control-plane@sha256:bf28e6729781f2d68c9ea121cee5c649a084ae9a4dadefded9321e2e381a2549" | |
| ], | |
| "sizeBytes": 826232564 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-deployer@sha256:88eedcec3c877bea09653f037174f50356071e265d93889964a954a73b03aa8e", | |
| "docker.io/openshift/origin-deployer:v3.11", | |
| "docker.io/openshift/origin-deployer:v3.11.0" | |
| ], | |
| "sizeBytes": 383673671 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-pod@sha256:91f0f130c991c2a6a9f0e9da43f7e56296bacdc46be0cd768125ed3e6f7ef09d", | |
| "docker.io/openshift/origin-pod:v3.11.0" | |
| ], | |
| "sizeBytes": 261526128 | |
| }, | |
| { | |
| "names": [ | |
| "docker.io/openshift/origin-pod@sha256:67b51982ad05b446e4045b7e2507768c2d25b2eb61ccadd7b6a718b1262c25b8" | |
| ], | |
| "sizeBytes": 258456390 | |
| }, | |
| { | |
| "names": [ | |
| "quay.io/coreos/etcd@sha256:43fbc8a457aa0cb887da63d74a48659e13947cb74b96a53ba8f47abb6172a948", | |
| "quay.io/coreos/etcd:v3.2.22" | |
| ], | |
| "sizeBytes": 37269372 | |
| } | |
| ], | |
| "nodeInfo": { | |
| "architecture": "amd64", | |
| "bootID": "c4c9e366-57c7-4334-933b-dab93702b583", | |
| "containerRuntimeVersion": "docker://1.13.1", | |
| "kernelVersion": "3.10.0-957.el7.x86_64", | |
| "kubeProxyVersion": "v1.11.0+d4cacc0", | |
| "kubeletVersion": "v1.11.0+d4cacc0", | |
| "machineID": "bfee8bc7a3cd4673a31ac520eb963eb2", | |
| "operatingSystem": "linux", | |
| "osImage": "CentOS Linux 7 (Core)", | |
| "systemUUID": "1C582A42-E6DF-7E8C-DBB8-DB6315262B0C" | |
| } | |
| } | |
| } | |
| ], | |
| "kind": "List", | |
| "metadata": { | |
| "resourceVersion": "", | |
| "selfLink": "" | |
| } | |
| }, | |
| "raw_failures": [], | |
| "rc": 0, | |
| "server_approve_results": [], | |
| "server_csrs": null, | |
| "state": "unknown", | |
| "unwanted_csrs": [ | |
| { | |
| "apiVersion": "certificates.k8s.io/v1beta1", | |
| "kind": "CertificateSigningRequest", | |
| "metadata": { | |
| "creationTimestamp": "2019-03-22T00:06:21Z", | |
| "generateName": "csr-", | |
| "name": "csr-bsf6g", | |
| "namespace": "", | |
| "resourceVersion": "19872", | |
| "selfLink": "/apis/certificates.k8s.io/v1beta1/certificatesigningrequests/csr-bsf6g", | |
| "uid": "52f26e8c-4c36-11e9-add9-005056aa572c" | |
| }, | |
| "spec": { | |
| "groups": [ | |
| "system:masters", | |
| "system:cluster-admins", | |
| "system:authenticated" | |
| ], | |
| "request": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQlJ6Q0I3Z0lCQURCRU1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14S3pBcEJnTlZCQU1USW5ONQpjM1JsYlRwdWIyUmxPbXM0Y3kxamRHd3dNUzVrWlhZdWRHOXdhV0V1Ym5vd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxCmhrak9QUU1CQndOQ0FBU2RzaEpwR1BvYlZrOFowdENJTDlpTE1tcjJwVklCY240bXZQdkM0RG4vSGNPQmJMTnQKRUJ6Q2NVV1pwQkRvOTh4UmlUWnludmErWXBJMFJLajBoTnAwb0Vnd1JnWUpLb1pJaHZjTkFRa09NVGt3TnpBMQpCZ05WSFJFRUxqQXNnaFpyT0hNdFkzUnNNREV1WkdWMkxuUnZjR2xoTG01NmdnQ0hCQW9FYjJXSEJLd1JBQUdICkJBcUFBQUV3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUlnVGVnSnluemkwZVl1ZHE4cU9LR3hOaXc1YlZheGkyK2sKbHV3NGVNUzVwOFFDSVFEUWRsaTY0LzJ6cldQajlQNjVYWFlSMUR0ME1pMzNERnRnQURQTHdPNnFtQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=", | |
| "usages": [ | |
| "digital signature", | |
| "key encipherment", | |
| "server auth" | |
| ], | |
| "username": "system:admin" | |
| }, | |
| "status": { | |
| "certificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQWFLZ0F3SUJBZ0lVQ3NzVlZMeTY4clhBcVhwbnczekEvQTg1TXRVd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRWtNQ0lHQTFVRUF3d2JiM0JsYm5Ob2FXWjBMWE5wWjI1bGNrQXhOVFV6TVRJNE56UTFNQjRYRFRFNQpNRE15TWpBd01ESXdNRm9YRFRJd01ETXlNVEF3TURJd01Gb3dSREVWTUJNR0ExVUVDaE1NYzNsemRHVnRPbTV2ClpHVnpNU3N3S1FZRFZRUURFeUp6ZVhOMFpXMDZibTlrWlRwck9ITXRZM1JzTURFdVpHVjJMblJ2Y0dsaExtNTYKTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFbmJJU2FSajZHMVpQR2RMUWlDL1lpekpxOXFWUwpBWEorSnJ6N3d1QTUveDNEZ1d5emJSQWN3bkZGbWFRUTZQZk1VWWsyY3A3MnZtS1NORVNvOUlUYWRLT0JqRENCCmlUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3d0NnWUlLd1lCQlFVSEF3RXdEQVlEVlIwVEFRSC8KQkFJd0FEQWRCZ05WSFE0RUZnUVVQTzJvQWlTaEc0bGlLZTRGSDFQTGltb0MwOHN3TlFZRFZSMFJCQzR3TElJVwphemh6TFdOMGJEQXhMbVJsZGk1MGIzQnBZUzV1ZW9JQWh3UUtCRzlsaHdTc0VRQUJod1FLZ0FBQk1BMEdDU3FHClNJYjNEUUVCQ3dVQUE0SUJBUUF3YmNITEVScHZTQkVZQUFUMlduVWlGOFppNE5rUGs5ZWRReisvcGJkeGNOeEIKUmZmSzFpRFNSYmRDNjh4ZjBHUjJxdklDWDdwMVhRUURueXJ3K2JvR2JUMUdxUVB4Z1RBTXdHVUtmcTFoMENWbQo5a0g5VE9sNTFWczRYZkhxZFZzdFA1Z3RUT0FpUFNzaWdVSmM3L3FKSHdNeHQ3aytNVmlJY1V1aUd1SjljRWFsClJ0TjZLeHBBTGx4WVA4aVZETTY4VTJISTgwalRrVFRjbW1yb2VzRnlPb1BtSlN5ZjRCNWExWWllcTB6SGNKcEwKYlRsSmpWTDl5QTRZQXdtQ0JzSmxyOCtVMlVweUhYY0hscERmV1ZvT1krRDhaQ2IzaFNRdnBlVkdJNE5OTCswOApQNVd4Ui9LK0xORElGTDVYR1ZDSG5rbm5IL041alZvNVEwUGY5bkZ5Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K", | |
| "conditions": [ | |
| { | |
| "lastUpdateTime": "2019-03-22T00:06:42Z", | |
| "message": "This CSR was approved by kubectl certificate approve.", | |
| "reason": "KubectlApprove", | |
| "type": "Approved" | |
| } | |
| ] | |
| } | |
| }, | |
| { | |
| "apiVersion": "certificates.k8s.io/v1beta1", | |
| "kind": "CertificateSigningRequest", | |
| "metadata": { | |
| "creationTimestamp": "2019-03-22T00:04:51Z", | |
| "generateName": "csr-", | |
| "name": "csr-v85dj", | |
| "namespace": "", | |
| "resourceVersion": "19857", | |
| "selfLink": "/apis/certificates.k8s.io/v1beta1/certificatesigningrequests/csr-v85dj", | |
| "uid": "1d5cb3b4-4c36-11e9-add9-005056aa572c" | |
| }, | |
| "spec": { | |
| "groups": [ | |
| "system:masters", | |
| "system:cluster-admins", | |
| "system:authenticated" | |
| ], | |
| "request": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQlNEQ0I3Z0lCQURCRU1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14S3pBcEJnTlZCQU1USW5ONQpjM1JsYlRwdWIyUmxPbXM0Y3kxamRHd3dNUzVrWlhZdWRHOXdhV0V1Ym5vd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxCmhrak9QUU1CQndOQ0FBU09yM1V1SXRpSmJSNU5QN2szOXBLa1FFZGRBQm01MHhBRGw2YmJtQ3VoaVMxZUt1dXMKbnNLU3VDakdRTEE1clhTYm9MN2FlZnJudzYra256Zys5WnNjb0Vnd1JnWUpLb1pJaHZjTkFRa09NVGt3TnpBMQpCZ05WSFJFRUxqQXNnaFpyT0hNdFkzUnNNREV1WkdWMkxuUnZjR2xoTG01NmdnQ0hCQW9FYjJXSEJLd1JBQUdICkJBcUFBQUV3Q2dZSUtvWkl6ajBFQXdJRFNRQXdSZ0loQVBFWlV4L0M0UHFncWhpKytBU3NCbWNCN3Z0SGNTcjMKUWJhL2dSS2hIUWwvQWlFQTJQbzVDK1lzQmFMcWlCMU1NcGQ4THFIS3d3ck0yL0NKczN0RHQyWDRwMkU9Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=", | |
| "usages": [ | |
| "digital signature", | |
| "key encipherment", | |
| "server auth" | |
| ], | |
| "username": "system:admin" | |
| }, | |
| "status": { | |
| "certificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQWFLZ0F3SUJBZ0lVTFVQcFY4eitRRHZseDhpa3AydjBpRTA2citRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRWtNQ0lHQTFVRUF3d2JiM0JsYm5Ob2FXWjBMWE5wWjI1bGNrQXhOVFV6TVRJNE56UTFNQjRYRFRFNQpNRE15TWpBd01ESXdNRm9YRFRJd01ETXlNVEF3TURJd01Gb3dSREVWTUJNR0ExVUVDaE1NYzNsemRHVnRPbTV2ClpHVnpNU3N3S1FZRFZRUURFeUp6ZVhOMFpXMDZibTlrWlRwck9ITXRZM1JzTURFdVpHVjJMblJ2Y0dsaExtNTYKTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFanE5MUxpTFlpVzBlVFQrNU4vYVNwRUJIWFFBWgp1ZE1RQTVlbTI1Z3JvWWt0WGlycnJKN0NrcmdveGtDd09hMTBtNkMrMm5uNjU4T3ZwSjg0UHZXYkhLT0JqRENCCmlUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3d0NnWUlLd1lCQlFVSEF3RXdEQVlEVlIwVEFRSC8KQkFJd0FEQWRCZ05WSFE0RUZnUVVwRGtRRjNXS3ArdmlhSmdlVWlBbHJPRXNueWt3TlFZRFZSMFJCQzR3TElJVwphemh6TFdOMGJEQXhMbVJsZGk1MGIzQnBZUzV1ZW9JQWh3UUtCRzlsaHdTc0VRQUJod1FLZ0FBQk1BMEdDU3FHClNJYjNEUUVCQ3dVQUE0SUJBUUNDL0JJL0hKYXBhWG1kTUpsd2MvejhlMkRiazBVSWp3UlhjM0lET0FxckRoeGsKR2VOd0tNcUJJc2Fic1pWcXlzMWtqSVBaS0VFWTdwVHFhZU54blFRT2FucWZYMmNuZTV1aGEvbUo0U2NmRUorQQpQOGJaSU1GRzVIS1FPNlBPOSt5SXhKR3I4emFpM1hsd2s0YUNGZ2xCWEQ3SzRsZ0Noc0xXZjdZQWtKcmRKRGEvCkRhNmpzbFQvQithZlpKbmJZeUJSZkNqeVdkUE5lbXdsU1V2ZS9SZ3Z6eXZsMllmTFlpMWJ3bmw2RXNrTW84aWoKUVdBR0tpd3FkUnNCdGNOSStJY1JHYzFDZWY4ZFFhdGVGWFl6c2VINHdONWpwRWZoYllMOVRCbEJ1QnQwWVpZOQpJTlZxUVRGVEZQVzBJWmZQcGU4ZXhUOFR3TUJlRjBoS05uUk03blZICi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K", | |
| "conditions": [ | |
| { | |
| "lastUpdateTime": "2019-03-22T00:06:36Z", | |
| "message": "This CSR was approved by kubectl certificate approve.", | |
| "reason": "KubectlApprove", | |
| "type": "Approved" | |
| } | |
| ] | |
| } | |
| } | |
| ] | |
| } | |
| to retry, use: --limit @/home/user/git/sourcetopia/openshift-install/openshift/ansible/playbooks/deploy_cluster.retry | |
| PLAY RECAP ********************************************************************************************************************************************************************** | |
| k8s-ctl01.dev.mydomain.com : ok=462 changed=85 unreachable=0 failed=1 | |
| k8s-worker01.dev.mydomain.com : ok=100 changed=11 unreachable=0 failed=0 | |
| k8s-worker02.dev.mydomain.com : ok=98 changed=11 unreachable=0 failed=0 | |
| k8s-worker03.dev.mydomain.com : ok=98 changed=11 unreachable=0 failed=0 | |
| localhost : ok=11 changed=0 unreachable=0 failed=0 | |
| INSTALLER STATUS **************************************************************************************************************************************************************** | |
| Initialization : Complete (0:00:30) | |
| Health Check : Complete (0:02:50) | |
| Node Bootstrap Preparation : Complete (0:02:50) | |
| etcd Install : Complete (0:00:29) | |
| Master Install : Complete (0:03:52) | |
| Master Additional Install : Complete (0:00:29) | |
| Node Join : In Progress (0:03:22) | |
| This phase can be restarted by running: playbooks/openshift-node/join.yml | |
| Failure summary: | |
| 1. Hosts: k8s-ctl01.dev.mydomain.com | |
| Play: Approve any pending CSR requests from inventory nodes | |
| Task: Approve node certificates when bootstrapping | |
| Message: Could not find csr for nodes: k8s-worker03.dev.mydomain.com, k8s-worker02.dev.mydomain.com, k8s-worker01.dev.mydomain.com |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment