Skip to content

Instantly share code, notes, and snippets.

@snoby
Created June 14, 2017 17:56
Show Gist options
  • Select an option

  • Save snoby/6eb80dd1051de007535e93a6c481d826 to your computer and use it in GitHub Desktop.

Select an option

Save snoby/6eb80dd1051de007535e93a6c481d826 to your computer and use it in GitHub Desktop.
logs from journalctl on master
Jun 14 17:54:50 ip-10-230-67-244 systemd[1]: Starting Run docker-healthcheck once...
Jun 14 17:54:50 ip-10-230-67-244 docker-healthcheck[5308]: docker healthy
Jun 14 17:54:50 ip-10-230-67-244 systemd[1]: Started Run docker-healthcheck once.
Jun 14 17:54:50 ip-10-230-67-244 kubelet[2484]: E0614 17:54:50.747874 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:382: Failed to list *v1.Service: Get https://127.0.0.1/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:51 ip-10-230-67-244 kubelet[2484]: E0614 17:54:51.045180 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://127.0.0.1/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersio
Jun 14 17:54:51 ip-10-230-67-244 kubelet[2484]: E0614 17:54:51.644003 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:390: Failed to list *v1.Node: Get https://127.0.0.1/api/v1/nodes?fieldSelector=metadata.name%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersion=0: d
Jun 14 17:54:51 ip-10-230-67-244 kubelet[2484]: E0614 17:54:51.748324 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:382: Failed to list *v1.Service: Get https://127.0.0.1/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:51 ip-10-230-67-244 sshd[5314]: error: Could not load host key: /etc/ssh/ssh_host_ed25519_key
Jun 14 17:54:51 ip-10-230-67-244 sshd[5314]: Did not receive identification string from 10.230.6.230
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: E0614 17:54:52.046239 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://127.0.0.1/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersio
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: E0614 17:54:52.487298 2484 event.go:208] Unable to write event: 'dial tcp 127.0.0.1:443: getsockopt: connection refused' (may retry after sleeping)
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.488887 2484 kubelet_node_status.go:230] Setting node annotation to enable volume controller attach/detach
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.488902 2484 kubelet_node_status.go:272] Adding node label from cloud provider: beta.kubernetes.io/instance-type=t2.small
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.488911 2484 kubelet_node_status.go:283] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/zone=us-east-2b
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.488916 2484 kubelet_node_status.go:287] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/region=us-east-2
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.492457 2484 aws.go:937] Could not determine public DNS from AWS metadata.
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.493856 2484 kubelet_node_status.go:379] Recording NodeHasSufficientDisk event message for node ip-10-230-67-244.us-east-2.compute.internal
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.493873 2484 kubelet_node_status.go:379] Recording NodeHasSufficientMemory event message for node ip-10-230-67-244.us-east-2.compute.internal
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.493880 2484 kubelet_node_status.go:379] Recording NodeHasNoDiskPressure event message for node ip-10-230-67-244.us-east-2.compute.internal
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: E0614 17:54:52.495180 2484 kubelet.go:1535] Failed creating a mirror pod for "kube-apiserver-ip-10-230-67-244.us-east-2.compute.internal_kube-system(90f08fcd18cdd5d149611cc65c5da571)": Post https://127.0.0.1/api/v1/namespaces/kube-system/pods: d
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: E0614 17:54:52.644527 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:390: Failed to list *v1.Node: Get https://127.0.0.1/api/v1/nodes?fieldSelector=metadata.name%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersion=0: d
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: E0614 17:54:52.749238 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:382: Failed to list *v1.Service: Get https://127.0.0.1/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.795596 2484 kuberuntime_manager.go:458] Container {Name:kube-apiserver Image:gcr.io/google_containers/kube-apiserver:v1.6.2 Command:[/bin/sh -c /usr/local/bin/kube-apiserver --address=127.0.0.1 --admission-control=NamespaceLifecyc
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: :true MountPath:/srv/sshproxy SubPath:} {Name:logfile ReadOnly:false MountPath:/var/log/kube-apiserver.log SubPath:}] LivenessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:8080,Host:127.0.0.1,Scheme:HTTP,HTTPHeader
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.795706 2484 kuberuntime_manager.go:742] checking backoff for container "kube-apiserver" in pod "kube-apiserver-ip-10-230-67-244.us-east-2.compute.internal_kube-system(90f08fcd18cdd5d149611cc65c5da571)"
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: I0614 17:54:52.795848 2484 kuberuntime_manager.go:752] Back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ip-10-230-67-244.us-east-2.compute.internal_kube-system(90f08fcd18cdd5d149611cc65c5da571)
Jun 14 17:54:52 ip-10-230-67-244 kubelet[2484]: E0614 17:54:52.795893 2484 pod_workers.go:182] Error syncing pod 90f08fcd18cdd5d149611cc65c5da571 ("kube-apiserver-ip-10-230-67-244.us-east-2.compute.internal_kube-system(90f08fcd18cdd5d149611cc65c5da571)"), skipping: failed to "StartContainer"
Jun 14 17:54:53 ip-10-230-67-244 kubelet[2484]: E0614 17:54:53.047281 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://127.0.0.1/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersio
Jun 14 17:54:53 ip-10-230-67-244 kubelet[2484]: E0614 17:54:53.645556 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:390: Failed to list *v1.Node: Get https://127.0.0.1/api/v1/nodes?fieldSelector=metadata.name%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersion=0: d
Jun 14 17:54:53 ip-10-230-67-244 kubelet[2484]: E0614 17:54:53.750236 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:382: Failed to list *v1.Service: Get https://127.0.0.1/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: W0614 17:54:54.028958 2484 cni.go:157] Unable to update cni config: No networks found in /etc/cni/net.d/
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: E0614 17:54:54.029564 2484 kubelet.go:2067] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: E0614 17:54:54.048260 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://127.0.0.1/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersio
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.197644 2484 kubelet_node_status.go:230] Setting node annotation to enable volume controller attach/detach
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.197673 2484 kubelet_node_status.go:272] Adding node label from cloud provider: beta.kubernetes.io/instance-type=t2.small
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.197680 2484 kubelet_node_status.go:283] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/zone=us-east-2b
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.197685 2484 kubelet_node_status.go:287] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/region=us-east-2
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.201756 2484 aws.go:937] Could not determine public DNS from AWS metadata.
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.203051 2484 kubelet_node_status.go:379] Recording NodeHasSufficientDisk event message for node ip-10-230-67-244.us-east-2.compute.internal
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.203394 2484 kubelet_node_status.go:379] Recording NodeHasSufficientMemory event message for node ip-10-230-67-244.us-east-2.compute.internal
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.203408 2484 kubelet_node_status.go:379] Recording NodeHasNoDiskPressure event message for node ip-10-230-67-244.us-east-2.compute.internal
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: I0614 17:54:54.203422 2484 kubelet_node_status.go:77] Attempting to register node ip-10-230-67-244.us-east-2.compute.internal
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: E0614 17:54:54.204472 2484 kubelet_node_status.go:101] Unable to register node "ip-10-230-67-244.us-east-2.compute.internal" with API server: Post https://127.0.0.1/api/v1/nodes: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: E0614 17:54:54.646044 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:390: Failed to list *v1.Node: Get https://127.0.0.1/api/v1/nodes?fieldSelector=metadata.name%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersion=0: d
Jun 14 17:54:54 ip-10-230-67-244 kubelet[2484]: E0614 17:54:54.750702 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:382: Failed to list *v1.Service: Get https://127.0.0.1/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:55 ip-10-230-67-244 kubelet[2484]: E0614 17:54:55.049858 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://127.0.0.1/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersio
Jun 14 17:54:55 ip-10-230-67-244 kubelet[2484]: E0614 17:54:55.646507 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:390: Failed to list *v1.Node: Get https://127.0.0.1/api/v1/nodes?fieldSelector=metadata.name%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersion=0: d
Jun 14 17:54:55 ip-10-230-67-244 kubelet[2484]: E0614 17:54:55.751160 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:382: Failed to list *v1.Service: Get https://127.0.0.1/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:56 ip-10-230-67-244 kubelet[2484]: E0614 17:54:56.050823 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://127.0.0.1/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersio
Jun 14 17:54:56 ip-10-230-67-244 sudo[5318]: admin : TTY=pts/0 ; PWD=/home/admin ; USER=root ; COMMAND=/bin/journalctl
Jun 14 17:54:56 ip-10-230-67-244 sudo[5318]: pam_unix(sudo:session): session opened for user root by admin(uid=0)
Jun 14 17:54:56 ip-10-230-67-244 kubelet[2484]: E0614 17:54:56.646961 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:390: Failed to list *v1.Node: Get https://127.0.0.1/api/v1/nodes?fieldSelector=metadata.name%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersion=0: d
Jun 14 17:54:56 ip-10-230-67-244 kubelet[2484]: E0614 17:54:56.751652 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:382: Failed to list *v1.Service: Get https://127.0.0.1/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:57 ip-10-230-67-244 kubelet[2484]: E0614 17:54:57.053028 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://127.0.0.1/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersio
Jun 14 17:54:57 ip-10-230-67-244 kubelet[2484]: E0614 17:54:57.647557 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:390: Failed to list *v1.Node: Get https://127.0.0.1/api/v1/nodes?fieldSelector=metadata.name%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersion=0: d
Jun 14 17:54:57 ip-10-230-67-244 kubelet[2484]: E0614 17:54:57.752118 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/kubelet.go:382: Failed to list *v1.Service: Get https://127.0.0.1/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:443: getsockopt: connection refused
Jun 14 17:54:58 ip-10-230-67-244 kubelet[2484]: E0614 17:54:58.054013 2484 reflector.go:190] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://127.0.0.1/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-230-67-244.us-east-2.compute.internal&resourceVersio
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment