rkt-kubelet-cadvisor

rkt-kubelet.log

-- Logs begin at Wed 2015-12-30 18:41:32 UTC, end at Wed 2015-12-30 18:52:00 UTC. --
Dec 30 18:42:24 kubernetes-minion-5gbv.c.coreos-gce-testing.internal systemd[1]: Starting Run Kubelet service...
Dec 30 18:42:24 kubernetes-minion-5gbv.c.coreos-gce-testing.internal configure-kubelet.sh[768]: Configuring hostname
Dec 30 18:42:24 kubernetes-minion-5gbv configure-kubelet.sh[768]: Configuring kubelet
Dec 30 18:42:24 kubernetes-minion-5gbv systemd[1]: Started Run Kubelet service.
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.034396     780 gce.go:201] Using existing Token Source &oauth2.reuseTokenSource{new:google.computeSource{account:""}, mu:sync.Mutex{state:0, sema:0x0}, t:(*oauth2.Token)(nil)}
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.034533     780 server.go:522] Successfully initialized cloud provider: "gce" from the config file: ""
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.034606     780 manager.go:128] cAdvisor running in container: "/system.slice/kubelet.service"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: W1230 18:42:26.034732     780 manager.go:132] Unable to connect to Docker: cannot connect to Docker endpoint
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.035367     780 fs.go:105] Filesystem partitions: map[/dev/sda6:{mountpoint:/usr/share/oem major:8 minor:6 fsType: blockSize:0} /dev/sda9:{mountpoint:/ major:8 minor:9 fsType: blockSize:0} /dev/sda3:{mountpoint:/usr major:8 minor:3 fsType: blockSize:0}]
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.039545     780 manager.go:163] Machine: {NumCores:2 CpuFrequency:2299944 MemoryCapacity:7848468480 MachineID:6c85860c7b7eca2151cbe3bf56ac765a SystemUUID:6C85860C-7B7E-CA21-51CB-E3BF56AC765A BootID:1a9fb9e3-0b25-4814-ab89-a2bde1037071 Filesystems:[{Device:/dev/sda9 Capacity:101552205824} {Device:/dev/sda3 Capacity:1031946240} {Device:/dev/sda6 Capacity:113229824}] DiskMap:map[8:0:{Name:sda Major:8 Minor:0 Size:107374182400 Scheduler:cfq}] NetworkDevices:[{Name:ens4v1 MacAddress:42:01:0a:f0:00:09 Speed:0 Mtu:1460}] Topology:[{Id:0 Memory:7848468480 Cores:[{Id:0 Threads:[0 1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:47185920 Type:Unified Level:3}]}] CloudProvider:GCE InstanceType:n1-standard-2}
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.039838     780 manager.go:169] Version: {KernelVersion:4.1.7-coreos ContainerOsVersion:CoreOS 766.4.0 DockerVersion:Unknown CadvisorVersion: CadvisorRevision:}
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.040413     780 server.go:545] Using root directory: /var/lib/kubelet
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.041743     780 server.go:792] cloud provider determined current node name to be kubernetes-minion-5gbv
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.041906     780 server.go:867] Adding manifest file: /etc/kubernetes/manifests
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.041925     780 file.go:47] Watching path "/etc/kubernetes/manifests"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.041937     780 server.go:877] Watching apiserver
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.052761     780 plugins.go:56] Registering credential provider: google-dockercfg-url
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.053678     780 plugins.go:56] Registering credential provider: google-container-registry
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.053690     780 plugins.go:56] Registering credential provider: .dockercfg
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.054212     780 plugins.go:56] Registering credential provider: google-dockercfg
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: W1230 18:42:26.056563     780 version.go:99] rkt: current binary version "0.14.0" is not recommended (recommended version "0.13.0")
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059411     780 plugins.go:268] Loaded volume plugin "kubernetes.io/aws-ebs"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059454     780 plugins.go:268] Loaded volume plugin "kubernetes.io/empty-dir"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059476     780 plugins.go:268] Loaded volume plugin "kubernetes.io/gce-pd"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059495     780 plugins.go:268] Loaded volume plugin "kubernetes.io/git-repo"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059516     780 plugins.go:268] Loaded volume plugin "kubernetes.io/host-path"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059535     780 plugins.go:268] Loaded volume plugin "kubernetes.io/nfs"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059553     780 plugins.go:268] Loaded volume plugin "kubernetes.io/secret"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059573     780 plugins.go:268] Loaded volume plugin "kubernetes.io/iscsi"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059718     780 plugins.go:268] Loaded volume plugin "kubernetes.io/glusterfs"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059746     780 plugins.go:268] Loaded volume plugin "kubernetes.io/persistent-claim"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059765     780 plugins.go:268] Loaded volume plugin "kubernetes.io/rbd"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059783     780 plugins.go:268] Loaded volume plugin "kubernetes.io/cinder"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059802     780 plugins.go:268] Loaded volume plugin "kubernetes.io/cephfs"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059827     780 plugins.go:268] Loaded volume plugin "kubernetes.io/downward-api"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059846     780 plugins.go:268] Loaded volume plugin "kubernetes.io/fc"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059870     780 plugins.go:268] Loaded volume plugin "kubernetes.io/flocker"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.059918     780 server.go:839] Started kubelet
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: E1230 18:42:26.063740     780 kubelet.go:813] Image garbage collection failed: unable to find data for container /
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.063811     780 server.go:104] Starting to listen on 0.0.0.0:10250
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.064713     780 server.go:121] Starting to listen read-only on 0.0.0.0:10255
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.085111     780 kubelet.go:843] Running in container "/kubelet"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.085481     780 container_manager_linux.go:171] Updating kernel flag: vm/overcommit_memory, expected value: 1, actual value: 0
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.085530     780 container_manager_linux.go:171] Updating kernel flag: kernel/panic, expected value: 10, actual value: 60
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.085565     780 manager.go:126] Starting to sync pod status with apiserver
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.085581     780 kubelet.go:2253] Starting kubelet main sync loop.
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.085599     780 kubelet.go:2285] SyncLoop (ADD, "file"): "fluentd-cloud-logging-kubernetes-minion-5gbv_kube-system(ff493c6d8ed6feddeefe29184bcb9167)"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: E1230 18:42:26.085638     780 kubelet.go:2208] error getting node: node 'kubernetes-minion-5gbv' is not in cache
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.086084     780 factory.go:203] System is using systemd
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: E1230 18:42:26.086540     780 manager.go:205] Docker container factory registration failed: unable to communicate with docker daemon: cannot connect to Docker endpoint.
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.087194     780 factory.go:94] Registering Raw factory
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.136502     780 manager.go:1005] Started watching for new ooms in manager
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.137249     780 oomparser.go:182] oomparser using systemd
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.138135     780 manager.go:249] Starting recovery of all containers
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.199567     780 kubelet.go:2285] SyncLoop (ADD, "api"): ""
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.233073     780 manager.go:254] Recovery completed
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: E1230 18:42:26.308844     780 kubelet.go:1625] Mirror pod not available
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.323565     780 kubelet.go:957] Adding node label from cloud provider: failure-domain.alpha.kubernetes.io/zone=us-east1-b
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.323582     780 kubelet.go:961] Adding node label from cloud provider: failure-domain.alpha.kubernetes.io/region=us-east1
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.330708     780 provider.go:91] Refreshing cache for provider: *gcp_credentials.dockerConfigUrlKeyProvider
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.333882     780 config.go:155] body of failing http response: &{0xc2088c9000 {0 0} false <nil> 0x5d42a0 0x5d4230}
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: E1230 18:42:26.333949     780 metadata.go:124] while reading 'google-dockercfg-url' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg-url
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.334023     780 kubelet.go:2614] Recording NodeHasSufficientDisk event message for node kubernetes-minion-5gbv
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.334045     780 kubelet.go:1068] Attempting to register node kubernetes-minion-5gbv
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.347123     780 kubelet.go:2285] SyncLoop (ADD, "api"): "fluentd-cloud-logging-kubernetes-minion-5gbv_kube-system(12639af1-af25-11e5-9118-42010af00006)"
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.362026     780 provider.go:91] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.362117     780 provider.go:91] Refreshing cache for provider: *gcp_credentials.dockerConfigKeyProvider
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.362683     780 config.go:155] body of failing http response: &{0xc2084a2f00 {0 0} false <nil> 0x5d42a0 0x5d4230}
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: E1230 18:42:26.362707     780 metadata.go:112] while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
Dec 30 18:42:26 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:26.546533     780 kubelet.go:1099] Successfully registered node kubernetes-minion-5gbv
Dec 30 18:42:39 kubernetes-minion-5gbv kubelet[780]: I1230 18:42:39.741703     780 kubelet.go:2285] SyncLoop (ADD, "api"): "kube-dns-v8-tcn2b_kube-system(1a6332a3-af25-11e5-9118-42010af00006)"
Dec 30 18:43:43 kubernetes-minion-5gbv kubelet[780]: I1230 18:43:43.679958     780 provider.go:91] Refreshing cache for provider: *gcp_credentials.dockerConfigUrlKeyProvider
Dec 30 18:43:43 kubernetes-minion-5gbv kubelet[780]: I1230 18:43:43.682480     780 config.go:155] body of failing http response: &{0xc208c69240 {0 0} false <nil> 0x5d42a0 0x5d4230}
Dec 30 18:43:43 kubernetes-minion-5gbv kubelet[780]: E1230 18:43:43.683022     780 metadata.go:124] while reading 'google-dockercfg-url' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg-url
Dec 30 18:43:43 kubernetes-minion-5gbv kubelet[780]: I1230 18:43:43.687399     780 provider.go:91] Refreshing cache for provider: *gcp_credentials.dockerConfigKeyProvider
Dec 30 18:43:43 kubernetes-minion-5gbv kubelet[780]: I1230 18:43:43.690419     780 config.go:155] body of failing http response: &{0xc208c696c0 {0 0} false <nil> 0x5d42a0 0x5d4230}
Dec 30 18:43:43 kubernetes-minion-5gbv kubelet[780]: E1230 18:43:43.690478     780 metadata.go:112] while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
Dec 30 18:46:23 kubernetes-minion-5gbv kubelet[780]: I1230 18:46:23.540318     780 container.go:435] Failed to update stats for container "/system.slice/var-lib-rkt-pods-run-af378ef3\x2d877c\x2d466a\x2dba87\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-kube2sky-rootfs.mount": open /sys/fs/cgroup/cpu,cpuacct/system.slice/var-lib-rkt-pods-run-af378ef3\x2d877c\x2d466a\x2dba87\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-kube2sky-rootfs.mount/cpuacct.stat: no such file or directory, continuing to push stats
Dec 30 18:46:23 kubernetes-minion-5gbv kubelet[780]: W1230 18:46:23.545863     780 handler.go:542] Error while processing event ("/sys/fs/cgroup/memory/system.slice/var-lib-rkt-pods-run-af378ef3\\x2d877c\\x2d466a\\x2dba87\\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-kube2sky-rootfs.mount": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/system.slice/var-lib-rkt-pods-run-af378ef3\x2d877c\x2d466a\x2dba87\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-kube2sky-rootfs.mount: no such file or directory
Dec 30 18:46:23 kubernetes-minion-5gbv kubelet[780]: W1230 18:46:23.545956     780 handler.go:542] Error while processing event ("/sys/fs/cgroup/cpu,cpuacct/system.slice/var-lib-rkt-pods-run-af378ef3\\x2d877c\\x2d466a\\x2dba87\\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-healthz-rootfs.mount": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpu,cpuacct/system.slice/var-lib-rkt-pods-run-af378ef3\x2d877c\x2d466a\x2dba87\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-healthz-rootfs.mount: no such file or directory
Dec 30 18:46:23 kubernetes-minion-5gbv kubelet[780]: W1230 18:46:23.546067     780 handler.go:542] Error while processing event ("/sys/fs/cgroup/blkio/system.slice/var-lib-rkt-pods-run-af378ef3\\x2d877c\\x2d466a\\x2dba87\\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-healthz-rootfs.mount": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/blkio/system.slice/var-lib-rkt-pods-run-af378ef3\x2d877c\x2d466a\x2dba87\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-healthz-rootfs.mount: no such file or directory
Dec 30 18:46:23 kubernetes-minion-5gbv kubelet[780]: W1230 18:46:23.546101     780 handler.go:542] Error while processing event ("/sys/fs/cgroup/memory/system.slice/var-lib-rkt-pods-run-af378ef3\\x2d877c\\x2d466a\\x2dba87\\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-healthz-rootfs.mount": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/system.slice/var-lib-rkt-pods-run-af378ef3\x2d877c\x2d466a\x2dba87\x2d13a3b81c1fdf-stage1-rootfs-opt-stage2-healthz-rootfs.mount: no such file or directory