Last active
October 29, 2021 03:07
-
-
Save FilBot3/e8709ef7809527075bb5dc1df7288782 to your computer and use it in GitHub Desktop.
k3s-ansible failing to connect workers and master failing to start on Ubuntu Server 21.10 arm64.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Oct 29 03:01:20 ubuntu-k3s-master systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE | |
| Oct 29 03:01:20 ubuntu-k3s-master systemd[1]: k3s.service: Failed with result 'exit-code'. | |
| Oct 29 03:01:20 ubuntu-k3s-master systemd[1]: k3s.service: Consumed 32.463s CPU time. | |
| Oct 29 03:01:26 ubuntu-k3s-master systemd[1]: k3s.service: Scheduled restart job, restart counter is at 43. | |
| Oct 29 03:01:26 ubuntu-k3s-master systemd[1]: Stopped Lightweight Kubernetes. | |
| Oct 29 03:01:26 ubuntu-k3s-master systemd[1]: k3s.service: Consumed 32.463s CPU time. | |
| Oct 29 03:01:26 ubuntu-k3s-master systemd[1]: Starting Lightweight Kubernetes... | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:26.844427808Z" level=info msg="Starting k3s v1.21.5+k3s2 (724ef700)" | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:26.845155773Z" level=info msg="Cluster bootstrap already complete" | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:26.910591057Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s" | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:26.910806645Z" level=info msg="Configuring database table schema and indexes, this may take a moment..." | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:26.911343818Z" level=info msg="Database tables and indexes are up to date" | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:26.929490456Z" level=info msg="Kine listening on unix://kine.sock" | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:26.930228940Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --enable-admission-plugins=NodeRestriction --etcd-servers=unix://kine.sock --insecure-port=0 --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: Flag --insecure-port has been deprecated, This flag has no effect now and will be removed in v1.24. | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: I1029 03:01:26.934107 20645 server.go:656] external host was not specified, using 192.168.86.90 | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: I1029 03:01:26.935106 20645 server.go:195] Version: v1.21.5+k3s2 | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: I1029 03:01:26.947081 20645 shared_informer.go:240] Waiting for caches to sync for node_authorizer | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: I1029 03:01:26.951810 20645 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: I1029 03:01:26.951909 20645 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota. | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: I1029 03:01:26.957699 20645 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. | |
| Oct 29 03:01:26 ubuntu-k3s-master k3s[20645]: I1029 03:01:26.957818 20645 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota. | |
| Oct 29 03:01:27 ubuntu-k3s-master k3s[20645]: I1029 03:01:27.047492 20645 instance.go:283] Using reconciler: lease | |
| Oct 29 03:01:27 ubuntu-k3s-master k3s[20645]: I1029 03:01:27.210896 20645 rest.go:130] the default service ipfamily for this cluster is: IPv4 | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: W1029 03:01:28.306166 20645 genericapiserver.go:425] Skipping API node.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: W1029 03:01:28.353592 20645 genericapiserver.go:425] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: W1029 03:01:28.370654 20645 genericapiserver.go:425] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: W1029 03:01:28.399720 20645 genericapiserver.go:425] Skipping API storage.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: W1029 03:01:28.411878 20645 genericapiserver.go:425] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: W1029 03:01:28.439420 20645 genericapiserver.go:425] Skipping API apps/v1beta2 because it has no resources. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: W1029 03:01:28.439518 20645 genericapiserver.go:425] Skipping API apps/v1beta1 because it has no resources. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: I1029 03:01:28.479852 20645 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: I1029 03:01:28.479949 20645 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota. | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.525238764Z" level=info msg="Waiting for API server to become available" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.525265764Z" level=info msg="Running kube-scheduler --address=127.0.0.1 --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --leader-elect=false --port=10251 --profiling=false --secure-port=0" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.527631638Z" level=info msg="Running kube-controller-manager --address=127.0.0.1 --allocate-node-cidrs=true --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --configure-cloud-routes=false --controllers=*,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --leader-elect=false --port=10252 --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=0 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.key --use-service-account-credentials=true" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.529529133Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --bind-address=127.0.0.1 --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --leader-elect=false --node-status-update-frequency=1m0s --port=0 --profiling=false" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.533198405Z" level=info msg="Node token is available at /var/lib/rancher/k3s/server/token" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.533333957Z" level=info msg="To join node to cluster: k3s agent -s https://192.168.86.90:6443 -t ${NODE_TOKEN}" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.537082746Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.537215817Z" level=info msg="Run: k3s kubectl" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.617610131Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:28 http: TLS handshake error from 192.168.86.93:50518: remote error: tls: bad certificate" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.641702844Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:28 http: TLS handshake error from 127.0.0.1:46448: remote error: tls: bad certificate" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.661967215Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:28 http: TLS handshake error from 127.0.0.1:46454: remote error: tls: bad certificate" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.719043521Z" level=error msg="runtime core not ready" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.724498402Z" level=info msg="certificate CN=ubuntu-k3s-master signed by CN=k3s-server-ca@1635475569: notBefore=2021-10-29 02:46:09 +0000 UTC notAfter=2022-10-29 03:01:28 +0000 UTC" | |
| Oct 29 03:01:28 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:28.744091047Z" level=info msg="certificate CN=system:node:ubuntu-k3s-master,O=system:nodes signed by CN=k3s-client-ca@1635475569: notBefore=2021-10-29 02:46:09 +0000 UTC notAfter=2022-10-29 03:01:28 +0000 UTC" | |
| Oct 29 03:01:29 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:29.065249778Z" level=info msg="Module overlay was already loaded" | |
| Oct 29 03:01:29 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:29.065383164Z" level=info msg="Module nf_conntrack was already loaded" | |
| Oct 29 03:01:29 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:29.065435811Z" level=info msg="Module br_netfilter was already loaded" | |
| Oct 29 03:01:29 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:29.065485958Z" level=info msg="Module iptable_nat was already loaded" | |
| Oct 29 03:01:29 ubuntu-k3s-master k3s[20645]: W1029 03:01:29.065602 20645 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 03:01:29 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:29.072873019Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log" | |
| Oct 29 03:01:29 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:29.073516283Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd" | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:30.077459705Z" level=info msg="Containerd is now running" | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:30.108517454Z" level=info msg="Connecting to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect" | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:30.118504625Z" level=info msg="Handling backend connection request [ubuntu-k3s-master]" | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:30.120927609Z" level=info msg="Running kubelet --address=0.0.0.0 --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --cni-bin-dir=/var/lib/rancher/k3s/data/5b1d238ed00ea9bbae6aa96d867c1264a96ad48b42e55d56570d54d05c8d8a7a/bin --cni-conf-dir=/var/lib/rancher/k3s/agent/etc/cni/net.d --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --container-runtime=remote --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=ubuntu-k3s-master --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-labels= --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key" | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:30.122806346Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=ubuntu-k3s-master --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables" | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: Flag --cloud-provider has been deprecated, will be removed in 1.23, in favor of removing cloud provider code from Kubelet. | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: Flag --cni-bin-dir has been deprecated, will be removed along with dockershim. | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: Flag --cni-conf-dir has been deprecated, will be removed along with dockershim. | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed. | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: W1029 03:01:30.123682 20645 server.go:220] WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP. | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: I1029 03:01:30.190560 20645 server.go:436] "Kubelet version" kubeletVersion="v1.21.5+k3s2" | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: E1029 03:01:30.207504 20645 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-master" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: I1029 03:01:30.287887 20645 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: W1029 03:01:30.288013 20645 manager.go:159] Cannot detect current cgroup on cgroup v2 | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:30.801576492Z" level=info msg="Handling backend connection request [ubuntu-k3s-wkr2]" | |
| Oct 29 03:01:30 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:30.802406789Z" level=info msg="Handling backend connection request [ubuntu-k3s-wkr1]" | |
| Oct 29 03:01:31 ubuntu-k3s-master k3s[20645]: E1029 03:01:31.401210 20645 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-master" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 03:01:33 ubuntu-k3s-master k3s[20645]: E1029 03:01:33.514817 20645 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-master" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 03:01:33 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:33.735463394Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:33 http: TLS handshake error from 192.168.86.93:50538: remote error: tls: bad certificate" | |
| Oct 29 03:01:33 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:33.820412072Z" level=error msg="runtime core not ready" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:35.112561057Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: W1029 03:01:35.329555 20645 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.349572 20645 server.go:660] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.351028 20645 container_manager_linux.go:291] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.351662 20645 container_manager_linux.go:296] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none Rootless:false} | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.351864 20645 topology_manager.go:120] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.352189 20645 container_manager_linux.go:327] "Initializing Topology Manager" policy="none" scope="container" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.352279 20645 container_manager_linux.go:332] "Creating device plugin manager" devicePluginEnabled=true | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.354402 20645 kubelet.go:404] "Attempting to sync node with API server" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.354616 20645 kubelet.go:272] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.354729 20645 kubelet.go:283] "Adding apiserver pod source" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.354800 20645 apiserver.go:42] "Waiting for node sync before watching apiserver pods" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.363116 20645 kuberuntime_manager.go:222] "Container runtime initialized" containerRuntime="containerd" version="v1.4.11-k3s1" apiVersion="v1alpha2" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.365317 20645 server.go:1191] "Started kubelet" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.378464 20645 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.378536 20645 server.go:149] "Starting to listen" address="0.0.0.0" port=10250 | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.381331 20645 server.go:409] "Adding debug handlers to kubelet server" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.390235 20645 volume_manager.go:271] "Starting Kubelet Volume Manager" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.391563 20645 desired_state_of_world_populator.go:141] "Desired state populator starts to run" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: E1029 03:01:35.413832 20645 cri_stats_provider.go:369] "Failed to get the info of the filesystem with mountpoint" err="unable to find data in memory cache" mountpoint="/var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: E1029 03:01:35.413984 20645 kubelet.go:1306] "Image garbage collection failed once. Stats initialization may not have completed yet" err="invalid capacity 0 on image filesystem" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.493875 20645 kuberuntime_manager.go:1044] "Updating runtime config through cri with podcidr" CIDR="10.42.0.0/24" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.495656 20645 kubelet_network.go:76] "Updating Pod CIDR" originalPodCIDR="" newPodCIDR="10.42.0.0/24" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.500792 20645 kubelet_node_status.go:71] "Attempting to register node" node="ubuntu-k3s-master" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.563958 20645 kubelet_network_linux.go:56] "Initialized protocol iptables rules." protocol=IPv4 | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.675192 20645 kubelet_network_linux.go:56] "Initialized protocol iptables rules." protocol=IPv6 | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.675309 20645 status_manager.go:157] "Starting to sync pod status with apiserver" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: I1029 03:01:35.676025 20645 kubelet.go:1846] "Starting kubelet main sync loop" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: E1029 03:01:35.676239 20645 kubelet.go:1870] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: E1029 03:01:35.777315 20645 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 03:01:35 ubuntu-k3s-master k3s[20645]: E1029 03:01:35.977605 20645 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: E1029 03:01:36.378718 20645 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.385007 20645 apiserver.go:52] "Watching apiserver" | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.647169 20645 dynamic_cafile_content.go:167] Starting request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.647176 20645 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.648230 20645 dynamic_serving_content.go:130] Starting serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.649185 20645 secure_serving.go:202] Serving securely on 127.0.0.1:6444 | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.649935 20645 tlsconfig.go:240] Starting DynamicServingCertificateController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.650762 20645 dynamic_serving_content.go:130] Starting aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.651610 20645 apiservice_controller.go:97] Starting APIServiceRegistrationController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.652260 20645 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.652806 20645 available_controller.go:475] Starting AvailableConditionController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.653363 20645 cache.go:32] Waiting for caches to sync for AvailableConditionController controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.654100 20645 autoregister_controller.go:141] Starting autoregister controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.654653 20645 cache.go:32] Waiting for caches to sync for autoregister controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.656068 20645 controller.go:83] Starting OpenAPI AggregationController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.650774 20645 apf_controller.go:294] Starting API Priority and Fairness config controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.660232 20645 crdregistration_controller.go:111] Starting crd-autoregister controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.660987 20645 shared_informer.go:240] Waiting for caches to sync for crd-autoregister | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.675533 20645 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.675667 20645 dynamic_cafile_content.go:167] Starting request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.675745 20645 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.675774 20645 shared_informer.go:240] Waiting for caches to sync for cluster_authentication_trust_controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.650867 20645 customresource_discovery_controller.go:209] Starting DiscoveryController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.687643 20645 controller.go:86] Starting OpenAPI controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.687748 20645 naming_controller.go:291] Starting NamingConditionController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.687805 20645 establishing_controller.go:76] Starting EstablishingController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.687855 20645 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.687902 20645 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.687996 20645 crd_finalizer.go:266] Starting CRDFinalizer | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.848229 20645 shared_informer.go:247] Caches are synced for node_authorizer | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.863554 20645 controller.go:611] quota admission added evaluator for: leases.coordination.k8s.io | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.875711 20645 trace.go:205] Trace[1833545207]: "GuaranteedUpdate etcd3" type:*coordination.Lease (29-Oct-2021 03:01:29.169) (total time: 7706ms): | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[1833545207]: ---"Transaction prepared" 7696ms (03:01:00.865) | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[1833545207]: [7.706377938s] [7.706377938s] END | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.878456 20645 trace.go:205] Trace[1686096278]: "Update" url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1,user-agent:k3s/v1.21.5+k3s2 (linux/arm64) kubernetes/724ef70,client:192.168.86.91,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/1.1 (29-Oct-2021 03:01:29.168) (total time: 7709ms): | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[1686096278]: ---"Object stored in database" 7708ms (03:01:00.878) | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[1686096278]: [7.709408595s] [7.709408595s] END | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.877879 20645 shared_informer.go:247] Caches are synced for cluster_authentication_trust_controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.885163 20645 apf_controller.go:299] Running API Priority and Fairness config worker | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.891497 20645 trace.go:205] Trace[113903439]: "GuaranteedUpdate etcd3" type:*coordination.Lease (29-Oct-2021 03:01:28.981) (total time: 7910ms): | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[113903439]: ---"Transaction prepared" 7884ms (03:01:00.865) | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[113903439]: [7.910319802s] [7.910319802s] END | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.891954 20645 trace.go:205] Trace[276779008]: "Update" url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr2,user-agent:k3s/v1.21.5+k3s2 (linux/arm64) kubernetes/724ef70,client:192.168.86.92,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/1.1 (29-Oct-2021 03:01:28.980) (total time: 7911ms): | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[276779008]: ---"Object stored in database" 7910ms (03:01:00.891) | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[276779008]: [7.911049268s] [7.911049268s] END | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.913031 20645 trace.go:205] Trace[338193932]: "GuaranteedUpdate etcd3" type:*coordination.Lease (29-Oct-2021 03:01:35.472) (total time: 1440ms): | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[338193932]: ---"Transaction prepared" 1412ms (03:01:00.884) | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[338193932]: [1.440830952s] [1.440830952s] END | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.913496 20645 trace.go:205] Trace[769123758]: "Update" url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-master,user-agent:k3s/v1.21.5+k3s2 (linux/arm64) kubernetes/724ef70,client:127.0.0.1,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/1.1 (29-Oct-2021 03:01:35.471) (total time: 1441ms): | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[769123758]: ---"Object stored in database" 1441ms (03:01:00.913) | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[769123758]: [1.441577677s] [1.441577677s] END | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.953140 20645 cache.go:39] Caches are synced for APIServiceRegistrationController controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.961081 20645 cache.go:39] Caches are synced for AvailableConditionController controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.963004 20645 shared_informer.go:247] Caches are synced for crd-autoregister | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.963234 20645 cache.go:39] Caches are synced for autoregister controller | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: I1029 03:01:36.980683 20645 trace.go:205] Trace[358137343]: "Create" url:/api/v1/namespaces/default/events,user-agent:k3s/v1.21.5+k3s2 (linux/arm64) kubernetes/724ef70,client:127.0.0.1,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/1.1 (29-Oct-2021 03:01:35.441) (total time: 1538ms): | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[358137343]: ---"Object stored in database" 1538ms (03:01:00.980) | |
| Oct 29 03:01:36 ubuntu-k3s-master k3s[20645]: Trace[358137343]: [1.538597031s] [1.538597031s] END | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.010768 20645 trace.go:205] Trace[1292014459]: "Create" url:/api/v1/nodes,user-agent:k3s/v1.21.5+k3s2 (linux/arm64) kubernetes/724ef70,client:127.0.0.1,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/1.1 (29-Oct-2021 03:01:35.523) (total time: 1486ms): | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: Trace[1292014459]: [1.486963759s] [1.486963759s] END | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: E1029 03:01:37.021562 20645 controller.go:156] Unable to remove old endpoints from kubernetes service: no master IPs were listed in storage, refusing to erase all endpoints for the kubernetes service | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.034102 20645 kubelet_node_status.go:109] "Node was previously registered" node="ubuntu-k3s-master" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.035308 20645 kubelet_node_status.go:74] "Successfully registered node" node="ubuntu-k3s-master" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.066653 20645 setters.go:577] "Node became not ready" node="ubuntu-k3s-master" condition={Type:Ready Status:False LastHeartbeatTime:2021-10-29 03:01:37.066370636 +0000 UTC m=+10.946961409 LastTransitionTime:2021-10-29 03:01:37.066370636 +0000 UTC m=+10.946961409 Reason:KubeletNotReady Message:container runtime status check may not have completed yet} | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: E1029 03:01:37.181482 20645 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.696253 20645 node.go:172] Successfully retrieved node IP: 192.168.86.90 | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.696360 20645 server_others.go:141] Detected node IP 192.168.86.90 | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.767439 20645 server_others.go:207] kube-proxy running in dual-stack mode, IPv4-primary | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.768070 20645 server_others.go:213] Using iptables Proxier. | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.768677 20645 server_others.go:220] creating dualStackProxier for iptables. | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: W1029 03:01:37.769280 20645 server_others.go:519] detect-local-mode set to ClusterCIDR, but no IPv6 cluster CIDR defined, , defaulting to no-op detect-local for IPv6 | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.821482 20645 server.go:643] Version: v1.21.5+k3s2 | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.823530 20645 storage_scheduling.go:148] all system priority classes are created successfully or already exist. | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.826146 20645 config.go:315] Starting service config controller | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.826228 20645 shared_informer.go:240] Waiting for caches to sync for service config | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.826304 20645 config.go:224] Starting endpoint slice config controller | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.826331 20645 shared_informer.go:240] Waiting for caches to sync for endpoint slice config | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.864353 20645 cpu_manager.go:199] "Starting CPU manager" policy="none" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.864434 20645 cpu_manager.go:200] "Reconciling" reconcilePeriod="10s" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.864506 20645 state_mem.go:36] "Initialized new in-memory state store" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.865153 20645 state_mem.go:88] "Updated default CPUSet" cpuSet="" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.865218 20645 state_mem.go:96] "Updated CPUSet assignments" assignments=map[] | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.865251 20645 policy_none.go:44] "None policy: Start" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.869314 20645 manager.go:600] "Failed to retrieve checkpoint" checkpoint="kubelet_internal_checkpoint" err="checkpoint is not found" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.875339 20645 plugin_manager.go:114] "Starting Kubelet Plugin Manager" | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.927037 20645 shared_informer.go:247] Caches are synced for endpoint slice config | |
| Oct 29 03:01:37 ubuntu-k3s-master k3s[20645]: I1029 03:01:37.927044 20645 shared_informer.go:247] Caches are synced for service config | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.783494 20645 topology_manager.go:187] "Topology Admit Handler" | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.786044 20645 topology_manager.go:187] "Topology Admit Handler" | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.787025 20645 topology_manager.go:187] "Topology Admit Handler" | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.787993 20645 topology_manager.go:187] "Topology Admit Handler" | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.788887 20645 topology_manager.go:187] "Topology Admit Handler" | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:38.840481311Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:38 http: TLS handshake error from 192.168.86.93:50552: remote error: tls: bad certificate" | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.849353 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6tft\" (UniqueName: \"kubernetes.io/projected/9e6fcb26-4756-4ba1-86d3-3d0c01c354d7-kube-api-access-s6tft\") pod \"local-path-provisioner-5ff76fc89d-4s8wv\" (UID: \"9e6fcb26-4756-4ba1-86d3-3d0c01c354d7\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.849510 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"values\" (UniqueName: \"kubernetes.io/configmap/3ff1ffb5-9742-4f2c-b44f-3076ace11ce6-values\") pod \"helm-install-traefik-vb664\" (UID: \"3ff1ffb5-9742-4f2c-b44f-3076ace11ce6\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.849636 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"content\" (UniqueName: \"kubernetes.io/configmap/3ff1ffb5-9742-4f2c-b44f-3076ace11ce6-content\") pod \"helm-install-traefik-vb664\" (UID: \"3ff1ffb5-9742-4f2c-b44f-3076ace11ce6\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.849775 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/127a7a55-1134-44da-8a24-9fbe9b076616-config-volume\") pod \"coredns-7448499f4d-4gc4z\" (UID: \"127a7a55-1134-44da-8a24-9fbe9b076616\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.849869 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k2zt\" (UniqueName: \"kubernetes.io/projected/127a7a55-1134-44da-8a24-9fbe9b076616-kube-api-access-9k2zt\") pod \"coredns-7448499f4d-4gc4z\" (UID: \"127a7a55-1134-44da-8a24-9fbe9b076616\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.849955 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae4b2f7c-6868-4c65-8dfc-8e9b29b57696-tmp-dir\") pod \"metrics-server-86cbb8457f-dt8lv\" (UID: \"ae4b2f7c-6868-4c65-8dfc-8e9b29b57696\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.850032 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6g5v\" (UniqueName: \"kubernetes.io/projected/ae4b2f7c-6868-4c65-8dfc-8e9b29b57696-kube-api-access-g6g5v\") pod \"metrics-server-86cbb8457f-dt8lv\" (UID: \"ae4b2f7c-6868-4c65-8dfc-8e9b29b57696\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.850115 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e6fcb26-4756-4ba1-86d3-3d0c01c354d7-config-volume\") pod \"local-path-provisioner-5ff76fc89d-4s8wv\" (UID: \"9e6fcb26-4756-4ba1-86d3-3d0c01c354d7\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.850193 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"values\" (UniqueName: \"kubernetes.io/configmap/1fc8d62b-7abd-4139-8971-5c3a78ccd908-values\") pod \"helm-install-traefik-crd-brjrl\" (UID: \"1fc8d62b-7abd-4139-8971-5c3a78ccd908\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.850298 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdcvj\" (UniqueName: \"kubernetes.io/projected/1fc8d62b-7abd-4139-8971-5c3a78ccd908-kube-api-access-jdcvj\") pod \"helm-install-traefik-crd-brjrl\" (UID: \"1fc8d62b-7abd-4139-8971-5c3a78ccd908\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.850389 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxspx\" (UniqueName: \"kubernetes.io/projected/3ff1ffb5-9742-4f2c-b44f-3076ace11ce6-kube-api-access-wxspx\") pod \"helm-install-traefik-vb664\" (UID: \"3ff1ffb5-9742-4f2c-b44f-3076ace11ce6\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.850473 20645 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"content\" (UniqueName: \"kubernetes.io/configmap/1fc8d62b-7abd-4139-8971-5c3a78ccd908-content\") pod \"helm-install-traefik-crd-brjrl\" (UID: \"1fc8d62b-7abd-4139-8971-5c3a78ccd908\") " | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: I1029 03:01:38.850513 20645 reconciler.go:157] "Reconciler: start to sync state" | |
| Oct 29 03:01:38 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:38.928134154Z" level=error msg="runtime core not ready" | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: E1029 03:01:39.558557 20645 remote_runtime.go:116] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9ae7b4e0b5685d2621a392cfb1d9ecd995df7dbd04abe321626df8db0eab3670\": open /run/flannel/subnet.env: no such file or directory" | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: E1029 03:01:39.558766 20645 kuberuntime_sandbox.go:68] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9ae7b4e0b5685d2621a392cfb1d9ecd995df7dbd04abe321626df8db0eab3670\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/helm-install-traefik-vb664" | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: E1029 03:01:39.558856 20645 kuberuntime_manager.go:790] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9ae7b4e0b5685d2621a392cfb1d9ecd995df7dbd04abe321626df8db0eab3670\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/helm-install-traefik-vb664" | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: E1029 03:01:39.559086 20645 pod_workers.go:190] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"helm-install-traefik-vb664_kube-system(3ff1ffb5-9742-4f2c-b44f-3076ace11ce6)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"helm-install-traefik-vb664_kube-system(3ff1ffb5-9742-4f2c-b44f-3076ace11ce6)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"9ae7b4e0b5685d2621a392cfb1d9ecd995df7dbd04abe321626df8db0eab3670\\\": open /run/flannel/subnet.env: no such file or directory\"" pod="kube-system/helm-install-traefik-vb664" podUID=3ff1ffb5-9742-4f2c-b44f-3076ace11ce6 | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: I1029 03:01:39.687129 20645 controller.go:132] OpenAPI AggregationController: action for item : Nothing (removed from the queue). | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: I1029 03:01:39.687240 20645 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue). | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:39.773444395Z" level=info msg="Waiting for cloud-controller-manager privileges to become available" | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:39.775932434Z" level=info msg="Kube API server is now running" | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:39.776602624Z" level=info msg="k3s is up and running" | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: Flag --address has been deprecated, see --bind-address instead. | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: I1029 03:01:39.816277 20645 controllermanager.go:175] Version: v1.21.5+k3s2 | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: I1029 03:01:39.819093 20645 deprecated_insecure_serving.go:56] Serving insecurely on 127.0.0.1:10252 | |
| Oct 29 03:01:39 ubuntu-k3s-master k3s[20645]: I1029 03:01:39.987816 20645 request.go:668] Waited for 1.030807477s due to client-side throttling, not priority and fairness, request: POST:https://127.0.0.1:6443/api/v1/namespaces/kube-system/serviceaccounts/metrics-server/token | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.033418 20645 remote_runtime.go:116] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5d9595d890b46ac5a0152423491dc54ca46bb0c1f88ae9eea73222124c87ec44\": open /run/flannel/subnet.env: no such file or directory" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.033624 20645 kuberuntime_sandbox.go:68] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5d9595d890b46ac5a0152423491dc54ca46bb0c1f88ae9eea73222124c87ec44\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/helm-install-traefik-crd-brjrl" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.033743 20645 kuberuntime_manager.go:790] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5d9595d890b46ac5a0152423491dc54ca46bb0c1f88ae9eea73222124c87ec44\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/helm-install-traefik-crd-brjrl" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.033960 20645 pod_workers.go:190] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"helm-install-traefik-crd-brjrl_kube-system(1fc8d62b-7abd-4139-8971-5c3a78ccd908)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"helm-install-traefik-crd-brjrl_kube-system(1fc8d62b-7abd-4139-8971-5c3a78ccd908)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"5d9595d890b46ac5a0152423491dc54ca46bb0c1f88ae9eea73222124c87ec44\\\": open /run/flannel/subnet.env: no such file or directory\"" pod="kube-system/helm-install-traefik-crd-brjrl" podUID=1fc8d62b-7abd-4139-8971-5c3a78ccd908 | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: W1029 03:01:40.097846 20645 authorization.go:47] Authorization is disabled | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: W1029 03:01:40.097925 20645 authentication.go:47] Authentication is disabled | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: I1029 03:01:40.097989 20645 deprecated_insecure_serving.go:54] Serving healthz insecurely on 127.0.0.1:10251 | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.182371 20645 remote_runtime.go:116] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7c838226c9cf05ed5202bf2e4b4f111dbd903bf86864e45824d1ba390efc7640\": open /run/flannel/subnet.env: no such file or directory" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.189387 20645 kuberuntime_sandbox.go:68] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7c838226c9cf05ed5202bf2e4b4f111dbd903bf86864e45824d1ba390efc7640\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/local-path-provisioner-5ff76fc89d-4s8wv" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.189520 20645 kuberuntime_manager.go:790] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7c838226c9cf05ed5202bf2e4b4f111dbd903bf86864e45824d1ba390efc7640\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/local-path-provisioner-5ff76fc89d-4s8wv" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.189727 20645 pod_workers.go:190] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"local-path-provisioner-5ff76fc89d-4s8wv_kube-system(9e6fcb26-4756-4ba1-86d3-3d0c01c354d7)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"local-path-provisioner-5ff76fc89d-4s8wv_kube-system(9e6fcb26-4756-4ba1-86d3-3d0c01c354d7)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"7c838226c9cf05ed5202bf2e4b4f111dbd903bf86864e45824d1ba390efc7640\\\": open /run/flannel/subnet.env: no such file or directory\"" pod="kube-system/local-path-provisioner-5ff76fc89d-4s8wv" podUID=9e6fcb26-4756-4ba1-86d3-3d0c01c354d7 | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.230508476Z" level=info msg="Stopped tunnel to 127.0.0.1:6443" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.231599379Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.232243365Z" level=info msg="Proxy done" err="context canceled" url="wss://127.0.0.1:6443/v1-k3s/connect" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.236556143Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-9.18.2.tgz" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.237501123Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-crd-9.18.2.tgz" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.239110218Z" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.271367307Z" level=info msg="Handling backend connection request [ubuntu-k3s-master]" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.277398086Z" level=info msg="Node CIDR assigned for: ubuntu-k3s-master" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: I1029 03:01:40.278630 20645 flannel.go:92] Determining IP address of default interface | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: I1029 03:01:40.279669 20645 flannel.go:105] Using interface with name eth0 and address 192.168.86.90 | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: I1029 03:01:40.296542 20645 kube.go:116] Waiting 10m0s for node controller to sync | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: I1029 03:01:40.297445 20645 kube.go:299] Starting kube subnet manager | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.342061575Z" level=info msg="labels have already set on node: ubuntu-k3s-master" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.419824655Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.420603083Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-delegator.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.421214570Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-reader.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.422331935Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-apiservice.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.423083938Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-service.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.423653962Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/resource-reader.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.424379928Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.425059766Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.425944950Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.426635065Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.427482344Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/aggregated-metrics-reader.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.428122274Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-deployment.yaml" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.443470 20645 remote_runtime.go:116] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3fe26e2f5928508c20018ce3f29b7d56cb2dfcc96fb7e34126a481cc2e97abc5\": open /run/flannel/subnet.env: no such file or directory" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.443642 20645 kuberuntime_sandbox.go:68] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3fe26e2f5928508c20018ce3f29b7d56cb2dfcc96fb7e34126a481cc2e97abc5\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/coredns-7448499f4d-4gc4z" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.443727 20645 kuberuntime_manager.go:790] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3fe26e2f5928508c20018ce3f29b7d56cb2dfcc96fb7e34126a481cc2e97abc5\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/coredns-7448499f4d-4gc4z" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.443904 20645 pod_workers.go:190] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-7448499f4d-4gc4z_kube-system(127a7a55-1134-44da-8a24-9fbe9b076616)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-7448499f4d-4gc4z_kube-system(127a7a55-1134-44da-8a24-9fbe9b076616)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3fe26e2f5928508c20018ce3f29b7d56cb2dfcc96fb7e34126a481cc2e97abc5\\\": open /run/flannel/subnet.env: no such file or directory\"" pod="kube-system/coredns-7448499f4d-4gc4z" podUID=127a7a55-1134-44da-8a24-9fbe9b076616 | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: I1029 03:01:40.526575 20645 network_policy_controller.go:144] Starting network policy controller | |
| Oct 29 03:01:40 ubuntu-k3s-master systemd[1]: Started Lightweight Kubernetes. | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.636835410Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.664589113Z" level=info msg="Event(v1.ObjectReference{Kind:\"Addon\", Namespace:\"kube-system\", Name:\"ccm\", UID:\"e3c9eada-799b-45f4-a9d7-f14df8554a2e\", APIVersion:\"k3s.cattle.io/v1\", ResourceVersion:\"295\", FieldPath:\"\"}): type: 'Normal' reason: 'ApplyingManifest' Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\"" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.673836546Z" level=info msg="Cluster dns configmap already exists" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.693593 20645 remote_runtime.go:116] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e22c61e06b2c637e4e6eeb211ac4e303519f5b0efbd63c2172d3f94ef79e036b\": open /run/flannel/subnet.env: no such file or directory" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.693822 20645 kuberuntime_sandbox.go:68] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e22c61e06b2c637e4e6eeb211ac4e303519f5b0efbd63c2172d3f94ef79e036b\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/metrics-server-86cbb8457f-dt8lv" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.693912 20645 kuberuntime_manager.go:790] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e22c61e06b2c637e4e6eeb211ac4e303519f5b0efbd63c2172d3f94ef79e036b\": open /run/flannel/subnet.env: no such file or directory" pod="kube-system/metrics-server-86cbb8457f-dt8lv" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: E1029 03:01:40.694362 20645 pod_workers.go:190] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"metrics-server-86cbb8457f-dt8lv_kube-system(ae4b2f7c-6868-4c65-8dfc-8e9b29b57696)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"metrics-server-86cbb8457f-dt8lv_kube-system(ae4b2f7c-6868-4c65-8dfc-8e9b29b57696)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e22c61e06b2c637e4e6eeb211ac4e303519f5b0efbd63c2172d3f94ef79e036b\\\": open /run/flannel/subnet.env: no such file or directory\"" pod="kube-system/metrics-server-86cbb8457f-dt8lv" podUID=ae4b2f7c-6868-4c65-8dfc-8e9b29b57696 | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: I1029 03:01:40.846053 20645 network_policy_controller.go:154] Starting network policy controller full sync goroutine | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.978366913Z" level=info msg="Starting helm.cattle.io/v1, Kind=HelmChartConfig controller" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.978553205Z" level=info msg="Starting helm.cattle.io/v1, Kind=HelmChart controller" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.979498370Z" level=info msg="Starting batch/v1, Kind=Job controller" | |
| Oct 29 03:01:40 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:40.980441128Z" level=info msg="Event(v1.ObjectReference{Kind:\"Addon\", Namespace:\"kube-system\", Name:\"ccm\", UID:\"e3c9eada-799b-45f4-a9d7-f14df8554a2e\", APIVersion:\"k3s.cattle.io/v1\", ResourceVersion:\"295\", FieldPath:\"\"}): type: 'Normal' reason: 'AppliedManifest' Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\"" | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: I1029 03:01:41.002956 20645 controller.go:611] quota admission added evaluator for: addons.k3s.cattle.io | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:41.076658020Z" level=info msg="Starting /v1, Kind=Node controller" | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:41.076973791Z" level=info msg="Starting /v1, Kind=Service controller" | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:41.077129121Z" level=info msg="Starting /v1, Kind=Pod controller" | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:41.077290859Z" level=info msg="Starting /v1, Kind=Endpoints controller" | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:41.077850495Z" level=info msg="Starting /v1, Kind=Secret controller" | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:41.116546392Z" level=info msg="Event(v1.ObjectReference{Kind:\"Addon\", Namespace:\"kube-system\", Name:\"coredns\", UID:\"8716daf0-1e90-452f-a04a-7ed17c777998\", APIVersion:\"k3s.cattle.io/v1\", ResourceVersion:\"313\", FieldPath:\"\"}): type: 'Normal' reason: 'ApplyingManifest' Applying manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\"" | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: I1029 03:01:41.144905 20645 controller.go:611] quota admission added evaluator for: helmcharts.helm.cattle.io | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: I1029 03:01:41.349305 20645 kube.go:123] Node controller sync successful | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: I1029 03:01:41.349545 20645 vxlan.go:123] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false | |
| Oct 29 03:01:41 ubuntu-k3s-master k3s[20645]: time="2021-10-29T03:01:41.358989929Z" level=fatal msg="flannel exited: operation not supported" | |
| Oct 29 03:01:41 ubuntu-k3s-master systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE | |
| Oct 29 03:01:41 ubuntu-k3s-master systemd[1]: k3s.service: Failed with result 'exit-code'. | |
| Oct 29 03:01:41 ubuntu-k3s-master systemd[1]: k3s.service: Consumed 32.257s CPU time. | |
| Oct 29 03:01:46 ubuntu-k3s-master systemd[1]: k3s.service: Scheduled restart job, restart counter is at 44. | |
| Oct 29 03:01:46 ubuntu-k3s-master systemd[1]: Stopped Lightweight Kubernetes. | |
| Oct 29 03:01:46 ubuntu-k3s-master systemd[1]: k3s.service: Consumed 32.257s CPU time. | |
| Oct 29 03:01:46 ubuntu-k3s-master systemd[1]: Starting Lightweight Kubernetes... | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:47.340823692Z" level=info msg="Starting k3s v1.21.5+k3s2 (724ef700)" | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:47.341547622Z" level=info msg="Cluster bootstrap already complete" | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:47.408536804Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s" | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:47.408663708Z" level=info msg="Configuring database table schema and indexes, this may take a moment..." | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:47.409137365Z" level=info msg="Database tables and indexes are up to date" | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:47.427163060Z" level=info msg="Kine listening on unix://kine.sock" | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:47.428346664Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --enable-admission-plugins=NodeRestriction --etcd-servers=unix://kine.sock --insecure-port=0 --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: Flag --insecure-port has been deprecated, This flag has no effect now and will be removed in v1.24. | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.432841 21077 server.go:656] external host was not specified, using 192.168.86.90 | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.433825 21077 server.go:195] Version: v1.21.5+k3s2 | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.445901 21077 shared_informer.go:240] Waiting for caches to sync for node_authorizer | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.451686 21077 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.451809 21077 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota. | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.458354 21077 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.458454 21077 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota. | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.561397 21077 instance.go:283] Using reconciler: lease | |
| Oct 29 03:01:47 ubuntu-k3s-master k3s[21077]: I1029 03:01:47.700355 21077 rest.go:130] the default service ipfamily for this cluster is: IPv4 | |
| Oct 29 03:01:48 ubuntu-k3s-master k3s[21077]: W1029 03:01:48.949755 21077 genericapiserver.go:425] Skipping API node.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:48 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:48.954774568Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:48 http: TLS handshake error from 192.168.86.93:50568: remote error: tls: bad certificate" | |
| Oct 29 03:01:48 ubuntu-k3s-master k3s[21077]: W1029 03:01:48.999493 21077 genericapiserver.go:425] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: W1029 03:01:49.016640 21077 genericapiserver.go:425] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: W1029 03:01:49.045889 21077 genericapiserver.go:425] Skipping API storage.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: W1029 03:01:49.057907 21077 genericapiserver.go:425] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: W1029 03:01:49.086073 21077 genericapiserver.go:425] Skipping API apps/v1beta2 because it has no resources. | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: W1029 03:01:49.086161 21077 genericapiserver.go:425] Skipping API apps/v1beta1 because it has no resources. | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: I1029 03:01:49.126459 21077 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: I1029 03:01:49.126574 21077 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota. | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.173440925Z" level=info msg="Waiting for API server to become available" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.173527812Z" level=info msg="Running kube-scheduler --address=127.0.0.1 --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --leader-elect=false --port=10251 --profiling=false --secure-port=0" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.175431921Z" level=info msg="Running kube-controller-manager --address=127.0.0.1 --allocate-node-cidrs=true --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --configure-cloud-routes=false --controllers=*,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --leader-elect=false --port=10252 --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=0 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.key --use-service-account-credentials=true" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.177466489Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --bind-address=127.0.0.1 --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --leader-elect=false --node-status-update-frequency=1m0s --port=0 --profiling=false" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.181115227Z" level=info msg="Node token is available at /var/lib/rancher/k3s/server/token" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.181257335Z" level=info msg="To join node to cluster: k3s agent -s https://192.168.86.90:6443 -t ${NODE_TOKEN}" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.184790150Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.184933110Z" level=info msg="Run: k3s kubectl" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.288330294Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:49 http: TLS handshake error from 127.0.0.1:46550: remote error: tls: bad certificate" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.308128082Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:49 http: TLS handshake error from 127.0.0.1:46556: remote error: tls: bad certificate" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.376057732Z" level=info msg="certificate CN=ubuntu-k3s-master signed by CN=k3s-server-ca@1635475569: notBefore=2021-10-29 02:46:09 +0000 UTC notAfter=2022-10-29 03:01:49 +0000 UTC" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.394614843Z" level=info msg="certificate CN=system:node:ubuntu-k3s-master,O=system:nodes signed by CN=k3s-client-ca@1635475569: notBefore=2021-10-29 02:46:09 +0000 UTC notAfter=2022-10-29 03:01:49 +0000 UTC" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.697029761Z" level=info msg="Module overlay was already loaded" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.697236534Z" level=info msg="Module nf_conntrack was already loaded" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.697301015Z" level=info msg="Module br_netfilter was already loaded" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.697348921Z" level=info msg="Module iptable_nat was already loaded" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: W1029 03:01:49.697460 21077 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.705044630Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log" | |
| Oct 29 03:01:49 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:49.705319421Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd" | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:50.709186104Z" level=info msg="Containerd is now running" | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:50.739042794Z" level=info msg="Connecting to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect" | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:50.748936309Z" level=info msg="Handling backend connection request [ubuntu-k3s-master]" | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:50.751170892Z" level=info msg="Running kubelet --address=0.0.0.0 --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --cni-bin-dir=/var/lib/rancher/k3s/data/5b1d238ed00ea9bbae6aa96d867c1264a96ad48b42e55d56570d54d05c8d8a7a/bin --cni-conf-dir=/var/lib/rancher/k3s/agent/etc/cni/net.d --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --container-runtime=remote --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=ubuntu-k3s-master --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-labels= --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key" | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:50.752933003Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=ubuntu-k3s-master --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables" | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: Flag --cloud-provider has been deprecated, will be removed in 1.23, in favor of removing cloud provider code from Kubelet. | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: Flag --cni-bin-dir has been deprecated, will be removed along with dockershim. | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: Flag --cni-conf-dir has been deprecated, will be removed along with dockershim. | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed. | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: W1029 03:01:50.755481 21077 server.go:220] WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP. | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: I1029 03:01:50.806053 21077 server.go:436] "Kubelet version" kubeletVersion="v1.21.5+k3s2" | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: E1029 03:01:50.847821 21077 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-master" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: I1029 03:01:50.904364 21077 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt | |
| Oct 29 03:01:50 ubuntu-k3s-master k3s[21077]: W1029 03:01:50.904514 21077 manager.go:159] Cannot detect current cgroup on cgroup v2 | |
| Oct 29 03:01:51 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:51.502283695Z" level=info msg="Handling backend connection request [ubuntu-k3s-wkr1]" | |
| Oct 29 03:01:51 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:51.504823864Z" level=info msg="Handling backend connection request [ubuntu-k3s-wkr2]" | |
| Oct 29 03:01:52 ubuntu-k3s-master k3s[21077]: E1029 03:01:52.008632 21077 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-master" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 03:01:53 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:53.981316729Z" level=info msg="Cluster-Http-Server 2021/10/29 03:01:53 http: TLS handshake error from 192.168.86.93:50572: remote error: tls: bad certificate" | |
| Oct 29 03:01:54 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:54.075366683Z" level=error msg="runtime core not ready" | |
| Oct 29 03:01:54 ubuntu-k3s-master k3s[21077]: E1029 03:01:54.075512 21077 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-master" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: time="2021-10-29T03:01:55.743474269Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: W1029 03:01:55.928621 21077 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.940412 21077 server.go:660] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.942749 21077 container_manager_linux.go:291] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.944050 21077 container_manager_linux.go:296] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none Rootless:false} | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.945336 21077 topology_manager.go:120] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.946494 21077 container_manager_linux.go:327] "Initializing Topology Manager" policy="none" scope="container" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.947324 21077 container_manager_linux.go:332] "Creating device plugin manager" devicePluginEnabled=true | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.953110 21077 kubelet.go:404] "Attempting to sync node with API server" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.954794 21077 kubelet.go:272] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.955855 21077 kubelet.go:283] "Adding apiserver pod source" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.958075 21077 apiserver.go:42] "Waiting for node sync before watching apiserver pods" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.962150 21077 kuberuntime_manager.go:222] "Container runtime initialized" containerRuntime="containerd" version="v1.4.11-k3s1" apiVersion="v1alpha2" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.964640 21077 server.go:1191] "Started kubelet" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.982576 21077 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.984627 21077 server.go:149] "Starting to listen" address="0.0.0.0" port=10250 | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: E1029 03:01:55.992976 21077 cri_stats_provider.go:369] "Failed to get the info of the filesystem with mountpoint" err="unable to find data in memory cache" mountpoint="/var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.993090 21077 server.go:409] "Adding debug handlers to kubelet server" | |
| Oct 29 03:01:55 ubuntu-k3s-master k3s[21077]: I1029 03:01:55.996010 21077 volume_manager.go:271] "Starting Kubelet Volume Manager" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: E1029 03:01:55.993109 21077 kubelet.go:1306] "Image garbage collection failed once. Stats initialization may not have completed yet" err="invalid capacity 0 on image filesystem" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.013052 21077 desired_state_of_world_populator.go:141] "Desired state populator starts to run" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.106950 21077 kuberuntime_manager.go:1044] "Updating runtime config through cri with podcidr" CIDR="10.42.0.0/24" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.118048 21077 kubelet_network.go:76] "Updating Pod CIDR" originalPodCIDR="" newPodCIDR="10.42.0.0/24" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.122000 21077 kubelet_node_status.go:71] "Attempting to register node" node="ubuntu-k3s-master" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.303166 21077 kubelet_network_linux.go:56] "Initialized protocol iptables rules." protocol=IPv4 | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.464550 21077 kubelet_network_linux.go:56] "Initialized protocol iptables rules." protocol=IPv6 | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.464657 21077 status_manager.go:157] "Starting to sync pod status with apiserver" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.464722 21077 kubelet.go:1846] "Starting kubelet main sync loop" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: E1029 03:01:56.464909 21077 kubelet.go:1870] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: E1029 03:01:56.570995 21077 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: E1029 03:01:56.775965 21077 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 03:01:56 ubuntu-k3s-master k3s[21077]: I1029 03:01:56.960547 21077 apiserver.go:52] "Watching apiserver" | |
| Oct 29 03:01:57 ubuntu-k3s-master k3s[21077]: E1029 03:01:57.176935 21077 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Oct 29 02:42:48 ubuntu-k3s-wkr1 systemd[1]: k3s-node.service: Main process exited, code=exited, status=1/FAILURE | |
| Oct 29 02:42:48 ubuntu-k3s-wkr1 systemd[1]: k3s-node.service: Failed with result 'exit-code'. | |
| Oct 29 02:42:48 ubuntu-k3s-wkr1 systemd[1]: Stopped Lightweight Kubernetes. | |
| Oct 29 02:42:48 ubuntu-k3s-wkr1 systemd[1]: k3s-node.service: Consumed 2.549s CPU time. | |
| -- Boot 916f2a3c80474e0794dd8ba58cda8c99 -- | |
| Oct 29 02:47:13 ubuntu-k3s-wkr1 systemd[1]: Starting Lightweight Kubernetes... | |
| Oct 29 02:47:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:14Z" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock" | |
| Oct 29 02:47:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:14Z" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/5b1d238ed00ea9bbae6aa96d867c1264a96ad48b42e55d56570d54d05c8d8a7a" | |
| Oct 29 02:47:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:22.175862974Z" level=info msg="Starting k3s agent v1.21.5+k3s2 (724ef700)" | |
| Oct 29 02:47:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:22.178068129Z" level=info msg="Running load balancer 127.0.0.1:6444 -> [192.168.86.90:6443]" | |
| Oct 29 02:47:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:22.526144774Z" level=error msg="Failed to configure agent: https://127.0.0.1:6444/v1-k3s/serving-kubelet.crt: 503 Service Unavailable" | |
| Oct 29 02:47:27 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:27.533919075Z" level=error msg="Failed to configure agent: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF" | |
| Oct 29 02:47:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:32.622221332Z" level=error msg="Failed to configure agent: https://127.0.0.1:6444/v1-k3s/serving-kubelet.crt: 503 Service Unavailable" | |
| Oct 29 02:47:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:37.752032108Z" level=error msg="Failed to configure agent: https://127.0.0.1:6444/v1-k3s/serving-kubelet.crt: 503 Service Unavailable" | |
| Oct 29 02:47:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:42.909486249Z" level=error msg="Failed to configure agent: https://127.0.0.1:6444/v1-k3s/serving-kubelet.crt: 503 Service Unavailable" | |
| Oct 29 02:47:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:47.916449232Z" level=error msg="Failed to configure agent: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF" | |
| Oct 29 02:47:52 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:52.996412360Z" level=error msg="Failed to configure agent: https://127.0.0.1:6444/v1-k3s/serving-kubelet.crt: 503 Service Unavailable" | |
| Oct 29 02:47:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:47:58.073757931Z" level=error msg="Failed to configure agent: https://127.0.0.1:6444/v1-k3s/serving-kubelet.crt: 503 Service Unavailable" | |
| Oct 29 02:48:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:04.514121234Z" level=info msg="Module overlay was already loaded" | |
| Oct 29 02:48:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:04.552447652Z" level=info msg="Module br_netfilter was already loaded" | |
| Oct 29 02:48:04 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:04.571599 3222 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 02:48:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:04.574636343Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400" | |
| Oct 29 02:48:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:04.574948304Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600" | |
| Oct 29 02:48:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:04.575553673Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072" | |
| Oct 29 02:48:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:04.579794715Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log" | |
| Oct 29 02:48:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:04.580150381Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd" | |
| Oct 29 02:48:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:05.588657505Z" level=info msg="Containerd is now running" | |
| Oct 29 02:48:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:10.679898951Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:48:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:10.693189430Z" level=error msg="Failed to connect to proxy" error="websocket: bad handshake" | |
| Oct 29 02:48:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:10.693286856Z" level=error msg="Remotedialer proxy error" error="websocket: bad handshake" | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:15.684241765Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:15.694078034Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:15.709869692Z" level=info msg="Running kubelet --address=0.0.0.0 --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --cni-bin-dir=/var/lib/rancher/k3s/data/5b1d238ed00ea9bbae6aa96d867c1264a96ad48b42e55d56570d54d05c8d8a7a/bin --cni-conf-dir=/var/lib/rancher/k3s/agent/etc/cni/net.d --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --container-runtime=remote --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=ubuntu-k3s-wkr1 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-labels= --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key" | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:15.712385722Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=ubuntu-k3s-wkr1 --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables" | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: Flag --cloud-provider has been deprecated, will be removed in 1.23, in favor of removing cloud provider code from Kubelet. | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: Flag --cni-bin-dir has been deprecated, will be removed along with dockershim. | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: Flag --cni-conf-dir has been deprecated, will be removed along with dockershim. | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed. | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:15.713560 3222 server.go:220] WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP. | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:15.761400 3222 server.go:436] "Kubelet version" kubeletVersion="v1.21.5+k3s2" | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:15.852608 3222 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-wkr1" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:15.857259 3222 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt | |
| Oct 29 02:48:15 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:15.857391 3222 manager.go:159] Cannot detect current cgroup on cgroup v2 | |
| Oct 29 02:48:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:16.937270 3222 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-wkr1" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 02:48:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:19.123342 3222 node.go:161] Failed to retrieve node info: nodes "ubuntu-k3s-wkr1" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:20.883132 3222 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.892056 3222 server.go:660] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.893109 3222 container_manager_linux.go:291] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.893333 3222 container_manager_linux.go:296] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none Rootless:false} | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.893599 3222 topology_manager.go:120] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.893649 3222 container_manager_linux.go:327] "Initializing Topology Manager" policy="none" scope="container" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.893681 3222 container_manager_linux.go:332] "Creating device plugin manager" devicePluginEnabled=true | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.894832 3222 kubelet.go:404] "Attempting to sync node with API server" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.894929 3222 kubelet.go:272] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.895082 3222 kubelet.go:283] "Adding apiserver pod source" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.895150 3222 apiserver.go:42] "Waiting for node sync before watching apiserver pods" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.898048 3222 kuberuntime_manager.go:222] "Container runtime initialized" containerRuntime="containerd" version="v1.4.11-k3s1" apiVersion="v1alpha2" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:20.898737 3222 probe.go:268] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating. | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.900758 3222 server.go:1191] "Started kubelet" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.905846 3222 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.912958 3222 volume_manager.go:271] "Starting Kubelet Volume Manager" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.915215 3222 desired_state_of_world_populator.go:141] "Desired state populator starts to run" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.917322 3222 server.go:149] "Starting to listen" address="0.0.0.0" port=10250 | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:20.918029 3222 cri_stats_provider.go:369] "Failed to get the info of the filesystem with mountpoint" err="unable to find data in memory cache" mountpoint="/var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:20.918891 3222 kubelet.go:1306] "Image garbage collection failed once. Stats initialization may not have completed yet" err="invalid capacity 0 on image filesystem" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:20.919806 3222 server.go:409] "Adding debug handlers to kubelet server" | |
| Oct 29 02:48:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:20.999050 3222 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ubuntu-k3s-wkr1\" not found" node="ubuntu-k3s-wkr1" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:21.014933 3222 kubelet.go:2291] "Error getting node" err="node \"ubuntu-k3s-wkr1\" not found" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:21.023729 3222 kubelet_node_status.go:71] "Attempting to register node" node="ubuntu-k3s-wkr1" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:21.039481 3222 kubelet_node_status.go:74] "Successfully registered node" node="ubuntu-k3s-wkr1" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:21.246005 3222 kubelet_network_linux.go:56] "Initialized protocol iptables rules." protocol=IPv4 | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:21.412186 3222 kubelet_network_linux.go:56] "Initialized protocol iptables rules." protocol=IPv6 | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:21.412295 3222 status_manager.go:157] "Starting to sync pod status with apiserver" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:21.412355 3222 kubelet.go:1846] "Starting kubelet main sync loop" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:21.412534 3222 kubelet.go:1870] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:21.513293 3222 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:21.713598 3222 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 02:48:21 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:21.896439 3222 apiserver.go:52] "Watching apiserver" | |
| Oct 29 02:48:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:22.113764 3222 kubelet.go:1870] "Skipping pod synchronization" err="container runtime status check may not have completed yet" | |
| Oct 29 02:48:22 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:22.655415 3222 cpu_manager.go:199] "Starting CPU manager" policy="none" | |
| Oct 29 02:48:22 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:22.655483 3222 cpu_manager.go:200] "Reconciling" reconcilePeriod="10s" | |
| Oct 29 02:48:22 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:22.655550 3222 state_mem.go:36] "Initialized new in-memory state store" | |
| Oct 29 02:48:22 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:22.668452 3222 policy_none.go:44] "None policy: Start" | |
| Oct 29 02:48:22 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:22.702881 3222 manager.go:600] "Failed to retrieve checkpoint" checkpoint="kubelet_internal_checkpoint" err="checkpoint is not found" | |
| Oct 29 02:48:22 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:22.732962 3222 plugin_manager.go:114] "Starting Kubelet Plugin Manager" | |
| Oct 29 02:48:22 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:22.928543 3222 reconciler.go:157] "Reconciler: start to sync state" | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.705211 3222 node.go:172] Successfully retrieved node IP: 192.168.86.91 | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.705391 3222 server_others.go:141] Detected node IP 192.168.86.91 | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.726742 3222 server_others.go:207] kube-proxy running in dual-stack mode, IPv4-primary | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.726844 3222 server_others.go:213] Using iptables Proxier. | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.726894 3222 server_others.go:220] creating dualStackProxier for iptables. | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:23.726942 3222 server_others.go:519] detect-local-mode set to ClusterCIDR, but no IPv6 cluster CIDR defined, , defaulting to no-op detect-local for IPv6 | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.728410 3222 server.go:643] Version: v1.21.5+k3s2 | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.729807 3222 config.go:315] Starting service config controller | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.729891 3222 shared_informer.go:240] Waiting for caches to sync for service config | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.729967 3222 config.go:224] Starting endpoint slice config controller | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.729999 3222 shared_informer.go:240] Waiting for caches to sync for endpoint slice config | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:23.759570 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:23.765115 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.830936 3222 shared_informer.go:247] Caches are synced for endpoint slice config | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:23.830958 3222 shared_informer.go:247] Caches are synced for service config | |
| Oct 29 02:48:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:23.846318673Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:24.895828101Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:48:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:24.910105 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:24.910171 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:24.910199 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:24.910474 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:24.910755 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:24.910756 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:24.910903 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1beta1.EndpointSlice ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:29.896606709Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:48:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:29.897584429Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:48:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:29.897726465Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.006778 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: the server is currently unable to handle the request (get nodes) | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.040794 3222 reflector.go:138] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Failed to watch *v1.Pod: failed to list *v1.Pod: the server is currently unable to handle the request (get pods) | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.111400 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.120934 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.135430 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.152527 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.155126 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.163849 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.164554 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.272872 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.547262 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: the server is currently unable to handle the request (get runtimeclasses.node.k8s.io) | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:31.772946 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:48:31 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:31.885127939Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:32.136205 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:48:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:32.181737 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: the server is currently unable to handle the request (get csidrivers.storage.k8s.io) | |
| Oct 29 02:48:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:32.338823 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:48:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:32.807273 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-proxy" cannot list resource "services" in API group "" at the cluster scope | |
| Oct 29 02:48:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:33.897598817Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:34 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:34.026894 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 02:48:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:34.898078301Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:48:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:34.926274961Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:48:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:35.910756270Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:37.924926946Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:38.143792 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-proxy" cannot list resource "services" in API group "" at the cluster scope | |
| Oct 29 02:48:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:38.602913 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 02:48:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:39.930101321Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:48:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:39.939326813Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:41.982536852Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:43.995104644Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:45.529897832Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:48:45 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:45.536564 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:45 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:45.537796 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:45 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:45.538299 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:45 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:45.539760 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:45 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:48:45.540159 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:48:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:50.534282865Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:48:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:50.537034059Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:48:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:50.537199778Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:51.313032 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:51.316359 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:51.319381 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:51.322689 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:51.325167 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:48:51.325278 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:51.761706 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: the server is currently unable to handle the request (get runtimeclasses.node.k8s.io) | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:51.809733 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:48:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:51.908154 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:52.035668573Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.151535 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.316715 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.320030 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.335929 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.365300 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.385885 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.385996 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.461084 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: the server is currently unable to handle the request (get nodes) | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.578817 3222 reflector.go:138] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Failed to watch *v1.Pod: failed to list *v1.Pod: the server is currently unable to handle the request (get pods) | |
| Oct 29 02:48:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:48:52.581102 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: the server is currently unable to handle the request (get csidrivers.storage.k8s.io) | |
| Oct 29 02:48:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:54.051177204Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:55.538066625Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:48:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:55.550889360Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:48:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:56.062465792Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:48:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:48:58.073312603Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:00.093813459Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:00.555892216Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:49:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:02.109785337Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:03.372190 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": net/http: request canceled (Client.Timeout exceeded while awaiting headers) | |
| Oct 29 02:49:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:04.120187620Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:06.131859286Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:07 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:07.687204 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:49:07 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:07.844242 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:49:07 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:07.980147 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1beta1.EndpointSlice ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:08.019909672Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:49:08 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:08.028285 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:08 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:08.029768 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:08 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:08.029768 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:08 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:08.029930 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:08 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:08.029971 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:08 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:08.031981 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:13.020665605Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:13.021783641Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:13.021956971Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:13.669475 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:13.673484 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:13.676943 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:13.679205 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:13.681510 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:13 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:49:13.681607 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:14.173722240Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:14.430801 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:14.433228 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:14.446392 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:14.458560 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:14.470004 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:14.470684 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:14.710352 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:49:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:14.913607 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:49:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:15.316333 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:49:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:16.186657498Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:18.022530848Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:49:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:18.041508693Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:49:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:18.207805857Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:20.222353993Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:22.236339479Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:23.046178919Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:49:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:24.249344697Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:26.261373396Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:28.274600556Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:30.404584 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:30.405046 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:30.444900833Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:49:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:49:30.451043 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:49:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:35.445925880Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:49:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:35.447092808Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:49:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:35.447262732Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.275562 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.278927 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.281911 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.284724 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.287266 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:49:36.287360 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:36.497849378Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.529833 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.531823 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.543637 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.554933 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.569583 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.569677 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.731378 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: the server is currently unable to handle the request (get csidrivers.storage.k8s.io) | |
| Oct 29 02:49:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:36.735405 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: the server is currently unable to handle the request (get nodes) | |
| Oct 29 02:49:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:37.307237 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:49:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:37.509730 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:49:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:37.912480 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:49:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:38.523540109Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:40.448214861Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:49:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:40.465239724Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:49:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:40.537764097Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:42.552214956Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:44.083509 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-proxy" cannot list resource "services" in API group "" at the cluster scope | |
| Oct 29 02:49:44 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:44.565724491Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:45.469193146Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:49:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:46.658077944Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:48.669202468Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:49:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:51.016276278Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:56.018480656Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:56.019220684Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:56.019338794Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:56.877738 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:56.881714 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:56.885775 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:56.887997 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:56.890328 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:49:56 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:49:56.890434 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:49:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:57.625904 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:57.627946 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:57.645534 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:57.659045 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:57.678046 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:57.678728 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:49:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:57.782196 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:49:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:57.915148 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:49:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:58.021590 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 02:49:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:49:58.068653420Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:49:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:58.120464 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:49:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:49:58.523228 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:00.084273064Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:01.020282946Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:50:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:01.040632645Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:50:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:02.095499111Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:04.107119789Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:06.045798853Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:50:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:06.120274686Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:08.138927397Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:10.157780248Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:11.628414099Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:50:11 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:11.638242 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:50:11 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:11.638242 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:50:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:16.629208532Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:50:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:16.630339817Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:50:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:16.630512593Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.737352 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.747303 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.762276 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.776680 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.788179 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.788275 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.890548 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.900539 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.920264 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.934740 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.946292 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:50:17.946754 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:50:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:17.957774 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:18.166454 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:18.214417972Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:18.569901 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:18.760735 3222 reflector.go:138] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Failed to watch *v1.Pod: failed to list *v1.Pod: the server is currently unable to handle the request (get pods) | |
| Oct 29 02:50:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:20.253096670Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:21 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:21.631707583Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:50:21 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:21.649245786Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:50:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:22.264707600Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:24.275705663Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:26.288842379Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:26.654158091Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:50:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:28.303119310Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:29 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:29.478676 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:50:29 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:29.482661 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:50:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:30.317530193Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:32.159411641Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:50:32 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:32.169360 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:50:32 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:32.169620 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1beta1.EndpointSlice ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:50:32 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:32.169620 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:37.160218536Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:37.161378026Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:37.161530284Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:37.775094 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:37.778487 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:37.781570 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:37.784743 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:37.787818 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:37 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:50:37.787921 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:38.314710 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:38.368215841Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:38.812802 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:38.831674 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:38.837027 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:38.861256 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:38.879085 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:38.898090 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:38.898224 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:50:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:39.026746 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:39.428798 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:40.380542770Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:42.161945158Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:50:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:42.181572847Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:50:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:42.392877481Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:44 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:44.405773388Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:46.418593285Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:47.186721876Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:50:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:48.464898529Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:50.483614160Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:52 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:52.594650392Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:50:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:53.104547801Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:50:53 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:53.117379 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:50:53 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:50:53.117379 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:58.104843933Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:58.105831666Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:58.106000850Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:58.877826 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:58.881246 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:58.884371 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:58.887478 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:58.890174 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:50:58 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:50:58.890275 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:50:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:50:59.633977048Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:59.917195 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:50:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:59.945130 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:59.951360 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:59.971154 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:59.983847 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:59.996520 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:50:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:50:59.996618 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:51:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:00.132245 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:51:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:00.534993 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:51:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:01.647399006Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:03 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:03.106978222Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:51:03 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:03.124880275Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:51:03 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:03.659011792Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:04.823659 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 02:51:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:05.669850411Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:07.680530541Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:08.130318656Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:51:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:09.692188666Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:11.704331607Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:13 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:13.849484971Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:14.716441750Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:51:14 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:51:14.723651 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:51:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:19.716727268Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:51:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:19.717720150Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:51:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:19.717888520Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:51:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:20.163293 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:20.166133 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:20.168954 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:20.171706 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:20.174560 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:51:20.174681 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:51:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:20.892713914Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.048468 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.050751 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.060978 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.072477 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.082893 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.082977 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.177060 3222 reflector.go:138] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Failed to watch *v1.Pod: failed to list *v1.Pod: the server is currently unable to handle the request (get pods) | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.189911 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.399722 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:51:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:21.802916 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:51:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:22.905131117Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:24.718308909Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:51:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:24.736000567Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:51:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:24.920418556Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:26.931743513Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:28.942052928Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:29.741395118Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:51:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:30.956382061Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:32.626454 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": net/http: request canceled (Client.Timeout exceeded while awaiting headers) | |
| Oct 29 02:51:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:32.974880260Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:34.985646267Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:37.120254706Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:51:37 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:51:37.130852 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:51:37 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:51:37.133908 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:51:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:42.120631202Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:51:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:42.123170741Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:51:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:42.123319962Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:51:43 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:43.061813 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:43 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:43.065845 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:43 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:43.069629 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:43 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:43.072054 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:43 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:43.074547 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:51:43 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:51:43.074653 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:51:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:43.173302295Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:43 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:43.796288 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: the server is currently unable to handle the request (get nodes) | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.092509 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.142464 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.144727 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.155855 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.167854 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.178701 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.178802 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.306050 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:51:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:51:44.709327 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:51:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:45.185295656Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:47.123725849Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:51:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:47.145989154Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:51:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:47.198747340Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:49.212671723Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:51.230707410Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:52 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:52.150929201Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:51:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:53.297490586Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:55.308136381Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:57.393075820Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:51:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:51:57.807556687Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:51:57 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:51:57.818766 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:52:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:02.808064272Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:52:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:02.809155950Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:52:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:02.809310209Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:52:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:03.414656 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:03.418888 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:03.421364 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:03.423713 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:03.426078 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:03 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:52:03.426190 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.259699 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.261466 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.261696 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.272265 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.282842 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.293589 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.293679 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:04.431892386Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.442644 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:04.661325 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:05 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:05.063988 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:06.445148159Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:07.810406732Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:52:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:07.827717517Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:52:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:08.462226203Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:10.474017035Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:12.492721283Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:12.831927846Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:52:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:14.504302381Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:16.516831742Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:18.404414418Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:52:18 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:52:18.411636 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:52:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:23.404979317Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:52:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:23.406075459Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:52:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:23.406238236Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:24.113001 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:24.116956 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:24.120394 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:24.122763 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:24.125158 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:52:24.125271 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:24.568647601Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:24.685117 3222 reflector.go:138] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Failed to watch *v1.Pod: failed to list *v1.Pod: the server is currently unable to handle the request (get pods) | |
| Oct 29 02:52:24 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:24.698144 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.194247 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.335923 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.339297 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.352349 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.375735 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.390872 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.391729 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.406809 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:25.809329 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:26.593682401Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:28.407376550Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:52:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:28.425897814Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:52:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:28.609169166Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:30.621917243Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:32.634499968Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:33.429625588Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:52:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:34.673446153Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:36.693024038Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:38.770189834Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:39.106277817Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:52:39 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:52:39.115486 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:52:39 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:52:39.115486 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:52:39 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:52:39.115591 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:44.107311177Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:44.108476170Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:44.108640855Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:44.529218 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:44.533217 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:44.535733 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:44.538271 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:44.540640 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:52:44 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:52:44.540747 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:45.438373 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:45.440302 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:45.449989 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:45.460515 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:45.470843 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:45.470918 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:45.559633 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:45.762562 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:45.815688513Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:52:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:46.165590 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:52:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:47.828325313Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:49.109616953Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:52:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:49.132397362Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:52:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:49.839478038Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:51.851106378Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:53 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:52:53.811201 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 02:52:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:53.861147893Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:54.137119706Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:52:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:55.873821212Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:57.910082435Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:52:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:52:59.568714405Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:53:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:04.569481200Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:53:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:04.570573895Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:53:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:04.570731595Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:53:05 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:05.135796 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:05 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:05.139828 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:05 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:05.142651 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:05 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:05.145256 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:05 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:05.147957 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:05 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:53:05.148072 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:53:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:05.952429380Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.132485 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: the server is currently unable to handle the request (get csidrivers.storage.k8s.io) | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.166934 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.376360 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.515149 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.518023 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.538830 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.553523 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.568371 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.569118 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.640145 3222 reflector.go:138] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Failed to watch *v1.Pod: failed to list *v1.Pod: the server is currently unable to handle the request (get pods) | |
| Oct 29 02:53:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:06.789419 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:07.964336191Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:09.571901678Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:53:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:09.589744131Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:53:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:09.974210837Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:11.983946209Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:13 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:13.994360941Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:14.594619071Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:53:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:16.022670121Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:18.032635616Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:20.067877748Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:20.235778338Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:53:20 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:53:20.245090 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:53:20 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:53:20.245091 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:53:22 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:53:22.666998 3222 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 02:53:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:25.237044651Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:53:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:25.238077831Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:53:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:25.238234680Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.187576 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.190055 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.203562 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.218792 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.244355 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:53:26.244731 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.275917 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.488124 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.576746 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.620590 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.622450 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.633114 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.643991 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.654657 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.654750 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:53:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:26.908484 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:27 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:27.106414890Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:27.385311 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: the server is currently unable to handle the request (get runtimeclasses.node.k8s.io) | |
| Oct 29 02:53:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:27.711932 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:29.125585973Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:30.239133448Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:53:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:30.257429936Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:53:31 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:31.136197779Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:33.147282098Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:35.159634332Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:35.261504546Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:53:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:37.171893544Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:39.183420744Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:41.015709063Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:46.016411050Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:46.017602581Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:46.017763856Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:46.872809 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:46.876295 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:46.879839 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:46.882742 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:46.884982 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:53:46 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:53:46.885075 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.237778 3222 reflector.go:138] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Failed to watch *v1.Pod: failed to list *v1.Pod: the server is currently unable to handle the request (get pods) | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:47.244657707Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.715630 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.719116 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.735890 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.759028 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.782920 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.783040 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.788110 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 02:53:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:47.909938 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:48.116521 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:53:48.519066 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:53:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:49.273205679Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:51.018687985Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:53:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:51.036506115Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:53:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:51.284684718Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:53.299670115Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:55.311813243Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:56.041456292Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:53:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:57.337555078Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:53:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:53:59.350265439Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:01.487645615Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:01.723857542Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:54:01 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:54:01.733134 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:54:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:06.724820129Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:54:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:06.725990033Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:54:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:06.726160789Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.362724 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.366172 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.369100 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.371968 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.374763 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:54:07.374880 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.800963 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.842296 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.844833 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.859204 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.878528 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.892787 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:07.892883 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:54:08 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:08.086858 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: the server is currently unable to handle the request (get csidrivers.storage.k8s.io) | |
| Oct 29 02:54:08 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:08.436892 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:08.530123041Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:08 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:08.646346 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:09.049185 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:10.562661768Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:11.726335860Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:54:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:11.745260817Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:54:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:12.574313799Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:14.584995434Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:16.596628595Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:16.750272063Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:54:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:18.613379203Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:20.625205341Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:22.703537382Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:54:22 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:54:22.712259 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:54:22 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:54:22.712357 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:54:27 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:27.704415400Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:54:27 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:27.705709323Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:54:27 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:27.705874876Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.498252 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.501301 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.504380 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.507363 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.509995 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:54:28.510091 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:28.767951480Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.956067 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.958639 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.970008 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.981554 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.994548 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:28.995289 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:54:29 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:29.515119 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 02:54:29 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:29.528582 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:29 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:29.731929 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:30 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:30.135440 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:30.781331017Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:32.706983887Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:54:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:32.725156829Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:54:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:32.793492061Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:34.806369372Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:36.819877722Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:37.729625234Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:54:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:38.866761960Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:40.876992333Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:42.923142473Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:43.555562837Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:54:43 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:54:43.563785 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:54:43 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:54:43.565164 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:48.556369736Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:48.557480274Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:48.557639568Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:48.841486 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:48.845221 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:48.848124 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:48.850712 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:48.853250 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:54:48 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:54:48.853363 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:54:49 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:49.895688 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:49.967925378Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:50 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:50.045379 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:50 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:50.047243 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:50 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:50.058209 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:50 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:50.068955 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:50 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:50.079551 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:54:50 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:50.079651 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:54:50 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:50.110246 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:50 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:50.512968 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:54:51.315190 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:54:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:51.979749302Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:53.557788181Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:54:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:53.573333799Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:54:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:53.990647201Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:56.000744461Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:58.011419685Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:54:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:54:58.578980987Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:55:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:00.038934095Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:02.049976917Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:04.235899138Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:55:04 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:55:04.244238 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:55:04 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:55:04.244433 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:55:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:09.237210989Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:55:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:09.238289937Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:55:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:09.238537767Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.192893 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.198175 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.215697 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.232679 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.245225 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:55:10.245587 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.265619 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:10.284755153Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.468476 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.518594 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: the server is currently unable to handle the request (get runtimeclasses.node.k8s.io) | |
| Oct 29 02:55:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:10.872712 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:11.134085 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:11.148942 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:11.164632 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:11.175857 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:11.187240 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:11.187334 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:55:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:11.688670 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:12.310808225Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:13.554200 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 02:55:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:14.239675118Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:55:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:14.259575841Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:55:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:14.323860427Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:16.337623081Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:18.351372026Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:19.264786573Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:55:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:20.369637289Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:22.379844501Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:24.438169497Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:55:24.735478 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:55:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:24.783831479Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:55:24 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:55:24.793350 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:55:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:29.784302515Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:55:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:29.785362964Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:55:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:29.785517259Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:55:30 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:30.401193 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:55:30 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:30.405293 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:55:30 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:30.409218 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:55:30 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:30.411436 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:55:30 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:30.413805 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:55:30 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:55:30.413902 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.242786 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.244910 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.256292 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.267648 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.278426 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.278531 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.342118 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.434501 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:31.471736586Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:31.653821 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:32.056839 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:33.482764091Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:34.785678495Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:55:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:34.802764883Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:55:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:35.494230819Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:37.505549663Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:39.521517134Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:39.807377810Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:55:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:41.543100422Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:43.554511410Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:44 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:55:44.347218 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:55:44 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:55:44.357362 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:55:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:45.436298908Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:55:45 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:55:45.442266 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:55:45 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:55:45.443505 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1beta1.EndpointSlice ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:55:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:50.436871966Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:55:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:50.437940157Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:55:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:50.438107266Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:51.430173 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:51.432368 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:51.442523 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:51.453295 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:51.464270 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:55:51.464603 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:51.476494 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:51.598840340Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:51 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:51.691689 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:52.101786 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:52.334204 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:52.343731 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:52.358683 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:52.374812 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:52.388773 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:55:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:52.389442 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:55:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:55:52.917124 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:55:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:53.612642258Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:55.438519559Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:55:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:55.455076191Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:55:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:55.626428362Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:57.638688371Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:55:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:55:59.653300864Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:00.459681724Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:56:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:01.700568923Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:03 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:03.710607143Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:05.833699827Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:06.203592807Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:56:06 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:56:06.213191 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:56:06 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:56:06.213320 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:56:06 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:56:06.213191 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:11.204244597Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:11.205425213Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:11.205607359Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:11.763132 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:11.766544 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:11.769779 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:11.772853 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:11.775569 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:11 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:56:11.775672 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:56:12 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:12.433651 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:12 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:12.439380 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:12 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:12.452046 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:12 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:12.463868 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:12 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:12.475150 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:12 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:12.475293 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:56:12 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:12.824983 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:12.869687419Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:13.028419 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:13.430814 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:14.883667731Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:16.205985984Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:56:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:16.225784312Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:56:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:16.894537482Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:18.906697453Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:20.916706237Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:21 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:21.230801469Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:56:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:22.965574237Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:24.984289235Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:26.711456576Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:56:26 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:56:26.718935 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:56:31 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:31.711724093Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:56:31 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:31.712921414Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:56:31 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:31.713241540Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:56:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:32.271712 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:32.275649 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:32.278243 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:32.280755 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:32.283347 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:56:32 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:56:32.283454 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:56:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:32.678202 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: the server is currently unable to handle the request (get nodes) | |
| Oct 29 02:56:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:32.919529 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:33.024603348Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.300440 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.509812 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.521292 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.529226 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.552561 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.568991 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.582246 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.582337 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:56:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:33.924817 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:35.035970471Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:36.714141764Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:56:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:36.732580526Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:56:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:37.045170668Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:39.056488447Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:41.067543666Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:41 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:41.631177 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 02:56:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:41.737609433Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:56:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:43.078774762Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:44.737272 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": net/http: request canceled (Client.Timeout exceeded while awaiting headers) | |
| Oct 29 02:56:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:45.105817660Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:47.115406455Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:49.123653584Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:56:49 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:56:49.135710 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:56:49 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:56:49.135889 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:56:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:54.124335990Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:56:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:54.125440405Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:56:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:54.125761494Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:55.174584020Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.563318 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.564949 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.575718 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.586697 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.597970 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:56:55.598565 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.609276 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.626369 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.628856 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.640795 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.651722 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.664074 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.664209 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:56:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:55.900049 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:56:56.303414 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:56:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:57.187446437Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:56:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:59.126252490Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:56:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:59.145183367Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:56:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:56:59.201522468Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:01.216121675Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:03 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:03.228520381Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:04.149514132Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:57:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:05.250824738Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:07.262043551Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:09.375247060Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:09 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:09.712709 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:57:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:09.748554284Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:57:09 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:09.763573 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:57:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:14.749175472Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:57:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:14.750207536Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:57:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:14.750379145Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.466730 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.468802 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.470711 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.472737 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.475174 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:57:15.475266 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.708193 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.711262 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.721946 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.735859 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.763627 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:15.763763 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:57:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:16.444213707Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:16.505176 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:16.708530 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:17.111576 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:17.223412 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:57:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:18.469547159Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:19.750646673Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:57:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:19.770327254Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:57:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:20.480931345Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:22.492602936Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:24.503098186Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:24.774905353Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:57:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:26.521840558Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:26 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:26.811332 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:57:26 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:26.816021 3222 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice | |
| Oct 29 02:57:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:28.542701943Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:30.180135610Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:57:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:30.188248 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1beta1.EndpointSlice ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:57:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:30.188519 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:35.181207321Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:35.182368236Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:35.182533679Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:35.776888 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:35.779972 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:35.783257 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:35.786386 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:35.788634 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:35 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:57:35.788728 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:57:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:36.583212097Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:36.834973 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:36.835001 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:36.838181 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:36.859290 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:36.897190 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:36.962120 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:36.962272 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:57:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:37.058176 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:37.460810 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:38.595130206Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:40.182999678Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:57:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:40.200545861Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:57:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:40.606057520Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:42.617871352Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:44 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:44.698238333Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:45.205382828Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:57:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:46.709848850Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:48.732652284Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:57:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:50.824183258Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:57:50 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:50.831466 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:57:50 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:50.831532 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:57:50 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:57:50.831534 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:57:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:55.827342994Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:57:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:55.828259949Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:57:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:55.828407873Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:57:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:56.356599 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:56.360030 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:56.362884 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:56.365303 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:56.367675 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:57:56 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:57:56.367779 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.022161 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.025893 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.043126 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.054590 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.065984 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.066081 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.176538 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.386124 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.596047 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:57.868999166Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:57:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:57:57.999016 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:57:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:57:59.893465703Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:00.828777802Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:58:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:00.848873152Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:58:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:01.904964639Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:03 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:03.916165039Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:05.853509675Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:58:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:05.927820339Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:07.941517192Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:09.956533935Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:11.516164149Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:16.516699960Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:16.517848283Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:16.518021763Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:16.873274 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:16.877402 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:16.879846 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:16.882166 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:16.884530 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:16 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:58:16.884630 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:58:17 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:17.917335 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:58:17 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:17.998426356Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:18.098320 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:58:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:18.107753 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:18.109838 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:18.122030 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:18.135435 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:18.148824 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:18.149680 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:58:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:18.740012 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 02:58:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:19.137309 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:58:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:20.012363134Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:21 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:21.518455483Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:58:21 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:21.537177351Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:58:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:22.026303644Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:22 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:58:22.667413 3222 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 02:58:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:24.041113081Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:26.056868760Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:26.542380520Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:58:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:28.069374142Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:30.081580978Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:32.144990881Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:32.348431026Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:58:32 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:58:32.357067 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:37.349221298Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:37.350158068Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:37.350316715Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:37.704453 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:37.708529 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:37.711708 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:37.713999 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:37.716216 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:37 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:58:37.716315 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:58:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:38.734480 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:58:38 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:38.937710 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.018397 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.212471 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.216028 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:39.221115813Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.241899 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.258580 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.272932 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.273049 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.355068 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:58:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:39.403866 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: the server is currently unable to handle the request (get runtimeclasses.node.k8s.io) | |
| Oct 29 02:58:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:41.246800984Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:42.351135610Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:58:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:42.368069386Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:58:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:43.259205924Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:45.269255219Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:47.279716235Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:47.372828324Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:58:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:49.326589561Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:51.336652415Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:53.369312849Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:58:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:54.095475027Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:58:54 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:58:54.103748 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:58:54 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:58:54.103749 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:59.095818742Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:59.097748669Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:58:59.097916112Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:59.472850 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:59.474780 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:59.477207 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:59.479166 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:58:59.481135 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:58:59 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:58:59.481227 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:00.353054 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:00.355317 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:00.368912 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:00.381827 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:00.402157 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:00.402867 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:00.411223291Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:00.517310 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:00.722678 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:01 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:01.072960 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:59:01 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:01.126685 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:01 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:01.930242 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:02.425283064Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:02 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:02.859622 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 02:59:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:04.100071246Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:59:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:04.119992111Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:59:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:04.440033625Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:06.455330620Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:08.472055341Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:09.124809576Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:59:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:10.481680488Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:12.493563703Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:14.546214192Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:15 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:15.763758813Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:59:15 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:59:15.773789 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:20.765666825Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:20.766568410Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:20.766728057Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:20.970517 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:20.974133 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:20.977672 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:20.980389 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:20.982329 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:20 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:59:20.982412 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.007387 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.009815 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.019692 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: the server is currently unable to handle the request (get runtimeclasses.node.k8s.io) | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.210744 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:22.602370942Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.617561 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.846156 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.848331 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.859393 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.870914 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.882880 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:22.882987 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:59:23 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:23.430551 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:24.616626557Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:25.766984300Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:59:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:25.786351802Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:59:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:26.626961901Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:28.638929165Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:30.654813647Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:30.791341531Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:59:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:32.665897196Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:34.678240779Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:36 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:59:36.676877 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:59:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:36.723487889Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:59:36 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:59:36.732517 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 02:59:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:41.724570136Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:59:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:41.725737996Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:59:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:41.725899328Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.479062 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.482217 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.485468 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.488669 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.490958 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: I1029 02:59:42.491052 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:42.772008442Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.936207 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.938230 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.959295 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.980190 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.994381 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 02:59:42 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:42.994501 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 02:59:43 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:43.517733 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:43 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:43.727993 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:44 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:44.131076 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 02:59:44 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:44.783997562Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:46.726804396Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 02:59:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:46.749245245Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:59:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:46.799738343Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:48.813801790Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:49 ubuntu-k3s-wkr1 k3s[3222]: E1029 02:59:49.789293 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 02:59:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:50.828479867Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:51.754631941Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 02:59:52 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:52.901496686Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:54.911764772Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:57.110795872Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 02:59:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T02:59:57.319961750Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 02:59:57 ubuntu-k3s-wkr1 k3s[3222]: W1029 02:59:57.329513 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:00:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:02.320804720Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:00:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:02.321721861Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:00:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:02.321872026Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:00:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:03.238665 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:03.242182 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:03.245667 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:03.248882 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:03 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:03.251123 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:03 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:00:03.251218 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.045699 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.048751 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.060834 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.077489 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.094106 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.094212 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:04.170421714Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.268649 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.471984 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.846324 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 03:00:04 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:04.874682 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:06.183852487Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:07.322326315Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:00:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:07.339779727Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:00:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:08.195485350Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:10.206634090Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:12.244285854Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:12.356274768Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:00:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:14.255206801Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:16.269101669Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:18.307400171Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:19.119825193Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:00:19 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:00:19.128280 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:00:19 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:00:19.128795 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:00:19 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:00:19.128796 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:00:19 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:00:19.129771 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:00:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:24.120732490Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:00:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:24.121766593Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:00:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:24.121933480Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:25.170224 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:25.172221 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:25.184616 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:25.197719 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:25.214114 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:00:25.214486 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:25.248197 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:25.347110012Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:25.451372 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:25 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:25.854207 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:26.143543 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:26.148641 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:26.168532 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:26.188028 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:26.202311 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:26.202416 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:00:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:26.670048 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:27 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:27.373629563Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:29.122570796Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:00:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:29.140777055Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:00:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:29.386707593Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:30 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:30.564999 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 03:00:31 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:31.400492004Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:33.414296394Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:34.148363975Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:00:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:35.534288256Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:37.545287518Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:39.641610290Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:40.023428424Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:45.023870914Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:45.024922442Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:45.025082552Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:45.771523 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:45.775146 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:45.777901 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:45.780727 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:45.783510 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:00:45 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:00:45.783629 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:00:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:46.249377 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:46.256392 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:46.268676 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:46.280189 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:46.291846 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:46.291936 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:00:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:46.677747184Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:00:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:46.801616 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:47.005159 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:47.081401 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: the server is currently unable to handle the request (get csidrivers.storage.k8s.io) | |
| Oct 29 03:00:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:00:47.408728 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:00:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:48.691674489Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:50.026163777Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:00:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:50.048439469Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:00:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:50.707792994Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:52 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:52.720716114Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:54.735874249Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:55.052368317Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:00:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:56.750974287Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:00:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:00:58.762911697Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:00.226112260Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:01:00 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:01:00.238521 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:01:00 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:01:00.238521 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:01:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:05.228347534Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:01:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:05.229266027Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:01:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:05.229421118Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.192809 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: the server is currently unable to handle the request (get nodes) | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.329154 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.331303 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.343027 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.357289 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.378054 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.378168 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.400126 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.552772 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.555062 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.566277 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.577748 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.588959 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:01:06.589320 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.599874 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:06.802733424Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:06 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:06.804748 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:07.209658 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:07 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:07.496919 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 03:01:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:08.818077317Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:10.230244316Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:01:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:10.250492469Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:01:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:10.828468910Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:12.840540595Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:14.852488922Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:15 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:15.255295897Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:01:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:16.865507753Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:18.876979293Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:20.779317990Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:01:20 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:01:20.784276 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:01:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:25.779786213Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:01:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:25.781442122Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:01:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:25.781614007Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:01:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:26.709996 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:26.714031 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:26.718104 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:26.720952 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:26 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:26.723234 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:26 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:01:26.723333 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:01:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:26.917928228Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:27.442562 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:27.451822 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:27.465299 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:27.488517 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:27.503025 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:27.503786 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:01:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:27.745403 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:27 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:27.948308 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:28 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:28.350864 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:28.943200930Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:30.782504271Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:01:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:30.800496669Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:01:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:30.957387152Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:32.971056316Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:34.988918190Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:35.804840499Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:01:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:37.007570960Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:39.018267267Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:41.102215765Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:41.483390963Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:01:41 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:01:41.493029 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:01:41 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:01:41.493029 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:01:41 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:01:41.493032 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:01:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:46.483578440Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:01:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:46.484779249Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:01:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:46.485022614Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.074530 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.078470 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.082012 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.083963 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.085994 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:01:47.086085 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.548021 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.550188 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.561520 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.580049 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.599515 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:47 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:47.599643 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:01:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:48.104864 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:48.142396242Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:01:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:48.233677 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 03:01:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:48.313458 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:48 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:48.716073 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:01:50 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:50.156468810Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:51.485630681Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:01:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:51.505390560Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:01:52 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:52.167726039Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:54.179282455Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:56.193598376Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:01:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:56.510706473Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:01:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:01:57.074574 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 03:01:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:01:58.282120273Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:00.295031870Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:02.352259095Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:02.964509065Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:02:02 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:02:02.979538 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:02:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:07.965343818Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:02:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:07.966518332Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:02:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:07.966688311Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:02:08 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:08.625798 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:08 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:08.629856 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:08 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:08.632713 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:08 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:08.635218 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:08 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:08.637962 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:08 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:02:08.638071 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:09.389479837Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:09.664196 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:09.678534 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:09.685431 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:09.712371 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:09.741164 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:09.774111 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:09.774803 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:02:09 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:09.901244 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:10 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:10.304635 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:11 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:11.401419497Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:12.967125880Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:02:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:12.988465431Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:02:13 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:13.420303538Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:15 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:15.432465648Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:17 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:17.445445680Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:17 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:17.995934903Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:02:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:19.460285056Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:21 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:21.116713 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": net/http: request canceled (Client.Timeout exceeded while awaiting headers) | |
| Oct 29 03:02:21 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:21.525697631Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:23.536984688Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:25.637013360Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:25.883400321Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:02:25 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:02:25.892316 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:02:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:30.884640218Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:02:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:30.885550942Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:02:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:30.885698235Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:02:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:31.767423 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:31.771584 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:31.775291 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:31.778061 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:31 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:31.780494 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:31 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:02:31.780600 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.147219 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: the server is currently unable to handle the request (get csidrivers.storage.k8s.io) | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.409888 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:32.696947208Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.796327 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.828303 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.831192 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.849078 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.861301 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.873729 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:32 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:32.873820 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:02:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:33.010707 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:33 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:33.413784 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:34 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:34.707622877Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:35.886549605Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:02:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:35.907742677Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:02:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:36.718285211Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:38.728468813Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:40 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:40.707715 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-proxy" cannot list resource "services" in API group "" at the cluster scope | |
| Oct 29 03:02:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:40.739354732Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:40.912151338Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:02:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:42.857102667Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:44 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:44.880951627Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:46 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:46.865700640Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:02:46 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:02:46.872019 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:02:46 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:02:46.872046 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:02:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:51.866082008Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:02:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:51.867090823Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:02:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:51.867236080Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.029747 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.033359 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.036769 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.039108 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.041299 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:02:52.041397 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:52.919038990Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.942041 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.944408 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.958547 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:52 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:52.983366 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:53 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:53.004464 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:02:53 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:53.005127 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:02:53 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:53.059613 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:53 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:53.265105 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:53 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:53.668133 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:54 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:02:54.471510 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:02:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:54.931420913Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:56.868002018Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:02:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:56.887799168Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:02:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:56.941791509Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:02:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:02:58.952456382Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:00.963956635Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:01.891407526Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:03:03 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:03.041132910Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:05.050936294Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:07.177343308Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:07.644967981Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:03:07 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:03:07.654473 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:03:07 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:03:07.654492 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:03:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:12.645760800Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:03:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:12.646744857Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:03:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:12.646898280Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:03:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:13.301449 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:13.305061 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:13.308131 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:13.310717 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:13 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:13.313277 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:13 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:03:13.313390 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.057349 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.059775 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.070551 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.081760 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.094665 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.094778 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:14.217777533Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.365640 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.570058 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:14 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:14.973308 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:16 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:16.233273240Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:17 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:17.648040621Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:03:17 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:17.668393127Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:03:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:18.248402431Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:20 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:20.260442780Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:22.242533 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-proxy" cannot list resource "services" in API group "" at the cluster scope | |
| Oct 29 03:03:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:22.273356035Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:22 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:03:22.666256 3222 sysinfo.go:203] Nodes topology is not available, providing CPU topology | |
| Oct 29 03:03:22 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:22.672699448Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:03:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:24.283732122Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:26 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:26.294435084Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:28.345637418Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:28.515520954Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:03:28 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:03:28.527200 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:03:28 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:03:28.527262 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:03:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:33.516845986Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:03:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:33.517895968Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:03:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:33.518068336Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:03:34 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:34.069254 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:34 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:34.073191 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:34 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:34.076782 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:34 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:34.078981 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:34 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:34.081133 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:34 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:03:34.081232 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:03:34 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:34.771600 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.099236 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.134658 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.136472 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.146231 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.157875 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.170191 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.170285 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.327681 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:35.407725838Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:35 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:35.730718 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:37 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:37.419200758Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:38.518671807Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:03:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:38.537093746Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:03:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:39.428479977Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:41.437928049Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:43.448179166Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:43.548344390Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:03:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:45.458009551Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:47.481033204Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:49.459424873Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:03:49 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:03:49.471467 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:03:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:54.460426559Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:03:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:54.461558374Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:03:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:54.461809778Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:03:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:55.108769 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:55.112618 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:55.115093 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:55.117470 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:55 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:55.119923 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:03:55 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:03:55.120031 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:03:55 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:55.522306372Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.171497 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.223499 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.234368 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.257396 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.270253 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.285223 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.285343 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.409411 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.570966 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: the server is currently unable to handle the request (get runtimeclasses.node.k8s.io) | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.714317 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 03:03:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:03:56.822247 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:03:57 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:57.535452825Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:03:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:59.462963185Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:03:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:59.478646733Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:03:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:03:59.546195331Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:01.557087329Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:03 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:03.569108859Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:04.483170726Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:04:05 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:05.609846357Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:07 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:07.637727162Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:09.555972909Z" level=error msg="Remotedialer proxy error" error="read tcp 192.168.86.91:60000->192.168.86.90:6443: read: connection reset by peer" | |
| Oct 29 03:04:09 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:04:09.563850 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:04:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:14.556445617Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:04:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:14.557507360Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:04:14 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:14.557658636Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:04:15 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:15.681406246Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:15.701214 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:15.703090 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:15.739398 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:15.773962 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:15.788658 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:15 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:04:15.789105 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:04:15 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:15.804098 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:16.006771 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:16.196732 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: the server is currently unable to handle the request (get nodes) | |
| Oct 29 03:04:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:16.343554 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:16.346072 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:16.366691 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:16.410136 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:16.433587 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:16 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:16.433696 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:04:17 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:17.696243411Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:19.558105573Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:04:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:19.583782291Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:04:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:19.709821564Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:21 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:21.723342483Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:22 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:22.102970 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 03:04:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:23.735757405Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:24 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:24.589025338Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:04:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:25.803767210Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:27 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:27.816491915Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:29.925232656Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:04:30.315009 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:04:30 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:30.351462033Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:04:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:04:30.352259 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:04:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:04:30.360656 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:04:30 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:04:30.360666 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:04:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:35.352343539Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:04:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:35.353392264Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:04:35 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:35.353679241Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.026593 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.030603 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.034248 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.036504 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.038716 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:04:36.038822 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.504754 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.508185 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.553670 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.567003 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.579473 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:36.579566 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:04:36 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:36.964282585Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:37.058154 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:37.260988 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:37 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:37.664476 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:38.978312714Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:40.354547633Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:04:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:40.373755181Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:04:40 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:40.988795561Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:42 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:42.999776064Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:45.011386851Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:45.378591372Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:04:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:47.028120392Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:49.042265263Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:04:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:51.088772842Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:04:51 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:04:51.095944 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:04:51 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:04:51.096230 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:56.090257761Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:56.091261320Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:56.091419670Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:56.870400 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:56.874532 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:56.878466 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:56.880742 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:56.883160 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:04:56 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:04:56.883268 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:04:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:57.412863 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: the server is currently unable to handle the request (get endpointslices.discovery.k8s.io) | |
| Oct 29 03:04:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:57.636748 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:57.638545 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:57.649259 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:57.660156 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:57.671483 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:57.671573 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:04:57 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:57.911812 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:58.117643 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:04:58 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:04:58.150666130Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:04:58 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:04:58.521229 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:00.173335761Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:01.092415898Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:05:01 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:01.112889585Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:05:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:02.184031851Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:04.194430880Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:06.122497129Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:05:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:06.217327775Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:08.228282850Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:10.239321394Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:12.479151945Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:13 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:13.059635635Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:18.061095783Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:18.062748871Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:18.062893499Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:18.808789 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:18.812225 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:18.815252 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:18.818338 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:18.821185 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:18 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:05:18.821284 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:19.513324456Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:19.719727 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:19.722061 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:19.737947 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:19.756371 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:19.771824 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:19.772874 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:19.855559 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:19 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:19.989978 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: the server is currently unable to handle the request (get services) | |
| Oct 29 03:05:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:20.057882 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:20.119525 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: the server is currently unable to handle the request (get runtimeclasses.node.k8s.io) | |
| Oct 29 03:05:20 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:20.460475 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:21 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:21.527126367Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:23.063438847Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:05:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:23.083960654Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:05:23 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:23.538405427Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:25 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:25.549354438Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:27 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:27.560688723Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:28 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:28.093431927Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:05:29 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:29.572119433Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:31 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:31.583556514Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:33 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:33.055068211Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:05:33 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:05:33.071298 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Node ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:05:33 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:05:33.071298 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:05:33 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:05:33.071345 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.CSIDriver ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:05:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:38.056244347Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:05:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:38.057276832Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:05:38 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:38.057424571Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.272302 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.274280 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.285538 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.296946 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.309802 3222 controller.go:187] failed to update lease, error: the server is currently unable to handle the request (put leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:05:39.310194 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.322256 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.525769 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:39.623376616Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.813098 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.822663 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.849091 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.872309 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.897559 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.897680 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:05:39 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:39.941620 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:40 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:40.744934 3222 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:05:41 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:41.634741062Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:43.057679535Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:05:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:43.118143492Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:05:43 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:43.646214652Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:45 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:45.656817619Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:46 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:46.497899 3222 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:kube-proxy" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope | |
| Oct 29 03:05:47 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:47.667974501Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:48 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:48.124024395Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:05:49 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:49.685550698Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:51 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:51.696265484Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:53 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:53.774422581Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:05:54 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:54.131996064Z" level=error msg="Remotedialer proxy error" error="websocket: close 1006 (abnormal closure): unexpected EOF" | |
| Oct 29 03:05:54 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:05:54.141660 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:05:54 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:05:54.141757 3222 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:05:54 ubuntu-k3s-wkr1 k3s[3222]: W1029 03:05:54.142242 3222 reflector.go:436] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: watch of *v1.Pod ended with: very short watch: k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:66: Unexpected watch close - watch lasted less than a second and no items received | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:59.133572959Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:59.134851849Z" level=error msg="Failed to connect to proxy" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:05:59.135018884Z" level=error msg="Remotedialer proxy error" error="dial tcp 192.168.86.90:6443: connect: connection refused" | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:59.558621 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:59.561861 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:59.564933 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:59.568184 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:05:59.570442 3222 controller.go:187] failed to update lease, error: Put "https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ubuntu-k3s-wkr1?timeout=10s": EOF | |
| Oct 29 03:05:59 ubuntu-k3s-wkr1 k3s[3222]: I1029 03:05:59.570543 3222 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease | |
| Oct 29 03:06:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:00.586626 3222 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:06:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:00.790546 3222 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:06:00 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:00.808757224Z" level=info msg="Waiting for node ubuntu-k3s-wkr1: the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:06:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:00.944854 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:06:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:00.950617 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:06:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:00.968006 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:06:00 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:00.991962 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:06:01 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:01.014910 3222 kubelet_node_status.go:470] "Error updating node status, will retry" err="error getting node \"ubuntu-k3s-wkr1\": the server is currently unable to handle the request (get nodes ubuntu-k3s-wkr1)" | |
| Oct 29 03:06:01 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:01.015012 3222 kubelet_node_status.go:457] "Unable to update node status" err="update node status exceeds retry count" | |
| Oct 29 03:06:01 ubuntu-k3s-wkr1 k3s[3222]: E1029 03:06:01.205247 3222 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: the server is currently unable to handle the request (get leases.coordination.k8s.io ubuntu-k3s-wkr1) | |
| Oct 29 03:06:02 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:02.821381966Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:06:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:04.135980307Z" level=info msg="Connecting to proxy" url="wss://192.168.86.90:6443/v1-k3s/connect" | |
| Oct 29 03:06:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:04.155471030Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:06:04 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:04.831585324Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:06:06 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:06.842062207Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:06:08 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:08.893217822Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:06:09 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:09.160177161Z" level=warning msg="Unable to watch for tunnel endpoints: unknown (get endpoints)" | |
| Oct 29 03:06:10 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:10.905846528Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" | |
| Oct 29 03:06:12 ubuntu-k3s-wkr1 k3s[3222]: time="2021-10-29T03:06:12.916518842Z" level=info msg="Waiting for node ubuntu-k3s-wkr1 CIDR not assigned yet" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment