canhnt

Container killed by OOM

When a container exit with code 137, it means the process is killed by OOM Killer

docker container ls -a
"task: non-zero exit (137)"

Check OOM in journal log

journalctl -k | grep -i -e memory -e oom

canhnt

Tips & tricks on k8s

Get leader

kubectl get --raw=/api/v1/namespaces/kube-system/endpoints/kube-controller-manager | jq .metadata.annotations

For kubectl via Rancher:

canhnt

Keybase proof

I hereby claim:

• I am canhnt on github.
• I am cngo (https://keybase.io/cngo) on keybase.
• I have a public key ASDceW9Aeeknx07KzDaju7gKF6jnOVHYswzSrkcG59jV7go

To claim this, I am signing this object:

canhnt

Access K8s cluster directly

Get kubeconfig from secret

CLUSTERID='c-1234'
kubectl get secret c-$CLUSTERID -n cattle-system -o json | jq -r .data.cluster | base64 -D | jq -r .metadata.state > kubeconfig-direct

Change endpoint to https://localhost:6443

canhnt

Grok pattern for glog

E%{MONTHNUM:month}%{MONTHDAY:day} %{TIME}\s*%{NUMBER:threadid} (?<file_source>%{WORD}\.%{WORD}):(?<file_line>%{NUMBER})\] (?<message>.*)

Example:

E0423 11:41:45.456532       1 reconcile.go:30] Upsert role 'xxx' failed: Failed to upsert ES role, expect StatusCode=200, but has 400

canhnt

```
awk 'function hextodec(str,ret,n,i,k,c){
    ret = 0
    n = length(str)
    for (i = 1; i <= n; i++) {
        c = tolower(substr(str, i, 1))
        k = index("123456789abcdef", c)
        ret = ret * 16 + k
    }
    return ret