Clayton Coleman smarterclayton
smarterclayton
/ gist:fd22a00423115d19aef049d6336ef918
Last active
October 31, 2017 14:55
heavy string allocation in iptables code on 3.7 node
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ### Not that busy node | |
| Showing nodes accounting for 51472970, 64.09% of 80315852 total | |
| Dropped 1872 nodes (cum <= 401579) | |
| Showing top 30 nodes out of 277 | |
| flat flat% sum% cum cum% | |
| 0 0% 0% 80185072 99.84% runtime.goexit /usr/lib/golang/src/runtime/asm_amd64.s | |
| 39878766 49.65% 49.65% 39878766 49.65% runtime.rawstringtmp /usr/lib/golang/src/runtime/string.go | |
| 0 0% 49.65% 35614612 44.34% runtime.slicebytetostring /usr/lib/golang/src/runtime/string.go | |
| 0 0% 49.65% 22676400 28.23% github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/proxy/iptables.(*Proxier).syncProxyRules /builddir/build/BUILD/atomic-openshift-git-0.ee33a8f/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/proxy/iptables/proxier.go |
smarterclayton
/ gist:e67d53f89b0a93a1f61a947eb05980dd
Created
August 24, 2017 17:29
node-config.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| allowDisabledDocker: false | |
| apiVersion: v1 | |
| authConfig: | |
| authenticationCacheSize: 1000 | |
| authenticationCacheTTL: 5m | |
| authorizationCacheSize: 1000 | |
| authorizationCacheTTL: 5m | |
| dnsBindAddress: 10.192.208.5:53 | |
| dnsDomain: cluster.local | |
| dnsIP: 10.192.208.5 |
smarterclayton
/ gist:78691c6cd2211755a17186ba1a38a8dc
Created
August 9, 2017 16:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| - hosts: localhost | |
| tasks: | |
| - stat: path=/usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/std_include.yml | |
| register: std_include | |
| become: no | |
| - include: /usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/std_include.yml | |
| when: hostvars['localhost']['std_include'] is defined and hostvars['localhost']['std_include'].exists |
smarterclayton
/ gist:665c39e9eba30356c0d2b4d2b230994f
Created
August 3, 2017 20:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| TASK [Gathering Facts] ********************************************************* | |
| Thursday 03 August 2017 20:18:52 +0000 (0:00:00.141) 0:03:56.731 ******* | |
| fatal: [ci-claytontest-ig-n-q7zh]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Warning: Permanently added '104.154.198.205' (ECDSA) to the list of known hosts.\r\nTraceback (most recent call last):\n File \"<stdin>\", line 136, in <module>\nNameError: name 'temp_path' is not defined\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1} | |
| fatal: [ci-claytontest-ig-n-d06s]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Warning: Permanently added '35.188.123.22' (ECDSA) to the list of known hosts.\r\nTraceback (most recent call last):\n File \"<stdin>\", line 136, in <module>\nNameError: name 'temp_path' is not defined\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1} | |
| fatal: [ci-claytontest-ig-n-l7s2]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Warning: Permanently added '146.148.10 |
smarterclayton
/ gist:894d4991f7743f52acd0179fca618752
Created
July 27, 2017 19:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ○ oc get --raw /metrics | grep grpc | |
| # HELP etcd_network_client_grpc_received_bytes_total The total number of bytes received from grpc clients. | |
| # TYPE etcd_network_client_grpc_received_bytes_total counter | |
| etcd_network_client_grpc_received_bytes_total 57586 | |
| # HELP etcd_network_client_grpc_sent_bytes_total The total number of bytes sent to grpc clients. | |
| # TYPE etcd_network_client_grpc_sent_bytes_total counter | |
| etcd_network_client_grpc_sent_bytes_total 1.7420058e+07 | |
| # HELP grpc_client_handled_total Total number of RPCs completed by the client, regardless of success or failure. | |
| # TYPE grpc_client_handled_total counter | |
| grpc_client_handled_total{grpc_code="OK",grpc_method="LeaseGrant",grpc_service="etcdserverpb.Lease",grpc_type="unary"} 2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| https://prometheus-openshift-devops-monitor.d800.free-int.openshiftapps.com/graph?g0.range_input=1h&g0.expr=container_memory_rss%7Bcontainer_name%3D%22prometheus%22%7D&g0.tab=0&g1.range_input=1h&g1.expr=drop_common_labels(sum+without+(instance)+(rate(apiserver_request_count%7Bcode!~'2%5C%5Cd%5C%5Cd'%7D%5B3m%5D)))&g1.tab=0&g2.range_input=1h&g2.expr=sum+by+(gitVersion)+(openshift_build_info)&g2.tab=0&g3.range_input=1h&g3.expr=drop_common_labels(sort_desc(sum+without+(cpu)+(rate(container_cpu_usage_seconds_total%5B5m%5D)))+%3E+0.1)&g3.tab=0&g4.range_input=1h&g4.expr=sum+without+(instance)+(rate(kubelet_docker_operations_errors%5B5m%5D))+%3E+0&g4.tab=0&g5.range_input=1h&g5.expr=drop_common_labels(sort_desc(sum+without+(instance)+(rate(apiserver_request_count%7Bverb%3D~%22PUT%7CPOST%7CPATCH%7CDELETE%22%7D%5B5m%5D))))&g5.tab=0 |
smarterclayton
/ gist:e2e8609ad0c62882d93ccf7a4bf3c496
Last active
June 7, 2017 21:12
endpoints ugly
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (pprof) top30 | |
| 3.47s of 7.15s total (48.53%) | |
| Dropped 416 nodes (cum <= 0.04s) | |
| Showing top 30 nodes out of 248 (cum >= 0.85s) | |
| flat flat% sum% cum cum% | |
| 0 0% 0% 7.04s 98.46% runtime.goexit | |
| 3.43s 47.97% 47.97% 3.44s 48.11% syscall.Syscall | |
| 0 0% 47.97% 2.38s 33.29% io/ioutil.ReadDir | |
| 0 0% 47.97% 2.20s 30.77% os.(*File).Readdir | |
| 0 0% 47.97% 2.20s 30.77% os.(*File).readdir |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| https://prometheus-openshift-devops-monitor.44fs.preview.openshiftapps.com/graph?g0.range_input=30m&g0.expr=up&g0.tab=0&g1.range_input=30m&g1.expr=sum+without+(instance)+(rate(apiserver_request_count%7Bcode!~%222.%3F.%3F%22%7D%5B5m%5D)+%3E+0)&g1.tab=0&g2.range_input=30m&g2.expr=sum+without+(kubernetes_io_hostname%2Cinstance%2Cnew_node)+(rate(kubelet_docker_operations_errors%5B5m%5D)+%3E+0)&g2.tab=0&g3.range_input=1h&g3.expr=container_memory_rss%7Bcontainer_name%3D%22prometheus%22%7D&g3.tab=0&g4.range_input=1h&g4.expr=count+by+(gitVersion)+(openshift_build_info)&g4.tab=0 |
smarterclayton
/ gist:53e56b0adfc0b862afc0239e6df9377f
Created
May 25, 2017 15:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| === RUN TestCreateSetsFields | |
| 2017-05-25 14:25:57.320097 I | integration: launching 1483262971511501821 (unix://localhost:14832629715115018210) | |
| unexpected fault address 0xc420b40000 | |
| fatal error: fault | |
| [signal SIGSEGV: segmentation violation code=0x1 addr=0xc420b40000 pc=0x1b6edd5] | |
| goroutine 211 [running]: | |
| runtime.throw(0x24d1860, 0x5) | |
| /usr/local/go/src/runtime/panic.go:566 +0x95 fp=0xc4205eb138 sp=0xc4205eb118 | |
| runtime.sigpanic() |
smarterclayton
/ gist:d8d29c28be5d33a499c194374dfab89b
Created
May 22, 2017 15:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var watchEvent watch.Event | |
| switch { | |
| case curObjPasses && !oldObjPasses: | |
| object, err := c.copier.Copy(event.Object) | |
| if err != nil { | |
| utilruntime.HandleError(fmt.Errorf("unexpected copy error: %v", err)) | |
| return | |
| } | |
| watchEvent = watch.Event{Type: watch.Added, Object: object} | |
| case curObjPasses && oldObjPasses: |