This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member | |
SuperQ commented on 23 Jul 2018 • | |
One additional debugging option, get a dump of the goroutines with http://localhost:9100/debug/pprof/goroutine?debug=1. Normally it would be only a few, like this. | |
@juliusv | |
Member | |
juliusv commented on 23 Jul 2018 | |
@SuperQ Yeah, I would recommend listing the active goroutines, as we already know it's connected to a goroutine leak. But prefer debug=2 over debug=1, so: http://localhost:9100/debug/pprof/goroutine?debug=2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For anyone who hits this, it's caused by api-services that no longer have backends running... | |
In my case it was KEDA, but there are a number of different services that install aggregated API servers. | |
To fix it: | |
kubectl get apiservice | |
Look for ones the AVAILABLE is False | |
If you don't need those APIs any more, delete them: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
echo "HTTP/1.1 200 OK | |
Content-Type: text/html; charset=UTF-8 | |
Server: netcat! | |
<!doctype html> | |
<html><body><h1>`hostname`</h1></body></html>" > index.http && while true; do cat index.http | nc -l -p 80; done |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If you want installable .deb package instead of putting everything to /usr/local, do this: | |
sudo apt-get build-dep curl | |
sudo apt-get install libc-ares-dev build-essential | |
apt-get source curl | |
cd curl-* | |
This will download curl sources with Debian/Ubuntu build files and patches. | |
Edit file debian/control: add line libc-ares-dev to Build-Depends |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ export DOCKER_REGISTRY=flwang | |
$ export HYPERKUBE_VERSION=v1.16.9 | |
$ export KUBE_GIT_TREE_STATE="clean" | |
$ bash build/run.sh make cross KUBE_FASTBUILD=true ARCH=amd64 && cd cluster/images/hyperkube/ && make VERSION=$HYPERKUBE_VERSION ARCH=amd64 REGISTRY=$DOCKER_REGISTRY && docker push $DOCKER_REGISTRY/hyperkube-amd64:$HYPERKUBE_VERSION | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openstack coe cluster template list --fields public,hidden,network_driver -f json | jq -c '.[] | select(.public==true and .hidden==false)' | |
glance --os-image-api-version 1 image-create --disk-format raw --container-format bare --file fedora-coreos-31.20200505.3.0-openstack.x86_64.raw --min-disk 10 --size 10737418240 --name fedora-coreos-31-x86_64 --is-public=True |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openstack coe cluster template create cc-v1.17.4 \ | |
--keypair feilong \ | |
--flavor ds1G \ | |
--master-flavor ds2G \ | |
--coe kubernetes \ | |
--external-network e0943db8-4e34-4ef0-8973-93ef72a735e4 \ | |
--network-driver calico \ | |
--docker-storage-driver=overlay2 \ | |
--image=0529c960-92cf-4913-b079-4b7b2984ef4a \ | |
--volume-driver=cinder \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kubectl port-forward prometheus-prometheus-prometheus-0 -n kube-system 9090:9090 | |
kubectl port-forward prometheus-operator-grafana-597f5fdffc-q5c7m -n kube-system 3000:3000 | |
kubectl delete apiservice v1beta1.custom.metrics.k8s.io |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
set-title(){ | |
ORIG=$PS1 | |
TITLE="\e]2;$@\a" | |
PS1=${ORIG}${TITLE} | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
--- | |
apiVersion: apps/v1 | |
kind: DaemonSet | |
metadata: | |
name: debug-app | |
namespace: kube-system | |
labels: | |
k8s-app: debug-app | |
spec: | |
selector: |
NewerOlder