This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| set -x | |
| echo "grafana is in http://127.0.0.1:7000/" | |
| echo "use login admin" | |
| echo "use password $(kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.adminPassword}" | base64 --decode)" | |
| kubectl -n monitoring port-forward service/prometheus-operator-grafana 7000:80 | |
| #echo "prometheus is in http://127.0.0.1:9090/" | |
| #kubectl -n monitoring port-forward service/prometheus-operator-prometheus 9090:9090 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| set -ue | |
| PGPASSWORD=$(kubectl get secret -n postgres pg-su -o jsonpath="{.data.password}" | base64 --decode) | |
| PGUSER=$(kubectl get secret -n postgres pg-su -o jsonpath="{.data.username}" | base64 --decode) | |
| BASH_CMD="PGPASSWORD=${PGPASSWORD} psql -h pg-stolon-proxy.postgres -U ${PGUSER} -d api_prod" | |
| kubectl -n postgres exec -ti pg-stolon-keeper-0 -- bash -c "${BASH_CMD}" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import React, { createRef } from "react" | |
| import throttle from "lodash/throttle" | |
| // Import styled components (see the Styles section below). | |
| import { | |
| TocDiv, | |
| TocLink, | |
| TocIcon, | |
| TocTitle, | |
| TocToggleOpener, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| FROM python:alpine | |
| RUN pip install --no-cache-dir awscli | |
| RUN apk update && apk add postgresql | |
| COPY dumpDb.sh . | |
| ENTRYPOINT [ "/bin/sh" ] | |
| CMD [ "./dumpDb.sh" ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Action item | Applicability to other incidents | Effectiveness for this incident | Verdict | |
|---|---|---|---|---|
| Fine-tune alerts | Low | High | Do: low-cost | |
| Fix the segfault bug | Low | High | Do: root cause | |
| Database timeout automated tests | Medium-low | High | Do: hygiene | |
| Retry circuit breaker or retry budget | High | High | Implement retry budget: key action item | |
| Circuit breaker | High | Medium | Not an action item for now: will take in the future | |
| Deadline propagation | High | Medium-high | Start as a long-term project |
OlderNewer