Skip to content

Instantly share code, notes, and snippets.

View grobie's full-sized avatar

Tobias Schmidt grobie

291 followers · 50 following
View GitHub Profile
@grobie
grobie / prometheus.rules
Created November 9, 2017 14:05
Prometheus alerts (in 1.x format) to monitor a Prometheus 2.0 server
ALERT PrometheusTSDBReloadsFailing
IF increase(prometheus_tsdb_reloads_failures_total[4h]) > 0
FOR 1d
LABELS {
system = "prometheus",
severity = "warning",
}
ANNOTATIONS {
summary = "Prometheus has issues reloading data blocks from disk",
description = "{{$labels.job}} at {{$labels.instance}} had {{$value | humanize}} reload failures over the last four hours.",
@grobie
grobie / Rules
Created March 18, 2017 19:07
Nanoc4 cache buster
# Rules
# Write asset files to new location including file hash. Also rewrite paths in CSS/text files.
compile '/assets/*' do
unless item.binary?
filter :cache_buster, directory: '/assets/'
end
write CacheBuster.path(item)
end
@grobie
grobie / prombench-ideas.md
Last active September 16, 2016 04:14
prombench ideas

exporters and processes

  • a bunch of exporters in go
  • option to start an external command
  • option to just scrape a specified address
  • option to run a Docker container somwhere (also for the prometheus server)
  • super fancy: Kubernetes integration to start jobs/processes (we have credits from Google to run stuff on their infrastructure)

tests

@grobie
grobie / mtail.log
Last active May 5, 2016 05:59
go version go1.6 linux/amd64 commit 6e0eac2
I0505 01:58:09.214256 29528 loader.go:107] Loaded program counter.mtail
I0505 01:58:09.214386 29528 vm.go:556] Starting program counter.mtail
I0505 01:58:09.214390 29528 tail.go:303] Tailing /home/grobie/code/go/src/github.com/google/mtail/test/logs/current
I0505 01:58:09.214575 29528 mtail.go:224] Listening on port 3903
I0505 01:58:15.582606 29528 tail.go:323] Unexpected event {"/home/grobie/code/go/src/github.com/google/mtail/test/logs/current"}
I0505 01:58:15.582756 29528 tail.go:210] Failed removing watches on/home/grobie/code/go/src/github.com/google/mtail/test/logs/current
I0505 01:58:15.582806 29528 tail.go:303] Tailing /home/grobie/code/go/src/github.com/google/mtail/test/logs/current
I0505 01:58:15.582825 29528 tail.go:323] Unexpected event {"/home/grobie/code/go/src/github.com/google/mtail/test/logs/current"}
I0505 01:58:15.901162 29528 tail.go:323] Unexpected event {"/home/grobie/code/go/src/github.com/google/mtail/test/logs/current"}
I0505 01:58:15.901219 29528 tail.go:210] Fai
@grobie
grobie / mtail.log
Created May 3, 2016 04:38
I0503 00:38:01.570728 17097 loader.go:107] Loaded program counter.mtail
I0503 00:38:01.570846 17097 tail.go:290] Tailing /home/grobie/code/go/src/github.com/google/mtail/test/logs/current
I0503 00:38:01.570978 17097 mtail.go:220] Listening on port 3903
I0503 00:38:01.571088 17097 vm.go:538] Starting program counter.mtail
I0503 00:38:04.437907 17097 tail.go:310] Unexpected event {"/home/grobie/code/go/src/github.com/google/mtail/test/logs/current"}
I0503 00:38:04.438229 17097 tail.go:209] Failed removing watches on/home/grobie/code/go/src/github.com/google/mtail/test/logs/current
I0503 00:38:04.438239 17097 tail.go:212] Seek to start on /home/grobie/code/go/src/github.com/google/mtail/test/logs/current
I0503 00:38:04.438272 17097 tail.go:290] Tailing /home/grobie/code/go/src/github.com/google/mtail/test/logs/current
I0503 00:38:04.438288 17097 tail.go:310] Unexpected event {"/home/grobie/code/go/src/github.com/google/mtail/test/logs/current"}
I0503 00:38:04.735764 17097 tail.go:310] Unexpec
@grobie
grobie / consul_collector.go
Last active April 26, 2016 01:00
package main
import (
"bufio"
"bytes"
"fmt"
"os/exec"
"strconv"
"strings"
@grobie
grobie / prometheus-downloads.txt
Created April 16, 2016 08:14
prometheus-0.13.3.linux-amd64.tar.gz: 539
prometheus-0.16.1.darwin-amd64.tar.gz: 700
prometheus-0.14.0.linux-amd64.tar.gz: 755
prometheus-0.16.0rc2.linux-amd64.tar.gz: 791
prometheus-0.16.0.linux-amd64.tar.gz: 1822
prometheus-0.16.2.linux-amd64.tar.gz: 2394
prometheus-0.17.0rc2.linux-amd64.tar.gz: 2561
prometheus-0.17.0.linux-amd64.tar.gz: 3188
prometheus-0.15.1.linux-amd64.tar.gz: 3419
prometheus-0.16.1.linux-amd64.tar.gz: 66395
@grobie
grobie / prometheus-node-metrics.json
Created April 16, 2016 00:09
{
"id": 138,
"title": "Prometheus node metrics",
"originalTitle": "Prometheus node metrics",
"tags": [
"prometheus",
"node"
],
"style": "dark",
"timezone": "",
@grobie
grobie / 4234.dashboard
Created March 3, 2016 04:04
{
"id": 2,
"title": "#4234",
"originalTitle": "#4234",
"tags": [],
"style": "dark",
"timezone": "browser",
"editable": true,
"hideControls": false,
"sharedCrosshair": false,
@grobie
grobie / consul.log
Created July 15, 2015 23:14
consul vote issues
2015-07-15_23:10:51.63143 2015/07/15 23:10:51 [WARN] raft: Heartbeat timeout reached, starting election
2015-07-15_23:10:51.63146 2015/07/15 23:10:51 [INFO] raft: Node at 10.1.2.3:5365 [Candidate] entering Candidate state
2015-07-15_23:10:51.63294 2015/07/15 23:10:51 [ERR] raft: Failed to make RequestVote RPC to 10.5.6.7:5365: dial tcp 10.5.6.7:5365: connection refused
2015-07-15_23:10:51.63665 2015/07/15 23:10:51 [INFO] raft: Node at 10.1.2.3:5365 [Follower] entering Follower state
2015-07-15_23:10:53.15118 2015/07/15 23:10:53 [WARN] raft: Heartbeat timeout reached, starting election
2015-07-15_23:10:53.15120 2015/07/15 23:10:53 [INFO] raft: Node at 10.1.2.3:5365 [Candidate] entering Candidate state
2015-07-15_23:10:53.15249 2015/07/15 23:10:53 [ERR] raft: Failed to make RequestVote RPC to 10.5.6.7:5365: dial tcp 10.5.6.7:5365: connection refused
2015-07-15_23:10:54.87441 2015/07/15 23:10:54 [WARN] raft: Election timeout reached, restarting election
2015-07-15_23:10:54.87443