Skip to content

Instantly share code, notes, and snippets.

@damien-roche
Created March 22, 2022 14:13
Show Gist options
  • Save damien-roche/62b5569b367d5101be759fc22c8c4de2 to your computer and use it in GitHub Desktop.
Save damien-roche/62b5569b367d5101be759fc22c8c4de2 to your computer and use it in GitHub Desktop.
Logstash grok for Heroku Postgres log sample

Given:

source=HEROKU_POSTGRESQL_EG addon=postgresql-perp-123 sample#current_transaction=541707859 sample#db_size=361264468527bytes sample#tables=50 sample#active-connections=80 sample#waiting-connections=0 sample#index-cache-hit-rate=0.99635 sample#table-cache-hit-rate=0.98647 sample#load-avg-1m=0.33 sample#load-avg-5m=0.355 sample#load-avg-15m=0.33 sample#read-iops=79.907 sample#write-iops=2.8729 sample#tmp-disk-used=609959936 sample#tmp-disk-available=72368832512 sample#memory-total=15612012kB sample#memory-free=128844kB sample#memory-cached=14176972kB sample#memory-postgres=334364kB sample#wal-percentage-used=0.06676170938635338

Pattern:

source=%{WORD:source} addon=%{NOTSPACE:addon} sample#current_transaction=%{NUMBER:current_transaction_sample} sample#db_size=%{NUMBER:db_size_bytes}bytes sample#tables=%{NUMBER:tables} sample#active-connections=%{NUMBER:connections} sample#waiting-connections=%{NUMBER:waiting_connections} sample#index-cache-hit-rate=%{NUMBER:index_cache_hit_rate} sample#table-cache-hit-rate=%{NUMBER:table_cache_hit_rate} sample#load-avg-1m=%{NUMBER:load_avg_1m} sample#load-avg-5m=%{NUMBER:load_avg_5m} sample#load-avg-15m=%{NUMBER:load_avg_15m} sample#read-iops=%{NUMBER:read_iops} sample#write-iops=%{NUMBER:write_iops} sample#tmp-disk-used=%{NUMBER:tmp_disk_used_bytes} sample#tmp-disk-available=%{NUMBER:tmp_disk_available_bytes} sample#memory-total=%{NUMBER:memory_total_kb}kB sample#memory-free=%{NUMBER:memory_free_kb}kB sample#memory-cached=%{NUMBER:memory_cached_kb}kB sample#memory-postgres=%{NUMBER:memory_postgres_kb}kB sample#wal-percentage-used=%{NUMBER:wal_percentage_used}

Result:

{
  "memory_free_kb": "128844",
  "load_avg_5m": "0.355",
  "addon": "postgresql-perp-123",
  "memory_postgres_kb": "334364",
  "source": "HEROKU_POSTGRESQL_EG",
  "tables": "50",
  "write_iops": "2.8729",
  "tmp_disk_used_bytes": "609959936",
  "waiting_connections": "0",
  "connections": "80",
  "index_cache_hit_rate": "0.99635",
  "load_avg_15m": "0.33",
  "current_transaction_sample": "541707859",
  "read_iops": "79.907",
  "tmp_disk_available_bytes": "72368832512",
  "db_size_bytes": "361264468527",
  "table_cache_hit_rate": "0.98647",
  "memory_total_kb": "15612012",
  "load_avg_1m": "0.33",
  "memory_cached_kb": "14176972",
  "wal_percentage_used": "0.06676170938635338"
}

Test:

https://grokdebug.herokuapp.com/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment