Skip to content

Instantly share code, notes, and snippets.

@portante
Last active June 22, 2018 16:03
Show Gist options
  • Save portante/7ccf04f4da41c72352b15b241254a9f7 to your computer and use it in GitHub Desktop.
Save portante/7ccf04f4da41c72352b15b241254a9f7 to your computer and use it in GitHub Desktop.
Simple awk script to calculate totals # of logs and total size of logs for "project" and "operations" indices from a /_cat/indices list retrieved from an OpenShift cluster with Aggregated Logging installed.
#!/usr/bin/awk -f
BEGIN {
if (length(DAY) > 0) {
gensub(/\./, "\\\\.", DAY)
dayfmt = " (" DAY ")"
} else {
dayfmt = ""
}
projects = "project\\..+\\." DAY
operations = "\\.operations\\." DAY
}
# Total the number of all logs
{ all_logs += $6 }
# Total # of logs for Apps
$3 ~ projects { app_logs += $6}
# Total # of logs for Ops:
$3 ~ operations { ops_logs += $6}
# Total size of logs (unreplicated):
{ all_logs_size += $9 }
# Total size of logs for Apps (unreplicated):
$3 ~ projects { app_logs_size += $9 }
# Total size of logs for Ops (unreplicated):
$3 ~ operations { ops_logs_size += $9 }
END {
if (length(SIMPLE) > 0) {
printf "%d %s %d %.2f %d %.2f %d %s %d %.2f %d %.2f\n", all_logs, dayfmt, app_logs, 100*(app_logs/all_logs), ops_logs, 100*(ops_logs/all_logs), all_logs_size, dayfmt, app_logs_size, 100*(app_logs_size/all_logs_size), ops_logs_size, 100*(ops_logs_size/all_logs_size)
} else {
printf "Total count of logs: %d\nTotal count of application logs%s: %d (%.2f)\nTotal count of operational logs%s: %d (%.2f)\nTotal size of all logs: %d\nTotal size of application logs%s: %d (%.2f)\nTotal size of operational logs%s: %d (%.2f)\n", all_logs, dayfmt, app_logs, 100*(app_logs/all_logs), dayfmt, ops_logs, 100*(ops_logs/all_logs), all_logs_size, dayfmt, app_logs_size, 100*(app_logs_size/all_logs_size), dayfmt, ops_logs_size, 100*(ops_logs_size/all_logs_size)
}
}
@portante
Copy link
Author

portante commented Jun 5, 2018

Use SIMPLE=1 to get a one line summary output:

# ./sum-es-indices.awk -v DAY=2018.06.03 -v SIMPLE=1 indices.2018-06-05.lis
486312696  (2018.06.03) 33101698 6.81 24223805 4.98 629301  (2018.06.03) 39001 6.20 22209 3.53

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment