portante’s gists

portante / watch-fluentd-memory

Created March 21, 2018 13:57

Show fluentd pods using more than 384 MB of memory.

	#!/bin/bash
	let num=0
	for pod in $(oc get pod -n logging -l component=fluentd -o name); do
	rss=$(oc exec -n logging $(basename $pod) -- ps auxww \| grep fluentd \| tail -n 1 \| awk '{ print $6 }')
	if [ "$rss" -gt 384000 ]; then
	let num=num+1
	echo "$pod $rss"
	fi
	done
	echo $num

portante / pre-create-new-indices.sh

Last active April 8, 2018 21:44

Very hacky way to avoid index creation becoming a bottle neck in OpenShift Aggregated Logging when we have large numbers of indices.

	#!/bin/bash

	function finish {
	rm -rf $TMPDIR
	}
	trap finish EXIT
	TMPDIR=$(mktemp -d)

	#TODAY='2018.03.27'
	TODAY=$(date "+%Y.%m.%d")

portante / shard-allocation.sh

Last active March 9, 2018 22:09

Enable/disable shard allocation in an OpenShift Elasticsearch cluster.

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_put='curl -s -X PUT --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	date
	$curl_put $ES_URL/_cluster/settings?pretty -d "{ \"transient\" : { \"cluster.routing.allocation.enable\" : \"$1\" } }"

portante / watch-fluentd-queues.sh

Last active September 12, 2019 16:22

A simple bash script to watch all of the running fluentd pods, gathering and displaying those with on-disk queue file counts greater than two (2) (long queues), or with queue files older than 2 minutes. This script relies on pssh (https://github.com/lilydjwg/pssh) to work efficiently.

	#!/bin/bash
	function finish {
	rm -rf $TMPDIR
	}
	trap finish EXIT
	TMPDIR=$(mktemp -d)
	mkdir $TMPDIR/output

	let num=0
	let slow=0

portante / dump-shards.sh

Created March 8, 2018 00:47

Dump the current state of all the index shards in an ES cluster under OpenShift.

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_get='curl -s -X GET --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	date
	$curl_get $ES_URL/_cat/shards?v\&bytes=b

portante / move-shard.sh

Created March 7, 2018 21:33

A simple script to move a replica shard from one node to another in an OpenShift ES cluster.

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_post='curl -s -X POST --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	date
	$curl_post $ES_URL/_cluster/reroute?pretty=true -d "{ \"commands\" : [ { \"move\" : { \"index\" : \"$1\", \"shard\" : $2, \"from_node\" : \"$3\", \"to_node\" : \"$4\" } } ] }"

portante / Dockerfile

Last active February 15, 2018 20:39

A simple systemd journal client that reads *all* messages, calculating a rate for the "LogSmasher" messages it is looking for, both for a data size interval of the messages we are looking for and for all ignored messages.

	FROM registry.fedoraproject.org/fedora
	RUN dnf -y install python
	COPY ./app.py /logsmasher
	RUN chmod +x /logsmasher
	#CMD ["/logsmasher"]

portante / app-reader.py

Created February 15, 2018 20:22

A simple systemd journal client that reads *all* messages, calculating a rate for the "LogSmasher" messages it is looking for, both for a data size interval of the messages we are looking for and for all ignored messages.

	#!/usr/bin/env python
	import sys
	import os
	import time
	import datetime
	from systemd import journal

	sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

	j = journal.Reader()

portante / view-kibana-logs.py

Last active December 11, 2018 21:51

Simple tool to interpret kibana logs under openshift via, "oc logs -c kibana <pod> | ./view-kibana-logs.py"

	#!/usr/bin/env python
	# oc logs -c kibana <pod> \| ./view-kibana-logs.py

	import sys
	import json
	import fileinput

	lines = 0
	valerrors = 0
	successes = 0

portante / sum-es-indices.awk

Last active June 22, 2018 16:03

Simple awk script to calculate totals # of logs and total size of logs for "project" and "operations" indices from a /_cat/indices list retrieved from an OpenShift cluster with Aggregated Logging installed.

	#!/usr/bin/awk -f

	BEGIN {
	if (length(DAY) > 0) {
	gensub(/\./, "\\\\.", DAY)
	dayfmt = " (" DAY ")"
	} else {
	dayfmt = ""
	}
	projects = "project\\..+\\." DAY

Peter Portante portante