Skip to content

Instantly share code, notes, and snippets.

View jcantrill's full-sized avatar

Jeff Cantrill jcantrill

View GitHub Profile
@portante
portante / format-es-logs
Last active January 11, 2018 14:11
Hacky set of python and bash scripts to format output of ES _search query output using the "scroll" API (see https://www.elastic.co/guide/en/elasticsearch/reference/2.4/search-request-scroll.html)
#!/usr/bin/env python
# Use via: curl -X GET http://localhost:9200/<index>/_search/?fields=@timestamp,level,hostname,kubernetes.host,kubernetes.pod_name,kubernetes.container_name,message&q=message:<string>
import sys, os, json
if len(sys.argv) == 1 or sys.argv[1] == '-':
json_doc = json.load(sys.stdin)
else:
json_doc = json.load(open(sys.argv[1]))
@jcantrill
jcantrill / gist:56c2e5f036113667bae185241fa4f035
Last active June 8, 2017 19:13
Origin v1.5.1 inventory file
$ cat ../jctoolbox/openshift/origin-15.inventory
[OSEv3:children]
masters
nodes
[OSEv3:vars]
ansible_become=true
ansible_ssh_user=vagrant
ansible_ssh_private_key_file=/home/jeff.cantrill/.ssh/id_rsa
ansible_user=root
@portante
portante / watch-fluentd-queues.sh
Last active September 12, 2019 16:22
A simple bash script to watch all of the running fluentd pods, gathering and displaying those with on-disk queue file counts greater than two (2) (long queues), or with queue files older than 2 minutes. This script relies on pssh (https://github.com/lilydjwg/pssh) to work efficiently.
#!/bin/bash
function finish {
rm -rf $TMPDIR
}
trap finish EXIT
TMPDIR=$(mktemp -d)
mkdir $TMPDIR/output
let num=0
let slow=0
@portante
portante / watch-fluentd-memory
Created March 21, 2018 13:57
Show fluentd pods using more than 384 MB of memory.
#!/bin/bash
let num=0
for pod in $(oc get pod -n logging -l component=fluentd -o name); do
rss=$(oc exec -n logging $(basename $pod) -- ps auxww | grep fluentd | tail -n 1 | awk '{ print $6 }')
if [ "$rss" -gt 384000 ]; then
let num=num+1
echo "$pod $rss"
fi
done
echo $num
@portante
portante / pending-tasks.sh
Created April 20, 2018 18:07
A simple script to fetch the list of pending tasks from Elasticsearch as deployed in OCP Aggregated Logging.
#!/bin/bash
ES_URL='https://localhost:9200'
curl_get='curl -s -X GET --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'
date
$curl_get $ES_URL/_cluster/pending_tasks?pretty
@portante
portante / sum-es-indices.py
Last active January 21, 2021 21:10
A script to generate a report of Elasticsearch index usage (from _cat/indices?v&bytes=b) by prefix for a set of known date suffixes.
#!/usr/bin/env python2
# A script to generate a report of Elasticsearch index usage
# (from _cat/indices?v&bytes=b) by prefix for a set of known
# date suffixes.
#
# E.g.
# $ curl -X GET http://localhost:9200/_cat/indices?v\&bytes=b -o indices.lis
# $ ./sum-es-indices.py indices.lis
#
@jcantrill
jcantrill / unassigned
Last active July 24, 2018 20:28
Unassiged shards
#!/bin/bash
pod=$1
oc exec -c elasticsearch $pod -- es_util --query=_cat/shards?h=index,shard,prirep,state,unassigned.reason| grep UNASSIGNED
@jcantrill
jcantrill / allocate-all-shards
Last active July 17, 2018 18:02
Allocate all non-assigned primary shards
#!/bin/bash
# This script allocates all primary shards that are unassigned to a a given node using
# the openshift binary. The binary must be in the path and the user executing the script
# must have access to logging project. The inputs are:
# pod An Elasticsearch pod name
# node An node Elasticsearch cluster which should be any one of the DC's. We could
# probably infer this from the pod name
pod=$1
node=$2
@jcantrill
jcantrill / allocate-shard
Created July 12, 2018 20:45
allocate a single shard
#!/bin/bash -e
#
# Copyright 2017 Red Hat, Inc. and/or its affiliates
# and other contributors as indicated by the @author tags.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#!/bin/bash
POD=$1
enable=${2:-"all"} #or none
PERSIST=${PERSIST:-"transient"}
oc exec -n logging -c elasticsearch $POD -- es_util --query=_cluster/settings -XPUT -d "{\"${PERSIST}\":{\"cluster.routing.allocation.enable\":\"${enable}\"}}"