portante’s gists

portante / index_usage_per_day.sh

Created June 23, 2017 00:17

A simple script to calculate the total size of data per day for all indices

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_get='curl -s -X GET --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	$curl_get $ES_URL/_cat/indices?bytes=m > /tmp/indices.lis

	for days_back in {1..7}; do
	thedate=$(date --date="-${days_back} day" +"%Y.%m.%d")
	echo "$thedate: $(grep $thedate /tmp/indices.lis \| awk '{sum += $8} END {print sum}')"

portante / turn-off-auto-exp-rep.sh

Created June 21, 2017 22:34

A simple script to turn off "auto_expand_replicas" setting for each index.

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_put='curl -s -X PUT --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	$curl_put $ES_URL/*/_settings -d '{ "index" : { "auto_expand_replicas" : false } }'

portante / health.sh

Last active February 12, 2019 20:34

A simple script to display the "health" of an ES cluster

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_get='curl -s -X GET --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	date
	$curl_get $ES_URL/_cat/health?v
	$curl_get $ES_URL/_cat/allocation?v\&h=node,host,ip,shards,disk.indices,disk.used,disk.avail,disk.total,disk.percent
	# See https://www.elastic.co/guide/en/elasticsearch/reference/2.4/cat-nodes.html for header meanings
	$curl_get $ES_URL/_cat/nodes?v\&h=name,host,r,m,hc,hp,hm,rc,rp,rm,fdc,fdp,fdm,load,uptime

portante / delete-an-index.sh

Last active March 22, 2018 15:25

Delete an index

-#!/bin/bash
-ES_URL='https://localhost:9200'
-curl_del='curl -s -X DELETE --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'
-$curl_del $ES_URL/${1}?pretty=true

portante / dump_indices.sh

Last active March 5, 2018 19:03

Dump the current state of all the indices in an ES cluster under OpenShift

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_get='curl -s -X GET --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	date
	$curl_get $ES_URL/_cat/indices?v\&bytes=b

portante / turn-off-fluentd

Created June 5, 2017 21:09

Simple command to turn off fluentd on all nodes

$ oc label node -l logging-infra-fluentd=true --overwrite logging-infra-fluentd=false

portante / close.sh

Last active June 7, 2017 13:06

Close indices matching a given pattern

	#!/bin/bash
	# Usage: close.sh \.2017.04.\
	pattern=$1
	ES_URL='https://localhost:9200'
	curl_post='curl -s -X POST --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	$curl_post $ES_URL/$pattern/_close

portante / change-rep-count-0.sh

Last active May 18, 2018 12:24

A simple script to set the replica count of all indices to 0

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_put='curl -s -X PUT --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	$curl_put $ES_URL/${1}/_settings?pretty -d '{ "index" : { "auto_expand_replicas" : "false" } }'
	$curl_put $ES_URL/${1}/_settings?pretty -d '{ "index" : { "number_of_replicas" : 0 } }'

portante / dumplogs.sh

Created May 29, 2017 12:26

A simple script to dump the logs through a filter (format-es-logs gist)

	#!/bin/bash

	ES_URL='https://localhost:9200'
	curl_get='curl -s -X GET --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'

	$curl_get $ES_URL/<index>/_search"?pretty&sort=@timestamp:desc&fields=@timestamp,kubernetes.host,kubernetes.pod_name,message" \| ./format-es-logs

portante / delete-kibana

Created May 17, 2017 14:50

A simple script to delete the openshift .kibana.* user indices

-#!/bin/bash
-ES_URL='https://localhost:9200'
-curl_del='curl -s -X DELETE --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key'
-$curl_del $ES_URL/.kibana.\*

Peter Portante portante