Shaunak Kashyap ycombinator

Keybase proof

I hereby claim:

I am ycombinator on github.
I am ycombinator (https://keybase.io/ycombinator) on keybase.
I have a public key whose fingerprint is 88A0 9B75 97D9 3C72 4661 E601 2F4B ECBF E8DD 0F18

To claim this, I am signing this object:

User perspective

Current state

As of: 7.5.0

ES:
- Recommended: User installs and configures MB along side ES. MB monitors ES node and sends monitoring data directly to monitoring cluster, monitoring-* indices.
- Alternative: User configures ES to send monitoring data to itself (aka production cluster). Production cluster forwards monitoring data to monitoring cluster, .monitoring-* indices.
Kibana:
Recommended: User installs and configures MB along side Kibana. MB monitors Kibana instance and sends monitoring data directly to monitoring cluster, monitoring-* indices.

Background

Filebeat consumes Elasticsearch logs via its elasticsearch module. Specifically, for each type of Elasticsearch log (server, gc, deprecation, etc.) there is a corresponding fileset under the Filebeat elasticsearch module. This fileset is responsible for parsing the Elasticsearch log files into structured event that can then be shipped to Elasticsearch or other outputs.

So whenever the structure of Elasticsearch logs changes, the changes must be tested with the Filebeat elasticsearch module to ensure two things:

that the module can handle the new log structure, and
that the module can continue to handle the previous log structure (in case a user is running an older version of Elasticsearch against a newer version of Filebeat)

If necessary, the ingest pipeline used by the fileset to do the parsing should be updated.

Request

POST .monitoring-kibana-*/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {

Test if Monitoring Cluster version X is able to monitor Production Cluster version Y, where X > Y.

Download and install Elasticsearch version X. This will be the Monitoring Cluster
Download and install Elasticearch version Y. This will be the Production Cluster
Download and install Kibana version Y. This will be the Production Kibana

Start the Monitoring Cluster:

bin/elasticsearch -E cluster.name=esmon -E node.name=esmon_1 -E http.port=9400

Start the Production Cluster:

	#!/bin/bash

	# Find dashboards to fix
	to_fix=""
	for p in packages/*; do
	pkg=$(echo $p \| cut -d"/" -f 2)
	echo "Checking package $pkg..."

	dashboard_filenames=$(find $p/kibana/dashboard -type f 2>/dev/null \| cut -d "/" -f 5)
	echo "Dashboards: "

	import yaml

	def print_yaml(yaml_list):
	print(yaml.safe_dump_all(yaml_list))

	def make_yaml_list(tree):
	yaml_list = []
	for key in tree:
	item = {
	"name": key

	#!/bin/bash

	# Usage:
	# ./find_jenkins_job.sh PR_NUMBER
	# Example:
	# ./find_jenkins_job.sh 15790

	set -e

	NUM_JOBS_TO_SEARCH=100

	#!/bin/bash

	CHILD_PIDFILE=$PWD/child.pid

	# Clean up old child, if any
	if [ -f $CHILD_PIDFILE ]; then
	OLD_CHILD_PID=$(cat $CHILD_PIDFILE)
	ps $OLD_CHILD_PID >/dev/null
	if [ $? -eq 0 ]; then
	kill -9 $OLD_CHILD_PID