graphite-web + logstash

README

logstash(search) - hits for search string
logstash(search,field,facet) - return stats on a field - facets are count, min, max, total, mean.
logstashGroup(search, group, field, facet) - return stats on a field, grouped by a value. 

functions.py

import pyes

def logstashSearch(requestContext, conn, query, field, resultfacet):
  start = requestContext["startTime"].isoformat()
  end = requestContext["endTime"].isoformat()
   
  boundedquery = "@timestamp:[%s TO %s] AND %s" % (start, end, query)
  q = pyes.StringQuery(boundedquery).search()
  q.facet.facets.append(pyes.facets.DateHistogramFacet('date_facet',
  key_field='@timestamp',
  value_field=field,
  interval='minute'))
  results = conn.search(query=q)
   
  interval=60
  start_timestamp = int(time.mktime(requestContext["startTime"].timetuple()))
  end_timestamp = int(time.mktime(requestContext["endTime"].timetuple()))
  time_range = (end_timestamp-start_timestamp)/interval
   
  values=[0]*time_range
   
  for facet in results.facets.date_facet.entries:
    appear_time = (facet['time']/1000)
    values[(appear_time-start_timestamp)/interval] = facet[resultfacet]
   
  result_series = TimeSeries(query,
            time.mktime(requestContext["startTime"].timetuple()),
            time.mktime(requestContext["endTime"].timetuple()),
            interval, values)

  return result_series
 

def logstashGroup(requestContext, query, group, field="@timestamp", resultfacet="count"):
  conn = pyes.ES("es1:9200")
   
  start = requestContext["startTime"].isoformat()
  end = requestContext["endTime"].isoformat()
   
  boundedquery = "@timestamp:[%s TO %s] AND %s" % (start, end, query)
  q = pyes.StringQuery(boundedquery).search()
  q.facet.add_term_facet(group)
  results = conn.search(query=q)
  resultList = [] 

  for v in results.facets[group]['terms']:
    subquery = query + " AND %s:%s" % (group, v['term'])
    result_series = logstashSearch(requestContext, conn, subquery, field, resultfacet)
    result_series.pathExpression = v['term']
    resultList.append(result_series)

  return resultList

def logstash(requestContext, query, field="@timestamp", resultfacet="count"):
  conn = pyes.ES("es1:9200")
  result_series = logstashSearch(requestContext, conn, query, field, resultfacet)
  return [result_series]

hi, can you give us please example usages, where do i define the elasticsearch index ?
thanx

This searches all indexes - if you want to choose a specific one you'd need to change the function.

You can do stuff like logstash("type:rails","time","mean") which would graph the mean of the time field for documents with type:rails.

Or logstashGroup("type:rails","user","time","mean") is the same, but per user