Dashing splunk® enterprise query

README.md

##Preview

Description

Simple Dashing job template to run splunk® enterprise blocking queries.

##Dependencies

splunk-sdk-ruby. More info at splunk Ruby SDK page.

Add it to dashing's gemfile:

gem 'splunk-sdk-ruby'

and run bundle install. Everything should work now :)

##Installation

1. Copy splunk_query_template.rb into the /jobs directory or type:

dashing install 11221b6ea30a35c7cdc6

2. Put the splunk_connection.yml file into the /config directory (create the directory if it does not exist).

##Usage

1. Put correct splunk connection information (host, port, username, and password) in the splunk_connection.yml file.
2. Change splunk_query_template.rb file name to something more appropriate, put the query you want to execute, and suitable code to process the results in the file. See splunk_query_list_example.rb and splunk_query_table_example.rb files to get an idea of the query format and result processing.
3. Add the necessary HTML snippet to the dashboard layout erb file.

Example:

<li data-row="1" data-col="1" data-sizex="1" data-sizey="2">
    <div data-id="myWidgetId" data-view="List" data-unordered="true" data-title="My cool splunk query result" data-moreinfo="Data from splunk" style="background-color:#2F4F4F"></div>
</li>

##Compatibility The widget was tested with the following splunk versions.

Splunk	Tested By	Verified
6.3.0	Antony Jesudhason	OK
6.2.5	Antony Jesudhason	OK

LICENSE.md

The MIT License (MIT)

Copyright (c) 2015 Antony Jesudhason

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

splunk_connection.yml

:scheme: :https
:host: 'localhost'
:port: 8089
:username: 'admin'
:password: 'changeme'
:basic: true

splunk_list_example.png

splunk_query_list_example.rb

require 'yaml'
require 'splunk-sdk-ruby'

connect_config = YAML.load_file('config/splunk_connection.yml')

query = 'search sourcetype=access_* status=200 action=purchase | top categoryId | eval percent=round(percent, 2)'

splunk = Splunk::connect(connect_config)

# :first_in sets how long it takes before the job is first run. In this case, it is run immediately
SCHEDULER.every '30m', :first_in => 0 do |job|

  stream = nil
  begin
    stream = splunk.create_oneshot(query)
  rescue Exception => e
    puts 'Splunk connect failed. Please check your configuration and ensure that splunk is running. Exception message: ' + e.message
    next
  end

  results = Splunk::ResultsReader.new(stream)

  list_items = Hash.new({ value: 0 })

  results.each do |result|
    list_items[result['categoryId']] = { label: result['categoryId'], value: result['percent']+'%' }
  end

  send_event('categories_by_salepercent', { items: list_items.values })
end

splunk_query_table_example.rb

require 'yaml'
require 'splunk-sdk-ruby'

connect_config = YAML.load_file('config/splunk_connection.yml')

query = 'search sourcetype=access_* status=200 action=purchase | top categoryId'

splunk = Splunk::connect(connect_config)

header = [{"cols" => [{"value" => "Category"},
                       {"value" => "No. of Purchases"},
                       {"value" => "Purchase Percent"}
                     ]}]

# :first_in sets how long it takes before the job is first run. In this case, it is run immediately
SCHEDULER.every '30m', :first_in => 0 do |job|
  
  stream = nil
  begin
    stream = splunk.create_oneshot(query)
  rescue Exception => e
    puts 'Splunk connect failed. Please check your configuration and ensure that splunk is running. Exception message: ' + e.message
    next
  end

  results = Splunk::ResultsReader.new(stream)

  rows = []

  results.each do |result|
    row = [{"value" => result['categoryId']}, 
           {"value" => result['count']},
           {"value" => result['percent']}
          ]
      rows << { "cols" => row }
  end

  send_event('top_categories_table', { hrows: header, rows: rows })
end

splunk_query_template.rb

require 'yaml'
require 'splunk-sdk-ruby'

# TODO: Update the splunk_connection.yml file to contain your splunk server connection information
connect_config = YAML.load_file('config/splunk_connection.yml')

# TODO: Your query here
query = 'search ...'

splunk = Splunk::connect(connect_config)

# :first_in sets how long it takes before the job is first run. In this case, it is run immediately
SCHEDULER.every '30m', :first_in => 0 do |job|

  stream = nil
  begin
    stream = splunk.create_oneshot(query)
  rescue Exception => e
    puts 'Splunk connect failed. Please check your configuration and ensure that splunk is running. Exception message: ' + e.message
    next
  end

  results = Splunk::ResultsReader.new(stream)

  results.each do |result|
    # TODO: Your query result record processing here
  end

  send_event('widgetId', { })
end

splunk_table_example.png

It didn't work for me until i manually set the app namespace. otherwise Splunk API will not allow the search access to results.

The ruby SDK doesn't play nice if you have a load balanced setup, unless you hard code to one of the search heads (this may not be possible if you are using splunk cloud). You may be better off using the python SDK instead.

If you do want to use python here are some snippets that may or may not help.

Method to send stuff to dashing (you can't use send_events a la ruby)...

post_data = {}
def send_results_to_tile(search_result, apikey=dashing_apikey):
  post_data['auth_token'] = apikey
  post_data['items'] = search_result
  url = 'http://localhost/widgets/your_tile_name'
  r = requests.post(url,json.dumps(post_data))

To collate the results and create something along the lines of the ruby hash...

dictList = []
rr = results.ResultsReader(job.results())
for result in rr:
  dictList.append({'label':result["field_name_1"],'value':result["field_name_2"]+'more text if required'})
  send_results_to_tile(dictList)