Skip to content

Instantly share code, notes, and snippets.

@markuskont
markuskont / init.sls
Last active November 21, 2016 14:56
salt_telegraf_win_repo
Telegraf:
1.1.1:
installer: 'salt://win/repo-ng/telegraf/telegraf-1.1.1.exe'
full_name: Influxdata Telegraf 1.1.1
reboot: False
install_flags: ' --service install'
uninstaller: '%ProgramFiles%/telegraf/telegraf.exe'
uninstall_flags: ' --service uninstall'
@markuskont
markuskont / database.py
Last active December 1, 2016 09:50
Use salt to gather windows disk usage metrics into influxdb
#!/usr/bin/env python
# coding: utf-8
from influxdb import InfluxDBClient
class influxData(object):
def __init__(self):
# InfluxDB connection parameters
self.host = 'FQDN'
select last("mem_heap_used_percent") from elasticsearch_jvm where time >'2016-12-06T00:00:00.000000000Z' AND time <'2016-12-07T00:00:00.000000000Z' group by time(3600s), "node_name"
select holt_winters(last("mem_heap_used_percent"), 10, 0) from elasticsearch_jvm where time >'2016-12-06T00:00:00.000000000Z' AND time <'2016-12-07T00:00:00.000000000Z' group by time(3600s), "node_name"
SELECT mean("load1") FROM "system" WHERE "host" = 'example' AND time >'2016-12-05T00:00:00.000000000Z' AND time <'2016-12-08T00:00:00.000000000Z' GROUP BY time(5m) fill(null)SELECT holt_winters_with_fit(last("mem_heap_used_percent"), 24, 0) FROM "elasticsearch_jvm" WHERE "node_name" = '1-gw-es.yellow.ex' AND time >'2016-12-05T08:30:00.000000000Z' AND time <'2016-12-09T08:30:00.000000000Z' GROUP BY time(1h) fill(null)
# Query actual data
SELECT mean("mem_heap_used_percent") FROM "elasticsearch_jvm" WHERE "node_name" = '1-gw-es.yellow.ex' AND $timeFilter GROUP BY time(1m) fill(null)
# Show processed data and predictions
var period = 7d
var every = 1m
var bucket = 1m
var cluster = 'influx.yellow.ex'
var outputDB = 'telegraf'
var outputRP = 'autogen'
var outputMeasurement = 'pred_elastic_jvm_heap_usage'
#!/usr/bin/env python3
from elasticsearch import Elasticsearch
import json
ES_INDEX='stackoverflow-surveys-2016'
es = Elasticsearch(hosts=['192.168.56.201'])
results = es.search(index=ES_INDEX, q='*', size=1)
@markuskont
markuskont / Vagrantfile
Last active December 17, 2016 23:05
Template for salt manifest devel environment. Template only manages initial installation. All salt commands are meant to be invoked from within saltmaster box (that includes initial key acceptance).
# vi: set ft=ruby :
SALT_VERSION = "v2016.11.2"
MASTER_IP = '192.168.56.174'
boxes = [
{
:name => "es-master-0",
:mem => "512",
:cpu => "1",
:ip => "192.168.56.170",
main_queue(
# queue.workerThreads="4"
# queue.dequeueBatchSize="1000"
# queue.highWatermark="500000" # max no. of events to hold in memory
# queue.lowWatermark="200000" # use memory queue again, when it's back to this level
# queue.spoolDirectory="/var/run/rsyslog/queues" # where to write on disk
# queue.fileName="stats_ruleset"
# queue.maxDiskSpace="5g" # it will stop at this much disk space
# queue.size="5000000" # or this many messages
# queue.saveOnShutdown="on" # save memory queue contents to disk when rsyslog is exiting
{
"query": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"gte": "now-7d/d",
"lt": "now/d"
}
{
"from": 0,
"size": 1,
"query": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
@markuskont
markuskont / grain2ela.py
Last active January 23, 2017 11:41
Push salt grains to elasticsearch
#!/usr/bin/env python
# apt-get install python-pip
# pip install elasticsearch
import sys, json, argparse, time, hashlib
import salt.client
from elasticsearch import Elasticsearch
def parse_arguments():