Skip to content

Instantly share code, notes, and snippets.

@griggheo
griggheo / mail_sendtime.pig
Created February 9, 2012 16:26
Pig script for analyzing mail sending times
REGISTER file:/home/hadoop/lib/pig/piggybank.jar;
DEFINE EXTRACT org.apache.pig.piggybank.evaluation.string.EXTRACT();
DEFINE CustomFormatToISO org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO();
DEFINE ISOToUnix org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();
DEFINE DATE_TIME org.apache.pig.piggybank.evaluation.datetime.DATE_TIME();
DEFINE FORMAT_DT org.apache.pig.piggybank.evaluation.datetime.FORMAT_DT();
@griggheo
griggheo / nginx_time_conversions.pig
Created February 9, 2012 17:11
Pig script for Nginx access log date/time conversions
REGISTER file:/home/hadoop/lib/pig/piggybank.jar;
DEFINE EXTRACT org.apache.pig.piggybank.evaluation.string.EXTRACT();
DEFINE CustomFormatToISO org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO();
DEFINE ISOToUnix org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();
DEFINE DATE_TIME org.apache.pig.piggybank.evaluation.datetime.DATE_TIME();
DEFINE FORMAT_DT org.apache.pig.piggybank.evaluation.datetime.FORMAT_DT();
DEFINE FORMAT org.apache.pig.piggybank.evaluation.string.FORMAT();
RAW_LOGS = LOAD '$INPUT' as (line:chararray);
@griggheo
griggheo / gist:2698152
Created May 14, 2012 23:51
dynamodb batchwriteitem in boto
import os
import sys
import subprocess
import re
import optparse
import boto
dynamodb_conn = boto.connect_dynamodb(aws_access_key_id='MY_ACCESS_KEY_ID', aws_secret_access_key='MY_SECRET_ACCESS_KEY')
table_name = 'mytable'
dynamodb_table = dynamodb_conn.get_table(table_name)
@griggheo
griggheo / compare_env.go
Created August 28, 2014 21:04
Inspect differences between Chef environments on Chef server vs local filesystem
package main
import (
"fmt"
"log"
"bytes"
"strings"
"flag"
"sort"
"os/exec"
@griggheo
griggheo / compare_env_v2.go
Created August 29, 2014 00:26
Version 2 of compare_env.go
package main
import (
"fmt"
"log"
"bytes"
"strings"
"flag"
"sort"
"os/exec"
{
"network": {
"servers": [ "{{logstash_elasticsearch_host }}:{{ logstash_listen_port_tcp }}" ],
"ssl ca": "{{ logstash_forwarder_etc_dir }}/{{ ssl_cert }}", "timeout": 15
},
"files": [
{
"paths": [
"/var/log/syslog",
"/var/log/*.log"
input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/logstash/my_wildcard.crt"
ssl_key => "/etc/pki/logstash/my_wildcard.key"
}
}
filter {
if ([message] =~ /ping/) {
filter {
grok {
add_tag => [ "valid" ]
match => { "message" => "%{APPLOGLINE}" }
}
json {
source => "payload"
}
if "valid" not in [tags] {
drop { }
@griggheo
griggheo / check_pingdom_and_post_cachet_status.go
Created October 13, 2015 23:28
Golang program which integrates Pingdom checks into a Cachet status page
package main
import (
"github.com/russellcardullo/go-pingdom/pingdom"
"fmt"
"time"
)
const PINGDOM_USERNAME="xxxx"
@griggheo
griggheo / keybase.md
Created November 3, 2015 18:03
keybase.md

Keybase proof

I hereby claim:

  • I am griggheo on github.
  • I am griggheo (https://keybase.io/griggheo) on keybase.
  • I have a public key whose fingerprint is 0BDA 74F2 92A1 4044 0D22 974C 03A6 7437 7506 1608

To claim this, I am signing this object: