Skip to content

Instantly share code, notes, and snippets.

View evidanary's full-sized avatar

evidanary evidanary

13 followers · 2 following
View GitHub Profile
@evidanary
evidanary / mixpanel_extract.rb
Last active August 29, 2015 13:56
Class file to download mixpanel data and convert to CSV
require 'mixpanel_client'
require 'csv'
require 'multi_json'
require 'net/http'
class MixpanelExtract
attr_reader :filename, :request_uri
def initialize(config, for_date)
@client = Mixpanel::Client.new(config[:mixpanel_auth])
@evidanary
evidanary / json_schema_extractor.rb
Last active August 29, 2015 13:56
Extract Schema from a JSON row. Generates a hive table. Flattens based on {"event" => ....., "properties" = > ....}
# This utilityi extracts schema of a JSON object
# It returns the keys of the JSON object separated by commas
# It also returns a create table statement in Hive
require '/var/lib/jenkins/scripts/lib/hive_utils.rb'
require 'json'
# This function should probably be in hive_utils.rb
def create_table_string(params)
header = "CREATE TABLE #{params[:hive_schema]}.#{params[:hive_table]}("
@evidanary
evidanary / pwgen.rb
Created December 20, 2013 18:57 — forked from lpar/pwgen.rb
#!/usr/bin/env ruby
# encoding: UTF-8
# Simple password generation in Ruby.
#
# Generate reasonably secure random passwords of any chosen length,
# designed to be somewhat easy for humans to read and remember.
# Each password has a capitalized letter and a digit.
#
# Example:
@evidanary
evidanary / gist:7651363
Created November 26, 2013 00:25
this should be in conf/activemq.xml
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
@evidanary
evidanary / gist:6418644
Last active December 22, 2015 04:39
Monitor Hadoop periodically to check if there is a change in the number of data nodes. Also checks if the utilized space on the cluster exceeds a thresold DFS_USED_PERCENT_THRESHOLD. Exits with error code = 1 when it is not able to run the dfsadmin -report command. Also prints the tmp directory space as this often gets filled up if running bad q…
#!/usr/bin/env ruby
# Checks Hadoop and alerts if there is a change in data nodes
require 'yaml'
DFS_USED_PERCENT_THRESHOLD = 75
error_flag = 0
report = `hdfs dfsadmin -report`
# Datanodes available: 17 (28 total, 11 dead)
@evidanary
evidanary / instant-search.html
Created May 7, 2012 00:00
Real time instant search
<style>
.loc {
color:gray;
font-style:italic;
font-size:11px;
}
</style>
<script type='text/javascript' src='https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js'></script>
<script type='text/javascript'>
$(document).ready(function() {