nmilford’s gists

nmilford / gist:886260

Created March 25, 2011 02:25

	# all the config is in attributes and is placed by the hadoop::default recipe.

	package "hadoop-0.20-datanode" do
	action :install
	version node[:Hadoop][:Version]
	end

	package "hadoop-0.20-tasktracker" do
	action :install
	version node[:Hadoop][:Version]

nmilford / chef template array

Created April 15, 2011 19:56

Trying to build a chef template that enumerates directories from an array

	directories:
	<% @node[:app][:directories].each do \|d\| %>
	- <% d %>
	<% end %>

	Whereas node[:app][:directories] is an array that currently has six entries...

	default[:app][:directories] = [ "/data/a/", "/data/b/", "/data/c/", "/data/d/", "/data/e/", "/data/f/" ]

	Compiled, the template ends up like this:

nmilford / gist:970641

Created May 13, 2011 14:34

	<html>
	<head>
	<title>Cassandra Ring Calculator</title>
	</head>
	<body>
	<input id='numNodes' size=10> <button onClick='calcRing()'>Submit</button>
	<script type='text/javascript'>
	function calcRing() {
	var nodes=document.getElementById('numNodes').value;

nmilford / gist:978659

Created May 18, 2011 14:20

blerg

	# Brisk mapred-site.xml
	<?xml version="1.0"?>
	<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

	<configuration>

	<property>
	<name>mapred.job.tracker</name>
	<value>${brisk.job.tracker}</value>
	</property>

nmilford / gist:1312881

Created October 25, 2011 14:19

bronto API test code

	#!/usr/bin/env python
	# http://app.bronto.com/api/?q=_default
	import suds
	import logging

	logging.basicConfig(level=logging.INFO)
	logging.getLogger('suds.client').setLevel(logging.DEBUG)

	brontoWSDL = "https://api.bronto.com/v4?wsdl"
	brontoApiToken = "REDACTED"

nmilford / gen-pid.clj

Created September 15, 2012 03:48

Have your Clojure program generate a .pid file.

	(ns io.milford.util
	(:import [java.lang.management ManagementFactory]))

	(defn get-pid []
	"Gets this process' PID."
	(let [pid (.getName (ManagementFactory/getRuntimeMXBean))]
	(first (re-seq #"[0-9]+" pid))))

	(defn write-pid-file [pid-file]
	"Writes this process' PID to a supplied file name."

nmilford / cassandra_helpers.rb

Created February 9, 2013 16:49

When chef-client is run manually node[:Cassandra][Keyspaces] populates properly from discover_cassandra_schema() and then the cassandra_maintenance.sh.erb template is generated properly. When Chef is run in the background (we keep it in cron) it does not populate, and the template is blank. I suspect it might have something to do with the way 's…

	# relevent bits from my library

	def discover_cassandra_schema
	# Discover schema information for this host so we can dynamically generate
	# maintenance scripts and collectd config.
	#
	# This depends on the discover-cassandra-schema.sh script. Eventually
	# I will us the Cassandra Ruby gem to get this data directly.

	# Gets schema data into an array in and the form of keyspace.columnfamily

nmilford / gist:4750315

Created February 10, 2013 17:28

I am plugged directly into my TWC cable modem box. My next hop after that box gets 30-60% packet loss and wildly varying latency.

	This is me tracerouting my next hop with TWC

	nathan@citadel:~$ traceroute 72.229.48.1
	traceroute to 72.229.48.1 (72.229.48.1), 64 hops max
	1 72.229.48.1 (72.229.48.1) 1021.072ms * 10.697ms

	This is my TWC supplied DNS server

	traceroute to 209.18.47.61 (209.18.47.61), 64 hops max
	1 * * 72.229.48.1 (72.229.48.1) 81.944ms

nmilford / gist:5250046

Created March 26, 2013 22:51

In which I never set up a secondary Namenode.

	2013-03-26 17:32:34,443 INFO org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file fsimage size 7196783 bytes.
	2013-03-26 17:51:14,689 INFO org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file edits size 14987736708 bytes.
	<--snip-->
	2013-03-26 18:10:20,534 INFO org.apache.hadoop.hdfs.server.common.Storage: Edits file /data/a/dfs/checkpoint/current/edits of size 14987736708 edits # 97361272 loaded in 1143 seconds.
	<--snip-->
	2013-03-26 18:12:04,277 WARN org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Checkpoint done. New Image Size: 798836083

nmilford / glu.spec

Created April 19, 2013 17:30

Initial glu spec attempt

	%define __jar_repack %{nil}
	%define glu_ver 4.4.0
	%define glu_pkg org.linkedin.glu.packaging-all-%{glu_ver}
	%define glu_home /opt/glu

	Name: glu
	Version: %{glu_ver}
	Release: 1%{?dist}
	Source: http://repo.nydc1.outbrain.com/Misc/glu/%{glu_pkg}.tgz
	License: Apache

Nathan Milford nmilford