ee08b397

tmux cheat sheet

(C-x means ctrl+x, M-x means alt+x)

Prefix key

The default prefix is C-b. If you (or your muscle memory) prefer C-a, you need to add this to ~/.tmux.conf:

remap prefix to Control + a

ee08b397

One-line Tree in Python

Using Python's built-in defaultdict we can easily define a tree data structure:

def tree(): return defaultdict(tree)

That's it!

ee08b397

OS X Screencast to animated GIF

This gist shows how to create a GIF screencast using only free OS X tools: QuickTime, ffmpeg, and gifsicle.

Instructions

To capture the video (filesize: 19MB), using the free "QuickTime Player" application:

ee08b397

ee08b397

UPDATE: A better way! (August 2015)

As pointed out by @johntyree in the comments, using git reflog is easier and more reliable. Thanks for the suggestion!

 $ git reflog
1ed7510 HEAD@{1}: checkout: moving from develop to 1ed7510
3970d09 HEAD@{2}: checkout: moving from b-fix-build to develop
1ed7510 HEAD@{3}: commit: got everything working the way I want
70b3696 HEAD@{4}: commit: upgrade rails, do some refactoring

ee08b397

import time
import uuid
import yappi
import timeit
import logging
import random

from threading import Event
from cassandra import ConsistencyLevel
from cassandra.cluster import Cluster

ee08b397

Best way to create Mutliple files into a single RDD
==================================

val fileRDD = sc.textFile(filename).repartition(1)

Where the filename is the location of your directory only.

ee08b397

How to create a MultiNode - MultiBroker Cluster for Kafka on AWS

PreRequisites :

1. Kafka Binary files : http://kafka.apache.org/downloads.html

2. Atleast 2 AWS machines : AWS EMR or EC2 will be preferable

3. A Kafka Manager Utility to watch up the Cluster : https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem

ee08b397

Setting up Flume NG, listening to syslog over UDP, with an S3 Sink

My goal was to set up Flume on my web instances, and write all events into s3, so I could easily use other tools like Amazon Elastic Map Reduce, and Amazon Red Shift.

I didn't want to have to deal with log rotation myself, so I setup Flume to read from a syslog UDP source. In this case, Flume NG acts as a syslog server, so as long as Flume is running, my web application can simply write to it in syslog format on the specified port. Most languages have plugins for this.

At the time of this writing, I've been able to get Flume NG up and running on 3 ec2 instances, and all writing to the same bucket.

Install Flume NG on instances

ee08b397

#!/usr/bin/env bash

# global parameters
g_tmp_folder="ncdc_tmp";
g_output_folder="ncdc_data";

g_remote_host="ftp3.ncdc.noaa.gov";
g_remote_path="pub/data/noaa";