prateek

-- 1. Place in ~/Library/Scripts and enable the Applescript menu via the Applescript Editor
--    (Or export to .app to run from spotlight.)
-- 2. Substitute "vpn.example.com" and "redacted" for your VPN server and password
-- 3. Open Security & Privacy System Preferences, go to Privacy, Accessibility
-- 4. Enable Applescript Editor and System UI Server (or for this .app if so exported)
-- 5. Trigger script from the menu (or run from spotlight)
-- 6. Enjoy being connected
-- 7. Run script again to close connection

prateek

#!/bin/bash

# paranoia with shell scripts, always to be encouraged
set -e
LOG=/tmp/shell-impala-$USER-$(date +%s).log

# echo-ing params to find logs on local fs in case of error
echo "HOST: ${HOSTNAME}"
echo "LOG: ${LOG}"

prateek

/**
 git clone https://github.com/twitter/scalding.git
 cd scalding
 ./sbt scalding-repl/console
*/

import scala.io.Source
val alice = Source.fromURL("http://www.gutenberg.org/files/11/11.txt").getLines
// Add the line numbers, which we might want later
val aliceLineNum = alice.zipWithIndex.toList

prateek

Offline Python Package Install Notes

Two common cases which make the install of Python packages harder due to networking issues are: a) Install behind a Proxy b) Install without access to the internet

(a) Install behind a Proxy

In the case where the target machine connects to the internet over a network proxy, export the following environment vars, as appropriate - http_proxy and https_proxy. Eg:

prateek

#!/usr/bin/env python
# encoding: utf-8

# File Transfer for Pythonista
# ============================
# This script allows you to transfer Python files from
# and to Pythonista via local Wifi.
# It starts a basic HTTP server that you can access
# as a web page from your browser.
# When you upload a file that already exists, it is

prateek

One-line Tree in Python

Using Python's built-in defaultdict we can easily define a tree data structure:

def tree(): return defaultdict(tree)

That's it!

prateek

New Distcp Usage Notes

---

Make sure you can access the S3 bucket with a command such as:

$ hdfs dfs -put localfile s3n://$BUCKET/

If this works, uncompress the attached jar and try the command below.

prateek

package mapreduce

/**
 * This is an attempt to find a minimal set of type classes that describe the map-reduce programming model
 * (the underlying model of Google map/reduce, Hadoop, Spark and others)
 * The idea is to have:
 * 1) lawful types that fully constrain correctness
 * 2) a minimal set of laws (i.e. we can't remove any laws,
 * 3) able to fully express existing map/reduce in terms of these types
 * 

prateek

#!/bin/bash

# Adapted from
# a) http://blog.cloudera.com/blog/2013/11/approaches-to-backup-and-disaster-recovery-in-hbase/#snapshots
# b) https://blog.cloudera.com/blog/2013/03/introduction-to-apache-hbase-snapshots/
# c) https://groups.google.com/forum/#!topic/nosql-databases/osC58F5PDsE

# 1. create table
hbase shell <<EOF
disable 'testTable'

prateek

#!/bin/bash
# restart-cm-service.sh
# Restart a Cloudera-Manager-managed service via the REST API
# -----------------------------------------------------------------------
# Copyright (C) 2014 Cloudera and Ben White

# Cloudera Manager credentials

USERNAME=admin
PASSWORD=admin