Prateek Rungta prateek

Create a FlameGraph to visualize where your code is spending its time.

Offline Python Package Install Notes

Two common cases which make the install of Python packages harder due to networking issues are: a) Install behind a Proxy b) Install without access to the internet

(a) Install behind a Proxy

In the case where the target machine connects to the internet over a network proxy, export the following environment vars, as appropriate - http_proxy and https_proxy. Eg:

	#!/usr/bin/env python
	# -- coding: utf-8 --
	#
	# golang_import_order_cleaner.py: cleans up import orders in golang files per the m3db convention
	#
	# Warning: This is very much alpha, take a backup of your files (e.g. commit to git) before running this.
	#
	# example usage:
	# (1) Update all *.go files in the local directory (except vendor/), in-place
	# $ golang_import_order_cleaner.py -o inplace


	// function to get list of files in a directory, recursively
	getFileList := func(path string) map[string]struct{} {
	fileList := make(map[string]struct{})
	err = filepath.Walk(path, func(path string, f os.FileInfo, err error) error {
	fileList[path] = struct{}{}
	return nil
	})
	require.NoError(t, err)
	return fileList

	package main

	import (
	"fmt"
	"io"
	"io/ioutil"
	"net"
	"os"
	"strings"

	// once upon a time, what did I see, but two RDDs
	val tableA = sc.parallelize(List((1,2), (3,4), (5,6)))
	val tableB = sc.parallelize(List((1,'A'), (3,'B'), (5,'C')))
	// one of them small enough to fit into map
	val mapTableB = tableB.collectAsMap
	// which everyone could read
	val broadcastB = sc.broadcast(mapTableB)
	// and join for all eternity
	val mapJoin = tableA.map({case (id, value) => (id, value, broadcastB.value.get(id))})
	// q.e.d.

	/**
	* Created by prungta on 7/22/15.
	*/
	import org.apache.oozie.client.AuthOozieClient;
	import org.apache.oozie.client.OozieClient;
	import org.apache.oozie.client.OozieClientException;
	import org.apache.oozie.client.WorkflowJob;

	import javax.security.auth.Subject;
	import javax.security.auth.login.AppConfigurationEntry;

	-- 1. Place in ~/Library/Scripts and enable the Applescript menu via the Applescript Editor
	-- (Or export to .app to run from spotlight.)
	-- 2. Substitute "vpn.example.com" and "redacted" for your VPN server and password
	-- 3. Open Security & Privacy System Preferences, go to Privacy, Accessibility
	-- 4. Enable Applescript Editor and System UI Server (or for this .app if so exported)
	-- 5. Trigger script from the menu (or run from spotlight)
	-- 6. Enjoy being connected
	-- 7. Run script again to close connection

	#!/bin/bash

	# paranoia with shell scripts, always to be encouraged
	set -e
	LOG=/tmp/shell-impala-$USER-$(date +%s).log

	# echo-ing params to find logs on local fs in case of error
	echo "HOST: ${HOSTNAME}"
	echo "LOG: ${LOG}"

	/**
	git clone https://github.com/twitter/scalding.git
	cd scalding
	./sbt scalding-repl/console
	*/

	import scala.io.Source
	val alice = Source.fromURL("http://www.gutenberg.org/files/11/11.txt").getLines
	// Add the line numbers, which we might want later
	val aliceLineNum = alice.zipWithIndex.toList