Campbell Allen camallen

AWS EMR bootstrap to install R packages from CRAN

This bootstrap is useful if you want to deploy SparkR applications that run arbitrary code on the EMR cluster's workers. The R code will need to have its dependencies already installed on each of the workers, and will fail otherwise. This is the case if you use functions such as gapply or dapply.

How to use the bootstrap

You will first have to download the gist to a file and then upload it to S3 in a bucket of your choice.
Using the AWS EMR Console create a cluster and choose advanced options.
In Step 3 you can configure your bootstraps. Choose to Configure and add a Custom action

Given an encoded (base64) JWT (access_token).

For example, see this sample OAuth2 response generated by JBoss APIMan/Keycloak

HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json
Date: Tue, 25 Aug 2015 19:25:12 GMT

	#!/usr/bin/env ruby

	if `git diff HEAD spec` =~ /^\+.*,\s?(:focus\|focus:\s?true\|:focus\s?=>\s?true)/
	puts "\e[31mPlease focus and remove your :focus tags before committing :)"
	exit 1
	end

	SELECT bl.pid AS blocked_pid,
	a.usename AS blocked_user,
	ka.query AS current_statement_in_blocking_process,
	now() - ka.query_start AS blocking_duration,
	kl.pid AS blocking_pid,
	ka.usename AS blocking_user,
	a.query AS blocked_statement,
	now() - a.query_start AS blocked_duration
	FROM pg_catalog.pg_locks bl
	JOIN pg_catalog.pg_stat_activity a ON a.pid = bl.pid

	function dethrottle {
	sudo ipfw delete 1
	}

	function throttle {
	if test $# -eq 0
	then
	echo 'throttle <port> <bandwidth (KB)>'
	elif test $# -eq 1
	then

	<!DOCTYPE html>

	<html lang="en">
	<head>
	<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
	<title>SoundCloud OAuth 2 User Agent Authentication Flow Demo</title>
	<script type="text/javascript" charset="utf-8" src="javascript/jquery-1.4.2.js"></script>
	<script type="text/javascript" charset="utf-8">
	$(function () {
	var extractToken = function(hash) {