Wesley Goi etheleon

Bash string manipulation cheatsheet

Assignment
Assign `value` to `variable` if `variable` is not already set. Value will be returned. Couple with `:` no-op if return value is to be discarded.	`${variable="value"}` `: ${variable="value"}`

Install ponysay: (You may have to sudo apt install software-properties-common first)
- sudo add-apt-repository ppa:vincent-c/ponysay
- sudo apt-get update
- sudo apt-get install ponysay

	jsc <- invoke_static(sc, "org.apache.spark.api.java.JavaSparkContext", "fromSparkContext", spark_context(sc))
	hconf <- jsc %>% invoke("hadoopConfiguration")

	# set the keys
	hconf %>% invoke("set","fs.s3a.access.key", AWS_ACCESS_KEY)
	hconf %>% invoke("set","fs.s3a.secret.key", AWS_SECRET_KEY)

	# API KEY for accessing it's service
	QUBOLE_API="MY_QUBOLE_API_ID"
	# This is the ID which is set by Qubole
	CLUSTER_ID=12830

	conf <- livy_config("custom_headers" = list(`X-AUTH-TOKEN`=QUBOLE_API))
	sc <- spark_connect(
	master = sprintf("https://us.qubole.com/livy-spark-%s", CLUSTER_ID),
	method = "livy",
	config = conf

	from rpy2.robjects.packages import importr
	import rpy2.robjects.lib.ggplot2 as ggplot2 # don't use importr cause the ggplot2 here has the .plot() function

	grdevices = importr('grDevices')
	cowplot = importr('cowplot') #cause i love cowplot
	rfeather = importr("feather", on_conflict="warn")

	# Out[57]:
	# country variable value
	# 0 H random 15.872105

	import os
	import pycurl
	from tqdm import tqdm


	downloader = pycurl.Curl()


	def sanitize(c):
	c.setopt(pycurl.UNRESTRICTED_AUTH, False)

	#!/usr/bin/env bash

	## create an ubuntu 14.04 hvm instance, then from your home directory:

	# 1. download this script
	# wget https://gist.githubusercontent.com/waylonflinn/506f563573600d944923/raw/install-python-data-science.sh

	# 2. make it executable
	# chmod a+x install-python-data-science.sh