Dat Tran datitran

What

Roll your own iPython Notebook server with Amazon Web Services (EC2) using their Free Tier.

What are we using? What do you need?

An active AWS account. First time sign-ups are eligible for the free tier for a year
One Micro Tier EC2 Instance
With AWS we will use the stock Ubuntu Server AMI and customize it.
Anaconda for Python.
Coffee/Beer/Time

Setting up github and bitbucket on the same computer (Windows)

Guide for Windows

mix3d asked for some help using this guide with windows so here we go. This was tested with Windows 10. Run all commands in Git Bash once it's installed.

Github will be the main account and bitbucket the secondary.

Git for Windows

Download and install Git for Windows
- In the installer, select everything but decide if you want a desktop icon (2nd step)

Apache Spark installation + ipython/jupyter notebook integration guide for macOS

Tested with Apache Spark 2.1.0, Python 2.7.13 and Java 1.8.0_112

For older versions of Spark and ipython, please, see also previous version of text.

Install Java Development Kit

Spark on Cloud Foundry

This document describes one means of running a simple Apache Spark cluster on Cloud Foundry. It makes heavy use of Cloud Foundry's container networking features.

You can see an example running at http://spark-ui-proxy.184.73.108.92.xip.io.

Deploy BOSH-Lite on AWS

This cluster was deployed using BOSH-Lite on AWS. Note, this Director cannot be targetted with the new BOSH CLI (see cloudfoundry-attic/bosh-lite#424), but you can use the "old" Ruby CLI just fine. You can use the new CLI for local workflows like manifest interpolation, and then the "old" CLI for remote workflows like deploying and SSH.

	"""
	From the man page for ssh:

	-f Requests ssh to go to background just before command execution. This is useful if ssh is going to ask
	for passwords or passphrases, but the user wants it in the background. This implies -n. The recommended way
	to start X11 programs at a remote site is with something like ssh -f host xterm.

	If the ExitOnForwardFailure configuration option is set to “yes”, then a client started with -f will wait
	for all remote port forwards to be successfully established before placing itself in the background.

	" Use Vim settings, rather then Vi settings (much better!).
	" This must be first, because it changes other options as a side effect.
	set nocompatible

	" ================ General Config ====================

	set number "Line numbers are good
	set backspace=indent,eol,start "Allow backspace in insert mode
	set history=1000 "Store lots of :cmdline history
	set showcmd "Show incomplete cmds down the bottom

	import theano
	import theano.tensor as T
	import numpy as np
	import cPickle
	import random
	import matplotlib.pyplot as plt

	class RNN(object):

	def __init__(self, nin, n_hidden, nout):

	from IPython.display import HTML
	import folium

	def inline_map(map):
	"""
	Embeds the HTML source of the map directly into the IPython notebook.

	This method will not work if the map depends on any files (json data). Also this uses
	the HTML5 srcdoc attribute, which may not be supported in all browsers.
	"""

	# Markus Gesmann
	library(arm) # for 'display' function only
	icecream <- data.frame(
	# http://www.statcrunch.com/5.0/viewreport.php?reportid=34965&groupid=1848
	temp=c(11.9, 14.2, 15.2, 16.4, 17.2, 18.1,
	18.5, 19.4, 22.1, 22.6, 23.4, 25.1),
	units=c(185L, 215L, 332L, 325L, 408L, 421L,
	406L, 412L, 522L, 445L, 544L, 614L)
	)
	basicPlot <- function(...){

	# Get everything ready on a Mac
	brew install docker docker-machine docker-compose

	# Create a new machine (based on the 1.8.1 boot2docker iso)
	docker-machine create --driver vmwarefusion --vmwarefusion-memory-size 2048 --vmwarefusion-boot2docker-url https://github.com/boot2docker/boot2docker/releases/download/v1.8.1/boot2docker.iso dev

	# Bring the machine up
	docker-machine start dev

	# Configure Shell