Philip K. Adetiloye mavencode01

PostgreSQL Streaming Replication With Docker

The *.txt files here hold user and database parameters. Specifically, replication.txt contains the user/role and password to use for replication. Whereas database.txt contains an initial database, user/role and password to create on the master.

Run the master:

$ fig run -d master

Wait for it to start up completely. Start the slave:

Install, Setup, and Test Spark and Cassandra on Mac OS X

This Gist assumes you already followed the instructions to install Cassandra, created a keyspace and table, and added some data.

Install Apache Spark

brew install apache-spark

The introduction to Reactive Programming you've been missing

(by @andrestaltz)

This tutorial as a series of videos

If you prefer to watch video tutorials with live-coding, then check out this series I recorded with the same contents as in this article: Egghead.io - Introduction to Reactive Programming.

	'''
	This script performs efficient concatenation of files stored in S3. Given a
	folder, output location, and optional suffix, all files with the given suffix
	will be concatenated into one file stored in the output location.

	Concatenation is performed within S3 when possible, falling back to local
	operations when necessary.

	Run `python combineS3Files.py -h` for more info.
	'''

	package com.databricks.spark.jira

	import scala.io.Source

	import org.apache.spark.rdd.RDD

	import org.apache.spark.sql._
	import org.apache.spark.sql.functions._
	import org.apache.spark.sql.sources.{TableScan, BaseRelation, RelationProvider}

	# You don't need Fog in Ruby or some other library to upload to S3 -- shell works perfectly fine
	# This is how I upload my new Sol Trader builds (http://soltrader.net)
	# Based on a modified script from here: http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash

	S3KEY="my aws key"
	S3SECRET="my aws secret" # pass these in

	function putS3
	{
	path=$1

	import org.junit.Test;
	import org.vertx.java.core.AsyncResult;
	import org.vertx.java.core.Handler;
	import org.vertx.java.core.file.AsyncFile;
	import org.vertx.java.core.http.HttpClient;
	import org.vertx.java.core.http.HttpClientResponse;
	import org.vertx.java.core.streams.Pump;
	import org.vertx.testtools.TestVerticle;

	public class DownloadTest extends TestVerticle {

	// Finds the connected component containing a particular vertex.
	// In response to http://apache-spark-developers-list.1001551.n3.nabble.com/GraphX-some-vertex-with-specific-edge-td8436.html

	import org.apache.spark.graphx._

	// Construct the graph in the above example
	val edges = sc.parallelize(List(
	Edge(1L, 2L, "e1"), Edge(2L, 3L, "e1"), Edge(3L, 4L, "e1")))
	val g: Graph[Int, String] = Graph.fromEdges(edges, 0)

	// The following code is from http://www.academicpub.org/PaperInfo.aspx?PaperID=14496 .
	import java.math.BigInteger;
	import java.security.*;
	import java.security.spec.*;
	import javax.crypto.KeyAgreement;

	public class ECCKeyAgreement {
	public static void main(String[] args) throws Exception {
	KeyPairGenerator kpg;
	kpg = KeyPairGenerator.getInstance("EC","SunEC");

	#!/usr/bin/env sh
	open "tel://$*"