JJS 0xYUANTI

A friend asked me for a few pointers to interesting, mostly recent papers on data warehousing and "big data" database systems, with an eye towards real-world deployments. I figured I'd share the list. It's biased and rather incomplete but maybe of interest to someone. While many are obvious choices (I've omitted several, like MapReduce), I think there are a few underappreciated gems.

###Dataflow Engines:

Dryad--general-purpose distributed parallel dataflow engine
http://research.microsoft.com/en-us/projects/dryad/eurosys07.pdf

Spark--in memory dataflow
http://www.cs.berkeley.edu/~matei/papers/2012/nsdi_spark.pdf

	# This code is part of a presentation on streaming analytics in Julia
	# It was inspired by a number of individuals and makes use of some of their ideas
	# 1. FastML.com got me thinking about inline processing after
	# reading his great Vowpal Wabbit posts
	# 2. John Lanford and his fantastic Vowpal Wabbit library.
	# Check out his NYU video course to learn more (see below)
	# 3. John Myles White's presentation on online SDG and his StreamStats.jl library
	# Thank you all!

	// A type-level SAT solver in Swift


	// Two types to model Booleans
	struct True { }

	struct False { }

	// Negation, Conjunction, Disjunction

	# List of all the valid pdf URLs ever posted to the #Clojure IRC channel.
	#
	# Many of them are interesting CS papers others are not that useful. What I've done:
	#
	# 1. crawled an IRC history archive for the channel
	# 2. extract pdf list in a file with: grep -riIohE 'https?://[^[:space:]]+pdf' * > pdf-links.txt
	# 3. remove dupes: cat pdf-links.txt \| sort \| uniq > pdf-links-uniq.txt
	# 4. filter only HTTP 200: cat pdf-links-uniq.txt \| xargs curl -o /dev/null --connect-timeout 2 --silent --head --write-out '%{http_code} %{url_effective}\n' \| grep "^200" > valid-pdf-links.txt
	#
	# Now your choice to download them all or not. If you want, use: cat valid-pdf-links.txt \| awk '{print $2}' \| xargs wget

	#include "tweetnacl.h"

	#define FOR(i,n) for (i = 0;i < n;++i)
	#define sv static void

	typedef unsigned char u8;
	typedef unsigned int u32;
	typedef unsigned long long u64;
	typedef long long i64;
	typedef i64 gf[16];