P. Oscar Boykin johnynek

Pomonoid: if x <= y, then zx <= zy for all x,y,z in X. Integral Pomonoid if x <= 1 for all x in X.

Lemma: If you have an Integral Pomonoid, then xy <= x. yx <= x

Suppose you have a key like (page, geo, day) and you want to make rollups/datacube so you can query for all pages, or all geos or all days.

Here is how you do it:

def opts[T](t: T): Seq[Option[T]] = Seq(Some(t), None)

val p: TypedPipe[(String, String, Int)] = ...

p.sumByLocalKeys

I hereby claim:

I am johnynek on github.
I am posco (https://keybase.io/posco) on keybase.
I have a public key whose fingerprint is 11C7 5CF0 8E84 D7FA EC33 1F78 7BE2 A709 A274 EDB0

To claim this, I am signing this object:

	/**
	* @avibryant and I have been interested in extracting as much of scalding out into Algebird,
	* so that it is portable across many execution systems, but how to model joins?
	*
	* In the FP world, Applicative[M] is a typeclass that gives you both Functor[M] (which provides map)
	* and in addition join:
	*/
	trait Functor[M[_]] {
	// law: map(map(a)(f))(g) == map(a)(f.andThen(g))
	def map[V,U](init: M[T])(fn: T => U): M[U]


	use std::comm::{Receiver, channel};
	use std::io;
	use std::mem::replace;
	use std::task::spawn;

	struct Future<'a, A> {
	state: FutureState<'a, A>
	}

	package mapreduce

	/**
	* This is an attempt to find a minimal set of type classes that describe the map-reduce programming model
	* (the underlying model of Google map/reduce, Hadoop, Spark and others)
	* The idea is to have:
	* 1) lawful types that fully constrain correctness
	* 2) a minimal set of laws (i.e. we can't remove any laws,
	* 3) able to fully express existing map/reduce in terms of these types
	*

	/**
	git clone https://github.com/twitter/scalding.git
	cd scalding
	./sbt scalding-repl/console
	*/

	import scala.io.Source
	val alice = Source.fromURL("http://www.gutenberg.org/files/11/11.txt").getLines
	// Add the line numbers, which we might want later
	val aliceLineNum = alice.zipWithIndex.toList

	// I was reading through these examples: http://apocalisp.wordpress.com/2010/06/08/type-level-programming-in-scala/
	// and I thought it would be nice to more quickly get to something useful, to show the power of the techniques.
	object SizedListExample {
	// Type of all Non-negative integers
	sealed trait Nat
	// This is zero.
	sealed trait _0 extends Nat
	// Successor to some non-negative number
	sealed trait Succ[N <: Nat] extends Nat

	// Use Gists to store code you would like to remember later on
	console.log(window); // log the "window" object to the console

	// Find the newest asserted at for each combo of subject & property
	.groupBy('subject, 'property) {
	_.sortedReverseTake[(String, Double)](('asserted_at, 'value) -> 'items , 1)
	}

	// Extract out that newest asserted at value
	.flatten[(String, Double)](('items) -> ('asserted_at, 'value))
	.discard('items)