MarkRoddy’s gists

MarkRoddy / dynFib.go

Last active August 29, 2015 14:03

	func dynFib(m int) int {
	inner := func(n int) int {
	switch {
	case n == 0:
	return 0;
	case n == 1:
	return 1;
	default:
	return inner(n-1) + inner(n-2)
	}

MarkRoddy / pig_counters.py

Created November 8, 2013 00:56


	from org.apache.pig.tools.pigstats import PigStatusReporter

	def increment(group, name, value = 1):
	status_reporter = PigStatusReporter.getInstance()
	if status_reporter != None:
	counter = status_reporter.getCounter(group, name)
	if counter:
	counter.increment(value);

MarkRoddy / gist:6936926

Created October 11, 2013 15:38

	pmroddy@mroddy-local-vm:~$ python
	Python 2.7.3 (default, Apr 10 2013, 05:46:21)
	[GCC 4.6.3] on linux2
	Type "help", "copyright", "credits" or "license" for more information.
	>>> int.__add__ = lambda x: 5
	Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
	TypeError: can't set attributes of built-in/extension type 'int'

MarkRoddy / gist:6906317

Created October 9, 2013 18:57

	mroddy@mroddy-local-vm:~$ python
	Python 2.7.3 (default, Apr 10 2013, 05:46:21)
	[GCC 4.6.3] on linux2
	Type "help", "copyright", "credits" or "license" for more information.
	>>> if float > 2.3:
	... print "wat?"
	...
	wat?
	>>>

MarkRoddy / gist:6717509

Created September 26, 2013 17:23


	Simius Quies: Block all networking on the instance using EC2 security groups (network failure)
	Simius Desertus: Null-route the 10.0.0.0 network (failure of the internal EC2 network)
	Simius Perditus: Cause packet loss (degradation of EC2 network)
	Simius Tardus: Add packet latency (degradation of EC2 network)
	Simius Politicus: Cause packet corruption (degradation of EC2 network)
	Simius Nonomenius: Block all DNS traffic (failure of DNS servers)
	Simius Amnesius: Null-route S3 traffic (S3 failure)
	Simius Noneccius: Null-route EC2 API traffic (EC2 control plane failure)
	Simius Nodynamus: Null-route DynamoDB traffic (DynamoDB failure)

MarkRoddy / inner.html

Created September 23, 2013 23:30

Only via Firefox: Javascript running inside an iframe that is hidden does not report the appropriate size of elements. If you remove style="display: none" from the iframe the correct size is reported.

	<html>
	<script src="http://code.jquery.com/jquery-1.8.2.js"></script>
	<body>
	Inner HTML
	<div class="some-container">Inside Container</div>
	<script>
	alert("Container width: " + $(".some-container").width());
	alert("Container outer width: " + $(".some-container").outerWidth());
	</script>
	</body>

MarkRoddy / millionsong.missing.store.pig

Last active December 21, 2015 09:09

	/*
	* Welcome to Mortar!
	*
	* STEP 1: Click ILLUSTRATE. You'll see a data sample flowing through every step of the script.
	*
	* STEP 2: Pull some additional data fields into the output. Add the tempo, artist_hotness, and year fields to
	* the 'song_density' alias's field list and press ILLUSTRATE again.
	*
	* STEP 3: Try running the script against the cluster. Click RUN, and run your job on a 2-node cluster.
	*/

MarkRoddy / top_density_songs

Last active December 21, 2015 09:09

	HadoopVersion PigVersion UserId StartedAt FinishedAt Features
	1.0.3 vagrant 2013-08-20 15:13:38 2013-08-20 15:21:36 ORDER_BY,FILTER,LIMIT

	Success!

	Job Stats (time in seconds):
	JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
	job_201308131552_0015 1 0 312 312 312 312 0 0 0 0 filtered_songs,song_density,songs MAP_ONLY
	job_201308131552_0016 1 1 6 6 6 6 12 12 12 12 density_ordered SAMPLER
	job_201308131552_0017 1 1 6 6 6 6 12 12 12 12 density_ordered ORDER_BY,COMBINER

MarkRoddy / gist:6137124

Created August 2, 2013 02:37

	Pig will short-circuit Boolean operations when possible. If the first (left) predicate of an and is false,
	the second (right) will not be evaluated. So in 1 == 2 and udf(x), the UDF will never be invoked.
	Similarly, if the first predicate of an or is true, the second predicate will not be evaluted. 1 == 1 or
	udf(x) will never invoke the UDF.

	http://chimera.labs.oreilly.com/books/1234000001811/ch05.html#filter

MarkRoddy / Map Reduce Plan

Created August 2, 2013 02:31

	2013-08-01 22:22:37,977 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimi\
	zer - MR plan size before optimization: 1
	2013-08-01 22:22:37,991 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimi\
	zer - MR plan size after optimization: 1
	#--------------------------------------------------
	# Map Reduce Plan
	#--------------------------------------------------
	MapReduce node 1-51
	Map Plan
	Store(fakefile:org.apache.pig.builtin.PigStorage) - 1-50

Mark Roddy MarkRoddy