doug ybarbo alexland

scala, spark, akka dev designing & building distributed systems--data pipelines, machine learning systems, search

alexland / index.html

Last active August 29, 2015 14:08


	<!DOCTYPE html>
	<head>
	<meta charset="utf-8">


	<meta http-equiv="X-UA-Compatible" content="IE=edge">
	<title>dynamic data in d3: data join & update-append-exit pattern</title>
	<meta name="description" content="">
	<meta name="viewport" content="width=device-width, initial-scale=1"><!-- Place favicon.ico and apple-touch-icon.png in the root directory -->

alexland / max_diff.py

Last active August 29, 2015 14:09

calculating max diff for a 1D sequence


	def fnx(arr):
	'''
	pass in: NumPy 1d array
	returns: 3-tuple (idx_min_val, idx_max_val, max_diff)
	'''
	arr = arr.reshape(-1, 1)
	arr_diff = arr.T - arr
	idx = NP.unravel_index(arr_diff.argmax(), arr_diff.shape)
	return (idx[0], idx[1], arr_diff[idx])

alexland / offset_merge_arrays.py

Last active August 29, 2015 14:11

in-place merge two halves of an array w/ specified offset


	>>> data = NP.arange(20)
	>>> data
	array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19])

	# use floor division to account for an odd number of items in the array,
	# ie, 20//2 and 21//2 return the same result
	>>> mp = data.shape[0] // 2

	'''

alexland / fizzbuzz_numpy.py

Last active May 23, 2022 23:56

fizz buzz test in three short lines with NumPy

	'''
	fizzbuzz refers to a quick test to filter applicants for programmer jobs who don't actually know how to code.
	(see eg, http://imranontech.com/2007/01/24/using-fizzbuzz-to-find-developers-who-grok-coding/)

	"pass in a sequence of integers from 1 to 100;
	for integers that are multiples of three: print “fizz”
	for integers that are multiples of five print “buzz”
	for integers that are multiples of five AND three print "fizzbuzz"
	for remaining integers, print the integer value
	'''

alexland / typed_mv.py

Created January 30, 2015 03:33

typed memoryviews in NumPy using the python buffer protocol (PEP 3118)


	import numpy as NP

	d = NP.random.randint(1, 10, 40).reshape(8, 5)

	mv = memoryview(d)

	def mem_addr(arr):
	'''
	returns: location of a NumPy array in memory, or

alexland / pipe-operator-in-scala

Last active August 29, 2015 14:20 — forked from SteveGilham/gist:e9ef541553e09be75994

pipe operator implemented in scala

	package operator

	object FunctionalPipeline {
	class PipedObject[T] private[Functional] (value:T)
	{
	def \|>[R] (f : T => R) = f(this.value)
	}
	implicit def toPiped[T] (value:T) = new PipedObject[T](value)
	}

alexland / spark-custom-install.sh

Last active August 29, 2015 14:20

apache spark custom installation using sbt and current scala version

	#---------------- custom install Apache Spark -------------------#

	# instructions to build Apache Spark from source w/ current Scala & using sbt (vs maven)


	# download Apache Spark src from the appropriate mirror


	# untar:
	tar zxf spark-1.3.1.tgz

alexland / geojson-conversion.sh

Last active August 29, 2015 14:21 — forked from benbalter/geojson-conversion.sh

	# Bulk convert shapefiles to geojson using ogr2ogr
	# For more information, see http://ben.balter.com/2013/06/26/how-to-convert-shapefiles-to-geojson-for-use-on-github/

	# Note: Assumes you're in a folder with one or more zip files containing shape files
	# and Outputs as geojson with the crs:84 SRS (for use on GitHub or elsewhere)

	#geojson conversion
	function shp2geojson() {
	ogr2ogr -f GeoJSON -t_srs crs:84 "$1.geojson" "$1.shp"
	}

alexland / sbtmkdirs.sh

Last active August 29, 2015 14:26 — forked from alvinj/sbtmkdirs.sh

A shell script to create an SBT project directory structure

	#!/bin/bash

	#------------------------------------------------------------------------------
	# Name: sbtmkdirs
	# Purpose: Create an SBT project directory structure with a few simple options.
	# Author: Alvin Alexander, http://alvinalexander.com
	# Info: http://alvinalexander.com/sbtmkdirs
	# License: Creative Commons Attribution-ShareAlike 2.5 Generic
	# http://creativecommons.org/licenses/by-sa/2.5/
	#------------------------------------------------------------------------------

alexland / postgres-cheatsheet.md

Last active August 29, 2015 14:26 — forked from Kartones/postgres-cheatsheet.md

PostgreSQL command line cheatsheet

Magic words:

psql -U postgres

Most \d commands support additional param of __schema__.name__ and accept wildcards like *.*