robinkraft’s gists

robinkraft / indexed_image.py

Created February 13, 2012 03:05

Convert an image file readable by GDAL into a row-indexed text file

	import numpy as np
	from osgeo import gdal

	def progress(n, interval=100):
	"Print progress at regular intervals"
	# don't want to divide by zero on first try
	if n > 0:
	if n % interval == 0:
	print "Parsing item %i" % n
	return

robinkraft / first_cascalog_queries.clj

Created February 13, 2012 03:10

Explorations with Cascalog

	;; Sam, I got up to 21, some of which now reside forma-clj

	(def nums
	[1
	2
	3
	4
	5
	6])

robinkraft / drops.do

Created February 13, 2012 19:23

what to drop from FORMA data

	drop if prob201108 == . & hansen == 0
	drop country ecoid gadm coastdist modh modv sample line prob200512
	drop if prob201108 < 50 & hansen == 0
	outsheet using "/Users/robin/Dropbox/code/forma-app/data/IDN.csv", comma noquote

robinkraft / gist:1829381

Created February 14, 2012 19:16

csv-ify some ints and a vector

robinkraft / gist:1952544

Created March 1, 2012 19:41

Want to define a function at master node repl, have it accessible in query

	(defn get-max
	[v]
	(apply max (flatten v)))

	(let [src (hfs-seqfile "s3n://formaresults/finaloutput/*")]
	(??<- [?count]
	(src ?s-res ?mod-h ?mod-v ?s ?l ?prob-series)
	(get-max ?prob-series :> ?max)
	(>= ?max 0.5)
	(c/count ?count)))

robinkraft / reli.clj

Created March 29, 2012 00:51

	(defmain through-reli
	[tmp-root pail-path ts-pail-path out-path run-key country-seq]
	(let [{:keys [s-res t-res est-end] :as est-map} (forma-run-parameters run-key)
	mk-filter (fn [vcf-path ts-src]
	(forma/filter-query (hfs-seqfile vcf-path)
	(:vcf-limit est-map)
	ts-src))]
	(assert est-map (str run-key " is not a valid run key!"))
	(workflow [tmp-root]
	vcf-step

robinkraft / gist:2348543

Created April 10, 2012 05:53

sample Cascalog query FOSS4G

	(defn modis->latlon
	"Converts the supplied MODIS coordinates into `[lat, lon]` based on
	the supplied resolution.

	Example usage:

	(modis->latlon \"1000\" 8 6 12 12)
	;=> (29.89583333333333 -115.22901262147285)"
	[res mod-h mod-v sample line]
	(->> (modis->global-mags res mod-h mod-v sample line)

robinkraft / gist:2472994

Created April 23, 2012 18:44

notes on converting latlon to google maps tile

	;; work in progress!

	(defn mf
	[lat lon zoom]
	(let [min-lat -90]))

	(defn my-func
	[lat lon zoom]
	(let [tile-range (bit-shift-left 1 zoom)
	y lat

robinkraft / gist:2490641

Created April 25, 2012 15:25

Convert period in original output for Vizzuality from 16-day to monthly periodicity

	(defn convert-16->32
	"Convert period in original output for Vizzuality from 16-day to monthly periodicity.

	For common data model"
	[]
	(let [in-res "16"
	out-res "32"
	start-epoch (date/datetime->period in-res "2000-01-01")
	start-epoch-new-res (date/datetime->period out-res "2000-01-01")
	src (hfs-seqfile "s3n://formaresults/analysis/xyzperiod")

robinkraft / gist:2496318

Created April 26, 2012 05:45

First shot at using Hadoop with CartoDB, courtesy of Tim Robertson of GBIF

	(ns validation
	(:use cascalog.api))

	(defn build-query
	[lat lon]
	(let [lat-f (float lat)
	lon-f (float lon)
	;;url_head "http://timrobertson100.cartodb.com/api/v2/sql?q="
	;;query_stub "select%20iso_a2%20from%20countries%20WHERE%20ST_DWithin(the_geom,%20ST_SetSRID(ST_Point("
	url_head "http://viz2.cartodb.com/api/v2/sql?q="

Robin Kraft robinkraft