Jacques Tardie jqtrde

Storytelling from Space: Tools/Resources

This list of resources is all about acquring and processing aerial imagery. It's generally broken up in three ways: how to go about this in Photoshop/GIMP, using command-line tools, or in GIS software, depending what's most comfortable to you. Often these tools can be used in conjunction with each other.

Acquiring Landsat & MODIS

Web Interface

USGS Earth Explorer - Browser and data access (create a login)
http://earthexplorer.usgs.gov/

Landsat archive

MODIS downloads are stored as .hdf files which makes them non-trivial to interact with.

I've previously mentioned using HEG but for a more streamlined approach we're going to want to use gdal.

Compiling GDAL with HDF4 support can actually be done fairly easily using brew although information on this was a bear to find. Thanks to Wood Sun we can do this via three commands

brew tap homebrew/versions
brew install hdf4
brew install https://raw.githubusercontent.com/geoalchimista/homebrew/master/formulae/gdal.rb --with-hdf4

Deploy your own PaaS!

Setting up Dokku with DigitalOcean and Namecheap

..or how I made my own heroku in a few hours for $3.98.

This write-up owes a great deal to dscape's Node.js Deployments with Docker, Dokku, & Digital Ocean, the dokku project itself, and the fine folks working on dokku's issues. I took dscape's article as a starting point when trying this out but found some details lacking so I documented my own process for getting dokku up and running.

1. Get a domain

General Background and Overview

Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
Models and Issues in Data Stream Systems
Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
[Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

Quick idea to understand what modules get imported.

Example:

>>> import importlog

# Run a bunch of code
>>> import shapely.geometry
>>> p = shapely.geometry.Point(0.0, 0.0)
&gt;&gt;&gt; b = p.buffer(1.0)

	<!DOCTYPE html>
	<html>
	<head>
	<title>Foo</title>
	<meta charset='utf-8' />
	<meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=0' />
	<style type='text/css'>
	div {
	cursor:pointer;
	cursor:hand;

	# https://www.mapbox.com/blog/processing-landsat-8/
	task :landsat_2014 => :environment do
	dirs = Dir["#{Rails.root.to_s}/db/initial/raster/*"].reject {\|q\| q =~ /tar\.gz/ }
	dirs.each do \|dir\|
	scene_id = dir.split("/")[-1]
	[4,3,2].each do \|band\|
	`gdalwarp -t_srs EPSG:3857 #{dir}/#{scene_id}_B#{band}.TIF #{dir}/#{scene_id}_B#{band}-projected.tif`
	end
	`convert -combine #{dir}/#{scene_id}_B{4,3,2}-projected.tif #{dir}/#{scene_id}_RGB-projected.tif && \
	convert -channel B -gamma 0.925 -channel R -gamma 1.03 -channel RGB -sigmoidal-contrast 50x16% #{dir}/#{scene_id}_RGB-projected.tif #{dir}/#{scene_id}_RGB-projected-corrected.tif && \

	<!DOCTYPE html>
	<html>
	<head>
	<meta charset='utf-8' />
	<title></title>
	<meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />
	<script src='https://api.tiles.mapbox.com/mapbox-gl-js/v0.7.0/mapbox-gl.js'></script>
	<script src='tx.js'></script>
	<link href='https://api.tiles.mapbox.com/mapbox-gl-js/v0.7.0/mapbox-gl.css' rel='stylesheet' />
	<style>