Skip to content

Instantly share code, notes, and snippets.

View hrwgc's full-sized avatar

Chris Herwig hrwgc

  • Google
  • San Francisco
View GitHub Profile
@hrwgc
hrwgc / mercantile-tiles-from-geojson.sh
Last active August 29, 2015 14:11
mercantile find web mercator tiles for a given zoom level based on geojson feature collection
#!/bin/bash
function to_tiles(){
geojson=$1;
zoom=$2;
jq -c .features[] < "$geojson" | mercantile tiles $zoom | sort | uniq
}
@hrwgc
hrwgc / aws-cli-s3cmd-du.sh
Last active June 19, 2023 15:32
aws-cli get total size of all objects within s3 prefix. (mimic behavior of `s3cmd du` with aws-cli)
#!/bin/bash
function s3du(){
bucket=`cut -d/ -f3 <<< $1`
prefix=`awk -F/ '{for (i=4; i<NF; i++) printf $i"/"; print $NF}' <<< $1`
aws s3api list-objects --bucket $bucket --prefix=$prefix --output json --query '[sum(Contents[].Size), length(Contents[])]' | jq '. |{ size:.[0],num_objects: .[1]}'
}
s3du $1;
@hrwgc
hrwgc / xml2json.js
Created September 17, 2014 18:28
convert xml to json nodejs command line utility
#!/usr/bin/env node
var parser = require('xml2json');
var fs = require('fs');
function onFile(err, data) {
if (err) {
console.log('Uh oh: ' + err);
return;
}
var json = parser.toJson(data);
@hrwgc
hrwgc / 1_how_to_fix.md
Last active December 29, 2015 11:19
postgres / postgis randomly stops working on osx mavericks - fix

Following advice of Psql: could not connect...:

Running the following command resets PostgreSQL's transaction log, which resolves the problem.

 pg_resetxlog -f /usr/local/Cellar/postgresql/9.3.1/data

you should replace /usr/local/Cellar/postgresql/9.3.1/data with your own pg_datadir (eg., pg_ctl -D $DATADIR start)

@hrwgc
hrwgc / validate.sh
Created November 13, 2013 19:57
bash wget - check if file exists at url before downloading
#!/bin/bash
# simple function to check http response code before downloading a remote file
# example usage:
# if `validate_url $url >/dev/null`; then dosomething; else echo "does not exist"; fi
function validate_url(){
if [[ `wget -S --spider $1 2>&1 | grep 'HTTP/1.1 200 OK'` ]]; then echo "true"; fi
}
Landsat 4-5 Wavelength Landsat 7 Wavelength Landsat 8 Wavelength (micrometers)
Band 1 - Coastal aerosol 0.43 - 0.45
Band 1 0.45-0.52 Band 1 0.45-0.52 Band 2 - Blue 0.45 - 0.51
Band 2 0.52-0.60 Band 2 0.52-0.60 Band 3 - Green 0.53 - 0.59
Band 3 0.63-0.69 Band 3 0.63-0.69 Band 4 - Red 0.64 - 0.67
Band 4 0.76-0.90 Band 4 0.77-0.90 Band 5 - Near Infrared (NIR) 0.85 - 0.88
Band 9 - Cirrus 1.36 - 1.38
Band 5 1.55-1.75 Band 5 1.55-1.75 Band 6 - SWIR 1 1.57 - 1.65
Band 7 2.08-2.35 Band 7 2.09-2.35 Band 7 - SWIR 2 2.11 - 2.29
@hrwgc
hrwgc / clean_git.sh
Created October 22, 2013 14:49
remove all non master branches from git repo
function clean_git_branches(){
git checkout master;
BRANCHES=`git branch | grep -Ev '^\s{0,}\*{0,}\s{0,}master$'`
git branch -D ${BRANCHES[@]}
}
@hrwgc
hrwgc / clipboard.py
Last active December 20, 2015 01:38
#!/usr/bin/env python
import os
def CopyToClipboard(text):
cmd = "echo '%s' | tr -d \"\\\n\" | pbcopy" % text
os.system(cmd)
@hrwgc
hrwgc / nearest_neighbor.sql
Last active December 19, 2015 06:19
postgis select across polygon layers based on nearest adjacent polygon from other layer.
create table
closest_point as
select distinct on (a.featureid) a.featureid as image_id,
(select b.eventid from fl_events as b order by st_distance(a.wkb_geometry,b.wkb_geometry) limit 1) as event_id,
a.wkb_geometry from images as a,
fl_events as b;
update images set eventid = closest_point.event_id from closest_point where images.featureid = closest_point.image_id;