Skip to content

Instantly share code, notes, and snippets.

View gka's full-sized avatar
🐢

Gregor Aisch gka

🐢
View GitHub Profile
@gka
gka / WikipediaPlainTables
Created August 3, 2011 17:09
Ever tried to copy&paste a table from Wikipedia to your favourite spreadsheet software? You just want the pure data and not all those hyperlinks and embedded images? This script removes unnecessary stuff from the tables. Simply run in firebug console and
// remove all images
$('table img').remove();
// remove all reference links
$('table .reference').remove()
// convert all links to plain text
$('table a').each(function(index, el) { $(el).replaceWith($(el).html()) });
@gka
gka / twitter_follower_intersect.py
Created September 15, 2011 16:16
Looks up how many followers two Twitter accounts do have in common
#!/usr/bin/env python
"""
This script looks up how many followers two different Twitter accounts do have in common.
Usage:
twitter_follower_intersect.py username username
You'll need Python and the Python Twitter Tools to get this running.
@gka
gka / csv2tsv
Created September 16, 2011 12:20
Converting a comma-separated and quoted file into a tab-separated and unquoted file
#!/usr/bin/env python2.7
"""
No more trouble with comma-separated and quoted CSV files.
"""
import csv, sys
if len(sys.argv) != 3: print 'Usage:\ncsv2tsv fromfile.csv tofile.tsv'
file_in = sys.argv[1]
file_out = sys.argv[2]
@gka
gka / gisutils.py
Created October 6, 2011 14:03
Some GIS utils for Python, like area computation for geo-polygons
"""
implementation taken from
http://forum.worldwindcentral.com/showthread.php?t=20724
algorithm expects a list of [lng,lat] pairs in degrees
"""
def haversine(x):
from math import cos
return ( 1.0 - cos(x) ) / 2.0
@gka
gka / shape_center.py
Created October 26, 2011 07:58
Computes the center of a shapefile multi-polygon
"""
computes the center of a multi-polygon
"""
def shape_center(shape):
"""
computes the center of gravity of a shapefile multi-polygon
shape must be an instance of shapefile._Shape of the Python Shapefile Library
http://packages.python.org/Python%20Shapefile%20Library/
Polygon class comes from here
http://pypi.python.org/pypi/Polygon/1.17
@gka
gka / USA-sat.svg
Created November 22, 2011 12:25
Map of US cities and state poverty levels
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@gka
gka / eu-disposable-incomes.csv
Created November 22, 2011 18:06
NUTS-2 Map of the European Union
We can make this file beautiful and searchable if this error is corrected: It looks like row 2 should actually have 3 columns, instead of 1 in line 1.
indic_na,unit,geo\time 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
AT11 11523.9 11771.1 12318.3 13027.5 13923.5 13937.3 14402.1 15052.7 16057.0 16840.4 17825.5 18470.1
AT12 12871.0 13021.4 13665.6 14525.4 15537.4 15403.4 15765.8 16322.2 16928.0 17831.3 18813.9 19544.9
AT13 14683.3 14857.3 15374.7 16069.4 17056.0 16630.1 16928.8 17261.8 17699.9 18279.5 18977.4 19485.4
AT21 11795.8 11922.2 12421.4 13006.9 14042.2 13967.3 14393.7 14954.2 15521.1 16523.0 17420.1 18127.9
AT22 12031.6 12222.0 12755.1 13429.9 14425.1 14242.8 14685.7 15180.8 15686.9 16638.9 17560.7 18347.0
AT31 12460.4 12643.3 13179.3 13796.6 14869.5 14654.6 15008.0 15522.6 16281.2 17261.5 18266.9 18949.4
AT32 12879.1 12985.2 13544.3 14257.6 15264.9 15007.9 15416.9 15921.1 16505.8 17563.2 18509.2 19289.0
AT33 12029.3 12018.6 12697.8 13400.2 14507.7 14402.3 15030.8 15632.8 16203.9 17043.9 17868.1 18588.3
AT34 12591.4 12620.3 13488.1 14107.7 15568.7 15154.4 15641.8 16167.9 16716.1 17793.2 18803.9 19367.5
@gka
gka / cartogram.py
Created December 5, 2011 12:09
Generator for Dorling cartograms
"""
Generator for packed circle cartograms
"""
import proj, gisutils
class Cartogram:
def loadCSV(self, url, key='id', value='val', lon='lon', lat='lat'):
import csv
doc = csv.reader(open(url))
@gka
gka / zip2nuts.py
Created December 13, 2011 23:37
This script tries to map European zip codes to NUTS2 regions by point-in-polygon checking the zip coordinate against a shapefile of NUTS regions..
#!/usr/bin/env python2.7
import csv, shapefile, Polygon, Polygon.IO, math
# shapefile comes from here:
# http://epp.eurostat.ec.europa.eu/cache/GISCO/geodatafiles/NUTS_10M_2006_SH.zip
shpsrc = "eu-nuts-2.shp"
# zip code database comes from here:
# http://www.clearlyandsimply.com/files/2010/10/european_cities_and_postcodes_us_standard.zip
@gka
gka / scrape_party_members.py
Created December 26, 2011 22:41
prototype for scraper of german party members from wikipedia
# coding=utf-8
from wikitools import wiki
from wikitools import api
site = wiki.Wiki('http://de.wikipedia.org/w/api.php')
#
# returns a list of categories
#
def getPageCategories(pages):