Skip to content

Instantly share code, notes, and snippets.

View pnorman's full-sized avatar

Paul Norman pnorman

View GitHub Profile
# First, clear out any old mapnik or node.js installs that might conflict
sudo apt-get purge libmapnik libmapnik-dev mapnik-utils nodejs npm
# Also clear out any old ppa's that might conflict
sudo rm /etc/apt/sources.list.d/*mapnik*
sudo rm /etc/apt/sources.list.d/*developmentseed*
sudo rm /etc/apt/sources.list.d/*chris-lea*
# add new ppa's
sudo apt-add-repository -y ppa:chris-lea/node.js
@pnorman
pnorman / gist:7064619
Created October 20, 2013 03:19
osm2pgsql highway distribution
value frequency
residential 25.91%
IS NULL 20.17%
service 8.58%
unclassified 6.88%
track 6.45%
footway 3.39%
tertiary 3.03%
path 2.26%
secondary 1.85%
@pnorman
pnorman / gist:7030961
Last active December 25, 2015 19:49
packages used by a server loading osm data and running tilemill
The following NEW packages will be installed:
apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common autoconf automake autotools-dev binutils comerr-dev cpp cpp-4.6 fontconfig fontconfig-config
g++ g++-4.6 gawk gcc gcc-4.6 gccxml gdal-bin git git-man iotop krb5-multidev libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libarmadillo2
libblas3gf libboost-date-time1.49-dev libboost-date-time1.49.0 libboost-filesystem-dev libboost-filesystem1.49-dev libboost-filesystem1.49.0
libboost-program-options-dev libboost-program-options1.49-dev libboost-program-options1.49.0 libboost-python-dev libboost-python1.49-dev
libboost-python1.49.0 libboost-regex-dev libboost-regex1.49-dev libboost-regex1.49.0 libboost-serialization1.49-dev libboost-serialization1.49.0
libboost-system-dev libboost-system1.49-dev libboost-system1.49.0 libboost-thread-dev libboost-thread1.49-dev libboost-thread1.49.0 libboost1.49-dev
libbz2-dev libc-dev-bin libc6-dev libcairo-gobject2 libcairo-script-interpreter2 li
@pnorman
pnorman / osm2pgsql_speed_commands.sh
Last active December 25, 2015 17:59
osm2pgsql speed benchmark commands
sudo apt-get update && sudo apt-get -y dist-upgrade && sudo shutdown -r now
sudo -i
cat > /etc/apt/sources.list.d/pgdg.list <<EOF
deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main
#deb-src http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main
EOF
wget --quiet -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc | apt-key add -
apt-get --no-install-recommends install -y python-software-properties
add-apt-repository -y ppa:chris-lea/node.js
@pnorman
pnorman / gist:6983476
Created October 14, 2013 22:40
query for node in building analysis
SELECT COUNT(*)
FROM planet_osm_point AS pt
JOIN planet_osm_polygon AS pl ON ST_Intersects(pt.way, pl.way)
WHERE pt.amenity='fast_food'
AND pl.building IS NOT NULL;
@pnorman
pnorman / carto_1k_x4.log
Last active December 24, 2015 13:49
Analysis of slow postgresql queries from 1k meta-tiles, repeated 4x
Restricted to queries>.25s
Standard indexes plus index for non-building polygons.
SELECT ST_AsBinary("way") AS geom,"landuse","natural","waterway" FROM (select way,"natural",waterway,landuse,name from planet_osm_polygon where (waterway in ('dock','mill_pond','riverbank','canal') or landuse in ('reservoir','water','basin') or "natural" in ('lake','water','land','glacier','mud')) and building is null order by z_order,way_area desc ) as water_areas WHERE "way" && <bbox>
* probable originating layers: none found
zoom count min time avg time max time tot time
---------------------------------------------------
7 16 21.9s 22.8s 24.9s 364.8
13 271 0.2s 0.8s 2.5s 216.8
@pnorman
pnorman / statements.sql
Created September 30, 2013 08:59
SQL statements from openstreetmap-carto
(select way, religion,
coalesce (aeroway, amenity, landuse, leisure, military, "natural", power, tourism, highway) as feature from (
select way,
('aeroway_' || (case when aeroway in ('apron', 'aerodrome') then aeroway else null end)) as aeroway,
('amenity_' || (case when amenity in ('parking', 'university', 'college', 'school', 'hospital', 'kindergarten', 'grave_yard') then amenity else null end)) as amenity,
('landuse_' || (case when landuse in ('quarry', 'vineyard', 'orchard', 'cemetery', 'grave_yard', 'residential', 'garages', 'field', 'meadow', 'grass', 'allotments', 'forest', 'farmyard', 'farm', 'farmland', 'recreation_ground', 'conservation', 'village_green', 'retail', 'industrial', 'railway', 'commercial', 'brownfield', 'landfill', 'greenfield', 'construction') then landuse else null end)) as landuse,
('leisure_' || (case when leisure in ('swimming_pool', 'playground', 'park', 'recreation_ground', 'common', 'garden', 'golf_course') then leisure else null end)) as leisure,
('military_' || (case when mil
@pnorman
pnorman / install.md
Last active September 15, 2019 15:26
Draft install instructions for osm2pgsql + carto + renderd

Manually building a tile server

This page describes how to install, setup and configure all the necessary software to operate your own tile server. The step-by-step instructions are written for Ubuntu Linux 12.04 LTS (Precise Pangolin), however they should transfer fairly straightforwardly to other versions of Ubuntu or Linux distributions.

##Software installation The OSM tile server stack is a collection of programs and libraries that work together to create a tile server. As so often with OpenStreetMap, there are many ways to achieve this goal and nearly all of the components have alternatives that have various specific advantages and disadvantages. This tutorial describes the most standard version that is also used on the main OpenStreetMap.org tile server.

This guide covers installation of osm2pgsql, loading a PostgreSQL/PostGIS database, and rendering tiles for an online webmap.The database can also be used to [develop stylesheets], or render data with other software.

Before starting you want t

@pnorman
pnorman / data.json
Created September 11, 2013 22:45
WIP of json-schema schema for OSM data
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "OSM Data",
"description": "OSM Data, as returned by the API",
"type": "object",
"properties": {
"version": {
"description": "The OSM API version",
"type": "string",
"enum": ["0.6"]
@pnorman
pnorman / cgimap
Last active December 22, 2015 15:29
An Apache config to handle some requests with cgimap and send others to the main API server
# Adjust as needed for the DocumentRoot. You will need to create the api
# directory, but nothing need be in it
<Directory /var/www/api>
# This section could be placed in a .htaccess file in /var/www/api, but
# the ProxyPreserveHost directive can't. If you don't have access to the
# apache config you won't be able to get OAuth to work
# Enable rewrites
RewriteEngine on