Skip to content

Instantly share code, notes, and snippets.

View pnorman's full-sized avatar

Paul Norman pnorman

View GitHub Profile
@pnorman
pnorman / install.md
Last active September 15, 2019 15:26
Draft install instructions for osm2pgsql + carto + renderd

Manually building a tile server

This page describes how to install, setup and configure all the necessary software to operate your own tile server. The step-by-step instructions are written for Ubuntu Linux 12.04 LTS (Precise Pangolin), however they should transfer fairly straightforwardly to other versions of Ubuntu or Linux distributions.

##Software installation The OSM tile server stack is a collection of programs and libraries that work together to create a tile server. As so often with OpenStreetMap, there are many ways to achieve this goal and nearly all of the components have alternatives that have various specific advantages and disadvantages. This tutorial describes the most standard version that is also used on the main OpenStreetMap.org tile server.

This guide covers installation of osm2pgsql, loading a PostgreSQL/PostGIS database, and rendering tiles for an online webmap.The database can also be used to [develop stylesheets], or render data with other software.

Before starting you want t

@pnorman
pnorman / statements.sql
Created September 30, 2013 08:59
SQL statements from openstreetmap-carto
(select way, religion,
coalesce (aeroway, amenity, landuse, leisure, military, "natural", power, tourism, highway) as feature from (
select way,
('aeroway_' || (case when aeroway in ('apron', 'aerodrome') then aeroway else null end)) as aeroway,
('amenity_' || (case when amenity in ('parking', 'university', 'college', 'school', 'hospital', 'kindergarten', 'grave_yard') then amenity else null end)) as amenity,
('landuse_' || (case when landuse in ('quarry', 'vineyard', 'orchard', 'cemetery', 'grave_yard', 'residential', 'garages', 'field', 'meadow', 'grass', 'allotments', 'forest', 'farmyard', 'farm', 'farmland', 'recreation_ground', 'conservation', 'village_green', 'retail', 'industrial', 'railway', 'commercial', 'brownfield', 'landfill', 'greenfield', 'construction') then landuse else null end)) as landuse,
('leisure_' || (case when leisure in ('swimming_pool', 'playground', 'park', 'recreation_ground', 'common', 'garden', 'golf_course') then leisure else null end)) as leisure,
('military_' || (case when mil
@pnorman
pnorman / carto_1k_x4.log
Last active December 24, 2015 13:49
Analysis of slow postgresql queries from 1k meta-tiles, repeated 4x
Restricted to queries>.25s
Standard indexes plus index for non-building polygons.
SELECT ST_AsBinary("way") AS geom,"landuse","natural","waterway" FROM (select way,"natural",waterway,landuse,name from planet_osm_polygon where (waterway in ('dock','mill_pond','riverbank','canal') or landuse in ('reservoir','water','basin') or "natural" in ('lake','water','land','glacier','mud')) and building is null order by z_order,way_area desc ) as water_areas WHERE "way" && <bbox>
* probable originating layers: none found
zoom count min time avg time max time tot time
---------------------------------------------------
7 16 21.9s 22.8s 24.9s 364.8
13 271 0.2s 0.8s 2.5s 216.8
@pnorman
pnorman / gist:6983476
Created October 14, 2013 22:40
query for node in building analysis
SELECT COUNT(*)
FROM planet_osm_point AS pt
JOIN planet_osm_polygon AS pl ON ST_Intersects(pt.way, pl.way)
WHERE pt.amenity='fast_food'
AND pl.building IS NOT NULL;
@pnorman
pnorman / osm2pgsql_speed_commands.sh
Last active December 25, 2015 17:59
osm2pgsql speed benchmark commands
sudo apt-get update && sudo apt-get -y dist-upgrade && sudo shutdown -r now
sudo -i
cat > /etc/apt/sources.list.d/pgdg.list <<EOF
deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main
#deb-src http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main
EOF
wget --quiet -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc | apt-key add -
apt-get --no-install-recommends install -y python-software-properties
add-apt-repository -y ppa:chris-lea/node.js
@pnorman
pnorman / gist:7030961
Last active December 25, 2015 19:49
packages used by a server loading osm data and running tilemill
The following NEW packages will be installed:
apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common autoconf automake autotools-dev binutils comerr-dev cpp cpp-4.6 fontconfig fontconfig-config
g++ g++-4.6 gawk gcc gcc-4.6 gccxml gdal-bin git git-man iotop krb5-multidev libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libarmadillo2
libblas3gf libboost-date-time1.49-dev libboost-date-time1.49.0 libboost-filesystem-dev libboost-filesystem1.49-dev libboost-filesystem1.49.0
libboost-program-options-dev libboost-program-options1.49-dev libboost-program-options1.49.0 libboost-python-dev libboost-python1.49-dev
libboost-python1.49.0 libboost-regex-dev libboost-regex1.49-dev libboost-regex1.49.0 libboost-serialization1.49-dev libboost-serialization1.49.0
libboost-system-dev libboost-system1.49-dev libboost-system1.49.0 libboost-thread-dev libboost-thread1.49-dev libboost-thread1.49.0 libboost1.49-dev
libbz2-dev libc-dev-bin libc6-dev libcairo-gobject2 libcairo-script-interpreter2 li
@pnorman
pnorman / gist:7064619
Created October 20, 2013 03:19
osm2pgsql highway distribution
value frequency
residential 25.91%
IS NULL 20.17%
service 8.58%
unclassified 6.88%
track 6.45%
footway 3.39%
tertiary 3.03%
path 2.26%
secondary 1.85%
# First, clear out any old mapnik or node.js installs that might conflict
sudo apt-get purge libmapnik libmapnik-dev mapnik-utils nodejs npm
# Also clear out any old ppa's that might conflict
sudo rm /etc/apt/sources.list.d/*mapnik*
sudo rm /etc/apt/sources.list.d/*developmentseed*
sudo rm /etc/apt/sources.list.d/*chris-lea*
# add new ppa's
sudo apt-add-repository -y ppa:chris-lea/node.js
@pnorman
pnorman / rowpatternmatch.sql
Created December 2, 2013 00:09
For example, that pattern describes a trading day during which the stock of XYZ Corp. began the day at a price of $100/share, then steadily fell throughout the morning until it hit a price of $50/share at noon, after which the price rose throughout the afternoon until it hit a peak of $150/share at 15:00 and then began to decline again. The info…
SELECT a_symbol, /* stock symbol */
a_tstamp, /* start time */
a_price, /* start price */
max_c_tstamp, /* inflection time */
last_c_price, /* lowest price */
max_f_tstamp, /* end time */
last_f_price, /* end price */
matchno
FROM stock_ticker
MATCH_RECOGNIZE (