Skip to content

Instantly share code, notes, and snippets.

@jsanz
Last active February 10, 2017 22:02
Show Gist options
  • Save jsanz/fcb8394e084919a4135188b7c30504ad to your computer and use it in GitHub Desktop.
Save jsanz/fcb8394e084919a4135188b7c30504ad to your computer and use it in GitHub Desktop.
SQL: Postgis workshop

Postgis queries workshop

With this viewer you will have a sandbox space to try different PostGIS capabilities using several datasets. Change the SQL and press Control+S or Command+S and check the results on the map on the right.

You can also play with the styles on the CartoCSS pane.

Contents

  • Transform to a different projection
  • Get the number of points inside a polygon
  • Know wether a geometry is within the given range from another geometry:
  • Create a buffer from points:
  • Get the difference between two geometries:
  • Create a straight line between two points:
  • Create great circles between two points:
  • Generating Grids with CDB functions

Transform to a different projection

SELECT
  cartodb_id,
  ST_Transform(the_geom, 54030) AS the_geom_webmercator
FROM
  ne_50m_land

About working with different projections in CARTO and ST_Transform.

Get the number of points inside a polygon

Using GROUP BY:

SELECT
  e.cartodb_id,
  e.admin,
  e.the_geom_webmercator,
  count(*) AS pp_count,
  sum(p.pop_max) as sum_pop
FROM
  ne_adm0_europe e
JOIN
  ne_10m_populated_places_simple p
ON
  ST_Intersects(p.the_geom, e.the_geom)
GROUP BY
  e.cartodb_id

Using LATERAL:

SELECT
  a.cartodb_id,
  a.admin AS name,
  a.the_geom_webmercator,
  counts.number_cities,
  counts.sum_pop
FROM
  ne_adm0_europe a
CROSS JOIN LATERAL
  (
    SELECT
      count(*) as number_cities,
      sum(pop_max) as sum_pop
    FROM
      ne_10m_populated_places_simple b
    WHERE
      ST_Intersects(a.the_geom, b.the_geom)
  ) AS counts

About ST_Intersects and Lateral JOIN


Note: Add this piece of CartoCSS at the end so you have a nice coropleth map:

#layer['mapnik::geometry_type'=3] {
  line-width: 0;
  polygon-fill: ramp([pp_count], ("#edd9a3","#f99178","#ea4f88","#a431a0","#4b2991"), quantiles(5));
}

This is using the new turbo-carto feature on CARTO to allow creating ramps from data without having to put the styles directly



Note: You know about the EXPLAIN ANALYZE function? use it to take a look on how both queries are pretty similar in terms of performance.


Know wether a geometry is within the given range from another geometry:

SELECT
  a.*
FROM
  ne_10m_populated_places_simple a,
  ne_10m_populated_places_simple b
WHERE
    a.cartodb_id != b.cartodb_id
  AND ST_DWithin(
      a.the_geom_webmercator,
      b.the_geom_webmercator,
      150000
    )
  AND a.adm0name = 'Spain'
  AND b.adm0name = 'Spain'

In this case, we are using the_geom_webmercator to avoid casting to geography type. Calculations made with geometry type takes the CRS units.

Keep in mind that CRS units in webmercator are not meters, and they depend directly on the latitude.

About ST_DWithin.

Create a buffer from points:

SELECT
  cartodb_id,
  name,
  ST_Transform(
    ST_Buffer(the_geom::geography, 250000)::geometry
    ,3857
  ) AS the_geom_webmercator
FROM
  ne_10m_populated_places_simple
WHERE
  name ilike 'trondheim'

Compare the result with

SELECT
  cartodb_id,
  name,
  ST_Transform(
    ST_Buffer(the_geom, 2)
    ,3857
  ) AS the_geom_webmercator
FROM
  ne_10m_populated_places_simple
WHERE
  name ilike 'trondheim'

Why this is not a circle?

About ST_Buffer.

Get the difference between two geometries:

SELECT
  a.cartodb_id,
    ST_Difference(
        a.the_geom_webmercator,
        b.the_geom_webmercator
  ) AS the_geom_webmercator
FROM
  ne_50m_land a,
  ne_adm0_europe b
WHERE
  b.adm0_a3 like 'ESP'

About ST_Difference.

Create a straight line between two points:

SELECT
  ST_MakeLine(
    a.the_geom_webmercator,
    b.the_geom_webmercator
  ) as the_geom_webmercator
FROM (
    SELECT * FROM ne_10m_populated_places_simple
    WHERE name ILIKE 'madrid'
  ) as a,
  (
    SELECT * FROM ne_10m_populated_places_simple
    WHERE name ILIKE 'barcelona'AND adm0name ILIKE 'spain'
  ) as b

About ST_MakeLine.

Create great circles between two points:

SELECT
  ST_Transform(
    ST_Segmentize(
      ST_Makeline(
        a.the_geom,
        b.the_geom
      )::geography,
      100000
    )::geometry,
  3857
  ) as the_geom_webmercator
FROM
  (SELECT * FROM ne_10m_populated_places_simple
  WHERE name ILIKE 'madrid') as a,
  (SELECT * FROM ne_10m_populated_places_simple
  WHERE name ILIKE 'new york') as b

About Great Circles.

Generating Grids with CDB functions

Rectangular grid

SELECT
  row_number() over () as cartodb_id,
  CDB_RectangleGrid(
    ST_Buffer(the_geom_webmercator,125000),
  250000,
  250000
  ) AS the_geom_webmercator
FROM
  ne_adm0_europe
WHERE
  adm0_a3 IN ('ITA','GBR')

About CDB_RectangleGrid

Adaptative Hexagonal grid

WITH grid AS
(SELECT
  row_number() over () as cartodb_id,
  CDB_HexagonGrid(
    ST_Buffer(the_geom_webmercator, 100000),
    100000
  ) AS the_geom_webmercator
FROM
  ne_adm0_europe
WHERE
  adm0_a3 IN ('ESP','ITA'))

SELECT
  grid.the_geom_webmercator,
  grid.cartodb_id
FROM
  grid, ne_adm0_europe a
WHERE
    ST_intersects(grid.the_geom_webmercator, a.the_geom_webmercator)
  AND a.adm0_a3 IN ('ESP','ITA')

About CDB_HexagonGrid

'use strict';
var DEFAULTS = {
endpoint: 'http://cartotraining.carto.com/api/v1/map',
sql: 'select * from ne_10m_populated_places_simple',
cartocss: document.getElementById("cartocss").textContent,
center: [0,0],
zoom: 2
};
var LOCAL_STORAGE_KEY = 'cartodb-viewer-config-sql-tests';
var Config = {
get: function() {
var config = {
endpoint: DEFAULTS.endpoint,
sql: DEFAULTS.sql,
cartocss: DEFAULTS.cartocss,
center: DEFAULTS.center,
zoom: DEFAULTS.zoom
};
if (window.localStorage) {
var storedConfig = localStorage.getItem(LOCAL_STORAGE_KEY);
if (storedConfig) {
try {
storedConfig = JSON.parse(storedConfig);
config.endpoint = storedConfig.endpoint || config.endpoint;
config.sql = storedConfig.sql || config.sql;
config.cartocss = storedConfig.cartocss || config.cartocss;
config.center = storedConfig.center || config.center;
config.zoom = storedConfig.zoom || config.zoom;
} catch (e) {
// pass
}
}
}
return config;
},
set: function(endpoint, sql, cartocss, center, zoom) {
var config = {};
if (endpoint !== DEFAULTS.endpoint) {
config.endpoint = endpoint;
}
if (sql !== DEFAULTS.sql) {
config.sql = sql;
}
if (cartocss !== DEFAULTS.cartocss) {
config.cartocss = cartocss;
}
config.center = center;
config.zoom = zoom;
if (window.localStorage) {
localStorage.setItem(LOCAL_STORAGE_KEY, JSON.stringify(config));
return true;
}
return false;
}
};
var map = L.map('map', {
scrollWheelZoom: false,
center: DEFAULTS.center,
zoom: DEFAULTS.zoom
});
function onMapChanged() {
console.log('Current zoom = %d', map.getZoom());
var center = [map.getCenter().lat, map.getCenter().lng];
Config.set(currentEndpoint(), sqlEditor.getValue(), cssEditor.getValue(), center, map.getZoom());
}
map.on('zoomend', onMapChanged);
map.on('dragend', onMapChanged);
L.tileLayer('http://{s}.basemaps.cartocdn.com/light_nolabels/{z}/{x}/{y}.png', {
attribution: '<a href="http://cartodb.com">CartoDB</a> © 2014',
maxZoom: 18
}).addTo(map);
L.tileLayer('http://{s}.basemaps.cartocdn.com/light_only_labels/{z}/{x}/{y}.png', {
attribution: '<a href="http://cartodb.com">CartoDB</a> © 2014',
maxZoom: 18
}).setZIndex(3).addTo(map);
var sqlEditor = CodeMirror.fromTextArea(document.getElementById('sql_editor'), {
theme: 'monokai',
lineNumbers: true,
lineWrapping: true,
mode: "text/x-plsql",
height: '400px'
});
var cssEditor = CodeMirror.fromTextArea(document.getElementById('css_editor'), {
theme: 'monokai',
lineNumbers: true,
lineWrapping: true,
mode: "text/x-scss",
height: "200px"
});
function tilesEndpoint(layergroupId) {
return currentEndpoint() + '/' + layergroupId + '/{z}/{x}/{y}.png?api_key=' + currentApiKey();
}
var tilesLayer = null;
function updateMap() {
if ( tilesLayer) {
map.removeLayer(tilesLayer);
}
var config = {
version: '1.5.0',
layers: [
{
type: 'cartodb',
options: {
sql: sqlEditor.getValue(),
cartocss: cssEditor.getValue(),
cartocss_version: '2.3.0'
}
}
]
};
var request = new XMLHttpRequest();
request.open('POST', currentEndpoint() + '?api_key=' + currentApiKey(), true);
request.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');
request.onload = function() {
if (this.status >= 200 && this.status < 400){
var layergroup = JSON.parse(this.response);
tilesLayer = L.tileLayer(tilesEndpoint(layergroup.layergroupid), {
maxZoom: 18
}).setZIndex(2).addTo(map);
onMapChanged();
} else {
throw 'Error calling server: Error ' + this.status + ' -> ' + this.response;
}
};
request.send(JSON.stringify(config));
}
function currentEndpoint() {
return document.getElementById('endpoint').value;
}
function currentApiKey() {
return document.getElementById('apikey').value;
}
document.getElementById('endpoint').addEventListener('blur', updateMap, false);
CodeMirror.commands.save = function() {
updateMap();
};
var config = Config.get();
document.getElementById('endpoint').value = config.endpoint;
sqlEditor.setValue(config.sql);
cssEditor.setValue(config.cartocss);
map.setView(config.center, config.zoom);
updateMap();
<!DOCTYPE html>
<html>
<head lang="en">
<meta charset="UTF-8">
<title></title>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/leaflet/0.7.3/leaflet.css" />
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/codemirror/4.7.0/codemirror.min.css">
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/codemirror/4.7.0/theme/monokai.min.css">
<style>
body {
margin: 0;
padding: 0;
border: 0;
height: 100%;
font-family: "Helvetica Neue Light", "HelveticaNeue-Light", "Helvetica Neue", Calibri, Helvetica, Arial, sans-serif;
}
.wrap {
width: 100%;
margin: 0 auto;
}
textarea {
padding: 0;
margin: 0;
border: solid 1px #999;
height: 64px;
}
.editor {
float: left;
width: 49%;
height: 100vh;
padding: 1%;
}
.editor p {
margin: 8px 0;
}
.editor label, .editor input, .editor select {
width: 80%;
margin-bottom: 8px;
margin-left: 10px;
display: block;
}
.CodeMirror {
float: left;
width: 100%;
height: 40vh;
margin-bottom: 8px;
}
#map {
width: 49%;
height: 100vh;
}
</style>
<style type="text/cartocss" id="cartocss">
#layer['mapnik::geometry_type' = 1] {
marker-fill: #F24440;
marker-opacity: 0.6;
marker-width: 8;
marker-line-color: darken(#F24440,10);
marker-line-width: 1;
marker-line-opacity: 0.9;
marker-placement: point;
marker-allow-overlap: true;
marker-comp-op: multiply;
}
#layer['mapnik::geometry_type'=2] {
line-color: #F24440;
line-width: 1.5;
line-opacity: 1;
}
#layer['mapnik::geometry_type'=3] {
polygon-fill: #F24440;
polygon-opacity: 0.6;
polygon-gamma: 0.5;
line-color: darken(#F24440,10);
line-width: 1;
line-opacity: 0.5;
line-comp-op: soft-light;
}
</style>
</head>
<body>
<div class="wrap">
<form class="editor">
<p>
<label for="endpoint">Maps API endpoint</label>
<input type="text" name="endpoint" id="endpoint" value="http://rochoa.cartodb.com/api/v1/map">
</p>
<p>
<label for="apikey">API key</label>
<input type="text" name="apikey" id="apikey" value="" placeholder="YOUR API KEY">
</p>
<p>
<label for="sql_editor">SQL</label>
<textarea id="sql_editor" class="code"></textarea>
</p>
<p>
<label for="css_editor">CartoCSS</label>
<textarea id="css_editor" class="code"></textarea>
</p>
</form>
<div id="map"></div>
</div>
</body>
<script src="//cdnjs.cloudflare.com/ajax/libs/leaflet/0.7.3/leaflet.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/4.7.0/codemirror.min.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/4.7.0/mode/javascript/javascript.min.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/4.7.0/mode/sql/sql.min.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/4.7.0/mode/css/css.min.js"></script>
<script src="app.js"></script>
</html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment