Cornell Data Science Hackathon

index.md

Cornell Data Science Hackathon

Andy Eschbacher, Map Scientist, CartoDB

@MrEPhysics | [email protected]

• Find notes here: bit.ly/cdb-cornell-notes
• Gist stored here: bit.ly/cornell-hack-cdb

What is CartoDB?

• Cloud-based spatial analytics platform. i.e., Data Visualizations + Maps
• Not just mapping platform -- flexible database (PostgreSQL) in the cloud

We have APIs

1. CartoDB.js -- JavaScript API
2. Torque.js -- Temporal mappping JS API
3. SQL API -- Data API
4. Maps API -- Public maps/private data, static maps (export as images), etc.
5. Import API -- Push files to your account

Quick tour of the Editor

Data import

Most major formats supported: CSV, GeoJSON, KML, etc.

• Drag and drop
• Chrome Extension (beta version): https://github.com/juanignaciosl/cartodb-chrome-extension

SQL Interface

Run SQL http://docs.cartodb.com/cartodb-editor.html#custom-sql

CartoCSS -- Map Stylings

http://docs.cartodb.com/cartodb-editor.html#cartocss

Analytics within a database

Mathematical/statistical operations in data tables: http://www.postgresql.org/docs/9.3/static/functions-aggregate.html

• Mean, median, mode, etc.
• Std dev, regression, covariance

String processing: http://www.postgresql.org/docs/9.3/static/functions-string.html

Pattern Matching/RegEx: http://www.postgresql.org/docs/9.3/static/functions-matching.html

Format juggling:

• XML
• JSON

Tons more: http://www.postgresql.org/docs/9.3/static/functions.html

Flexible Data Export using SQL API

Format:

http://{account_name}.cartodb.com/api/v2/sql?q={custom_sql_query}&format={desired_format}&filename={output_file_name}

format: csv, geojson, shp, etc.

R's read.csv() and Panda's pd.read_csv() are not only for local files

Can easily use with Python pandas (pd.read_csv) and R (read.csv). Check out demo on Plotly I made last week: https://plot.ly/ipython-notebooks/cartodb/

Spatio-temporal Analysis

PostGIS -- the geospatial extension to PostgreSQL

SELECT superhero.name
FROM city, superhero
WHERE ST_Contains(city.the_geom, superhero.the_geom)
AND city.name = 'Gotham'

Resources:

• Workshop: http://cartodb.github.io/training/inter/harvard.html
• Map Academy: http://academy.cartodb.com/courses/04-sql-postgis.html

Torque (documentation)

Examples:

• Twitter analysis: http://cartodb.s3.amazonaws.com/static_vizz/sunrise.html
• Zoom-dependent binning of data: https://team.cartodb.com/u/eschbacher/viz/68d07502-bf9a-11e4-b867-0e853d047bba/embed_map
• Heat maps + SQL API: http://bl.ocks.org/pramsey/922f6170dec2a3cd5da6

Accounts

Get (better) free accounts here: https://cartodb.com/signup?plan=academy

If you already have an account and want the student upgrade, email me ([email protected]) and I'll bump you up.

Resources

• Me: @MrEPhysics
• GIS StackExchange/cartodb
• Doucmentation: SQL API, CartoDB.js, Import API, Maps API
• Map Academy
• Tutorials, etc.

Need help?? Tweet at me @MrEPhysics