Skip to content

Instantly share code, notes, and snippets.

@msund

msund/gist:5481bd48a21e9532a36f

Last active August 29, 2015 14:16
Show Gist options
  • Save msund/5481bd48a21e9532a36f to your computer and use it in GitHub Desktop.
Save msund/5481bd48a21e9532a36f to your computer and use it in GitHub Desktop.
Download ZIP
Next Version for CartoDB
Raw
gistfile1.txt
{
"metadata": {
"name": "",
"signature": "sha256:125dfe0d83824b70aef36b1d777d19fa4bf5ba3bde0874041ba6bd5b9a4e4aa8"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "heading",
"level": 3,
"metadata": {},
"source": [
"<div style=\"margin: auto; width: 800px; display: block;\">\n",
"<img src=\"https://plot.ly/static/learn/images/plotly-hist-logo.png\" style=\"width: 200px; display: inline;\" /> <span style=\"font-size: 5em;\">+</a>\n",
"<img src=\"http://cartodb.s3.amazonaws.com/static/logos_full_cartodb_light.png\" style=\"width: 350px; display: inline;\" /></div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[CartoDB](http://cartodb.com/) lets you easily make web-based maps driven by a PostgreSQL/PostGIS backend, so data management is easy. [Plotly](https://plot.ly) is a cloud-based graphing and analytics platform with [Python, R, & MATLAB APIs](https://plot.ly/api) where collaboration is easy. This IPython Notebook shows how to use them together to analyze earthquake data."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"# Import needed libraries\n",
"%pylab inline\n",
"import pandas as pd\n",
"import plotly.plotly as py\n",
"from plotly.graph_objs import *\n",
"import plotly.tools as tls"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"prompt_number": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Getting started**\n",
"\n",
"1. Setup a free CartoDB account at [https://cartodb.com/signup](https://cartodb.com/signup) or use data linked in this notebook\n",
"2. Use Plotly's sandbox account, or [sign-up](https://plot.ly/python/getting-started/). No downloads required."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"py.sign_in('Python-Demo-Account', 'gwt101uhh0')"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 2
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Pandas's [`read_csv`](http://pandas.pydata.org/pandas-docs/dev/generated/pandas.io.parsers.read_csv.html) allows import via HTTP, FTP, etc. It's perfect for CartoDB's [SQL API](), which has the following template:\n",
"```\n",
"http://{account_name}.cartodb.com/api/v2/sql?q={custom_sql_statement}&format=csv\n",
"```\n",
"\n",
"To get data from the data table in my CartoDB account, the following query grabs values we can graph, and converts the timestamp to work easily with plotly.\n",
"\n",
"```sql\n",
"SELECT\n",
" mag,\n",
" magtype,\n",
" type,\n",
" to_char(time,'yyyy-mm-DD HH24:MI:SS') AS time_plotly,\n",
" place,\n",
" depth\n",
"FROM\n",
" all_month\n",
"```\n",
"\n",
"All we need to do is replace the white space with `%20` so the URL is properly encoded."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"url = \"http://andye.cartodb.com/api/v2/sql?\"\\\n",
" \"q=SELECT%20mag,magtype,type,to_char(time,'yyyy-mm-DD%20HH24:MI:SS')%20AS%20time_plotly,place,depth%20FROM%20all_month\"\\\n",
" \"&format=csv\"\n",
"df = pd.read_csv(url)\n",
"df = df.sort(['mag'], ascending=[0]);"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 3
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df.head()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>mag</th>\n",
" <th>magtype</th>\n",
" <th>type</th>\n",
" <th>time_plotly</th>\n",
" <th>place</th>\n",
" <th>depth</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>3749</th>\n",
" <td> 7.3</td>\n",
" <td> mww</td>\n",
" <td> earthquake</td>\n",
" <td> 2014-10-14 03:51:35</td>\n",
" <td> 67km WSW of Jiquilillo, Nicaragua</td>\n",
" <td> 40.00</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1686</th>\n",
" <td> 7.1</td>\n",
" <td> mww</td>\n",
" <td> earthquake</td>\n",
" <td> 2014-10-09 02:14:32</td>\n",
" <td> Southern East Pacific Rise</td>\n",
" <td> 15.50</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4602</th>\n",
" <td> 7.1</td>\n",
" <td> mwc</td>\n",
" <td> earthquake</td>\n",
" <td> 2014-11-01 18:57:22</td>\n",
" <td> 141km NE of Ndoi Island, Fiji</td>\n",
" <td> 434.41</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2855</th>\n",
" <td> 6.6</td>\n",
" <td> mww</td>\n",
" <td> earthquake</td>\n",
" <td> 2014-10-09 02:32:05</td>\n",
" <td> Southern East Pacific Rise</td>\n",
" <td> 10.00</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7186</th>\n",
" <td> 6.3</td>\n",
" <td> mwp</td>\n",
" <td> earthquake</td>\n",
" <td> 2014-10-11 02:35:46</td>\n",
" <td> 154km ENE of Hachinohe, Japan</td>\n",
" <td> 13.48</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 4,
"text": [
" mag magtype type time_plotly \\\n",
"3749 7.3 mww earthquake 2014-10-14 03:51:35 \n",
"1686 7.1 mww earthquake 2014-10-09 02:14:32 \n",
"4602 7.1 mwc earthquake 2014-11-01 18:57:22 \n",
"2855 6.6 mww earthquake 2014-10-09 02:32:05 \n",
"7186 6.3 mwp earthquake 2014-10-11 02:35:46 \n",
"\n",
" place depth \n",
"3749 67km WSW of Jiquilillo, Nicaragua 40.00 \n",
"1686 Southern East Pacific Rise 15.50 \n",
"4602 141km NE of Ndoi Island, Fiji 434.41 \n",
"2855 Southern East Pacific Rise 10.00 \n",
"7186 154km ENE of Hachinohe, Japan 13.48 "
]
}
],
"prompt_number": 4
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's take a look at the magnitude in a histogram. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"mag_histogram_plot = [{'x': df['mag'], \n",
" 'type': 'histogram'\n",
"}]"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 5
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"data_histogram = Data(mag_histogram_plot)\n",
"\n",
"fig_histogram = Figure(data=data_histogram)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 6
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"py.iplot(fig_histogram, filename='magnitude_histogram')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<iframe id=\"igraph\" scrolling=\"no\" style=\"border:none;\"seamless=\"seamless\" src=\"https://plot.ly/~Python-Demo-Account/1535.embed\" height=\"525\" width=\"100%\"></iframe>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 7,
"text": [
"<plotly.tools.PlotlyDisplay at 0x7fe42b8e10d0>"
]
}
],
"prompt_number": 7
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's check out the same data in a box plot. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"mag_jitter_plot = [{'y': df['mag'], \n",
" 'name': 'Earthquake Magnitude',\n",
" 'type': 'box',\n",
" 'boxpoints': 'outliers', \n",
" 'jitter': 0.9,\n",
"}]"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 8
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"data_jitter = Data(mag_jitter_plot)\n",
"\n",
"fig_jitter = Figure(data=data_jitter)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 9
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"py.iplot(fig_jitter, filename='boxplot_with_jitter')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<iframe id=\"igraph\" scrolling=\"no\" style=\"border:none;\"seamless=\"seamless\" src=\"https://plot.ly/~Python-Demo-Account/1532.embed\" height=\"525\" width=\"100%\"></iframe>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 10,
"text": [
"<plotly.tools.PlotlyDisplay at 0x7fe42ae04bd0>"
]
}
],
"prompt_number": 10
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we want to put the plot in a report, email, or presentation we can export the static version. The plot URL contains the data, code to reproduce the plot with MATLAB, R, and Python, and can be embedded. \n",
"<br>\n",
"<br>\n",
"<br>\n",
"- https://plot.ly/~Python-Demo-Account/1534.png\n",
"- https://plot.ly/~Python-Demo-Account/1534.svg\n",
"- https://plot.ly/~Python-Demo-Account/1534.pdf\n",
"- https://plot.ly/~Python-Demo-Account/1534.eps\n",
"- https://plot.ly/~Python-Demo-Account/1534.m\n",
"- https://plot.ly/~Python-Demo-Account/1534.py\n",
"- https://plot.ly/~Python-Demo-Account/1534.r\n",
"- https://plot.ly/~Python-Demo-Account/1534.jl\n",
"- https://plot.ly/~Python-Demo-Account/1534.json\n",
"- https://plot.ly/~Python-Demo-Account/1534.embed\n",
"<br>\n",
"<br>\n",
"<br>\n",
"You and others you share the plot with can also collaborate and style the plot in the GUI. \n",
"<br>\n",
"<br>\n",
"<br>\n",
"<img src=\"http://i.imgur.com/Cwtea9h.gif\" /></a>\n",
"<br>\n",
"<br>\n",
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's take another pass at it, and this time put both magnitude and depth in the same plot. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"location = df['place'] # manages serialization in early versions of Plotly Python client\n",
"for i in range(len(location)):\n",
" try:\n",
" location[i] = str(location[i]).decode('utf-8')\n",
" except:\n",
" location[i] = 'Country name decode error'"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 11
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"trace1 = Scatter(\n",
" x=df['depth'],\n",
" y=df['mag'],\n",
" text=location,\n",
" mode='markers',\n",
" marker=Marker(\n",
" color='rgba(31, 119, 180, 0.15)', # add opacity for visibility\n",
" )\n",
")\n",
"layout = Layout(\n",
" title='Earthquake Magnitude vs. Depth',\n",
" xaxis=XAxis( type='log', title='depth' ),\n",
" yaxis=YAxis( type='log', title='magnitude' ),\n",
" hovermode=\"closest\",\n",
")\n",
"data = Data([trace1])\n",
"fig = Figure(data=data, layout=layout)\n",
"py.iplot(fig, filename='Earthquake_basic')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<iframe id=\"igraph\" scrolling=\"no\" style=\"border:none;\"seamless=\"seamless\" src=\"https://plot.ly/~Python-Demo-Account/1536.embed\" height=\"525\" width=\"100%\"></iframe>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 12,
"text": [
"<plotly.tools.PlotlyDisplay at 0x7fe42adfe190>"
]
}
],
"prompt_number": 12
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you click and drag, you can zoom in on the plot. Hover your mouse to see data about each earthquake. Now, for our final plot, we can make a scatter plot over time, showing the magnitude on the y axis with the point sized for depth. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"depth_time_plot = [Scatter({'y': df['mag'], \n",
" 'x': df['time_plotly'],\n",
" 'name': 'Earthquake Depth',\n",
" 'mode': 'markers',\n",
" 'text': df['place'],\n",
" 'marker': {\n",
" 'size': 20.0 * (df['depth'] + abs(df['depth'].min())) / (df['depth'].max() + abs(df['depth'].min()))\n",
" }})]"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 13
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"data_depth = Data(depth_time_plot)\n",
"\n",
"layout_depth = Layout(yaxis=YAxis(title='Magnitude of the Event'),xaxis=XAxis(title='Date of Event'),hovermode='closest')\n",
"\n",
"fig_depth = Figure(data=data_depth, layout=layout_depth )"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 14
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"py.iplot(fig_depth)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<iframe id=\"igraph\" scrolling=\"no\" style=\"border:none;\"seamless=\"seamless\" src=\"https://plot.ly/~Python-Demo-Account/1541.embed\" height=\"525\" width=\"100%\"></iframe>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 15,
"text": [
"<plotly.tools.PlotlyDisplay at 0x7fe42adfead0>"
]
}
],
"prompt_number": 15
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"from IPython.display import HTML\n",
"HTML('<iframe width=100% height=520 frameborder=0 src=http://andye.cartodb.com/viz/e44ac140-b8ad-11e4-b156-0e4fddd5de28/embed_map allowfullscreen webkitallowfullscreen mozallowfullscreen oallowfullscreen msallowfullscreen></iframe>')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<iframe width=100% height=520 frameborder=0 src=http://andye.cartodb.com/viz/e44ac140-b8ad-11e4-b156-0e4fddd5de28/embed_map allowfullscreen webkitallowfullscreen mozallowfullscreen oallowfullscreen msallowfullscreen></iframe>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 16,
"text": [
"<IPython.core.display.HTML at 0x7fe42adfe3d0>"
]
}
],
"prompt_number": 16
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"from IPython.core.display import HTML\n",
"import urllib2\n",
"HTML(urllib2.urlopen('http://bit.ly/1Bf5Hft').read())"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style>\n",
"\n",
"html {\n",
" font-size: 62.5% !important; }\n",
"body {\n",
" font-size: 1.5em !important; /* currently ems cause chrome bug misinterpreting rems on body element */\n",
" line-height: 1.6 !important;\n",
" font-weight: 400 !important;\n",
" font-family: \"Raleway\", \"HelveticaNeue\", \"Helvetica Neue\", Helvetica, Arial, sans-serif !important;\n",
" color: #222 !important; }\n",
"\n",
"div{ border-radius: 0px !important; }\n",
"div.CodeMirror-sizer{ background: rgb(244, 244, 248) !important; }\n",
"div.input_area{ background: rgb(244, 244, 248) !important; }\n",
"\n",
"div.out_prompt_overlay:hover{ background: rgb(244, 244, 248) !important; }\n",
"div.input_prompt:hover{ background: rgb(244, 244, 248) !important; }\n",
"\n",
"h1, h2, h3, h4, h5, h6 {\n",
" color: #333 !important;\n",
" margin-top: 0 !important;\n",
" margin-bottom: 2rem !important;\n",
" font-weight: 300 !important; }\n",
"h1 { font-size: 4.0rem !important; line-height: 1.2 !important; letter-spacing: -.1rem !important;}\n",
"h2 { font-size: 3.6rem !important; line-height: 1.25 !important; letter-spacing: -.1rem !important; }\n",
"h3 { font-size: 3.0rem !important; line-height: 1.3 !important; letter-spacing: -.1rem !important; }\n",
"h4 { font-size: 2.4rem !important; line-height: 1.35 !important; letter-spacing: -.08rem !important; }\n",
"h5 { font-size: 1.8rem !important; line-height: 1.5 !important; letter-spacing: -.05rem !important; }\n",
"h6 { font-size: 1.5rem !important; line-height: 1.6 !important; letter-spacing: 0 !important; }\n",
"\n",
"@media (min-width: 550px) {\n",
" h1 { font-size: 5.0rem !important; }\n",
" h2 { font-size: 4.2rem !important; }\n",
" h3 { font-size: 3.6rem !important; }\n",
" h4 { font-size: 3.0rem !important; }\n",
" h5 { font-size: 2.4rem !important; }\n",
" h6 { font-size: 1.5rem !important; }\n",
"}\n",
"\n",
"p {\n",
" margin-top: 0 !important; }\n",
" \n",
"a {\n",
" color: #1EAEDB !important; }\n",
"a:hover {\n",
" color: #0FA0CE !important; }\n",
" \n",
"code {\n",
" padding: .2rem .5rem !important;\n",
" margin: 0 .2rem !important;\n",
" font-size: 90% !important;\n",
" white-space: nowrap !important;\n",
" background: #F1F1F1 !important;\n",
" border: 1px solid #E1E1E1 !important;\n",
" border-radius: 4px !important; }\n",
"pre > code {\n",
" display: block !important;\n",
" padding: 1rem 1.5rem !important;\n",
" white-space: pre !important; }\n",
" \n",
"button{ border-radius: 0px !important; }\n",
".navbar-inner{ background-image: none !important; }\n",
"select, textarea{ border-radius: 0px !important; }\n",
"\n",
"</style>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 17,
"text": [
"<IPython.core.display.HTML at 0x7fe42adfe490>"
]
}
],
"prompt_number": 17
}
],
"metadata": {}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment