Skip to content

Instantly share code, notes, and snippets.

@flamingbear

flamingbear/Daily_Sea_Ice_Statistics_by_Year_Month.ipynb

Created April 25, 2015 18:43
Show Gist options
  • Save flamingbear/db24b18c98393f26caa1 to your computer and use it in GitHub Desktop.
Save flamingbear/db24b18c98393f26caa1 to your computer and use it in GitHub Desktop.
Download ZIP
Daily Sea Ice Statistics iPython Notebook
Display the source blob
Display the rendered blob
Raw
Daily_Sea_Ice_Statistics_by_Year_Month.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"# Compute summary statistics for the daily sea ice index.\n",
"\n",
"# From the CSV files determine the day of maximum and minimum extent for each month and how that month's max and min ranks with all other months\n",
"\n",
"The input data format is just a date and extent for each day we have data.\n",
"```\n",
"Year, Month, Day, Extent, Missing, Source Data\n",
"YYYY, MM, DD, 10^6 sq km, 10^6 sq km, Source data product web site: http://nsidc.org/d....\n",
"1978, 10, 26, 10.231, 0.000, ftp://sidads.colorado.edu/pub/DATASETS/nsidc0051....\n",
"1978, 10, 28, 10.420, 0.000, ftp://sidads.colorado.edu/pub/DATASETS/nsidc0051....\n",
"1978, 10, 30, 10.557, 0.000, ftp://sidads.colorado.edu/pub/DATASETS/nsidc0051....\n",
"....\n",
"```\n"
]
},
{
"cell_type": "code",
"execution_count": 143,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
"import datetime as dt\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import os\n",
"%pylab inline\n",
"import pandas as pd\n",
"pd.options.display.mpl_style = 'default'\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"code to read the CSV files."
]
},
{
"cell_type": "code",
"execution_count": 144,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"\n",
"def parse_the_date(year, mm, dd):\n",
" return dt.date(int(year), int(mm), int(dd))\n",
"\n",
"def slurp_csv(filename):\n",
" data = pd.read_csv(filename, header = None, skiprows=2,\n",
" names=[\"year\", \"mm\", \"dd\", \"extent\", \"missing\", \"source\"],\n",
" parse_dates={'date':['year', 'mm', 'dd']},\n",
" date_parser=parse_the_date, index_col='date')\n",
" data = data.drop('missing', axis=1)\n",
" return data\n",
"\n",
"def read_a_hemisphere(hemisphere):\n",
" production_topdir = '/projects/DATASETS/NOAA/G02135/'\n",
" final_prod_filename = os.path.join(production_topdir, '{hemisphere}/daily/data/{hemi}H_seaice_extent_final.csv'.format(hemisphere=hemisphere, hemi=hemisphere[0:1].upper()))\n",
" nrt_prod_filename = os.path.join(production_topdir, '{hemisphere}/daily/data/{hemi}H_seaice_extent_nrt.csv'.format(hemisphere=hemisphere, hemi=hemisphere[0:1].upper()))\n",
"\n",
" final = slurp_csv(final_prod_filename)\n",
" nrt = slurp_csv(nrt_prod_filename)\n",
" all_data = pd.concat([final, nrt])\n",
" return all_data\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"Read CSV data"
]
},
{
"cell_type": "code",
"execution_count": 145,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>extent</th>\n",
" <th>source</th>\n",
" </tr>\n",
" <tr>\n",
" <th>date</th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>1978-10-26</th>\n",
" <td>17.634</td>\n",
" <td>ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1978-10-28</th>\n",
" <td>17.815</td>\n",
" <td>ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1978-10-30</th>\n",
" <td>17.671</td>\n",
" <td>ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1978-11-01</th>\n",
" <td>17.534</td>\n",
" <td>ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1978-11-03</th>\n",
" <td>17.493</td>\n",
" <td>ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" extent source\n",
"date \n",
"1978-10-26 17.634 ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...\n",
"1978-10-28 17.815 ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...\n",
"1978-10-30 17.671 ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...\n",
"1978-11-01 17.534 ftp://sidads.colorado.edu/pub/DATASETS/nsidc0...\n",
"1978-11-03 17.493 ftp://sidads.colorado.edu/pub/DATASETS/nsidc0..."
]
},
"execution_count": 145,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"north = read_a_hemisphere('north')\n",
"south = read_a_hemisphere('south')\n",
"south.head()"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"Add columns for year and month: We have do this because when we read the CSV file\n",
"we converted the existing year/month/day into a python datetime object.\n",
"also drop the source because we don't care where the data came from (near real time or production)"
]
},
{
"cell_type": "code",
"execution_count": 146,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def add_year_month_columns(df):\n",
" a = df.copy()\n",
" a = a.drop('source',1)\n",
" a = a.reset_index()\n",
" a['year'] = pd.to_datetime(a.date).dt.year\n",
" a['month'] = pd.to_datetime(a.date).dt.month\n",
" return a"
]
},
{
"cell_type": "code",
"execution_count": 147,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"north = add_year_month_columns(north)\n",
"south = add_year_month_columns(south)"
]
},
{
"cell_type": "code",
"execution_count": 148,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>year</th>\n",
" <th>month</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1978-10-26</td>\n",
" <td>10.231</td>\n",
" <td>1978</td>\n",
" <td>10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>1978-10-28</td>\n",
" <td>10.420</td>\n",
" <td>1978</td>\n",
" <td>10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>1978-10-30</td>\n",
" <td>10.557</td>\n",
" <td>1978</td>\n",
" <td>10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1978-11-01</td>\n",
" <td>10.670</td>\n",
" <td>1978</td>\n",
" <td>11</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>1978-11-03</td>\n",
" <td>10.787</td>\n",
" <td>1978</td>\n",
" <td>11</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" date extent year month\n",
"0 1978-10-26 10.231 1978 10\n",
"1 1978-10-28 10.420 1978 10\n",
"2 1978-10-30 10.557 1978 10\n",
"3 1978-11-01 10.670 1978 11\n",
"4 1978-11-03 10.787 1978 11"
]
},
"execution_count": 148,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"north.head()"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"Use a groupby to compute the row locations that represent the minimum and\n",
"maximum extent and grab those rows into new variables. AKA: Filter out everything\n",
"but the minimum/maximum extent for each month and year"
]
},
{
"cell_type": "code",
"execution_count": 149,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def select_min_and_max_extent_rows_by_year_and_month(df):\n",
" min_groups = df.loc[df.groupby(['year','month'])['extent'].idxmin()]\n",
" max_groups = df.loc[df.groupby(['year','month'])['extent'].idxmax()]\n",
" return {'min': min_groups,\n",
" 'max': max_groups}"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"create dictionaries of max and min values for each hemisphere"
]
},
{
"cell_type": "code",
"execution_count": 150,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"n = select_min_and_max_extent_rows_by_year_and_month(north)\n",
"s = select_min_and_max_extent_rows_by_year_and_month(south)"
]
},
{
"cell_type": "code",
"execution_count": 151,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def add_rank(df):\n",
" df['rank'] = df.groupby('month').extent.rank()\n",
" return df"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"add rank column for each month and hemsiphere's max and min:"
]
},
{
"cell_type": "code",
"execution_count": 152,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"n['max'] = add_rank(n['max'])\n",
"n['min'] = add_rank(n['min'])\n",
"s['max'] = add_rank(s['max'])\n",
"s['min'] = add_rank(s['min'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"add min and max columns and combine into a single data frame"
]
},
{
"cell_type": "code",
"execution_count": 153,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def merge_min_max_data(df_min, df_max):\n",
" # Add column of type to the data\n",
" df_min['type'] = 'MIN'\n",
" df_max['type'] = 'MAX'\n",
"\n",
" # pivot the data so that year, month and type are the index \n",
" mins = df_min.set_index(['year','month', 'type'])\n",
" maxs = df_max.set_index(['year','month', 'type'])\n",
"\n",
" mins.head()\n",
"\n",
" # broadcast month back up into the columns across the top.\n",
" mins = mins.unstack('month')\n",
" maxs = maxs.unstack('month')\n",
"\n",
" combined = pd.concat([mins, maxs])\n",
"\n",
" return combined\n"
]
},
{
"cell_type": "code",
"execution_count": 154,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def swap_column_level_and_sort(df):\n",
" df.columns = df.columns.swaplevel(1,0)\n",
" df = df.sortlevel(0, axis=1)\n",
" return df\n"
]
},
{
"cell_type": "code",
"execution_count": 155,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"combo_north = merge_min_max_data(n['min'], n['max'])\n",
"combo_south = merge_min_max_data(s['min'], s['max'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"add hemisphere info."
]
},
{
"cell_type": "code",
"execution_count": 156,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def add_hemisphere(df, hemi):\n",
" df = df.stack('month').reset_index()\n",
" df['hemisphere'] = hemi\n",
" df = df.set_index(['year', 'type', 'month','hemisphere']).unstack('month')\n",
" return df"
]
},
{
"cell_type": "code",
"execution_count": 157,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"combo_north = add_hemisphere(combo_north, 'Northern')\n",
"combo_south = add_hemisphere(combo_south, 'Southern')\n"
]
},
{
"cell_type": "code",
"execution_count": 158,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"combo_south = swap_column_level_and_sort(combo_south.sortlevel(1))\n",
"combo_north = swap_column_level_and_sort(combo_north.sortlevel(1))"
]
},
{
"cell_type": "code",
"execution_count": 159,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"all_data = pd.concat([combo_north, combo_south])"
]
},
{
"cell_type": "code",
"execution_count": 160,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr>\n",
" <th></th>\n",
" <th></th>\n",
" <th>month</th>\n",
" <th colspan=\"3\" halign=\"left\">1</th>\n",
" <th colspan=\"3\" halign=\"left\">2</th>\n",
" <th colspan=\"3\" halign=\"left\">3</th>\n",
" <th>4</th>\n",
" <th>...</th>\n",
" <th>9</th>\n",
" <th colspan=\"3\" halign=\"left\">10</th>\n",
" <th colspan=\"3\" halign=\"left\">11</th>\n",
" <th colspan=\"3\" halign=\"left\">12</th>\n",
" </tr>\n",
" <tr>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>...</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" </tr>\n",
" <tr>\n",
" <th>year</th>\n",
" <th>type</th>\n",
" <th>hemisphere</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>1978</th>\n",
" <th>MAX</th>\n",
" <th>Northern</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>1978-10-30</td>\n",
" <td>10.557</td>\n",
" <td>36</td>\n",
" <td>1978-11-29</td>\n",
" <td>12.694</td>\n",
" <td>36</td>\n",
" <td>1978-12-31</td>\n",
" <td>14.604</td>\n",
" <td>37</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1979</th>\n",
" <th>MAX</th>\n",
" <th>Northern</th>\n",
" <td>1979-01-30</td>\n",
" <td>15.912</td>\n",
" <td>37</td>\n",
" <td>1979-02-25</td>\n",
" <td>16.579</td>\n",
" <td>37</td>\n",
" <td>1979-03-01</td>\n",
" <td>16.635</td>\n",
" <td>37</td>\n",
" <td>1979-04-08</td>\n",
" <td>...</td>\n",
" <td>24</td>\n",
" <td>1979-10-31</td>\n",
" <td>9.924</td>\n",
" <td>25</td>\n",
" <td>1979-11-30</td>\n",
" <td>12.076</td>\n",
" <td>27</td>\n",
" <td>1979-12-30</td>\n",
" <td>14.116</td>\n",
" <td>32</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1980</th>\n",
" <th>MAX</th>\n",
" <th>Northern</th>\n",
" <td>1980-01-31</td>\n",
" <td>15.519</td>\n",
" <td>31</td>\n",
" <td>1980-02-28</td>\n",
" <td>16.251</td>\n",
" <td>33</td>\n",
" <td>1980-03-05</td>\n",
" <td>16.302</td>\n",
" <td>33</td>\n",
" <td>1980-04-06</td>\n",
" <td>...</td>\n",
" <td>34</td>\n",
" <td>1980-10-31</td>\n",
" <td>10.198</td>\n",
" <td>32</td>\n",
" <td>1980-11-30</td>\n",
" <td>12.626</td>\n",
" <td>35</td>\n",
" <td>1980-12-28</td>\n",
" <td>14.172</td>\n",
" <td>33</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1981</th>\n",
" <th>MAX</th>\n",
" <th>Northern</th>\n",
" <td>1981-01-31</td>\n",
" <td>15.627</td>\n",
" <td>33</td>\n",
" <td>1981-02-24</td>\n",
" <td>15.736</td>\n",
" <td>26</td>\n",
" <td>1981-03-14</td>\n",
" <td>15.801</td>\n",
" <td>25</td>\n",
" <td>1981-04-01</td>\n",
" <td>...</td>\n",
" <td>27</td>\n",
" <td>1981-10-30</td>\n",
" <td>9.719</td>\n",
" <td>20</td>\n",
" <td>1981-11-29</td>\n",
" <td>11.884</td>\n",
" <td>20</td>\n",
" <td>1981-12-31</td>\n",
" <td>14.237</td>\n",
" <td>36</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1982</th>\n",
" <th>MAX</th>\n",
" <th>Northern</th>\n",
" <td>1982-01-30</td>\n",
" <td>15.755</td>\n",
" <td>35</td>\n",
" <td>1982-02-27</td>\n",
" <td>16.325</td>\n",
" <td>35</td>\n",
" <td>1982-03-05</td>\n",
" <td>16.315</td>\n",
" <td>35</td>\n",
" <td>1982-04-02</td>\n",
" <td>...</td>\n",
" <td>28</td>\n",
" <td>1982-10-31</td>\n",
" <td>10.615</td>\n",
" <td>37</td>\n",
" <td>1982-11-30</td>\n",
" <td>12.695</td>\n",
" <td>37</td>\n",
" <td>1982-12-26</td>\n",
" <td>14.183</td>\n",
" <td>34</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>5 rows × 36 columns</p>\n",
"</div>"
],
"text/plain": [
"month 1 2 \\\n",
" date extent rank date extent rank \n",
"year type hemisphere \n",
"1978 MAX Northern NaN NaN NaN NaN NaN NaN \n",
"1979 MAX Northern 1979-01-30 15.912 37 1979-02-25 16.579 37 \n",
"1980 MAX Northern 1980-01-31 15.519 31 1980-02-28 16.251 33 \n",
"1981 MAX Northern 1981-01-31 15.627 33 1981-02-24 15.736 26 \n",
"1982 MAX Northern 1982-01-30 15.755 35 1982-02-27 16.325 35 \n",
"\n",
"month 3 4 ... 9 \\\n",
" date extent rank date ... rank \n",
"year type hemisphere ... \n",
"1978 MAX Northern NaN NaN NaN NaN ... NaN \n",
"1979 MAX Northern 1979-03-01 16.635 37 1979-04-08 ... 24 \n",
"1980 MAX Northern 1980-03-05 16.302 33 1980-04-06 ... 34 \n",
"1981 MAX Northern 1981-03-14 15.801 25 1981-04-01 ... 27 \n",
"1982 MAX Northern 1982-03-05 16.315 35 1982-04-02 ... 28 \n",
"\n",
"month 10 11 \\\n",
" date extent rank date extent rank \n",
"year type hemisphere \n",
"1978 MAX Northern 1978-10-30 10.557 36 1978-11-29 12.694 36 \n",
"1979 MAX Northern 1979-10-31 9.924 25 1979-11-30 12.076 27 \n",
"1980 MAX Northern 1980-10-31 10.198 32 1980-11-30 12.626 35 \n",
"1981 MAX Northern 1981-10-30 9.719 20 1981-11-29 11.884 20 \n",
"1982 MAX Northern 1982-10-31 10.615 37 1982-11-30 12.695 37 \n",
"\n",
"month 12 \n",
" date extent rank \n",
"year type hemisphere \n",
"1978 MAX Northern 1978-12-31 14.604 37 \n",
"1979 MAX Northern 1979-12-30 14.116 32 \n",
"1980 MAX Northern 1980-12-28 14.172 33 \n",
"1981 MAX Northern 1981-12-31 14.237 36 \n",
"1982 MAX Northern 1982-12-26 14.183 34 \n",
"\n",
"[5 rows x 36 columns]"
]
},
"execution_count": 160,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"all_data.head()"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"we want to reorder the index so that Hemisphere appears first and year last."
]
},
{
"cell_type": "code",
"execution_count": 161,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"all_data = all_data.swaplevel(0,2)"
]
},
{
"cell_type": "code",
"execution_count": 162,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr>\n",
" <th></th>\n",
" <th></th>\n",
" <th>month</th>\n",
" <th colspan=\"3\" halign=\"left\">1</th>\n",
" <th colspan=\"3\" halign=\"left\">2</th>\n",
" <th colspan=\"3\" halign=\"left\">3</th>\n",
" <th>4</th>\n",
" <th>...</th>\n",
" <th>9</th>\n",
" <th colspan=\"3\" halign=\"left\">10</th>\n",
" <th colspan=\"3\" halign=\"left\">11</th>\n",
" <th colspan=\"3\" halign=\"left\">12</th>\n",
" </tr>\n",
" <tr>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>...</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" </tr>\n",
" <tr>\n",
" <th>hemisphere</th>\n",
" <th>type</th>\n",
" <th>year</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th rowspan=\"2\" valign=\"top\">Northern</th>\n",
" <th rowspan=\"2\" valign=\"top\">MAX</th>\n",
" <th>1978</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>1978-10-30</td>\n",
" <td>10.557</td>\n",
" <td>36</td>\n",
" <td>1978-11-29</td>\n",
" <td>12.694</td>\n",
" <td>36</td>\n",
" <td>1978-12-31</td>\n",
" <td>14.604</td>\n",
" <td>37</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1979</th>\n",
" <td>1979-01-30</td>\n",
" <td>15.912</td>\n",
" <td>37</td>\n",
" <td>1979-02-25</td>\n",
" <td>16.579</td>\n",
" <td>37</td>\n",
" <td>1979-03-01</td>\n",
" <td>16.635</td>\n",
" <td>37</td>\n",
" <td>1979-04-08</td>\n",
" <td>...</td>\n",
" <td>24</td>\n",
" <td>1979-10-31</td>\n",
" <td>9.924</td>\n",
" <td>25</td>\n",
" <td>1979-11-30</td>\n",
" <td>12.076</td>\n",
" <td>27</td>\n",
" <td>1979-12-30</td>\n",
" <td>14.116</td>\n",
" <td>32</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>2 rows × 36 columns</p>\n",
"</div>"
],
"text/plain": [
"month 1 2 \\\n",
" date extent rank date extent rank \n",
"hemisphere type year \n",
"Northern MAX 1978 NaN NaN NaN NaN NaN NaN \n",
" 1979 1979-01-30 15.912 37 1979-02-25 16.579 37 \n",
"\n",
"month 3 4 ... 9 \\\n",
" date extent rank date ... rank \n",
"hemisphere type year ... \n",
"Northern MAX 1978 NaN NaN NaN NaN ... NaN \n",
" 1979 1979-03-01 16.635 37 1979-04-08 ... 24 \n",
"\n",
"month 10 11 \\\n",
" date extent rank date extent rank \n",
"hemisphere type year \n",
"Northern MAX 1978 1978-10-30 10.557 36 1978-11-29 12.694 36 \n",
" 1979 1979-10-31 9.924 25 1979-11-30 12.076 27 \n",
"\n",
"month 12 \n",
" date extent rank \n",
"hemisphere type year \n",
"Northern MAX 1978 1978-12-31 14.604 37 \n",
" 1979 1979-12-30 14.116 32 \n",
"\n",
"[2 rows x 36 columns]"
]
},
"execution_count": 162,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"all_data.head(2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"Lastly we want to rename the Month columns"
]
},
{
"cell_type": "code",
"execution_count": 163,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import calendar\n",
"month_names = [calendar.month_name[x] for x in range(1,13)]"
]
},
{
"cell_type": "code",
"execution_count": 164,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"all_data.columns = all_data.columns.set_levels(month_names, level=0)\n"
]
},
{
"cell_type": "code",
"execution_count": 165,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr>\n",
" <th></th>\n",
" <th></th>\n",
" <th>month</th>\n",
" <th colspan=\"3\" halign=\"left\">January</th>\n",
" <th colspan=\"3\" halign=\"left\">February</th>\n",
" <th colspan=\"3\" halign=\"left\">March</th>\n",
" <th>April</th>\n",
" <th>...</th>\n",
" <th>September</th>\n",
" <th colspan=\"3\" halign=\"left\">October</th>\n",
" <th colspan=\"3\" halign=\"left\">November</th>\n",
" <th colspan=\"3\" halign=\"left\">December</th>\n",
" </tr>\n",
" <tr>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>...</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" <th>date</th>\n",
" <th>extent</th>\n",
" <th>rank</th>\n",
" </tr>\n",
" <tr>\n",
" <th>hemisphere</th>\n",
" <th>type</th>\n",
" <th>year</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th rowspan=\"3\" valign=\"top\">Northern</th>\n",
" <th rowspan=\"3\" valign=\"top\">MAX</th>\n",
" <th>1978</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>1978-10-30</td>\n",
" <td>10.557</td>\n",
" <td>36</td>\n",
" <td>1978-11-29</td>\n",
" <td>12.694</td>\n",
" <td>36</td>\n",
" <td>1978-12-31</td>\n",
" <td>14.604</td>\n",
" <td>37</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1979</th>\n",
" <td>1979-01-30</td>\n",
" <td>15.912</td>\n",
" <td>37</td>\n",
" <td>1979-02-25</td>\n",
" <td>16.579</td>\n",
" <td>37</td>\n",
" <td>1979-03-01</td>\n",
" <td>16.635</td>\n",
" <td>37</td>\n",
" <td>1979-04-08</td>\n",
" <td>...</td>\n",
" <td>24</td>\n",
" <td>1979-10-31</td>\n",
" <td>9.924</td>\n",
" <td>25</td>\n",
" <td>1979-11-30</td>\n",
" <td>12.076</td>\n",
" <td>27</td>\n",
" <td>1979-12-30</td>\n",
" <td>14.116</td>\n",
" <td>32</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1980</th>\n",
" <td>1980-01-31</td>\n",
" <td>15.519</td>\n",
" <td>31</td>\n",
" <td>1980-02-28</td>\n",
" <td>16.251</td>\n",
" <td>33</td>\n",
" <td>1980-03-05</td>\n",
" <td>16.302</td>\n",
" <td>33</td>\n",
" <td>1980-04-06</td>\n",
" <td>...</td>\n",
" <td>34</td>\n",
" <td>1980-10-31</td>\n",
" <td>10.198</td>\n",
" <td>32</td>\n",
" <td>1980-11-30</td>\n",
" <td>12.626</td>\n",
" <td>35</td>\n",
" <td>1980-12-28</td>\n",
" <td>14.172</td>\n",
" <td>33</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>3 rows × 36 columns</p>\n",
"</div>"
],
"text/plain": [
"month January February \\\n",
" date extent rank date extent rank \n",
"hemisphere type year \n",
"Northern MAX 1978 NaN NaN NaN NaN NaN NaN \n",
" 1979 1979-01-30 15.912 37 1979-02-25 16.579 37 \n",
" 1980 1980-01-31 15.519 31 1980-02-28 16.251 33 \n",
"\n",
"month March April ... September \\\n",
" date extent rank date ... rank \n",
"hemisphere type year ... \n",
"Northern MAX 1978 NaN NaN NaN NaN ... NaN \n",
" 1979 1979-03-01 16.635 37 1979-04-08 ... 24 \n",
" 1980 1980-03-05 16.302 33 1980-04-06 ... 34 \n",
"\n",
"month October November \\\n",
" date extent rank date extent rank \n",
"hemisphere type year \n",
"Northern MAX 1978 1978-10-30 10.557 36 1978-11-29 12.694 36 \n",
" 1979 1979-10-31 9.924 25 1979-11-30 12.076 27 \n",
" 1980 1980-10-31 10.198 32 1980-11-30 12.626 35 \n",
"\n",
"month December \n",
" date extent rank \n",
"hemisphere type year \n",
"Northern MAX 1978 1978-12-31 14.604 37 \n",
" 1979 1979-12-30 14.116 32 \n",
" 1980 1980-12-28 14.172 33 \n",
"\n",
"[3 rows x 36 columns]"
]
},
"execution_count": 165,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"all_data.head(3)"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"and write out the statistics file"
]
},
{
"cell_type": "code",
"execution_count": 166,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"all_data.to_csv('Sea_Ice_Statistics_from_daily_data.csv')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
},
"name": "Daily_Sea_Ice_Statistics_by_Year_Month.ipynb"
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment