Skip to content

Instantly share code, notes, and snippets.

@JiaweiZhuang

JiaweiZhuang/ncdiff.ipynb

Last active September 21, 2017 22:36
Show Gist options
  • Save JiaweiZhuang/7a71b8470e26ff28d5c8ffe3b2bb5654 to your computer and use it in GitHub Desktop.
Save JiaweiZhuang/7a71b8470e26ff28d5c8ffe3b2bb5654 to your computer and use it in GitHub Desktop.
Download ZIP
Diff GEOS-Chem output
Display the source blob
Display the rendered blob
Raw
ncdiff.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import xarray as xr"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"ds = xr.open_dataset('GEOSChem.inst.20130701.nc4.Ref')\n",
"ds1 = xr.open_dataset('GEOSChem.inst.20130701.nc4.Identical')\n",
"ds2 = xr.open_dataset('GEOSChem.inst.20130701.nc4.Different')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use DataSet methods"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`identical` checks all metadata, while `equals` only checks numerical values."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(True, False)"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds1.identical(ds), ds2.identical(ds)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(True, True)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds1.equals(ds), ds2.equals(ds)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use testing functions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The followling 4 assertions all pass."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"xr.testing.assert_allclose(ds, ds1)\n",
"xr.testing.assert_allclose(ds, ds2)\n",
"xr.testing.assert_equal(ds, ds1)\n",
"xr.testing.assert_equal(ds, ds2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`assert_allclose` is less strict than `assert_equal`, and you can specifiy error tolerance. "
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"xr.testing.assert_allclose(ds, ds2,rtol=1e-10, atol=1e-12)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`assert_identical` fails on ds and ds2 because of their different metadata"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"xr.testing.assert_identical(ds,ds1)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"ename": "AssertionError",
"evalue": "<xarray.Dataset>\nDimensions: (ilev: 48, lat: 46, lev: 47, lon: 72, time: 1)\nCoordinates:\n * time (time) datetime64[ns] 2013-07-01T01:00:00\n * lev (lev) float64 0.9925 0.9775 0.9625 0.9475 0.9325 0.9175 ...\n * ilev (ilev) float64 1.0 0.985 0.97 0.955 0.94 0.925 0.91 0.895 ...\n * lat (lat) float64 -89.0 -86.0 -82.0 -78.0 -74.0 -70.0 -66.0 ...\n * lon (lon) float64 -180.0 -175.0 -170.0 -165.0 -160.0 -155.0 ...\nData variables:\n AREA (lat, lon) float32 2.16047e+09 2.16047e+09 2.16047e+09 ...\n hyam (lev) float64 0.02402 3.321 9.864 16.37 22.85 29.33 35.78 ...\n hybm (lev) float64 0.9925 0.9742 0.9526 0.9311 0.9096 0.8882 ...\n hyai (ilev) float64 0.0 0.04805 6.594 13.13 19.61 26.09 32.57 ...\n hybi (ilev) float64 1.0 0.985 0.9634 0.9419 0.9204 0.8989 ...\n P0 float64 1e+03\n CHEM_SPC_Rn (time, lev, lat, lon) float64 1.378e-21 1.378e-21 ...\n CHEM_SPC_PASV (time, lev, lat, lon) float64 3.452e-09 3.452e-09 ...\n CHEM_SPC_Be7 (time, lev, lat, lon) float64 3.971e-22 3.971e-22 ...\n CHEM_SPC_Pb (time, lev, lat, lon) float64 1.124e-20 1.124e-20 ...\nAttributes:\n title: GEOS-Chem diagnostic collection: inst\n history: \n format: NetCDF-3 with large file support\n conventions: COARDS\n ProdDateTime: \n reference: www.geos-chem.org; wiki.geos-chem.org\n contact: GEOS-Chem Support Team ([email protected])\n<xarray.Dataset>\nDimensions: (ilev: 48, lat: 46, lev: 47, lon: 72, time: 1)\nCoordinates:\n * time (time) datetime64[ns] 2013-07-01T01:00:00\n * lev (lev) float64 0.9925 0.9775 0.9625 0.9475 0.9325 0.9175 ...\n * ilev (ilev) float64 1.0 0.985 0.97 0.955 0.94 0.925 0.91 0.895 ...\n * lat (lat) float64 -89.0 -86.0 -82.0 -78.0 -74.0 -70.0 -66.0 ...\n * lon (lon) float64 -180.0 -175.0 -170.0 -165.0 -160.0 -155.0 ...\nData variables:\n AREA (lat, lon) float32 2.16047e+09 2.16047e+09 2.16047e+09 ...\n hyam (lev) float64 0.02402 3.321 9.864 16.37 22.85 29.33 35.78 ...\n hybm (lev) float64 0.9925 0.9742 0.9526 0.9311 0.9096 0.8882 ...\n hyai (ilev) float64 0.0 0.04805 6.594 13.13 19.61 26.09 32.57 ...\n hybi (ilev) float64 1.0 0.985 0.9634 0.9419 0.9204 0.8989 ...\n P0 float64 1e+03\n CHEM_SPC_Rn (time, lev, lat, lon) float64 1.378e-21 1.378e-21 ...\n CHEM_SPC_PASV (time, lev, lat, lon) float64 3.452e-09 3.452e-09 ...\n CHEM_SPC_Be7 (time, lev, lat, lon) float64 3.971e-22 3.971e-22 ...\n CHEM_SPC_Pb (time, lev, lat, lon) float64 1.124e-20 1.124e-20 ...\nAttributes:\n title: GEOS-Chem diagnostic collection: inst\n history: Thu Sep 21 16:38:23 2017: ncatted -a History,global,o,c,Pr...\n format: NetCDF-3 with large file support\n conventions: COARDS\n ProdDateTime: \n reference: www.geos-chem.org; wiki.geos-chem.org\n contact: GEOS-Chem Support Team ([email protected])\n History: Produced on 2017-09-21 15:46\n NCO: 4.6.8",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mAssertionError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-8-6b424e70e616>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mxr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtesting\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0massert_identical\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mds\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mds2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m~/Research/Computing/miniconda3/envs/geo/lib/python3.6/site-packages/xarray/testing.py\u001b[0m in \u001b[0;36massert_identical\u001b[0;34m(a, b)\u001b[0m\n\u001b[1;32m 83\u001b[0m \u001b[0massert_identical\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_to_temp_dataset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_to_temp_dataset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 84\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mxr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataset\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mxr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mVariable\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 85\u001b[0;31m \u001b[0;32massert\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0midentical\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'{}\\n{}'\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 86\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 87\u001b[0m raise TypeError('{} not supported by assertion comparison'\n",
"\u001b[0;31mAssertionError\u001b[0m: <xarray.Dataset>\nDimensions: (ilev: 48, lat: 46, lev: 47, lon: 72, time: 1)\nCoordinates:\n * time (time) datetime64[ns] 2013-07-01T01:00:00\n * lev (lev) float64 0.9925 0.9775 0.9625 0.9475 0.9325 0.9175 ...\n * ilev (ilev) float64 1.0 0.985 0.97 0.955 0.94 0.925 0.91 0.895 ...\n * lat (lat) float64 -89.0 -86.0 -82.0 -78.0 -74.0 -70.0 -66.0 ...\n * lon (lon) float64 -180.0 -175.0 -170.0 -165.0 -160.0 -155.0 ...\nData variables:\n AREA (lat, lon) float32 2.16047e+09 2.16047e+09 2.16047e+09 ...\n hyam (lev) float64 0.02402 3.321 9.864 16.37 22.85 29.33 35.78 ...\n hybm (lev) float64 0.9925 0.9742 0.9526 0.9311 0.9096 0.8882 ...\n hyai (ilev) float64 0.0 0.04805 6.594 13.13 19.61 26.09 32.57 ...\n hybi (ilev) float64 1.0 0.985 0.9634 0.9419 0.9204 0.8989 ...\n P0 float64 1e+03\n CHEM_SPC_Rn (time, lev, lat, lon) float64 1.378e-21 1.378e-21 ...\n CHEM_SPC_PASV (time, lev, lat, lon) float64 3.452e-09 3.452e-09 ...\n CHEM_SPC_Be7 (time, lev, lat, lon) float64 3.971e-22 3.971e-22 ...\n CHEM_SPC_Pb (time, lev, lat, lon) float64 1.124e-20 1.124e-20 ...\nAttributes:\n title: GEOS-Chem diagnostic collection: inst\n history: \n format: NetCDF-3 with large file support\n conventions: COARDS\n ProdDateTime: \n reference: www.geos-chem.org; wiki.geos-chem.org\n contact: GEOS-Chem Support Team ([email protected])\n<xarray.Dataset>\nDimensions: (ilev: 48, lat: 46, lev: 47, lon: 72, time: 1)\nCoordinates:\n * time (time) datetime64[ns] 2013-07-01T01:00:00\n * lev (lev) float64 0.9925 0.9775 0.9625 0.9475 0.9325 0.9175 ...\n * ilev (ilev) float64 1.0 0.985 0.97 0.955 0.94 0.925 0.91 0.895 ...\n * lat (lat) float64 -89.0 -86.0 -82.0 -78.0 -74.0 -70.0 -66.0 ...\n * lon (lon) float64 -180.0 -175.0 -170.0 -165.0 -160.0 -155.0 ...\nData variables:\n AREA (lat, lon) float32 2.16047e+09 2.16047e+09 2.16047e+09 ...\n hyam (lev) float64 0.02402 3.321 9.864 16.37 22.85 29.33 35.78 ...\n hybm (lev) float64 0.9925 0.9742 0.9526 0.9311 0.9096 0.8882 ...\n hyai (ilev) float64 0.0 0.04805 6.594 13.13 19.61 26.09 32.57 ...\n hybi (ilev) float64 1.0 0.985 0.9634 0.9419 0.9204 0.8989 ...\n P0 float64 1e+03\n CHEM_SPC_Rn (time, lev, lat, lon) float64 1.378e-21 1.378e-21 ...\n CHEM_SPC_PASV (time, lev, lat, lon) float64 3.452e-09 3.452e-09 ...\n CHEM_SPC_Be7 (time, lev, lat, lon) float64 3.971e-22 3.971e-22 ...\n CHEM_SPC_Pb (time, lev, lat, lon) float64 1.124e-20 1.124e-20 ...\nAttributes:\n title: GEOS-Chem diagnostic collection: inst\n history: Thu Sep 21 16:38:23 2017: ncatted -a History,global,o,c,Pr...\n format: NetCDF-3 with large file support\n conventions: COARDS\n ProdDateTime: \n reference: www.geos-chem.org; wiki.geos-chem.org\n contact: GEOS-Chem Support Team ([email protected])\n History: Produced on 2017-09-21 15:46\n NCO: 4.6.8"
]
}
],
"source": [
"xr.testing.assert_identical(ds,ds2)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.2"
},
"toc": {
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"toc_cell": false,
"toc_position": {},
"toc_section_display": "block",
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment