Skip to content

Instantly share code, notes, and snippets.

@kwilcox
kwilcox / test_open.py
Last active November 10, 2015 16:52
netCDF4-python dataset objects getting garbaged collected inside context manager?
#!/usr/bin/env conda-execute
# coding=utf-8
"""
Testing netCDF4 garbage collection differences between 2.7 and 3.4
**Fails**
- python 3.4 / netCDF4-python 1.2.1, 1.2.0, 1.1.9, and 1.1.8
**Passes**
- python 3.4 / netCDF4-python 1.1.6
Frontiers in Ecology and the Environment, September, Vol. 13, No. 7 : 369-377
Arctic tundra fires: natural variability and responses to climate change
Feng Sheng Hu, Philip E Higuera, Paul Duffy, Melissa L Chipman, Adrian V Rocha, Adam M Young, Ryan Kelly, and Michael C Dietze
(doi: 10.1890/150063)
Frontiers in Ecology and the Environment, September, Vol. 13, No. 7 : 363-368
Filling the gaps: sensor network use and data-sharing practices in ecological research
Christine M Laney, Deana D Pennington, and Craig E Tweedie
(doi: 10.1890/140341)
@kwilcox
kwilcox / 2500_constant_tag.influx
Last active August 29, 2015 14:27
InfluxDB profiling with 2500 inserts. Varying tags is 17x faster.
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.611,unit="m/s",value=2.682 1028854800000000000
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.611,unit="m/s",value=2.682 1028858400000000000
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.611,unit="m/s",value=2.682 1028862000000000000
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.611,unit="m/s",value=2.682 1028865600000000000
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.611,unit="m/s",value=2.682 1028869200000000000
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.611,unit="m/s",value=2.235 1028872800000000000
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.611,unit="m/s",value=2.235 1028876400000000000
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.611,unit="m/s",value=1.788 1028880000000000000
wind_speed,depth=4.6 authority="wban",label="27516",lat=71.321,lon=-156.
@kwilcox
kwilcox / getlegendgraphic.ipynb
Created July 13, 2015 14:37
GetLegendGraphic in sci-wms
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
$ python collect.py --projects BW2011 --output="./out"
2015-07-02 11:04:49 terrapin root[5654] DEBUG Could not find CF mapping for EPIC code 624
2015-07-02 11:04:49 terrapin root[5654] DEBUG Could not find CF mapping for EPIC code 624
2015-07-02 11:04:49 terrapin root[5654] DEBUG Could not find CF mapping for EPIC code 500
2015-07-02 11:04:49 terrapin root[5654] DEBUG Could not find CF mapping for EPIC code 502
2015-07-02 11:04:49 terrapin root[5654] INFO Translating /home/kwilcox/Development/usgs-cmg-portal/woods_hole_obs_data/download/9061ysi-a.nc into CF1.6 format: /home/kwilcox/Development/usgs-cmg-portal/woods_hole_obs_data/out/BW2011/9061ysi-a.nc
2015-07-02 11:04:49 terrapin root[5654] DEBUG Could not find CF mapping for EPIC code 500
2015-07-02 11:04:49 terrapin root[5654] DEBUG Could not find CF mapping for EPIC code 502
2015-07-02 11:04:49 terrapin root[5654] DEBUG Could not find CF mapping for EPIC code 624
(larva)kwilcox@terrapin:~/Development/paegan-transport (master)$ py.test -s -k test_run_from_multiple_files_without_cache
================================================================ test session starts ================================================================
platform linux2 -- Python 2.7.6 -- py-1.4.28 -- pytest-2.7.1
rootdir: /home/kwilcox/Development/paegan-transport, inifile:
collected 106 items
tests/test_model_controller.py test_model_controller.ModelControllerTest.test_run_from_multiple_files_without_cache
*** BaseModelController ***
***** Transport *****
upstream sciwms {
server 127.0.0.1:{{sciwms_gunicorn_port}};
keepalive 32;
}
server {
listen {{sciwms_nginx_port}};
client_max_body_size 4G;
keepalive_timeout 5;
server_name _;
@kwilcox
kwilcox / gist:042a76d27b5e2760d11d
Created April 10, 2015 20:01
9141wh-a.nc processing
(usgs_cmg)kwilcox@terrapin:woods_hole_obs_data$ python collect.py --download -p HURRIRENE_BB -f ./test_issue_51_download/ -o ./test_issue_51_output
2015-04-10 15:53:50 terrapin thredds_crawler[14577] INFO Crawling: http://geoport.whoi.edu/thredds/catalog/usgs/data2/emontgomery/stellwagen/Data/HURRIRENE_BB/catalog.xml
2015-04-10 15:53:51 terrapin thredds_crawler[14577] INFO Ignoring dataset based on 'selects'. ID: usgs/data2/emontgomery/stellwagen/Data/HURRIRENE_BB/9141p-cal.cdf
2015-04-10 15:53:51 terrapin root[14577] INFO Found 3 datasets in HURRIRENE_BB!
2015-04-10 15:53:51 terrapin root[14577] INFO Found 3 TOTAL datasets!
2015-04-10 15:53:51 terrapin root[14577] INFO Downloading http://geoport.whoi.edu/thredds/fileServer/usgs/data2/emontgomery/stellwagen/Data/HURRIRENE_BB/9141wh-a.nc
2015-04-10 15:53:54 terrapin root[14577] INFO 9141wh-a.nc saved (1/3)
2015-04-10 15:53:54 terrapin root[14577] INFO Downloading http://geoport.whoi.edu/thredds/fileServer/usgs/data2/emontgomery/stellwagen/Data/HURRIRENE_BB/91
@kwilcox
kwilcox / epic.py
Created November 20, 2014 17:49
EPIC to CF Conversion
#!python
# coding=utf-8
metadata_codes = []
metadata_codes += [1215, 1216, 1217, 1218, 1219, 1220, 1404, 1411, 1412, 1413] # Instrument and platform pitch/roll/orientation
metadata_codes += [9991, 9992, 9993, 9994] # x/y/z/t axes
metadata_codes += [1207, 1208] # ADCP records/pings
metadata_codes += [106] # Battery voltage
metadata_codes += [4000, 40001, 40002, 4003, 4004, 4005, 4006] # Rotor speed
metadata_codes += [106, 113, 114, 115] # Battery voltage
@kwilcox
kwilcox / gist:ac9e37d46d1e4b0e83b0
Created November 13, 2014 20:14
Ghosted FreeIPA Host
[root@auth ~]# ipa host-find hassium.axiompvd
--------------
1 host matched
--------------
Host name: hassium.axiompvd
Principal name: host/hassium.axiompvd@AXIOM
Password: False
Keytab: True
SSH public key fingerprint: xxxxxxxxxxxx root@hassium (ssh-dss), xxxxxxxxxxxxxx root@hassium (ecdsa-sha2-nistp256), xxxxxxxxxxxxx root@hassium (ssh-rsa)
----------------------------