Skip to content

Instantly share code, notes, and snippets.

View lukecampbell's full-sized avatar

Luke Campbell lukecampbell

View GitHub Profile
@lukecampbell
lukecampbell / breakpoint.py
Last active December 30, 2015 01:59
breakpoint
def breakpoint(scope=None, global_scope=None):
import traceback
from IPython.config.loader import Config
ipy_config = Config()
ipy_config.PromptManager.in_template = '><> '
ipy_config.PromptManager.in2_template = '... '
ipy_config.PromptManager.out_template = '--> '
ipy_config.InteractiveShellEmbed.confirm_exit = False
@lukecampbell
lukecampbell / logging.local.yml
Created October 16, 2013 20:55
logging.yml
root:
level: INFO
formatters:
minimal:
format: '%(message)s'
handlers:
dotfile:
class: logging.handlers.RotatingFileHandler
formatter: minimal
@lukecampbell
lukecampbell / supervisord.conf
Created October 16, 2013 19:43
supervisord
; Sample supervisor config file.
;
; For more information on the config file, please see:
; http://supervisord.org/configuration.html
;
; Note: shell expansion ("~" or "$HOME") is not supported. Environment
; variables can be expanded using this syntax: "%(ENV_HOME)s".
[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
@lukecampbell
lukecampbell / github.rb
Created October 4, 2013 18:25
github.rb A Ruby script to make dealing with github easier and from the command line
#!/usr/bin/env ruby
#
require 'slop'
require 'git'
class GitHub
def initialize(remote=nil, branch=nil, url='https://github.com/')
if File.directory? '.git'
@g = Git.open('./')
@lukecampbell
lukecampbell / assessment.md
Last active December 8, 2016 15:59
HDF Corruption Assessment

HDF Corruption

CI engineers observed an issue with attempting to retrieve data from a PREST raw data product. The dataset 8f13001a08a14162abfcc0288840f491 comprises a ViewCoverage, a ComplexCoverage and several SimplexCoverages. Each SimplexCoverage comprises several HDF5 files, each file contains a time-series dataset for a specific parameter. The ComplexCoverage combines the SimplexCoverages and provides a seamless API that aggregates the data over a universal time axis. The ViewCoverage provides a filtered view of the data, in this case all of the data is presented so it is a transparent API that delegates all calls to the ComplexCoverage. When the dataset was queried by PyDAP an exception was raised and logged. Initial investigation led to the conclusion that the data file for the data product was

@lukecampbell
lukecampbell / table.md
Last active December 17, 2015 11:19
Parameters

Begin

                  name                      |   dp_id    | combine_qc_flags |   polyval_qc   | glblrng_qc |   gradtst_qc   |   loclrng_qc   |   modulus_qc   |   solarel_qc   | spketst_qc | stuckvl_qc |   trndtst_qc

------------------------------------------------|------------|------------------|----------------|------------|----------------|----------------|----------------|----------------|------------|------------|---------------- Density | DENSITY_L2 | not applicable | not applicable | applicable | applicable | applicable | not applicable | not applicable | applicable | applicable | not applicable Flux of CO2 from the Ocean into the Atmosphere | CO2FLUX_L2 | not applicable | not applicable | applicable | not applicable | not applicable | not applicable | not applicable | applicable | applicable | not applicable Mean Point Water Velocity | VELPTMN_L1 | not applicable | applicable | applicable | not applicable | appl

@lukecampbell
lukecampbell / python.supervisord.plist
Last active December 17, 2015 01:09
launchd plist file for starting supervisord on startup
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>KeepAlive</key>
<true/>
<key>Label</key>
<string>python.supervisord</string>
<key>ProgramArguments</key>
<array>
@blazetopher
blazetopher / 00_coi-services.py
Last active December 15, 2015 20:29
Read OOI Notebook Walkthrough.md below for instructions. NOTE: There are unresolved issues with gevent. Also, this has not been tested with any kind of service/deploy support in coi-services.
# Fix the ipython path
project_path = '/Users/cmueller/Development/OOI/Dev/code/coi-services'
virtualenv_path = '/Users/cmueller/Development/OOI/Dev/virtenvs/clean27'
import IPython
IPython.sys.path = ['',
'{0}/eggs/gevent-0.13.7-py2.7-macosx-10.6-intel.egg'.format(project_path),
'{0}/eggs/coverage-3.5.2-py2.7-macosx-10.6-intel.egg'.format(project_path),
'{0}'.format(project_path),
'{0}/eggs/objgraph-1.7.2-py2.7.egg'.format(project_path),
@lukecampbell
lukecampbell / timer.py
Last active December 11, 2015 20:38
Quick with context to get the time it takes to do something in Python
#!/usr/bin/env python
'''
@author Luke Campbell
'''
import gevent_profiler
import time
class TimeIt(object):
def __init__(self, message=''):
@lukecampbell
lukecampbell / load_datasets.py
Last active December 11, 2015 10:38
Load datasets and streamer utilities
# IPython log file
from ion.services.dm.utility.granule_utils import time_series_domain
from interface.services.dm.ipubsub_management_service import PubsubManagementServiceClient
from interface.services.dm.idataset_management_service import DatasetManagementServiceClient
from interface.services.dm.iingestion_management_service import IngestionManagementServiceClient
from interface.services.sa.idata_product_management_service import DataProductManagementServiceClient
from interface.services.dm.idata_retriever_service import DataRetrieverServiceClient
from ion.services.dm.inventory.dataset_management_service import DatasetManagementService
from interface.services.sa.idata_acquisition_management_service import DataAcquisitionManagementServiceClient