CI engineers observed an issue with attempting to retrieve data from a PREST raw data product. The dataset
8f13001a08a14162abfcc0288840f491
comprises a ViewCoverage
, a ComplexCoverage
and several SimplexCoverage
s. Each
SimplexCoverage
comprises several HDF5 files, each file contains a time-series dataset for a specific parameter. The
ComplexCoverage
combines the SimplexCoverage
s and provides a seamless API that aggregates the data over a universal
time axis. The ViewCoverage
provides a filtered view of the data, in this case all of the data is presented so it is a
transparent API that delegates all calls to the ComplexCoverage
. When the dataset was queried by PyDAP
an exception
was raised and logged. Initial investigation led to the conclusion that the data file for the data product was
def breakpoint(scope=None, global_scope=None): | |
import traceback | |
from IPython.config.loader import Config | |
ipy_config = Config() | |
ipy_config.PromptManager.in_template = '><> ' | |
ipy_config.PromptManager.in2_template = '... ' | |
ipy_config.PromptManager.out_template = '--> ' | |
ipy_config.InteractiveShellEmbed.confirm_exit = False | |
root: | |
level: INFO | |
formatters: | |
minimal: | |
format: '%(message)s' | |
handlers: | |
dotfile: | |
class: logging.handlers.RotatingFileHandler | |
formatter: minimal |
; Sample supervisor config file. | |
; | |
; For more information on the config file, please see: | |
; http://supervisord.org/configuration.html | |
; | |
; Note: shell expansion ("~" or "$HOME") is not supported. Environment | |
; variables can be expanded using this syntax: "%(ENV_HOME)s". | |
[unix_http_server] | |
file=/tmp/supervisor.sock ; (the path to the socket file) |
#!/usr/bin/env ruby | |
# | |
require 'slop' | |
require 'git' | |
class GitHub | |
def initialize(remote=nil, branch=nil, url='https://github.com/') | |
if File.directory? '.git' | |
@g = Git.open('./') |
Begin
name | dp_id | combine_qc_flags | polyval_qc | glblrng_qc | gradtst_qc | loclrng_qc | modulus_qc | solarel_qc | spketst_qc | stuckvl_qc | trndtst_qc
------------------------------------------------|------------|------------------|----------------|------------|----------------|----------------|----------------|----------------|------------|------------|---------------- Density | DENSITY_L2 | not applicable | not applicable | applicable | applicable | applicable | not applicable | not applicable | applicable | applicable | not applicable Flux of CO2 from the Ocean into the Atmosphere | CO2FLUX_L2 | not applicable | not applicable | applicable | not applicable | not applicable | not applicable | not applicable | applicable | applicable | not applicable Mean Point Water Velocity | VELPTMN_L1 | not applicable | applicable | applicable | not applicable | appl
<?xml version="1.0" encoding="UTF-8"?> | |
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> | |
<plist version="1.0"> | |
<dict> | |
<key>KeepAlive</key> | |
<true/> | |
<key>Label</key> | |
<string>python.supervisord</string> | |
<key>ProgramArguments</key> | |
<array> |
# Fix the ipython path | |
project_path = '/Users/cmueller/Development/OOI/Dev/code/coi-services' | |
virtualenv_path = '/Users/cmueller/Development/OOI/Dev/virtenvs/clean27' | |
import IPython | |
IPython.sys.path = ['', | |
'{0}/eggs/gevent-0.13.7-py2.7-macosx-10.6-intel.egg'.format(project_path), | |
'{0}/eggs/coverage-3.5.2-py2.7-macosx-10.6-intel.egg'.format(project_path), | |
'{0}'.format(project_path), | |
'{0}/eggs/objgraph-1.7.2-py2.7.egg'.format(project_path), |
#!/usr/bin/env python | |
''' | |
@author Luke Campbell | |
''' | |
import gevent_profiler | |
import time | |
class TimeIt(object): | |
def __init__(self, message=''): |
# IPython log file | |
from ion.services.dm.utility.granule_utils import time_series_domain | |
from interface.services.dm.ipubsub_management_service import PubsubManagementServiceClient | |
from interface.services.dm.idataset_management_service import DatasetManagementServiceClient | |
from interface.services.dm.iingestion_management_service import IngestionManagementServiceClient | |
from interface.services.sa.idata_product_management_service import DataProductManagementServiceClient | |
from interface.services.dm.idata_retriever_service import DataRetrieverServiceClient | |
from ion.services.dm.inventory.dataset_management_service import DatasetManagementService | |
from interface.services.sa.idata_acquisition_management_service import DataAcquisitionManagementServiceClient |