Skip to content

Instantly share code, notes, and snippets.

@omad

omad/index.html

Last active December 8, 2017 01:08
Show Gist options
  • Save omad/b00a74eb143ff93fbd2bb993df200144 to your computer and use it in GitHub Desktop.
Save omad/b00a74eb143ff93fbd2bb993df200144 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
index.html
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 2017-12-08 Fri 11:59 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Discussion points - DataCube S3 Refactor</title>
<meta name="generator" content="Org mode" />
<meta name="author" content="Damien Ayers" />
<style type="text/css">
<!--/*--><![CDATA[/*><!--*/
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
font-size: medium;
font-weight: bold;
margin-top:0; }
.todo { font-family: monospace; color: red; }
.done { font-family: monospace; color: green; }
.priority { font-family: monospace; color: orange; }
.tag { background-color: #eee; font-family: monospace;
padding: 2px; font-size: 80%; font-weight: normal; }
.timestamp { color: #bebebe; }
.timestamp-kwd { color: #5f9ea0; }
.org-right { margin-left: auto; margin-right: 0px; text-align: right; }
.org-left { margin-left: 0px; margin-right: auto; text-align: left; }
.org-center { margin-left: auto; margin-right: auto; text-align: center; }
.underline { text-decoration: underline; }
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
border: 1px solid #ccc;
box-shadow: 3px 3px 3px #eee;
padding: 8pt;
font-family: monospace;
overflow: auto;
margin: 1.2em;
}
pre.src {
position: relative;
overflow: visible;
padding-top: 1.2em;
}
pre.src:before {
display: none;
position: absolute;
background-color: white;
top: -10px;
right: 10px;
padding: 3px;
border: 1px solid black;
}
pre.src:hover:before { display: inline;}
/* Languages per Org manual */
pre.src-asymptote:before { content: 'Asymptote'; }
pre.src-awk:before { content: 'Awk'; }
pre.src-C:before { content: 'C'; }
/* pre.src-C++ doesn't work in CSS */
pre.src-clojure:before { content: 'Clojure'; }
pre.src-css:before { content: 'CSS'; }
pre.src-D:before { content: 'D'; }
pre.src-ditaa:before { content: 'ditaa'; }
pre.src-dot:before { content: 'Graphviz'; }
pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-fortran:before { content: 'Fortran'; }
pre.src-gnuplot:before { content: 'gnuplot'; }
pre.src-haskell:before { content: 'Haskell'; }
pre.src-hledger:before { content: 'hledger'; }
pre.src-java:before { content: 'Java'; }
pre.src-js:before { content: 'Javascript'; }
pre.src-latex:before { content: 'LaTeX'; }
pre.src-ledger:before { content: 'Ledger'; }
pre.src-lisp:before { content: 'Lisp'; }
pre.src-lilypond:before { content: 'Lilypond'; }
pre.src-lua:before { content: 'Lua'; }
pre.src-matlab:before { content: 'MATLAB'; }
pre.src-mscgen:before { content: 'Mscgen'; }
pre.src-ocaml:before { content: 'Objective Caml'; }
pre.src-octave:before { content: 'Octave'; }
pre.src-org:before { content: 'Org mode'; }
pre.src-oz:before { content: 'OZ'; }
pre.src-plantuml:before { content: 'Plantuml'; }
pre.src-processing:before { content: 'Processing.js'; }
pre.src-python:before { content: 'Python'; }
pre.src-R:before { content: 'R'; }
pre.src-ruby:before { content: 'Ruby'; }
pre.src-sass:before { content: 'Sass'; }
pre.src-scheme:before { content: 'Scheme'; }
pre.src-screen:before { content: 'Gnu Screen'; }
pre.src-sed:before { content: 'Sed'; }
pre.src-sh:before { content: 'shell'; }
pre.src-sql:before { content: 'SQL'; }
pre.src-sqlite:before { content: 'SQLite'; }
/* additional languages in org.el's org-babel-load-languages alist */
pre.src-forth:before { content: 'Forth'; }
pre.src-io:before { content: 'IO'; }
pre.src-J:before { content: 'J'; }
pre.src-makefile:before { content: 'Makefile'; }
pre.src-maxima:before { content: 'Maxima'; }
pre.src-perl:before { content: 'Perl'; }
pre.src-picolisp:before { content: 'Pico Lisp'; }
pre.src-scala:before { content: 'Scala'; }
pre.src-shell:before { content: 'Shell Script'; }
pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
/* additional language identifiers per "defun org-babel-execute"
in ob-*.el */
pre.src-cpp:before { content: 'C++'; }
pre.src-abc:before { content: 'ABC'; }
pre.src-coq:before { content: 'Coq'; }
pre.src-groovy:before { content: 'Groovy'; }
/* additional language identifiers from org-babel-shell-names in
ob-shell.el: ob-shell is the only babel language using a lambda to put
the execution function name together. */
pre.src-bash:before { content: 'bash'; }
pre.src-csh:before { content: 'csh'; }
pre.src-ash:before { content: 'ash'; }
pre.src-dash:before { content: 'dash'; }
pre.src-ksh:before { content: 'ksh'; }
pre.src-mksh:before { content: 'mksh'; }
pre.src-posh:before { content: 'posh'; }
/* Additional Emacs modes also supported by the LaTeX listings package */
pre.src-ada:before { content: 'Ada'; }
pre.src-asm:before { content: 'Assembler'; }
pre.src-caml:before { content: 'Caml'; }
pre.src-delphi:before { content: 'Delphi'; }
pre.src-html:before { content: 'HTML'; }
pre.src-idl:before { content: 'IDL'; }
pre.src-mercury:before { content: 'Mercury'; }
pre.src-metapost:before { content: 'MetaPost'; }
pre.src-modula-2:before { content: 'Modula-2'; }
pre.src-pascal:before { content: 'Pascal'; }
pre.src-ps:before { content: 'PostScript'; }
pre.src-prolog:before { content: 'Prolog'; }
pre.src-simula:before { content: 'Simula'; }
pre.src-tcl:before { content: 'tcl'; }
pre.src-tex:before { content: 'TeX'; }
pre.src-plain-tex:before { content: 'Plain TeX'; }
pre.src-verilog:before { content: 'Verilog'; }
pre.src-vhdl:before { content: 'VHDL'; }
pre.src-xml:before { content: 'XML'; }
pre.src-nxml:before { content: 'XML'; }
/* add a generic configuration mode; LaTeX export needs an additional
(add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
caption.t-bottom { caption-side: bottom; }
td, th { vertical-align:top; }
th.org-right { text-align: center; }
th.org-left { text-align: center; }
th.org-center { text-align: center; }
td.org-right { text-align: right; }
td.org-left { text-align: left; }
td.org-center { text-align: center; }
dt { font-weight: bold; }
.footpara { display: inline; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
.inlinetask {
padding: 10px;
border: 2px solid gray;
margin: 10px;
background: #ffffcc;
}
#org-div-home-and-up
{ text-align: right; font-size: 70%; white-space: nowrap; }
textarea { overflow-x: auto; }
.linenr { font-size: smaller }
.code-highlighted { background-color: #ffff00; }
.org-info-js_info-navigation { border-style: none; }
#org-info-js_console-label
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
.org-svg { width: 90%; }
/*]]>*/-->
</style>
<script type="text/javascript">
/*
@licstart The following is the entire license notice for the
JavaScript code in this tag.
Copyright (C) 2012-2017 Free Software Foundation, Inc.
The JavaScript code in this tag is free software: you can
redistribute it and/or modify it under the terms of the GNU
General Public License (GNU GPL) as published by the Free Software
Foundation, either version 3 of the License, or (at your option)
any later version. The code is distributed WITHOUT ANY WARRANTY;
without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU GPL for more details.
As additional permission under GNU GPL version 3 section 7, you
may distribute non-source (e.g., minimized or compacted) forms of
that code without the copy of the GNU GPL normally required by
section 4, provided you include this license notice and a URL
through which recipients can access the Corresponding Source.
@licend The above is the entire license notice
for the JavaScript code in this tag.
*/
<!--/*--><![CDATA[/*><!--*/
function CodeHighlightOn(elem, id)
{
var target = document.getElementById(id);
if(null != target) {
elem.cacheClassElem = elem.className;
elem.cacheClassTarget = target.className;
target.className = "code-highlighted";
elem.className = "code-highlighted";
}
}
function CodeHighlightOff(elem, id)
{
var target = document.getElementById(id);
if(elem.cacheClassElem)
elem.className = elem.cacheClassElem;
if(elem.cacheClassTarget)
target.className = elem.cacheClassTarget;
}
/*]]>*///-->
</script>
</head>
<body>
<div id="content">
<h1 class="title">Discussion points - DataCube S3 Refactor</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#org772f239">1. Must Change</a></li>
<li><a href="#orgcfdea83">2. Use Cases</a>
<ul>
<li><a href="#org6397913">2.1. Simple</a></li>
<li><a href="#org50348e2">2.2. Compute jobs at NCI</a></li>
</ul>
</li>
<li><a href="#org4733516">3. Configuration</a></li>
<li><a href="#orga3a5ec0">4. Types of drivers/extension points</a>
<ul>
<li><a href="#org169fa1a">4.1. Database extension</a></li>
<li><a href="#org663be17">4.2. Storage read</a></li>
<li><a href="#orgb70fda0">4.3. Storage write</a></li>
</ul>
</li>
<li><a href="#orga50e33b">5. Proposed architecture</a></li>
<li><a href="#org4216c9d">6. Driver resolution for querying for data</a></li>
<li><a href="#org9cb81e4">7. Driver resolution for loading data</a></li>
</ul>
</div>
</div>
<div id="outline-container-org772f239" class="outline-2">
<h2 id="org772f239"><span class="section-number-2">1</span> Must Change</h2>
<div class="outline-text-2" id="text-1">
<ol class="org-ol">
<li>URI schema cannot be <code>s3://</code>, that is already a standard protocol, the
<code>uri_body</code> that is generated is meaningless and is ignored.</li>
<li>Change how connections to database are managed</li>
<li></li>
</ol>
</div>
</div>
<div id="outline-container-orgcfdea83" class="outline-2">
<h2 id="orgcfdea83"><span class="section-number-2">2</span> Use Cases</h2>
<div class="outline-text-2" id="text-2">
</div>
<div id="outline-container-org6397913" class="outline-3">
<h3 id="org6397913"><span class="section-number-3">2.1</span> Simple</h3>
<div class="outline-text-3" id="text-2-1">
<div class="figure">
<p><object type="image/svg+xml" data="simple_use_case.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>
</div>
</div>
<div id="outline-container-org50348e2" class="outline-3">
<h3 id="org50348e2"><span class="section-number-3">2.2</span> Compute jobs at NCI</h3>
<div class="outline-text-3" id="text-2-2">
<div class="figure">
<p><object type="image/svg+xml" data="nci_compute_job.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>
</div>
</div>
</div>
<div id="outline-container-org4733516" class="outline-2">
<h2 id="org4733516"><span class="section-number-2">3</span> Configuration</h2>
<div class="outline-text-2" id="text-3">
<ul class="org-ul">
<li>When the library is loaded, it will find a list of available datacubes/environments</li>
<li>An environment specifies which database to connect to, it could also specify which
database driver to use.</li>
<li></li>
</ul>
<p>
I think it only makes sense for a `Datacube`
</p>
</div>
</div>
<div id="outline-container-orga3a5ec0" class="outline-2">
<h2 id="orga3a5ec0"><span class="section-number-2">4</span> Types of drivers/extension points</h2>
<div class="outline-text-2" id="text-4">
</div>
<div id="outline-container-org169fa1a" class="outline-3">
<h3 id="org169fa1a"><span class="section-number-3">4.1</span> Database extension</h3>
<div class="outline-text-3" id="text-4-1">
<ul class="org-ul">
<li>Options
<ul class="org-ul">
<li></li>
<li>Only one can exist for a single instance of DataCube</li>
</ul></li>
<li></li>
</ul>
<div class="figure">
<p><object type="image/svg+xml" data="proposed_database_extension_classes.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>
</div>
</div>
<div id="outline-container-org663be17" class="outline-3">
<h3 id="org663be17"><span class="section-number-3">4.2</span> Storage read</h3>
<div class="outline-text-3" id="text-4-2">
<ul class="org-ul">
<li>Multiple should be available in the same Datacube Instance</li>
<li>Need a way to</li>
</ul>
<div class="figure">
<p><object type="image/svg+xml" data="proposed_classes.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>
</div>
</div>
<div id="outline-container-orgb70fda0" class="outline-3">
<h3 id="orgb70fda0"><span class="section-number-3">4.3</span> Storage write</h3>
<div class="outline-text-3" id="text-4-3">
<ul class="org-ul">
<li>Multiple can be available, but we don't need automatic selection. User can
specify using either a configuration file or command line arguments.</li>
</ul>
<p>
<b>*</b>
</p>
</div>
</div>
</div>
<div id="outline-container-orga50e33b" class="outline-2">
<h2 id="orga50e33b"><span class="section-number-2">5</span> Proposed architecture</h2>
</div>
<div id="outline-container-org4216c9d" class="outline-2">
<h2 id="org4216c9d"><span class="section-number-2">6</span> Driver resolution for querying for data</h2>
</div>
<div id="outline-container-org9cb81e4" class="outline-2">
<h2 id="org9cb81e4"><span class="section-number-2">7</span> Driver resolution for loading data</h2>
<div class="outline-text-2" id="text-7">
<p>
dea-clean.e1607197
</p>
</div>
</div>
</div>
<div id="postamble" class="status">
<p class="author">Author: Damien Ayers</p>
<p class="date">Created: 2017-12-08 Fri 11:59</p>
<p class="validation"><a href="http://validator.w3.org/check?uri=referer">Validate</a></p>
</div>
</body>
</html>
Display the source blob
Display the rendered blob
Raw
nci_compute_job.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
proposed_classes.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
proposed_database_extension_classes.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
simple_use_case.svg
<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="400px" preserveAspectRatio="none" style="width:608px;height:400px;" version="1.1" viewBox="0 0 608 400" width="608px" zoomAndPan="magnify"><defs><filter height="300%" id="f1m3spv0441yhc" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><text fill="#000000" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacingAndGlyphs" textLength="149" x="230" y="23.5352">Simple Data Loading</text><line style="stroke: #A80036; stroke-width: 1.0; stroke-dasharray: 5.0,5.0;" x1="26" x2="26" y1="116.9766" y2="312.8398"/><line style="stroke: #A80036; stroke-width: 1.0; stroke-dasharray: 5.0,5.0;" x1="224.5" x2="224.5" y1="116.9766" y2="312.8398"/><line style="stroke: #A80036; stroke-width: 1.0; stroke-dasharray: 5.0,5.0;" x1="423" x2="423" y1="116.9766" y2="312.8398"/><line style="stroke: #A80036; stroke-width: 1.0; stroke-dasharray: 5.0,5.0;" x1="528" x2="528" y1="116.9766" y2="312.8398"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacingAndGlyphs" textLength="31" x="8" y="114.0234">User</text><ellipse cx="26.5" cy="43.4883" fill="#FEFECE" filter="url(#f1m3spv0441yhc)" rx="8" ry="8" style="stroke: #A80036; stroke-width: 2.0;"/><path d="M26.5,51.4883 L26.5,78.4883 M13.5,59.4883 L39.5,59.4883 M26.5,78.4883 L13.5,93.4883 M26.5,78.4883 L39.5,93.4883 " fill="#FEFECE" filter="url(#f1m3spv0441yhc)" style="stroke: #A80036; stroke-width: 2.0;"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacingAndGlyphs" textLength="31" x="8" y="325.375">User</text><ellipse cx="26.5" cy="338.3281" fill="#FEFECE" filter="url(#f1m3spv0441yhc)" rx="8" ry="8" style="stroke: #A80036; stroke-width: 2.0;"/><path d="M26.5,346.3281 L26.5,373.3281 M13.5,354.3281 L39.5,354.3281 M26.5,373.3281 L13.5,388.3281 M26.5,373.3281 L39.5,388.3281 " fill="#FEFECE" filter="url(#f1m3spv0441yhc)" style="stroke: #A80036; stroke-width: 2.0;"/><rect fill="#FEFECE" filter="url(#f1m3spv0441yhc)" height="30.4883" style="stroke: #A80036; stroke-width: 1.5;" width="78" x="183.5" y="81.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacingAndGlyphs" textLength="64" x="190.5" y="102.0234">Datacube</text><rect fill="#FEFECE" filter="url(#f1m3spv0441yhc)" height="30.4883" style="stroke: #A80036; stroke-width: 1.5;" width="78" x="183.5" y="311.8398"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacingAndGlyphs" textLength="64" x="190.5" y="332.375">Datacube</text><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacingAndGlyphs" textLength="39" x="401" y="114.0234">Index</text><path d="M405.5,64.4883 C405.5,54.4883 423.5,54.4883 423.5,54.4883 C423.5,54.4883 441.5,54.4883 441.5,64.4883 L441.5,90.4883 C441.5,100.4883 423.5,100.4883 423.5,100.4883 C423.5,100.4883 405.5,100.4883 405.5,90.4883 L405.5,64.4883 " fill="#FEFECE" filter="url(#f1m3spv0441yhc)" style="stroke: #000000; stroke-width: 1.5;"/><path d="M405.5,64.4883 C405.5,74.4883 423.5,74.4883 423.5,74.4883 C423.5,74.4883 441.5,74.4883 441.5,64.4883 " fill="none" style="stroke: #000000; stroke-width: 1.5;"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacingAndGlyphs" textLength="39" x="401" y="325.375">Index</text><path d="M405.5,338.3281 C405.5,328.3281 423.5,328.3281 423.5,328.3281 C423.5,328.3281 441.5,328.3281 441.5,338.3281 L441.5,364.3281 C441.5,374.3281 423.5,374.3281 423.5,374.3281 C423.5,374.3281 405.5,374.3281 405.5,364.3281 L405.5,338.3281 " fill="#FEFECE" filter="url(#f1m3spv0441yhc)" style="stroke: #000000; stroke-width: 1.5;"/><path d="M405.5,338.3281 C405.5,348.3281 423.5,348.3281 423.5,348.3281 C423.5,348.3281 441.5,348.3281 441.5,338.3281 " fill="none" style="stroke: #000000; stroke-width: 1.5;"/><rect fill="#FEFECE" filter="url(#f1m3spv0441yhc)" height="30.4883" style="stroke: #A80036; stroke-width: 1.5;" width="137" x="460" y="77.4883"/><rect fill="#FEFECE" filter="url(#f1m3spv0441yhc)" height="30.4883" style="stroke: #A80036; stroke-width: 1.5;" width="137" x="456" y="81.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacingAndGlyphs" textLength="123" x="463" y="102.0234">Disk/S3/HTTP/etc</text><rect fill="#FEFECE" filter="url(#f1m3spv0441yhc)" height="30.4883" style="stroke: #A80036; stroke-width: 1.5;" width="137" x="460" y="311.8398"/><rect fill="#FEFECE" filter="url(#f1m3spv0441yhc)" height="30.4883" style="stroke: #A80036; stroke-width: 1.5;" width="137" x="456" y="315.8398"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacingAndGlyphs" textLength="123" x="463" y="336.375">Disk/S3/HTTP/etc</text><polygon fill="#A80036" points="212.5,143.9766,222.5,147.9766,212.5,151.9766,216.5,147.9766" style="stroke: #A80036; stroke-width: 1.0;"/><line style="stroke: #A80036; stroke-width: 1.0;" x1="26.5" x2="218.5" y1="147.9766" y2="147.9766"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacingAndGlyphs" textLength="153" x="33.5" y="143.5449">Request a region of data</text><polygon fill="#A80036" points="411.5,173.2871,421.5,177.2871,411.5,181.2871,415.5,177.2871" style="stroke: #A80036; stroke-width: 1.0;"/><line style="stroke: #A80036; stroke-width: 1.0;" x1="224.5" x2="417.5" y1="177.2871" y2="177.2871"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacingAndGlyphs" textLength="175" x="231.5" y="172.8555">Query for available datasets</text><polygon fill="#A80036" points="235.5,202.5977,225.5,206.5977,235.5,210.5977,231.5,206.5977" style="stroke: #A80036; stroke-width: 1.0;"/><line style="stroke: #A80036; stroke-width: 1.0; stroke-dasharray: 2.0,2.0;" x1="229.5" x2="422.5" y1="206.5977" y2="206.5977"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacingAndGlyphs" textLength="94" x="241.5" y="202.166">list of datasets</text><polygon fill="#A80036" points="516.5,231.9082,526.5,235.9082,516.5,239.9082,520.5,235.9082" style="stroke: #A80036; stroke-width: 1.0;"/><line style="stroke: #A80036; stroke-width: 1.0;" x1="224.5" x2="522.5" y1="235.9082" y2="235.9082"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacingAndGlyphs" textLength="207" x="231.5" y="231.4766">Load data from disk/s3/http/etc</text><polygon fill="#A80036" points="235.5,261.2188,225.5,265.2188,235.5,269.2188,231.5,265.2188" style="stroke: #A80036; stroke-width: 1.0;"/><line style="stroke: #A80036; stroke-width: 1.0; stroke-dasharray: 2.0,2.0;" x1="229.5" x2="527.5" y1="265.2188" y2="265.2188"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacingAndGlyphs" textLength="157" x="241.5" y="260.7871">data loaded into memory</text><polygon fill="#A80036" points="37.5,290.5293,27.5,294.5293,37.5,298.5293,33.5,294.5293" style="stroke: #A80036; stroke-width: 1.0;"/><line style="stroke: #A80036; stroke-width: 1.0; stroke-dasharray: 2.0,2.0;" x1="31.5" x2="223.5" y1="294.5293" y2="294.5293"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacingAndGlyphs" textLength="174" x="43.5" y="290.0977">Return an in memory xarray</text><!--
@startuml
' note right of Index : Typically PostgreSQL
' note right of [First Component]
' A note can also
' be on several lines
' end note
title Simple Data Loading
actor User
participant Datacube
database Index
collections "Disk/S3/HTTP/etc" as Storage
User -> Datacube: Request a region of data
Datacube -> Index : Query for available datasets
Index - -> Datacube : list of datasets
Datacube -> Storage : Load data from disk/s3/http/etc
Storage - -> Datacube : data loaded into memory
Datacube - -> User : Return an in memory xarray
@enduml
PlantUML version 1.2017.18(Sat Oct 07 03:56:32 AEDT 2017)
(GPL source distribution)
Java Runtime: Java(TM) SE Runtime Environment
JVM: Java HotSpot(TM) 64-Bit Server VM
Java Version: 1.8.0_152-b16
Operating System: Mac OS X
OS Version: 10.13.2
Default Encoding: UTF-8
Language: en
Country: AU
--></g></svg>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment