tnachen

• http://christophermeiklejohn.com/distributed/systems/2013/07/12/readings-in-distributed-systems.html
• http://dancres.org/reading_list.html
• http://rxin.github.io/db-readings/
• http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html
• http://www.eecs.berkeley.edu/GradAffairs/CS/Prelims/db.html
• http://pdos.csail.mit.edu/dsrg/papers/
• http://scalingsystems.com/2011/09/07/reading-list-for-distributed-systems/
• http://courses.engr.illinois.edu/cs525/sp2011/sched.htm
• http://henryr.github.io/distributed-systems-readings/
• http://qr.ae/NQjNG

tnachen

Drill Window Notes:
New Function Scope for Window queries
Function rename in DrillOptiq for window functions, add "_win" perhaps

Let's focus on these window functions:
---------------------------------------
lead
rank
lag
percentile

tnachen

#include <stdio.h>
#include <stdlib.h>
#include <iostream>
#include <sys/sysinfo.h>
#include <errno.h>

int main(int argc, char** argv)
{
  struct sysinfo info;
  if (sysinfo(&info) != 0) {

tnachen

FROM stackbrew/ubuntu:13.10
MAINTAINER Mesosphere [email protected]

RUN ["env", "DEBIAN_FRONTEND=noninteractive", "apt-get", "update"]
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "apt-get", "install", "-y", "--fix-missing", "--force-yes", "python2.7", "libsasl2-2", "libcurl3", "curl", "wget", "unzip"]
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "apt-get", "install", "-y", "--fix-missing", "--force-yes", "python", "python-setuptools", "python-pip", "python-protobuf"]
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "apt-get", "install", "-y", "--fix-missing", "--force-yes", "default-jre-headless"]

ADD http://downloads.mesosphere.io/master/ubuntu/13.10/mesos_0.19.0-xcon2_amd64.deb /tmp/mesos.deb
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "dpkg", "-i", "/tmp/mesos.deb"]

tnachen

Bio

Timothy Chen is a Apache Mesos PMC and committer, and lead the development for the Docker on Mesos integration at Mesosphere. He's also a Apache Drill committer and contributed to Apache Kafka, Spark and Hive. His previous work experiences includes PaaS CloudFoundry at VMWare and Cloud services for the game industry including the Halo series. 

Abstract

"Scale your Docker containers with Mesos"

Docker provided an awesome way for developers to isolate and ship dependencies of their production applications, and there is a vibrant community that provides lots of different images. However, to deploy and scale these docker containers as distributed systems still remains a challenge for the users. Apache Mesos is a resource management system that provides the primitives to scale and operate distributed systems. With the latest release Docker is integrated into Mesos and in this talk I'll talk about the motivation and architecture of the integration, and how to scale Docker containers and tap into the rich fe

tnachen

struct Size {
  int diff;
  int orig;
  Duration duration;
};

const int STEP = 16 * 1024;

static void benchDiff(int size, double diffPercent, bool silent)
{

tnachen

Matching: )^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@, Event method: )^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@

Matched!

Update Future finished in test.

Method: )^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
0   libmesos-0.22.0.dylib               0x000000010d9cf42e _ZN7process8internal8dispatchERKNS_4UPIDERKNSt3__110shared_ptrINS4_8functionIFvPNS_11ProcessBaseEEEEEERKNS4_12basic_stringIcNS4_11char_traitsIcE\
ENS4_9allocatorIcEEEE + 238
1   libmesos-0.22.0.dylib               0x000000010cefa65f _ZN7process8dispatchIN5mesos8internal5slave10LimitationENS3_15IsolatorProcessERKNS1_11ContainerIDES6_EENS_6FutureIT_EERKNS_3PIDIT0_EEMSD_FSB_T1_\

tnachen

diff --git a/src/launcher/fetcher.cpp b/src/launcher/fetcher.cpp
index 4bda91c..bcd90ae 100644
--- a/src/launcher/fetcher.cpp
+++ b/src/launcher/fetcher.cpp
@@ -89,8 +89,7 @@ Try<bool> extract(const string& filename, const string& directory)
 // Attempt to get the uri using the hadoop client.
 Try<string> downloadWithHadoopClient(
     const string& uri,
-    const string& directory,
-    const string& basename)

tnachen

I0627 00:04:39.045922 24200 slave.cpp:620] Successfully attached file '/tmp/mesos/slaves/20150623-193734-918679562-5050-12782-S22/frameworks/20150623-193734-918679562-5050-12782-004│·····················
9/executors/test-task/runs/08285ac6-ad2f-42e8-8edd-0a80b8a9e828'                                                                                                                     │·····················
I0627 00:04:39.052626 24208 store.cpp:200] Stored layer with hash: 511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158                                                  │·····················
I0627 00:04:39.053170 24208 store.cpp:200] Stored layer with hash: df7546f9f060a2268024c8a230d8639878585defcc1bc6f79d2728a13957871b                                                  │·····················
I0627 00:04:39.053649 24208 store.cpp:200] Stored layer with hash: ea13149945cb6b1e746bf28032f02e9b5a793523481a0a18645fc77ad53c4ea2                                                  │·····················
I062

tnachen

53a5f78a-f0f3-4273-8ba1-f29ac11341b5,mesos.internal.log.PromiseRequest,ff2f5c26-e4cc-4626-bf06-3e27b2bdbdec,,(3)@10.0.77.237:5050,log-replica(1)@10.0.77.237:5050,1438288302620932096,message_outbound_queued
53a5f78a-f0f3-4273-8ba1-f29ac11341b5,mesos.internal.log.PromiseResponse,c75db6b9-7066-4ecc-a19d-3fe5f9c07702,ff2f5c26-e4cc-4626-bf06-3e27b2bdbdec,log-replica(1)@10.0.77.237:5050,(3)@10.0.77.237:5050,1438288302626352128,message_outbound_queued
f95fac79-d1ed-4e17-9c71-6b9a19594cab,mesos.internal.log.WriteRequest,a2cc030c-7128-4f06-a828-60d8789f6d3b,,(5)@10.0.77.237:5050,log-replica(1)@10.0.77.237:5050,1438288302656142080,message_outbound_queued
f95fac79-d1ed-4e17-9c71-6b9a19594cab,mesos.internal.log.WriteResponse,f27a49ef-f14e-4c75-8f2a-d8d2142d6282,a2cc030c-7128-4f06-a828-60d8789f6d3b,log-replica(1)@10.0.77.237:5050,(5)@10.0.77.237:5050,1438288302663973120,message_outbound_queued
9f6bb86d-7d3d-4b7b-93eb-842b35411d45,mesos.internal.log.LearnedMessage,36b1e706-b33c-482e-8694-28ec66168792,,@0.0.0.0:0,log-replic