Spark 1.1.0 release candidate benchmarking

.gitignore

results/

Spark 1.1.0 Release Candidate Benchmarking.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:0cda8a7f156fe5aa7a7ad230db6562ead56b60b12f6878d91aef417e04cf40a3"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Spark 1.1.0 Release Candidate Benchmarking (snapshot1)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This notebook contains the results of performance benchmarks that [I'm](https://github.com/JoshRosen) running to test the Spark 1.1.0 release candidates.  It will be updated as I run more tests and as new release candidates are published.\n",
      "\n",
      "These benchmarks use the [spark-perf](https://github.com/databricks/spark-perf) test suite."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### @pwendell's tips for cluster configuration\n",
      "\n",
      "1. By default, spark-ec2 will use the existing formatted EXT3 drives that exist on most instance types. These will kill performance  benchmarks.\n",
      "2. For r3.X, the current spark-ec2 scripts will actually format new  drives with ext4. Might be good to start with these:  https://github.com/mesos/spark-ec2/blob/v3/setup-slave.sh#L20 \n",
      "3. For other types, I have a special forked version of spark-ec2 that can format and mount the drives as XFS (I think this is out of date though): https://github.com/pwendell/spark-ec2/commits/xfs "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Plotting code:\n",
      "from glob import glob\n",
      "import os\n",
      "import numpy as np\n",
      "import pandas as pd\n",
      "from pandas import DataFrame as df\n",
      "import seaborn  # Fancier plots\n",
      "seaborn.set_style(\"whitegrid\")\n",
      "\n",
      "def load_spark_results(log_dir):\n",
      "    logs = glob(os.path.join(log_dir, \"*.out\"))\n",
      "    results = {}\n",
      "    for log in logs:\n",
      "        test_name = os.path.split(log)[-1][:-4]  # Strip \".log\" suffix\n",
      "        with open(log) as f:\n",
      "            result_line = f.readlines()[-1]\n",
      "        assert \"results:\" in result_line\n",
      "        comma_separated_results = result_line.replace(\"results:\", \"\", 1).strip()\n",
      "        times = eval('[' + comma_separated_results + ']')\n",
      "        results[test_name] = np.array(times)\n",
      "    return results\n",
      "    \n",
      "    \n",
      "def plot_results(results, exclude_first_run=True):\n",
      "    if exclude_first_run:\n",
      "        results = results.applymap(lambda x: x[1:])\n",
      "    means = results.applymap(mean)\n",
      "    errors = results.applymap(std)\n",
      "    means.plot(xerr=errors, kind='barh')\n",
      "    seaborn.despine(trim=True);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Default `spark-perf` Spark Benchmarks on 20 m1.xlarge nodes (default filesystem)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "_Note_: These were run with the default filesystem, so these may not be representative of the best possible performance.  However, this preliminary tests show that 1.1.0 doesn't have any glaring performance regressions."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "results = df({\n",
      "    \"1.0.2\": load_spark_results(\"results/2014-08-08/20_m1.xlarge/spark_perf_output_origin-tag-v1.0.2_2014-08-08_21-44-00_logs\"),\n",
      "    \"1.1.0-snapshot-1\": load_spark_results(\"results/2014-08-08/20_m1.xlarge/spark_perf_output_origin-tag-v1.1.0-snapshot1_2014-08-08_21-13-04_logs/\")\n",
      "})\n",
      "\n",
      "\n",
      "plot_results(results, exclude_first_run=True);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAFRCAYAAACc+BKVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VNW99/FPCBEUI4j1RlWQ6M9TThQ1oi2tipV6Kd6K\nlwqcVjBWsWL1cPoIXhClKm1VTlFRkIt4BdtHtKIVL+AFVNSkVkVtfwo8VMBj8JFgEgoCmeePvcMz\nDBNym7Ans7/v1yuvTGbWXus3s435staaPSAiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiKS\nNfKiLkAyb8mSJYl///d/j7oMaaYPP/wQnb+2S+ev7dK5a9vy8vKalGnatVYhEp0f/ehHUZcgLbBh\nw4aoS5AW0Plru3Tu4kUBSERERGJHAUhERERiRwFIREREYkcBSERERGJHAUhERERiRwEoB82dOzfq\nEkRERLJa+6gLkMxbsWIFhYWFUZchDSgqKiI/Pz/qMkQkyZYtW1i6dGlG+9TvenZSAMpBfyibQacV\nCkDZrGZNFdOGTsDMoi5FRJIsXbqUn137GLt13icj/a1fV8HD4wc3+Lv+3nvvcccdd/Dwww9vc/+C\nBQu49957ad++Peeeey7nn3/+No+vXr2a6667ji1btgAwbtw4Dj744IzUnusUgHJQp70LKezWJeoy\nRETapN0678Pue357p403depUnn76aTp16rTN/Zs2beK3v/0tTzzxBB07dmTQoEH88Ic/ZK+99tra\n5q677uJnP/sZJ598MosWLWLChAncfffdO632tkx7gERERCLUvXt37rnnHhKJxDb3L126lIMOOojC\nwkIKCgooKSnhnXfe2abNqFGjOPHEEwHYvHkzHTp02Gl1t3UKQCIiIhE65ZRT0u4Rqq6u3mY/Z6dO\nnaiqqtqmzZ577kn79u1ZtmwZv//97xkxYkSr15srtASWg175zdOced/Poy4j1j6a8MYOH09srqV0\nTikFBQXbPbZx48Zt/hW3YMGCjNcnItmvsLCQmpqarT/X1NTQuXPn7dotXryYcePGcfvtt9OjR4+d\nWGHblvEZIDPrZ2azmnnsBw08/nn4/b/N7MDmjJHU1wlmdnh4+39a0lcjx2v265LUx4FmdkamahIR\nkezVs2dPVqxYwbp16/jmm2945513OPLII7dps3jxYm677TamT5+uT7JvotaYAUo03KRl3P0/M9DN\nxcAs4AN2Qs0ZGuNk4DDgmQz0Ja2o18i+O3y8anUlEwfcnPadIeXl5ZSUlLRWaSLSgPXrKiLpKy8v\nD4BnnnmG9evXc8EFFzB69GhKS0upra3lvPPOY5999qGyspIxY8Zw9913M378eDZv3sw111wDwMEH\nH8y4ceMyVn8uy2tKYwv+b/0AsIlg9mgwMBroA+wCjAXWAXcCFcA+wFx3vzmcbZkYjvl/CQJINTAZ\nOAL4DDjW3bub2Uxglrs/b2anAT9192Fm9rm7729mrwCXAYOAHuE43YH/dPcXwlmSm8Na1gLvu/vN\nSc+jhCBEfAGcCbwHPAccFNZ2HnAD0BfoBJQCA4CfApuB19x9tJndBHzu7lPM7N+A+9z9pHTjA6/U\n87q8ArwLHAXUAhcCvYDL3H1QWO/nwLeBj4BdgSvcvd4Q1HHP3RJaAstuCkC5S+ev7SovL+fII4/U\ndYDaqLy6BNlITZ0B6g8sBkYBxwNDga7ufpyZdQFGAvOBjsDZYf//JAgDU4Gh7v53M7sYuAYoB3Zz\n9++a2beAT8NxEvz/GZN0MyfJj21w9x+bWX/gv8zsJYKg9V13X2Nmj6T24e7lZjaPIGR9Zma7A9e6\n+z/N7GWCMJIAPnT3/wzD2/nA99x9i5k9YWYD0tVmZu12MH661yUBvBSOMwK4HpiT2q+715rZeOCw\nHYUfgERtgqrVlTtqIhGrWVPF8uXL0z5W34Us9T9RkdaXn5+v63PFRFMD0HSC8DOPYHbjbYJAhLtX\nAjeaWT9gibtvAjaZ2ebw2O8A94X/YRUAnwA1wDvh8V+a2cdpxmxon9Lfwu8rCQLG3sDX7r4mvH8h\nsJ+ZXUEws5MA/iOlj6/c/Z/h7f8Bdgtve/j934DF7r4lqc/Uxda65Jl2/PB2utcF4MXw++sEM02p\n8pK+N5hwhw0p5a+LujbUTCI2fvUn5LWr51+az2y7LW39ugpGDTmC7t2774TKpKXKy8ujLkGaSecu\nPpoagM4GFrr7ODMbBNxK8EcbM+tMsKfmd6Sftfk78DN3X2lmJwB7ESz5DAYmmtmeQF3s3gB0C28f\n3UBNqWNVAIVm9i13/xL4HrDc3ScBk+oamVktkF9PH3Vqw+8fAyPNLD+87wTgQYJgtH9KnWnHb2Cc\n4wiCUl+CPUkb6vo1s+5AXZrZQiM2rl9y8VBuemxlQ82kjSkuLta/TNsALYG1XTp38dLUd4GVAePM\nbD5wKXAusNbMFhLMCk0M2yX/oa+7fTnwcNj2FuADd/8z8LmZvQXMIJh9AZgG/KeZvUgQhBqzHAaQ\ncPcEMAL4S3j8gQR7llK9BYwP9+7UF0wSAO6+BPgjQdh7iyBQ/Rl4HPhx8rJZA+One10Argj3Ap1K\nECrLgEozWwzcBCwL230AnG1mF9RTr4iIiDRCkzYMtRVmNhqY4O7fmNnDwPPu/kg2jh+Gp3Pd/atM\njV9WVpbQDFBuqV67iimj+2sGqA3QLELbpXPXtrX2Jui2ogpYbGbrCZafHo/Z+BKxZfPHZ7S/RO1m\nSktnpb1wYkvoIosi29KnwcdHTgag1P0+2Ty+u5/UyuWIiEgjLV26lEtmjqTT3tu/E7M5atZUMW3o\nhGZ/GjzAv/71L4YNG8Ztt91Gz549t3nsq6++4te//jUbN25kn332Yfz48XTs2DEjtTfH6NGjGTBg\nAMcff3yDbdetW8fChQs544z01/ddsWIFI0aMYO7cuZkuE8jRACQStZ4nX5vR/rQEJrLzdNq7kMJu\nXXbaePV9GjzABx98wNixY6moqCDdCs+9997LWWedxTnnnMP999/P7NmzGTp06E6oOr2mrEL9/e9/\nZ8GCBWkD0FNPPcXDDz/M2rVrM1neNvRhqCIiIhGq79PgATZt2sS9997LwQcfnPbYv/71r1tnW044\n4QTefPPN7dq88MILXHDBBQwePJirr76aRCLB3XffzejRo/nFL37BgAEDWLRoEQDz5s3j5z//OYMH\nD2bIkCGsXbuWt956i+HDh3PxxRdz1llnMWtW8KlOjz76KBdccAEXXnght9xyy9bxHn/8cS666CIG\nDhzI+++/D8CMGTM477zzuPDCC7njjjsAmDx5MosXL+ZPf/rTdjV36dKFRx5p3a27mgESERGJ0Cmn\nnMLKlenfuHL00Tu+EkzyJ8an+7R4gGeffZZLLrmEU045haeeeorq6mry8vLYZZddmDp1Km+88QYz\nZszgBz/4AStWrOD++++nY8eO3HjjjSxatIh9992XyspKHn30Ub755hvOOussTj31VJ588kluuukm\niouLmTVrFlu2BJfKKy4uZvjw4Tz55JM8+eSTdOzYkXnz5vH444+Tn5/PlVdeySuvvMLll1/O7Nmz\nOf/887eruV+/fk18FZtOAShHVa9dFXUJkkGZ/GwiEckdu+++O9XV1XTt2pWamhr22GMPysvL+cMf\n/gDAJZdcwrXXXsuUKVN4+OGH6dmzJ/379wegV69eAOy7775s3LgRgK5duzJq1Ch22203li9fzlFH\nHQVAnz59yM/PZ9ddd+XQQw/ls88+Y/z48cyYMYOVK1dy5JFHbp3BqvtQ1m9961ts2LCBZcuW0bt3\n760bwUtKSvjkk0/o3bv31udx2WWXsX79eg477DBuuOGGnfDKKQDlrCmj+0ddgjTTkiVLKC4u3u7+\noqKiCKoRkWx29NFH8+qrr/KTn/yE1157jWOOOYaSkpJtNlNPnDiRK6+8kq5du3LjjTfy0ksvpe2r\nurqau+++m1dffZXa2louvvjiraFmyZIlQLAhe+nSpXTv3p1JkyZx8803s8suu1BaWsq77767TX91\nx/bs2ZMHHniALVu20K5dO8rKyjjnnHNo164dtbXB9YanTJmS8demIQpAOUqbZduuqqoqnT+RCNWs\n2X4ZaWf0le7T4NNJ/jT4yy+/nFGjRvHHP/6Rrl27cuedd27X/ogjjuCyyy6jU6dOdOrUiX79+vHI\nI49ss2E5Ly+P3XffnaOPPpqf/vSndO3alYMPPpg1a9ZwwAEHUF1dzbBhw1i3bh0jRoygS5cumBmD\nBw+mU6dO7LfffvTu3Zs5c+Zs7bfuu5lx+umnM2jQIGpraznmmGPo378/X3zxBe7OQw89xM9/vvM/\nwDsnL4QYd2VlZQldzKvt0sXY2jadv7ZLnwaf3ltvvcULL7zAmDFjoi5lh3QhRBERkWbSp8FvLy8v\nr0lvb28rFIBERESkXsceeyzHHnts1GVknK4DJCIiIrGjACQiIiKxowAkIiIisaMAJCIiIrGjACQi\nIiKxowAkIiIisaMAJCIiIrGjACQiIiKxowAkIiIisaMAJCIiIrGjACQiIiKxowAkIiIisaMPQ81R\n7h51CbFSVFREfn5+1GWIiEgjKQDlqKueHRt1CbFRs6aKaUMnYGZRlyIiIo2kAJSjCrt1iboEERGR\nrKU9QCIiIhI7CkAiIiISO1oCk1j6aMIbGesrsbmW0jmlFBQUZKS/22+/PSP9iIhI/bJqBsjMZprZ\nqTthnBH13P9BM/rqYWZvtrwqMLPeZjZmB493MLPSTIwlIiISZ9k2A5QIv1rb9cA9O2GcJnH394D3\ndtBkf+ASYPrOqSh39RrZN2N9Va2uZOKAmzP2LrDy8vKM9CMiIvXLeACy4K/AA8AmghmmwcBooA+w\nCzAWeAa4HziA4I/60+4+JqmPQoI/8p2BbsAkd5/c0DjuvtLM7gS+HzZ7zN3vMrOZQFdgL+BZoKuZ\n3ePuqTNBnc1sDrAP8K67X2lmi4BL3f0jMzsdOMPdr0jzvNsBDwIfuPvvzexKYBBBoJsNTAL+AfRx\n90ozuxzY3d1vT+qjH3CZuw8ys0+ARcBhwBfAuQTBrZeZ3eDut+zoPIiIiEj9WmMJrD+wOPw+FhgK\ndHX344CTgGOAA4E33f004DhgeNLxecAhwCx3PxU4FRjZiHE6m9kZQA93/y7wA2CwmRUThJD57v59\nd78N+CpN+AEoJAg7PwD2MbMzgWnAReHjFwNT0xzXHngUeD0MP72ACwiC2AnAOeFzepQgFAEMAWam\n6avOwcAN7t4X2JsgQN4CfKTwIyIi0jKtsQQ2HRgFzAPWAW8TBBXcvRK40cz2APqY2UnA10CHlD4q\ngKvNbGD4eHsz60Qwe5MAXgAmpIxzHfBvwMJwrM1mthjoFfa53aWRzewWgpCSIAhSH7v7l+HDbxLM\nvtwLlJvZHcC33f1vZjYNKArrvAboHdZQGB5bDHQHFoQ/dyEIQDOA2Wb2GvCFu68xs2eATsAHwBNJ\n5X3p7qvC25+Fr1Hedq92PapWVza2qbRQzZoqli9fnrH+tmzZkrG+REQkvdYIQGcDC919nJkNAm4F\nXgcws87ALOA5oNLdh5vZIcClKX2MJJghmhyGpAHuXgP0q2tgZhekjDOKIEAMA/5gZgVAX4JlqdOB\n2qT+8wDc/YbkQc3sUDPbE6gEjgcmu/t6M3sZmAg8HB53SdIxPYBy4AzgbTObB/wd+NDdTw/bjATe\nD5foKgmWsqaFfZ2R1NfW50f6vVBbaOSs3VeLdFXinWn86k/Ia7e0xf2sX1fBqCFH6GM12jjt42q7\ndO7iozUCUBnwoJl9Q/DH+lxgmJktDMe7iWBG4zEzKwFWAGVm1i08PgHMBe42s58AHwJVZlbg7pvq\nGScfuDqcnelnZm8Q7Dd63N3fDTenJgeKj8zsIXf/eUrtawj2Fe0DvOruL4b3TyXYjzOc9BLuviHc\n1/MQwbLe/HD/UEeCGbDVSX3dRbAEtl0/SXWmC0AVwC5mNt7dr62nFgD26XH0jh6WLFdSUhJ1CdJM\n5eXlOn9tlM5dvDR6SSXOzOwYYIS7D81AX+cBxe5+U0v7qk9ZWVnipsdWtlb30oqq167iyjP2Y+DA\ngVGXIs2kP6Jtl85d25aXl9ekTJNtb4PPOuE1gy4Gzs9AX7cBJxIsl0mOWDZ/fMb6StRu5ncf5nPP\nPZm7SsOCBQsabiQiEjMKQA1w93vI0DWD3P26TPQjIiIiLaMAJNJCPU/e4XasJtESmIjIzpFVH4Uh\nIiIisjMoAImIiEjsaAksR1WvXdVwI8k669dVAPtFXYaISM5TAMpRU0b3j7oEaabKSl3FW0SktSkA\n5ahMfTK57Hy6Eq2ISOvTHiARERGJHQUgERERiR0FIBEREYkdBSARERGJHQUgERERiR0FIBEREYkd\nBSARERGJHQUgERERiR0FIBEREYkdBSARERGJHQUgERERiR0FIBEREYkdBSARERGJHQUgERERiR0F\nIBEREYkdBSARERGJnfZRFyCtw92jLiGnFRUVkZ+fH3UZIiLSTApAOeqqZ8dGXULOqllTxbShEzCz\nqEsREZFmUgDKUYXdukRdgoiISNbSHiARERGJHc0AiaT4aMIbO3w8sbmW0jmlFBQU7LDdggULMlmW\niIhkkGaAREREJHZyYgbIzGYCs9z9+VYeZ4S735Pm/g/c/fAm9tWDoObvZao+yYxeI/vu8PGq1ZVM\nHHCzNkGLiLRhuTIDlAi/Wtv1O2EMERERaWVZMwNkwT+nHwA2EQSzwcBooA+wCzAWeAa4HzgA2B94\n2t3HJPVRCEwHOgPdgEnuPrmhcdx9pZndCXw/bPaYu98Vzix1BfYCngW6mtk97j4ipfzOZjYH2Ad4\n192vNLNFwKXu/pGZnQ6c4e5XpHne7YAHgQ/c/fdmdiUwiCDQzQYmAf8A+rh7pZldDuzu7rc3/tUV\nERGRZFkTgID+wGJgFHA8MBTo6u7HmVkXYCTwHvCmu083s47AZ0BdAMoDDiFYVnrSzLoBrwDbBKA0\n43Q2syOBHu7+XTNrDywyswUEIWS+u08EMLMr04QfgEKCsPOlmT1uZmcC04CLwnEuBm5Nc1x74FHg\nVXefbGa9gAsIglg74AXg+bDNIOA+YAjwk4ZezKrVlQ01kWaqWVPF8uXLW9SHLqQoIhKtbApA0wnC\nwjxgHfA2QVDB3SuBG81sD6CPmZ0EfA10SOmjArjazAaGj7c3s04EszcJgkAxIWWc64B/AxaGY202\ns8VAr7DP7S6pbGa3EISUBEGg+tjdvwwffhM4DLgXKDezO4Bvu/vfzGwaUBTWeQ3QO6yhMDy2GOgO\n1L19qAtBqJsBzDaz14Av3H1NQy/mV4u0P6U1jV/9CXntljbr2PXrKhg15Ai6d+9eb5vy8vLmliZZ\nQOev7dK5i49sCkBnAwvdfZyZDSKYMXkdwMw6A7OA54BKdx9uZocAl6b0MZJghmhyGJIGuHsN0K+u\ngZldkDLOKOAJYBjwBzMrAPoSLEudDtQm9Z8H4O43JA9qZoea2Z5AJcGs0mR3X29mLwMTgYfD4y5J\nOqYHUA6cAbxtZvOAvwMfuvvpYZuRwPvhEl0lwR6kaY15MffpcXRjmklEiouL691EXV5eTklJyU6u\nSDJF56/t0rmLl2zaBF0GjDOz+QTB5lxgrZktJJitmQjMB04zsxcJ9geVhUtdEMzGzAWuMLPngTOB\nqjDQ1DfOZcBd7v4ssNzM3iCYwfmTu7+b1G+dj8zsoTS1ryHYV/Q64O7+Ynj/VIJg92g9zznh7huA\ny4GHCALQfDNbZGZlQE9gdVJfx4evhYiIiLRA1swAufsygj/wyd5N0/TINPcNS7q9w7ej1zMO7v6/\n0tw3LOXnH9bTZ6909wP5BGHq6zTH/B+CmSbc/XXgqPChO8KvdH1Nd/ed8W43ERGRnJY1ASjXmNkI\ngs3P52egr9uAEwmWyyQCy+aPz1hfidrNlJbOqvdK0hs3bqRDh9TtbTumq06LiDSNAlArCS+YuN1F\nE5vZ13WZ6EdEREQCCkAijdDz5Gsz1lf12lVMGd1fm6BFRCKUTZugRURERHYKzQDlqOq1q6IuQeqx\nfl1F1CWIiMSeAlCOmjK6f9QlyA4UFRVFXYKISKwpAOUofVK5iIhI/bQHSERERGJHAUhERERiRwFI\nREREYkcBSERERGJHAUhERERiRwFIREREYkcBSERERGJHAUhERERiRwFIREREYkcBSERERGJHAUhE\nRERiRwFIREREYkcBSERERGJHAUhERERiRwFIREREYkcBSERERGKnfdQFSOtw9xYdX1RURH5+foaq\nERERyS4KQDnqqmfHNvvYmjVVTBs6ATPLYEUiIiLZQwEoRxV26xJ1CSIiIllLe4BEREQkdjQDJFt9\nNOENABKbaymdU0pBQQELFiyIuCoREZHM0wyQiIiIxE5OzACZ2Uxglrs/H2ENewKnufusFvTRE3gW\neAvoDgwHKlrab2P1GtkXgKrVlUwccLM2QYuISM7KlRmgRPgVpd7AWS3s4wfAs+4+NPw5kaF+RURE\nJEnWzABZMN3wALCJIJgNBkYDfYBdgLHAM8D9wAHA/sDT7j4mqY9CYDrQGegGTHL3ySnj7BqOc1DY\n7wigPLzvYCAfmODufzSzV4BL3d3NbDiwLzATmA38EygC3nb3XwLXA0eY2SXuPi1pvDnAre5ebmZ/\nB6519yfN7AXgInf/PGx3IHAdsKuZfRoenpfU7y+A7wNdgb2AAe5e2ZzXWkREJO6yaQaoP7A4/D4W\nGAp0dffjgJOAY4ADgTfd/TTgOIIlojp5wCEES2GnAqcCI9OMMxxY5u59gQvDfi4FvnD374fj32Jm\ne7HtrFLy7UOBi4FjgR+b2b7ALcCC5PATehI43cx6ABuA/ma2B9ChLvwAuPtnwHjg0ZTQVtfv1LCG\n+e7+/YbCT9XqymZ/1ayp2lHXIiIibV7WzAARzNyMAuYB64C3CQIR4R/7G8Pg0MfMTgK+Bjqk9FEB\nXG1mA8PH25tZJ4J9NQngRYIQ9VzY76fARDO7B3gpvK/azD4imN1JlhwWP3X3GgAz+zysI6/uQTN7\nBugEvE8Q5v4MfAn8jiCUnQ7MNbPvEwQcgNvD71v7qefnf9AIXy1q3v6df1V9xaVnfZ/KykrKy8ub\n1Ye0nF77tk3nr+3SuYuPbApAZwML3X2cmQ0CbgVeBzCzzsAsguBS6e7DzewQgpmbZCMJZogmhyFp\nQBhU+tU1MLNfESyrPR1uOr6ZIGgdDzwVLqMdDiwnmLHpBjhwNLAy7CZ1v1EesIUwJLn7GckPmtl6\n4KfAQOA84CpgiLsvJ5jdqms3NM3rsrXfesZOa58eRzem2Xaq167ilFP6awN0hMrLyykpKYm6DGkm\nnb+2S+cuXrJpCawMGGdm8wmCzbnAWjNbSDArNBGYD5xmZi8S7A8qM7Nu4fEJYC5whZk9D5wJVJlZ\nQco4U4Ce4f6eB4H/JthXtFc41svATe6+BrgLuNfM5hG8VnXhIzWEJIClwOFhwEr1Z2BXd18LPB/e\nXp6mXepm7uR+r6pnbBEREWmi1OUVyQFlZWWJmx5b2XDDNKrXrmLKaM0ARUn/Cm3bdP7aLp27ti0v\nL69JmSablsAkYsvmjydRu5nS0lm6CrSIiOS0bFoCExEREdkpNAMkW/U8+VotgYmISCxoBkhERERi\nRzNAOap67apmHbd+XUWGKxEREck+CkA5asro/s0+tqgo9RqQIiIiuUUBKEdpD4+IiEj9tAdIRERE\nYkcBSERERGJHAUhERERiRwFIREREYkcBSERERGJHAUhERERiRwFIREREYkcBSERERGJHAUhERERi\nRwFIREREYkcBSERERGJHAUhERERiRwFIREREYkcBSERERGJHAUhERERiRwFIREREYqd91AVI63D3\nJrUvKioiPz+/laoRERHJLgpAOeqqZ8c2um3NmiqmDZ2AmbViRSIiItlDAShHFXbrEnUJIiIiWUt7\ngERERCR2FIBEREQkdrQEFmMfTXgDgMTmWkrnlFJQUADAggULoixLRESk1WkGqJHMbKaZnRpxDXua\n2aAoaxAREckFmgFqvET4FaXewFnArEx01mtkXwCqVlcyccDNeheYiIjERqwCkAV/4R8ANhHMfg0G\nRgN9gF2AscAzwP3AAcD+wNPuPiapj0JgOtAZ6AZMcvfJKePsGo5zUNjvCKA8vO9gIB+Y4O5/NLNX\ngEvd3c1sOLAvMBOYDfwTKALedvdfAtcDR5jZJe4+LZOvjYiISJzEKgAB/YHFwCjgeGAo0NXdjzOz\nLsBI4D3gTXefbmYdgc+AugCUBxwCzHL3J82sG/AKsE0AAoYDy9z9QjM7BBgAlABfuPt/mNnuwF/N\nbD7bziol3z40rPdfwDIz2xe4BRjemPBTtbqyUS8IBNcB2rJlS6Pbi4iItHVxC0DTCcLPPGAd8DZB\nIMLdK4EbzWwPoI+ZnQR8DXRI6aMCuNrMBoaPtzezTsCzBAHmReBA4Lmw30+BiWZ2D/BSeF+1mX1E\nMLuTLHlP1qfuXgNgZp+HdeQ19ol+tajxy1n/qvqKJUctYf369Y0+RlpXeXl51CVIC+j8tV06d/ER\ntwB0NrDQ3ceFm4lvBV4HMLPOBHtrngMq3X14OHtzaUofIwlmiCaHIWlAGFT61TUws18RLKs9bWY9\ngZsJgtbxwFPhMtrhwHJgA8FSmgNHAyvDblL3G+UBW2jkxvV9ehzdmGYAVK9dRe/evbUHKEuUl5dT\nUlISdRnSTDp/bZfOXbzE7V1gZcC4cOnpUuBcYK2ZLSSYFZoIzAdOM7MXCfYHlYVLXRCEkrnAFWb2\nPHAmUGVmBSnjTAF6hvt7HgT+m2Bf0V7hWC8DN7n7GuAu4F4zm0dwPuqCT2oASgBLgcPDgCUiIiLN\nFKsZIHdfRjALk+zdNE2PTHPfsKTbhzcwzkZgSJqHhqZp+xzhclmKvkltvpd0f68djS0iIiINi1UA\nkm0tmz8egETtZkpLZ1FQUKCLIIqISCzEbQlMRERERDNAcdbz5GuBYBP0lNH9tQlaRERiQzNAIiIi\nEjsKQCIhS0OcAAAVYUlEQVQiIhI7WgLLUdVrVzW67fp1Fa1YiYiISPZRAMpRU0b3b1L7oqLUi1KL\niIjkLgWgHKUNzSIiIvXTHiARERGJHQUgERERiR0FIBEREYkdBSARERGJHQUgERERiR0FIBEREYkd\nBSARERGJHQUgERERiR0FIBEREYkdBSARERGJHQUgERERiR0FIBEREYkdBSARERGJHQUgERERiR0F\nIBEREYmd9lEXIK3D3aMuQZppxYoVFBYWRl2GNJPOX9ulc9d0RUVF5OfnR11Gs+RFXYBkXllZWeKG\n1+6IugwREclhNWuqmDZ0AmYWdSkA5OXlNSnTaAYoRxV26xJ1CSIiIllLe4BEREQkdhSAREREJHa0\nBCYiIiIAfDThjUa3TWyupXROKQUFBY1qv2DBguaW1SpabQbIzGaa2amt1X8DY/cws7nNOK6fmc1q\njZoaGHdfM5u0s8cVERGJq9acAUqEX21JJPW6+xfAFVGMLSIiUqfXyL6Nblu1upKJA27OmneBNdUO\nA5AFz+oBYBPBbNFgYDTQB9gFGAs8A9wPHADsDzzt7mOS+igEpgOdgW7AJHefnDLOHsBUoEtyGzM7\nFrgHqAIqgA3uPszMxgDnAGuA3YAx7v5qSvkHmdlfgK7A08BvgX8Afdy90swuB3Z399uTjskL69kN\neAJ4yN1nmdl44AdAPjABeBEoBw5194SZ/Q4oc/c/JT2nocCPgV2BIuB37v6gmZ0I3Bi+nruHr+km\nYBZwKTDR3X8Y9vEMcEP42t0CbAGWApe5++Y0p0xEREQaoaElsP7A4vD7WGAo0NXdjwNOAo4BDgTe\ndPfTgOOA4UnH5wGHALPc/VTgVGBkmnGKgNlp2kwGLnL3kwn+8GNmRwCnhWOfQxC60s3cdAIGAn2B\nAUAx8CgwKHx8CDAz5ZgEUEgQmCaF4ed0oIe7Hw/8ELg+bLsIOM3M8sN6nkxTwx7ufiZwFkFwBOgF\n/Ie7nwTMAc6vq9/dPwA6mtlBZrY/sJe7/40gHP7E3fsBqwjOg4iISKSWL1+Ou+PubNmyJepymqSh\nJbDpwChgHrAOeJsgEOHulcCN4exNHzM7Cfga6JDSRwVwtZkNDB9vb2adgGcJ/vC/ADyY2iY8dn93\n/zi8vRC4EPgO8La7J4ANZlYG5JnZVIKwVQFcE7bZAGBm7wAGzABmm9lrwBfuviacZekEfEAw63MC\n8D7QMRz3cKDEzF5Oes16EISSXxGEyBeBDmb2UvicXiQIKn8Lj1mZ1N9q4C4zqwa+TRCkUl/znwMb\ngRlmtjewH/CncJpx1/A126Gq1ZUNNREREWm2mjVV3Hj/G+zWeSnr11UwasgRdO/ePeqyGq2hAHQ2\nsNDdx5nZIOBW4HUAM+tMsGzzHFDp7sPN7BCCZZxkIwlmiCaHIWmAu9cA/eoamNmdqW3Chz4zs++E\nIeh74X0fAleaWR7BMtxRQMLdf5HUXw+gt5l1IFg2+i5wn7v/08wqCWZxpgG4+xlJx/UjCGZXAQvN\n7HXgY+Bld7/MzNoD1wGfunuNmU0ESoHr0zyni0g/M3U/0DM8fibbz8LNBhaEdf8I+BdBgDrL3avM\n7BzgqzT9bmPDB8c31ERERKTZ8oG9D9qPvHbBR2EUFxe3qf1ADQWgMuBBM/uG4A/1ucAwM1sYHnsT\n8BnwmJmVACuAMjPrFh6fAOYCd5vZTwjCS5WZFbj7pqRx0rYBfkkwC1INfAOsdPcl4d6excCXBPtn\nkvuqG3cd8GdgT+Bhd/9H+NhU4C6CJbBUCYIwVWFmY4EH3P208N1hrxHs2ZkThh0IltTOS5qlStdf\n6u1HCMLVauDvBEt4Wx8Pg9HfgPy6cczsKuAvZtYufF4/r2e8rXbf89sNNREREYmtrP4sMDP7JfBH\nd//SzH5DsCw0hSB03BfO8CwBTnL3lY3s8zyg2N1vykB9vwa+dPeZLe0rk8rKyhI3Pdaol0NERKTF\nqteuYsro/pHOAOXaZ4F9AbwQzgBVAheF3/uY2TCCWZOpTQg/twEnAmc01LYRfc0k2JtzZkv7EhER\nyRbL5o9v8jGJ2s2Uls5q9EUR60R5ccSsDkDu/gTBxuRUFzezv+taVtE2fQ3NVF8iIiKyc2V1ABIR\nEZGdq+fJ1zb5mGxYAmsqfRiqiIiIxI4CkIiIiMSOlsByVPXaVVGXICIiMbF+XUXUJTRZVr8NXpqn\nrKwsUVhYGHUZ0kxLliyhuLg46jKkmXT+2i6du5YpKioiPz8/svFz7W3w0kxtaSOabKuqqkrnrw3T\n+Wu7dO7iRXuAREREJHYUgERERCR2FIBEREQkdhSAREREJHYUgERERCR2FIBEREQkdhSAREREJHYU\ngERERCR2FIBEREQkdhSAREREJHYUgERERCR2FIBEREQkdhSAREREJHYUgERERCR2FIBEREQkdtpH\nXYC0DnePugRpphUrVlBYWBh1GdJM2XT+ioqKyM/Pj7oMkaykAJSjrnp2bNQlSEus+FPUFUhLZMH5\nq1lTxbShEzCzqEsRyUoKQDmqsFuXqEsQERHJWtoDJCIiIrGjACQiIiKxoyUwEYm1jya8EXUJrSKx\nuZbSOaUUFBREXUqrWLBgQdQlSBuXVTNAZjbTzE6NaOweZja3Gcf1M7NZGarhVDP7xQ4e39PMBmVi\nLBERkTjLthmgRPjVlmSsXnd/voEmvYGzgIwELhGBXiP7Rl1Cq6haXcnEATfrXWAi9ch4ALLgt+0B\nYBPBDNNgYDTQB9gFGAs8A9wPHADsDzzt7mOS+igEpgOdgW7AJHefnDLOHsBUoEtyGzM7FrgHqAIq\ngA3uPszMxgDnAGuA3YAx7v5qSvkHmdlfgK7A08BvgX8Afdy90swuB3Z399uTjskL69kNeAJ4yN1n\nmdl44AdAPjABeBEoBw5194SZ/Q4oc/et75c1s6HAYcBkYDbwT6AIeNvdfwlcDxxhZpe4+7QGToWI\niIjUozWWwPoDi8PvY4GhQFd3Pw44CTgGOBB4091PA44DhicdnwccAsxy91OBU4GRacYpAmanaTMZ\nuMjdTwaWApjZEcBp4djnEISudDM3nYCBQF9gAFAMPArULTsNAWamHJMACgkC06Qw/JwO9HD344Ef\nEgQXgEXAaWaWH9bzZJq+6hwKXAwcC/zYzPYFbgEWKPyIiIi0TGssgU0HRgHzgHXA2wSBCHevBG4M\nZ2/6mNlJwNdAh5Q+KoCrzWxg+Hh7M+sEPEsQEl4AHkxtEx67v7t/HN5eCFwIfIdgFiUBbDCzMiDP\nzKYShK0K4JqwzQYAM3sHMGAGMNvMXgO+cPc1ZvYMQVj6gGDW5wTgfaBjOO7hQImZvRz+3B7oQTBj\n9SuC4Pki0MHMXgqf04vAqqTX4FN3rwlr+Tx8jfIaeO23qlpd2dimIpKDatZURV2CSFZrjQB0NrDQ\n3ceFG3ZvBV4HMLPOBPtXngMq3X24mR0CXJrSx0iCGaLJYUgaEIaBfnUNzOzO1DbhQ5+Z2XfCEPS9\n8L4PgSvNLI9gGe4oIOHuv0jqrwfQ28w6AFuA7wL3ufs/zaySYBZnGoC7n5F0XD+CYHYVsNDMXgc+\nBl5298vMrD1wHWGgMbOJQClwfZrndFHSa5A6Q5UX1tWoWbuvFmndX6Q1/KvqKy496zC6desWdSk7\n1h0qKyspLy+PupI2Ra9XfLRGACoDHjSzbwj+WJ8LDDOzheF4NwGfAY+ZWQmwAigzs7r/mySAucDd\nZvYTgvBSZWYF7r4paZy0bYBfAjPMrBr4Bljp7kvCvT2LgS8J9icl91U37jrgz8CewMPu/o/wsanA\nXQRLYKkSBGGqwszGAg+4+2nhu8NeA3YH5tTN5hAsqZ2XNEuVrr/k78n3LwUON7Nfuftd9RwPwD49\njt7RwyLSTNVrV3HKKf3r3VxcXl5OSUnJTq5KMkHnLl4yHoDcfRlwfMrd76ZpemSa+4Yl3T68gXFe\nSdcm3AR9prt/aWa/ATaa2d7AWnc/LpzhWUIQwpL7W0Gw9yedfGB6uISWWserwKvh7dkEm5dx9//a\nQV9T63lODyb92Dfp/u8l3d+rnn5FRESkkbLtbfCZ8AXwQjgDVAlcFH7vY2bDCGZSprr7ysZ0Zma3\nAScCZzTUthF9zQT2A85saV8iuWLZ/PFRl9AkidrNlJbOqvcCgxs3bqRDh9RtjdHSRQNFtpdzAcjd\nnyDYmJzq4mb2d13LKtqmr6GZ6ktERESaL+cCkIi0LT1PvjbqEpqkeu0qpozWHiCRti6rPgpDRERE\nZGdQABIREZHY0RJYjqpeu6rhRiLSZOvXVURdgohkgAJQjpoyun/UJUgzLVmyhOLi4qjLkB0oKiqK\nugQRaSEFoBylT4Buu6qqqnT+RERamfYAiYiISOwoAImIiEjsKACJiIhI7CgAiYiISOwoAImIiEjs\nKACJiIhI7CgAiYiISOwoAImIiEjsKACJiIhI7CgAiYiISOwoAImIiEjsKACJiIhI7CgAiYiISOwo\nAImIiEjsKACJiIhI7CgAiYiISOy0j7oAaR3uHnUJOamoqIj8/PyoyxARkRZSAMpRVz07NuoSck7N\nmiqmDZ2AmUVdioiItJACUI4q7NYl6hJERESylvYAiYiISOxoBkgkjY8mvLHdfYnNtZTOKaWgoGCb\n+xcsWLCzyhIRkQzRDJCIiIjETk7MAJnZTGCWuz8fwdg9gLvd/cwmHtcPuMzdB7VGXdIyvUb23e6+\nqtWVTBxwszZBi4jkgFyZAUqEX21JW6tXREQkZ2TNDJAF/6x+ANhEEMwGA6OBPsAuwFjgGeB+4ABg\nf+Bpdx+T1EchMB3oDHQDJrn75JRx9gCmAl2S25jZscA9QBVQAWxw92FmNgY4B1gD7AaMcfdXU8o/\nyMz+AnQFngZ+C/wD6OPulWZ2ObC7u9+edExeWM9uwBPAQ+4+y8zGAz8A8oEJwItAOXCouyfM7HdA\nmbv/qWmvsIiIiNTJmgAE9AcWA6OA44GhQFd3P87MugAjgfeAN919upl1BD4D6gJQHnAIwVLYk2bW\nDXgF2CYAAUXA7DRtJgND3P1jM7sF+LaZHQGcBhwDdAA+IP3MTSdgIPANsJAgqD0KDALuA4YAP0k5\nJgEUEgSmP7j7M2Z2OtDD3Y8Pn9+bBAFoEXCamb0Q1nN9Qy9m1erKhppIE9WsqWL58uU7bKMLJYqI\ntA3ZFICmE4SfecA64G2CQIS7VwI3hrM3fczsJOBrglCSrAK42swGho+3N7NOwLMEgeMF4MHUNuGx\n+7v7x+HthcCFwHeAt909AWwwszIgz8ymEoStCuCasM0GADN7BzBgBjDbzF4DvnD3NWb2DEFY+oBg\n1ucE4H2gYzju4UCJmb0c/twe6EEwY/UrgpmxF919845eyGkzZvLV/2ifSmsYv/oT8totTfvY+nUV\njBpyBN27d2/xOOXl5S3uQ6Kj89d26dzFRzYFoLOBhe4+zswGAbcCrwOYWWdgFvAcUOnuw83sEODS\nlD5GEswQTQ5D0gB3rwH61TUwsztT24QPfWZm3wlD0PfC+z4ErjSzPIJluKOAhLv/Iqm/HkBvM+sA\nbAG+C9zn7v80s0qC2ZppAO5+RtJx/QiC2VXAQjN7HfgYeNndLzOz9sB1wKfuXmNmE4FSGjH788BD\nj/GjS2c01ExaQXFxcYs3SZeXl1NSUpKhimRn0/lru3Tu4iWbNkGXAePMbD5BsDkXWGtmCwlmhSYC\n8wmWgl4k2B9UFi5jQTDDMxe4wsyeB84EqsysIGWc+tr8EpgR9t0H+MbdlwB/IZiJmkOwP2lTSn8J\nghmrPxMEtkfc/R/hY1MJlvPmpXm+CYIwVUGwv+kBd58LVIezRm8DtWGAg2BJbd+kWSoRERFppqyZ\nAXL3ZQRhIdm7aZoemea+YUm3D29gnFfStQk3QZ/p7l+a2W+AjWa2N7A23IfUAVhCsO8oub8VwPbv\nmQ7kA9PDJbTUOl4FXg1vzwZmh7f/awd9Td3RcxMREZHGyZoAlAW+AF4ws2qgErgo/N7HzIYRzNhM\ndfeVjenMzG4DTgTOaKhtI/qaCexHMGPVoO+d/5uWDhlLy+aPb9HxidrNlJbO2u5K0U11++23N9xI\nRERaRAEo5O5PEGxMTnVxM/u7rmUVbdPX0Ez1JSIiIgpAIlv1PPnaFh1fvXYVU0b3z8gmaBERaV3Z\ntAlaREREZKfQDFAOWr+uIuoSYkmvu4hI26EAlINGDTmC4uLiqMuIpaKioqhLEBGRRlAAykEjRoxg\n9erVUZchIiKStbQHSERERGJHAUhERERiRwFIREREYkcBSERERGJHAUhERERiRwEoB82dOzfqEkRE\nRLKaApCIiIjEjgKQiIiIxI4CkIiIiMSOApCIiIjEjgKQiIiIxI4CUA4688wzoy5BREQkqykAiYiI\nSOwoAImIiEjsKACJiIhI7CgAiYiISOwoAImIiEjs5EVdgGReWVnZTcDYqOsQERHZiW4+5phjboq6\nCBERERERERERERERERERERERERERERERERERyQS9DT6HmFk74F7gCGAjcIm7L422KmkKM/srsC78\ncZm7l0ZZjzTMzI4DfuvuJ5nZIcBMoBZYAlzh7oko65P6pZy7o4C5wCfhw/e5+x+jq052xMwKgBlA\nd6ADcAvwMU34/Wvf+mXKTnQOsIu79w1/se8M75M2wMw6Arj7SVHXIo1jZtcA/wFUh3dNAK5z99fM\n7D7gbOCpqOqT+qU5dyXABHefEF1V0gRDgDXu/jMz2xN4D3iXJvz+6UrQueX7wDwAd38LOCbacqSJ\negO7mdnzZjY/DLGS3T4FBvL/Z9OPdvfXwtvPAf0jqUoaI/XclQADzOxVM5tmZrtHV5o0wp+AG8Pb\n7YBNNPH3TwEot+wBfJ3085ZwWUzahhrgdnc/FRgOPKrzl93cfQ6wOemu5G0F1UDnnVuRNFaac/cW\n8Gt3PxFYhq6mn9Xcvcbdq82skCAM3cC2mabB3z/9zzW3fA0UJv3czt1roypGmsyBRwHc/RPg/wL7\nR1qRNFXy71shUBlVIdJkT7r7u+Htp4CjoixGGmZmBwILgIfcfRZN/P1TAMotrwM/BjCz7wLvR1uO\nNNEwgn1bmFk3ghm9zyOtSJrqXTM7Mbx9OvDajhpLVplnZn3C2ycDZVEWIztmZvsCLwDXuPvM8O4m\n/f5pE3RueRL4kZm9Hv48LMpipMmmAw+YWd0v7TDN4LUZde80+S9gqpntAnwE/O/oSpJGqjt3w4FJ\nZraJ4B8el0ZXkjTCdQRLXDeaWd1eoKuAu/T7JyIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIi\nIiIiIiIiIiIiIlnn/wFHup3XhFoW2wAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10c3302d0>"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Testing sort-based shuffle with GroupBy"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `AggregateByKeyNaive` test in `spark-perf` triggers huge group-bys, making it a good candidate for testing Spark 1.1's new [sort-based shuffle](https://github.com/apache/spark/pull/1499).  The tests below were run on an 8-node `r3.8xlarge` cluster with the ext4 filesystem.\n",
      "\n",
      "The `spark-perf` settings were\n",
      "\n",
      "```\n",
      "aggregate-by-key-naive --num-trials=10 --inter-trial-wait=3 --num-partitions=3600 --reduce-tasks=3600 --random-seed=5 --persistent-type=memory  --num-records=1800000000 --unique-keys=180000 --key-length=10 --unique-values=9000000 --value-length=10  --storage-location=hdfs://:9000/spark-perf-kv-data\n",
      "```\n",
      "\n",
      "(Compared to the defaults, this is `SCALE_FACTOR = 9.0`)\n",
      "\n",
      "This test failed to complete on v1.0.2."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Both results here used 1.1.0-snapshot-1, since the same job failed to complete in a timely manner on 1.0.2. \n",
      "results = df({\n",
      "    \"1.1.0 default shuffle\": load_spark_results(\"results/2014-08-08/8_spot_nodes_rx.8xlarge/spark_perf_output_origin-tag-v1.1.0-snapshot1_2014-08-09_02-02-21_logs/\"),\n",
      "    \"1.1.0 sort shuffle\": load_spark_results(\"results/2014-08-08/8_spot_nodes_rx.8xlarge/spark_perf_output_origin-tag-v1.1.0-snapshot1_2014-08-09_07-45-59_logs/\")\n",
      "})\n",
      "\n",
      "\n",
      "plot_results(results, exclude_first_run=True);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAFRCAYAAACYDcKoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHMhJREFUeJzt3XucVXW9//HXcBGPCAgJHknTAL8na9ISrJN5QUNLFFMT\nRXQURQgt0wf0UEFIpdKTEaeENEKMNBO0gyeko0lRieaRoFS8fo0f4i0dPAgO99v8/lgbmoERNjCw\nZ6/v6/l48GBm7XX5fPZl9nu+a+35giRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJZa+i1AWo8T333HO1\nn/jEJ0pdxi57/vnnsY+mIQ89QD76yEMPYB9NSR56AKioqNipTNOssQtR6Z188smlLqFRrF69utQl\nNIo89JGHHiAffeShB7CPpiQPPewKg5AkSUqWQUiSJCXLICRJkpJlEJIkSckyCEmSpGQZhHLooYce\nKnUJkiSVhRalLkCSpMa0YcMGFixY0Kj77Nq1K82bN2/UfappMAhJknJlwYIFVA3/Jfu069Qo+1u5\nrJp7bulPCGG76z7zzDOMGTOGe+65p8HbV61axSWXXMLNN99Mly5d6t22ZMkSvvnNb7JmzRo6derE\nLbfcwt577/2Bx+rTp882zwB8//vfZ/bs2YwaNYqjjz56u7Vvcuyxx/L444/z8ssvU1NTQ48ePba7\nzec//3meeOKJovY/c+ZMxowZw4UXXsjbb7/N7NmzGTlyJFOmTGHs2LFF19lYDEKSpNzZp10n9m3/\n4T16zIkTJzJ9+nRat27d4O3z58/nhhtuoLq6mob+CPLtt9/OGWecwZlnnslPf/pTpkyZwoABA3a6\nnt/+9rdMnz6dffbZZ6e2f/TRR+nYsWNRQWhH/qjzrFmzGD58OD179qRXr15Mnz6d+fPn71SNjcFr\nhCRJagSHHHII48ePp7a2tsHb161bx+23385HP/rRBm//61//ynHHHQfA8ccfz5NPPlnv9o0bNzJq\n1CjOPfdcvvGNb7B8+XIA/vGPfzBo0CCqqqoYNGgQb7/9NuPHj6e6upqvfvWrrFq1iuuvv56BAwdy\nxhln8MMf/hCA6667jtmzZwPw2GOPMXz48M3Heuedd3jwwQeZPHlyvZCyZs0ahgwZQlVVFeecc87m\nUaC1a9cybNgw+vfvzxVXXMH69esZN24cU6ZMAQqjdFVVzJo1i9mzZzN27Nh6Na5Zs2bzMR5++GH6\n9etH//79+cEPflD8A7CTHBGSJKkRnHLKKbzxxhsfePtRRx21ze2XL19OmzZtAGjdujU1NTX1bv/d\n737HqlWruP/++1myZAmnnHIKAN/73veoqqraHJ7GjBnDmDFjmDZtGpMmTWLx4sV86lOfom/fvqxZ\ns4YTTjiBq6++moqKis0jOVuO6BxwwAGcffbZdOzYkU9+8pObl7/22mssXbqUO++8kyVLlrBw4UIA\nVq5cybBhw+jcuTNVVVW88MILDY4SnXTSScycOZPTTjuNY489dnONf/vb3wBYtmwZ48ePZ9q0abRq\n1YprrrmGP//5zxxzzDHbvO92hUEoh/r06cNbb71V6jIkSTtg3333Zfny5XTo0IEVK1bQtm3bercv\nXLhwcyjp0KEDXbt2BSDGyIQJE5g4cSIALVu2rLddu3btmD9/Pk899RT77rsva9eu3erYGzdubLCm\nLUe3DjvsMPr168ewYcNYv349VVVVm4/RuXNnADp27LjT85ctWrSIJUuWcNlllwGwYsUKXn/99Z3a\nV7EMQpIkNQFHHXUUf/rTnzjrrLN47LHHtro2p2vXrsyYMYOLL76YZcuW8eqrr25efumll/LpT3+a\nGCPPPvtsve2mTZtG27ZtGT16NIsWLeL+++8HYK+99qK6uppDDjmEF154Yat6KioqtgpIMUZWrFjB\nhAkTqK6u5vzzz6dnz54Njv7stddeLF68GMhmuC/GQQcdxIEHHsjkyZNp3rw5v/rVr+qNSO0OBiFJ\nUu6sXFZdsn3VDQUzZsxg5cqVnHvuuQ2uu3TpUkaNGsW4ceO4/PLLufbaa7n//vvp0KHDVtfH9OrV\ni6eeeoq+ffvSqVMn9t9/fwCuueYabrzxRtauXcvq1asZOXJkvTqOOeYYhg0bxvPPP0/nzp2prKzk\nnXfeoW/fvowYMYKWLVtyxBFHbFVbZWUlt956K926deMzn/kMAIceeijjx4/n4YcfZuPGjVx11VUf\neB/07t2bq6++mjlz5lBZWdlgWKp7aq6iooIOHTowYMAALrjgAjZu3MhBBx3E6aefvs37e1cVf5m3\nysaBBx5Ym4dTY/PmzaN79+6lLmOX5aGPPPQA+egjDz3A7u1jT/4doTw8HnnoAaBiRz66VocjQpKk\nXGnevHlRf/NHAj8+L0mSEmYQyiHnGpMkqTgGIUmSlCyDkCRJSpYXS0uScsXZ57UjDEKSpFxZsGAB\nl00eSuuObRplfysW13DngLFNbvb5HTF16lS+8pWv0KLFtt/2p02bxsKFCxk2bFhR+x06dCivv/46\n119/PTfddBPt27fngAMOoHfv3pvnTWvqPDUmScqd1h3b0Kbzfo3yr9hANXHiREaOHMm6desavH3+\n/PlccMEFvPHGG9ucff7ee+/l8MMP3zxhaWOYMGHCB06jUdeO/imeJ598kgceeIB169Zx8MEHc9dd\nd+3UfkrJIJRDffr0KXUJkpSc3T37/JIlS7jooouoqqrivPPO46WXXgLgrrvu4pxzzqFfv36MGTMG\ngHHjxnHppZfSr18/7r33Xt59912GDh1ab3/z5s3j3HPPZfTo0QwcOJAVK1ZQW1vL008/zcCBAznr\nrLM2T8dx0kknbZ6jbMyYMTz44IPcdNNN1NTUMGjQIL773e8yd+5cxo0bB2RzlK1fv54RI0Zw4YUX\n0r9/f+bMmbMzd+tu56kxSZIawe6efX7+/Pm0b9+eW2+9lb///e+sXLmSl19+mUceeYSpU6fSvHlz\nrrzySv74xz9SUVFBt27dGDFiBACTJk1i7Nix9fb3+9//nt69e1NZWcnSpUt5//33gWzS1kmTJvHm\nm28yePDgraYH2TQdxg033MDMmTOZOHEic+bMYerUqVx55ZUMHz4cYPNUITfffDPvvfceVVVVzJgx\no4h7cs8yCEmS1ARsb/b5448/nldffZUrrriCFi1acPnll7Nw4UKOPPLIzRdyd+/enVdeeQXI5gXb\nliFDhnDHHXfw61//mm7dunHkkUdSUVHBxz/+cQD2339/Vq1atdV2DY141dbWbrX8lVdeYe7cuTzz\nzDNAdhH70qVL2W+//Yq7Q/YQT41JktQEbJp9Hmhw9vmnnnqKjh07MmnSJIYMGcLYsWPp0qULzz77\nLBs2bKC2tpa5c+duPvXWrNk/3+KbNWu21TVC06dP5+yzz2bkyJEcdthhTJ06FWj4+p5WrVpRXV1N\nbW0tL774YlH9dOnShdNPP5177rmHO+64g1NPPZV27doVf4fsIY4ISZJyZ8Ximu2vtJv2tbtmn//Y\nxz7G0KFDue+++9iwYQNf//rXCSFw6qmncv7557Nx40Z69OhBr169eOmll+rV0aNHDwYPHszdd9+9\nedkRRxyx+eLu9u3bM3r0aObMmVNvu01fX3bZZQwePJgPf/jDDY7obDpdVvf78847j1GjRlFVVcXy\n5cvp379/k7yIuulVpF3m7PNNSx76yEMPkI8+8tADOPt8U5KHHsDZ51WHc41JSpmzz2tHeI2QJElK\nlkFIkiQlyyAkSZKSZRCSJEnJMghJkqRkGYRyyLnGJEkqjkFIkiQlyyAkSZKSZRCSJEnJMghJkqRk\nGYQkSVKyDEI55FxjkiQVxyAkSZKSZRCSJEnJMghJkqRkGYQkSVKyDEKSJClZBqEccq4xSZKKYxCS\nJEnJMghJkqRkGYQkSVKyDEKSJClZBiFJkpQsg1AOOdeYJEnFMQhJkqRkGYQkSVKyDEKSJClZBiFJ\nkpSsFqUuQI1v0aJFtGnTptRl7DL7aDry0APU76Nr1640b968xBVJKjWDUA71v/RCTrzhy6Uuo3Es\neqDUFTSOPPSRhx4AFj3AisU13DlgLCGEUlcjqcQMQjlU0ayCNp33K3UZkiQ1eV4jJEmSkmUQkiRJ\nyTIISZKkZBmEJElSsgxCOdRz1BmlLkGSpLJgEJIkSckyCEmSpGQZhCRJUrIMQpIkKVkGIUmSlCyD\nUA798dvTS12CJEllwSAkSZKSZRCSJEnJMghJkqRkGYQkSVKyDEKSJClZBqEccq4xSZKKYxCSJEnJ\nMghJkqRkGYQkSVKyDEKSJClZBiFJkpQsg1AOOdeYJEnFMQhJkqRkGYQkSVKyDEKSJClZBiFJkpQs\ng5AkSUqWQSiHnGtM2rbXfvEcAwcOLHUZkpoAg5AkSUqWQUhScj5yYSWTJk0qdRmSmgCDkCRJSpZB\nSJIkJcsgJEmSkmUQyiHnGpMkqTgGIUmSlCyDkCRJSpZBSJIkJcsgJEmSkmUQkiRJyTII5ZBzjUmS\nVByDkCRJSpZBSJIkJcsgJEmSkmUQkiRJyTIISZKkZBmEcsi5xiRJKo5BSJIkJcsgJEmSkmUQkiRJ\nyTIISZKkZBmEJElSsgxCOeRcY5IkFccgJEmSkmUQkiRJydptQSiEMDmE8MXdtf/tHPvQEMJDO7Fd\nzxDCfbujpu0c94AQwo/39HElSUpdi92479rCv3JSknpjjO8AXyvFsSVJStk2g1AIIQA/A9aRjR71\nB64Djgb2Am4AZgA/BQ4CDgSmxxhH1dlHG2AS0A7oDPw4xviTLY7TFpgI7Fd3nRDCZ4DxQA1QDayO\nMV4SQhgFnAksBvYBRsUY/7RF+R8JIfwP0AGYDvwH8DJwdIxxaQjhcmDfGOP362xTUahnH+C/gLtj\njPeFEG4BjgWaA2OBmcA84LAYY20I4XvA3BjjA3V6GgD0Bv4F6Ap8L8b48xDCCcC3CvfnvoX7dB1w\nHzAY+FGM8aTCPmYAIwv33XeADcAC4KsxxvUNPGSSJGkHbO/UWC/gfwv/3wAMADrEGD8LnAj0AA4G\nnowxfgn4LDCkzvYVQDfgvhjjF4EvAkMbOE5XYEoD6/wEuDjG+AWyAEAI4QjgS4Vjn0kWvhoayWkN\nnA0cA5wGVAL3AucXbr8AmLzFNrVAG7Lg9ONCCDoVODTGeBxwEnB9Yd3HgS+FEJoX6nmwgRraxhj7\nAGeQBUiAjwMXxhhPBKYBfTfVH2OcD+wdQvhICOFA4EMxxqfJQuJZMcaewJtkj8MHcq4xSZKKs71T\nY5OAa4FHgGXAHLJgRIxxKfCtwmjO0SGEE4H3gVZb7KMauDqEcHbh9hYhhNbAb8gCwKPAz7dcp7Dt\ngTHGFwtfzwb6AYcDc2KMtcDqEMJcoCKEMJEsdFUD1xTWWQ0QQvgLEIC7gCkhhMeAd2KMiwujLq2B\n+WSjQMcDzwJ7F477SaB7COEPde6zQ8nCyTfIwuRMoFUI4XeFnmaSBZanC9u8UWd/bwG3hRCWAx8m\nC1Rb3ucXAWuAu0IIHYF/BR7IBuj4l8J99oFqN9ZS89bSba0iJW3F4hoWLlxY6jJ2yqJFi2jTpk2p\ny9hl9rH7de3alebNm5e6jCZve0Hoy8DsGOPoEML5wHeBJwBCCO3ITuc8DCyNMQ4JIXQjO71T11Cy\nEaOfFMLSaTHGFUDPTSuEEH6w5TqFm14PIRxeCEOfKyx7HrgyhFBBdnru00BtjHFQnf0dChwZQmhF\ndjrp34E7YoyvhRCWko3q3AkQYzy9znY9yQLaVcDsEMITwIvAH2KMXw0htABGAH+PMa4IIfwIGAhc\n30BPF9PwSNVPgS6F7Sez9ajcFGBWoe6TgVVkQeqMGGNNCOFMYEkD+91s45oWLHk8bGsVKXm3vPUK\nFc0WlLqMnTPj7VJX0DjsY7dZuayaay84gkMOOaSo9efNm7ebK2q6theE5gI/DyGsJXvD/gpwSQhh\ndmHbG4HXgV+GELoDi4C5IYTOhe1rgYeAcSGEs8hCTE0IoWWMcV2d4zS4DnAF2ajIcmAt8EaM8bnC\ntT//C7xLdn1N3X1tOu4y4NdAe+CeGOPLhdsmAreRnRrbUi1ZqKoOIdwA/CzG+KXCp8keI7umZ1oh\n9EB2qu2cOqNWDe1vy69/QRay3gJeIju1t/n2QkB6Gmi+6TghhKuA/wkhNCv0ddEHHA+AZi1a0unQ\no7a1iiQp5yorKymcSdimefPm0b179z1QUdNUUeoCtiWEcAVwf4zx3RDCt8lOF00gCx93FEZ8ngNO\njDG+UeQ+zwEqY4w3NkJ93wTejTFO3tV9NaZW+3aoPXnwXaUuQ5JUIsvfe5MJ1/VKKghVVFTsVKbZ\nnR+fbwzvAI8WRoSWAhcX/j86hHAJ2SjKxB0IQTcDJwCnb2/dIvY1mezanT67ui9JklQaTToIxRj/\ni+wC5i1dupP7G7FrFdXb14DG2ldj+1zfb5e6BEmSyoJTbEiSpGQZhCRJUrIMQpIkKVkGIUmSlCyD\nkCRJSpZBKIeefGDU9leSJEkGIUmSlC6DkCRJSpZBSJIkJcsgJEmSkmUQkiRJyTII5ZBzjUmSVByD\nkCRJSpZBSJIkJcsgJEmSkmUQkiRJyTIISZKkZBmEcsi5xiRJKo5BSJIkJcsgJEmSkmUQkiRJyTII\nSZKkZBmEJElSsgxCOeRcY5IkFadFqQuQJKnc/b/f31LqEuqp3biegQPvo2XLlttdd82aNbRq1WoP\nVJWZNWvWHjtWMRwRkiRJyXJESJKkXdTlC8NLXUI9y997kwnX9SKEsN11582bR/fu3fdAVU2TI0KS\nJClZBiFJkpQsg1AOOdeYJEnFMQhJkqRkGYQkSVKyDEKSJClZBiFJkpQsg5AkSUqWQSiHnGtMkqTi\nGIQkSVKyDEKSJClZBiFJkpQsg5AkSUqWQUiSJCXLIJRDzjUmSVJxDEKSJClZBiFJkpQsg5AkSUqW\nQUiSJCXLICRJkpJlEMoh5xqTJKk4BiFJkpQsg5AkSUqWQUiSJCXLICRJkpJlEJIkSckyCOWQc41J\nklQcg5AkSUpWi1IXoMZXu3EDy997s9RlSJJKZOWy6lKXUDYMQjnUrvVeTLiuV6nL2GXPPfcclZWV\npS5jl+Whjzz0APnoIw89gH3sCV27di11CWXBIJRDLVq0IIRQ6jJ2WU1NjX00EXnoAfLRRx56APtQ\n0+E1QpIkKVkGoRx66KGHSl2CJEllwSAkSZKSZRCSJEnJMghJkqRkGYQkSVKyDEKSJClZBqEc6tOn\nT6lLkCSpLBiEJElSsgxCkiQpWQYhSZKULIOQJElKlkFIkiQlyyCUQ841JklScQxCkiQpWQYhSZKU\nLIOQJElKlkFIkiQlyyAkSZKSZRDKIecakySpOAYhSZKULIOQJElKlkFIkiQlyyAkSZKSZRCSJEnJ\nMgjlkHONSZJUHIOQJElKlkFIkiQlyyAkSZKSZRCSJEnJMghJkqRkGYRyyLnGJEkqjkFIkiQlyyAk\nSZKSZRCSJEnJMghJkqRkGYQkSVKyDEI55FxjkiQVxyAkSZKSZRCSJEnJMghJkqRkGYQkSVKyDEKS\nJClZBqEccq4xSZKKYxCSJEnJMghJkqRkGYQkSVKyDEKSJClZBiFJkpQsg1AOOdeYJEnFMQhJkqRk\nGYQkSVKyDEKSJClZBiFJkpQsg5AkSUqWQSiHnGtMkqTiGIQkSVKyDEKSJClZBiFJkpQsg5AkSUqW\nQUiSJCXLIJRDzjUmSVJxDEKSJClZBiFJkpQsg5AkSUqWQUiSJCXLICRJkpJlEMoh5xqTJKk4BiFJ\nkpQsg5AkSUqWQUiSJCXLICRJkpJlEJIkSckyCOWQc41JklQcg5AkSUqWQUiSJCXLICRJkpJlEJIk\nSckyCEmSpGQZhHLIucYkSSqOQUiSJCXLICRJkpJlEJIkSckyCEmSpGQZhCRJUrIMQjnkXGOSJBXH\nICRJkpJlEJIkSckyCEmSpGQZhCRJUrIMQpIkKVkGoRxyrjFJkopjEJIkSckyCEmSpGQZhCRJUrIM\nQpIkKVkGIUmSlCyDUA4515gkScUxCEmSpGQZhCRJUrIMQpIkKVkGIUmSlCyDkCRJSpZBKIeca0yS\npOIYhCRJUrIMQpIkKVkGIUmSlCyDkCRJSpZBSJIkJcsglEPONSZJUnEMQpIkKVkGIUmSlCyDkCRJ\nSpZBSJIkJcsgJEmSkmUQyiHnGpMkqTgGIUmSlCyDkCRJSpZBSJIkJcsgJEmSkmUQkiRJyaoodQFq\nfHPnzr0RuKHUdUiStAfd1KNHjxtLXYQkSZIkSZIkSZIkSZIkSZIkSZIkqZT8+HyOhBCaAbcDRwBr\ngMtijAtKW1VxQggtgbuAQ4BWwHeAF4HJwEbgOeBrMcbaUtVYrBBCJ2Ae8AWy2idTfj0MB/oALYHx\nwBOUWR+F18OdQCCrexCwgTLpI4TwWeA/YownhhC60UDdIYRBwGBgPfCdGONvSlZwA7bo4VPAbWSP\nwRrgohhjdVPvAer3UWdZf+DrMcZjCt+XVR+Fn1MTgf3IssBFMcZXm3ofW/TwMbLXeC0Qyd7zdvh1\n4R9UzJczgb0KL8zrgB+UuJ4dcQGwOMZ4PPAl4Mdk9Y8oLKsAvlzC+opSCHQTgBVkNY+l/HroCXyu\n8DzqCXShDB8L4BSgdYzxWGA0cDNl0kcI4RqyN6lWhUVbPY9CCP8KXAkcA3wRuCWEsFcp6m1IAz38\nkCw4nAhMA64NIRxAE+4BGuyDEMKngUvrfN+kHwtosI9bgXtijCcA3wIqm3ofDfRwI1nQOa6w7LSd\n6cEglC+fBx4BiDE+BfQobTk75AGyFyNkz8t1wFExxscKyx4GepWisB30feAO4B+F78uxh1OA+SGE\n/wYeAqYD3cuwj1VAuxBCBdAOWEv59PF34Gz+OWrf0PPoaOCJGOO6GOP7hW2O2OOVfrAte+gXY3y2\n8HVLssfnMzTtHmCLPkIIHwK+C1zNP3sruz7IgsLBIYSZZL+IzqLp97FlD6uADxVe423IXuM73INB\nKF/aAu/X+X5D4fRAkxdjXBFjXB5CaEMWikZS//m5nOzNrMkKIQwgG9V6tLCogvqnn5t8DwUdge7A\nOcAQ4JeUZx9PAHsDL5GN0t1GmfQRY5xGNqy/Sd26a8jqbgssa2B5k7BlDzHGtwFCCMcAXwP+kybe\nA9Tvo/DzdBIwlOz5s0lZ9VFwKLAkxngy8BpwLVmYaLJ9NNDDOOBHwAtAJ+BP7MRjURZvkira+2RP\n5E2axRg3lqqYHRVCOJjst5K7Y4z3kV0PsUkbYGlJCiveJcDJIYQ/AJ8Cfk4WKjYphx4A3gUejTGu\njzFGYDX1f5CUSx/XkP1m+G9kj8fdZCMRm5RLH1D/tdCWrO4tX+9tgPf2ZFE7KoRwHtmIae8Y4/9R\nfj10B7qR9XAf8PEQwliyN95y6gPg/8hGeyEb+e1B+T0evwCOizEeDtxDdup7hx8Lg1C+PAH0Bggh\n/Dvw7LZXbzoK1wo8ClwTY5xcWPy3EMIJha9PBR5raNumIsZ4QoyxZ+EaiKeBi4BHyqmHgsfJrtMi\nhNAZ2Af4fRn20Zp/jpC+B7SgzJ5TdTRU9xzguBBCqxBCO+Bwsgupm6QQwoVkI0E9Y4yvFhaXVQ8x\nxr/EGCsLr/F+wAsxxqHAXyijPgoeB04rfH0CWb1l9XiQ/WyqKXz9D7ILv3e4hxa7tUTtaQ+SjUg8\nUfj+klIWs4NGkI06fCuEsOlaoauA2woXur0A/KpUxe2kWmAYMLGceogx/iaEcHwIYQ7ZL0tXAK9S\nZn2QXa/1sxDCbLKRoOFkn+Yrpz42faJtq+dR4dMxtwGzyR6nETHGtSWqc1tqC6eUfgQsAqaFEAD+\nGGO8qUx6gH8+FptUbFoWY3y7DPsYBtwZQricbISxf4xxWZn0samHy4BfhRBWk30ScVCM8Z0y6UGS\nJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJGnn/X/+iA6rI5XxIgAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10c299d90>"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Testing impact of broadcasting task code on short-lived jobs"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "TODO: Add a suite that tries to launch _jobs_ as fast as possible, where each job has a small number of tasks.  This will help to measure the impact of Spark 1.1's [broadcast-based shipping of task code](https://github.com/apache/spark/pull/1498)."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}