Hyperband demo with sklearn

Hyperband-example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hyperband\n",
    "Hyperband is useful when limited by computational resources. Some examples cases are when\n",
    "\n",
    "* there are many parameters to search over\n",
    "* models take a long time to train\n",
    "\n",
    "Hyperband does only require *one* input, the computational budget. For more information on this, see the documentation: https://dask-ml.readthedocs.io/en/latest/hyper-parameter-search.html\n",
    "\n",
    "Hyperband is an *adaptive* algorithm: it spends as much time as possible on high-performing models by \"killing\" off the lower portion. More detail in mentioned in the `HyperbandCV` class description: https://dask-ml.readthedocs.io/en/latest/modules/generated/dask_ml.model_selection.GridSearchCV.html#dask_ml.model_selection.HyperbandCV\n",
    "\n",
    "Below, we'll simulate having many parameters to search over by having one parameters. We would have two, but we want to have a easy-to-interpret visualization at the end.\n",
    "\n",
    "Hyperband is very similar to `RandomizedSearchCV` and works best with continuous random variables. We simulate log-uniform random variable with lots of samples: `np.logspace(-4, 1, num=1000)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import dask.array as da\n",
    "\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "\n",
    "import dask_ml\n",
    "from dask_ml.datasets import make_classification\n",
    "from dask_ml.wrappers import Incremental\n",
    "from dask_ml.model_selection import HyperbandCV, GridSearchCV, train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from distributed import Client, LocalCluster\n",
    "client = Client()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "n, d = int(10e3), int(100)\n",
    "X, y = make_classification(n_features=d, n_samples=n,\n",
    "                           n_informative=d // 10,\n",
    "                           chunks=(n // 10, d))\n",
    "classes = da.unique(y)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "\n",
    "kwargs = dict(penalty='elasticnet', max_iter=1.0, warm_start=True, loss='log')\n",
    "model = Incremental(SGDClassifier(**kwargs))\n",
    "params = {'alpha': np.logspace(-4, 1, num=1000)}\n",
    "#           'loss': ['hinge', 'log', 'modified_huber', 'squared_hinge']}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "alg = HyperbandCV(model, params, max_iter=81)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.21 s, sys: 223 ms, total: 2.44 s\n",
      "Wall time: 2.68 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "HyperbandCV(asynchronous=True, eta=3, max_iter=81,\n",
       "      model=Incremental(SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,\n",
       "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
       "       learning_rate='optimal', loss='log', max_iter=1.0, n_iter=None,\n",
       "       n_jobs=1, penalty='elasticnet', power_t=0.5, random_state=None,\n",
       "       shuffle=True, tol=None, verbose=0, warm_start=True)),\n",
       "      params={'alpha': array([1.00000e-04, 1.01159e-04, ..., 9.88542e+00, 1.00000e+01])},\n",
       "      random_state=None, scoring=None, test_size=0.15)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "alg.fit(X_train, y_train, classes=da.unique(y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.774"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alg.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alpha': 0.11562801312073753}"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hyperband_alpha = alg.best_params_['alpha']\n",
    "# hyperband_loss = alg.best_params_['loss']\n",
    "# hyperban\n",
    "alg.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now will we compare with an exhaustive evaluation, which we can do because we're only simulating being computationally limited.\n",
    "\n",
    "We will use `GridSearchCV`, and set the loss of the model to be the loss Hyperband found. We'll do this because this is the really the only to show an fair visualization: otherwise we're comparing `alpha`s across loss functions, which doesn't make sense.\n",
    "\n",
    "Note that this visualization hides the fact that Hyperband was searching between 5 different loss functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4.42 s, sys: 2.45 s, total: 6.86 s\n",
      "Wall time: 1.16 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# model.set_params(loss=hyperband_loss)\n",
    "params = {'alpha': np.logspace(-4, 1, num=50)}\n",
    "grid = GridSearchCV(model.estimator, params, return_train_score=False)\n",
    "grid.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alpha': 0.03556480306223129}"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEPCAYAAABRHfM8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4VOXZ+PHvPZOELJAFEvYtKCCEHUQCylIVUVFRqUrRAkrBjdLV5a1b1be1rf3VF6UtuIFLAXdRcVdEBQoooOwiWwLIEgIJZJ+5f3/MJA4hyyRkMjPJ/bmuc83MOc85c58MzD3Pcp4jqooxxhhTHUewAzDGGBMeLGEYY4zxiyUMY4wxfrGEYYwxxi+WMIwxxvjFEoYxxhi/WMIwxhjjF0sYxhhj/GIJwxhjjF8sYRhjjPFLRLADqEvJycnauXPnYIdhjDFh5auvvjqsqinVlWtQCaNz586sWbMm2GEYY0xYEZHd/pQLaJOUiIwRka0isl1E7qpge0cR+VRE1orINyJyic+2u737bRWRiwIZpzHGmOoFrIYhIk5gNnAhkAmsFpHFqrrJp9g9wEuq+i8R6QksATp7n18HpAFtgY9EpJuqugIVrzHGmKoFsoYxGNiuqjtUtQhYCFxRrowC8d7nCcA+7/MrgIWqWqiqO4Ht3uMZY4wJkkAmjHZAhs/rTO86Xw8A14tIJp7axYwa7GuMMaYeBTJhSAXryt+taQIwT1XbA5cAz4uIw899PW8iMk1E1ojImkOHDp1WwMYYYyoXyISRCXTwed2eH5ucSt0EvASgqiuAaCDZz33x7jdXVQep6qCUlGpHhRljjKmlQCaM1UBXEUkVkSg8ndiLy5XZA5wPICI98CSMQ95y14lIExFJBboCqwIYqzHGmGoEbJSUqpaIyO3A+4ATeEZVN4rIg8AaVV0M/BZ4UkR+jafJabJ6bjK+UUReAjYBJcBtNkLKGGOCSzzfzw3DoEGD1C7cM6YKH33kebzgguDGYUKKiHylqoOqK9egrvQ2xlTj4Yc9j5YwTC3Y5IPGGGP8YgnDGGOMXyxhGGOM8YslDGOMMX6xTm9jGpM5c4IdgQljljCMaUy6dw92BCaMWZOUMY3JW295FmNqwWoYxpyGa+esAGDR9PTTLl/TY9XK3//uebzsssC9h2mwrIZhTJi6ds6KsiRjTH2whGFMiKtpYrBEYgLFEoYxPir7sm0IX9rXzlnBxn05lW4L9/MzgWd9GKbBCmSfgNutbNqfw76j+Sjw6ZaDnNWmGa3joxGRU8ruOHycdRnH2Hn4BAXFLn725EqcDkFEcAo4RNh2IBenQ3hg8UaaRUfQLDqC+OhIsk4U4QA+2XLAW15wiHAsv5gIh5BTUEx8dGSdnyPUU7+KCRuWMEzYq8uO56oUlrh4aXUGn28/zJfbD3PkRFHZtinzVgOQGBvJWa2bsTvrBCLCxKdW8k3GMXILSwBwCMREOSkqceNSxa2ehOJWpaDYs+61rzPJLSyh/ETSN86reCbmPg98QGJsJB2SYunYPJY9R/KIiXRyMLeAls2iTyo7e8p9/PP6gTU6b2NKWcIwpgJHThSxcd8xNu7LYeO+HNZnHKWgxM26jG9IadaEkd1SOK9bMs8t341D4M6Le7Dlhxw2789l8/4cDuYWogrJTZtwRf+29G2fSL8Oifzh9W8RkWpHSbndyomiEnILSpj+/BpU4X+v7I1LFVXF5Yb739xAiVsZP7A9Gdl57DmSz6b9OfxwrAAFBv/vx/Rul8Co7imMOqslqkpW81bQocMp722MPyxhmAYp63ghe47kUVjiZu6y7+nfMYne7RKIjnSeUvZoXhHrM4+xPuMoWw/kcqKwhAEPfVi2vV1iDDFRTlrGN2HuzwfRvVWzsmanhasyABic2pzBqc3L9rnm38tR4OWbh570XuWbqyrjcAjNoiNpFh1JbJTnv2nfDoknlYmP8TRDTR9xxknrr/n3cvKKXFzcuw2fbDnIE59uZ9Yn24lwCKn5R/jDowdo0r0bTSIdNIlw0CTCyf5jBbjdyv++s4ncghJyCorJyS9hw95jiAhT568mISaKpNhIEmMjOZBTQNMm9vXR2NgnbhqUIyeKmLPse55bvpv8YhdRTgd/WrIFgEin0LNNPD/kFNAkwsGvFq5lfaanX6FUdKSD+OhIpo/oQlrbBHq2iScpLqrs1/9ZreP9ikNE8C811D0RIa5JBLeNOpPbRp1J9okiln13iAff2sSxXHhvfxGFuRkUlrgodp3c7vXCyj3Ex0TQLDqS+OgIIpwOVJW9RwvYtC+Ho/nF5BV5bn4pwHMrdnHDkE7VJkLrC2kYLGGYBqHY5eYv721h/vJd5Be7uLxvW3YeOkFMlJMnfjaAtXuyWZtxlK93Z3MotxC3QkGxm34dEhk/sD39OiTSu30Cv5jv6SeYNvyMat6x/pzul2xSXBRX9GvHf/67h/uev5O0tvGwdCng6T8pcrmZ+ORKHA45pUZU0Rd9YYmLa+esYNfhPO57cyMb9h7joXG9aBLhtMTQwFnCMGFv/7F8MrPzWZtxlLF92jLz/DM5s2Wzsi+vlGZNGJ3WmtFprQFPk02JW3nt1mEBi6myL8yqvkhr+iVbF1/KDocQ7XAS4fR/hH2TCCdNIpx0a9WUc7q04PFPtrPtwHHm3FB5Z3pRiZu8ohJU1e9mORN6LGGYsLZ4/T72HMknMSaSl25Op1urZtXuIyJEOmv2pVWbBBAs9RWTiPDb0d3p2Sae3768nrGPf0FK0yiaeYf4ZhzJ470NP/Duhv2szTgKwCdbDnJ+j1b1Ep+pe5YwTNj6/tBx7n71G5o2iaBrq6anJItQ/DIPN/78DS/u3YYuKU35xXNr2Lw/l5RmTbh01udlFwmmtY2nfWIM+3MKWPLtD5YwwpglDBOW8otc3Pbi1zSJdNKxeSyOOmrmaMhJZtH0dLj6g4Acu3vrZiy+fRgj/raUg7mFtE+K4Q+X9OCitNZ0bBHLtXNWUFDi4qPNByh2uYmsQROYCR0BTRgiMgb4P8AJPKWqj5Tb/g9glPdlLNBSVRO921zAt95te1T18kDGakKDv52m9y/ewNYDucyfMpjZn26v0Xs05KRQreTkClfXRf9JYmwU3Vs1xa3wyi1DT9meFBvFdweP898dRzi3a8VxmNAWsDQvIk5gNnAx0BOYICI9fcuo6q9VtZ+q9gMeB17z2Zxfus2ShfH18poMXlqTyYxRZzK8W0qwwwkv8+Z5lgAREZyOU2t7i6ans/j2c4mJdPLexv0Be38TWIGsYQwGtqvqDgARWQhcAWyqpPwE4P4AxmPC3LVzVpBXVMJ3B4+T3qUFMy/oFuyQwk9pspg8ud7fOibKyaizUnh/4wEevLwXjgoSiwltgWxIbAdk+LzO9K47hYh0AlKBT3xWR4vIGhFZKSLjAhemCYbazHbqcivfHTxOs+hI/m9Cv7JfsoumpzfuZqYwclFaaw7lFvL1nuxgh2JqIZA1jIp+PmgF6wCuA15RVZfPuo6quk9EugCfiMi3qvr9KW8iMg2YBtCxY8fTjdnUA7db2Xs0n8O5hYz826c4HD/OwJqRnYcAE+au9MzWGhNZNmvrdwePU1Ds5tnJ/U+ZVM+EhuoS90/OakmU08F7G35gUOfmVZY1oSeQCSMT8J3lrD2wr5Ky1wG3+a5Q1X3exx0ishToD5ySMFR1LjAXYNCgQZUlJBMijuUX89uX1pGZnU98dAR92ifiVs9srS63cjC3AFUocbvZnZVHbkExuQUlZbO9dmweQ/oZLYJ8Fqa2mkVHcm7XZN7d8AN/uLSHXcQXZgKZMFYDXUUkFdiLJyn8rHwhEekOJAErfNYlAXmqWigiycAw4K8BjNXUg60/5DL9+TVkZufTqUUsrZo1YdaE/ieVqWyUlMutXDtnRYUdqia8jElrzSdbDrJxXw692iUEOxxTAwFLGKpaIiK3A+/jGVb7jKpuFJEHgTWquthbdAKwUPWk2f97AHNExI2nn+URVa2ss9yEsNIEMHFIJ+585RuaRUewcNqQGjdHOB0Vj74xNbRkSbAj4IKerXC+Lry34QdLGGEmoNdhqOoSYEm5dfeVe/1ABfstB3oHMjZTP9yqZBzJ45cL1nJ25yRm/2wALeOt/yFoYmODHQHN46I4J7U5727Yz+8u6h7scEwN2JXepk5U1JRU7HLz3YHjHM0vZsqwzvzPJT1O6wpfGwlVB/75T8/jrbcGNYwxvVpz35sb2X4wlz+8vgGwzzcc2PX5JiBUlT+8/i1H84vp3CKW+y9L8ytZ2BDZAHvpJc8SZKN7emYOfm/DD0GOxNSEJQwTELM+3s5LazJplxhNK2uCMuW0TohmQMdE3ttoCSOcWJOUqXMvr8ngHx9t4+oB7Xn0p31s6KSp0JherfnTki30bV/xrXNN6LEahqlTy7Yd4u7XvuXcM5P581W9LVmYSo1JawNAdl5RkCMx/rKEYerMicISbn3xa85s2ZR/XT+AqAj752Uq17FFLD3bxHPkRHGwQzF+sv/Rpk4UlrjYeiCXZtERzJsyuOyuaybELF1adj/vUDCmV2uOF5ZQVOIOdijGD5YwzGkrcbnZduA4bjc8O+VsWidYJ7fxz5hentFSR61ZKixYwjA1UtEss69+nUlekYvUlDjOah0fpMiMXx591LOEiK4tm+IQyCt2VV/YBJ0lDHNaCopdPPbRd8Q1cdI81pqhQt7bb3uWECEiREc6KSi2JqlwYAnDnJbnV+xm/7ECOibF2ogoUysxkU4KrIYRFixhmFrLKShm9tLtDO+WQnyM1S5M7URHOigscVvSCAOWMEytPblsB0fzirnDJpAzp6H0or3dWXlBjsRUxxKGqZVDuYU8/cVOxvZpY1NUh5OYGM8SQmK8CWPn4eNBjsRUx6YGMbUy+9PtFJa4+e1oq12ElXffDXYEpyitYXx/6ESQIzHVsYRhaqyg2MWL/93NtWd3IDU5DrCpqU3tOR1CpFPYedgSRqizJqlaqOhahMZk79F8HCLMPL9rsEMxNfXQQ54lhCyans7ATknsOGRNUqHOEgaWAGoir6iEw8eLmDIs1aYtD0cff+xZQkyXlKbssBpGyLOEYWokIzsfp0O4ZcQZwQ7FNCBdkuM4mldM9gmbIiSUWcIwfntj7V6O5hXTNiGaBLuq29ShLimevrAdNlIqpFnCaMRq0hS3ZtcR7njlG5pFR9jkgqbOdUluCthIqVBno6RMtTKO5DH9+a9omxhN87goHDYFSPhq0SLYEVSofVKMjZQKAwGtYYjIGBHZKiLbReSuCrb/Q0TWeZdtInLUZ9skEfnOu0wKZJy14VYNdgj1IregmJvmr6bY5ebpyWcT6bRKaVh79VXPEmIinA46No+1kVIhLmA1DBFxArOBC4FMYLWILFbVTaVlVPXXPuVnAP29z5sD9wODAAW+8u6bHah4a+JoXhHfHTzOexv2M6ZXm9M6VmmTUChex1DicjNjwVq+P3SC524czBkpTYMdkmnAuqQ0ZYc1SYW0QP5cHAxsV9UdqloELASuqKL8BGCB9/lFwIeqesSbJD4ExgQwVr8VlrjYlZWHW+H3L3/DngY8/83/LtnM0q2HePCKNIadmRzscExduPtuzxKCuiTHsTsrD5e7cdTew1Eg+zDaARk+rzOBcyoqKCKdgFTgkyr2bReAGGvsqc93UljiJrVFLFknirh9wde8fHM6TSKcwQ6txrLzijiUW8jDb2+iU3IcnVvE0rlFHL99aR0HcwvZlZXHjcNSmXhOp2CHaurKitC93qhLShxFLjd7s/Pp2CI22OGYCgQyYVTUM1rZT4frgFdUtXR+Y7/3FZFpwDSAjh071jRGz4H97I/YfyyfJz7ZTlJsJC3jo7nrkh5Mf/4rHnl3C/dfllblvqHW9JR1vJAdh06gwAv/3X3SDWwEzx97VPcU/nBpj2CFaBqZLt4mz+8PH6dji9iQ+z9j/GiSEpFWIvK0iLzrfd1TRG7y49iZQAef1+2BfZWUvY4fm6NqtK+qzlXVQao6KCUlxY+wKgg0O5/th05Q4qr6rl//+85m3Kp0au759XNRWmumDOvMs1/u4r0NP9TqvStTm6vPa7LPQ29vwuVWeraJZ9Mfx7Dy7vNZOG0If7m6N60TommTEM2sCf1xOmxElKkfpfOS7bR+jJDlTx/GPOB9oK339TbgV37stxroKiKpIhKFJyksLl9IRLoDSYDvN937wGgRSRKRJGC0d11ARDiFIyeKmLlwHcWVJI0V32fx9jf7uWXkGTSJ/LH56e6Le9CnfQJ3vLKejCN5dTbNiMutnCgsOe3jVOSzbYd4Y90+2iZGExvlxOEQWidEM6RLC649uyMdm8fSsXkszaLt4jxTf1rERREfHWEX74UwfxJGsqq+BLgBVLUEqPbWWN5yt+P5ot8MvKSqG0XkQRG53KfoBGCh+rQLqeoR4CE8SWc18KB3XUC0SYihY/MY3vl2PzMXri1LGqVf/iUuNw8s3kj7pBhuLjclRlSEgycmDECB2xesPe3hti638tKaDNZnHmXDvhye+WKnX/sdyytm+8HjHMotPGWbbxLLKyrhD69/y5ktm9I2seL7Iiyanm7NAA1V+/aeJQSJCF1Smtq1GCHMnz6MEyLSAm8fgogMAY75c3BVXQIsKbfuvnKvH6hk32eAZ/x5n7rQJiGGn6d35uF3NqO6llkT+pdte37lbrYeyGXODQPL5u731bFFLH8b34ebX/ia1vHRdKplh93n3x3if9/ZzJYfcolr4iQuysGDb28iwin8PL1zpfsdzC3g50+vIutEEVkninjq8x1MPa9LhWX/3wfbyMzO5+Wb03n0/a21itOEsRdeCHYEVeqSHMeKHVnBDsNUwp+E8Rs8TUlniMiXQAowPqBRBUnpl+zD72zml97agsut/L8Pt3Fe12RG92wFVNwJN6ZXGyald2L+it0UlLh45N0tdEmJ44yUuLJpDyqTV1TCpGdW8dm2Q3RoHsPjE/rz/IpdKJAYG8V9b27EIcL1QzyjlXw7AzOz87j+qf9yMLeQ7q2aekY9vbOZnPxifn1hN8TnquxvMo/yzJc7mXhOR87u3LxWfyOreZhA6pISx2tr95JXFJjmWHN6qkwYIuIAooERQHc8A2i2qmpxPcQWFL5JIyk2EqdDKCh28cDlaSd9+Vbkfy7twZJvfyC3sJinv9hBsevH5qkIhxDhFH7y6NKT9tl3NJ+CEjfx0RHcc2kPbkjvRJMIJ5f19XQZFZa4uOWFr7nnjQ1EOITrBv84Emz7wVyuf2oVeUUlvDD1HP7y7hYSYiJJTW7KrE+2k1NQwn1jewKeK9PvevVbUpo14c6Lz6qLP5UJR7/ydj8+9lhw46hEqvfHlTVLhaYqE4aqukXk76qaDmysp5iCbup5XRARHnrbc1H69OFd/LrKuUmEs2zWzRennkNmdj47Dh9nx6ETzF22A5dbSSt3/+tj+cU0j3OwZOZ5JMZGVXjMf04cwPTnv+Lu17/F4R21dKKwhGvmrMQhwqLp6fRoE1/2619VaRYdwVNf7CSnoBhV5YdjBWRk5/Pv6wcS7+3MttpCI7RuXbAjqFLZrLU2Uiok+dMk9YGIXA28pv5esNAA3HRuKs+v2MWRE0XMqMWd5SKcDjonx9E5OY6fnAUfbjoAwOM+fSPwY/NSRcmiVHSkkzk3DOQXz63hzle/oXV8NAdzCmmdEM2LU8+hs3c4YikR4Q+X9iAhJpK/f7iNhJhIcgqKuSitFWN6ta7xuRhTX0qH1lrCCE3+jJL6DfAyUCQiOSKSKyI5AY4rJLSKj6ZHm3iaNgn+pL7RkU7m3jCI9C4t2H+sgMgI4ZVb0k9JFqVEhBnnd+WBy3pyLL8YB8KDV/Sq56iNqZnoSCftEmPYaUNrQ1K134Sq2qw+AmkoAtnMExPl5KlJg7j4sc9p0TSKNgkVD4v1NXlYKgtW7cHpELulqgkLXVLi2HH4BDEVjEg0weXXT2fvdRPDvS+XqurbgQvJVCU2quY3MKqqucs0Mt26BTuCanVJjuPVr/fSs02zageamPpVbcIQkUeAs4EXvatmisi5qnrK/S2MMSFu7txgR1Ct1OQ4jheWUOxSoiIsYYQSf2oYlwD9VNUNICLzgbVAg0kYlTUj1WXzko1IMsY/pZMQFhS7iIqwG3aFEn97cxOB0qk5EqoqaALPko+ptWnTPI8hXNMoHVpbUOwiPsbmMwsl/iSMPwNrReRTPBfuDQdC8w4sxpiqbdsW7Aiq1TYhhiYRDvKLq5492tQ/f0ZJLRCRpXj6MQS4U1Xrdi5vE1BWIzHhxOEQUpPj+OFYQbBDMeX40+l9JfCJqi72vk4UkXGq+kbAo2sE7MvcmFOlJsexK8su3gs1/vQo3a+qZbPTqupR4P7AhWSMaey6pMRRWOw+7dsFmLrlTx9GRUkl+Jc+G2Nqrl+/YEfgly7JTVGg0PoxQoo/X/xrROT/AbPx3BNjBvBVQKMyxgRGiM5SW15q6Uipkmrv1WbqkT9NUjOAImARnjmlCoDbAhmUMaZxOyP5x2sxTOjwZ5TUCbwX6YmIE4jzrjPGhJvrr/c8hvid9xJiI4lwCPlF1iQVSqqtYYjIf0QkXkTi8NwTY6uI/D7woRlj6lxmpmcJA1ERDopdljBCiT9NUj1VNQcYh+f+3B2BGwIalTGm0YtwCCVuGyUVSvxJGJEiEoknYbzpvT2rfYrGmICKcDoocVsNI5T4kzDmALuAOGCZiHQCGsUNlIwxwRPhEEpc9ts0lFSbMFR1lqq2U9VLvLdo3QOMKt0uIpMq21dExojIVhHZLiIVzm4rIteIyCYR2Sgi//FZ7xKRdd5lcc1OyxhTofR0zxIGIpyeJim3NUuFjBpfgOdNGiU+q2YC88uX846omg1cCGQCq0Vksapu8inTFc9EhsNUNVtEWvocIl9Vw+MqI2PCxZ//HOwI/Bbh8PyezSkotpuAhYi6mGy+sjucDAa2q+oOVS0CFgJXlCvzC2C2qmYDqOrBOojHGNMARDg8Xy1H84qDHIkpVRcJo7L6Yjsgw+d1pnedr25ANxH5UkRWisgYn23RIrLGu35cHcRpjLn6as8SBiKcnoSRnVcU5EhMqbqYE6qyGkZF68snlwigKzASaA98LiK9vBMcdlTVfSLSBfhERL5V1e9PeRORacA0gI4dO9byFIxpJLKygh2B30qbpKyGETr8uXAvtZp1X1ayaybQwed1e2BfBWXeVNViVd0JbMWTQFDVfd7HHcBSoH9Fb6Kqc1V1kKoOSklJqe50jDFhItJqGCHHnyapVytY90rpE1W9vZL9VgNdRSRVRKKA64Dyo53ewDviSkSS8TRR7RCRJBFp4rN+GLAJY0yjUdqHkW01jJBRaZOUiJwFpAEJInKVz6Z4ILq6A6tqiYjcDrwPOIFnVHWjiDwIrPHekOl9YLSIbAJcwO9VNUtEhgJzRMSNJ6k94ju6yhjT8DnLOr2thhEqqurD6A6MBRKBy3zW5+IZ3VQtVV2CZzoR33X3+TxX4DfexbfMcqC3P+9hjKmB888PdgR+ExEiHGJNUiGk0oShqm8Cb4pIuqquqMeYjDGBcu+9wY6gRiKcYk1SIcSfPowrvbPVRorIxyJyWESuD3hkxphGL8LhsCapEOJPwhjtna12LJ5RTd0Am97cmHB08cWeJUxEOITsE1bDCBX+XIcR6X28BFigqkdEKrv0whgT0vLzgx1BjUQ4xWoYIcSfhPGWiGwB8oFbRSQFz21ajTEmoCIcDuvDCCH+zFZ7F5AODPLeCyOPU+eEMsaYOhfhFPKLXXZv7xDhz5XescBtwL+8q9oCgwIZlDHGgE1AGGr8aZJ6FvgKGOp9nQm8DLwdqKCMMQEydmywI6iRCKd3Pqn8IlonVHu9sAkwfxLGGap6rYhMAFDVfLFeb2PC0+9+F+wIaqRsehAbKRUS/BlWWyQiMXhnmhWRM4DCgEZljDH4NknZSKlQ4E/CeAB4D+ggIi8CHwN3BjIoY0yAjBzpWcLAounpPDnJ011qI6VCQ7VNUqr6gYh8BQzBc4+Lmap6OOCRGWMavSTvrVltPqnQ4M8oqY9VNUtV31HVt1X1sIh8XB/BGWMat+hIJ9GRNj1IqKhqevNoIBZIFpEkfryDXjyeobXGGBNwSbFR1iQVIqpqkpoO/ApPcviKHxNGDjA7wHEZYwwAibFRVsMIEVVNb/5/wP+JyAxVfbyyciJyoap+GJDojDF165prgh1BjSXFRloNI0T40+ldabLw+gtgCcOYcHDrrcGOoMaSYqPY/ENOsMMw+Destjp2EZ8x4SIvz7OEkcTYSJsaJET4c6V3dbQOjmGMqQ+XXOJ5XLo0qGHURJK3D8PtVhwO+30aTHVRwzDGmIBJjI3ErZBbUBLsUBq9ukgYu+rgGMYYU6FEu3gvZPjVJCUiQ4HOvuVV9Tnv41UBicwYY/CMkgI4mm/9GMFWbcIQkeeBM4B1QOldTBR4LoBxGWMMYDWMUOJPDWMQ0FNVa9y5LSJjgP8DnMBTqvpIBWWuwTPBoQLrVfVn3vWTgHu8xR5W1fk1fX9jTDmTJwc7ghorq2FYwgg6fxLGBqA1sL8mBxYRJ54rwi/Ec9Ol1SKyWFU3+ZTpCtwNDFPVbBFp6V3fHLgfT7JS4Cvvvtk1icEYU05YJgxvDcPuiRF0/iSMZGCTiKzC5z4Yqnp5NfsNBrar6g4AEVmI517gm3zK/AKYXZoIVPWgd/1FwIeqesS774fAGGCBH/EaYypz2DvRdHJycOOogfiYSESshhEK/EkYD9Ty2O2ADJ/XmcA55cp0AxCRL/E0Wz2gqu9Vsm+7WsZhjCk1frznMYyuw3A6hIQYmx4kFPgzNchntTx2RVfYlO8HiQC6AiOB9sDnItLLz309byIyDZgG0LFjx1qGaowJZZ4Za62GEWz+3A9jiIisFpHjIlIkIi4R8Wdil0ygg8/r9sC+Csq8qarFqroT2IongfizLwCqOldVB6nqoJSUFD/CMsaEG5seJDT4c+HeE8AE4DsgBpjqXVed1UBXEUkVkSjgOmBxuTJvAKMARCQZTxPVDuB9YLSIJHnvxTHau84Y0whZDSM0+HXhnqpuFxGooxp+AAAfUklEQVSnqrqAZ0VkuR/7lIjI7Xi+6J3AM6q6UUQeBNao6mJ+TAyb8Fzj8XtVzQIQkYfwJB2AB0s7wI0xjU9ibCRbf8gNdhiNnj8JI89bQ1gnIn/FM7w2zp+Dq+oSYEm5dff5PFfgN96l/L7PAM/48z7GGD/dckuwI6gVq2GEBn8Sxg14mq5uB36Np2/h6kAGZYwJkGuvDXYEtZIUG0lekYvCEhdNIpzBDqfR8meU1G4RiQHaqOof6yEmY0ygZHhHq3foUHW5EFM6PcjRvGJaxVvCCBZ/Rkldhmceqfe8r/uJSPnOa2NMOLjhBs8SZhLLpgexkVLB5M8oqQfwXLV9FEBV1+GZudYYY+pFkk1AGBL8SRglqnos4JEYY0wlEm0CwpDg1+SDIvIzwOmdLPCXQLXDao0xpq78WMOwJqlg8qeGMQNIwzPx4H+AY8DMQAZljDG+rEkqNPhTw+jpXSK8yxXA5UCfAMZljAmE3/422BHUSkyUkyYRDuv0DjJ/EsaLwO/w3BfDHdhwjDEBddllwY6g1pJio8g+YTWMYPInYRxS1bcCHokxJvC2bvU8du8e3DhqITHWpjgPNn8Sxv0i8hTwMSffQOm1gEVljAmM6dM9j2F0P4xSSbFRNkoqyPxJGFOAs4BIfmySUsAShjGm3iTF2QSEweZPwuirqr0DHokxxlQhMTbKOr2DzJ9htStFpGfAIzHGmCokxUZyNL8YzyTXJhj8qWGcC0wSkZ14+jAEz8zkNqzWGFNvkmKjcLmVnIISEmIigx1Oo+RPwhgT8CiMMfXjnnuCHUGtlSaJo3lFljCCxK/pzesjEGNMPbjggmBHUGtJPlOcd2oR5GAaKX/6MIwxDcW6dZ4lDCXFeWoVNj1I8Ph1T29jTAPxq195HsPwOgzfmyiZ4LAahjEmLNgEhMFnCcMYExYSYiIRsSnOg8kShjEmLDgdQnx0pE0PEkQBTRgiMkZEtorIdhG5q4Ltk0XkkIis8y5Tfba5fNbbPcSNMSTZBIRBFbBObxFxArOBC4FMYLWILFbVTeWKLlLV2ys4RL6q9gtUfMY0Sn/6U7AjOC2JsVF8tvUg185ZwaLp6cEOp9EJ5CipwcB2Vd0BICIL8dx8qXzCMMbUl6FDgx3BaUmKjaTEbVODBEsgm6TaARk+rzO968q7WkS+EZFXRKSDz/poEVkjIitFZFwA4zSm8Vi+3LOEqaTYKEsYQRTIGoZUsK78J/0WsEBVC0XkZmA+8BPvto6quk9EugCfiMi3qvr9KW8iMg2YBtCxY8e6i96Yhuh//sfzGIbXYYCnSarEZTf+DJZA1jAyAd8aQ3tgn28BVc1S1dKbMj0JDPTZts/7uANYCvSv6E1Uda6qDlLVQSkpKXUXvTEm5CTFRuJWcNuMtUERyISxGugqIqkiEgVcB5w02klE2vi8vBzY7F2fJCJNvM+TgWFY34cxjV5inOfivRKXJYxgCFiTlKqWiMjtwPuAE3hGVTeKyIPAGlVdDPxSRC4HSoAjwGTv7j2AOSLixpPUHqlgdJUxppFJ9M5SW+K2ZqlgCOhcUqq6BFhSbt19Ps/vBu6uYL/lgN3lzxhzktLpQayGERw2+aAxjcljjwU7gtOSGFtaw7CEEQyWMIxpTPqF97WwSaV9GNYkFRQ2l5QxjclHH3mWMJVUWsOwJqmgsBqGMY3Jww97HsP0znsxkU5ErEkqWKyGYYwJGyJChEPs4r0gsYRhjAkrEQ6H1TCCxBKGMSasRDjF+jCCxBKGMSasRDiEYhslFRTW6W1MYzJnTrAjOG1Nm0SQnVfMrsMn6JwcF+xwGhWrYRjTmHTv7lnCWIumTQB4fe3eIEfS+FjCMKYxeestzxLGmkQ4iI+O4PW1e1GbtbZeNfgmqeLiYjIzMykoKAh2KMacIjo6mvbt2xMZGVk/b/j3v3seL7usft4vQJKbNmHH4RN8tTubQZ2bBzucRqPBJ4zMzEyaNWtG586dEanonk7GBIeqkpWVRWZmJqmpqcEOJ6w0j4ti/7ECXv16ryWMetTgm6QKCgpo0aKFJQsTckSEFi1aWO23FpwOYUyv1rzzzT4Kil3BDqfRaPAJA7BkYUKW/dusvSv7tyOnoIRPthwMdiiNRqNIGMHWtGnTk17PmzeP22+/PaDvuXTpUsaOHRuQY3fu3JnDhw+XPT8d9fG3OB2+53o6ZUzdG3ZmMi2bNeG1r220VH1p8H0YjVFJSUmwQzCh6vnngx1BnXE6hHH92/HMFzvJOl5YNtzWBI7VMIIoNzeX1NRUiouLAcjJyaFz584UFxczcuRIfvWrXzF06FB69erFqlWrADhx4gQ33ngjZ599Nv379+fNN98EPL/Uf/rTn3LZZZcxevTosuNdeeWV9OzZk5tvvhm39+rYW265hUGDBpGWlsb9999fFk/nzp25//77GTBgAL1792bLli0AZGVlMXr0aPr378/06dNPGsqYkpICwP79+xk+fDj9+vWjV69efP7556ec7+rVqxk6dCh9+/Zl8ODB5ObmArBv3z7GjBlD165dueOOO8rKL1iwgN69e9OrVy/uvPNOAFwuF5MnT6ZXr1707t2bf/zjHwB8//33jBkzhoEDB3LeeeeVxT558mR++ctfMnToULp06cIrr7xS4Wcxbtw4Bg4cSFpaGnPnzj1l+65duzjrrLOYNGkSffr0Yfz48eTl5ZVtf/zxx0/5u61atYqhQ4fSv39/hg4dytatWyt873rVoYNnaSCuGtCOErfy9jf7gx1K46CqDWYZOHCglrdp06ZT1tU3h8Ohffv2LVs6dOigt912m6qqTp48WV9//XVVVZ0zZ47+5je/UVXVESNG6NSpU1VV9bPPPtO0tDRVVb377rv1+eefV1XV7Oxs7dq1qx4/flyfffZZbdeunWZlZamq6qeffqpNmjTR77//XktKSvSCCy7Ql19+WVW1rExJSYmOGDFC169fr6qqnTp10lmzZqmq6uzZs/Wmm25SVdUZM2boH//4R1VVffvttxXQQ4cOnXSOjz76qD788MNlx83JyTlpe2FhoaampuqqVatUVfXYsWNaXFyszz77rKampurRo0c1Pz9fO3bsqHv27NG9e/dqhw4d9ODBg1pcXKyjRo3S119/XdesWaMXXHBB2XGzs7NVVfUnP/mJbtu2TVVVV65cqaNGjVJV1UmTJun48ePV5XLpxo0b9YwzzqjwMyr9m+Tl5WlaWpoePny47G9y6NAh3blzpwL6xRdfqKrqlClT9G9/+1uVf7fSc1RV/fDDD/Wqq66q8L3r9d/owoWeJYxd8+/les2/l5e9HvPYMr388c+DGFH4A9aoH9+xja9JauTIU9ddcw3ceivk5cEll5y6ffJkz3L4MIwff/K2pUurfcuYmBjWrVtX9nrevHmsWbMGgKlTp/LXv/6VcePG8eyzz/Lkk0+WlZswYQIAw4cPJycnh6NHj/LBBx+wePFiHn30UcAzCmzPnj0AXHjhhTRv/uMQw8GDB9OlS5eyY33xxReMHz+el156iblz51JSUsL+/fvZtGkTffr0AeCqq64CYODAgbz22msALFu2rOz5pZdeSlJS0innePbZZ3PjjTdSXFzMuHHj6Ffuzm5bt26lTZs2nH322QDEx8eXbTv//PNJSEgAoGfPnuzevZusrCxGjhxZVoOZOHEiy5Yt495772XHjh3MmDGDSy+9lNGjR3P8+HGWL1/OT3/607JjFhYWlj0fN24cDoeDnj17cuDAgQo/o1mzZvH6668DkJGRwXfffUeLFi1OKtOhQweGDRsGwPXXX8+sWbP43e9+V+nf7dixY0yaNInvvvsOESmrSQbVv/7lebz22uDGUYeuHtCOh9/ZzPaDxzmzZdPqdzC1Zk1SQTZs2DB27drFZ599hsvlolevXmXbyo+gERFUlVdffZV169axbt069uzZQ48ePQCIi4s7pXz51zt37uTRRx/l448/5ptvvuHSSy89aVhnkyaedmCn03lSX0h1o3mGDx/OsmXLaNeuHTfccAPPPffcSdtVtdJjlL6n7/tqJVfwJiUlsX79ekaOHMns2bOZOnUqbrebxMTEsr/JunXr2Lx5c4XHr+i4S5cu5aOPPmLFihWsX7+e/v37VzjUtaK/Z/n38P273XvvvYwaNYoNGzbw1ltv2fDZALm8b1scAq+vzQx2KA1e40sYS5eeutx6q2dbbGzF2ydP9mxPTj51Wx34+c9/zoQJE5gyZcpJ6xctWgTAF198QUJCAgkJCVx00UU8/vjjZV98a9eurfS4q1atYufOnbjdbhYtWsS5555LTk4OcXFxJCQkcODAAd59991q4xs+fDgvvvgiAO+++y7Z2dmnlNm9ezctW7bkF7/4BTfddBNff/31SdvPOuss9u3bx+rVqwFP/01VnfPnnHMOn332GYcPH8blcrFgwQJGjBjB4cOHcbvdXH311Tz00EN8/fXXxMfHk5qayssvvwx4ksL69eurPa9Sx44dIykpidjYWLZs2cLKlSsrLLdnzx5WrFgBePpXzj333GqP265dO8BTqzSB0TI+mvO6pvDksp1c8+/lAX2va+es4No5K05Zv3HfMZ5fuZsfjp36o6CyfepqfX0KaMIQkTEislVEtovIXRVsnywih0RknXeZ6rNtkoh8510mBTLOYJs4cSLZ2dllTVClkpKSGDp0KDfffDNPP/004PnVWlxcTJ8+fejVqxf33ntvpcdNT0/nrrvuolevXqSmpnLllVfSt29f+vfvT1paGjfeeGNZE0tV7r//fpYtW8aAAQP44IMP6Nix4yllli5dSr9+/ejfvz+vvvoqM2fOPGl7VFQUixYtYsaMGfTt25cLL7ywyl/cbdq04c9//jOjRo2ib9++DBgwgCuuuIK9e/cycuRI+vXrx+TJk/nzn/8MwIsvvsjTTz9N3759SUtLKxsM4I8xY8ZQUlJCnz59uPfeexkyZEiF5Xr06MH8+fPp06cPR44c4ZZbbqnyuHfccQd33303w4YNw+Wyi8sC6aoB7ShyucktqL8RgsUuN2+t38f4fy3n0llfcO8bGzj3L59w23++ZtXOIw1ynisJ1EmJiBPYBlwIZAKrgQmqusmnzGRgkKreXm7f5sAaYBCgwFfAQFU99aetj0GDBmlp30CpzZs3lzXZhKpXXnmFN998k+d9hjyOHDmSRx99lEGDBgUxMlNq165djB07lg0bNtT5sev132hpH14d1Y6DofRX9qLp6WXr8otcpN3/Hi3iolh9z4UBfe+iEjcjuqfwn//u4WBuIR2bx/Lz9E6kn9GCN9buZdHqDHIKSujRJp5J6Z145atMnA45Kd7KzqM26+uCiHylqtV+2QSy03swsF1Vd3gDWghcAWyqci+Pi4APVfWId98PgTHAggDFGjQzZszg3XffZcmSJcEOxTQGlQwrDicVfWHGRDlpHhdF1okinvliJ+MHtSc++scJHeviS9jtVjKO5LH/WAFrM44ysnsKf0nvzIhuKTgcnv6stLYJ/ObC7ry5bi/zlu/irte+xSEQ6XQw/K+fnnS8AzmeGnZN1kc4hYWr9nBFv3bERDlrfA6nK5AJox2Q4fM6EzingnJXi8hwPLWRX6tqRiX7tgtUoMH0+OOPV7h+aRj/AmyIOnfuHJDaRb1LTg52BAHTLjGGgmI3D769iUc/2MpVA9rx8/TOdGvV7LSPnVdUwq8WrmPfsQKSm0bx8s1DSa3k5k0xUU6uG9yRa8/uwJrd2dz24te43MrATiePLly27RBAjdbnFbm467Vv+dOSzVx7dgduGNL5tM+tJgKZMCoaElO+/estYIGqForIzcB84Cd+7ut5E5FpwDSgwrZ1Y4yP0s730oEcDUh0pJO0tvHcc2lPnluxi5fWZPLCyj2kd2lB1vFCIiMcrNp55KR9cguKiY2q+mvwh2MF3DR/NZv359CpeSyt4ptUmix8iQhnd25eVvYf15481Ly0ZlCT9arK7y46i/krdvHMl7t46oudJERH0iq+CW63ltV0AiWQCSMT8L2ktD2wz7eAqmb5vHwS+IvPviPL7bu0ojdR1bnAXPD0YZxOwMY0eA04YZTq3T6Bv/20L3df0oNFqzN4YeVu9h7NB+CaCkYZOUX441sbuWFIJ7qknHwdx7eZx5j63GqOF5Tw1KRBzPlsR72cQ2VEhMGpzRmc2pwfjhXwn1V7+NfS7Ww9UMzqXUc4p0uL6g9yGgKZMFYDXUUkFdgLXAf8zLeAiLRR1dJr+i8HSgfPvw/8SURK62SjgbsDGKsxpoFpHhfFLSPPYNrwLlz2+Oe4Fe4d2/OkMn98ayNZx4t4YeVunv1yF8O7pTApvRMju7fkw00H+PWidTSPi+LVW4dyVuv4oCcMX60TovnNhd1Y8f1hjuYVMzg18PcFCVjCUNUSEbkdz5e/E3hGVTeKyIN4LkNfDPxSRC4HSoAjwGTvvkdE5CE8SQfgwdIOcGOMqUhlnb5Oh9DM2wE+7MyT+3CSYqNIio3i8Z/1Z+GqDF78725umr+GNgnR7D9WQL8Oicz9+UBaNouu8j1qE1ddrXeI0Dwuql6myg/o1CCqugRYUm7dfT7P76aSmoOqPgM8E8j4KlOfow7qy2OPPca0adOIjY2t8b5vvPEG3bp1o2fPntUXNiYMtWwWzS/P78otI8/g/Y0/8OLKPQw7M5mHx/UiOtIZ7PBCRuObS6qReuyxx7j++utrnTDGjh1brwnD5XLhdNp/VFM3/P3VHul0MLZPW8b2aVsfYdWJ+vxh2/imBgmC0qmxp06dSq9evZg4cSIfffQRw4YNo2vXrqxatarSact37drFeeedx4ABAxgwYADLl3umPli6dCkjR45k/PjxnHXWWUycOLHSK0tnzZrFvn37GDVqFKNGjQLggw8+ID09nQEDBvDTn/6U48ePA3DXXXfRs2dP+vTpw+9+9zuWL1/O4sWL+f3vf0+/fv34/vvvK32P0v2uu+46AI4fP86UKVPo3bs3ffr04dVXXwUqnrYcPDeauu+++zjnnHNYsWIFX331FSNGjGDgwIFcdNFF7N9vU1iftiVLPIsxteHPlLbhstTV9Oblp08+XTt37lSn06nffPONulwuHTBggE6ZMkXdbre+8cYbesUVV1Q6bfmJEyc0Pz9fVVW3bdumpef46aefanx8vGZkZKjL5dIhQ4bo559XPsVz6TTdqqqHDh3S8847T48fP66qqo888oj+8Y9/1KysLO3WrZu63e6yOFQ9U4SXTo1emTZt2mhBQcFJ+91xxx06c+bMsjJHjhypdNpyVVVAFy1apKqqRUVFmp6ergcPHlRV1YULF+qUKVP8+nuHm1CYgt80btj05qf641sb2bQvp9pym/Z7yvgz0VfPtvHcf1lateVSU1Pp3bs3AGlpaZx//vmICL1792bXrl1kZmZWOG1527Ztuf3221m3bh1Op5Nt27aVHXPw4MG0b98egH79+rFr165qJ8QDWLlyJZs2bSqbR6qoqIj09HTi4+OJjo5m6tSpXHrppTW6xWufPn2YOHEi48aNY9y4cQB89NFHLFy4sKxMUlISy5Ytq3Da8nHjxuF0Orn66qsBz3ToGzZs4MILPdM8uFwu2rRp43c8phL//KfnsXTCTWNqoFEljGDynWLb4XCUvXY4HJSUlOB0Onn11Vfp3r37Sfs98MADtGrVivXr1+N2u4mOjq7wmOWnI6+KqnLhhReyYMGpM62sWrWKjz/+mIULF/LEE0/wySef+HXMd955h2XLlrF48WIeeughNm7cWOGU5lrF3GXR0dFl/RaqSlpaWtnssKaOvPSS59EShqmFRpUw/KkJQHBGSZVOW/74448jIqxdu5b+/ftz7Ngx2rdvj8PhYP78+bWe9bRZs2bk5uaSnJzMkCFDuO2229i+fTtnnnkmeXl5ZGZm0rZtW/Ly8rjkkksYMmQIZ5555kn7VsbtdpORkcGoUaM499xz+c9//sPx48cZPXo0TzzxBI899hgA2dnZnHPOOcycOZPDhw+TlJTEggULmDFjxinH7N69O4cOHWLFihWkp6dTXFzMtm3bSEvz7zM0xtQ96/QOEZVNW37rrbcyf/58hgwZwrZt2065SZK/pk2bxsUXX8yoUaNISUlh3rx5TJgwgT59+jBkyBC2bNlCbm4uY8eOpU+fPowYMaLsftnXXXcdf/vb3+jfv3+Fnd4ul4vrr7+e3r17079/f37961+TmJjIPffcQ3Z2Nr169aJv3758+umnlU5bXl5UVBSvvPIKd955J3379qVfv35lHf7GmOAI2PTmwVBX05s3xOswTOiy6c1NsPk7vbnVMIwxxvilUfVh+CucaxZXXnklO3fuPGndX/7yFy666KI6Of5tt93Gl19+edK6mTNnnnJ7WROirGZhToMljAbm9ddfD+jxZ8+eHdDjG2NClzVJGWOM8UujSBgNqWPfNCz2b9OEkwafMKKjo8nKyrL/mCbkqCpZWVknXYxpTChr8H0Y7du3JzMzk0OHDgU7FGNOER0dXTa9izGhrsEnjMjISFJTU4MdhjHGhL0G3yRljDGmbljCMMYY4xdLGMYYY/zSoOaSEpFDwG4gATjms8n3dWXPk4HDdRBG+fc+nbKVba/q/CpaF+7n7M86fz/zUDtnfz/jitbZOds5lz4/3fPtpKop1Zby5y5L4bYAcyt7XcVzv+44VdP3Pp2ylW2v6vwa4jn7s64Gn3lInbO/n7Gds51zVedcV+db3dJQm6TequJ1Zc8D9d6nU7ay7VWdX0Xrwv2c/Vnn72deV+rqnP39jCtaZ+ds5xyIc65Ug2qSOh0iskb9mN63IbFzbhzsnBu++jrfhlrDqI25wQ4gCOycGwc754avXs7XahjGGGP8YjUMY4wxfrGEYYwxxi+WMIwxxvjFEoafRCRORL4SkbHBjqU+iEgPEfm3iLwiIrcEO576ICLjRORJEXlTREYHO55AE5EuIvK0iLwS7FgCyft/d773s50Y7HjqQ6A+2wafMETkGRE5KCIbyq0fIyJbRWS7iNzlx6HuBF4KTJR1qy7OWVU3q+rNwDVAyA9PrKNzfkNVfwFMBq4NYLinrY7Od4eq3hTYSAOjhud/FfCK97O9vN6DrSM1OedAfbYNPmEA84AxvitExAnMBi4GegITRKSniPQWkbfLLS1F5AJgE3CgvoOvpXmc5jl797kc+AL4uH7Dr5V51ME5e93j3S+UzaPuzjcczcPP8wfaAxneYq56jLGuzcP/cw6IBn8/DFVdJiKdy60eDGxX1R0AIrIQuEJV/wyc0uQkIqOAODwfSL6ILFFVd0ADPw11cc7e4ywGFovIO8B/Ahfx6aujz1mAR4B3VfXrwEZ8eurqMw5XNTl/IBNP0lhHGP9IruE5bwpEDGH7xztN7fjxFwd4/kG1q6ywqv5BVX+F50vzyVBOFlWo0TmLyEgRmSUic4AlgQ4uQGp0zsAM4AJgvIjcHMjAAqSmn3ELEfk30F9E7g50cPWgsvN/DbhaRP5FPU+lUQ8qPOdAfbYNvoZRCalgXbVXMKrqvLoPpd7U6JxVdSmwNFDB1JOanvMsYFbgwgm4mp5vFhCOibEyFZ6/qp4AptR3MPWksnMOyGfbWGsYmUAHn9ftgX1BiqW+2Dk3/HNubOdbXmM8/3o958aaMFYDXUUkVUSigOuAxUGOKdDsnBv+OTe28y2vMZ5/vZ5zg08YIrIAWAF0F5FMEblJVUuA24H3gc3AS6q6MZhx1iU754Z/zo3tfMtrjOcfCudskw8aY4zxS4OvYRhjjKkbljCMMcb4xRKGMcYYv1jCMMYY4xdLGMYYY/xiCcMYY4xfLGEYY4zxiyUMY0KQiMwTkfGnW8aYumQJwxg/iEhjnajTmDKWMEyjISKdRWSLeG7X+Y14bj8bKyL3ichqEdkgInO998VARJaKyJ9E5DNgpohcJiL/FZG1IvKRiLTylnvAe8wPRGSXiFwlIn8VkW9F5D0Riawipgrfu1yZXSLyFxFZ5V3O9Nk8XESWi8iO0tqGiDQVkY9F5GtvDFfU7V/SNFaWMExj0x2Yq6p9gBzgVuAJVT1bVXsBMZx8s6FEVR2hqn/Hc/fBIaraH1gI3OFT7gzgUjw3r3kB+FRVewP53vWVqeq9feWo6mDgCeAxn/VtgHO9+z3iXVcAXKmqA4BRwN8rSkTG1JQlDNPYZKjql97nL+D5sh3lrTl8C/wESPMpv8jneXvgfW+535cr966qFgPfAk7gPe/6b4HOVcRT1Xv7WuDzmO6z/g1VdavqJqCVd50AfxKRb4CP8NxkpxXGnCZLGKaxKT/bpgL/BMZ7awRPAtE+20/4PH8cT42gNzC9XLlCAO/dGIv1x1k93VRyozIRia7mvSuL2/d5oe8hvY8TgRRgoKr2w3Mv+sqOa4zfLGGYxqajiJT+Qp+Ap5kJ4LCINAWqGnWUAOz1Pp9UB7GUfon7897X+jyuqOa4CcBBVS0Wz/3oO51emMZ42MgP09hsBiZ571X+HfAvIAlP09EuPDekqcwDwMsishdYCaSeTiCqelREnvTzvZuIyH/x/MibUM2hXwTeEpE1wDpgy+nEaUwpux+GaTREpDPwtreDOWyIyC5gkKoeDnYspnGzJiljjDF+sRqGMfVARF7n1CasO1X1/WDEY0xtWMIwxhjjF2uSMsYY4xdLGMYYY/xiCcMYY4xfLGEYY4zxiyUMY4wxfvn/F5s5zns5OEgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "df = pd.DataFrame(grid.cv_results_)\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "df.plot(x='param_alpha', y='mean_test_score',\n",
    "        yerr='std_test_score',\n",
    "        logx=True, ax=ax)\n",
    "ax.plot(2 * [hyperband_alpha], plt.ylim(), 'r--',\n",
    "         label=\"Hyperband's chosen alpha\")\n",
    "plt.legend(loc='lower left')\n",
    "plt.ylabel('mean_test_score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

1. HyperbandSearchCV now

2. algo.fit gives "Too many values to unpack" error

I'm unsure what you're asking about. I think the HyperbandSearchCV docs, user guide and blog post are better resources than this gist.

Just commenting in case other people stumble upon it too. Wanted to use it as an example, but it ended up being deprecated.

On Thu, Oct 22, 2020 at 3:57 PM Scott Sievert ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ I'm unsure what you're asking about. I think the HyperbandSearchCV docs <https://ml.dask.org/modules/generated/dask_ml.model_selection.HyperbandSearchCV.html#dask_ml.model_selection.HyperbandSearchCV>, user guide <https://ml.dask.org/hyper-parameter-search.html#adaptive-hyperparameter-optimization> and blog post <https://blog.dask.org/2019/09/30/dask-hyperparam-opt> are better resources than this gist. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://gist.github.com/94cecc2679ddedc07a1887080d4afa19#gistcomment-3500268>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADZR6DSTUHIMTYWAC7NYW3SMCFDZANCNFSM4S3URZPA> .

-- -- Sincerely yours, Dave Liu