pac_bayes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data\n",
    "\n",
    "**Gintare Karolina Dziugaite, Daniel M. Roy (2017)**\n",
    "\n",
    "- [Paper](https://arxiv.org/pdf/1703.11008.pdf)\n",
    "- [PAC-Bayes Tutorial](https://arxiv.org/pdf/1901.05353.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Generalization Bounds\n",
    "\n",
    "PAC bounds:\n",
    "\n",
    "$$\\mathbb P(\\mathcal E (c) \\le \\delta) \\ge 1 - \\epsilon,$$\n",
    "- $\\mathcal E (c)$ — error rate of the classifier $\\phi$\n",
    "- $\\delta, \\epsilon \\in (0, 1)$\n",
    "\n",
    "Here $\\delta$ is a threshold that usually depends on $\\epsilon$ and data.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example: Simple Generalization Bound\n",
    "\n",
    "- Countable set of classifiers $c \\in \\mathcal C$, and a prior distribution $P(c)$ over them.\n",
    "\n",
    "Then with probability at least $1 - \\delta$ for _all_ classifiers $c$ correctly classifying (all) $m$ training examples\n",
    "\n",
    "$$\\mathcal E (c) \\le \\frac {\\log \\frac 1 {P(c)} + \\log \\frac 1 \\delta}{m}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**Proof:**\n",
    "Suppose we have a classifier $c$ such that the error rate is greater than the bound\n",
    "$$\n",
    "    \\mathcal E (c) \\ge \\frac {\\log \\frac 1 {P(c)} + \\log \\frac 1 \\delta}{m}.\n",
    "$$\n",
    "The probability of this event it agreeing with $m$ data points is\n",
    "$$\n",
    "    (1 - \\mathcal E (c))^m \\le e^{-\\mathcal E(C) m} \\le \n",
    "    e^{- \\log \\frac 1 {P(c)} + \\log \\frac 1 \\delta} = P(c) \\delta.\n",
    "$$\n",
    "By union bound we get the statement of the theorem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## PAC-Bayes Bound\n",
    "\n",
    "Idea: get generalization bounds for stochastic classifiers.\n",
    "\n",
    "Setting:\n",
    "- Binary classification with a neural network\n",
    "- We will get a generalization bound for a model average under an arbitrary distribution $Q$ over weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Notation\n",
    "\n",
    "- $S_m$ — dataset of $m$ datapoints\n",
    "- $\\hat e(Q, S_m) = \\mathbb E_{h \\sim Q} (\\hat e(h, S_m))$ — empirical error of the model average under distribution $Q$ on $S_m$\n",
    "- $e(Q)$ — true (expected) error of the model average under distribution $Q$\n",
    "- $KL(\\hat e(Q, S_m) \\vert \\vert e(Q))$ — KL-divergence between _Bernoulli_ random variables with success probabilities $\\hat e(Q, S_m)$ and $e(Q)$:\n",
    "$$\n",
    "  \\hat e(Q, S_m) \\cdot \\log \\frac{\\hat e(Q, S_m)}{e(Q)} + (1 - \\hat e(Q, S_m)) \\cdot \\log \\frac{1 - \\hat e(Q, S_m)}{1 -e(Q)}\n",
    "$$\n",
    "<span style=\"color:blue\">This will be our way of measuring difference between true and empirical error rate!</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### The Bound \n",
    "\n",
    "**PAC-Bayes Bound** ([McAllester 1999](https://link.springer.com/article/10.1023/A:1007618624809)):\n",
    "\n",
    "Suppose $P$ is a prior distribution over the parameters of the network (can not depend on the data). Then with probability at least $1 - \\delta$ for all distributions $Q$ over the parameters we have:\n",
    "$$\n",
    "    KL(\\hat e(Q, S_m) \\vert \\vert e(Q)) \\le\n",
    "    \\frac{KL(Q \\vert \\vert P) + \\log \\frac m \\delta}{m-1}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Idea\n",
    "\n",
    "$$\n",
    "    KL(\\hat e(Q, S_m) \\vert \\vert e(Q)) \\le\n",
    "    \\frac{KL(Q \\vert \\vert P) + \\log \\frac m \\delta}{m-1}.\n",
    "$$\n",
    "\n",
    "The bound implicitly implies a lower bound on $e(Q)$. Let's try to optimize the bound with respect to $Q$!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Optimizing the bound\n",
    "\n",
    "$$\n",
    "    KL(\\hat e(Q, S_m) \\vert \\vert e(Q)) \\le\n",
    "    \\frac{KL(Q \\vert \\vert P) + \\log \\frac m \\delta}{m-1}.\n",
    "$$\n",
    "\n",
    "First, we only have an implicit dependence on $e(Q)$ in the bound. We need to _invert the KL_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import math\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "sns.set_style('whitegrid')\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def kl_bernoulli(q, p):\n",
    "    return q * math.log(q / p)+ (1 - q) * math.log((1 - q) / (1 - p))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcQAAAFCCAYAAABvtq3aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeVyU5f7/8dfMMOzILioKisulIii4UJlaZpZWuKRG2kmpTKtj306WZcd+p/P1m9U52mYnNStbMDtWaqVmpZaKmAvijncu4AIosu8wMPP7Y8QkZXWYewau5+PBQxnue+43I/KZ67qvRWMymZAkSZKk1k6rdgBJkiRJsgWyIEqSJEkSsiBKkiRJEiALoiRJkiQBsiBKkiRJEiALoiRJkiQB4KB2gOZy4MABk5OTk9ox7Ep5eTnyNWsa+do1nXztmk6+do1XUlKS1b9/f//rfa3FFkQnJyd69eqldgy7kpycLF+zJpKvXdPJ167p5GvXeImJiWdq+5rsMpUkSZIkZEGUJEmSJEAWREmSJEkCZEGUJEmSJEAWREmSJEkCWvAoU0myBKPRyPnz5ykuLq7zOIPBQHJyspVStSwNee3c3Nzo2LEjWq18Dy81H1kQJakOWVlZaDQahBB1/jIuLS3FxcXFislajvpeO6PRSFpaGllZWbRt29aKyaTWRr7dkqQ65OXlERAQIFsmKtJqtQQEBJCfn692FKmFk//LJakOVVVV6PV6tWO0enq9nsrKSrVjSC2cLIh1OLLtPL/vvaB2DEllGo1G7Qitnvw3aN2+PZDG57/VusCMxch7iLUwmUzEf3USk9FEcKgvTq6ylSBJkmRt+SUGZq8+iE6r4aGooGZ9cyRbiLXQaDQEdGmD0Wji7NEcteNIkiS1Sr/+nkml0UREkFez9xTIgliHLn39AEg5lKVyEkmSpNZpS3ImACN6BTT7tWRBrEPnMHNBPHs0m6oqo8ppJMk2rVy5kuHDhxMWFsb48ePZt29fncfv3buXmTNnMmTIEIQQfPvtt1ZKKtkbQ5WRXxVZEG2CV4ArXgGulJdUcuGkHPItSX+2ceNGFixYwMyZM1m3bh0RERFMnz6d9PT0Ws8pKSmhR48e/P3vf8fZ2dmKaSV7szc1h4KySrr6u9HZz63ZrycLYj26hF/uNj0su00l+5KSkkJsbCxhYWGMGDGCbdu2ERYWRkJCgsWusWLFCsaNG8ekSZPo2rUrL7/8Mv7+/qxatarWc4YNG8azzz7L3XffLed3SnW60l3au/lbhyBHmdarc7gfST+fJfVgFoPv7yaHf0t2ITU1lYkTJ/Lggw/yyiuvkJKSwrx586ioqKBnz541jl26dCnLli2r8/mWL1/OgAEDajxWUVHB0aNHeeSRR2o8PnjwYJKSkizzjUitlslkYnPyRcA63aUgC2K92oW0wdlNT/6lUvIuluDdrvmb7ZLtil2xh1+US1a95u3CnxWxgxp1zvz58xk+fDizZ88GIDg4mCFDhhAfH4+Pj0+NY2NiYhg1alSdzxcQcO0vpNzcXKqqqvDz86vxuK+vr0VboVLrdOpSEWeyS/B21RMZ5G2Va8qCWA+tTktwH1+U3RdIOZQlC6Jk8zIyMoiPj2fNmjU1Htfr9de0DgG8vLzw8vKyVjxJapDNl7tLb+/ZFp3WOj1zsiA2QOdwP5TdF0g9lEXkyGC140gqqq2lZkuLex87dgydTkf37t1rPK4oClFRUdcc39QuU29vb3Q6HVlZNe+vZ2dn4+/v38T0kmS2+Zh1u0tBFsQGCertg1an4cKpfMqKDDi7y1VrJNul0WgwGo1UVlbi6OgIQGJiIklJSUydOvWa45vaZero6EhoaCgJCQk1zk9ISGDkyJE3+F1IrVlOcQX7z+biqNMytIf13lzJgtgAji4OBPbw4lxyLmeOZCFuaq92JEmqVWhoKHq9nkWLFjFt2jQURWHhwoUAFu8yjY2NZc6cOYSHhxMZGcmqVavIzMwkJiYGgLi4OOLi4ti0adOVc4qLizl79ixg3trpwoULJCcn4+npSYcOHZqUQ2pZfjmeidEEUSE+uDtZr0zJMc8N1Dlcrloj2YeAgABeffVVtmzZQnR0NGvXrmXy5Mm4uroSHGzZLv/Ro0czd+5clixZwpgxY9i/fz8ffPABgYGBgHngTUpKSo1zjhw5wtixYxk7dixlZWUsWbKEsWPH8u6771o0m2S/rD26tJpsITZQ5zA/dvz3BGeP5VBlMKLTy/cSku2Kjo4mOjr6yueLFy+ud5PjppoyZQpTpky57tdmzZrFrFmzajwWFRWFoihXPrel+6+S+sorq9j+u3kk9x29rLshtPyt3kBt/FzwDXTDUFZF2olcteNIUqMoioIQQu0YklSv307nUFxRRc92HnT0drXqtWVBbITqbtPUQ9kqJ5Gkxjl+/Di9evVSO4Yk1WvL5e7SO620Os3VZEFshD8KYhYmk0nlNJLUcJs3b74y0EWSbJXJZLqyXNsdVr5/CLIgNkpAcBtc2jhSmFNGdlqx2nEkSZJalOSMQtLySvH3cCI80NPq15cFsRE0Wg2dw3wBcytRkiRJspzq7tI7erZFa6XVaa4mC2IjVe+RmCp3v5AkSbKozcfV6y4FWRAbrVMvH3QOWi6mFFCcX652HEmSpBYhs6CMg+fycHLQcms3v/pPaAayIDaS3klHx17mldfPHJGjTSVJkixh6+XW4a3d/HBx1KmSQRbEJqjuNk05KLtNJUmSLKF6dRq1uktBFsQmqS6I55NzqKyoUjmNJEmSfSszVBF/0tzAsPbqNFeTBbEJ3L2d8A/yoNJg5LwiV62RJEm6ETtPZlFmMBLe0ZOANs6q5ZAFsYnkYt+SJEmWcaW7tKd63aUgC2KTdblcEM/IVWskSZKazGj8Y3WaEb3V6y4FWRCbzK+TO+7eThTnV3DpbKHacSRJNStXrmT48OGEhYUxfvx49u3bV+/x9913H5GRkURGRvLwww/z66+/WiesZHOOpOeTWVhOB09nerdvo2oWWRCbSKPREBwmu02l1m3jxo0sWLCAmTNnsm7dOiIiIpg+fTrp6em1nhMQEMBzzz3H2rVr+eabbxg4cCBPPfUUx48ft2JyyVZsPmbuLh3eqy0ajfVXp7maLIg3oMtVi31Lkq1JSUkhNjaWsLAwRowYwbZt2wgLCyMhIcFi11ixYgXjxo1j0qRJdO3alZdffhl/f39WrVpV6zkjRoxg2LBhBAcH06VLF2bNmoWbmxsHDhywWC7Jfmyu7i5VcbpFNblB8A0IFF44OOnIOldEYU4ZHj7qjY6SpKulpqYyceJEHnzwQV555RVSUlKYN28eFRUV9OzZs8axS5cuZdmyZXU+3/LlyxkwYECNxyoqKjh69CiPPPJIjccHDx5MUlJSg3JWVVWxadMmSkpKiIiIaNA5UsuRllfKsYwCXB113BTiq3YcWRBvhINeR1AvH04fuMSZw1n0GdZR7UhSc1s5EU78dM3Dzbrfe/eRMOWrRp0yf/58hg8fzuzZswEIDg5myJAhxMfH4+PjU+PYmJgYRo0aVefzBQRc++49NzeXqqoq/PxqLrPl6+tbbytUURRiYmIoLy/HxcWF9957T25g3AptvTy6dGh3f5z16qxOczVZEG9Q53BfTh+4RMqhbFkQJZuQkZFBfHw8a9asqfG4Xq+/pnUI4OXlhZeXl7XiAdClSxfWrVtHYWEhGzZs4IUXXuDzzz+nR48eVs0hqWvzlb0P1R1dWk0WxBsU3McPNHBeyaGirBJHZ/mStmi1tNRKS0txcWnWdmKDHTt2DJ1OR/fu3Ws8rigKUVFR1xzf1C5Tb29vdDodWVk176FnZ2fj7+9f5/M5OjoSHBwMQNeuXUlOTuaTTz5hwYIFdZ4ntRxF5ZXsOpWNRgO395QFsUVwbeNIuy5tuHC6gPPJuYRE1P2LQJKam0ajwWg0UllZiaOjIwCJiYkkJSUxderUa45vapepo6MjoaGhJCQk1Dg/ISGBkSNHNiqz0WikoqKiUedI9i3+xCUqqoz0D/bGz91J7TiALIgW0TncjwunC0g5nCULoqS60NBQ9Ho9ixYtYtq0aSiKwsKFCwEs3mUaGxvLnDlzCA8PJzIyklWrVpGZmUlMTAwAcXFxxMXFsWnTpivnLFy4kNtuu4127dpRXFzMunXr2LNnT72tVKllsbXuUrCBgiiE6AR8BgQAJuADRVHe+dMxGuAdYDRQAkxTFGW/tbPWpnOYH7+tO82Zw1kYjSZVdnqWpGoBAQG8+uqrvPnmm6xZs4ZbbrmFyZMn89Zbb13pprSU0aNHk5uby5IlS8jMzKRHjx588MEHBAYGAuaBNykpKTXOycrK4vnnn+fSpUt4eHjQvXt3li9fzpAhQyyaTbJdVUbTle2ebGG6RTXVCyJQCcxWFGW/EMIDSBRC/KwoyrGrjhkFdL/8EQUsufynTfDp4EYbP2cKssrITC2gXYin2pGkVi46Opro6Ogrny9evBghBFqt5aceT5kyhSlTplz3a7NmzWLWrFk1Hnv99ddrfG5L918l6zhwLpec4gqCfFzp3tZd7ThXqD4xX1GUjOrWnqIohUAyEPinw8YAnymKYlIU5TfASwjR3spRa6XRaP7YI1FO0pdskKIoclqDZDOu7i5Ve3Waq9lCC/EKIURnIALY/acvBQLnrvr8/OXHMmp7rvLycpKTky0dsVZar3IAlL1peAn7HBxQVlZm1dfMHhgMBkpLS+s9zmQyNeg4tSQnJzNt2jSbzNjQ185gMMifzz+x1/+zG5LMv857uFn393R9bKYgCiHcgW+AZxRFKbjR53NycqJXr143HqyBqrobOfbDDoqzK+ng1xlPf/vrAkpOTrbqa2YPkpOTG9SdZ+vdflu2bFE7Qq0a+trp9Xr58/kn9vh/9kx2MWfzT+Ph7MCEYf3Q66zbUZmYmFjr11TvMgUQQugxF8OViqKsuc4haUCnqz7vePkxm6Fz0BIUal56SK5tKkmSdH3V3aXDevhbvRjWR/U0l0eQfgQkK4ryZi2HfQc8LITQCCFuAvIVRam1u1Qt1ZsGpx6WBVGSJOl6tlxeru3O3rYzurSaLXSZDgb+AhwWQlQvd/8SEASgKMpSYCPmKRcnMU+7iFUhZ72C+/ii0WpI/z2P8tJKnFxs4eWVJEmyDfmlBvak5KDTarith+3MP6ym+m9sRVHigTqHGSmKYgKesk6ipnN209O+qyfpJ/I4ezSb7gNs7x2QJEmSWrb9folKo4moLj54uurVjnMN1btMW5rOco9ESZKk67Ll7lKQBdHiqjcNPnMkG2OVUeU0kiRJtsFQZeSX49XzD2VBbBW8AlzxCnClvKSSjFP5aseRJEmyCftScykoq6Srvxtd/NzUjnNdsiA2A9ltKkmSVFN1d6ktrV36Z7IgNoMu4eb5iHIZN0mSJPNqRJsvF0Rb7S4FWRCbRbsQT5zcHMjPLCX3QrHacSRJklR16lIxqdkleLvqiQxq2lZj1iALYjPQ6rQE96letSZb5TSS1LxWrlzJ8OHDCQsLY/z48ezbt6/O46t33qj+6NevH4MHD7ZSWkkN1d2lt4u2ONjY6jRXs91kdu6P3S8uqZxEkprPxo0bWbBgATNnzmTdunVEREQwffp00tPT6zyvS5cuxMfHEx8fz+bNm/n++++tlFhSgz10l4IsiM0mKNQXrU7DhVP5lBUZ1I4jtUIpKSnExsYSFhbGiBEj2LZtG2FhYSQkJFjsGitWrGDcuHFMmjSJrl278vLLL+Pv78+qVavqPM/BwQF/f3/8/f3x8/PDx8fHYpkk25JbXEHimVz0Og1De/ipHadOqq9U01I5uTjQobsX54/ncuZoNiKqndqRpFYkNTWViRMn8uCDD/LKK6+QkpLCvHnzqKiooGfPnjWOXbp0KcuWLavz+ZYvX86AAQNqPFZRUcHRo0d55JFHajw+ePBgkpKS6ny+c+fOceutt+Lo6EifPn14/vnn6dSpU53nSPbpFyUTowkGh/ji4Wx7q9NcTRbEZtQ53I/zx3NJOZglC2IL8eTmJ9mRtsOq1xwSOIT3R7zfqHPmz5/P8OHDmT17NgDBwcEMGTKE+Pj4a1pjMTExjBo1qs7nCwi4tqsrNzeXqqoq/Pxqvuv39fWtsxUaHh7Oa6+9RkhICDk5Obz33nvExMSwfv16vL29G/otSnbiSndpT9tbu/TPZEFsRl3C/YhffYKzx7KpqjSic5A91FLzy8jIID4+njVrau6kptfrr2kdAnh5eeHlZb2Rf8OGDavxuRCCe++9l3Xr1hEba5Pr9ktNVF5ZxfbfzdPPbP3+IciC2Kza+Lng08GNnPRi0n/Po1NveZ/E3tXWUrOlDYKPHTuGTqeje/fuNR5XFIWoqKhrjm9ql6m3tzc6nY6srJrzbbOzs/H3929wXldXV7p160ZqamqDz5Hsw+7TORSVV9KznQedfFzVjlMvWRCbWZdwP3LSi0k5nCULomQVGo0Go9FIZWUljo6OgHmX8KSkJKZOnXrN8U3tMnV0dCQ0NJSEhIQa5yckJDBy5MgG5y0vLyclJeW6xVqyb/awOs3VZEFsZp3D/UjcdIbUQ1kMmdQdjabOna4k6YaFhoai1+tZtGgR06ZNQ1EUFi5cCGDxLtPY2FjmzJlDeHg4kZGRrFq1iszMTGJiYgCIi4sjLi6OTZs2XTnnjTfe4Pbbb6d9+/bk5OSwePFiSkpKGDduXJMySLbJvDpN9WLetn//EGRBbHYBndvg4qGnMLuMnPRifAPd1Y4ktXABAQG8+uqrvPnmm6xZs4ZbbrmFyZMn89ZbbxEcHGzRa40ePZrc3FyWLFlCZmYmPXr04IMPPiAwMBAwD7xJSUmpcc6FCxd49tlnycvLw9vbm7CwMFavXn3lHKllOH6hkLS8Uvzcnejb0XZXp7maLIjNTKPV0DnMj+SEDFIOZsmCKFlFdHQ00dHRVz6vXh1Gq7X8wK4pU6YwZcqU635t1qxZzJo1q8Zjb731Vo3Pben+q2Q5m4/9MbpUq7WPnjE57NEKQiLMAwyObE+j0lClchqpNVIUBSGE2jGkVqLMUEXc7jMA3N3HfqacyYJoBcGhvvh2dKc4r5yj2+te0kqSmsPx48fp1auX2jGkVmLl7rNcLCind/s2DOvR8BHHapMF0Qo0Wg1R93UBIPHHMxgqZCtRsq7NmzdfGegiSc2ppKKSJb+eBODZO3vYTXcpyIJoNZ3D/Wgb7EFpQQWHfz2vdhxJkqRm8dmuM2QVVdC3k5fdjC6tJguilWg0GgZFhwCQ9ONZKsoqVU4kSZJkWUXllSzbdgowtw7tbZqZLIhWFNTbh3YhnpQVGzi0VbYSJUlqWVbEp5BbYmBAsDdDu9v2zhbXIwuiFWk0GqKizfcSD2w+S3mJ3BZKkqSWIb/UwPIdpwF4dqT9tQ5BFkSr69jTh8AeXpSXVHJwyzm140iSJFnER/EpFJRVcnOIL7d0tb/WIciCqIpB95nvJR7cco6yYtlKlCTJvuUWV/BxvHlFomdH9lA5TdPJgqiCDt296NTLm4qyKpJ+Pqt2HEmSpBvywY7TFJVXMqS7HwM72+8mBrIgqqR6xOmhX85TWlihchpJkqSmySoq55OdqQDMHmnfqyHJgqiSdl08CQ7zpbK8iv0/nlE7jiRJUpMs/fUUpYYq7ujZln6d7GMR79rIgqiiqMv3Eg9vS6M4v1zlNJIkSY1zsaCMz38zv6H/2532e++wmiyIKvIP8iCknz9VBiOJm2QrUbJPK1euZPjw4YSFhTF+/Hj27dtX5/HDhw9HCHHlo1+/fgghePzxx62UWLKU9385SXmlkbtD29En0FPtODdMFkSVDbqvC2jg6I40CnPK1I4jSY2yceNGFixYwMyZM1m3bh0RERFMnz6d9PTaF7H/+uuviY+Pv/Lx5ZdfotFoGDVqlBWTSzcqLa+UVXvOodG0jNYhyIKoOt9Ad7r1b4ux0kTiD6lqx5FakJSUFGJjYwkLC2PEiBFs27aNsLAwEhISLHaNFStWMG7cOCZNmkTXrl15+eWX8ff3Z9WqVbWe4+Pjg7+//5WPHTt24O7uLguinXlv60kqqozcG94B0c5D7TgWITcItgGD7u3CqcRMkndmEHlXMG385Gap0o1JTU1l4sSJPPjgg7zyyiukpKQwb948Kioq6NmzZ41jly5dyrJly+p8vuXLlzNgwIAaj1VUVHD06FEeeeSRGo8PHjyYpKSkBuU0mUysW7eO6OhonJ2dG3SOpL6z2SV8te8cWg08M6K72nEsRhZEG+Ddzo0eg9qh7L7A3o2p3PGw3LfOVp2dMYPibdutek23YUMJqqdg/dn8+fMZPnw4s2fPBiA4OJghQ4YQHx+Pj0/NeWIxMTH1ts4CAgKueSw3N5eqqir8/GquSuLr69vgVujOnTtJS0tj0qRJDTpesg3vbj1BpdHE+MhAuvq7qx3HYmRBtBED7unM73svovx2gf53BeMV4Kp2JMlOZWRkEB8fz5o1a2o8rtfrr2kdAnh5eeHlpc5w+dWrVxMaGnrdXJJtOn2piDX7z6PTavifO1pO6xBkQbQZXm1d6XlzO5J3ZrB3Qwp3PhKqdiTpOmprqZWWluLiYhtd3ceOHUOn09G9e81fVoqiEBUVdc3xTe0y9fb2RqfTkZWVVePx7Oxs/P3r3yU9OzubrVu38uKLL9Z7rGQ73tlyAqMJYgZ0JNjXTe04FiULog0ZMLozym8X+H3vRfqP6oxP+5b1wyZZh0ajwWg0UllZiaOjIwCJiYkkJSUxderUa45vapepo6MjoaGhJCQk1Dg/ISGBkSNH1ptzzZo16PV6OZjGjpy4WMh3B9PR6zT8dXg3teNYnCyINqSNrwu9B3fgyPY09q5P4a7pfdSOJNmh0NBQ9Ho9ixYtYtq0aSiKwsKFCwEs3mUaGxvLnDlzCA8PJzIyklWrVpGZmUlMTAwAcXFxxMXFsWnTphrnmUwmvv76a+655x5cXeXtAXvx9uYTmEwQMzCIjt4t799NTruwMf1HBaNz0HIyMZOs80Vqx5HsUEBAAK+++ipbtmwhOjqatWvXMnnyZFxdXQkODrbotUaPHs3cuXNZsmQJY8aMYf/+/XzwwQcEBgYC5oE3KSkp15y3e/duUlNT5WAaO3IsvYANhzNwdNDy1O0tr3UIsoVoc9y9nQkd0oFDv5xnz/enGf1EuNqRJDsUHR1NdHT0lc8XL16MEAKt1vLvgadMmcKUKVOu+7VZs2Yxa9asax6/6aabUBQFMN9/lWzfW5t/B2BKVBDtPFvmFBnZQrRBkXcH46DXknIwi8wzBWrHkVoARVEQwr53IpDUc+h8Hj8fu4izXssTt3VVO06zkQXRBrl5OtHnto4A7Pn+2u4mSWqs48eP06uXnN8qNc2bP5tbh1Nv7kxbj5bZOgQb6DIVQnwM3AtkKopyzSgSIcRtwLdAdWVYoyjK/1ovoToiRwZxZHsaZ45kc+F0Pu1C7H/hXEk9mzdvVjuCZKcSz+Tyq3IJN0cdM4a13NYh2EYL8RPg7nqO2aEoSr/LHy2+GAK4eDjS93ZzK3H3d6dVTiNJUmv15s/me72xg7vg4+aocprmpXpBVBRlO5Cjdo7ryf7oY3L/u1q16/e7MwhHZx3nj+eSfiJXtRySJLVOv53OZufJbDycHZg+JES9IPtWwM53m/0yqneZNtDNQoiDQDrwnKIoR+s7oby8nOTk5Bu76uLFUF7OhfbtoAErbzSHwL4upOwu4pcvjxJxvw8ajabZrlVWVnbjr1kLYzAYGjQK0mQyydGSTdTQ185gMMifzz9pzv+zJpOJ/9uUAcCYnh6knzlJ7Zt6NR+H4gy6rf8bJgdnFJ87m/dazfrslrEfCFYUpUgIMRpYB9S7gJ6Tk9MNDyJIu+MOCjZswO/AQfyfvnbouDWEdK7k88MJ5KVV4KFpR6dePvWf1ETJycly4MWfJCcnN2hJNltaus3eNPS10+v18ufzT5rz/+yOE5c4kpmCl6ueF8YOxMNZ3yzXqdfWNYAJTa97LfK9JiYm1vo11btM66MoSoGiKEWX/74R0Ash/Oo5zSK8Lk8azvvmG0yVlda45DWcXByIGBkEmO8lmkwmVXJIktR6mEwmFv1kHln6+NAQ9YphVSUkfW7+e/9pzX45my+IQoh2QgjN5b8Pwpw52xrXdh00EMfgYCovXqRo+w5rXPK6wm7riIuHnospBZw5YpVvXZKkVuwXJZMD5/LwdXNk6s2d1Qty4icozADfbhA8uNkv16guUyHETZhHhN4EdABcgCxAAbYB6xRFadToDyHEKuA2wE8IcR74B6AHUBRlKTABeEIIUQmUAjGKolilmaTRaPCaNInMf/+bvNWr8Rh+uzUuew1HZwciRgaT8M1J9nyfQnAf32a9lyhJUutlMpmuzDt84rauuDmpeGct8RPzn/2ngRV+5zXoOxVCTAWeA0KBQuAgcAJzgfIBooC/AP8RQqwG/qkoSoNmlCuK8mA9X38PeK8hz9UcPMeNJfPttynavh1DRgb69u1VydFnWCAHfj7LpbOFpBzMIqSfOoN8JElq2X46dpEjaQW09XDioZssu/Zto+Sdg5M/g84R+k62yiXr7TIVQhwCXgc2Av0BL0VRhiqKcr+iKA8pijJaUZRemAvjdKAtcEwI8UBzBrcWBx8f2tw5AoxG8r5ZU/8JzUTvqKP/KPMP557vUzAZ5b1ESZIsy2g08dbl1uFTt3fDWa9TL0xSHJiM0Os+cPO1yiUbcg/xI6CLoigvKIqSVFt3paIo+YqirFQUZTTmLtU8SwZV05XBNV9/jamqSrUcvW/tgLu3E9lpRRzbqcYAaEmSWrJVe89y/EIh7T2diRnUSb0gVh5MU63egqgoyjuKopQ15kkVRTmoKMqPTY9lW1wHDUIfHETlhQsU7VBvcI2DXsct483brsSvPkHuhWLVskhStZUrVzJ8+HDCwsIYP348+/btq/P4oqIiXn31VW6//XbCw8N5+OGHOXTokJXSSrU5mVnI/PXHAHhpdC+cHFRsHZ7cDAVp4BMCnYdY7bKNHmUqhPAWQowRQjwihBgphGjxk680Wi3eEycCkLf6K1WzdB8YgPfpHNQAACAASURBVLipHZUGIz99dJQqg1HVPFLrtnHjRhYsWMDMmTNZt24dERERTJ8+nfT02nsw5s2bR3x8PK+//jrff/89N998M7GxsVy8eNGKyaWrlRmqmLXqAGUGI/dHduS+vh3UDWTlwTTVGlUQhRDDgdPAGuBDYBOQKYRYIIRo0YvceY4bB3o9Rb/+ikHl/7hDY3rQxt+FrHNF7Fp7StUsku1KSUkhNjaWsLAwRowYwbZt2wgLCyMhIcFi11ixYgXjxo1j0qRJdO3alZdffhl/f39WrVp13ePLysr46aefmD17NlFRUQQHB/PEE08QHBzMF198YbFcUuO8sek4yRkFdPZ15Z9jQtUNk58GJ34Erd5qg2mqNbaF+Bbm5dNuB9oBkcB/gKeAbUIIN8vGsx0Ovr543HHH5cE136iaxdHZgZGPhqLVaji49ZycmyhdIzU1lYkTJ9KnTx/Wr1/PvHnzmDdvHhUVFfTs2bPGsUuXLiUiIqLOj+t1g1ZUVHD06FEGD645P2zw4MEkJSVdN1dlZSVVVVU4OTnVeNzJyYn9+/ff4HctNcUvxzNZsTMVB62Gd2IicFdzmgVcNZjmXnC37mj6xn7nPTDPA9x++fNM4IAQ4l1gO/Aq8IwF89kU70kTKdy0ibyvv8Zvxgw0OvX62AM6tyFqTAi71p5iy6fHiHk5Ctc2LbqRbhPWv3fQ6m9Agvv4cu9f+zbqnPnz5zN8+HBmz55tfo7gYIYMGUJ8fDw+PjWX/4uJiWHUqFF1Pl9AQMA1j+Xm5lJVVYWfX82Fo3x9fWtthbq7uxMREcGSJUvo0aMHfn5+bNiwgQMHDhAUFNSYb1GygMzCMp776iAAz90l6NvJS91AxirY/5n571YcTFOtsS3ENOCaNcwURUkH5gMtYqpFbVxvugl9p05UpmdQvHOn2nGIuDOIQOFNaaGBLZ8ck1MxJAAyMjKIj49n6tSpNR7X6/XXtA4BvLy8CA4OrvPD2dlym8L+61//QqvVMnToUMLCwvjiiy+455570GptfuGsFsVoNDF79UGyiysY3M2Xx9XczaLayS1QcB68u0DnoVa/fGNbiCuBvwkhNl5n+sU5wMMysWyTRqvFa+JELr35JnlffYX7UOv/g9XMo+HO2N58OX8PZ4/lcHDrOfqNkO+ym1NtLTVbWtz72LFj6HQ6unevuQa+oihERUVdc/zSpUtZtmxZnc+5fPlyBgwYUOMxb29vdDodWVlZNR7Pzs7Gv47dYYKCgoiLi6OkpISioiI8PDyYO3cunTqpOMy/FfooPoUdJ7LwdtXz5qR+aLU2sPrV/k/Nf/afCiq8QWpsQeyJeVWaLUKIlxRF+Q1ACKEFYoFfLJzP5niNG8uld9+lcOsvGDIz0bdtq2oeNy8nhk/txcb3D7Fr7SkCe3jjH9Si35dI9dBoNBiNRiorK3F0NHejJyYmkpSUdE2rEZreZero6EhoaCgJCQk1zk9ISGDkyJH15nR1dcXV1ZWLFy8SHx/P888/X+85kmUcScvnXz8eB+DfE/oS0MZyPQBNVpAByg+gdYB+U1SJ0NiC2BnQYV57dKcQIh1zN2owUA6MtmQ4W+Tg74/H8OEU/vQT+WvW4jdzhtqR6BLuR9iwQA5vS+Onj44y6aWB6J1UnEMkqSo0NBS9Xs+iRYuYNm0aiqKwcOFCgFq7TL28mnbvKDY2ljlz5hAeHk5kZCSrVq0iMzOTmJgYAOLi4oiLi2PTpk1XztmxYwdGo5GQkBDOnj3L66+/TkhICOPHj29SBqlxissreXpVEoYqE1NvDmZE72vf7KjiQByYqqDXGHBXp6HRqDapoihRmLtF+wKPAGsx31N0B4KAw0KIM0KINUKIlywd1lZcWbnmq68wGW1jHuAt93fDp4MbeRdL2LH6d7XjSCoKCAjg1VdfZcuWLURHR7N27VomT56Mq6srwcGWXZty9OjRzJ07lyVLljBmzBj279/PBx98QGBgIGAeeJOSUnNZ48LCQubPn8+oUaN44YUXiIiI4KOPPkKvV2mLoVbmn98f5XRWMSLAg7mjbWR/SaMREtUbTFNNY4n99S5vz9QT81qnEZf/7KsoivcNP3kTJScnm5pr40yT0cipO0diSEuj04cf4n5r829L0hDZaUV89fo+qgxG7preh279G/cuS24QfK2Gvia2dA/xehYvXszOnTv58ssv1Y5yjYa+dvLn81qNfU3WH0rnr18k4eSg5ftZt9IjwEZur5zcDHH3g1cwPH2gWe8fJiYmJvbv33/A9b5mkQknlwfYJF/+iLPEc9oy8+CaCVx6+x3yVq+2mYLoG+jO4Pu7sf3L3/kl7jhtO3vQxtd2f0lL1qMoCkIItWNIKjqfW8LcNYcBmHdvb9sphvDHyjSRD6symKZavQVRCPHwjVxAUZTPbuR8W+U5bjyXFr9H4datVF66hEMdo+qsqc+wQM4eyyH1UBabVxxj7N8i0OrkcPbW7vjx4zz22GNqx5BUUlll5JkvD1BYVsmdvQN4KMqGRqMXXjAPptHoIOIhVaM0pIX4yQ08vwlokQVRH9AW99tvo2jzFvLWrsPv8elqRwLMIwyHP9yT/87fQ8bJfPb9cIZB93ZRO5akss2bN6sdQVLR4q0n2Xcml4A2Trxxf7htbTB+YCUYK6HnveDRTtUo9RZERVFk86IW3pMmmQviV1/h+9ijaGxkYrGLuyN3xPbmu3cOsG9DCp16etO+m8orUEiSpIq9qTks3noCjQbemtQPHzcbWtHKaITE6rmHsepmoQm7XUh/cBs8GIcO7TGcO0fJb7+pHaeGTj19iBwZhMkEP318lPISg9qRJEmysvwSA898eQCjCWYO68ot3fzqP8maUn6FvDPgGQRdb1c7jSyIN0Kj0+E1YQIAuSpvC3U9g6JDaBvsQVFOOb+uVLDEiGJJkuyDyWTipbWHScsrpW8nL569s4faka5VYzCN+nOnZUG8QV733w9aLYWbN1P5pyWs1KbTabnz0VD0TjpOJmaSnJChdiS7JN9IqE/+GzTe6n3n2HA4AzdHHe/G9ENva4PrijLh+IbLg2nUWZnmz+Qo0xukDwjA/bbbKNq6lfx16/C1sZF8Xm1dGfpgD7Z8ksyO//5O+66eeLdrsbt0WZxOp8NgMFxZAk1Sh8FgwMFB5W2J7MipS0W88t0xAOaP7UOwrw3+n68eTCPugTYqb0h8mRxlagFekyZStHUruV99hc8jj9jM4JpqIqodZ4/mcGLvRX7++Bj3P98fnd62MtoqLy8vLl68SGBgoNyNQSVGo5GLFy/i6empdhS7UF5ZxdOrkig1VDG2XwfGR3ZUO9K1agymmaZqlKvJUaYW4D5kCA7t2mE4c5aSPXtwu+kmtSPVoNFoGDZZcDEln0tnC/nt21MMntC9/hMl/Pz8OH/+PIqi1HmcwWCQS481UUNeOzc3t2v2XZSu79+bFI6mF9DJx4X5Y/uoHef6UrdDbgq06Qjd7lA7zRWyD8ICqgfXZL33HnmrV9tcQQRwcnHgzkdCWbNwPwc2n6NTLx+CQn3VjmXztFptgzaulcuKNZ187SznVyWTD+NT0Gk1vBMTgYezjb5Js7HBNNUa1foTQjwvhLhDCKHaGqW2yuv+8aDVUvDzZipzctSOc13tQjyvTNLf/GkyJQUVKieSJMlSLhWW89xXBwF49s4eRAbZ6K/pokuQvB40WtVXpvmzxnaHvgL8BGQJIVKEEN8IIf4uhBglhLCRPUTUoW/f3rxhsMFA/tp1asepVeTdwXTo7kVpQQVbP0uWo/ckqQUwGk08//VBsooquCnEh5nDuqodqXYHvwCjAbrfBZ6BaqepobEF8WYgA9gL7AT8gbnAeiBdCJEmhPheCPFPIUS0ZaPavivbQq1ebbOFRqvVMCK2N06uDpw5ks2hrefVjiRJ0g1akZDKr8olvFz1vPVAP3RaG1qa7Wom0x/dpTY0mKZaYwviMmCloig3KYrykKIoQ4EQ4GPMGwV/ATgBT2LeK7FVcR86BIeAACrOnKFkz16149TKw8eZ4X8x37NJ+OYkZ45kq5xIkqSm+uV4Jq9tTAbgjfvDae9pwzvcpO6AnNPQJhC6jVA7zTUaWxD7ARuvfkBRlExFUaYD24FMRVFGKoriD7S6FaU1Dg7mifqYW4m2LCTCn4g7gzAaTWxadpj0E3lqR5IkqZGOXCxlZlwilUYTM4aGcFeouotj16u6dRjxF9DZ3pjOxhbEDKC2cbyfAjOqP1EU5WxTQ9kzrwn3g0ZD4U8/UZmbq3acOt08viu9bmlPpcHIhv8cpDBTrncqSfbiSFo+/9hygfJKIw8M6MSLo3qqHaluxVmQ/L1NDqap1tiC+BHwihAi8jpf8wNa9cAaAH2HDrgNHYLJYCB/3bdqx6mTRqPhtod60jXSn4qyKg6syyH3QrHasSRJqsepS0VM/XgPJQYT94S1Z8H4MNva0ul6Dq6Cqgrodid4dVI7zXU1tiC+DmwF9gghvhRCTBJC3CaEeBxYCCRaPKEd8raDwTXVtFoNd8aG0qm3D4ZSI9+9c4DCnDK1Y0mSVIu0vFL+8uFusosr6N/BxbYH0VSz8cE01RpVEBVFqVIU5QHgUSAc+BLYAiwF8oGZFk9oh9yHDcPB35+KlBRK9+1TO069dHoto2aE4dleT1FuOd+9c0DOUZQkG3SpsJyHPtxNen4ZA4K9mXdbAI4OdrCY2JmdkH0SPNpD95Fqp6lVk15JRVE+VRSlN+aBM0OBXkAfRVGOWzKcvdI4OOA5wTy4xha3hboevZOO8GgffAPdybtYwveLD8g9FCXJhuSXGnj44z2kZBXTq30bPpo2EGd7WZPYxgfTVLuhV1NRlDOKouxUzIyWCtUSeN0/wTy45scfbX5wTTW9s5bo/+mHp78LWeeK2PD+IQwVVWrHkqRWr7Siikc/2UtyRgFd/Nz47JFBeLrY6LJsf1aSA8e+BTQQ+Re109Sp3oIohPhOCBHR0CcUQjgLIZ4VQrTq7lPHjoG43XorpooKCr77Tu04DebaxpHoZ/rh7u1Exsl8Ni07TFWlfK8jSWqpqDQyIy6RfWdyae/pTNxjUfh7OKkdq+GuDKYZAV71rwuspoa0EFOB34QQu4UQTwshIoUQNdq8QogOQoixQoiPME/NeBTYb/m49sVr0kTA3G1q64NrrtbG14Xo/+mHs7ues0dz+PnjYxiN9pNfklqKKqOJv/33ANt/v4SPmyOfPxpFoJcNT7z/MzsZTFOt3oKoKMrTQG9gD+a1TPcCZUKIHCFEhhCiFDgHrAFCgWeAcEVR9jRbajvhcdtt6Pz9qDh1itL99vX+wLudG/fN6oveWcep/ZlsW3ncroq6JNk7k8nES2sOs+FwBh5ODnz2yCC6tXVXO1bjnN0FWb+DezvocZfaaerVoLubiqKcAmYJIWZjXs80CugAOAPZwHFgu6IoZ5orqD3S6PV4jb+f7GXLyFu9Gtf+/dWO1Chtg9tw71PhfPfuQY7tzMDJVc/N47va/nwnSbJzJpOJ1344zn/3ncPJQctH0wbSJ9AON0i+MpjmIdDZ/j3PRg33URSlAth2+UNqAK+JE8hetoyCHzYRMHcuOi8vtSM1Sofu3tz9eB9+WHKYpJ/P4uTmQP+7O6sdS5JatPd/PcUH20/joNWw9KH+DOrio3akxivJgaPrsIfBNNXsZMyu/XLs2BG3wYMxVVSQt852t4WqS+cwP0bE9gYN/LbuNEe2yR0yJKm5fLYrlX//qKDRwFsP9OP2nm3VjtQ0B1dBVTl0HQ7endVO0yAWK4hCiLuFEK3+vuH1eD8YA0DWf97HcOGCymmapvvAAIY9KADY9uXv/L7HPr8PSbJl65LS+H/fHgXg1bFh3Ne3g8qJmij/PPz6hvnvAx9VN0sjWLKF6ATY100yK3G/4w7cb78dY2EhGS+9hMlon9MY+gwN5OZxXcEEmz9JJvVQltqRJKnF2HzsIrMv73j/4qieTI6y7SkKtTIaYd2TUJ4PYrT5w06o3mUqhPhYCJEphDhSy9c1Qoh3hRAnhRCHallY3KZpNBraz/9fdN7eFCfsInfVKrUjNVnkXcFE3hWEyWhi0/IjpP1uH4sOSJIt23Uqmye/2E+V0cQTt3W17R3v67P3Q0jZBq6+cN87YEeD8BoyMT/8z/MOLewT4O46vj4K6H7543FgSTNmaTYOfn60e+UVADL/vZDylBR1A92Am8Z2JXRIB6oMRja8f4jMMwVqR5Iku3XwXB6PfbqXikojU6KCmHOXUDtS02WdgJ//n/nv970D7vZ1/7MhLcQDQJEQYv/l1tzTQoihQog2lgigKMp2IKeOQ8YAnymKYlIU5TfASwjR3hLXtrY2d43Ec0w0prIy0l98EVNlpdqRmkSj0TD0QUG3AW0xlFXx/bsHyUmX20ZJUmOduFjI1BV7KK6o4r6+HfjfMX3sd1pTVSWsnQGVpdD3Qeh1n9qJGq0hLb8ngb5AP2ASMA0wAQghzmAumAeA5poxGoh54n+185cfy6jrpPLycpKTk5sp0g2YOBHid1J28BDHF7wGEyeoneiKsrKyRr1mHW/SkpvlRHZqOd8s2kv/ib64eNruwr3NqbGvnfSH1vraXSg08NwP6eSVVjGooyuPhzvxeyP3R7Cl18736Me0TUvE4BrA6ZBHMNpIrsZoyG+vHxRFWQrm+3lAD8zFsfrjZmDs5WNtZikTJycnevXqpXaM6yr+9784+8ijsHo1XSbcj3Pv3mpHAiA5ObnRr1mPHlV8/+4BMk7mc2htPvf8tS/+nTyaKaHtasprJ5m1xtfuaHo+L67dS3ZpFYO6+PDZI4Nw1usa/Tw289qlH4BjHwOgn/ABImSQyoFql5hY+7a9Deky/U4I4QpwudtSURTlv4qizFUUZZSiKO2B9sBo4CWLJK4pDbh6e+WOlx+zW2633IL3Qw9BZSXpL7yAsbxc7UhNpnfUcc9TfWnfzZPi/ArWLtzPmSPZaseSJJv1i5LJpKW7uFhQzqAuPnw4dUCTiqHNMJSZu0qNlRA1E0JuUztRkzWkILYDVtZ1gKIoFxVF2QRcd6ToDfoOePjyaNObgHxFUersLrUHbWc/i2PnzpSfOMmld95VO84NcXJxYMz/RNB9YACG8io2vH+Iozvs+j2LJDWLlbvP8Nin+yiuqGJMvw58/ugg2jjb/pJmddo6Hy4dB9/ucMc/1E5zQxpSEO8HRgkhXq3rICHE34FvGxtACLEK2GX+qzgvhHhUCDHzqu2jNgKngZPAcsz3NO2e1sWFDv96A3Q6clasoGTvXrUj3RCdXsudsb3pPyoYk9HErysVdq09hUnukiFJGI0mXv/hOH9fe4Qqo4lZw7vx9gP9cHKw45YhQGo87PoPaHQwbhk4uqqd6IbUew9RUZR4IcSTwHIhxBFFUWpMohNCuGCeOjERWNHYAIqiPFjP103AU419XnvgEh6O34zHyXp/CekvzqXLt9+ic3dTO1aTabQabhrTlTa+Lvz6hcL+H89QkF3KHVN74WDPXUKSdAPKDFXM/uogGw5loNNqWDCuDw8MtNNJ91crK4B1TwAmGPocdLT/dVkaNDFfUZSPgXeBD4UQA6ofF0IEAwmYB9U8pSiK/azRYyP8Zs7EuXdvDGlpXHz9NbXjWETvWztw71/D0TvrOLkvk+/ePkBZkUHtWJJkdTnFFTz04W42HMrA3cmBFdMGtoxiCPDjXMg7C+37wtDn1U5jEY1ZqWY2sANYJ4RoJ4S4DfPeiAHA7Yqi2OWEebVpHB3p8MbraBwdyf/6Gwq3/qJ2JIsI6u3L+Of64+blRMapfL7+1z7yMkvUjiVJVpOaVcz9SxKu7HT/9RM3M7SHv9qxLOP4RkiKA50TjPvALrZ2aoiGrFQTIYTQK4pixDwPsRCIB37EfF+vv6IoCc0bs2Vz6t4d/7/9DYCMl1+mMqeudQrsh19Hdya8MADfju7kZ5byzb8SuXA6X+1YktTsEs/kMO79naRkFdO7fRvWPjmYnu0sspaJ+oqz4PunzX8f8Q9o21PdPBbUkBZiIuaVapKAt4EtQAjwFTC0JYz4tAU+Ux/GdeBAqrKzufCPV1rM7vTu3k6Mfy6SoFAfyooMrHsziZOJmWrHkqRms+FQBg8u301uiYHbhD+rZ95MO09ntWNZhskE3/8PFF+C4Fsh6gm1E1lUQwri45hHdxYDE/hjlOeDwHEhxFdCiL8LIUYLIex0rxL1abRa2r/2Glo3Nwp//pmC779XO5LFODo7cM+T4fQe0oGqSiM/Lj9C0k9nW0zRlyQw73K/bNspnvpiPxWVRiZHBfHhwwNwd2pBqzcd+i8cXw+OHjBuCWhV3x/CohoyyvTDqz8XQnTnj1VqIoDBmKdmgHmlGjmcsIkcOwYS8NJLZPz971yY/3+4DhyIvr1dLtt6Da1Oy22TBZ5+Luxae4qENScpyCplyAPd0epa1n8qqfWprDLyj++OsnL3WcC8fdOMoSH2uy7p9eSfh42XB8+MegO8WsjgoKs0+q2LoigngBOYu0wBEEK0BSIxr3kq3QDP8eMo3LKFoq1bSX/pJYI++ghNC3kXptFoiLwrGA9fZ7Z8ksyR7WkU5pQx8rFQHJ1b0LtoqVUpLq/kr1/s5xflEo4OWt6c1Jd7w1tYZ9mVPQ4LQNwD/SarnahZWOQ3raIomYqibFIU5Q1LPF9rptFoaP+//0Tn7U3Jrt/I/cJ+906sTfcBAUQ/0w8nNwfOHMlm7aL9FOfZ7/J1Uut1saCMSct28YtyCW9XPV88FtXyiiHA3uWX9zj0s7s9DhujZTQ9WhgHPz/a/e8/AchcuJDy0/a7d2JtOnTzYsKcAbTxdyHrXBFfv7GP7LQitWNJUoMdv1DA2P/s5Gh6AZ19XVnz5GAGdPZRO5blXfr9qj0O3wb3FjJ15DpkQbRRbe68E88xY+x+78S6eAW4MmFOf9qFtKEot5w1/07kXHLLmHIitWw7TlxiwpJdZOSX0T/YmzVPDqaLn/2uMlWrK3sclkHfyXa5x2FjyIJowwL+/hIO7dtTdugQ2cuXqx2nWbh4ODLmmQi6RvpTUVbF+sUHSU5IVzuWJNVq9d5zxK7YS1F5JfeEtWflY1H4uDmqHat5xL8J6fuhTUcY9braaZqdLIi1MJlMzNk2h38k/EO16QG6Nm3osMC8pvql/7xP6dGjquRobg6OOu56rA8RdwZhNJrY+tlxtq9SqDRUqR1Nkq4oM1Tx/749wpxvDlFpNDFjaAiLH4yw762b6pKeBNsuDwsZ+z44e6oSw2Qy8Y+EfzBn+5xm/10sC2ItNBoNiZmJrDmxhk2pm1TL4XbzzXj/5S8tYu/Eumi0Gm65vxvDJgu0Og2Ht6Xx9ev7yEkvVjuaJHHiYiFj/7OTz3adQa/T8H9j+zB3dC+02pY5uARDKayp3uPwCQgZplqUH1J+YM2JNSReTGz2aSyyINbhyb7mNQgW7VtEiUG9dTjbPvs3HLt0oeLkKS69/Y5qOayhz9BA7p/TH09/F7LTivnqtb0c3ZEmJ/FLqjCZTHyx+yz3vRfP8QuF5sEzTwzmoZuC1Y7WvLb+H2Qp4NfDvDybSkoMJSxKXATAU/2af9MjWRDrMLbbWHr79uZiyUU+OvKRajm0Li50eON1896Jn3xC8Z49qmWxhrbBbZj094GIm9pRaTDy60qFH5cfoaxY7pghWU9+iYGnvtjPS2sPU2Ywcn9kR9Y/PYSwjup0HVpNyo6r9jhcCnoX1aJ8ePhDMksyCfUNZWy3sc1+PVkQ66DT6pg7aC4Anxz5hPOF51XLYt47cQaYTGS8OJeqggLVsliDo7MDI6b1ZkRsb/ROOk7tv8R/X91Dxim5OLjU/Pal5jD63R1sPHwBdycH3n6gH4sm9W1Zy7BdT2meeQI+JvOWToHq7XF4rvAcnx79FIAXB72IVtP85UoWxHr0a9uPe0PupcJYwcJ9C1XN4vfETJxDQzGkp3N2+nSqCgtVzWMNIqodD8wbSNtgD4pyylm7aD/7NqZgNMouVMnyqowm3t1ygknLdpGWV0rfjp5sePpWxkYEqh2t+ZXlQ9x4yD8L7fuZN/1V0cK9C6kwVnBfyH30a9vPKteUBbEB/tb/b7g4uLDl7BZ2pe9SLYdGr6fj4nfRBwZSdvAQ5x6bTlVRy5/M7unvyvjn+xMxMgiT0cTu71L49q0kinLL1I4mtSAZ+aVMXv4bb/78O0YTzBzWla9m3kKwbwucX/hnZQXw+XhISzSvUfpAnKp7HCakJ7D13FZcHFx4pv8zVruuLIgN0Na1LY+HPw7AG3vewGBU716WvkMHgj79FH2HDpQePHi5KLb8kZg6By23jO/GfU/3xaWNI+kn8vjy//Zw+sAltaNJLcCPRy8w6p0d7E7Jwc/dic8fHcSLo3ri6NAKfkWWFUDc/ZC2z1wMp20Ar06qxTEYDbyxxzzdY0b4DNq6trXatVvBv7ZlPNz7YTp5dOJU/in+e/y/qmZx7BhI0Gef4tChPaUHDnDu8cdbRVEECOrtS8y8QQSF+lBeXMkPSw+b5yxWyDmLUuOVGap4ed0RZnyeSF6JgWE9/Nn0zBCGdG+5y5PVUF4IKyfA+T3g2Qmmrld9F4svj3/J6fzTBHkE8Zfef7HqtWVBbCBHnSNzBs4B4P0D75NTpu4SY44dOxL86ac4tG9P6f79nJsxA2Nx6yiKrm0cufepvgye0O2POYtvyDmLUuP8frGQMe/t5PPfzHML593TixXTBuLn7qR2NOsoL4S4CXBut3klmmnrwVvd6STZpdksObAEgDkD5+Cos+4KQLIgNsKwjsMY3GEwhYZC3t3/rtpxcOzUieBPP8GhXTtKExM524qKokarod+IICa8MADPtnLOotRw1XMLo9+LR7lYSBc/N9Y+OZjHhoS03In2f1ZeBCsnwbnfoE3g5WLYWe1ULE5aTKGhkMGBgxnaqcN+VQAAIABJREFUcajVry8LYiNoNBrmDJqDg8aBNSfWcCz7mNqRcAwKMhfFgABK9yVybsZMjCXqLSJgbf5BHkx6aSA95ZxFqQHySww8ufKPuYUT+ndk/axb6RPYwucWXq28CL6YBGcTwKODuRj6dFE7FUezj7LmxBocNA68MPAFVTZXlgWxkUI8Q5jSawomTLy2+zWbaI04Bgebi2LbtpTs28e5mU+0qqLo6OzAHdVzFp2vmrN4Mk/taJIN2Zuaw6h3tvPDEfPcwndi+rFwYl/cWvrcwqtVFMMXD8CZneDR/nIxDFE7FSbT5d+nmHio90N08VSnQMuC2AQz+s7Ax9mHA5cOsCFlg9pxAHDs3JmgTz/Bwd+fkj17OPfEkxhLS9WOZVUiqh0P/L3mnMU961OoqjSqHU1SUUWlkbc3/84Dy3aRnl9G305ebHx6CGP6tYK5hVerKLlcDOPBvZ15NKlvV7VTAbD+9HoOXjqIr7MvM8JnqJZDFsQm8HD04JlI89yYt/a9peo6p1dz6tKFoE8/RefvR8nu3a2yKFbPWYy8KwiTCfauT2H1gr2ky9Ziq7Q3NYd73t3B25tPXJlb+PXMmwnydVU7mnVVlMCqByB1x+ViuN5mimGxoZi3Et8C4Jn+z+Du6K5aFlkQm2hMtzH08e1DZmkmyw/bzl6FTiFdCK4uir/9xvmnnsJY1romsOsctNw8rhtjnumHp78LOenFrF24n18+T5b3FluJvJIKXvzmEBOX7uJEZhGdfV354rEoXhzVE72ulf3aM5TCqhhI2Q7uAeZi6Ndd7VRXLD+0nEullwjzCyO6a7SqWVrZT4blaDVaXox6EYBPj37KuYJzKif6g1NICMGffILOz4/ihF2cf7L1FUWAjj19iPl/gxgwujNanYZjOzP44pXfUHZfsIl7v5LlmUwm1iad545F2/hy7zn0Og1P39GdTc8M5ZZufmrHsz5DKax6EFK2gVtb8zxDGyqGZwvO8tmxzwDrrVdaF1kQb0Bf/77/n73zDo/iuvrwu027q9Wq9y4kGAkQvTvGptkkjrEd23Hv3XGP4ySf4zgxKY6TuMZObIhxrzi2SdwAU206CBAgLagX1Pv2Nt8fIwkJBAgk7a5g3ueZZ3aKdg+XmfnNPffcc1iUuQiX18UzO57xtzm90GZmkvbGMlRRUVg2baLqvvvP2FqKJ0KtUTF90Qiu+s00EkeGY+twsXrZAVa8sJvWusBwdcsMDiUNZq7/91Ye/nAPTRYn0zMi+erB2TyyYNSZW8T3RLjs8MG1ULIWDDFSzzBmlL+t6sVft/8Vl9fFosxFjIsZ529zZEEcKA9NeohgdTDrKtfxffX3/janF9qsLEkUIyOxfPfdWSuKAJEJBi59ZCJzb8xBZ9BQVdjCB4u3sf2LUjwuOehmOONwe3hh9SEWvrCR74uaiAjW8NcrxvHBnTPIivXfeJRfcdnhw+ugeA0ER0s9wxjB31b14rvq71hXtQ6DxtAdk+FvZEEcIDHBMdw1XoqKenrb07g8gTVGpR05ktQ3lqGKiMCycSNVDzyA1+n0t1l+QaFQkDMrgWt/P53smfF43F62/beUD/6wjeqDLf42T+Y02FzcxA9f2Mhzqw/idEvzCr/9+flcOSXFL/PYAgK3Az68HopWd4rhfyE2299W9cLl6Z2vNCY4MFLlyYI4CFyfcz1poWmUtZfxXuF7/jbnGHSjRpH6xhuowsOxrN9A9f1nrygC6EOCmHfTaC59eCLhccG01ln57Nk8vn3zADbz2dsuw4lmi5Off7SHa5ZsoaTBwogYA+/fMYO/XTmeSINv030FFG4HfHgDFK2C4Ci4aQXEjfa3VcfwXuF7lLWXkR6azvU51/vbnG5kQRwEeuY5/deef9Foa/SzRceiE0ZJPcXwcMzr11P94EOIZ7EoAiQJEVz9m2lMuzgDlVpJ4eZa3ntyKwWbauSgmwBFFEU+2lHJvL+v45NdVQSplTyyYBRfPXguMzOj/G2ef3E74KMb4dA3oI+EG1dA3Bh/W3UMjbZG/rlHylf6i6m/QOPHMlNHIwviIDE7eTbnJp2L2WUOiDynfaHLziZ12euowsIwr11L1cOPnPWiqNIomXpRBlc/MY0kIQK7xcWatwr47Nk8WmrPjryww4WiejNXv7aFx5bvpcXq4pysKL55aDYPzBuJVn0WBs30xO2Ej2+Gg1+DPkLqGcaP9bdVffLCrhewuCzMTp7tl3ylJ0IWxEHksamPoVaq+azoM/Y17vO3OX2iy8khddnrKMPCMH/7LVUPPnRWFBk+GeFxwVzy0ATm3zIavVEj1VtcvI2tK0pwu+TSUv7E7vLw7EoTP3xhQ2e9wiCev2oC79w2nYzos6B478mwt8NHN4DpS9CFSz3D+Fx/W9Un+Q35fFb0GWqluturFkjIgjiIpIelc0PODVKe021/xisGZvSibvRoUl//tySKa9dSdvkV2AsL/W2W31EoFAjT47n2dzMY/YNEvB6RHV+W8cHibVQW+rfc19nKd4caWfj8Bl5cU4TLI3LNtBRWP3Iel05MOnuDZnqgbTkEr50v9Qx14XDj55Dg/+kLfeEVvTy97WkAbhh9A2mh/i011ReyIA4yd467k2h9NHsb9vJFSWDkOe0L/ZgxpH/wPtpRo3CWl1N21dWwerU8dgboDBrmXJ/NZY9OIjLRQFu9jRXP7+abJftoa5DnLvqCskYLP3tvF9f/eytlTVZGxYWw/O6Z/Pkn4wgPPouDZroQRdj5Junf3g7NxRA3Fm7/FhIn+Nuy4/K/kv+xt3Ev0fpov+YrPRGyIA4yIUEh3XNqnt35LBZX4I5DaTMySP/oQ8KuuBzR4YCXX6HmV78+qyplnIjErHB++n9TmXHpCFQaJUU763nvya1s+OAg1vaze+x1qGg0O3jy833Mf3Y9X+ytQatW8thCgf/dfy5T0iP9bV5g4LTAp3fDfx9A6XHApBvh9tUQneVvy46L2Wnuzlf68OSHMWgC09UtC+IQcHHmxYyLHkejrZFX977qb3NOiFKnI/EPfyDh6T+DVkvb559T+tOf4igu9rdpAYFKrWTywnSu+/0Msmcl4BVF8tdV8c4Tm9n+RSlOu9vfJp4RWBxuXlh9iPOeWcubm8vxiiJXTk5m7aPnc+/5WQSp5UcVAPWFsGQu7P0ANMEcnvZbWPQSaPT+tuyEvLb3NRptjYyLGcePR/zY3+YcF8WZ6iIrKCgQc3Jy/Pb7+Q35XPvltVKQzSWfBaS//GgKvvmGoBdfwllcjEKvJ+H3vyNskX+T7QYaTdVmtnxWTFl+EwD60CCmXZSOIqqdMWMDb75XoOPyeHl+xXY+3N9Bo1nKojQvO5bHFmYjxBv9bF2AsedD+N9D4LJCtAA/fZOCJvDnc64/lLWVcdmKy3B73bx/0fuMjfZv9OvOnTt3Tp48eUpfx+TXriEiNyaXS7Muxe1188z2wMpzelxSU8n46ENCL74Y0Wbj8GO/pOaJ356VicGPR1RSCBf9bDyXPjKR2PRQbO1O1r9/kG3vNFC8q14eg+0noijyZX4NFzy3gZe3NtJodjAhJZwP75zBv2+eKothT1w2WPEAfHqnJIbjroI71kBsYAthF89sfwa3181lWZf5XQxPxllUKtr3PDjpQVaVr2JD1QY2VG0IuDk3faE0GEh85i8ET51C3R/+SOvHH2PLzyf5+ecISk/3t3kBQ9KoCK745WSKdzWw5fNi2uptfP3aPuIyQpn1k0wSR0b428SAZWtJE3/+qpDdlVKNyqRQDb+5OJeFY+PlyNGjaSqGj26CunxQaeFHf5XGDIdJO22o2sDG6o2EaEJ4YNID/jbnpMg9xCEkWh/NPePvATqzugdYntPjoVAoiPjpT0n/8AM0aak4CgspvfwK2r/+2t+mBRQKhYKsybFc8+R0Rs0JRR8aRF1pO5/+PY8vXt5DU7U8v7MnptoObn1jO1e9toXdla1Eh2hZfOlY/nVJMj/MTZDF8Gj2fwqvnieJYeQIKXBm8k3DRgxdHle3d+zu8XcTrQ/88lsB0UMUBGEh8AKgApaaTKanjzp+M/BXoLpz1z9MJtNSnxp5mlybfS3LDy6nrL2MV/a8woOTHvS3Sf1Gl5NDxvLl1Dz+GzpWrqT6oYexXr+T2Md+gTJIDn3vQqVSkjzOwHmXTGDPt5XkraygLL+J8n1NCDMTmPbjDIyROn+b6TcOt9p4dtVBPtlVhSiCIUjFnbMzuf3cDAxaNQUFBf42MbBwO2Dlb2Dba9L26Etg0T9AF+pfu06RV/a8Qnl7ORlhGVybfa2/zekXfhdEQRBUwMvAAqAK2C4IwgqTyXTgqFM/NJlM9/ncwAGiUWn49fRfc/equ1mavxQFCu6feP+weRtWGY0kvfA8Le+8S90zz9DyzjvY9uwh6bnnCEpO8rd5AUWQTs3UizIYc24SO74oZf/GwxRuquHQ9jrGzUlm0oVp6AyBk7dxqGmzunhlXRHLNpXhdHtRKxVcNyOV++eNJDpE62/zApOWMikF2+E8UGrgwj/CtDuHTa8QpPHhF/Ne7H7e/WrarwIqX+mJCASX6TSgyGQylZhMJifwAXCJn20aVGYlzuJP5/4JlULFkvwl/GX7X4ZV8IVCoSDyhutJf/cdNImJ2PPzKf3JT+hYs8bfpgUkwaFBzL5G4Jonp5M1ORaPy0veygreeWIzeSsrzvhUcHaXh9c2FDP7r2t5dUMJTreXi8YlsPqR8/j9JWNlMTwehV/Cq7MlMQxLhVu/gel3DSsx9Ipe/rL9LyzNX4pKoeLP5/6ZWYmz/G1Wv/H7tAtBEK4AFppMpts7t28ApvfsDXa6TP8MNAAHgYdNJlPlib539+7dolYbWDfetuZtPF/8PG7RzdyYudyZfidKRSC8k0jY7XZ0upO49jo64MWXYMcOafvSS+G6a0Htd2eDXzlR27XXOin6voPWKmkyv9aoIn2KgfjRwajUw+dhdzKcHi+risx8mN9Cg0US/fHxOm6ZHIkQffzrql/X3ZmM103s3n8SZXoXgI7EH3B42hN4tWEn/dNAajuv6OXV0ldZ27gWtULNQ1kPMS1imr/NOgar1XrcaRfD5Sn2X+B9k8nkEAThLuBNYO6J/kCr1Qbc/JwccshKz+KhtQ+xpmENWoOWP577RzTKwHAnFBQU9KvNxLfepHnZMuqffQ4++wx9eTlJzz2LJj7eB1YGJidsuxyYdr5Ixf5mNn9aRFO1BdPadip32pkwP5UxsxMJ0g2XW/FYzA43720tZ8nGUho6pLmE2fFGfvnDbM4fFXPS4YH+XndnJG3VsPwWqNwKChXM/x3GWfcj9LNXGCht5/K6+L+N/8faxrXoVDpemPMCs5ICs2e4c+fO4x4LhLuwGkjpsZ3MkeAZAEwmU1OPzaXAMJnYdyznJJ3DP+f/k/vW3MdXZV9h89j423l/Q6sKrN7siVAolUTddhv6CROofvgRbHl5lF72ExIWP0XIvHnDZnzUlygUCtLGRpEyOpLiXfXs/Lqcpiozm/5TxM6vyxg3J5lxc1LQhQTGy1F/aLE4eWNTGW9sKqPNJkVQ5ySEcu/5mfwoNwGVUr4OjosoQuEXsOJ+sDWDMRGuXAapM/xt2Snj8Dh4dN2jrKtah0Fj4OV5LzM5brK/zTotAkEQtwMjBUHIQBLCq4FeIUmCICSYTKaazs1FwLAOS5sSP4UlC5Zw9+q7WVe5jvu/vZ/n5zxPsCbY36adEsGTJ5Px2acc/sVjWL7/nqr77scw+1zi/+//5DmLx0GpVDByShxZk2Mp39fEzq/KqS1pY/sXZeStrmTsuYlMmJ+KITxwX5Dq2u0s3VjCu1srsDol1+iUtAh+NieL84WT9wjPepqK4atfSlXtATLnwk+WgCHwpyUcjdVl5cG1D7KlZguhQaG8uuDVgJ98fyL8Logmk8ktCMJ9wDdI0y5eN5lM+wVBeArYYTKZVgAPCIKwCHADzcDNfjN4kMiNyeX1C1/nzlV3srlmM/esvod/zPsHxqDhlaFDHRlJypLXaHnnXRpefBHLho2UbF5E5K23En3XnSiDh5fI+wqFQkF6bjRpY6OoKWpl51flVBxoZvfqSvauqyJnZgITL0gjLCZwclSWN1n41/oSPtlZhdMjlTabPSqG++ZkMS1DTrx9UpwW2Ph32PQSeJygDYO5j8PUO0AZOLEE/aXD2cHPvv0ZefV5ROoieW3BawiRgr/NGhB+D6oZKvydy7S/lLaVcsfKO6iz1jEmagz/mv8vwnXhfrFloOMR7sZG6v/+LG2ffgqAOiGBuF/+EuOFF5zxvYbBGMupL29n19flFO9uAFEKLhw5NY5JF6YRlRQySJaeOqbaDl5ZV8R/9xzG22nXD8fGc895WeQmnzzw42QEyjjYkCGKcOBz+OZxaK+S9k24HuY/CSGxA/pqf7Vdq72Vu1bfxYGmA8QFx7H0gqWkh6X73I7T4US5TGVBDACqzdXc/s3tVJmryArPYskFS/yS1WGwbi5rXh61ixfjOCB5tg2zZhL3+ONoMzMH/N2BymA+mFpqLez6ppyDW+vweqX7M2N8NJMWphGfMXAB6i95FS28vLaY1QV1AKiVCi6dmMTd52WSFTt4An1GC2KDCb56DErWSdsJ4+FHf4OUwYm+9EfbNdoauWPlHRS1FpEckszSC5eSFDJ85iSfSBD97jKVgaSQJN5Y+AZ3rJIuspu/vpklC5aQEJLgb9NOi+CJE8n4+GNaP/6Y+ueex7JpMyWXXErkTTcSfc+9qEICsxZaoBARb2DeTaOZ+uMMdq+q5MD3hynd00jpnkaSsyOYvDCNJCFiSHrdoiiyqbiJl9cWsalYimXTqpVcPTWFO2aPIDlCdoH3C0cHrP8LbPkneN1SNft5v4XJN4NS5W/rTpsacw23r7ydio4KRoSNYMkFS4gNHlgvN5CQBTFAiDPE8cbCN7hr1V0UNhdy09c3sfSCpaSGpvrbtNNCoVIRcfXVGC+8kIbnnqf1449p/vfrtP/3f8Q+9hihF/3ojHejDpTQKD2zrx7FlB+ls+fbSvLXV1FV2EJVYQtxGaFMXphGem40ikGI5vR6RVYX1PHyumL2dCbdDtGquWFmGreek0GMMXCDfAIKUYT85VLqNXMtoJBEcO5vwRDlb+sGREV7BbevvJ0aSw05kTn8a8G/iNSdWWPHsss0wGh3tnPP6nvY27CXGH0MSy5YQma4b1yNQ+l+seXnU7v4D9j37gUgeOpU4p74DbpRo4bk93yNL1xXDquL/HXV7FlTid0sTXOISDAwbk4yo6bFndZcRovDzad51by5qYxD9VIy8khDELeek84NM9MJ0w/9NJAzxmVatx++/AWUfy9tJ02W3KNJk4bsJ33VdkUtRdyx6g4abY2MjxnPK/NfITRoeOVW7UIeQxxmWFwW7l9zP9trtxOuDefVBa8yOmroi88O9c0ler20/ec/1P/9WTwtLaBSEXHdtcTcfz8q4/CKrj0aXz7UXQ4PB747TN6qCiyt0kT4IL2anJkJjD0vifC4k7s1SxrMvL2lnOU7quhwuAFICNNx5+wRXDU1heAg3zmPhr0g2lph3dNSMm7RA8FRMP93UuDMEEeP+qLt9jft5+5Vd9PqaGVa/DRemvvSsJsi1hNZEIchdredR9Y90l1L7J/z/8mE2AlD+pu+ejB5WltpePElWj74ALxeVFFRxD76KGGXLEIxDMPPwT8PdY/bS0leA/nrqqgpbuvenzomktzzk0kdE4WyhzvV4xVZZ6rnzc3lbDjY0L1/SloEN85KZ+GYeILUvm//YSuIXi/s/QBW/RYsDaBQwtTbYc7/gd439TCHuu3y6vO4d/W9mF1mZifP5u/n/R2dOjBSxZ0usiAOU1weF7/c+EtWla9Cr9bz0tyXmJ4wfch+z9cPJntBAbWL/4Bt1y4A9BMnEv/Eb9CNHvre8GDj74d6Q0UH+eurOLitDo9LmiMYGq1j7HnJJE6IYkVBHW9vKaei2QpIgTKXTkjihplpjE3yXeRqX/i77U6Lmj3wxaNQtU3aTpkhFe9NGOdTM4ay7bbUbOGBNQ9gc9tYkLaAv5z7l2FTteJEyII4jHF73Ty56UlWFK8gSBnEc3OeY3by7CH5LX88mERRpO3zz6n/29/xNDaCUknE1VcRfc89qGNifGrLQAiUh7rd7KJgUw3566voaLID4ELkQJCHPK0bXYyOG2ak8dMpKYQHB0ZNy0Bpu37RUQcbnoEdr4PoBUMsXLAYxl3ll6oUQ9V26yvX88i6R3B6nSzKXMTvZ/0etfLMiME8kSAOT//UWYRaqWbxOYu5SrgKp9fJg2se5Juyb/xt1qChUCgIv/RSMr/6ksibbgSFgpb33qdo3nxqn1qMq7r65F8i041Kr6IqTs1/k0U+MTgoVXvQoGC8U83NHToeUoQxJzgEo/bMeLj5jNYK+OLn8HwubF8KKGDGz+D+HTD+6mFVoulkfF32NQ+tfQin18lVwlUsPmfxGSOGJ+Ps+FcOc5QKJY9Pf5xgdTDL9i/jsQ2Pkd+Qz70T7h3Wg9s9URmNxP3614RdfjmNL71Ex6rVtLz3Hi0ffUTYxRcTdecdaDMy/G1mwFLfbue9bRW8t7WC+s6KE8YQNcbzE5mdFUtrfguFm2qoLW6jtriN4LAgxpybxJhzEzGEyVMqjkvjIfjuOdj7oTSfECD7xzDncYgbfq79E2F1WXll9yu8XfA2XtHLLWNv4eFJD59V06NkQRwmKBQKHp78MMYgI//Y/Q/ePPAmq8pX8fiMx4fMheoPdKNGkfzSSzgOHaLxtSW0f/EFbZ9+SttnnxH6w4VE3XUXOmF450scLERRZGd5C29uLuer/BrcnVltRsWFcOPMdC6bmIShqyeYE8P0RSM4uLWW/PXVNB+2sP1/pez8sozMSTHkzkkhfkToWfXwOyG1+VLe0f2fIeXRU0LuT+HcRyB2mLh3T4ENVRv4w5Y/UGOpQalQ8sDEB7g99/az7nqQxxBPhLUZVEGg9V8eyb7Y37if323+HYXNhQBcmH4hv5r2qwGnewvEsRxnRQVNS5bS+tln4JLm3oXMmUP03XehHz/ez9YdwZdt19Dh4PPd1SzfWUVhbQcAKqWCC0bHcePMdGaMiDzhg0wURQ4fbCV/XRUlexoRO4U0KslA9swERk2LJzjUd+OLAXXdVW6HjX+Dg19L20oNTLgWfvAQRI7wr219MNC2a7A28PS2p1lZvhKAnMgcnpz5JGOixwyWiYOCx2wBtwtV+MDzPMtBNaeD1wvP5khZ6a9bDsmBVd/L7XXzbsG7vLz7ZWxuG0aNkYenPMzlIy9HqTi9oeGAejAdhaumhqbXl9H68ceIdilYJHjmDKLvupvg6dP8/iY71G3ndHtZU1jH8p1VrDU14OkSMUMQ10xL5drpqSSGn3pljI5mO/s3VnPgu8PYOqQXDoVSqt2YPTOe9NxoVEM8FcPv150oQukGSQhLN0j71Hopw8ys+yEscPN0nm7beUUvyw8u5/mdz9Ph6kCv1vOzCT/jupzrAm680LZnD5V33Y1CoyFr/boBT82SBfF0+fhm2P8paAxw1duQNW9QbBtMqs3V/HHLH9lYvRGAibETeXLmk6eV3cbvD6Z+4G5qovnNt2h59128FgsA+gkTiLr7LkLOO89vwjgUbSeKIvsPt7N8ZxWf766mxSoJllqpYE52LFdMTmaOEDsocwc9bi/l+U0UbK6hfF9Td69RZ9Awaloc2TMTiE4JGZL29dt1J4pw8BtJCKu2S/u0odJcwhn3QkjgRzmfTtsVtRTx1JanyKvPA+DcpHN5fMbjAZmg27zxO6oeeADRZiP0Rz8k6dlnB/ydsiCeLh4XfH6fNPlWqYHL/gW5VwyOgYOIKIp8U/4NT299miZ7E2qlmlvH3sqd4+5Eq+p/wMRwEMQuPG1tNL/7Li1vvoWnTZqUrs3OJvquOzFecAEKlW8TKA9m2zWaHXyW19slCpAdb+SKyclcOjGJ6JChC4Sxtjs5uK2Wws01NFVbuvdHJYWQPTN+0F2qPr/uvB6pHNPGZ6EuX9qnj5REcNodoPdP+bXT4VTazuFx8OqeV1m2fxlur5tofTS/nPZLLky70O8elr5o+98XHP71r8HlIuySS0j4w2IUmoHPg5QFcSB4vbDqCdj8D0ABP/wLTL9r4N87BLQ723l+5/N8fPBjANJC03hixhP9nsw/nASxC6/FQsuHH9G07HU8DY0ABGVkEHXnnYT9+KJBuYH6w0DbTnKJ1rN8ZxXrTPXdATIRwRoumZDEFZOTGZPo26AXURRprDRTuLmGg9vqsFukHqpSqSB1bBQ5MxNIy40asEvVZ9edxwV7P5KiRpsOSftC4iW36OSbAy5WoD/0t+221mzlqc1PUdFRAcCVo67kockPBWw+0ua33qbuT38CIPLWW4l99OeDlsVKFsTB4PsXpBRNALMfk9IzBeBbFcCuul08tfkpituKAViUuYhHpzxKhO7E6aSGoyB24XU4aPvPf2hashTX4cMAaJKSiLzlFsIuWTTkuVJPp+2O5xJVKRXMESSX6NzswXGJDpShdKkO+XVnb4M9H0qV6tskQSAsVQqUmXAdaIZvKrKTtV2LvYW/7fgbK4pXAJAZlsmTs55kYuxEX5l4SoiiSMOLL9L0z38BEPuLR4m67bZB/Q1ZEAeLvHdgxf1ShorJt8BFfw/Y2mYuj4vX973Oa3tfw+l1Eq4N5xdTf8HFIy4+7kNrOAtiF6LLRdv/vqDptddwlpYCoNDpCF24kPArr0A/aZLfx8FO5hK9ZEJSQJdb6nKpFmyqofnwwF2qQ3LdiSJUbIFdb0pTJ9y2TiNHSlMncq+EMyAN2fHaThRFVhSv4G87/karo5UgZRB3jb+LW8bcErDp10SPh9rfP0XrRx+BSkXCU08RfvlPBv13ZEEcTAq/hOW3gNsOOYvg8qXjaP/mAAAgAElEQVSgDtyHV1lbGYu3LGZbrZRzcXrCdH4747d91lk8EwSxC9HjkSb3v/8+1q1bu/cHZWYSfsUVhF2yCHXk4NVyO1nbtVqdrNxfx//ya9hU1NjtEg0P1nCpn1yiA6XLpVqwuYaD22pxWKSJ60qlguTsCDInxzJiQgw6w4kfwIN63VkaYc/7sOstaDx4ZH/GbJhyG+RcHLAvsadDX21X3l7O4s2L2VorXffT46fzxMwnSAtN84eJ/cLrcHD40V/QsWoVCq2WpOeexTh37pD8liyIg035JnjvanC0Qfq5cPV7oAtMXzwc+7aoVWm5a9xd3Dzm5l5vi2eSIPbEWV5O6/JPaP30UylfKoBGg3H+PCKuvJLgGTMGPD7RV9t1ieAX+TV830MEJZdojBQlmh2LVj38H9Ael5eyfY0Ubq7t5VJVKhUk50SQOen44jjg687rhdJ1sPNNKPwCvJLrGUMsTLwOJt4AUb6pKeprerady+Ni2f5lvLrn1X57hQIBj9lM1b0/w7ptG0qjkZR/vkLwlD71alCQBXEoqN0H7/wEzHUQPw6u/wRCYofu9waBo8cTssKzeHz640yJl66NM1UQuxBdLszr19Py8cdYNn4nPUgBTXIy4VdcTthll6GJizut7+5quzari28O1PLF3mNFcFZmFD/KTeDCMfFEGgIjsfZQYDM7KclroGhnPdUHW48Rx6zJsWSMPyKOp33dtR+GvHch7y0p1yhIGWWyFsCkG2HUhWeEW/REdLXd9trt/GnrnyhqLQL6Hzfgb9yNjVTceSeOAwWoY2JIWbpkyDNRyYI4VLSUwduXQXOJlMXihk8hIn1of3MQ2FKzhcWbF3dHnI2LGceNo28kyZrE2NFj/Wydb3DV1ND6n//Q+sknuA/XSDuVSkLOO4/wK68gZPZsFOr+TVBus7p4Y3UeeU3w3aHeIjhzRBQXjTvzRfB42DqclOzuQxxVCpKzI8maHINL38K4if3MjOJxw6GV0tjgoZXSeD5AWIrUE5x4HYQlD9G/JrBwe928tfktvm37lr0NewFINaby25m/HdIycYOFs7KSittux1VRgSYtldR//5ug5KH/v5MFcSgx18M7l0PtXgiJg+v/A/GBLyp2t51l+5bxTsE7tDvbAYgJiuHmcTfzk5E/ISRo+IWgnw6ix4Nl02ZaP/6YjjVrwC2Ng6ljYwn7yWWEX3FFnzdpm9XFygO13e5Ql6e3CEo9wTiihnC+4HCjlziaWuh69CiUkJIT1dlzjO57zLG5FPLelnqE5lppn1INwo9g8k0wYs4ZNTZ4IsxOM/859B/eLXiXwxYpojo0KJTrc67nlrG3DIsCvvbCQiruuANPQyO60aNJWfIa6qgon/y2LIhDjb0dPrgWyjaCNgyu/QDSZvnmtweI1WVlRfEK3j7wdneP0aAxcPnIy7ku5zoSQxL9bKHvcDc20vb557R+vBxnWVn3fsOsmYRfeSXembNZVdTMl/k1fNdDBJUKGB+v58oZWbII9pOe4lhlaoHOx5BSpSAlJ5LMSbFkjA1FV/G1FCBTsu7IH0dlSS7R8dcOi2wyg0W1uZr3Ct7jk0OfYHFJ0b0J2gRum3AbF2dePGwq31i3b6fynnvxms0Ez5hB8j9eQhXiuxdwWRB9gcsO/7kDClaAWgdXLIPsH/nu9weIV/Ty7uZ3WdO+hh11OwCp7NSCtAXcOPpGxsX4thK4PxFFEduOHbR8/DHt36wEh1ROyarWsjV+NN8n5LIrXmCSkMhFuYlcOCaO+sqSM3r8dSjZs3MfKksExbt69xyVuEkO2kOGbjtpwfswjvsBTLpJetkM4CCRwWZPwx7e2v8WqytW4+10EU+Nn8qNo28kpiOGMaMDKxH3iej49luqH34E0enEeOGFJP71GZRBvh1KkAXRV3g9UhHRnctAoYJFL8LE631rwwDoGqA/0HSAtw+8zdelX+MWJRfihJgJ3DD6Buamzg245L+DiccrsruyhW8L6llTWE9VRR1zqnZxQfl2stp6FCvW6jDO/gHGBQsImTOHg1VVsiCeJqY92xAohYIVWE3bKLFMoNg+i2rnWESOuEGjkkJIz40ifVw0semhKJVnrii6vW7WVKzhrQNvsadhDwBqhZqFGQu5YfQNjI6SajEOp0C41k8+oeaJ34LXS/hVVxH/2yd8nmIRZEH0LaIIa/8EG56Rtuf/Hs55cFi80R59c9VZ6ni/8H0+OvgRHU5pAnlSSBLX5VzHZVmXnTHjjG02FxsONrC2sJ61pvrujDEAhiAV546MYW5OLOeFOFB/v572lSux79l75As0GsjNJf6ySzHOmzeo8xvPWCyN0hSJghWIJetQdBXfBUieCjkXY035EeVVBsrym6g80IzL4ek+RReiIW1MFGm5UaSOjkQbfGZEk5qdZj4t+pR3C96l2iy9gBmDjPx01E+5Jvsa4gy9o6CHgyCKokjT0qU0/F1KzB19771E33+f36aCyILoD7a+Cl/9EhBh5n2wYDEMUi6+oeJ4N5fVZeXz4s95+8DbVHZUAhCiCekeZ0wISfC1qQNCFEWKGyysLazn28I6tpe1dJdTAkiNDGZeTixzs2OZlhHZ5zxBV20tHatW07FqFdYdO7qncKBUEjx1KsYFCzAumH/a0zjOSNoPQ8H/pGGF8u+7I0RFhRJF2jlSoovsi/ost+RxeTl8qJWy/EbK8htpb7R3H1MoFSRmhZGWG016bhThccEBPe+uLw6bD3ePD5pdZkCKGL1+9PVcknnJcccHA10QRa+X+mf+SvMbb4BCQdzjjxN5/XV+tUkWRH+Rvxw+vVuaKDz+Glj0UkDPizrZzeXxelhftZ63DrzFzrqdAKgUqu5xxtyYXF+Zeso43V62lTbzbWEdawrrKW+ydh9TKRVMTY9gXnYcc7JjyYwxnNID1d3UxKF33sWwbx+WLVu6CxmDVJrKuGABxgsv8ElIecDRXAoF/5VEsKvEEkjVY0acBzmLOKjMYtTEc/r9laIo0lpnpWxvE+X7Gjlc1NY9nQMgNEYvuVZzo0kcGT7k9RwHwt6Gvbx14C1Wl6/GI0o94Mlxk7lx9I2cl3weqpNEzgayIIouFzW/+Q1tn68AjYbEp/9M2EUX+dssWRD9StG38OEN4LJIcxVn/kyKjgsKvIiwU7m59jfu560Db/FN2TfdN/LIiJEsSF3AvLR5jAwf6fe39Pp2O+sONrCmoJ6NhxqwOI+43CKCNcwRYpmTHcvsUTGE6Qf2otLVdp72dszr1tG+ciWWjd8hdgbkAGhH5xC6YAHGCy4gaMQIv7fPkCCKUsq0Ayug4HOozT9yTK2DrPlST3DUhd1llgb6UHdYXVQcaKYsv5GKfc3dVTkANDoVKTmRpOdGkTomCkOYfyOARVHkUOshVpevZnXFag61SFU31Ao1F6RfwI2jbzylavWBKIhem422zz6j6Y03cJVXoAgOJvnFFwn5Qf9feoYSWRD9TdVOKf9pa7m0rY+UipBOuyOgstuczs1Va6nlvcL3WH5wefc4I0ilp+anzmdB2gJGR432ycO/1epkS0kTm4qlpaje3Ot4dryRudmxzMuJZUJKBKpBDMroq+28VivmDRvpWLkS87p1eK1HeqXqhAQMM2dimDkDw4wZqGOG8fSBjjooXS9NjShZD+1VR44FGSXxy7kYRi6AIMMxfz6YD3WvV6SutJ2y/EbK8xt71XMEiIgPJlmIIDk7ksRR4SfNszoYiKLI/qb93SJY3l7efcwYZOSKUVdwbfa1xBviT/m7A0kQ3Y2NtLz3Hi3vvY+ntRWQskAlPft39OMCJ0pdFsRAwOOW3EabXoLDu6R9Ki2Mv0oaY4wZ2nRF/WEgN5fL42JLzRZWV6xmTcUaWh2t3ccSDAnMS53HgrQFTIidgFIxOC4si8PN9rLmTgFsZP/hdnpezsFBKqZnRDI3J4652bEkhesH5Xf74mRt53U4sGzaRMeq1ZjXrsXT0tLruHZkFsEzZmKYOZPgaVN9Oi/rlLG3S2OAXQLYUND7eHAUjPqhJIIjzj9peaWhfKh3NNspz2+kLL+J6kOtuHsE5qCAmBQjSUIEydkRJGSGEaQbnAhqj9fDnoY9rCpfxbcV31Jjqek+FqGNYG7qXOalzmNGwowBVZ8IBEF0FBfT/MYbtH2+AtHpBECXm0vUbbdinD+/3xmffIUsiIGEKELFZkkYTV/RPSN51EKpUGnaOX6LSB2sm8vtdbOrblf3w6DB1tB9LFofzdyUucxPm8+U+ClolP1/GDjcHvIqWiUBLGpkd2Vrd5o0gCCVkklp4czKjGZWZhTjU8LRqHwzfnQqbSd6vThMJiybt2DZvBnrjh2INtuRE1Qq9Lm5BM+cgWHmTPQTJvh8rlYv3A6o3HakF1i9C8QewqIJlq7bEedJAhg75pQCyHz1UPe4vdSXtVNlaqGqsIXa0ja87iPXj1KpIC4jlKTsCJKFCOIzwlBp+v/vcHld7Kjdwery1aypXEOjrbH7WKw+lrmpc1mQtoBJcZMGbeqSvwRRFEWs27bT/PrrmNevl3YqFITMnUvULTejnzw5YIcEZEEMVBoPweaXpXI17s6ouYQJkjCOvhRUvn2zGoqbyyt62duwt9td1BVKDhCmDWNOyhzmp85nZuJMglS9H/puj5d9h9vZVNzI5uImtpc1Y3d5u48rFZCbHM45mVHMyoxmcloE+iD/pO8aSNuJTie2vXuxbNqMZcsWbHv2gOeI4Ch0OoInT8YwaybBM2agy8kZtOrhfeL1SqkIuwSwfPOReoIgpUxLmnJEAJOmgPr0BdtfD3WX00NtcRtVhS1UmVpoKO/tYVBrlCRkhUk9SCGSmNQQlEe9YDk9TjYf3syq8lWsq1pHm6Ot+1hSSBLzU+czP20+42LGDZpnpCe+bjvR5aL9m5U0L1uGff9+ABRaLWGXXkrkTTehHZHhM1tOF1kQAx1LI2xfCtteA2uTtC8sBWbcI6Wo0g5ttfcuhvrmEkWRguaCbnEsbSvtPmbQGDg3aTbZxnNwm0exvdTC1pImOhzuXt+RHW9kZmYU52RGM21EJKG6wIjaHcy285gtWHdsx7p5M5bNW3AcPNjruCo8nODp0zHMnEHwtOkEZaQP7G1cFKGpCEo3SCJYugFsvV26xI6RxG/EeVKmmEG8JgPB7QfgsLk5fFASx2pTyzHjj0F6NYkjw4kdaaDGWMx39m/ZcHh9dxo1gIywjO6x8+zI7CHvJfmsd2020/rxcprffqs7Gb4qIoKI664j4tprhtXcW1kQhwsum9Rb3PQPaC6W9mlDYfLNMP3uPudnDSa+fjBtqz7ARwe+ZFv9elrcZd37Ra8ajy0Fjy2daLXArOTJnJeVyszMKKIDNE/oULadu7ERy5atWLZsxrppM67Dh3sdV4aFoR87Ft24XPS549CPy0UdHX38L+yok8axq3ceWextvc8JSz3SA8yYPaTBX4EiiEdjbXdSfbCF0gN1lBc04jzqHcGldNJgqMAd00H6qDjOmzyN0SmjfGrjULedq7aW5rffpvXDj/CapSC1oPR0Im+5hbBLFqHUBX4i8aORBXG44fXCwa+lccaKTdI+pRrGXi4F4CQMTcTWUN5cLo+XA4fbyatoYVdFK7sqWqhqOeKGU2iaUBv3ERJZgFtT1utvFSgYGTGSibETmRQ7iUlxk04rIm8o8dVDXRRFXJWV3e5V684deBoajzlPnZggieNoAV1CEHpDC8qmvdL4X1vlsV8cEg+pM46IYESGz8ayA00Qay217Krbxa76XeTV53Go5RAiIiGOCJLaRpLYPpJUq4DeEnbM3xqjdMSPCCN+RChxGWFEp4SgGsJx7KFqO3tBAU3LltH+5VfdFWCCp0wh8tZbCDn//KF12Q8xsiAOZ6p2wuaX4MDnR2q/pZ8rhbKn/0AqTjxIZW8G8+aqb7ezq1P88ipa2FvVhsPt7XWOIUjF+JRwJqVGMDE1nImpEUQagmi1t7K7Ybf0QKrLY1/TPtze3q7TBENCt0BOjJtIVnjWkIzR9Bd/Bje46+qw7d2Lfc8ebLu2Yi84hNfu7H2iQkQb6kYX6UQfp0KfMxLthBkoUqdC0mQI9V9VE38Kolf0UtRaRF5dXrcA9owIBVAr1YyNGsvEOOl6mxAzgXBdODazk7qSdmpL2qgtbae+rL1XejkAlUZJbJqR+Iww4kaEEj8ibFDnQg5W24keD/aCQqzbtmFetw7rtm3SAaUS44UXEHXLLQE1dWIgyIJ4JtBSBlv+JZXC6TFmgTZMGtNJ/0GnQOaetkCe7s1ldbox1XaQ19nzy6topbrVdsx5I6INTEyNYFJaOBNTIhDijf2aC2h329nftJ+8+jx21e1id/1uOlwdvc4xaoxMiJ3ApLhJTIydyNjosWhVvnOv+vyh7nFJhalr84+4PWv2gNuO6AVnhxpbkwZbixZ7Wyj2Rg94e9/rCq0W3ejR6Mflossdhy4nm6DUVBQa347L+rLt7G47+xr3SddS/S721O856bU0JmpMv2oMer0izYfN1Ja0U9cpkq111mPOM0bqunuQcSNCiUoMQaP17T0rejzYCwuxbtuOdds2rDt24O040g6K4GDCr7icyBtvPOMyLMmCeCZhawHT11D2nVR/sbW893FdmBQC3yWQcWP7LZAnnUvnFalutXGgpp3Cmg4Ka9sprO2grMnC0ZeRIUjFhFSp9zcpNYIJKeFEDFLF+P681WuUGkZHjSY3Opes8CwywzPJCMsgTHusm2swGLKHussmRSM3HoSGQmgwSUtzMRzVawYgMlPq8XUt8bmg0eF1OHAUFGDbm48tPx/73r04y8uP/Xu1mqC0NLSZmQRljkCbmYU2cwRBGRlDNl40VG3X5mijtK2U4tZiilqLyG/MZ3/T/mO8DYmGREkAh8DbYDe7qC1to65U6knWlR7bi0QBYTF6opNCiEoOISophOjkEIyROhQneWHsb9uJHo801WfbNqxbjxVAkCbRB0+bRvC0qRjnzEEVNjT3ir+RBfFMprWiUxy7BLKi93FdeB8C2ffN3vPm6rC7MNV2UFDbQWGNJHym2g7MjmMfwmqlghExBsYlH3F/jorrX+9vsKi11Hb3IPPq8zjYchCRY6/tGH0MI8JHMCJsBJlhmYwIH0FmeCaRuoFFyQ34oW5vl4SvoRAaO0WvoRBayqGPfwcoIDwVYkd3it8kSJwIwf3/d3haW7Ht2489fy+2vfk4Dh3CVV3NMW83AAoFmpQUtCNGoM3KJGhEZvdaFXJs9plTYaBt12xvpri1mJLWEorbpHVJW0mv+a9dKFAwKmKU5G7v7AH6cjxa6kVaqCtto7akjfryDlprrXi9x7a5RqciKlESyegkA1FJklgG6Y9Mxzpe23XNdbVu24alSwDb23t/f1KSJIDTp2GYOhVN0tAG7QUKsiCeTbSUS1lESjdKAnl0AEWXQGacC+k/wBMzmrJmG4U1HXy3r4QGl5bC2vZeAS89iQ7RkpNgJCchlOx4I9nxoWTGGvqsCOFPOpwd7K7fjanFRHFrMcWtxZS2lWL32Ps8P0IbcUQowzO71zH6mH6Fzvfroe5xg7lWemlp6CF6jQehvbrvv1GoICoTokdBTHbnMgqiRg5JPlyvzYaztBRHcTGO4mKcxcU4ikuk3qTH0+ffqOPjj+lRapKSUMfE9CtLSX/aThRFGmwNkvC1lfT6P21xtPT5N3q1nvTQdDLDM8kMz0SIEJgQOwFjkG+mMfUXj8tLS52FxiozTVVmmqrNNFZbsLU7+zzfGKXr7kXaaWX89GyMUVpcRYewbt2KZdt2SQDbekcOaxITCZ4+XRLBqVMJSj47BPBoZEE8m2kpx1G0HvuhDQRVfY/e2jtk3y5qqBRjqehcuj7XKOLRxmaQkRBLToIkfEK8kRhjYE576A9e0UuNpeaY3kRxW3GvuWQ9MWqMZIRnkBmWSVJIErHBsb2W0KBQFAoFBQcOkJMWJ72AtFdDW9WRpWu7o+ZIYNTRqLQQPVJK4ReTfUQAI0cMaNL7YCE6nTgrKnAUFeMoLsJZXCIJZmlpd7quY1AqUcfFoYmPR5OQgDohHk18AprEBNTx8WgSE1GFh1NYWEh2djbtznbqrfW9lmpzNcVtxZS2lh4z1teFQWM40tvv0etPMCT4NdBqoFjbnTRVSwLZVGWmsbKd5lor3j7eS5QeJ8G2enS2RvT2JnT2JgzBIlHZKURPHUPorGlnrQAeTcALoiAIC4EXABWw1GQyPX3UcS3wFjAZaAKuMplMZSf6zsEQxNc2FLP/cDsJYXoSw3UkhulJCNeRFK4nTK8JmNREDreH6hYblS02qlqsVDbbqGyxUtVsparFRpPlyAMrWdHATOV+ZigLmK4sIFlxbMh+LwyxEJHex5IGxsSAr/HYH0RRpM5a1+1q6xLKotYi2p3tJ/xbLQpivCKxTiexbhexHg+xbk+vdYzHg04UAQWExEFY8rHiF5E+aNHCvkT0eHBVVeEoLpGEsqgYR1kp7sM1uBuOdVkejUujpDlUSYPRS0OoSKMRmkIVNIZK62Yj2IIAhYLQoFCywrN6Cd+IsBHEBccFzL04EESvF3ddHc7KSlyVVbiqq3BWVuGqrMRZVYWnsRGvQolNH4s5JAmzIQlzSCJmQxKOk7j89UYNxkgdodF6QqN1GKP0hEZJ28ZI3SmlqBtKRFHEYXHT0WLH3OLA3GzH3GKno9lBTKqRiQtSB/wbJxJEv2ddFQRBBbwMLACqgO2CIKwwmUwHepx2G9BiMpmyBEG4GvgLcNVQ2uX1iry8tpg2m6vP43qNqlscE8J0JITppc/hum4BDQ46teYVRRGr04PF6cbi8GBxuKWl57bTQ6vVSVWLjcpOwavrsPc57NNFkEpJcoSepAg9KZGppEScS1CEnsbIYIwhXsIch6GljDrTVuI0NimitaVMCtix1EtL1bZjv1gVJI1jRaRDeJqU1FkbAkEhUiaToJC+tzUG3wup1wvODrC1gr2111phbyXe1kq8vZVZPfaLdgtNDgslXislahU1ahUNKjX1ahX1KhX1ahUWpZIqJVTpgoDj9+RCNSHEBscRZ4gnWh9NSFAIwWoVBudhDA1tGFoPEKwJxqAxYFAbMGgM3dvB6uCT1sUbSjxeD1a3FYvLgtUlrS1uS+/tGAuWcBvW8eGYnSNpsIXT1B6Jq64WbZOZ6HaR6HaIOmptcHiJa/IS19TzF3tfzKJSiTI0BHVoCKowG6rQSlRhbShDi1CGhtEcFooyNBRVaBiqsFBUoaEoOz8rDQafz5kTvV68Vhtei6X3YrUcs8/d0oKrqhpXZSWu6mpEV9/PGwCFRoMuORljSjLJySloUlKoV0DGzFw8EfGYLQram2x0NNlpb7TR3mSXPjfZsHW4sHW4qC/vo5etAEOYltAoHcZoHaFRenQGDRqtCo1OhUarIkinQqNV99qn0apO+UXE5fBgbrFjbnZIotcsCV9H59rcYsft7NuLUnGgiQnzUk4aaDQQ/C6IwDSgyGQylQAIgvABcAnQUxAvAX7X+Xk58A9BEBQmk2nIurdKpYL/3f8DtpY2c7jVRk2bjcOt9s7PdswONyUNFkoa+na1AYQHazqFUkdcqA6vKGJ2eLA63Jg7hc7q8GB2uLuF8HQ67CqlgoRwHSkRwaRE6knuXKdEBJMcEUysUYvyhBdRJMSPpZlM4nr2qr1eyc3XJZBHL5Z6KeVXU9EpWqyQygAdVzCDJdei1yMlke65PmafW7LzePtcVkn8HO3Hd1ce30qiO5dpGgMYoyAkSUqrF5YEYclYDNHUB+nZWVONNj6Cems9DbaGXm6/BmsD7S4z7W1mitqKT7GtJPRqPcHq4F4iadAY+jUdoL/Y3XZJ5DrFz+KyYHPbsLn7Hk/uFwZQh2hoDY6hPTgWc3AstuBY7PoYXMGxxBGKbX8lY4xxqBpacNXU4K6pxVVTg6u2BndDI1itiK1tuFrbOL5cHAelEqXRKImkTgcqlSSQarW07tpWqVCoVKBSolCq+tjuXKtVKBRKvLY+BK9rsdn6DkzqB6roaIKSk9GkpBCUkowmuXOdkoI6NvYYca8vKEAnCF1NTVxG6DHfKXpFLG1OOpokkTwiljbaGyUhsrRKS01x2zF/f1wUdAtjkE7d43OnYOrUKJQKLK1dgmfHYekjMvooNDoVxkgdIRFaQiKktTFSR+LI8CEVQwgAl6kgCFcAC00m0+2d2zcA000m0309ztnXeU5V53Zx5znH9fft3r1b1GqHbrzL4vTSYHFTb3HTaHHT0LVYj3x2n9rzFwCtSoFeo0SvUaBXK/v8bAhSEmtQEx+iId6oJjpYPSgRnXa7Hd0phNYr3DaCzIfRWKRF5exA6baidFmltdty5LPLiqpr+ziBLUONRx2MN8iIR2PEE2TEGxSKp6/tICMeTah0budxTlKi50Rt5xW9dLg7aHY20+JqodXVitVjxe6xY/PYsHls2L3S5+59Xumz3WPH7rX3GTHrKxQo0Cl16FTSolfq0av00meVHp1SWnftC1YFE64JJ0ITQWRQJEa18YRjeSe97lwusFikxWwGswUsPdd97es81+6faw2dDvQ60OlBf/SiO/LZEAJxsRAXB7Gx0t+dAqd6z/aF1yviMHuwt3mwtXuwt3twO714nCIel4jHKeJ2Hdl2O0U8Lm+fs35OhkIFuhAVWqPqyNqoRBuiQmeUFrV2aHv0Vqs1cF2mQ4VWq/VrOiivV6TJ4uzsWdqoa3egUioI0aoxaNUYglTSWqvGoO38HDQ4wna6nF74+6RT/yGvB5xmcJh7rDuObLus0p2jVPVYK6X0db32da5PtE+tkyqz68JQqTSogKGYdj6Uk8u9ore799blruxyVTo8jkH7Ha1Ke8RV2+m27eqFDmVwylC2nehy4enowNPWJgX/eDyIHi943Ihe75FtrwfRIy14vdLa40X0uFMFT2EAAAWQSURBVHtve6W1Uq9DaTD0XkJCpLVeL/UufYBfs/x4RVwODy67G5fDg9N+1GeHB69HxBAe1Nnj06EP0Qx5L+9k7Ny587jHAkEQq4GUHtvJnfv6OqdKEAQ1EIYUXBOwKJUKYoxaYoxaxiWH+9ucwEKpkhII6M7Mib+DjVKhJFgjuUtjiPG3OcMKhUaDOjJyWFVjGC4olQq0ejVafSDIyOAQCP+S7cBIQRAykITvauDao85ZAdwEbAauANYM5fihjIyMjMzZh99jbU0mkxu4D/gGKAA+MplM+wVBeEoQhEWdp/0biBIEoQh4BPiVf6yVkZGRkTlTCYQeIiaT6Uvgy6P2/bbHZztwpa/tkpGRkZE5e/B7D1FGRkZGRiYQkAVRRkZGRkYGWRBlZGRkZGQAWRBlZGRkZGQAWRBlZGRkZGQAWRBlZGRkZGQAWRBlZGRkZGSAAEjuPVTs3LmzASj3tx0yMjIyMgFF2uTJk/vMgXjGCqKMjIyMjMypILtMZWRkZGRkkAVRRkZGRkYGkAVRRkZGRkYGkAVRRkZGRkYGkAVRRkZGRkYGCJDyTzK+RRCEhcALgApYajKZnj7q+CPA7YAbaABuNZlM8hQWTt52Pc67HFgOTDWZTDt8aGLA0p+2EwThp8DvABHYYzKZji4WflbSj3s2FXgTCO8851edZfVkTgG5h3iWIQiCCngZ+CEwGrhGEITRR52WB0wxmUzjkB7qz/jWysCkn22HIAhG4EFgq28tDFz603aCIIwEfg2cYzKZxgAP+dzQAKSf191vkIqrTwSuBl7xrZVnBrIgnn1MA4pMJlOJyWRyAh8Al/Q8wWQyrTWZTNbOzS1Aso9tDFRO2nadLAb+Ath9aVyA05+2uwN42WQytQCYTKZ6H9sYqPSn7UQgtPNzGHDYh/adMciCePaRBFT22K7q3Hc8bgO+GlKLhg8nbTtBECYBKSaT6QtfGjYM6M91NwoYJQjC94IgbOl0E8r0r+1+B1wvCEIV8CVwv29MO7OQBVHmuAiCcD0wBfirv20ZDgiCoASeBX7ub1uGKWpgJHA+cA2wRBCEcL9aNHy4BnjDZDIlAz8C3u68HmVOAbnBzj6qgZQe28md+3ohCMJ84HFgkclkcvjItkDnZG1nBMYC6wRBKANmACsEQZjiKwMDmP5cd1XACpPJ5DKZTKXAQSSBPNvpT9vdBnwEYDKZNgM6INon1p1ByFGmZx/bgZGCIGQg3VRXA70i+QRBmAi8CiyUx3F6ccK2M5lMbfR4CAmCsA54VI4yBfpx3QGfIfV0lgmCEI3kQi3xqZWBSX/argKYB7whCEIOkiA2+NTKMwC5h3iWYTKZ3MB9wDdAAVJk2n5BEJ4SBGFR52l/BUKAjwVB2C0Iwgo/mRtQ9LPtZPqgn233DdAkCMIBYC3wC5PJ1OQfiwOHfrbdz4E7BEHYA7wP3GwymeTKDaeIXO1CRkZGRkYGuYcoIyMjIyPz/+3dIY4VQRSF4R8WgGUJZQgWjSBIUASBhBWggUlAIAl2PJmgCJKwCwy1BByToHmIhxoyupqe71N93XEnN+mqqhQiAFQKEQAqhQgAlUIEgEohAkClEAGgUogAULm6DXZpjHFSvapuV++rO9V5dVqdzDl/r0sH22RDhH37VH2tHlYfqhfVy6WJYKNsiLBvp3POt3+/v4wxblTPxxjv5pw/VwaDrbEhwr59vDCfdby4/daCLLBpChH27ccl88UX1+HKU4iwbzcvmf95FBquOoUI+/bowvy4+lV9W5AFNs1PNbBvz8YY1zu+un6/etrx2MX52liwPTZE2LcH1b3qc/WkelO9XpoINsqGCPv2fc55d3UI+B/YEAEghQgAVV07HA6rMwDAcjZEAEghAkClEAGgUogAUClEAKgUIgBU9QcyazQrpGcfhQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(7, 5))\n",
    "ps = np.linspace(0.05, 0.95, 20)\n",
    "qs = np.linspace(0.1, 0.9, 5)\n",
    "kls = [[kl_bernoulli(q, p) for p in ps] for q in qs]\n",
    "for i, q in enumerate(qs):\n",
    "    plt.plot(ps, kls[i], label=r\"$q = {:.1f}$\".format(q), linewidth=2)\n",
    "    \n",
    "plt.ylabel(r\"$KL(q || p)$\", fontsize=16)\n",
    "plt.xlabel(\"p\", fontsize=16)\n",
    "plt.legend(fontsize=14);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Inverting the KL\n",
    "\n",
    "\n",
    "For evaluating the bound, we can invert $KL$ numerically. However, in order to train the distribution $Q$ we need a way of getting the gradients. We will use the bound:\n",
    "\n",
    "$$KL(q \\vert \\vert p) \\ge 2(q - p)^2.$$\n",
    "\n",
    "So, if $KL(q \\vert \\vert p) \\le \\epsilon$, we have \n",
    "$$\n",
    "p \\le q + \\sqrt{\\frac \\epsilon 2}.\n",
    "$$\n",
    "Combining this expression with the McAllester bound, we get\n",
    "$$\n",
    "    e(Q) \\le \\hat e(Q, S_m) + \\sqrt{\n",
    "        \\frac{KL(Q \\vert \\vert P) + \\log \\frac m \\delta}{2(m-1)}\n",
    "    }\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAasAAAFCCAYAAABPWvInAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeVxVdf7H8RebLIIgq4oIinIEBNfU3FIz17LFqbT6ZU010+o0Wc00WdNmzbTXVNpe5rRn6ZS2aKWZWYpLKpeDKKAgsm8CFy7c8/vjAAKiINwVPs/Hg4ecc8+93w8n8u35nu/5fl00TUMIIYRwZK72LkAIIYRoi4SVEEIIhydhJYQQwuFJWAkhhHB4ElZCCCEcnoSVEEIIh+dur4b37NmjeXp62qt5p1RdXY2cs46Rc9dxcu46Ts7d2ausrCwYPXp0SMv9dgsrT09PYmNj7dW8UzIYDHLOOkjOXcfJues4OXdnLykpKbO1/dINKIQQwuFJWAkhhHB4ElZCCCEcnoSVEEIIhydhJYQQwuHZbTRgW8rKysjLy8NkMtm7FIdhMpkwGAz2LsMpWfrceXh4EBoaSq9evSz2mUKI03PIsCorKyM3N5fw8HC8vb1xcXGxd0kOoaqqCm9vb3uX4ZQsee40TaOqqors7GwACSwhbMAhuwHz8vIIDw/Hx8dHgko4HBcXF3x8fAgPDycvL8/e5QjRLThkWJlMJrmCEA7P29tbuqmFsBGHDCtArqiEw5PfUSF0a/dk8972VieesBiHvGclhBDCORRV1HDXx3vxcHPh/8ZHWq0dh72yEkII4fi+Sz5OnVnjnKhAq7YjYSWEEKLDvtp3HIB5CX2t2o6ElQMqLS1lwoQJHDly5Kze9+uvvzJ9+vQ297VlyZIlvPXWW2f1HiFE91NSWcO2tALcXF2YGd/Hqm1JWFnB4sWLWbp0abN9H3/8McOHD2fFihVomsbixYu59957W33/ypUrOe+88xgwYIAtyj3FbbfdxsqVKykvL7dL+23573//y/Tp00lISOCyyy5j586dbb7no48+OuN7Xn31VRYsWMCoUaMYP348N998M6mpqdb6EYToEr5LzqXWrDF+UCCBPXtYtS0JKytITk5m2LBhANTU1HD//ffzzDPP8NJLL3HLLbfg4uJCcnIy8fHxp7y3qqqKTz/9lAULFti67EaKotC/f3/WrVtntxpOZ/369Tz++OPcfPPNfPHFF4wcOZKbbrqJY8eOnfE9Tz311Bnf89tvv3HVVVfx4Ycf8u677+Lm5sb1119PSUmJLX4sIZzShv16F+CcYdbtAgQJK4s7cuQIZWVlxMfHc+zYMRYtWsSBAwf49NNPmTx58inHtLR582ZcXFwYPXr0Ka9lZmZy/fXXk5CQwIwZM9i8eTMJCQls27at3fUVFRWhKArvvPMOCxYsICEhgVmzZrF169Zmx02fPp0vv/zyrH729PT0TtfXlrfffptLL72UK664gujoaB544AFCQkL44IMPzvieiy666IzvefPNN1mwYAExMTEoisKTTz5JUVERu3btsljtQnQlZUYTPx3Mx9UFZlm5CxCcaOj69W//xg9qvk3bnKaE8Pb1Y8/qPfv378fFxYXS0lKWLFnClClTeOSRR/Dy8mp2jKura6sriO7cuZP4+PhTnuHJyMjg6quv5qqrruKhhx4iPT2dZcuWUVNTw9ChQ9tdX0pKCgCffPIJy5Yto0+fPjz33HMsXbqUzZs3N9aZmJjIypUrMRqNzWo/nYyMDC6//HIWLVrUZn0rV67k1VdfPePnvf7664wZM6bZvpqaGg4cOMAf//jHZvsnTpzI7t27W/2chvdcc8017X4PQEVFBWazWaZSEuI0NibnYqrTuwBD/Dyt3p7ThJWzOHDgAK6urvzlL39h6dKl3HDDDa0eExUVRc+ePU957dixY4SGhp6y/9FHH2Xq1KmN98IiIyOZPHkyW7duJTCw/UNGDQYDbm5uvPTSSwwcOBCAu+++mwsuuIDDhw8TFxcHQGhoKCaTiby8vHbdO3v00UeZPn16u+pbuHAhc+bMOePnhYWFnbKvuLiYuro6goODm+0PCgo67dVbw3uCgoLa/R6A5cuXExsby8iRI89YpxDd1fr6UYBzrTwKsIHThNXZXuHYS3JyMuPGjSMzM5Pk5OTTHtNaFyBAdXX1KX8Z5+TksHXr1lO6ujw8PM7qqgr0sJo2bVpjUAH4+vqeclzD1ZTRaGzzMxvqW7NmTbvqCwgIICAg4KzqtqUnnniCpKQkPvjgA9zc3OxdjhAOp9xoYsvBfFxcYLYNugBB7llZXHJyMhMnTmTlypX88MMPvPjii60e0zAAo6WAgABKS0tPOd7NzY3Bgwc326+qaqtdiWeSkpJyynt27dqFp6dnswBrqKE9V20N9Q0ZMqRd9a1cuZKRI0ee8au1EX69e/fGzc2NgoKCZvsLCwsJCQlptbaG9xQWFrbrPY8//jhfffUV7777LhEREW3+7EJ0R9+n5FFTa+acyEBCe7V9m8ASnObKyhkcPXqUkpIS4uPjiYmJ4ZlnnuG2224jKiqK+fPnNzumobutpbi4OD7//PNm+1xcXDCbzdTW1jbuS0pKYvfu3SxevLjd9VVXV5Oeno6mac32v/3228ybN6/Z5MGpqamEhYWdcpXXmqb19ejRo836OtoN2KNHD+Lj49m2bVuz92/bto2ZM2e2+jkN79m+fTsXX3zxGd/z2GOPsWHDBlatWkV0dPQZ6xOiO1u/LweAOQm2uaoCCSuLOnDgAEBjF9+0adNYunQp999/P/369WPMmDGNx7i7uzd7jsfNzY3o6GgmTZrE008/TXFxMb179278PA8PD1544QVuvPFGVFXl6aefBjirbsDU1FQ0TePLL7/k3HPPJTAwkBUrVpCZmcnzzz/f7NikpCQmTZrUrs9tqO+ZZ57huuuua7O+znQDXn/99dx7770kJiYyatQoPvjgA/Ly8li4cGHjMatXr2b16tV8/fXXje+55557GDVq1Gnf8/DDD7N27VpefvllevXqRX6+PpjHx8en1XuLQnRXFdW1/Fg/2G32MAkrp3TgwAEiIiKajSC74YYbSEtL4/bbb+fjjz9uDKtFixY1e29MTAz/+9//UBSFxMRE1q9fz9VXXw3oVxnLly/nmWeeYd26dUyYMIGrrrqK5557jsjI9k8cmZKSQmRkJLfffjt33XUXpaWljB8/nvfff79Zl1h1dTXfffcdb775ZrP3r1mzhvvuu49NmzbRv3//xv0N9T377LOsWbOmw/W1x9y5cykuLmbFihXk5eURExPDa6+9Rnh4eOMxxcXFpKenN3tPXl7eGd/z/vvvA3Ddddc1a+/222/njjvusOjPIIQz+0HNo7rWzKgBAfT1t+FSTpqm2eUrOTlZO50zvdYdbN68WZs5c6ZWW1vbbH9lZWXj9y+++KJ25ZVXNnt9+/bt2rRp00677+GHH9aWLFnSZvurV6/Wrr/++lP2v/DCC9rcuXM1k8nU5me0Vp89NT13ltQdfle7w89oLV3x3N26OkmL/NuX2utbDlnl83fu3LlTayUz5MrKAU2ZMoWMjAyOHz/e7F//TamqiqIoZ/W5BoOh8cHkM3F3d2fZsmWn7N+8eTMPPvgg7u5t/9p0pD4hhGOrqqnj+xR9dew5Nhqy3kBGAzqoa6+99rRBBa2P6jsTTdPaHSBXXnklgwYNOmX/Z599xrhx49rV3tnWJ4RwfD+qeVSZ6hgeEUB4gG1Xc5crKye1cePGU/aFh4dz7bXXtrrPxcXFplMHtVafEMK5ra+fC3CuDQdWNJArqy6kf//+pwwQaG2fEEKcLaOpju8NuYDtZq1oSsJKCCFEm7ak5lNRU0dCuD8RgT42b1/CSgghRJvs8SBwUxJWQgghzqi6to6NBn0U4FwbrF3VGgkrIYQQZ7T1YAEnqmuJ69uLqGD7zOgiYSWEEOKMTi4HYp8uQJCwEkIIcQY1tWa+S65fvt4OowAbSFgJIYQ4rZ8PFVBmrEUJ8yM65NS172ylzYeCFUWJAFYBYYAGvKaq6gstjpkKrAUaZg9do6rqI5YtVQghhK1tqB8FaI9nq5pqzwwWtcBSVVV3KYriByQpivKdqqotl8H9SVXVCy1forC3nJwc7r33XgoLC3Fzc+PWW29tcz0qIYTzM9WZ+Ta54UFg+92vgnaElaqqOUBO/ffliqIYgHCg9TXbRZfj5ubGP/7xD2JjY8nPz+eyyy7jvPPOw8fH9g8GCiFsZ/vhQkoqTQwO9WVImJ9dazmre1aKokQBI4FfW3n5XEVR9iqKskFRlHhLFNdVlZaWMmHCBI4cOWLvUlq1ZMkS3nrrrcbt0NDQxklpQ0JC6N27d+Oy90KIrqvhQWB7zAXYUrsnslUUxRf4DLhTVdWyFi/vAiJVVT2hKMpc4AtgyJk+r7q6GoPB0OprJpOJqqqq9pbmUN588002bdpEZmYmPXr0ICEhgSVLljB48ODGY1566SUmTpxISEjIWf2cmqbZ5LzceOON3HDDDVx44YX4+TX/11RycjK1tbUEBAQ41X8ja507k8l02t/jrsJoNHb5n9FanPnc1Zk1vtqbDcBQXwf4OVpb5KrlV0xMjEdMTMw3MTExd7Xz+IyYmJjgMx3TVRdf/OMf/6h9+umnmqqqWkpKinbrrbdqEyZM0IqLizVN0xcBHDNmjLZjx46z/mxrLSDYmksvvVRbvXp1s33FxcXa3LlztaSkJJvVYSmy+GLHdYef0Vqc+dz9fDBfi/zbl9q0p37QzGazzdo93eKLbXYDKoriArwJGFRVffY0x/SpPw5FUcaidy8WWjJUncWbb77JggULiImJQVEUnnzySYqKihqX59i8eTMuLi6MHj36lPemp6dz/fXXk5CQwIwZM9i8eTMJCQls27bNYvUVFRWhKArvvPMOCxYsICEhgVmzZrF169Zmx02fPp0vv/yycbumpobbbruNm266iVGjRlmsHiGEY1q//+RcgC4uLnaupn3dgBOB/wP2KYqyp37fP4ABAKqqrgT+ANyiKEotUAUsVFVVs0K9TqeiogKz2UyvXr0A2LlzJ/Hx8af8x8/IyODyyy9n0aJFPPTQQ6Snp7Ns2TJqamoYOnSoxepJSUkB4JNPPmHZsmX06dOH5557jqVLl7J582a8vLwASExMZOXKlRiNRjw9Pfn73//O+PHjueSSSyxWixDCMdWZNb7er48CnGOnuQBbas9owK3AGWNVVdWXgJcsVVRXsnz5cmJjYxk5ciQAx44dIzQ09JTjHn30UaZPn87SpUsBiIyMZPLkyWzdupXAwECL1WMwGHBzc+Oll15i4MCBANx9991ccMEFHD58mLi4OEAfVGEymcjLyyMvL4/169ejKErjoopPPvmkLFsvRBe1I6OIghPVRAb5EN+vl73LAZxppeD/Xg4Hv7Vtm0NmwtWfdPjtTzzxBElJSXzwwQe4ubkB+sCS4ODgZsfl5OSwdetW1qxZ02y/h4dHq1dVzz33HCtXrjxj26tWrWp1CXqDwcC0adMagwrA1/fUp9IbrrCMRiNjxoxpvCITQnR9DQ8CzxnW1yG6AMGZwsrJPP7446xfv553332XiIiIxv0BAQGnDPtOTk7Gzc2NIUOaD6BUVbXVwFm8eDHz588/Y/v9+vVrdX9KSgqzZ89utm/Xrl14eno2C7CGGi15VSeEcHxms8aG/fafuLYl5wmrTlzh2Npjjz3Ghg0bWLVqFdHR0c1ei4uL4/PPP2+2z8XFBbPZTG1tLT169AAgKSmJ3bt3s3jx4lM+PzAwsEMhUl1dTXp6OprW/Hbi22+/zbx58/D29m7cl5qaSlhY2ClXgUKIrm3XkWLyyqvp39ubhHB/e5fTSCaytbCHH36YNWvW8PTTT9OrVy/y8/PJz8+noqICgEmTJnHo0CGKi4sb3xMfH4+HhwfPPPMMR48eZePGjdx///0AFh1ckZqaiqZpfPnll+zcuZPDhw9zzz33kJmZyV133dXs2KSkJCZNmmSxtoUQzuGrJnMBOkoXIEhYWdz7779PRUUF1113HZMmTWr8apgRQlEUEhMTWb9+feN7wsLCWL58OZs2bWL+/Pl8/vnnXHXVVfj4+BAZGWmx2lJSUoiMjGTJkiXcddddXHrppZSVlfH+++8TEhLSeFx1dTXfffcdV1xxhcXaFkI4PrNZ4+v6LsA5DjBrRVPO0w3oJFRVbfOY2267jeXLl7Nw4cLGgRfz589vdh/qP//5D4qi4OpquX9PGAwGYmJimDdvHvPmzTvtcZ9++inDhw9nxIgRFmtbCOH49mSVkFNqpJ+/FyMiAuxdTjMSVnYwZcoUMjIyOH78OOHh4a0eo6qqxYeGGwwGJk+e3OZx7u7uLFu2zKJtCyEcX8MowNkONAqwgXQD2sm111572qACvcuuYfJYS9A0rd0BeOWVVzJo0CCLtS2EcHyapjUuXz8v0bG6AEGurBxWw8O3luLi4tI45ZMQQrT0e1Yp2SVVhPXyZGREb3uXcwq5shJCCHFyLsBhfXF1dawuQJCwEkKIbk/TNDbsc8xRgA0krIQQops7cKyMI0WVhPh5MibKMWetkbASQohurmFF4NnxfXBzwC5AkLASQohuTR8FeHLtKkclYSWEEN1YyvFyMgorCerZg7EO2gUIDhxWLSdbFcLRyO+o6AoaHgSeNawP7m4OGwmOGVYeHh5UVVXZuwwhzqiqqgoPDw97lyFEp6xvWA7EQVYEPh2HDKvQ0FCys7OprKyUf70Kh6NpGpWVlWRnZ7e66rMQziI1t5y0vBP09vFg3CDH7QIEB53BolcvfRnlY8eOYTKZ7FyN4zCZTPIv+Q6y9Lnz8PAgLCys8XdVCGfUMLBiZlwfPBy4CxAcNKxADyz5i6A5g8Fg0fkCuxM5d0KcquFB4LmJjt0FCA7aDSiEEMK60vJOoOaW4+/twYToIHuX0yYJKyGE6IYaugAviAtz+C5AkLASQohup7TKxNs/pwMwf3g/O1fTPhJWQgjRzbzyYxrFlSbGDQxk8pBge5fTLhJWQgjRjWQVV/L2zxkA3D8v1uFWBD4dCSshhOhGnv5GpabWzMUj+pHYP8De5bSbhJUQQnQTv2eV8MWeY/Rwd+XumYq9yzkrElZCCNENaJrG8q8MAFw/MYqIQB87V3R2JKyEEKIb2GjI49f0Inr7eHDr1MH2LuesSVgJIUQXZ6oz88QG/apqyflD8Pd2vmnbJKyEEKKL+3DHUQ7nVxAV5MPV4yLtXU6HSFgJIUQXVm408fx3qQD8fc5Qerg751/7zlm1EEKIdlm5+RCFFTWMiezNrHjHXba+LRJWQgjRRR0rqeKNn/RplZzpAeDWSFgJIUQX9fS3KtW1ZuYl9mXkgN72LqdT2lzPSlGUCGAVEAZowGuqqr7Q4hgX4AVgLlAJXKeq6i7LlyuEEKI99meX8vnubDzcXPjbrKH2LqfT2nNlVQssVVU1DhgP3KYoSlyLY+YAQ+q//gSssGiVQggh2k3TNB5fb0DTYPG5UQwIcq4HgFvTZlipqprTcJWkqmo5YADCWxx2MbBKVVVNVdXtQICiKI6/9KQQQnRBP6r5bDtUiL+3B7dPd74HgFtzVsvaK4oSBYwEfm3xUjhwtMl2Vv2+nNN9VnV1NQaD4Wya7/aMRqOcsw6Sc9dxcu46zh7nrs6s8c91WQBcGe9HTuah0/9F7ETaHVaKovgCnwF3qqpa1tmGPT09iY2N7ezHdCsGg0HOWQfJues4OXcdZ49z9/6vRzhSaiIi0Jull4zF093Npu13VlJSUqv72zUaUFEUD/Sg+q+qqmtaOSQbiGiy3b9+nxBCCBupqK7l2foHgP82e6jTBdWZtGc0oAvwJmBQVfXZ0xy2DrhdUZQPgXFAqaqqXeHKUwghnMarWw5TcKKaEREBzEvoWsMG2tMNOBH4P2Cfoih76vf9AxgAoKrqSmA9+rD1NPSh69dbvlQhhBCnc7zUyGtbDgGwzMkfAG5Nm2GlqupW4Iw/taqqGnCbpYoSQghxdp79TsVoMjM7vg9jogLtXY7FyQwWQgjh5Aw5ZXySlIW7qwt/m+P8DwC3RsJKCCGc3BMbUtA0uGZ8JAODe9q7HKuQsBJCCCe2OTWfLan5+Hm5s+T8IfYux2okrIQQwknVmTWeWK8/dHz7tMEE9uxh54qsR8JKCCGc1GdJWaQcLyc8wJvFE6LsXY5VSVgJIYQTqqyp5elvVQDuna3g5dF1HgBujYSVEEI4oTd+SievvJrE/v5clNjP3uVYnYSVEEI4mbxyIys36w8A/2NuLK6uXesB4NZIWAkhhJN57ruDVNbUMSM2jPGDguxdjk1IWAkhhBNJzS3nox1HcHN14e9d9AHg1khYCSGEE3livQGzBleNHcDgUF97l2MzElZCCOEkvj1wnB/UfHw93fnLjK77AHBrJKyEEMIJZBRUsPSTvQDcOWMIwb6edq7ItiSshBDCwVXV1HHz6iTKjbXMjAvjhkkD7V2SzUlYCSGEA9M0jWVf7CfleDkDg3vy9BXDu9xaVe0hYSWEEA7s/d+O8NmuLLw8XFlxzSh6eXnYuyS7kLASQggHtfdoCQ+vSwbgX5clMrRPLztXdBqaBmazVZuQsBJCCAdUVFHDLauTqKkz83/jI7lkZLi9S2pdfiqsnASvT9VDy0okrIQQwsHUmTX+8uFujpUaGRERwLILY+1dUutSvoLXp0PufnD3tmpT7lb9dCGEEGfthY2p/HSwgMCePXjl6lF4ujvYjOpmM2z+F2z+t74dfylc/DJYceCHhJUQQjiQ71NyefH7NFxd4D+LRtIvwLpXLGfNWApr/gypG8DFFc7/J0z8i1WDCiSshBDCYRwprOTOD/cAsHSmwsTBwXauqIX8VPjwKig8CF4B8Ie3YPD5NmlawkoIIRyA0aQ/+FtmrGVGbCi3nBdt75KaS1kPa/4ENeUQGg8LV0PgIJs1L2ElhBB2pmkaD3yxn+ScMiKDfHjmihGOs0aV2azfm9r8L3274f5Uj542LUPCSggh7OyjHUf5JKn+wd+rR+Pv7SAP/hpL4fObQV1v0/tTrZGwEkIIO9qXVcqD6w4AsPySBOL6OciDv3a8P9UaCSshhLCT4ooabl6dRE2tmavHDWDB6P72Lkln5/tTrZGwEkIIO6gza9z50R6yS6oY3t+fBy+Ks3dJDnN/qjUSVkIIYQcvbjrI5tR8evt48Mo1o+3/4K8D3Z9qjYSVEELY2A9qHi9+fxAXF3hh4UjC7f3gr4Pdn2qNhJUQQtjQ0SL9wV9Ng6UXxDAlJsS+BTng/anWSFgJIYSNGE113PrfXZRWmZg+NJTbpg22XzFmM2x5En58Qt92oPtTrZGwEkIIG3lo3QH2ZZcSEejNc/Z88LeySL8/dfAbh7w/1RoJKyGEsIGPdxzlwx1H8XSvf/DXx04P/mbthE+ug9Kj9fen3oTBM+xTy1loM6wURXkLuBDIU1V1WCuvTwXWAun1u9aoqvqIJYsUQghntj+7lGVr9wPw6CXDGBbub/siNA1+fRW+XQZmE4SPgcvfhoABtq+lA9pzZfUO8BKw6gzH/KSq6oUWqUgIIbqQ7JIq/vye/uDvorERXDEmwvZFGEth7e1gWKdvj78VZjwM7j1sX0sHtblSsKqqW4AiG9QihBBdSt6JWha9tp3skipGRATwz4vibV9Ezl549Tw9qDx7wRWrYPYTThVUYLl7VucqirIXOAbcrarqAQt9rhBCOKXskir+9s0xjp+oJbG/P+9ePxYvDxs++KtpkPQObPgb1FVDn0S44l2HHJbeHpYIq11ApKqqJxRFmQt8AQxp603V1dUYDAYLNN99GI1GOWcdJOeu4+Tcnb28E7WNQTUkyJNlkwI4lpnGMRu172KqpG/Sv/HP/AaA4uhLyR15J1puNeQ653/LToeVqqplTb5fryjKK4qiBKuqWnCm93l6ehIbG9vZ5rsVg8Eg56yD5Nx1nJy7s3OspIo/v7a9Mag+ve082478yzPAx7dAgQoePeGi5+mdeAW9bVdBpyQlJbW6v9NhpShKHyBXVVVNUZSx6PfBCjv7uUII4WyOlVSx8LXtHCmqJLG/P8smBdg2qPZ8AF/dBaZKCInVu/1CFNu1b0XtGbr+ATAVCFYUJQv4J+ABoKrqSuAPwC2KotQCVcBCVVU1q1UshBAOqGlQJYT7894fx3EsM802jZuqYP09sPs9fXv4Ipj3jMPORtERbYaVqqqL2nj9JfSh7UII0S21DKrVN4zD38fDNveoCtLgk8WQux/cvWDuUzDy/xx6NoqOkBkshBCiE46VVLHo9VODyiYOfA5r79AnoQ2M1rv9+iTYpm0bk7ASQogOagiqzEIbB1VttT4TxW+v6dtxl8D8/4BXL+u3bScSVkII0QF2C6riDH1uv2O7wdUDZj0OY2/qct1+LUlYCSHEWcoptVNQJa+FdXfo0yf5D4Ar3oHw0dZv1wFIWAkhxFnIKdUHU2QWVjIsvJdtgqqmAr7+O+yqn6I1Zg5cugK8neXpqc6TsBJCiHZqGVT/vWG89YPq2G747EYoTAM3T5j5WLfo9mtJwkoIIdrB5kFlNsMv/4FNj+pLeoTGwYI3IMwOk+E6AAkrIYRog82DqiwHPv8zpG/Wt8f+CS54BDy8rdemg5OwEkKIM8gprWKRLe9RpXylrz1VVQQ+wXDJKxAzy3rtOQkJKyGEOI2GoMpoElQBPlZaB6qmEr69H3a+pW9Hnw+XrAC/MOu052QkrIQQohU2Daqc3/VBFAUquPXQV/EddzO4trk+brchYSWEEC38nlXCn1YlcbzMaN2gMpvh1xWw8SGoq4FgRR9E0TfR8m05OQkrIYRoYu2ebO799Heqa82MiezNG4vHWCeoynPhi1vg0CZ9e8wfYeZy6OFj+ba6AAkrIYQA6swaT3+rsuLHQwAsPCeCRy4eRg93K3TFpX4DX9wKlQX6g73zX4LYCy3fThciYSWE6PbKjSb+8uEevk/Jw83VhQcvjOPacyNxsfSDt6Yq+O7BkxPQDjwPLn0VevW1bDtdkISVEKJbSy+o4KZVO0nLO0GAjwevXDWKCYODLd9QbjJ8dhJd/2QAACAASURBVAPkJesT0J7/AJx7hwyiaCcJKyFEt/XTwXxu++8uyoy1xIT58sa15zAgyML3jMxm2PE6fPsA1FVD0GB9EEW/kZZtp4uTsBJCdDuapvHWzxks/yoZswYzYsN4fuEIfD0t+1eiR8UxeO9eSN+i7xh1Lcz+V5dabt5WJKyEEN1KdW0dyz7fzydJWQDcMX0wf50Rg6urBe9PaRrsepeBX98HtZXgEwQXPg9x8y3XRjcjYSWE6Dbyyo3c/F4Su46U4OXhytOXD+fCxH6WbaQ0W19z6tAm3ABi58O8Z8E3xLLtdDMSVkKIbmFfVil/em8nOaVG+vl78dq1YxgW7m+5BjQN9n4AG/4O1aXg3ZvsEXcRPvOObrechzVIWAkhuryWD/quuGY0IX6elmugPBf+9xdI3aBvx8yBi56nLKuYcAkqi5CwEkJ0WS0f9L1yTASPXBKPp7ubZRrQNNj/Gay/G6qKwdMf5vwLhi+qv5oqtkw7QsJKCNE1tXzQ94F5sSyeEGW5B30rCuDLv4Jhnb4dfT7M/w/4h1vm80UzElZCiC6n5YO+L181iomWfNA3eS18eZc+XVIPX5i1HEYtlntTViRhJYToUpo+6Dsk1Jc3Fo8hMshCzzVVFsH6e2D/p/p21GS4+GXoHWmZzxenJWElhOgSTHVmXv4hjf98n0adWWNGbCjPXTkCPy8Lreqrfg3/WwIncsHDR19mfswNMl2SjUhYCSGcXlreCZZ+vIe9WaUA3D5tMHddYKEHfatK4Ov7YO/7+vaAc/WrqaDozn+2aDcJKyGE0zKbNVb9ksETG1KorjXTz9+Lp68YzoRoC92fStsI65ZAWTa4e8H5D9av4Guh0YSi3SSshBBOKae0ins++Z2taQUAXDYqnIfmx9PLEt1+VcX6xLO739O3w8fAJSsgJKbzny06RMJKCOF01u7J5oEv9lNmrKW3jwePX5rAnAQLrAnV8NzU13+Hinxw6wFT74MJS8BN/rq0Jzn7QginUVJZw7Iv9vPl7zkATB8ayr8WJBDq59X5Dy/OhK+WQtp3+vaACXDR8xCidP6zRadJWAkhnMLm1Hzu/XQvuWXV+PRw44EL41h4TkTnH/Ktq4XfXoXvHwNTpT4LxcxHYOS1MtLPgUhYCSEcWmVNLU+sT+G97ZkAjI7szbNXDLfMs1M5e/UBFDl79O34S2H2v8EvrPOfLSxKwkoI4bB2Hynmro/3kl5QgYebC3+9IIY/T4nGrbND0msq4Mcn4JdXQKuDXv1h3jOgzLZM4cLi2gwrRVHeAi4E8lRVHdbK6y7AC8BcoBK4TlXVXZYuVAjRfZjqzPxn00Fe/vEQdWYNJcyPZ68cTnw/CyzpcXAjfPVXKDkCLq4w/laYdj94+nb+s4XVtOfK6h3gJWDVaV6fAwyp/xoHrKj/UwghzlpaXjl//Wgv+7JLcXGBP00ZxF0XxODl0clnm07kwzf3wb5P9O2wBJj/AoSP7nzRwuraDCtVVbcoihJ1hkMuBlapqqoB2xVFCVAUpa+qqjmWKlII0fWZzRrvbMvg31/rD/iGB3jzzBXDGT8oqHMfrGmw57/wzf1gLAF3b5h2n35F5WahqZiE1VninlU4cLTJdlb9PgkrIUS7HCup4p5P9/JzWiEAl4/uz4MXxXV+Xr+CNPjyTsj4Sd+Onq4vMR84sJMVC1uz2wCL6upqDAaDvZp3SkajUc5ZB8m56zhrnrtas8b/DKW8t7eYKpOGv5crS84NYcKAHmSlp3X8g+tMBKmrCT7wNq7mGmo9A8gdcSdlkbMg1wi5tvldkN87y7FEWGUDEU22+9fvOyNPT09iY2Mt0Hz3YTAY5Jx1kJy7jrPWudt+uJB/rj2AmlsOwMy4MB6/LIFg304uN3/0N304en59SIy4GveZjxHuE4itl0WU37uzl5SU1Op+S4TVOuB2RVE+RB9YUSr3q4QQp5NbZuTx9QbW7jkGwIBAH/55URznx3by2aaKAtj4EOxeDWgQOAgufB4GndfpmoX9tWfo+gfAVCBYUZQs4J+AB4CqqiuB9ejD1tPQh65fb61ihRDOy1Rn5p2fM3h+YyoVNXV4urty69TB/Pm8QZ0b6VdXCzvfhB+Wg7EUXD1gwh1w3r3g4W25H0DYVXtGAy5q43UNuM1iFQkhupxfDhXy4Nr9HMw7AcAFcWE8eGEcEYE+nfvgjK2w/l7IO6BvR58Pc/4NwUM6WbFwNDKDhRDCao6XGlm+3sD/9updfpFBPjx0UTzThoZ27oNLs+G7B/QZ0gECImH2E6DMhc7OFSgckoSVEMLiTHVm3v45nRc2HqSipg4vD1dumzqYm6Z0ssuvthp+eRm2PA2mCn1BxEl3wcQl0uXXxUlYCSEsaltaAQ+uO0BafZffrPgwHrgwjv69O9nll/qtvs5U0SF9O3Y+zFoOAQM6WbFwBhJWQgiLyCmt4rGvDHxVv9ZUVJAPD82PZ6rSyS6/osPw9T8gdYO+HRwDc56E6GmdrFg4EwkrIUSn1NSaeXNrOv/5/iCV9V1+d0wfwo2TB+Lp3okuv5pK2Pos/Pwi1FVDDz+Y+jcY+2dw72G5H0A4BQkrIUSHbT1YwIPr9nM4vwKAOcP6sOzCOMIDOnH/SNMg+Qv4ZhmUZen7hi+CGQ+BX59O1yyck4SVEOKspeaW8/Q3Kt8m5wIwKLgnD82PZ0pMSOc+OM8AG+6F9C36dp9EmPs0DJCFHLo7CSshRLsdLarkuY2pfL47G00Dbw837jh/MDdM6mSXn7EUfvwX/Pqqvhiid284/0EYtRhcO7k0iOgSJKyEEG3KKzfy8vdpvP/bEUx1Gh5uLiwaO4Dbpw8m1M+r4x9cZ4Kkd/SgqizQF0MccwNMXwY+gRarXzg/CSshxGmVVpl4Z1cR697PpMpUh4sLXDYynL9eENO52Sc0DVK+go3/hML62dUjxsPcp6BvomWKF12KhJUQ4hRVNXW8vS2dlT8eosxYC+hTJN09U0Hp49e5Dz+6Q5994sgv+nZgtD54IvYimX1CnJaElRCiUU2tmY92HOHF79PIL68GYHgfL/552ShGDejduQ8vPASbHobktfq2TxBMvQ9GXycr9oo2SVgJIagza6zdk81zG1M5WlQFQGJ/f+6ZpRBkyieuM0FVUQhbnoQdb4LZpE+RdO5tMPFO8OploZ9AdHUSVkJ0Y5qm8V1yLs98m9q4CGJ0SE/unqkwe1gfXFxcMBgKOvbhpirYvgK2PgfVZYALjLgGpv0D/G29DKJwdhJWQnRT2w4V8NQ3KruPlAAQHuDNX2YM4bKR4bi7uXb8g8118PtH8P1jUFa/aPjgGTDjYegzzAKVi+5IwkqIbub3rBKe+kblp4P6FVNQzx7cNm0wV48f0LlnpQAOfQ/fPgi5+/TtPglwwaMyj5/oNAkrIbqJHRlFrPjxEN+n5AHg5+nOTVMG8cdJA/H17ORfBcf3wXcP6mEF0Ks/nP8AJFwBrp24ShOinoSVEF2Ypmn8oObxyg+H2JlZDICXhyvXnhvFLedF07tnJyeELc3Wu/v2fgBo4OkPk++CcX+W9aWERUlYCdEF1daZ+WpfDit+PETKcX3gRC8vdxZPiOK6CVEE+Xp2roGKQtj2gj49Uq0RXD3gnBthyj3QM8gCP4EQzUlYCdGFGE11fJKUxWtbDjUOQQ/18+TGyQO5alxk57v7Kovgl5f0kKrRF1ck/lJ9Hr/AQZ2sXojTk7ASogsoM5pYvT2Tt7ZmUHBCf5h3YHBP/jxlEJeOCu/8wImqEtj+CvzyCtToV2oMmQlT/w7hoztZvRBtk7ASwonllRt5++cMVv+SSXm1Pi1SfL9e3Dp1MLOH9cHNtZPTFxnLCD7wFnzxEVSX6vuip8PUf0DEOZ2sXoj2k7ASwgkdKazk1S2H+CQpi5paMwDnDgrilqnRTB4SjEtn59irLoffXoOfXyTEqD+HxcApekhFntvJ6oU4exJWQjgRQ04ZK348xJe/H8Os6ftmxoVxy9RoRnZ27j6AmgrY8Qb8/AJUFgJQETKSnvMeg6hJnf98ITpIwkoIB6dpGjsyilnxYxo/qPkAuLu6cNmocG4+bxCDQzs5CzpATSXsfAt+fh4q9DaIGAfT/sERYyixUXGdb0OITpCwEsJBGU11rN2TzbvbMknOKQP0lXkXjo3gxsmDCA+wwHNMJqO++OHWZ+GEvkQ94WNg2n0Qfb6+ZIfB0Pl2hOgkCSshHMzRokpWb8/ko51HKak0AfqUSFePj+S6CVEEdvZBXoDaati1Cn56FsqP6fv6joBp98OQC2RdKeFwJKyEcACapvFzWiHv/pLBJkNu4/2o4f39WTwhinmJfTs//Bz0kNrzPmx5Gsqy9H1hCfpM6MocCSnhsCSshLCjE9W1rNmVxbvbMjiUXwGAh5sLFyf2Y/GEKEZEBFimIWOZ3t23/RUoz9H3hcbpix8OvVDm7xMOT8JKCDs4nH+CVb9k8llSVuPzUWG9PLlmXCQLxw4gxK+T0yE1OJEHv66E3944+ZxUaJw+LVLcJRJSwmlIWAlhI2azxo+pebyzLZMtqfmN+8dGBbJ4QhQz48Pw6Mw6Uk0VHYZt/4Hd/4U6fUYLBkyASX+Ve1LCKUlYCWFlpZUmPkk6yqpfMjlSVAnoM59fMiKca8+NIq6fBZd2z9kLW5+H5C9A0x8WRpkHk+6EiLGWa0cIG5OwEsJKko+V8d72TL7YnU2VqQ6A/r29ufbcSK4YE0GAjwVG9QFoGqRv1kPq8A/6PlcPGH4VTFwCIYpl2hHCjiSshLCg0ioT6/Zk8/HOLPZllzbunzwkmMXnRjFtaGjn5+trYK4Dwzo9pHL26Pt6+MLo62D8reAfbpl2hHAA7QorRVFmAy8AbsAbqqr+q8Xr1wFPAdn1u15SVfUNC9YphMMymzW2Hy7k451H2bD/ONX1c/X5e3tw6chwrhkfyeBQX8s1aDLqix1ue1G/NwXgEwzjb9bXlPK2wLRLQjiYNsNKURQ34GXgAiAL2KEoyjpVVZNbHPqRqqq3W6FGIRzSsZIqPkvK4pOkrMZ7UQCTBgdzxTkRzIwLw8vDAs9GNTCWwo43YfsKqNCXpicgEibcASOvkZV5RZfWniursUCaqqqHARRF+RC4GGgZVkJ0eTW1ZjYacvlox1F+Opjf+PBuP38v/jAmgstH9yci0MeyjZYcgd9eh51vn1xLqk8CTLxTH37uJr35outrz295OHC0yXYWMK6V4xYoijIFSAX+qqrq0VaOEcIpqcfL+WjHUb7Yk01RRQ0APdxcuSA+jCvHRDBxcLDl7kXByUETv74GqRtOjuyLmqwPP4+eLsPPRbdiqX+S/Q/4QFXVakVR/gy8C0w/0xuqq6sxyASZZ8VoNMo566COnLuKGjOb00/wbVo5akF14/6o3j2YNdiP6YN86eXlBnUFpKoFFqnT1VSBf8YGeqd9imdZBgCaqztlAy6gaMiVGIPiwASkpFikvfaQ37uOk3NnOe0Jq2wgosl2f04OpABAVdXCJptvAE+29aGenp7Exsa2p0ZRz2AwyDnroPaeO7NZY0dGER/tPMr6fTkYTfoVjZ+nO/NH9OPKcyJICPfv/OKGLRWkwY7X9Xn7qvUZ1vHrC2P+iMvo6/D3DcXfsi22m/zedZycu7OXlJTU6v72hNUOYIiiKAPRQ2ohcFXTAxRF6auqav2EY8wH5J8Swqmox8tZuyebtXuOkV1S1bh//KBArjwngtnxffHuYcHBEqAPPT/4nb4i76FNJ/cPmABjb4LYi8DNw7JtCuGk2gwrVVVrFUW5HfgGfej6W6qqHlAU5RFgp6qq64AliqLMB2qBIuA6K9YshEVkFVfyv705rN2TTcrx8sb9fXp5sWB0OJePjiAquKflG64qht2r9RV5izP0fe7ekHg5nHMT9E20fJtCOLl23bNSVXU9sL7FvgebfH8fcJ9lSxPC8ooqali/Tw+oHRnFjfv9vT2Ym9CXi0f0Y2xUIK6WHCzR4Ph+/Srq94+htv7qLWCAHlAjrwGfQMu3KUQXIWNeRZdXWVPLj4dP8PSvO9icmk9t/XhzT3dXLogL4+IR4UyJCbbMelEt1Zkg5Ss9pDJ/Prl/0DQY92cYMhNcrdCuEF2MhJXokkx1ZrYeLGDtnmy+Tc6lskafm8/N1YUpMSFcMqIfM+P74Otppf8FSo7Ang/0NaQaVuLt4QcjrtJnmQiJsU67QnRRElaiy9A0jaTMYtbuOcZX+3Ian4cCGBriyaJzBzM3oa/l1opqyVSlX0Xtfg8ObwbqnxgOjoGxf4LEK8HLgjOsC9GNSFgJp6ZpGvuzy1i/P4f/7T1GVvHJkXyDQ325ZEQ/5g8PpyIvk9jYKGsUoE8iu3s17PtEnxIJwM1TH8038hoYNFUe4BWikySshNMxmzV2Hy3h6/05bNh/vFlA9fX3Yv7wfswf0Y+4vr0an4cy5Fm4iIoCfaDE7tWQd+Dk/n4j9YAatkAmlBXCgiSshFOoM2v8ll7E1/tz+OZALsfLjI2vhfh5Mis+jHkJ/Rg30Eoj+QDqauHQ93o3n7oBzCZ9v3cgDF8II66GPsOs07YQ3ZyElXBYpjozvxwqZMP+43x74DiFTe5BhQd4Myu+D3MS+jB6QG/rBRTos0vsWa0PmDhxXN/n4gpDZsHIqyFmDrhbaCFFIUSrJKyEQzGa6th6sIAN+4+z0ZBLaZWp8bWoIB9mD+vLnGF9SOxvhSmPmqo+oS8Nv3s1HPnl5P7AaL2bb/gi6NXXeu0LIZqRsBJ2V1lTy2Y1n/X7j/NDSh4nqmsbX4sJ820MqKF9/KwbUOY6yPhJHyix/3MwVej7PXrCsEthxDUwYLwMlhDCDiSshF0UV9TwY2oe3+zP5cfUvMYJYwHi+/VizrA+zB7W17Ir7LbGbIaj22H/Z5C8FiryT742YILezRd3CXhauQ4hxBlJWAmb0DSNQ/kn2GjIY5Mhl6TM4saFCwFGDgjQAyq+LwOCLLx44anFQNZOOLAGDnxx8qFdgMBBEH+Z3s0XPNi6dQgh2k3CSlhNTa2ZHRlFbDTk8n1KHpmFJ5d+93BzYcLAIGbEhjJrWB/6+lt5SXZNg2O7YX99QJUeOflawACIv1QPqb7DpZtPCAckYSUsqriihh/UPDal5LFFzae8yf2n3j4eTBsayozYMCYPCcbPy8rLX2ga5B6AA2uI3v0RnMg6+ZpfPz2ghl0G4aMloIRwcBJWolM0TSMtT+/e+z7l1O69mDBfzo8N4/yhoYwc0NuyS7+fTr5afwW1BgpSAegB0DMU4i/Rr6AixoGrq/VrEUJYhISVOGs1tWZ+Sy9iU0oumwx5HClq3r03cVAQ04eGcv7QMOvff2pQeAgOfK6HVNMZJbwDIW4+mX5jiJxylcxwLoSTkrAS7XK0qJItB/PZkprPtrTCZt17gT17MFUJsV33HuizSRz9FVK/1r/qr6AA8PKHoRfpw80HngduHlQaDBJUQjgxCSvRqorqWrYfLuSngwVsSc3ncEFFs9cbuvdmxIYyIsJG3XtVxZC2SQ+ng9+BseTka57+EDNLn5MverrMKCFEFyNhJQB9cljD8TK2pOrhtDOzCFPdyZtPfl7uTIwOZkpMCJOHBBMRaKPuvYKDejipX+szSWh1J18LGgwxs/WvAePBzQZXdEIIu5Cw6sYKTlTz08F8tqQW8NPBAgpOVDe+5uICwyMCOG+IHlAjIgJwd7PBgIQ6E2Rug9Rv9JAqOnTyNVd3iJx8MqDkOSghug0Jq26kptZMUmZx472nA8fKmr3ep5cXU2L0cJo0OJgAHxt1pVUWwcFv9XBK2wTVTery7q0v/R4zC6LPB+8A29QkhHAoElZdmNmskXK8nG2HCvjlUCHbDxdSUXOyG83T3ZWxAwM5LyaEKTEhDAn1te7cew3qTPoDuumb4eBGyPoNtJPTLREy9OTVU/9zwE1+TYXo7uRvgS6k4ZmnXw4XNoZTcaWp2TExYb5MGaKH09iBgXh52GCEnLkOju+D9C3615FfoObEydddPfRRe8oc/SoqcKD1axJCOBUJKyemaRqZhZX8criQbYf0gGp63wmgn78X46ODOHdQEJOGBFt/WiO9MMgz6DOYp2/R/2xY7r1BcAxETdaXfI+eBp5+1q9LCOG0JKycTHZJFdvSChqvnnJKjc1eD/b1ZEJ0EOdGBzEhOogBgT7W79rTNCg6rHfrpf+kh1PT2csBAiJh4BT9K2qyrAUlhDgrElYOLq/M2BhMPxpyOH7icLPXe/t4MH5QUGNARYfY6L5TydGTV03pW6Asu/nrfn2bh1PvSOvXJITosiSsHIimaWQUVrIjo4idGUXszCg+5WFcPy93xg08eeWkhPlZd0l3gNoayN0P2Un60hpHf4Xi9ObH+ATpodQQUEGDZXJYIYTFSFjZkanOzIFjZY3BtDOziIITNc2O8enhxtiBgZw7KIg+ruVcOHG4dWeL0DQoydRDqSGccvZCXfN7YXj6Q9TEk+EUEisTwwohrEbCyobKjSZ2HylhZ0YROzKK2X20uNkKuQDBvj0YExnImKjenBMVSFy/XnjUP4xrMBgsH1RVJXooNQRTdhJUFpx6XNAQ6D9GX06j/xjokyhz7QkhbEbCyoqOlxobu/R2ZBSTcrys2fIZAINCenJOk3CKDLLigIg6k96d1/SqqfDgqcf5BEH4mJPhFD5KfzhXCCHsRMLKQqpr6zDklLP3aAm7jxSzM7OYrOKqZse4u7owPMKfc6ICGR3ZmzGRvQny9bROQRWFkJesDyHPOwC5yXD8d6htPnoQN0/om9g8nHpHyf0mIYRDkbDqgDqzxqH8E+w9WsLerBJ+zyrFkFPWbOJXAD9Pd0ZF9uacqN6MiQpkeP8AvHtYuOus+gTkpzQJpmQ9mCryWj8+MLo+lMZA/9EQliAzlAshHJ6EVRs0TSO7pIrfs0obw2lfVmmzaYtAvxAZEupLYv8ARkT4MyYqkJgwP8vdY6qtwbMkDfYd0JdqbwimkszWj/foCaGx9V9x+p99h4NPoGXqEUIIG5KwaqGooka/WjpaWn/VVHLKCD2A8ABvEvv7MzwigOH9AxgW3qvziw6azVCeowdQcQYUZ+qLCuYlQ2Eag8y1p77H1QNClBbBFAf+ETI6TwjRZXTbsKozaxwtqkTNLSf1eDkpueX8nlXC0aKqU44N8PHQr5j6+5PYP4DECH9C/bw61rCx9GQQFWc0D6aSI6cOEW/kQrVvBJ79h5+8UgqNg6BoWcdJCNHldfmw0jSN42VG1OPlpOaWox4/QWpuOQfzyk8ZNg7g5eHKsH71V0wRAQzv7392UxbVVOpXRy2DqGG7qvjM7+8Zog9wCIjU/wwcBGFxEKxw+FAmsbGxZ3cChBCiC2hXWCmKMht4AXAD3lBV9V8tXvcEVgGjgULgSlVVMyxbatsKT1Q3XimpuSc4mFuOmltOubGV7jMgrJcnMWF+KGF+xIT5MSzcn5gw3+aLDGqavr7SiXx90MKJPH3eu4r8k9+fyKt/LR9MFa221cjdWw+h3lH6FERNgylgAHj6WuhsCCFE19FmWCmK4ga8DFwAZAE7FEVZp6pqcpPDbgCKVVUdrCjKQuDfwJXWKLipbYcK+PZALqm5+lVTa/eWAIK8XUkI9SA+2A2ltwvRAS5E+mr4ulZDTZY+oq7mBCQXwI68JsFUH0qn7ZprhVsP8O2jB1FDCDUNpp4hMixcCCHOUnuurMYCaaqqHgZQFOVD4GKgaVhdDDxU//2nwEuKorioqtriEVjL0TSNT1e9xMi6fSS4GOlJNX6e1QR5mPB3q6anazXe5irc6ypxqTVCLvpXR3j0BN8Q6BkKvqF64PQMOfm9b2j9ayHg2UvCSAghLKw9YRUOHG2ynQWMO90xqqrWKopSCgQBrczbYxkumsa/e7yBh6m8+Qu19V/Nj4YevtCjZ5Ov+m1P35PbPkGnhk/PEP11IYQQdmO3ARbV1dUYDIZOfYb35KfwKjmI2d0Hs7t38z89vOu/74nm5nl2VzsacAI4UQUc6VSNlmQ0Gjt9zrorOXcdJ+eu4+TcWU57wiobiGiy3b9+X2vHZCmK4g74ow+0OC1PT8/Oj2zrZiPjDAaDjAbsIDl3HSfnruPk3J29pKSkVve3J6x2AEMURRmIHkoLgataHLMOWAz8AvwB+N6a96uEEEJ0L21OcaCqai1wO/ANYAA+VlX1gKIojyiKMr/+sDeBIEVR0oC7gL9bq2AhhBDdT7vuWamquh5Y32Lfg02+NwKXW7Y0IYQQQieTxwkhhHB4ElZCCCEcnoSVEEIIhydhJYQQwuFJWAkhhHB4ElZCCCEcnoSVEEIIh+eiafaZaCIpKSkfyLRL40IIIRxV5OjRo0Na7rRbWAkhhBDtJd2AQgghHJ6ElRBCCIcnYSWEEMLhSVgJIYRweBJWQgghHJ7dlrUXp6coymzgBcANeENV1X+1eP0u4EagFsgH/qiqqjwGQNvnrslxC4BPgXNUVd1pwxIdVnvOnaIoVwAPARqwV1XVlguxdkvt+H92APAuEFB/zN/rl14S7SRXVg5GURQ34GVgDhAHLFIUJa7FYbuBMaqqJqL/hfukbat0TO08dyiK4gf8BfjVthU6rvacO0VRhgD3ARNVVY0H7rR5oQ6onb93y9AXrh2Jvtr6K7at0vlJWDmesUCaqqqHVVWtAT4ELm56gKqqP6iqWlm/uR3ob+MaHVWb567eo8C/AaMti3Nw7Tl3NwEvq6paDKCqap6Na3RU7Tl3GtCr/nt/4JgN6+sSJKwcTzhwtMl2Vv2+07kB2GDVipxHm+dOUZRRQISqql/ZsjAn0J7fuxggRlGUnxVF2V7f9SXad+4eAq5RFCULfdX1O2xTWtchYeXEFEW5BhgDPGXvWpyBoiiuwLPAUnvX4qTcgSHAVGAR8LqiKAF2rch5LALeUVW1PzAXeK/+91G0k5wsfEEAXQAAAmhJREFUx5MNRDTZ7l+/rxlFUWYA9wPzVVWttlFtjq6tc+cHDAN+VBQlAxgPrFMUZYytCnRg7fm9ywLWqapqUlU1HUhFD6/urj3n7gbgYwBVVX8BvIBgm1TXRchoQMezAxiiKMpA9F/4hUCzEVeKoowEXgVmy32DZs547lRVLaXJXxCKovwI3C2jAYF2/N4BX6BfIbytKEowerfgYZtW6Zjac+6OAOcD7yiKEoseVvk2rdLJyZWVg1FVtRa4HfgGMKCPIDqgKMojiqLMrz/sKcAX+ERRlD2KoqyzU7kOpZ3nTrSinefuG6BQUZRk4AfgHlVVC+1TseNo57lbCtykKMpe4APgOlVVZRbxsyCzrgshhHB4cmUlhBDC4UlYCSGEcHgSVkIIIRyehJUQQgiHJ2ElhBDC4UlYCSGEcHgSVkIIIRyehJUQQgiHJ9MtCWEHiqI8BPwTSAReBMYBpcDrwEOqqprtV50QjkeurISwry+AjcAlwPvAA8CDdq1ICAckV1ZC2NfrTZZA/1ZRlF7AUkVRnldVtcSehQnhSOTKSgj7+rjF9ofokxQPs0MtQjgsCSsh7Cv3NNtnWh1aiG5HwkoI+wo7zfYpC24K0Z1JWAlhX1e02F4InAD22aEWIRyWDLAQwr5uUhTFFX212VnAjehD10vtW5YQjkWurISwr4uBC4B1wDXAY8Cjdq1ICAckV1ZC2FeKqqrT7F2EEI5OrqyEEEI4PAkrIYQQDs9F0zR71yCEEEKckVxZCSGEcHgSVkIIIRyehJUQQgiHJ2ElhBDC4UlYCSGEcHj/v9HKahSMglEwCkbBoAcALaIX2RWfBBMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(7, 5))\n",
    "ps = np.linspace(0.05, 0.95, 20)\n",
    "qs = [0.02]\n",
    "kls = [[kl_bernoulli(q, p) for p in ps] for q in qs]\n",
    "for i, q in enumerate(qs):\n",
    "    plt.plot(ps, kls[i], label=r\"$KL(q || p), q = {:.2f}$\".format(q), linewidth=2)\n",
    "    plt.plot(ps, [2 * (q - p)**2 for p in ps], label=r\"$2 (q - p)^2$\", linewidth=2)\n",
    "    \n",
    "# plt.ylabel(\"KL(q || p)\", fontsize=16)\n",
    "plt.xlabel(\"p\", fontsize=16)\n",
    "plt.legend(fontsize=14);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Parameterize the $Q$\n",
    "\n",
    "For $Q$ we will use the standard $N(m, s)$ form, where $s$ is a diagonal matrix.\n",
    "\n",
    "We will optimize the bound\n",
    "\n",
    "$$\n",
    "    e(Q) \\le \\hat e(Q, S_m) + \\sqrt{\n",
    "        \\frac{KL(Q \\vert \\vert P) + \\log \\frac m \\delta}{2(m-1)}\n",
    "    } = \n",
    "    \\mathbb E_{h \\sim N(m, s)} \\hat e(h, S_m)\n",
    "    + \\sqrt{\n",
    "        \\frac{KL(N(m, s) \\vert \\vert P) + \\log \\frac m \\delta}{2(m-1)}}.\n",
    "$$\n",
    "\n",
    "<span style=\"color:blue\">The bound is of the form _Expected Loss + Complexity penalty_. Notice the similarity with variational ELBO.</span>\n",
    "\n",
    "We will optimize the bound with respect to $m, s$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Prior Choice\n",
    "\n",
    "- We will use $P(w) = N(\\mu, \\lambda I)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- For $\\mu$ we will use random initialization $w_0$\n",
    "    - <span style=\"color:blue\">The idea is to have a compact prior. In particular, we want to avoid covering all the symmetries in the parameterization.</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- $\\lambda$ will be trained to optimize the bound\n",
    "    - We need to account for optimizing the $\\lambda$ in the bound. \n",
    "    - Use a discrete set of values for $\\lambda$ of the form $\\lambda = 0.1 \\exp(- j / 100)$, $j \\in \\mathbb N$\n",
    "    - Use different $\\delta$ values that sum up to one for different values of $\\lambda$: $\\delta_j = \\frac \\delta {\\pi^2 j^2}.$\n",
    "    - By union bound, the bound will then hold simultaneously for all $\\lambda$ with probability $\\delta$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Final bound:\n",
    "\n",
    "$$\n",
    "    \\mathbb E_{h \\sim N(m, s)} \\hat e(h, S_m)\n",
    "    + \\sqrt{\n",
    "        \\frac{KL(N(m, s) \\vert \\vert N(w_0, \\lambda I)) +  2 \\log(b \\log \\frac c \\lambda) + \\log \\frac {\\pi^2 m} {6 \\delta}}{m-1}}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We minimize the bouns with respect to $m, s$ and $\\lambda$!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Example: MLP on MNIST"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/izmailovpavel/anaconda3/envs/py37/lib/python3.7/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.decomposition.pca module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.decomposition. Anything that cannot be imported from sklearn.decomposition is now part of the private API.\n",
      "  warnings.warn(message, FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "import torch\n",
    "import torch.nn.functional as F\n",
    "import torchvision\n",
    "import numpy as np\n",
    "import tqdm\n",
    "\n",
    "from swag import data, models, utils, losses\n",
    "from swag.posteriors import SWAG\n",
    "from swag.utils import eval, train_epoch\n",
    "from swag.losses import cross_entropy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are going to run models on the test set. Are you sure?\n"
     ]
    }
   ],
   "source": [
    "# Making  MNIST dataloaders for binary classification\n",
    "\n",
    "loaders, num_classes = data.loaders(\n",
    "    \"MNIST\",\n",
    "    \"~/datasets/\",\n",
    "    128,\n",
    "    4,\n",
    "    torchvision.transforms.ToTensor(),\n",
    "    torchvision.transforms.ToTensor(),\n",
    "    use_validation=False,\n",
    "    split_classes=None,\n",
    "    shuffle_train=False\n",
    ")\n",
    "loaders['train'].dataset.targets = 1 - 2*(loaders['train'].dataset.targets < 5).int()\n",
    "loaders['test'].dataset.targets = 1 - 2*(loaders['test'].dataset.targets < 5).int()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Stochastic Linear Layer\n",
    "\n",
    "class StochLinear(torch.nn.Module):\n",
    "    def __init__(self, in_features, out_features, bias=True, \n",
    "                 init_inv_softplus_sigma=-3.0, eps=1e-6):\n",
    "        super().__init__()\n",
    "        self.weight_mu = torch.nn.Parameter(torch.zeros(out_features, in_features))\n",
    "        self.weight_ispsigma = torch.nn.Parameter(torch.empty(out_features, in_features).fill_(\n",
    "                                                                                init_inv_softplus_sigma))\n",
    "        if bias:\n",
    "            self.bias_mu = torch.nn.Parameter(torch.zeros(out_features))\n",
    "            self.bias_ispsigma = torch.nn.Parameter(torch.empty(out_features).fill_(init_inv_softplus_sigma))\n",
    "        self.eps = eps\n",
    "        self.with_bias = bias\n",
    "    \n",
    "    \n",
    "    def forward(self, x):\n",
    "        weight = self.weight_mu + torch.randn_like(self.weight_mu) * self.weight_sigma\n",
    "        if self.with_bias:\n",
    "            bias = self.bias_mu + torch.randn_like(self.bias_mu) * self.bias_sigma\n",
    "        else:\n",
    "            bias = None\n",
    "        return torch.nn.functional.linear(x, weight, bias)\n",
    "    \n",
    "    def copy_weights(self, linear_layer):\n",
    "        self.weight_mu.detach().copy_(linear_layer.weight)\n",
    "        if self.with_bias:\n",
    "            self.bias_mu.detach().copy_(linear_layer.bias)\n",
    "            \n",
    "    @property\n",
    "    def weight_sigma(self):\n",
    "        return torch.nn.functional.softplus(self.weight_ispsigma) + self.eps\n",
    "    \n",
    "    @property\n",
    "    def bias_sigma(self):\n",
    "        return torch.nn.functional.softplus(self.bias_ispsigma) + self.eps\n",
    "            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class PACBound:\n",
    "    def __init__(self, prior_mean, lambda_init, likelihood_fn, num_data,\n",
    "                 delta=0.025, b=100, c=0.1):\n",
    "        self.prior_mu = prior_mean\n",
    "        self.log_lambda_ = torch.tensor([math.log(lambda_init)],\n",
    "                                   requires_grad=True, device=prior_mean.device)\n",
    "        self.likelihood_fn = likelihood_fn\n",
    "        self.num_data = torch.tensor([num_data], requires_grad=False, \n",
    "                                     dtype=float, device=prior_mean.device)\n",
    "        self.delta = delta\n",
    "        self.b = b\n",
    "        self.c = c\n",
    "        \n",
    "    def kl(self, stoch_network):\n",
    "        mu_net, sigma_net = self._get_mu_sigma(stoch_network)\n",
    "        prior_sigma = torch.ones_like(sigma_net) * self.prior_sigma\n",
    "        return self._kl_gaussians(mu_net, sigma_net, self.prior_mu, prior_sigma)\n",
    "    \n",
    "    def train_bound(self, stoch_network, x, y):\n",
    "        preds = stoch_network(x).flatten()\n",
    "        acc = (preds * y > 0).float().mean()\n",
    "        likelihood = self.likelihood_fn(preds, y)\n",
    "        regularizer =  torch.sqrt(self.B_RE(stoch_network) / 2)\n",
    "        return likelihood + regularizer, preds, \\\n",
    "               {'accuracy': acc, 'regularizer': regularizer}\n",
    "        \n",
    "    def B_RE(self, stoch_network):\n",
    "        kl = self.kl(stoch_network)\n",
    "        return (kl + \n",
    "                2 * torch.log(torch.abs(self.b * (math.log(self.c) - self.log_lambda_))) + \n",
    "                math.log(math.pi**2 * self.num_data / (6 * self.delta))) / (self.num_data - 1)\n",
    "        \n",
    "    @property\n",
    "    def prior_sigma(self):\n",
    "        return 1 / torch.sqrt(torch.exp(self.log_lambda_))\n",
    "        \n",
    "    @staticmethod\n",
    "    def _kl_gaussians(mu_1, sigma_1, mu_2, sigma_2):\n",
    "        # sigma_i -- vector of stds\n",
    "        d = mu_1.numel()\n",
    "        kl = 0.5 * (torch.sum(torch.log(sigma_2) - torch.log(sigma_1)) * 2 +\n",
    "                    torch.sum(sigma_1**2 / sigma_2**2) +\n",
    "                    torch.sum((mu_2 - mu_1)**2 / sigma_2**2) - d)\n",
    "        return kl \n",
    "        \n",
    "    @staticmethod\n",
    "    def _get_mu_sigma(stoch_network):\n",
    "        mu_net, sigma_net = [torch.cat(a) for a in \n",
    "                             zip(*[\n",
    "                                  [torch.cat([layer.weight_mu.flatten(), layer.bias_mu.flatten()]),\n",
    "                                  torch.cat([layer.weight_sigma.flatten(), layer.bias_sigma.flatten()])]\n",
    "                                  for \n",
    "                                  layer in stoch_network if\n",
    "                                  isinstance(layer, StochLinear)])\n",
    "                            ]\n",
    "        return mu_net, sigma_net"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def logistic_loss(preds, y):\n",
    "    loss = torch.log(1 + torch.exp(-preds.flatten() * y))/ math.log(2)\n",
    "    loss = loss.mean(dim=0)\n",
    "    \n",
    "    return loss\n",
    "\n",
    "# single hidden layer\n",
    "def get_model():\n",
    "    return torch.nn.Sequential(torch.nn.Flatten(), \n",
    "                               torch.nn.Linear(28**2, 600), \n",
    "                               torch.nn.ReLU(), \n",
    "                               torch.nn.Linear(600, 1))\n",
    "\n",
    "\n",
    "def get_stoch_model(isp_sigma=-3.):\n",
    "    return torch.nn.Sequential(torch.nn.Flatten(), \n",
    "                               StochLinear(28**2, 600, init_inv_softplus_sigma=isp_sigma), \n",
    "                               torch.nn.ReLU(), \n",
    "                               StochLinear(600, 1, init_inv_softplus_sigma=isp_sigma))\n",
    "\n",
    "def logistic_loss_swag(model, x, y):\n",
    "    preds = model(x).flatten()\n",
    "    acc = (preds * y > 0).float().mean()\n",
    "    return logistic_loss(preds, y), preds, {'accuracy': acc}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 10/10 [00:16<00:00,  1.61s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train:  {'accuracy': tensor(0.9966, device='cuda:0')}\n",
      "Test:  {'accuracy': tensor(0.9815, device='cuda:0')}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# we will start by pre-training the model\n",
    "\n",
    "model = get_model()\n",
    "model.cuda()\n",
    "\n",
    "# Save the random initialization for the prior\n",
    "torch.save(model.state_dict(), \"model_init.pt\")\n",
    "\n",
    "loss_fn = logistic_loss_swag\n",
    "opt = torch.optim.SGD(model.parameters(), lr=0.05, momentum=0.9)\n",
    "\n",
    "for epoch in tqdm.tqdm(range(10)):\n",
    "    train_res = utils.train_epoch(loaders['train'], model, loss_fn, opt, regression=True)\n",
    "    test_res = eval(loaders[\"test\"], model, loss_fn, regression=True)\n",
    "print(\"Train: \", train_res['stats'])\n",
    "print(\"Test: \", test_res['stats'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Initialize the stochastic model and the PAC bound\n",
    "\n",
    "init_model = get_model()\n",
    "init_model.cuda()\n",
    "init_model.load_state_dict(torch.load(\"model_init.pt\"))\n",
    "\n",
    "prior_mu = torch.cat([p.detach().flatten() for p in init_model.parameters()])\n",
    "pac_bound = PACBound(prior_mu, lambda_init=1e-3, likelihood_fn=logistic_loss, \n",
    "                     num_data=len(loaders['train'].dataset))\n",
    "\n",
    "stoch_model = get_stoch_model(isp_sigma=-2)\n",
    "for stoch_layer, layer in zip(stoch_model, model):\n",
    "    if isinstance(stoch_layer, StochLinear):\n",
    "        stoch_layer.copy_weights(layer)\n",
    "stoch_model.cuda();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 50/50 [02:06<00:00,  2.52s/it]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Iteration')"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1IAAAFCCAYAAAD7b+y1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd1SU19bA4d/QO0gRO/YjqFiwEEuK6c0UEzVRE9NMz81Nbnr9Ynpyk5ueqDHRaGK6msT0ZosNGygexYJSRFCRPpSZ7493QFSQAQeGgf2sxRLmLbM5K5lhz9lnH5PVakUIIYQQQgghhP3cnB2AEEIIIYQQQrgaSaSEEEIIIYQQop4kkRJCCCGEEEKIepJESgghhBBCCCHqSRIpIYQQQgghhKgnSaSEEEIIIYQQop48nB2As2zcuNHq7e3d4OvNZjOncn1LIeNgkHEwyDjIGFSydxyKiopy4uLiIpogJJcj71OOIeNgkHEwyDgYZBwc8z7VahMpb29voqOjG3x9cnLyKV3fUsg4GGQcDDIOMgaV7B2HhISE1CYIxyXJ+5RjyDgYZBwMMg4GGQfHvE9JaZ8QQgghhBBC1JMkUkIIIYQQQghRT5JICSGEEEIIIUQ9SSIlhBBCCCGEEPUkiZQQQgghhBBC1JMkUkIIIYQQQghRT5JICSGEEEIIIUQ9SSIlhBBCCCGEEPUkiZQQQgghhBBC1JOHswMQQghRf+UVFjzc6/dZWHmFBZPJhLubya7zS8oq+HLdPtIOF3PL6d0JD/BuSKjCxVksVt79K4XScgsjeoYzqEsI3h7uzg5LCCGcThIpIYRwMduz8pk0azU9Ivz5YMoQgn0967xm36Eirpu9hkOFpVwV14lJw7vQPSKgxnNLyir4dPVe3v97JwfyzQAsWLuPhy/sw4QhnXGzMxETLcMvW/fz6i/bAXjzjxR8PN0Y2jWUkT3DGdkjnJgOQXYn50II0ZJIIiWEEC4kt6iUW+auIzvfTHa+mcmzVvPJTcMI8fOq9Zo9OYVcO3MVGUdKAPhw+W4+XL6bUT3DmRzfhXOiI/Fwd6O4tIL5q1P5YOkusm0JVEz7IEL9vVieksMj3yTy5bp9PHdFf6LbBzXJ7yuc78PluwEY3SucA3lmdFY+y3bksGxHDgCh/l7cd25vJg3vgskkCZUQovWQREoIIVxEeYWFuz/bQOrBImLaB1FYWk5i+hGumbmaeTcNI6yG0ruUAwVMmrWKrDwzcVFtePB8xdfr01i8KYPlKTksT8khMsibc6Ij+XlLFjkFRgLVt0MQ/zq7F+fGRALwQ2Imz3y3lfV7c7nkreXcOLIr957TG39veRtpyTan5bJ2z2ECvT14b3IcAd4eZOebWbkzh5UpB1mxM4e0w8U8vjCJn5L28+K4/nRq4+fssIUQoknIO6AQQriIF3/cxrIdOYT5ezHz+iG4m0xcO3MVyZl5XDNzFfNvjici8GgypfcbJYA5BWaGdwtl9tSh+Ht7MLx7GI9dFMPX69OYvzqVndmFzF+9F4D+HYP519m9ODu67TGzC5fEduD03hG89st25v6zh5nLdvP95kzeumYQQ7qGNvVQiCYy2zYbNXFYZwJsSXNEoDeXDezIZQM7AvD95gyeWJjE8pQcLvjfMh6/OJoJQzvL7JQQosWTrn1CCFEDi8VKfklZoz/Hmt2HeOv3Hfyz8+BJz/06IY1Zy3fj4WbivclxdAzxpV2wDwtujadX2wC2ZxUwccY/ZOUZ5XtbM4zkKqfAzKie4Xx8w7BjZo+C/Ty5cVQ3frvvDD69ZTi3n9mD2VOHsPiukZwTE1njH8FBPp48PbYvC+8cSf+OwWQeKWHG0l2OHRTRbOw/UsL3mzNxM8H1I7rWet4lsR345d9ncH7fSArM5Tz8TSLXf7SWzCPFTResEEI4gcxICSFENQcLzHyxLo1P16SSkVvCv8/pxR1n9nRYgwWr1cqmtCN8tymDHzZnst+W+ACM6dOWhy7og2oXeMw1m/bl8si3iQD832V9Gdbt6AxQ20AfFkyLZ9Ks1Wzbn8+ED/7h0YuieeCrzRwpLuNMFcH7k+Pw8ay5y5rJZGJEj3BG9Ai3+3eI7RTCwjtH8vf2A7JWqgWb+88eyi1WLurfrs5yvYhAb96fHMfiTRk8uWgLS7dnc97rS5l+WT8uH9SxaQIWQogmJomUEKLVs1qtJKQeZt6qVJYk7qe0wlJ17NVftrNmz2FeHz+gxjVI9t4/OTOf7zZn8P3mDPYdOvpJfccQX07rEcZPSfv5Y9sB/tIHuDquM/8+tzftgn04kFfCtE/WUVpuYdLwLkwaHnXC/cMCvPnslnimzF5NUnoe0z5JAOCc6EjemTSoUVpVu7uZGNMn0uH3Fc1DcWkFn64xyj1vGtXNrmtMJhOXDezIad3DePTbRH5LPsC9n2/E18ud8/u2a8xwhRDCKSSREkK0WiVlFXyVkMa8Vals258PgMkEZ/dpy+T4KKxYuf+LTSzdns3Fby7nrWsHMbQe64FSDuTz3aZMvtucwa7swqrH2wZ6c3Fsey6J7cDgLiGYTCYevrAPb/2+g/mr9/L5un0s2pTOzaO6s3JnDll5ZoZ1DeWpS/vW+lxt/L2Yf3M8181ew6Z9uVzYrx1vTByEl4dUcIv6+2ZDGrlFZQzoHMLgLm3qdW3bIB9mXjeEt/5I4bVft3Pf5xv55o6RJ8y0CiGEq5NESgjR6lRYrHyzPo3//baD9Fxjdig8wIsJQztzzbAux5QxLfnXaO7+dAPrUg8zccYq/nOe4tbTu9dY6me1WsnIK+OPP1P4blNGVXIGRovoC/q149LYDgzrFnrCvjvhAd7832X9mDqyG6/8vI0lift5+88UADoE+/Du5MF1JkXBvp58Pi2ezWlHiItqI3v7iAaxWKxVTSZuHNm1QU0jTCYTd4/pScqBAhZvyuDmuWtZfOco2vjX3qZfCCFcjSRSQohWw2q18vOWLP77i2bHgQIAVGQgd47pyQV929WYqLQP9uWzafG8+ovmg7938dJP21iz+yA3jupG2uFiUg8WsfdQIakHi0g9WESBubzq2kAfDy7o245LBnRgRI8wPN3rnh3qFu7Pu5PiSEg9zEs/bWNXdiEzrhtCuJ1lhT6e7sesoRKivpbuyGZndiHtg324qH/7Bt/HZDLx0rhYducUkph+hDs/Xc+cG4fZ9f+BEEK4AkmkhBB2sVqtPPtDMkWlFUy/rC8eTvhjyGq1sn7vYXKLyugdGUjHEF+7m0Cs3JnDSz9pNu3LBaBTG1/uP683Ywd0rHPmxtPdjUcujGZY11Du/3ITf+ps/tTZNZ4b4OXG2THGzNPo3uENXp8UF9WGL249DavVKm2kRZOq3ID3utO6nnLS4+vlzozr4rj0rRWs3HmQ535I5umxtZeoCiGEK5FESghhl/V7D1f9gRXi58lDF/Rpsue2Wq0s3ZHDG79tZ/3e3KrH/b3c6RkZiIoMoHdkIN3C/Skwl5OVV8L+I2ay8ks4kFdC5pES0g4fLeG75+xeTBzapd7rh86OjuSHe0bz3A9bOZBnJirMn6gwP6LC/OgS6kdUmD/7U1OIiYlx2O8uSZRoStuz8lm2IwdfT3euGdbZIfdsH+zLB1MGc82M1Xy8cg/R7QOZMLSLQ+4thBDOJImUEMIus5btrvr+vb92MrhLG86NadyubVarlb+2Z/PGbzvYaJtJauPnSXT7IHYcKCA738ymfblVs0wnE+jtwa1ndOeGkd2O2U+pvjqG+PLupLhaj2dJ4iNc2EcrjP/Px8V1JMTPceuZ4qJCefbyfjz49WYeX5hEj4gA2chZCOHyJJESQtRp78Eift6yH093E9ed1pUPl+/mvi828sPdo+kSVvv+MqXlFl78cRvbs/J5Y+JAu9uHW61W/tLZ/O/3HVVJUqi/F7eM7s6U06IIsCVChwpL2Z6VX/WVerCIIF9PIgN9iAzypl2wD21t33cI8a11LyUhhPH/0zfr0wG4YaR9Lc/rY/zQzmzNzOPjlXu4bV4Ci+8aRYcQX4c/jxBCNBVJpIQQdZq9YjcWK1w+oAOPXxxN6sEifkvO4vb5CXx9+4gaE5QjRWXcNi+Bf3YdBODJxVt459rBdj3fM99v5aMVewAI8/di2undmRwfdcJMUqi/F/Hdw4jvHnZqv6AQDqKU8gGWAt4Y77Ffaa2fUkp9DJwBHLGdOlVrvdE5Udbs09WpmMstnKUi6BER0CjP8fjF0ew4kM+KlIPcMncdX952Gn5e8qeIEMI1SescIcRJHSku44t1+wC4eVR3TCYT/x0/gC6hfmzJyOP/vttywjWpBwu54r0V/LPrIBGB3vh5ufPD5ky+35xR5/P9lLSfj1bswcvdjUcv6sOyh87i1jN6nFI5nhBNyAyM0VoPAAYCFyil4m3HHtBaD7R9NaskCmDp9hwArq1h02dH8XB3451rBxMVZrx+3P/FJiwWa6M9nxBCNCZJpIRowVbtOsjjCxN57dftfLp6L78nZ5GUfoTsfLPdf7wsWLOXotIKRvYMI6ZDEGDsV/TuJGNfo8/W7OOrhLSq89ftOcTl76xgV3YhfdoFsvDOkTx6UTQATyxMIjvfXOtzpecW89DXmwF4+MI+TDu9h3xaLVyK1tqqtS6w/ehp+3KJTCHf1rq/fbBPoz5PiJ8XH14/hEBvD35M2s8bv+9o1OcTQojG0uR/oSilLgDeANyBWVrrF487HgXMBiKAQ8BkrXWaUuos4PVqp/YBJmqtF7pCyYQQTS23qJTb5yVwuKisxuMebiauHtKJ5y7vX2sL8bIKCx+v3AMYs1HV9esYzPTL+vLQ14k89m0ifTsE8eeuAv63cg+lFRbO6B3B29cOItDHk0nDu/BT0n6Wp+Tw+MJE3p8cd0I3uvIKC/cu2MCR4jLO7tOWG0Z2PeUxEMIZlFLuQALQE3hHa71aKXU78JxS6kngd+BhrXXtnyo4QVGpkUj5eTX+WsKebQN589pB3PTxWt74fQe9IgO4JLZDoz+vEEI4UpMmUrY3l3eAc4E0YK1SarHWemu1014F5mqt5yilxgAvAFO01n9ilEmglAoFUoBfql33gNb6q6b4PYRwBa/9up3DRWUM6BTMGb0jyMozsz+vhKy8Eg7kmzlUWMpna/YR6u/FA+fX3Mp8SWImmUdK6BHhzxm9I044PmFoF9btOcyXCWlcO3NVVdI2JT6Kpy6NqdprymQy8eK4/lzwv2X8vCWLxZsyuGxgx2Pu9eYfKazdc5jIIG9euXqAtP0WLktrXQEMVEqFAN8qpfoBjwD7AS9gBvAQ8MzJ7mM2m0lOTm5wHCUlJfW6Pq/QyOsy9u7GnNP4fx60A26OC2XGukPc//lGLHkH6BVmX0Oa+qjvOLRUMg4GGQeDjINjxqCpZ6SGASla610ASqkFwGVA9UQqBrjP9v2fwMIa7nMV8KPWuqgRYxXCZW3NyGPeqlTc3Uy8dFUsfdoFnXDO0u3Z3PDxWt75cye92gZy+aBjExur1VrV8vzm0d1rnbWafnk/kjLySM7MwwQ8cUkMN4zsekIi1KmNH49dHM0j3yTy5KItnNY9jLZBRgnRql0HefuPHZhM8PqEgYT6O67tshDOorXOVUr9CVygtX7V9rBZKfUR8J+6rvf29iY6OrrBz5+cnFyv682WVAAG9ouu6ozZ2Pr0sZLLZr5Yl8bzSw+y+K6RVa8LjlLfcWipZBwMMg4GGQf7xyAhIaHWY02dSHUE9lX7OQ0Yftw5m4ArMcr/rgAClVJhWuuD1c6ZCLx23HX1Kplo6k/6WioZB0NzGger1cqDP2ViscJYFYj1cDrJh9NPOC8CmDYklPfWHOSBLzdhyTtAdNujf8Ak7i8mMf0IQd5uRPvmn/T3e2hEMJ9stHBaBy9OCyth27ZtNZ43IMDK4A6+rM8o5u65//DUmEjyzRbu/C4NixWuiQ2hTWk2ycnZpzwOztKc/ltwptY6DkqpCKDMlkT5YlRgvKSUaq+1zlRKmYDLgSSnBnoci8VKUWkFAL5NuE2AyWRi+uX92J1TyNo9h7nlkwQ+nxYvWxUIIVxCc1zF/R/gbaXUVIwWsulAReVBpVR7oD/wc7Vr6l0y0dSf9LVUMg6G5jQOizamk3RgN6H+XkwfH0+wn2et5/bpYyXfLYl5q/by/LIcFt01io62fV1eW7sOgKmjejCwf++TPmc0cOZQ+8bh7Q7dOP/1paxOKyK5OIgfkzI5WFTBkKg2TJ8QX1UO6Kqa038LzuSIT/pcVHtgjq2U3Q34Qmv9vVLqD1uSZQI2Arc5M8jjFZUdTaLca5l9bizeHu68NzmOy95ewaZ9uTz89WZenzBQynuFEM1eUydS6UDnaj93sj1WRWudgTEjhVIqABintc6tdsp44FutdVm1azJt39pdMiFES1RoLuf5JcYswIPnq5MmUWB8GvzUpX3ZnVPIipSD3DxnHV/ddhoH8s38lpyFl4cbU+Id2wq5Q4gvT1waw4NfbeahrzdTbrES5OPB/yYOdPkkSgit9WZgUA2Pj3FCOHYrsnXs8/d2zkxQeIA3H04dwrh3V7JwYwZdwvy579yTf4AjhBDO1tR/tawFeimluimlvDBK9BZXP0EpFa6UqozrEYwOftVdA3x23DXtbf82y5IJIZrK23+mkJVnJrZTMOOHdK77AsDT3Y13r42je7g/yZl5/PvzjXy4fBdWK1wxsCMRgY5f/H11XCfOUhGU21qwv3xVLJ3a+Dn8eYQQ9im0lfU5c7+2Pu2CeOvaQbiZ4M3fd/D52r1Oi0UIIezRpImU1rocuAujLC8Zo+Rhi1LqGaXUWNtpZwJaKbUdiASeq7xeKdUVY0br7+NuPV8plQgkAuHAs435ewjRHO3OKWTWsl0A/N/YvrU2h6hJsJ8ns64fQpCPB79szWLeKuMPmJtGd2uUWI0ufrEM6BzC3WN6ckG/9o3yPEII+xSaK1ufO7fif0yfSKZf3g+AR79N4i99wKnxCCHEyTT5K6bWegmw5LjHnqz2/VdAjW3MtdZ7MBpWHP94sy6ZEKKxWa1W/u+7LZRVWLk6rhODurSp9z26RwTw7qQ4rv9oDRUWK6f3jqB3ZGAjRGuIDPJh0Z0jG+3+Qgj7VSZS/k2wh1RdJg2PIv1wMe/+tZM756/n81tPo1/HYGeHJYQQJ5AFCUK0AL8nH+AvnU2gtwcPXlDznlD2GNUrnBev7E9UmJ+sTxCiFSlqBqV91f3nPMVlAztQWFrBDR+vJe2w7HYihGh+mscrphCiTlarlcLSCo4Ul5FbVMqR4jLyisvILSrj3b92AnDvub1PeU3T1UM6c7Wd66uEEC1DYalzm00cz83NxMtXxZKVV8KqXYeY+tFavr5tRJ0NdIQQoilJIiVEM5Kdb2b17oNk5paQlVfC/rwSDuSZyco3fi4ps9R6ba+2AVx3mmM77AkhWofmskaqOm8Pdz6YMoSr31/J9qwCpn2yjrk3DcPbo3kke0II0XxeMYVwQVarlR+T9uNXUkZDdg6yWq3szC7gl61Z/LY1iw37crFaaz/f28ONNn5ehPh5EuTrSYivJyF+nrTx82L80M54SvtwIUQDFJptpX3NYI1UdcG+nnx0wzCufHcFq3cf4tFvkvjv+AHODksIIQBJpIQ4JT9vyeKO+esJ9HZjbmRnu5s8JKUfYfGmDH7dmsXunMKqx7083IjvHkaPCH8ig3yIDPK2/Wt8BTST9QtCiJalqKq0r/m9xnQM8WX21KGMe28lX69P46w+EVwS28HZYQkhhCRSQpyKz9YYbcLzzRaunbmadycP5izVttbzyyosvP7rdt77e2fVzFOInydj+rTlvJhIRveKaJZ/yAghWrbmsI/UyfTtEMxjF8fwxMIkHvs2iSFRobQL9nF2WEKIVq55vmIK4QLSc4tZuiMbL3c3RnTx5a/dhdw8Zx0vj4tlXFynE85PO1zEvxZsJCH1MG4mmBwfxcWx7RkS1QYPKckTQjjR0TVSzau0r7rJw7vw29Ys/t6ezQNfbWLujcMwmezfL08IIRxN/noTooG+WpeG1Qrn92vHg6PbcseZPaiwWLn/y03MWLrzmHN/SsrkojeWkZB6mHZBPnx2SzzTL+9HfPcwSaKEEE53dI1U8/181WQy8cpVsYT4ebJsRw5z/0l1dkhCiFZO/oITogEqLFa+WLcPgIlDO2MymXjwgj48eUkMAM8v2cZzP2ylpKyCJxYmcdu89eSVlHNOdFt+/NdohncPc2b4QghxjOa8Rqq6tkE+vHBFfwCeX5JMyoECJ0ckhGjNJJESogFWpOSQnltM51BfTquWFN04qhtvTByIp7uJmct2E//C73yyKhVPdxNPXhLDzOuG0Mbfy4mRCyHEiSrXSPk1k32kTubC/u0ZN7gT5nIL//58I2UVtW8LIYQQjUkSqSby1u87mDZ3HaXl8oLfEny+1piNGh/XGTe3Y2v0LxvYkQ+vH4qflzu5RWV0DfPjm9tHcuOoblLPL4RolirXSDXn0r7qnhobQ8cQXxLTj/DW7zucHY4QopWSRKoJlFVYeOevFH7ZmkVC6mFnhyNO0aHCUn7Zuh83E1w15MSmEgCn947g2ztG8vjF0Xx/z2j6dwpu4iiFEMJ+rtBsorogH0/+O34AJhO8/WcK6/fKe6sQoulJItUEtmXmU1JmzERtSst1cjTiVH2zPo2yCitn9I6gfbBvreepdoHcPLq77P0khGj2imylfa70ehXfPYxpo7tjscK/P99YlQwKIURTkUSqCVT/pGzjXkmkmsJna/Zyzmt/s3bPIYfe12q1VpX1TRjaxaH3FkIIZ6lsNuEKa6Squ++83vRpF0jqwSL+99t2Z4cjhGhlJJFqAhuqJ1L7JJFqbNv25/HkoiRSDhRw+7z1ZOWV2H2txWI96fH1e3PZcaCA8AAvzo6ufeNdIYRwJQUutkaqkreHO69cZZT4zV6xh23785wdkhCiFZFEqgmst81CmUywP6+E/Ufs/8Ne1E9ZhYX7v9hEWYUVPy93cgrM3Dl/vV1dneb+s4f+T//Ms99vpaKWhOoL22zUuLhOeMr+T0KIFqDCYq0qP/f1dK0ZKYD+nYKZEh9FhcXK498m1fmBmBBCOIr8JdjIcgrM7D1UhJ+XO8O7hQIyK9WY3v1zJ1sy8ujUxpef/nU67YJ8WJd6mBeWbDvpdR8u382Ti7ZQWFrBrOW7uWnOWvJLyo45p8BcznebMwAYP6Rzo/0OQgjRlKr2kPJyP6ELqau4/zxFeIAX61IP8/X6NGeHI4RoJSSRamTrbV36YjsFExfVBpCGE41lS8YR3vrDaIP78lWxdAnz451Jg/F0NzF7xW4Wb8qo8bpZy3Yx/futANwyuhtt/Dz5S2cz7r2V7DtUVHXe95syKCqtYFjXUHpEBDT+LySEEE2gqGoPKdcq66su2NeTxy6OBuCFH7eRW1Tq5IiEEK2BJFKNrLKsb3CXNgzoFAJIw4nGUFpulPSVW6xcf1oUI3qEAxAX1YYnLokB4OGvN7M9K/+Y62Yu3cWzPyQD8NwV/Xjs4hgW3TmKnm0D2J5VwGXvrGCdrWHFgqomEzIbJYRoOY6uj3K9sr7qLh/YkeHdQjlUWMrLP2tnhyOEaAUkkWpklR37Bndpw8AuRiKVmH6k1jU4omHe/mMH2/bn0yXUj4cu7HPMsSnxUVwxqCNFpRXc9klCVcneB3/v5LklRhL1wpX9mTQ8CoAuYX58c8cITu8dwaHCUq6duZrXftFs3JdLoLcHF/Vv37S/nBBCNKIis21GysUaTRzPZDLx7OX98HAz8dmavcc0ehJCiMYgiVQjKquwsNlWxjeoSwhtA33oGOJLgbmcndkFTo6u5UhMO8I7f+3EZIJXrx5wwh8DJpOJ56/oT592gezKKeSBLzfz3l87eeFHY93Ui1f255phx7YyD/LxZPb1Q5g6oiulFRbe/CMFgLEDO+Dr4p/aCiFEdYW2NVKutIdUbXpFGvv3Wa3w+MIk+dBSCNGoXP9Vsxmr3Ii3a5gfYQHeAAzoHEx6bjEb9+bSOzLQyRG6PnN5Bfd/uZEKi5UbR3ZjmK2hx/F8vdx5f3Icl769nJ+27OenLfsxmeClK2MZX0upnoe7G0+P7UvPtgE8tXgLFRYrE2XvKCHESSilfIClgDfGe+xXWuunlFLdgAVAGJAATNFaN4uFPK66h1Rt7jm7J4s3prMlI495q1IZ1sbZEQkhWiqZkWpEG/YdLeurNLCzbZ2UNJxwiDd+28H2rAK6hfvzwPnqpOd2DffntfEDAaMV/Uvjak+iqpscH8V3d41i3k3D6d8p2CFxCyFaLDMwRms9ABgIXKCUigdeAl7XWvcEDgM3OTHGYxTYSvtcbQ+p2vh5efDU2L4AvPqz5lBxuZMjEkK0VC3jVbOZquzYNyjqaCLVmhtOWK1WvkpIw81kYlxcp1O+39Lt2bz/d2VJX6xdJXfnxkQy76bheHu6MbRrzbNXNYnpEHQqoQohWgmttRWorN32tH1ZgTHAtbbH5wBPA+81dXw1KbI1m/BrQWXL58VEMqZPW/7YdoCZaw8xcrCzIxJCtERNnkgppS4A3gDcgVla6xePOx4FzAYigEPAZK11mlLqLOD1aqf2ASZqrRc215KJox37Qqoe698pGHc3Ezorn+LSilaz3sZisfLckmQ+XL4bgMFRbegW7t/g+yWlH+H2eQlYrHDXWT2Ji7I/KRrVK7zBzyuEEHVRSrljvBf1BN4BdgK5WuvKqZE0oKOTwjtBoa39uX8LWCNVyWQy8fSlfVmRksNfuwtYkpgpjYKEEA7XpK+atjeXd4BzMd5I1iqlFmutt1Y77VVgrtZ6jlJqDPACRmL0J0aZBEqpUCAF+MV2TWXJxAKl1PsYJRNO/aSv+ka8qtpaKD8vD3pHBpKcmUdSxpF6zYq4qrIKCw99vZlv1qdXPbZgzV4euSi6Qffbd6iIGz5eS2FpBWMHdOC+c3s7KlQhhDhlWusKYKBSKgT4FuODv3ozm80kJyc3OI6SkhK7rk9NM6onivNzT+n5mqOb4trw7uqDPPjlRgLMOUT4t5xksb7s/e+hpZNxMMg4OGYMmvoVZRiQorXeBaCUWgBcBlRPpGKA+2zf/wksrM6+RXEAACAASURBVOE+VwE/aq2LlFImmmHJRPWNeD3cj12KNrBzMMmZeWzcm9viE6mSsgru+nQ9vyUfwNfTnbvG9OSVnzVfJqRx33m98fao34xcblEpUz9aQ3a+mdO6h/HK1bG4uZkaKXohhGg4rXWuUupP4DQgRCnlYZuV6gSkn/xq8Pb2Jjq6YR84ASQnJ9t1ve+eZOAwXTpEEh3ds8HP1xz16WNlbdqfrE0v5oONhXxy4/BW+55h738PLZ2Mg0HGwf4xSEhIqPVYUzeb6Ajsq/ZzTeUNm4Arbd9fAQQqpcKOO2ci8Jnt+zCaYclE9Y14j9daGk7klZRx3ew1/JZ8gGBfT+bfMpw7zuxBdPsgDhWW8lPS/nrdr6SsgpvnrGNndiF92gXywXVx9U7EhBCiMSmlImwzUSilfDEqMJIxPhi8ynba9cAi50R4oqKqZhMt7/XUZDLx75ERhPl7sSLlYFV5uRBCOEJznOP+D/C2UmoqRgvZdKCi8qBSqj3QH/j5VJ6ksUsmlm/LACDCreCE8wLLjOVba3dmu/y0am3jkFtcweO/ZbLzUClhvu48d24kvoX72bZtP2O6eJGcCbP+3EZv7zy7nqfCYuWFpQdYl1pIuJ87j45qQ/rulLo/0m0iMkVukHGQMajUisehPTDHVsruBnyhtf5eKbUVWKCUehbYAHzozCCrq9xHqiWtkaquja8HL18Vy01z1vHKz5oRPcPo20E6sAohTl1Tv2qmA9X7TZ9Q3qC1zsA2I6WUCgDGaa2rT92MB77VWpfZfj5IMyuZKKuwkPLpHgDGjuhftYdUpd4WK34/ZXKgsJzwTt2JCPSu4S6uoaZxyMgt5o5Zq9l9qJSuYX58ctNwOof6VR3v1K2Mjzb8TmJWCZ5hnenZNuCkz2G1Wvm/77ayIrWQQB8P5k8bgWrXvPbgkilyg4yDjEElR5RMuCKt9WZgUA2P78Iob292qmakWmgiBXB2dCST47swb9Ve/rVgI9/fPQofz5Y3AyeEaFpNXdq3FuillOqmlPLCKNFbXP0EpVS4UqoyrkcwOvhVdw1Hy/oqW802q5IJvf/EjXirc3cz0b+j8WnYpn0tq7zPYrFyz2cb2J1TSEz7IL68bcQxSRRAoI8nlw3sAMBna/bWec8Pl+/m45V78HJ3Y8aUIc0uiRJCCFdWOSPVktqf1+Sxi2LoEeFPyoECXljSKmdLhRAO1qSJlG3G6C6MsrxkjJKHLUqpZ5RSY22nnQlopdR2IBJ4rvJ6pVRXjBmtv4+79UPAfUqpFIw1U04tmVi/98SNeI830NYSfVMLWyc1b3Uq61IPEx7gzfybh9c623btsCgAvl6fRklZRY3nACSmHeHFH7cB8Or4AZzW4/jlckIIIU5Foblll/ZV8vVy542Jg/B0NzHnn1T+3HbA2SEJIVxck79qaq2XAEuOe+zJat9/BXxVy7V7qKGRRHMrmahpI97jDapsONGCZqTSDhfxki3pefbyvrTx96r13P6dgunfMZjE9CP8mJTJFYNO3KC3pKyCez/fQLnFyg0juzJ2QIdGi10IIVqrosp9pLxadiIF0K9jMPefp3jxx2088NUmfrr3dMJrqBwRQgh7NHVpX6tQ00a8xxtQLZGyWKxNEldjslqtPPZtEoWlFVzUvx0X9Kt748Nrh3cBYP6qmsv7XvxxGzuzC+nZNoCHLmjQNixCCCHqcLTZRMsu7at0y+juxHcPJaeglGlz11FcWntVhBBCnIwkUg5W20a8x2sf7EtkkDf5JeXsPljYhBE2jm/Wp/P39myCfT15emxfu64ZO6ADAd4erEs9zPas/GOOLduRzccr9+DhZuJ/EwbKomAhhGgkhbZmE36tYEYKjHXKb0wcRMcQX9bvzeX2+QmUVVicHZYQwgVJIuVgJ9uI93gDOtlmpfa6dnlfdr6ZZ7439lR+4pIY2gb62HWdv7dHVdOJT1cfnZXKLSrlP19uAuDf5/amX0dpUyuEEI3l6Bqp1vOBVWSQD3NvGkaovxd/6Wwe+HJTi6gOEUI0LUmkHOxkG/Eer6U0nHh68RaOFJdxeu8Ixg2u317IleV9X69Po7i0wigRXJhEVp6ZuKg23Hp698YIWQghBFBeYcFcbsFkAt9WNvPfIyKAj6YOxc/LnYUbM5j+w1asVkmmhBD2k0TKwezp2FdpYDNqOLEru6BBpQ0rUgv5ITETPy93nr+iHyaTqV7X9+0QzMDOIeSXlPP95gwWb8rgh83G/V4bP6DOWT0hhBANV1R2tNFEfV+/W4IBnUOYMWUIXu5ufLRiD+/+tdPZIQkhXIj8lepA5RUWNttmlwadpNFEpf4dgzGZIDkz76QtwBvbl+v2Mea/fzP+g38osi06tseRojLeWZ0DwEMX9KFTG786rqhZ5azUzGW7eHxhEgBPXhJDVJh/g+4nhBDCPpVlfS19D6mTGdUrnNcnDMRkgld+1nbtbyiEEGBnIqWUOr2xA2kJttWxEe/xAn086dU2gLIKK1sz85ogwhPtP1LCM98Z65s27M3l9nnr7Z6Zem7JVg4XVzAkqg1T4qMaHMOlsR0I9PFge1YB+SXlnBPdlglDOzf4fkIIIexT2Wiipe8hVZeLY9sz/bJ+ADz2bSI/JWU6OSIhhCuwd0bqL6XUFqXU3UopWflfi01Vs1F1l/VVcmbDCaNleSL55nKGdwsl1N+Lv7fXvei2rMLC04u38MW6NDzdTLw4LhY3t4aXhPh6uXPlIGNtVZi/Fy9cGdsqS0yEEKKpFbWy1ucnMzk+ivvO7Y3FCvd8tpFkJ33AKYRwHfYmUhcB24H/AhlKqdlKqeGNF5Zryi0qA6BdsH1d6+Bow4kE29qqprRoYwa/bztAoI8Hb14ziI9vGIp/HYtuDxaYmTxrNR+v3IOnu4l7R4TTs23AKcdy51k9uTi2Pe9NjiMiUDZHFEKIptDaWp/X5e4xPZkwpDOlFRbuXbDRqWX3Qojmz65ESmv9k9b6CiAKeBEYA6xUSm1QSt2qlDr1v6RbgMpN/Xw87P9kb1TPcEwm+HVrFtn55sYK7QTZ+Wae/m4LYLQsjwzyIbZTCDOuq33RbVL6Eca+vYLVuw/RNtCbBdNOY0yP2vfKqo+2QT68c+1ghnULdcj9hBBC1K2q9XkrXiNVnclk4qmxMXQL90dn5fPqz9rZIQkhmrF6NZvQWmdqracD3YCxQBHwLsYs1ftKqT6NEKPLqPzkytfL/mGNCvPn7D6RlJZb+GRVamOFdoKnFieRW1TG6F7hXB3XqerxkT2PXXS7wLboduGGdMa9t5L03GIGdg7hu7tHERdlfwmjEEKI5qfQVtrn18rXSFXn5+XB6xMG4u5mYtby3axMyXF2SEKIZqqhXfvOAq4DhgCHgW+AS4HNSqmbHBSbyykpt81I1XMvjltGdwNg3qrUJikjWJKYyZLE/fh7ufPClf1PWI90cWx7nrEtun3020Run5fAvZ9vxFxuYcKQznx+azyRQfaXLwohhGieimyVFAFS2neMgZ1DuHtMTwDu/3ITR2yl+0IIUZ3diZRSKkwpdb9SSgO/AV2BaUBHrfVU288fA884PEoXUVxqdLurT2kfwLBuocR2CuZQYSlfr09rjNCqHC4s5clFRovxhy+KrrVl+ZT4KO49pxcWK/yYtB8PNxPTL+/Hi+P6413P308IIUTzVNX+XJpNnOCus3oysHMImUdKeML2vimEENXZ2/58PpCGkSQtB4ZqrYdrredorc0AWusyYA7QvrGCbe6qZqTqWWtuMpm4eXR3AD5ctvukHfNO1TPfbyWnoJTh3UKZNKzLSc/919m9uPOsHvRpF8int8QzJT5KuukJIUQLUtX+XGakTuDh7sbrEwbi6+nO4k0ZLNqY7uyQhBDNjL0zUkOARzFmn27SWifUcl4icKFDInNB5rLKZhP1r5i8qF87Oob4siunkD+2HXB0aAD8npzFtxvS8fF04yU7WpabTCYeOL8PP917ujSBEEKIFqioVGakTqZbuD9PXBIDwOMLk8jILXZyREKI5sTern1Ka/261vqkmx1prfO01j87JjTXU1zWsDVSYHzydcPIrgDMWLbLkWEBsG7PIe7/chMA/zlP0TXc3+HPIYQQwrVUNpsIkGYTtbpmWGfO7tOW/JJy7v/i5PssCiFaF3tL+85XSk2r5dg0pdS5jg3LNZWUGWukfBvYRnbC0M4EenuwZvchNqc5boPeHzZncu2s1eQWlXFOdCQ3jOzmsHsLIYRwXUWyj1SdTCZj4/kwfy/+2XWQWcsd/2GnEMI12VuD9jQQVsuxEOAph0Tj4krK6r+PVHWBPp5cM9xYtzRz2e5TjsdqtfLB3zu589P1lJZbmBzfhfcnD8a9jpI+IYQQrUPljJTsI3VyEYHevDQuFoCXf9Ks2X3IyREJIZoDexOpGKC2dVEbgL6OCce1HS3ta2hXeZg6oisebiaWJGaSfgq12OUVFp5ctIUXftwGwMMX9mH6Zf3wcG94bEIIIVqWqmYTUtpXp3NiIpl2enfKLVbu/HQ9B/JKnB2SEMLJ7P2r2h3wreWYH+DlmHBcm9lW2teQNVKVOoT4cnFseyosVj5a3rBZqaLScm79JIFPVqXi5e7GW9cM4rYzekjHPSGEEMeompGSZhN2efB8RXz3ULLzzdwxfz1lFRZnhySEcCJ7P4JKBCYCi2o4NgGQDRaoVtp3CokUwC2ju7NoYwYL1u7jnnN6EeTjWXWswFzOksRMFm1M51BhGQHe7vh7e+Dv7UGg7d/Vuw+SlJ5HiJ8nM68bwtCu0nFPCCHEiWSNVP14uLvx1jWDufSt5axLPczzS5J56lIpyhGitbL3lfN1YIFSqhyYibGnVEeMDXknANc2TniuxRGlfQD9OgYT3z2UVbsO8fmafdw0qhurdx/iy4R9/Ji4v+p5TqZzqC8f3zCMHhEBpxSLEEII+ymlOgNzgUjACszQWr+hlHoauAXItp36qNZ6iXOiPOroGilJpOwVEejNO5MGM3HGP3y0Yg8DO4dw2cCOzg5LCOEEdr1yaq2/UEp1wdiQt3rSVAI8qLX+vDGCcyVWq9VhM1JgzEqt2nWI9//eyZx/9pB2+Oh6qWFdQ7kqrhMxHYIoMJdTaC63/VtBgbkMN5OJKwZ1JCzA+5TjEEIIUS/lwP1a6/VKqUAgQSn1q+3Y61rrV50Y2wkKzVLa1xBxUW148pIYnli0hYe/TkS1C6RPuyBnhyWEaGJ2fwSltX5VKTUbGI3RwS8HWKa1PtxYwbmSsgorFit4uJnwdEBDh7NUW7pH+LMruxAKoUOwD+PiOjFucCfZA0oIIZoprXUmkGn7Pl8plYxRwdEsFZZKs4mGmhwfxYa9uXyzIZ3bPklg8d2jjinFF0K0fPV65dRaH6LmdVKt3qlsxlsTNzcT/5swkEUbMxjTpy2ndQ/DTdqWCyGEy1BKdQUGAauBkcBdSqnrgHUYs1ZO/SCyrMJCabkFNxN4e0hH1/oymUw8d0V/kvfnk5yZx/1fbOKDyXHyXi1EK1KvREop5Q/0AHyOP6a1XmPnPS4A3sDoBDhLa/3iccejgNlABHAImKy1TrMd6wLMAjpj1J5fpLXeo5T6GDgDOGK7zVSt9cb6/G6nyuzgRAogtlMIsZ1CHHY/IYQQTUMpFQB8Ddyrtc5TSr0HTMd475oO/Be48WT3MJvNJCcnNziGkpKSk15fUFq596GJbdu2Nfh5mru6xuFU/Sc+mH/9UMCvW7OY/tUqxvdvnu/bjT0OrkLGwSDj4JgxsCuRUkp5Ae8DkzESoJrUmUEopdyBd4BzMRpWrFVKLdZab6122qvAXK31HKXUGOAFYIrt2FzgOa31r7Y3qep9Rx/QWn9lz+/TGEqqWp/Lp3pCCNGaKaU8MZKo+VrrbwC01lnVjs8Evq/rPt7e3kRHRzc4juTk5JNen5FbDKQS5Htqz9Pc1TUOpyoaeCMokpvmrGPOhkOcF9eb03qENdrzNVRjj4OrkHEwyDjYPwYJCbVtpWv/PlKPAhcDtwMm4H7gLmAtsBO40s77DANStNa7tNalwALgsuPOiQH+sH3/Z+VxpVQM4KG1/hVAa12gtS6y83kbnaNL+4QQQrgepZQJ+BBI1lq/Vu3x9tVOu4JmsG1Ika1jn580mjhlZ0dHcseZPbBY4e7PNshmvUK0EvYmUhMwOvZ9bPt5qdb6Pa11PLAVON3O+3QE9lX7ubKNenWbOJqYXQEEKqXCgN5ArlLqG6XUBqXUK7YZrkrPKaU2K6VeV0o1ebu6yo59vpJICSFEazYSo4pijFJqo+3rIuBlpVSiUmozcBbwb6dGCRTa9pCS1ueOcd+5vYnvHkpOgZm7PttAuWzWK0SLZ++rZxSQqLWuUEqVAX7Vjs3A+PTtfgfF9B/gbaXUVGApkA5U2GIdjbFwdy/wOTDV9tyPAPsBL1s8D2EkfrVydO253m+0J7eUndp9XY3U2BpkHAwyDjIGlVrrOGitl2NUbhzP6XtGHa9yDyk/L/kA0BE83N1485pBXPzmctbsPsSrv2zn4Qv7ODssIUQjsjeROghU7uyaBsQCy20/hwD29uNOx2gUUamT7bEqWusMbDNStnVQ47TWuUqpNGCj1nqX7dhCIB740NZuFsCslPoIIxk7KUfXnme5HQAyCQ0OaFU1p1Jja5BxMMg4yBhUckTtuWhclTNSAdL63GHaBvrw9jWDuHbWat7/eydxUW04NybS2WEJIRqJvaV9azGSJ4CFwDNKqX8rpe4GXgFW1uM+vZRS3WwNLCYCi6ufoJQKV0pVxvUIRge/ymtDlFIRtp/HYJQVVtWe22rTL8cJtedHm03IJ3tCCCGav6NrpCSRcqTh3cN48HwFwP1fbGTvwWaznFsI4WD2JlIvA3ts30/H2BPjVYw25lnAHfbcRGtdjtGk4mcgGfhCa71FKfWMUmqs7bQzAa2U2g5EAs/Zrq3AmGn6XSmViFE6MdN2zXzbY4lAOPCsnb+Xw5jLpdmEEEII13F0jZS8bznatNO7c25MJHkl5dzxaULVOmohRMti18dQWutVwCrb97nAxUqpQMBXa32gPk+otV7CcbXiWusnq33/FVBjG3Nbx77YGh4fU58YGkNx1X4c0v5cCCFE81c1IyXNJhzOZDLx6tUDuOStZSSl5/HkoiReGheLySSb9QrRktT5V79SyksptVIpdU71x7XW+fVNolqyqq598smeEEIIF1BgNhKpAGl/3iiCfT15b1IcXh5ufLEujScXbcFqtTo7LCGEA9WZSNn2e4rB6JwnalFSLmukhBBCuI4iWyWFrJFqPP06BvPBFCOZ+mRVKk8sSsJikWRKiJbC3jq0P4CzGzMQVyelfUIIIVxJoW1GStZINa6zVFtm2JKpeav2SjIlRAti78dQLwOfKaXA6NqXCRzzKmBrW95qlVQ2m5A3JCGEEC6gakZK1kg1ujNVW2ZdN4Rb5q5j/uq9WKzw3OX9cHOTNVNCuDJ7p09WYmzK+yhGx769wL7jvlo1c2X7cw9JpIQQQjR/lWuk/KW0r0mc3juCWdcPwdvDjc/W7OWxhYkyMyWEi7P31fMOjpuBEseqKu2TNVJCCCFcQGXXPn9pNtFkRveKYPbUodw0Zy2frdmH1QrPX9FfZqaEcFH2tj9/v7EDcXWVpX2+XrJGSgghRPNXuY+UlPY1rZE9w5l9/VBunLOWBWv34enuxjOX9ZXW6EK4IPmr30Eq259LaZ8QQghXIDNSzjOiZzizpw6t6ub35u8pzg5JCNEAdn0MpZR6t45TrFrrOx0Qj8sqLpP250IIIVxH5YyUv8xIOcWIHuG8dc0gbp+XwOu/bSc0wIsp8VHODksIUQ/2vnpeyYlrpEIAb6DA9tWqE6mqGSlJpIQQQriAwlJpNuFs5/dtx/NX9OfhbxJ5clESoX5eXBzb3tlhCSHsZO8aqXbHP6aUMgHnAq8DEx0cl8sxVyVSUi0phBCi+SuqWiMlHwA608RhXThYWMorP2vu/XwDwb6ejOoV7uywhBB2aPBf/Vprq9b6F+BN4G3HheSaimVGSgghhIsoLbdQWmHB3c2Et2wk73R3nNmDG0d2o6zCyq2frGNzWq6zQxJC2MERr54pwBAH3MelldjWSPlKIiWEEKKZq2o04eUu3eKaAZPJxOMXR3P5wA4UllYw9aO17MoucHZYQog6nFIiZSvvmwRkOCYc1yVrpIQQQriKQtveh7I+qvlwczPxytUDOFNFcKiwlCkfriH1YKGzwxJCnIS9XfuW1PCwF9AHaA/c48igXFGxrJESQgjhIorMxoyUrI9qXjzd3Xh30mCmfLiGhNTDjP/gH+bfHE/PtgHODk0IUQN7/+oPBdoc9+UO/AZcrLV+p3HCcx1maX8uhBDCRciMVPPl5+XBnBuHMaxbKFl5ZibO+IfkzDxnhyWEqIG9XfviGzsQV1ZhsVJaYcFkQhbtCiGEaPYKzZVrpCSRao4CvD2Yc8Mwpn2yjmU7crhm5irm3jiM2E4hzg5NCFGN/NXvAJXro7w93GTRrhBCiGavKpHyliqK5srXy51Z1w/hnOhIcovKmDRzNev2HHJ2WEKIauxKpJRSLymlPqrl2EdKqecdG5ZrqUykpGOfEEIIV1BUWrmHlMxINWfeHu68N3kwF8e2J99czpQP17AyJcfZYQkhbOydkRoH/FXLsT+BqxwSjYsqKZf1UUIIIVxHYanMSLkKT3c33pw4iHGDO1FcVsENH6/l7+3Zzg5LCIGda6SAjkBqLcf2Ap0cE45rKi6V1udCCCFAKdUZmAtEAlZghtb6DaVUKPA50BXYA4zXWh92VpyyRsq1uLuZeOWqWHw83Zi/ei+3fZLAZ9PiGdhZ1kwJ4Uz2zkjlAd1qOdYdaNUbHcgeUkIIIWzKgfu11jFAPHCnUioGeBj4XWvdC/jd9rPTFJptpX3Stc9luLmZePbyflwVZ8xM3fixbNorhLPZm0j9Djxm+0Stiu3nR2zHWy1zuewhJYQQArTWmVrr9bbv84FkjKqOy4A5ttPmAJc7J0JDUWVpn+wj5VJMJhMvXNmfs2yb9l43ew0H8kqcHZYQrZa9f/k/gbGXVIpSarZS6hml1IdAiu3xxxsrQFdQXGpbI+Uhb0hCCCEMSqmuwCBgNRCptc60HdqPUfrnNJX7SMmMlOvxdHfjnUmDGdA5hLTDxVz/0VrySsqcHZYQrZK9+0jtVEoNA54HrgCCgVzgF+BxrXVK44XY/FV17ZNP9oQQQgBKqQDga+BerXWeUqrqmNbaqpSy1nUPs9lMcnJyg2MoKSmp9fqMA0Yb7bycLJKTixr8HK7gZOPgyh4ZEcz9PxaSnJnH5PeXMv2c9ni5174FS0sdh/qScTDIODhmDOz+KMqWLI0HUEq5aa0tDXlCpdQFwBuAOzBLa/3iccejgNlABHAImKy1TrMd6wLMAjpjLOK9SGu9RynVDVgAhAEJwBStdWlD4muIEintE0IIYaOU8sRIouZrrb+xPZyllGqvtc5USrUHDtR1H29vb6KjoxscR3Jycq3Xe6wpBAro2a0L0dHtGvwcruBk4+DqFkR158r3VrJ5fwkzNpXw5jWDcHerOZlqyeNQHzIOBhkH+8cgISGh1mP27iPVxpbgAFA9iVJKRSml7Gobo5RyB94BLgRigGtsi3CrexWYq7WOBZ4BXqh2bC7witY6GhjG0Teil4DXtdY9gcPATfbE4yhVXfuktE8IIVo1pZQJ+BBI1lq/Vu3QYuB62/fXA4uaOrbqjq6RktI+V9Y51I85Nwwj0NuDHxIzeea7LVitdU52CiEcxN4plPcx1knV5DHgPTvvMwxI0Vrvss0YLcBYgFtdDPCH7fs/K4/bEi4PrfWvAFrrAq11ke1Nawzwle2aJl/EW7WPlJT2CSFEazcSmAKMUUpttH1dBLwInKuU2gGcY/vZaY6ukZL3LVcX0yGID66Lw8vdjTn/pHLfF5uqlhwIIRqXvR9FjQburuXYT8Cbdt6nI7Cv2s9pwPDjztkEXIlR/ncFEKiUCgN6A7lKqW8wWrH/htE+tg2Qq7Uur3bPjnbG4xDmMpmREkKIlsRWQdEHiAUGALFa64vquk5rvRyobaHK2Y6L8NRU7iMVIM0mWoQRPcJ5d9Jg7v5sA99uSGdXdgEfTBlCu2AfZ4cmRItm7ytoKEbJXE1ygXDHhAPAf4C3lVJTgaVAOlCBEetojA5IezE2NpxKA8sjHLmId2+6MTSFeYdb3cI9WaxokHEwyDjIGFRypXFQSoVjS5aq/RsNeGEkRWYg0WkBNoIiWyLlJ5UULcY5MZF8c8cIbpm7jk1pR7j07eW8PzmOuKg2zg5NiBbL3kQqHRjK0ZK76oZitHK19z6dq/3cyfZYFa11BsaMVGXXo3Fa61ylVBqwUWu9y3ZsIcZmh7OBEKWUh21W6oR71sSRi3gD924DDtOpXVuio3s1+J6uSBYrGmQcDDIOMgaVHLGIt7EppW7AWIvbodrDhcAajPW8G21fyVrrFlUrVVnaJ2ukWpbo9kEsvmsUd85fzz+7DnLNjFU8e0U/xg/pXPfFQoh6s3eN1LcYG/IeU5Zg+/kRjO5E9lgL9FJKdVNKeQETMRbgVr9nuFKqMq5HMBKlymtDlFIRtp/HAFu11laMtVRX2R5v8kW8JWW2NVKe8smeEEK4kBcxEqeHMUrJlwPewN/AQ1rreVrrpJaWRMHRZhOyRqrlCfX3Yu5Nw5g6oiulFRYe/GozTy/eQoVFmlAI4Wj2JlJPA9uBX5RSu5RSS5VSOzH2kdoBPGXPTWwzRncBP2Ps9v6F1nqLbYPfsbbTzgS0Umo7xoaFz9murcAo+/tdKZWIUW4x03bNQ8B9SqkUjBboH9r5ezlEcZm0PxdCCBcUAdyhtX5Fa71Ya30GcB/wALBaKdXPueE1DnN5BWUVVjzdTXjL2t4WydPdjafH9uWlcf3xdDfx8co9lzBTGgAAIABJREFUPPJLJgfySpwdmhAtir0b8hYopUZitBU/FyNZ2QS8AnyktTbb+4Ra6yXAkuMee7La919xtAPf8df+ilG7fvzjuzA6AjpFSVUiJW9IQgjhQroD2dUf0Fq/o5T6AWPPwnVKqenACw3dO7E5KjLbOvZJWV+LN2FoF3q2DeC2eetJzCrhojeX8fqEgYzuFVH3xUKIOtVnQ14z8K7t6xhKKa+m3AC3uTFLaZ8QQrgcrfWekzx+jlJqGvAycLlSaqrWeksThtdoCqv2kJL3rNYgLiqUH+4ZxbTZK9mYWcx1s9dw55k9ufecXni4SyWNEKfilP4PUkqdppR6D8h0UDwuqVhmpIQQosXRWs8A+gE5gPO6YjhYUWWjCWl93mq0DfTh2XPacd+5vTEBb/+ZwrUzV7P/iJT6CXEq6v0qqpTqgrHZ4HVAT6Ac+M7BcbmUytI+X0mkhBCiRdFapwEXKqWud3YsjlJQ2fpcEqlWxd3NxD1n92Jo11DuWbCBNXsOcdGby3ht/ADOVG2dHZ4QLsmuGSmllL9S6nql1B/ALmA6RhL1OtBJa33VSW/QwpVIswkhhGjRtNb/396dx1dZnvkf/2RPIKwJIBBkEbgERVBQUcEKtR1b0Wpb27p0pGPbaaud2o5tdX7T1rHT1u621Zkuikun1TpqHUXrjgJqKSAiS7xYAkLCFogJWxYS8vvjeU6IKUtyclbO9/168eKc5zznea7cryR3rnPf93Xfn+wYYiWyRkpT+zLTOSeV8JevTGf6mFJq9jUx+97F/PjZt1XVTyQKR/3L38zeb2YPEOwTdS9wCvArgt3Zs4An3L36KJfICPVaIyUiImkiskZKxSYyV2lxAfd/5ixu+uBYsrPgrnnr+cc5i9i5t9O1w0SEoyRSZraJoLz5FcAzwEeAoe7+VeCNxISXHhq1RkpERNJEZA+pYu0hldGys7O4YeYY/ue6syktzufVdbuY9cuFLH2nJtmhiaSNo41IlRGMOi0A7gOeCveBkg40tU9ERNLF3kj5c62REuDc0aXM/fJ0pgzvx7bdDXzyN39lzsINtLZqqp/IsRztL/8Lgd8D5wBPAFvM7GdmNikhkaURVe0TEZF0sb9R5c/lvU7oU8iDn5/KZ6eNpPlgK7fNXc0NDy5rK0wiIod3xETK3V9y92uBEwg24i0HvkJQAnYR0AoUJyLIVNcQrpFS1T4REUl1+5q0Ia/8vbycbP591nj+6+ozKC7I5am3tnLpnQtZX7032aGJpKxjzkVz933ufp+7zwBGAreG78sC/s/M5prZR+MbZupqbW2loVkjUiIikh4iI1LFmtonh/HhCYN54obzsEG9qKjex2V3vsoLq7cnOyyRlNSlRT3uvsndv+vuY4FpBJX8zgP+Nx7BpYPG5oO0tkJeThY52VnJDkdEROSo2qr2qdiEHMGoAcU89qVz+fCEE9jT2MxnH1jCL19cy0GVSBd5j6irI7j7a+7+eYKpf1fFLqT00qjS5yIikkb2te0jpREpObKeBbncddUZfP0fjKws+Nnza/jC/yzVuimRdrpdZs7dG939T7EIJh1pWp+IiKST/W37SKnfkqPLysri+hmjmTP7THoV5vLc6u1cfterbNi5L9mhiaQE1evupvomlT4XEZH0ERmR0hop6awZNpAnbpjG6IHFrN2xl0vvXMgzK7clOyyRpNNf/90UGZFSxT4REUkHh9ZIKZGSzhtZ2pPHrz+PD44fxJ6GZr7wP0v514eXs7vhQLJDE0kaJVLd1KA1UiIikkb2aR8piVJxQS6/vmYy37lkPAW52Tz6RiUfumMBr63fmezQRJJCiVQ3tU3ty1WHJCIiqW9/ZB8pjUhJFLKzs/jMeSN56l+mcVpZH6pq67nqd4u47cnVNBxoSXZ4IgmlRKqb2opN6JM9ERFJA5ERqWJV7ZNuGD2wF49+8VxuvHAMOdlZzHl1A7N+tZAVlXXJDk0kYTr9W9TMPglcCZwIFHZ4udXdT4llYOmi8UBkREo5qYiIpLbW1lb2hSNSRfoAULopLyebGy8cy8yTB/LVP73Juh17uey/XuWz00dy4/vH6ntMjnudSqTM7Gbg+8BaYBXQGM+g0kn9AZU/FxGRgJnNAWYBO9z91PDYrcDngOrwtH9z96eTEV9j80FaDraSn5NNvj4AlBg5rawvT/3LdH70jHPvaxv4zSsV/GXFNr5/+QSmjSlNdngicdPZEanPA//l7jfEM5h0FCk2oap9IiIC3AfcCTzQ4fjP3f0niQ/nvQ6tj1KfJbFVmJfDty8Zz6yJg7nl0RX49j1cc88iPnrGUL518Xj69cxPdogiMdfZj6MGAo/FM5B01XBA+0iJiEjA3ecDNcmO40gOVezT+iiJjzNO7Mfcf5nG1//ByM/N5rE3qnj/z17h8WVVtLa2Jjs8kZjq7F//C4FT4xlIutLUPhER6YQbzOwtM5tjZv2SFURkD6meGpGSOMrLyeb6GaN55ivTmTqqPzX7mrjxT29y1e8WsbJKxSjk+NHZj6RuAB41s23A0+6+N44xpRXtIyUiIsfw38B3gdbw/58C/3SsNzU2NlJeXh71TRsaGv7u/eU7GgDIajnQrWunk8O1QyZKVjt8e1ofnjshm7uX1PB6xS5m/WohM0cVc+3p/RlYnPiRUX0/BNQOsWmDzn4HlwNZwIMAZtZxo4BWdy/oViRpqlEjUiIichTuvj3y2Mx+B8ztzPsKCgoYN25c1PctLy//u/fvzK0GtlDat7hb104nh2uHTJTMdhg/HmZf2MSdL63jgdff4aWKvSzctJ9/Om8kX5pxEr0L8xIWi74fAmqHzrfB0qVLj/haZxOpnxJ8ktZtZnYR8AsgB7jb3W/v8PpwYA4wgGCe+TXuXhm+1gKsCE/d5O6XhsfvA94HRMaLZ7v7m7GI91jqtUZKRESOwswGu/vW8OnlwMpkxbKvMeiztEZKEq1vj3z+fdZ4rj13BD961nly+RZ+/cp6Hl6ymX+ZOZqrzh6uSpKSdjr1m9Tdb47FzcwsB7gL+ABQCSw2syfcfXW7034CPODu95vZTOAHwKfD1+rdfdIRLv91d38kFnF2RaTYhKr2iYiImT0IXACUmlkl8B3gAjObRPCB5Ebgn5MV3+6GAwAUFyqRkuQY1r8Hv7rydK6bNpLvP1XO3zbWcOuTq7nn1Q386weMSycOITs7K9lhinRKon+TngWsc/cKADN7CPgI0D6RGg98LXw8D3g8oRF2kdZIiYhIhLtfeZjD9yQ8kCOo2x8kUn2LVIpakmvSsL786Z+n8vzq7fzoWWfdjr3c+Kc3+fUr6/n6PxgzTx5IVpYSKkltR0ykzOwbBCND28LHR9Pq7j/uxP2GApvbPa8Ezu5wznLgowTT/y4HeplZibvvAgrNbAnQDNzu7u2TrO+Z2beBF4Gb3T0hmwarap+IiKSLuvowkeqRuDUpIkeSlZXFB085gfePG8Rjb1Ty8+fX8Pa2PVx3/xKmDO/HNz90MmeO6J/sMEWO6GgjUrcDLwPbwsdH0wp0JpHqjJuAO81sNjAfqAIixS2Gu3uVmY0CXjKzFe6+HrgljDMf+C3wTeC2o90kVtWQdtXuBqB6WxXlOe9Gfb10paovAbVDQO2gNohQO6Sm2vomQImUpJac7CyumDKMSyYO4Q+LNnHXvHUseeddrvj161w4biDfmjWe4SU9kx2myN85WiJV1G5UpyhG96sChrV7XhYea+PuWwhGpDCzYuBj7l4bvlYV/l9hZi8DpwPr2y3ibTSzewmSsaOKVTWk3FdqgXrGjBrBuFElUV8vXanqS0DtEFA7qA0iYlENSWKvNpza16dIiZSknsK8HK6bNpJPTCnj7gUbuHtBBS+U72D+2p184fxRfPGC0RTlawaQpI4jJlLtp8bFcJrcYmCMmY0kSKA+BVzV/gQzKwVq3P0gwUjTnPB4P2C/uzeG55wH/Ch8bbC7bzWzLOAyElgRqV7FJkREJE0cmtqnNVKSunoV5vHVD4zl6qkncvvTb/PYsip++dI6Hn2jim9fMp4Pjh+k9VOSEhJaZ9Ldmwk2932WYG+qh919lZndZmaXhqddALiZrQEGAd8Lj48DlpjZcoIiFLe3q/b3BzNbQVAavRT4z4R8QRyq2qc1UiIikupq24pNaERKUt/AXoX87JOT+N8vnMPJJ/Siqraef/79Umbfu5iK6r3JDk+k81X7zOwfgS8CBhR2fN3de3TmOu7+NPB0h2Pfbvf4EeDvypi7+2vAhCNcc2Zn7h0PDc3aR0pERNJDZI2UpvZJOjlzRH/mfnkaf1i0iZ8857yyppqL7ljAFVPK+Pz5o7R+SpKmU3/9m9mVBOVb1wJ9gUeBJwmq51USVNjLSPVNQflzTe0TEZFU1zYipWITkmZyc7K59twRzLvpAq6YXEZTy0H+sGgTM37yMtf/8Q1WVtUlO0TJQJ0dRvlX4IfAZ8LnP3f3TwKjgQMEGwxmpMZwal+BEikREUlhLQdb2dPQTFZWsAZFJB2VFhfw4ysm8vxXz+eKyWXkZGfx1FtbmfWrhVxz9yIWrK2mtbU12WFKhuhsIjWWYF3SQYJS5/kA7r4D+C6HNtDNOJraJyIi6WB3WGiid2EeOdlaqC/pbcygXvz4ionM/8YMPjd9JD3zc1i4biefvudvzPrVQh57o5LG5pZjX0ikGzr7138DgLu3EuzXNKLda3UEZcwzTnPLQQ60tJKdBfk5SqRERCR11WozXjkODe5TxP+7eDyv3fx+vv4PRmlxPqu27OZrDy9n2g/n8csX17Jzb6yKT4u8V2eLTawmmMb3IvAqcLOZOcEaqW8Da+ITXmpraA7WRxXm5agMp4iIpLTa/eFmvCo0IcehPj3yuH7GaK6bNpIn3tzCnFc38Pa2Pfzs+TXcOW8dl00awmfOG5nsMOU409lE6h4g8t33LYKEakn4fD/hBrqZRqXPRUQkXURGpPpoDyk5jhXm5fCJM4dxxZQyXl+/izmvbuTFt7fz8JJKHl5SycQTCrkxq4T3jR1Atqa4Sjd1KpFy99+3e+xmdgowHegBLHD3rXGKL6XVN2kzXhERSQ91YcU+lT6XTJCVlcW5o0s5d3QpG3fu477XNvK/SzazfFsDn7lvMaMHFvPZaSO57PSh+kBconbMRMrM8oH/AB5x96UA7l4HzI1zbCkvsoixQIUmREQkxdXVazNeyUwjSnty66Wn8NUPjOWXc5fw1Nr9rNuxl5sfW8FPnnM+PXUE10w9kZLigmSHKmnmmBmAuzcBXwG021kHDQfCNVK5+iRDRERSm/aQkkzXpyiPj5/alwXfnMEdn5zEKUN6s3NvEz9/YQ3n/fAlvjt3NdV7VJhCOq+zQynLgfHxDCQd1YdrpIrylUiJiEhqq60Pik1oap9kurycbC47fShzvzyNBz83lZknD6ThwEHuWbiB6T96ie8/Xa5Kf9IpnU2kvgF808wujGcw6eZQsQlN7RMRkdRW1zYipWITIhCsozrnpBLmzD6TuV+exoXjBtFw4CC/nV/B9B/O4wd/KadmX1Oyw5QU1tmqfXOAvsCzZrafYC+p9ttGt7q7xTq4VKepfSIiki5qtUZK5IhOHdqHu6+dworKOu54YQ0vvr2D37xSwe9ff4dPTBnGJRMHc/qwfqr0J+/R2URqKe9NnIRDU/sKNbVPRERSXNs+UlojJXJEE8r6cM/sM1m+uZY7XljDPK/mvtc2ct9rGxnSp5CLTxvMrNOGcFpZH+0hKp0uf/6peAeSjtqm9mlESkREUlzbPlIakRI5ponD+nLvZ85iZVUdjy+r4qkVW9lS18DvFmzgdws2MKx/ERdPGMKs0wZzypDeSqoy1BETKTOrAC539+UJjCetNGqNlIiIpIndbRvyKpES6axTh/bh1KF9+LcPj+ONTe8y962tPL1iK5tr6vn1K+v59SvrGVnak4snDObi0wZz8gm9lFRlkKONSI0AVFD/KNqq9mkjNxERSWGtra1t5c81IiXSddnZWUwZ0Z8pI/rzrVnjWbyxhrlvbeGZldvYsHMfd85bx53z1nHSgJ5cfNoQLp04mNEDeyU7bImzzq6RksNoKzahREpERFLYvqYWmg+20iM/hwJNRxfplpzsLKaOKmHqqBJuveQU/rahhrkrtvLMym2sr97HL19cyy9fXMukYX35xJRhzJo4mN6F+gDjeHSsREoFJo5C5c9FRCQdtBWa0GiUSEzl5mRz7uhSzh1dym2XnsLrFbuYuzyY/vfm5lre3FzLbXNX8eFTB3PFlGGcPbK/Kv8dR46VSP2Hme3sxHVa3f3aWASUTtqq9mlESkREADObA8wCdrj7qeGx/sCfCKbMbwQ+4e7vJjKutml92kNKJG5yc7KZPmYA08cM4NZLT+GZVVt5eHElr1fs4rFlVTy2rIoT+/fgQ6eewPljBzBlRD+NEKe5YyVSk4DObO2ckSNXmtonIiId3AfcCTzQ7tjNwIvufruZ3Rw+/2Yig6rTHlIiCVWUn8Plp5dx+ellbNq1n0eWbuaRpZVsqtnPb+ZX8Jv5FRTl5XDuSSW8zwbwvrEDGF7SM9lhSxcdK5G6zN3/lpBI0lCjRqRERKQdd59vZiM6HP4IcEH4+H7gZRKcSKnQhEjynFjSg6990PjKhWNZtGEXr3g1r6yp5u1te3jx7R28+PYOAEaU9ODCcYO4cPwgpgzvR26Olo6kOhWb6AZV7RMRkU4Y5O5bw8fbgEGJDqBtREqlz0WSJic7i3NPKuXck0q55cPj2FbXwPy1QVK1cO1ONu7az90LN3D3wg30Kcpj5skDuXDcIM4fW0ovFatISUqkukHFJkREpCvcvdXMOjUdvrGxkfLy8qjv1dDQ0Pb+te/UAtBcv7tb10xH7dshk6kdAqnWDhN6woTTi/jixDLKqxtZtHkff928n8rdB/jzsir+vKyK3GyYNLiI940s5pxhPemZ3/2/O1OtHZIhFm2gRKobVGxCREQ6YbuZDXb3rWY2GNjRmTcVFBQwbty4qG9aXl7e9v78DeVADSeVDWbcuJOivmY6at8OmUztEEjldjgVuCJ8vL56Ly+Wb+eF1TtY8k4NS6rqWVJVT37uLmbYAC6ZOIT3nzyIovzo/gZN5XZIlM62wdKlS4/42hETKXfXMMsxqNiEiIh0whPAtcDt4f//l+gAImukNLVPJD2cNKCYkwYU8/nzT2LX3kb+snIbTy7fwt821vDsqu08u2o7PfJzeP+4Qcw8eQDTRg9gQK+CZIedcRI+ImVmFwG/AHKAu9399g6vDwfmAAOAGuAad68MX2sBVoSnbnL3S8PjI4GHgBJgKfBpd2+K99eiqX0iItKemT1IUFii1Mwqge8QJFAPm9l1wDvAJxIdV2299pESSVclxQVcM3U410wdzvbdDTz11laefGsLyzbV8uTyLTy5fAsA4wb35vwxpZw/dgCTh/fTB/0JkNBEysxygLuADwCVwGIze8LdV7c77SfAA+5+v5nNBH4AfDp8rd7dJx3m0j8Efu7uD5nZr4HrgP+O2xcSatDUPhERacfdrzzCS+9PaCAdHNpHSomUSDob1LuQf5o2kn+aNpLNNft5dtU2FqzdyaINuyjfupvyrbv5zfwKCvOymTqqhBk2kBk2kBNLeiQ79ONSokekzgLWuXsFgJk9RFAWtn0iNR74Wvh4HvD40S5oZlnATOCq8ND9wK0kJJEKpvapap+IiKSySNU+lT8XOX4M69+Dz04fxWenj6LhQAtL33mX+Wuqmb92J+Vbd/OyV/OyV/MdVjFqQE9m2EBmnjyQKSP6JTv040aiE6mhwOZ2zyuBszucsxz4KMH0v8uBXmZW4u67gEIzWwI0A7e7++ME0/lq3b253TWHHiuQWFRD2tcYTJXYtGE9dYWZmUyp6ktA7RBQO6gNItQOqeVQ+fP8JEciIvFQmJfDeaNLOW90KbcAO/Y0MH/NTua9vYP5a6upqN5HRfUG7lm4gR75OYwfkM/M7XlMHVXChKF9yNOeVVFJxap9NwF3mtlsYD5QBbSErw139yozGwW8ZGYrgLpobhKLakhNYVSnnXIyPfJTsSnjT1VfAmqHgNpBbRARi2pIEjttxSY0IiWSEQb2KuTjk8v4+OQyDrQc5I133mWeV/Oy7+DtbXvCKoAOQI/8HCYP78fZI/tz9qgSTivrQ0FuZg4QdFWi//qvAoa1e14WHmvj7lsIRqQws2LgY+5eG75WFf5fYWYvA6cDjwJ9zSw3HJX6u2vGQ2trK43NYdU+fbOJiEiKajjQQv2BFvJysugRZalkEUlfeTnZnD2qhLNHlXDzh05mW10Djy1cwebGQhZt2EVF9T4WrN3JgrU7AcjPzWZSWV/OHNmPM0f0Z/LwftoQ+AgSnUgtBsaEVfaqgE9xaG0TAGZWCtS4+0HgFoIKfphZP2C/uzeG55wH/Cjc3HAe8HGCyn0JKS3b1BLsp5ifm012dla8byciIhKV3W3ro/LJylJ/JZLpTuhTyIxRxW2zBnbsaWBRRQ2LNuzibxtqWLN9L3/bWMPfNtYA68nOCioCnjOqhGljSjl7ZEnU+1cdbxKaSLl7s5ndADxLUP58jruvMrPbgCXu/gRB2dgfhDu/zweuD98+DviNmR0EsgnWSEWKVHwTeMjM/hNYBtwT76+lsTlIpApzNadURERSV2299pASkSMb2KuQSyYO4ZKJQwB4d18TS995l8VhMrWiso5VW3azastu7l64gfycbCYP78f0saVMHz2AU4b0zthBhYQv7HH3p4GnOxz7drvHjwCPHOZ9rwETjnDNCoKKgAnTGI5IKSMXEZFUpvVRItIV/Xrmc+H4QVw4fhAA9U0tLNv0LgvX7WThup2sqKrj9YpdvF6xix/h9O2Rx6RhfZlY1pdJw/pyWlkfSoozY3PgzKyQEAORqX3aQ0pERFJZnUakRKQbivJzOHd0KeeOLuUbBCNWr63fxYK11SxYu5Oq2vq2UusRw/oXMbGsL5OH92PqqBJsUK/jctRKiVSUVGhCRETSQe3+YKuO3hqREpEY6Nczn4tPG8zFpw2mtbWVynfreauyjuWVtby5uZYVlXVsrqlnc009c9/aGrynRx5njyxh6qj+TD2phLEDj4/ESolUlCJT+wo1tU9ERFJY24hUkfaQEpHYysrKYlj/Hgzr34OLTxsMQHPLQdbu2Mubm2tZvKGG1yt2sbWugWdWbeOZVduAILGaUNaX8YN7M35Ib8YP7s3I0p7kpFlypUQqSk0qNiEiImmgbY2UpvaJSALk5mQzbnBvxg3uzZVnnUhrayubavbz14pdvL4+WFu1fXcj89dUM3/NoemARXk5nDy4F+MH9+aUIX04ZUhv7IReKb2MRolUlBq1RkpERNJAbX0wtU+JlIgkQ1ZWFsNLejK8pCefPPPEtumAq7bsZvWWOlZv3c3qLbvZUtfAsk21LNtU2/benOwsxgwsZvyQILmaMDRIsHoWpEYKkxpRpKHIGqkiJVIiIpLCIiNSfbRGSkRSQPvpgBedekLb8Xf3NbUlVau2BCXX11fv5e1te3h72x4ee6MqfD+MKu3JhKF9OLXdv+IkJFdKpKJ0qGqfpvaJiEjqOlS1T2ukRCR19euZz3mjSzlvdGnbsfqmFsq3BXtYraqqY+WWOnzbHtZX72N99T4ef3MLECRXI0t6hklVb04d2odThvSJ+wdISqSi1LYhr0akREQkhR0qNqERKRFJL0X5OZxxYj/OOLFf27HG5hbWbNvLiqo6VlTVsbKqjre37aZi5z4qdu7jieVb2s49sX8PPnf+KD49dXhc4lMiFSWtkRIRkXSgqX0icjwpyM1hQlkfJpT1aTvW1HyQNdv3sDIctVpZtZvyrbvZVLOfv6zYqkQq1TS1hPtIKZESEZEUFtlHSsUmROR4lZ+b3bZWKuJAy0He2bWPE/oUxe2+SqSidGhqn9ZIiYhIamo52MruhmaysqBXoRIpEckceTnZjB7YK673UBYQpUgipap9IiKSqnaH66N6F+al3UaXIiKpTolUlJq0RkpERFJcbb024xURiRclUlGK7COlqX0iIpKq2tZHqdCEiEjMKQuIkqr2iYhIqouUPu+jPaRERGJOiVSUNLVPRERSXVsipREpEZGYUyIVJW3IKyIiqS6yh5Sm9omIxJ7Kn0epMdxHSlX7RESkM8xsI7AHaAGa3X1KvO/Zlkip2ISISMwpkYpSk/aREhGRrpvh7jsTdbPa+qDYhKb2iYjEnrKAKKnYhIiIpLq6thEpFZsQEYk1JVJR0oa8IiLSRa3Ac2a21Mw+n4gbtu0jpREpEZGY09S+KEWq9hVoap+IiHTONHevMrOBwPNm9ra7zz/SyY2NjZSXl0d9s4aGBrbtqgOgtnoL5eU1UV8rnTU0NHSrHY8XaoeA2iGgdohNGyiRitKhDXk1IiUiIsfm7lXh/zvM7M/AWcARE6mCggLGjRsX9f3Ky8tpJBdoZOLJoxkzqFfU10pn5eXl3WrH44XaIaB2CKgdOt8GS5cuPeJrGk6JUmSNlKb2iYjIsZhZTzPrFXkMfBBYGe/7HtqQV1P7RERiLeEjUmZ2EfALIAe4291v7/D6cGAOMACoAa5x98p2r/cGVgOPu/sN4bGXgcFAfXjaB919R7y+hgMtBznYCjnZWeTlKBcVEZFjGgT82cwg6Hv/6O7PxPOGra2tbeXPVbVPRCT2EppImVkOcBfwAaASWGxmT7j76nan/QR4wN3vN7OZwA+AT7d7/bscfirE1e6+JE6hv0f9gRYACnOVRImIyLG5ewUwMZH3rG9upflgKz3ycyjI1ewJEZFYS3QmcBawzt0r3L0JeAj4SIdzxgMvhY/ntX/dzCYTfKr3XAJiPaKGMJEqylfHJCIiqWlPY9BXqWKfiEh8JDqRGgpsbve8MjzW3nLgo+Hjy4FeZlZiZtnAT4GbjnDte83sTTP7lpnvIY4KAAAOXUlEQVRlxTLojhoPBIUm9AmfiIikqr1NQV/VR3tIiYjERSpW7bsJuNPMZhNM4asCWoAvAU+7e2U4x7y9q8OSsr2ARwmmAj5wtJt0p6zsO+8GO8VntzardKTKZwJqhwi1g9ogQu2QfHsag0RKI1IiIvGR6ESqChjW7nlZeKyNu28hHJEys2LgY+5ea2bnANPN7EtAMZBvZnvd/eZ2JWX3mNkfCaYQHjWR6k5Z2QOVtUAlfYqLVDpS5TMBtUOE2kFtEBGLsrLSPZGpfSo0ISISH4lOpBYDY8xsJEEC9SngqvYnmFkpUOPuB4FbCCr44e5XtztnNjDF3W82s1ygr7vvNLM8YBbwQjy/iIZwal+hpvaJiEiK2hNO7eur0uciInGR0DVS7t4M3AA8C5QDD7v7KjO7zcwuDU+7AHAzW0NQWOJ7x7hsAfCsmb0FvEmQoP0uHvFHtFXt0x5SIiKSovY2RtZIKZESEYmHhK+Rcvengac7HPt2u8ePAI8c4xr3AfeFj/cBk2Md59E0KJESEZEUd6hqn4pNiIjEgzZCisKhRErNJyIiqUlT+0RE4kuZQBQ0IiUiIqlur/aREhGJKyVSUYgUmyhSIiUiIilqT5PWSImIxJMSqShoap+IiKS6yD5SKn8uIhIfygSioKp9IiKS6tqKTfRQsQkRkXhQIhWFtn2klEiJiEiKais2oREpEZG4UCIVBRWbEBGRVNZwoIXG5lbycrLoka++SkQkHpRIRUFrpEREJJXtrj8AQJ+ifLKyspIcjYjI8UmZQBQiiZSq9omISCqqCxMp7SElIhI/SqSioDVSIiKSymojiZTWR4mIxI0SqSjUa2qfiIiksNr9kal9SqREROJFmUAUVGxCRERSWe3+JkCb8YqIxJMSqSg0NGtqn4iIpK62NVJF2kNKRCRelEhFoaEpHJHKVSIlIiKpJzK1T8UmRETiR4lUFBqaw6p92ptDRERSUG19MLVPiZSISPwokYqC9pESEZFUVlffDKjYhIhIPCkTiEK9pvaJiEgKixSb6NtDa6REROIlN9kBpKNIsQlN7RMRkc4ys4uAXwA5wN3ufnu87hUpNqERKRGR+NGIVBcdPNhKU5hIFeSq+URE5NjMLAe4C/gQMB640szGx+t+bcUmlEiJiMSNMoEuOtjaSlFeDn0Lc8jKykp2OCIikh7OAta5e4W7NwEPAR+J182ysyA3G0p7FcTrFiIiGU9T+7ooNyebB647iy2bNyU7FBERSR9Dgc3tnlcCZ8frZnddfQYrfT3FBermRUTiRb9ho3DmiP6U129PdhgiInIca2xspLy8PKr3ZgNj+2ZF/f7jSUNDg9oBtUOE2iGgdohNGyiREhERib8qYFi752XhsSMqKChg3LhxUd+wvLy8W+8/XqgdAmqHgNohoHbofBssXbr0iK8pkRIREYm/xcAYMxtJkEB9CrgquSGJiEh3qNiEiIhInLl7M3AD8CxQDjzs7quSG5WIiHRHwkekjrWPhpkNB+YAA4Aa4Bp3r2z3em9gNfC4u98QHpsM3AcUAU8DX3H31vh/NSIiIp3j7k8T9FEiInIcSOiIVCf30fgJ8IC7nwbcBvygw+vfBeZ3OPbfwOeAMeG/i2IcuoiIiIiISJtET+3rzD4a44GXwsfz2r8ejjwNAp5rd2ww0Nvd/xqOQj0AXBa/L0FERERERDJdohOpw+2jMbTDOcuBj4aPLwd6mVmJmWUDPwVuOsw1K9s9P9w1RUREREREYiYVq/bdBNxpZrMJpvBVAS3Al4Cn3b3SzLp9k+7szwGqvx+hdgioHQJqB7VBhNpBRESOd4lOpI65j4a7byEckTKzYuBj7l5rZucA083sS0AxkG9mewkKV5Qd7ZqHo/05YkPtEFA7BNQOaoOIWOzPISIiksoSnUgdcx8NMysFatz9IHALQQU/3P3qdufMBqa4+83h891mNhVYBPwj8Kv4fykiIiIiIpKpslpbE1sl3Mw+DNxBUP58jrt/z8xuA5a4+xNm9nGCSn2tBFP7rnf3xg7XmE2QSEXKn0/hUPnzvwBfPlb586VLl1YD78TyaxMRkS4bPnny5AHJDiIVqZ8SEUkJR+ynEp5IiYiIiIiIpLtEV+0TERERERFJe0qkREREREREukiJlIiIiIiISBcpkRIREREREekiJVIiIiIiIiJdlOh9pNKemV1EsAlwDnC3u9+e5JASxszmALOAHe5+anisP/AnYASwEfiEu7+brBjjzcyGAQ8AgwhK9P/W3X+Rge1QSLA9QQHB75FH3P074R5xDwElwFLg0+7elLxIE8PMcoAlQJW7z8rEdjCzjcAeoAVodvcpmfZzkUoyta9SP6V+KkL91Hupn4pPP6URqS4IvwnvAj4EjAeuNLPxyY0qoe4DLupw7GbgRXcfA7wYPj+eNQP/6u7jganA9eH3QKa1QyMw090nApOAi8JNsX8I/NzdRwPvAtclMcZE+gpQ3u55prbDDHef5O5TwueZ9nOREjK8r7oP9VPqpwLqp95L/VQgpv2UEqmuOQtY5+4VYdb+EPCRJMeUMO4+H6jpcPgjwP3h4/uByxIaVIK5+1Z3fyN8vIfgl9JQMq8dWt19b/g0L/zXCswEHgmPH/ftAGBmZcDFwN3h8ywysB2OIKN+LlJIxvZV6qfUT0WonzpE/dRRdevnQolU1wwFNrd7Xhkey2SD3H1r+HgbwVSCjGBmI4DTgUVkYDuYWY6ZvQnsAJ4H1gO17t4cnpIpPx93AN8ADobPS8jMdmgFnjOzpWb2+fBYxv1cpAj1Ve+Vsd+H6qfUT4XUTwVi3k8pkZKYcfdWgm/S456ZFQOPAje6++72r2VKO7h7i7tPAsoIPgE/OckhJZyZRdZiLE12LClgmrufQTCd7HozO7/9i5nycyGpLZO+D9VPqZ8C9VMdxLyfUiLVNVXAsHbPy8JjmWy7mQ0GCP/fkeR44s7M8gg6pz+4+2Ph4Yxrhwh3rwXmAecAfc0sUsQmE34+zgMuDRewPkQwVeIXZF474O5V4f87gD8T/NGSsT8XSaa+6r0y7vtQ/dR7qZ9SPwXx6aeUSHXNYmCMmY00s3zgU8ATSY4p2Z4Arg0fXwv8XxJjibtwXvE9QLm7/6zdS5nWDgPMrG/4uAj4AME8/HnAx8PTjvt2cPdb3L3M3UcQ/D54yd2vJsPawcx6mlmvyGPgg8BKMuznIoWor3qvjPo+VD8VUD8VUD8ViFc/ldXaetyP7MaUmX2YYK5pDjDH3b+X5JASxsweBC4ASoHtwHeAx4GHgROBdwjKRnZc6HvcMLNpwAJgBYfmGv8bwfzzTGqH0wgWZeYQfCDzsLvfZmajCD7x6g8sA65x98bkRZo4ZnYBcFNYVjaj2iH8ev8cPs0F/uju3zOzEjLo5yKVZGpfpX5K/VSE+qm/p34q9v2UEikREREREZEu0tQ+ERERERGRLlIiJSIiIiIi0kVKpERERERERLpIiZSIiIiIiEgXKZESERERERHpotxjnyIih2Nms4F7gTHuvs7MbgQ2tdv8MNHx9AVuBJ5w9zc6vPYygLtfkPjIREQkGdRPicSXEimR2LkRWAgkpYMC+hLsmVIJvNHhtS8lPhwREUkx6qdEYkiJlEgKM7OCWGyQ5+6rYxGPiIhIe+qnJJNpQ16RKLWfMgG8AAzvcMr97j47PHci8F1gOlBI8Encze6+oN317gMuBK4AfgqcDvzW3b9iZp8CPg9MAIqAtcAd7n5/+N4RwIbDhPkZd7/vcFMmzMyA24EZQAGwHLjV3Z9pd86tBJ8ejgXuAN4H7ALuAf7T3Q92qrFERCTh1E+pn5L4UrEJkdi4HNgGPAucE/77LoCZnQG8BvQHPgd8jOCX/AtmNrnDdfoADwEPAh8C/hgeHwU8AlwNXAY8CdxtZl8IX98KfDR8/IN2MTx1uGDNbAjB9I6JwA3AJ4Ba4Ckz+9Bh3vJn4KXw3o8D/wFce/QmERGRFKJ+SiTGNLVPJAbcfZmZNQI73f2vHV7+MbAJmOnuTQBm9iywEvgWwS/9iGLgGnf/vw7X/37ksZllAy8Dg4EvAr9290YzWxaeUnGYGDr6GtAPOMfd14XXfRpYDXwP+EuH83/q7veGj18ws5nAlQSfdIqISIpTPyUSexqREokjMysimGbwv8BBM8s1s1wgi2Caxfkd3nIAmHuY64wxswfNrCo85wDwWcCiDO184K+RzgnA3VsIPmGcZGa9O5zf8RPDlcCJUd5bRERShPopkegpkRKJr/5ADsEnegc6/LsB6Bd+chdRHXYUbcysGHieYHrDzQTz188E5hDMGY82rq2HOb6NoPPs1+F4TYfnjQRz6EVEJL2pnxKJkqb2icRXLXAQuAt44HAndFgIe7jqL+cQLBCe7u4LIwfDTwyjVQOccJjjJ4QxvNuNa4uISPpQPyUSJSVSIrHTSFCpqI277zOzBQSf0r0RZfWgHuH/ByIHzKwf8JHD3J+OMRzBK8CNZjbC3TeG18wBPgksc/fdUcQpIiKpTf2USAwpkRKJndXAdDObRTD1YGf4y/9rwHzgWTO7h2CqQilwBpDj7jcf47qvAbuBu8zsO0BP4N+BnQTVkyK2E1RZ+pSZvQXsAza4+67DXPPnwGzg+fCauwk2QxwLXNzFr1tERNKD+imRGNIaKZHYuQVw4GFgMXArgLu/QTBXfBfwS+A54BcEe23MP9ZF3b2aoGxtDkFp2R8AdwP/0+G8gwQLe/sRLBBeDFxyhGtuAaYBq4D/Dq/bH7i4/f4cIiJyXFE/JRJD2pBXRERERESkizQiJSIiIiIi0kVKpERERERERLpIiZSIiIiIiEgXKZESERERERHpIiVSIiIiIiIiXaRESkREREREpIuUSImIiIiIiHSREikREREREZEuUiIlIiIiIiLSRf8f5SgaH2A+SD8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1008x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Optimize the bound\n",
    "loss_fn = pac_bound.train_bound\n",
    "opt = torch.optim.Adam([{'name': 'stoch_net','params': stoch_model.parameters(), 'lr': 5e-3}, \n",
    "                       {'name': 'lambda','params': pac_bound.log_lambda_, 'lr': 1e-2}]\n",
    "                      )\n",
    "lambdas = []\n",
    "train_accs = []\n",
    "b_re_values = []\n",
    "\n",
    "for epoch in tqdm.tqdm(range(50)):\n",
    "    train_res = utils.train_epoch(loaders['train'], stoch_model, loss_fn, opt, regression=True)\n",
    "    \n",
    "    train_accs.append(train_res['stats']['accuracy'].item())\n",
    "    lambdas.append(torch.exp(pac_bound.log_lambda_).item())\n",
    "    b_re_values.append(pac_bound.B_RE(stoch_model).item())\n",
    "    \n",
    "\n",
    "f, arr = plt.subplots(1, 2, figsize=(14,5))\n",
    "\n",
    "arr[0].plot(train_accs, lw=2)\n",
    "arr[0].set_ylabel(\"Train Accuracy\", fontsize=16)\n",
    "arr[0].set_xlabel(\"Iteration\", fontsize=16)\n",
    "arr[1].plot(lambdas, lw=2)\n",
    "arr[1].set_ylabel(r\"$\\lambda$\", fontsize=16)\n",
    "arr[1].set_xlabel(\"Iteration\", fontsize=16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/izmailovpavel/.local/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: divide by zero encountered in double_scalars\n",
      "  # Remove the CWD from sys.path while we load stuff.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Iteration')"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAb4AAAFCCAYAAACO3biLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3xcdZ3/8dckaSY0Se+QXlKghfbTlHtbbqIIKFgUAYEiuO5DXC+7q6yy6q6wF2RRVlbXC+uv7g1d0FWxRSxFqgVFQBfQNqUF2ukHSqE0bWlL77ckzeX3x5mk02kyOWnmdjLv5+PBozNnznzPh+8j6affe6yzsxMREZFSUVboAERERPJJiU9EREqKEp+IiJQUJT4RESkpSnwiIlJSlPhERKSkVBQ6gGxYvnx5ZzweH3A5LS0tZKOcwUr10zfVUWaqn76pjjLrT/3s37//rZkzZx6bfn1QJL54PE5DQ8OAy0kkElkpZ7BS/fRNdZSZ6qdvqqPM+lM/jY2N63q6rq5OEREpKUp8IiJSUpT4RESkpCjxiYhISVHiExGRkqLEJyIiJUWJT0RESooSn4iIlBQlPhERKSlKfEBHRydzf7uGJU37Cx2KiIjk2KDYsmygtu9v5euLnZrKMq6/qJ2qIeWFDklERHJELT5gTE2c6eOGsbe1g18nNhc6HBERySElvqTrZ9UDMG9pU4EjERGRXFLiS7rqzAlUlMHvXtnKxp0HCh2OiIjkiBJf0sjqSs6fWE1nJzy0TK0+EZHBSokvxWVTagGY39hEZ2dngaMREZFcUOJLcda4Yxg7rIp12/bzx9e2FzocERHJASW+FOVlMa6dOQHQJBcRkcFKiS/NnJkTAVj04ib2trQVOBoREck2Jb40J46p5pwTR3HgYDuPvrCx0OGIiEiWKfH1YI7W9ImIDFpKfD1472njqK4sp3HdDl7durfQ4YiISBYp8fWgOl7B+04fB8B8tfpERAYVJb5eXD8rmOTy0LIm2to7ChyNiIhkS95PZzCz2cA9QDlwr7vfnfb5CcD3gWOB7cCH3T3vza6ZJ4xk8phq1r61j6df2col0+ryHYKIiORAXlt8ZlYOzAUuB6YDN5rZ9LTb/hX4gbufDtwJfDWfMXaJxWJc1zXJZYm6O0VEBot8d3WeA6xx97Xu3go8AFyVds904Ink69/28HneXDujnrIY/Gb1ZrbtbSlUGCIikkX57uqcAKxPed8EnJt2zwrgGoLu0A8AtWY22t239VZoS0sLiURiwME1NzcfUc7M8cewZMMB/nPx83xg+vABPyPKeqofOZzqKDPVT99UR5llo36K8QT2LwD/z8xuAp4GNgDtmb4Qj8dpaGgY8IMTicQR5fxZ2wiW/GgZT69v5bZrphGLxQb8nKjqqX7kcKqjzFQ/fVMdZdaf+mlsbOzxer4T3wZgYsr7+uS1bu6+kaDFh5nVANe6+868RZjmXQ11VFeWs/rNPezYf5BR1ZWFCkVERLIg32N8S4ApZjbJzCqBG4CFqTeY2Rgz64rrNoIZngVTWVHGqJog2e1pPljIUEREJAvymvjcvQ24GVgMJIB57r7SzO40syuTt10EuJm9DNQBd+Uzxp7UxIcAsKdZm1aLiERd3sf43H0RsCjt2u0prx8EHsx3XJnUxoNq0mkNIiLRp51bQqipSiY+tfhERCJPiS+EGrX4REQGDSW+ELpafHuU+EREIk+JL4TuMT51dYqIRJ4SXwiHujq1nEFEJOqU+ELQ5BYRkcFDiS+ErhafxvhERKJPiS+EWrX4REQGDSW+ELp2btFyBhGR6FPiC6FrjG+fEp+ISOQp8YWgMT4RkcFDiS8EjfGJiAweSnwhaMsyEZHBQ4kvhKGV5cRisL+1nfaOzkKHIyIiA6DEF0IsFqOmUq0+EZHBQIkvpO7dW5T4REQiTYkvpBptVC0iMigo8YV0qMWnjapFRKJMiS+k7rV8avGJiESaEl9ItRrjExEZFJT4QtIYn4jI4KDEF5I2qhYRGRyU+ELqmtyiMT4RkWhT4gupVtuWiYgMCkp8IdVoo2oRkUFBiS8kbVQtIjI4KPGF1D3Gp8QnIhJpFfl+oJnNBu4ByoF73f3utM+PB+4HRiTvudXdF+U7znRdY3w6hV1EJNry2uIzs3JgLnA5MB240cymp932D8A8dz8LuAH4bj5j7I3G+EREBod8d3WeA6xx97Xu3go8AFyVdk8nMCz5ejiwMY/x9UpjfCIig0O+uzonAOtT3jcB56bdcwfwmJn9FVANvDs/oWVWm1zAvqdZm1SLiERZ3sf4QrgRuM/dv2Fm5wM/NLNT3b2jty+0tLSQSCQG/ODm5uZey+k6eX1vSxurVq0iFosN+HlRk6l+JKA6ykz10zfVUWbZqJ98J74NwMSU9/XJa6k+BswGcPdnzawKGANs6a3QeDxOQ0PDgINLJBIZy6kaso7mgx2cePJUhlYW478Zcquv+hHVUV9UP31THWXWn/ppbGzs8Xq+x/iWAFPMbJKZVRJMXlmYds8bwLsAzKwBqAK25jXKXnTv16kJLiIikZXXxOfubcDNwGIgQTB7c6WZ3WlmVyZv+zzwCTNbAfwEuMndO/MZZ29qtZZPRCTy8t5fl1yTtyjt2u0pr1cBF+Q7rjB0NJGISPRp55Z+0JIGEZHoU+LrBx1NJCISfUp8/aCjiUREok+Jrx8ObVumRewiIlGlxNcPGuMTEYk+Jb5+0NFEIiLRp8TXD7VaziAiEnlKfP3QPcanFp+ISGQp8fVD15ZlOoxWRCS6lPj6oWtyi9bxiYhElxJfP9Sqq1NEJPKU+PpByxlERKJPia8fDi1gV+ITEYkqJb5+6B7jU4tPRCSylPj6IV5RRkVZjNa2Dlra2gsdjoiIHAUlvn6IxWLd3Z37WpT4RESiSImvn3QYrYhItCnx9dOhcT6d0CAiEkUVvX1gZhf2pyB3f3rg4RS/Ws3sFBGJtF4TH/Ak0Jl8HUt53ZvybARU7LSWT0Qk2jIlvotTXo8AvgO8BDwAbAbqgBuBU4BP5yrAYlNTFezXqcQnIhJNvSY+d3+q67WZ3Qc85u4fT7vtB2b2PeAa4JGcRFhktF+niEi0hZ3cchXw014++2ny85Kg/TpFRKItbOIrA07u5bMplMj4Hmg5g4hI1GUa40v1KPBVM3sLeMjd282sHLgW+Arwi1wFWGw0uUVEJNrCJr7PABMJujXbzGwHMDL5/d8nPy8JXTu3aIxPRCSaQiU+d38LeIeZXQqcB4wDNgHPuvuvcxhf0amNd21ZpsQnIhJFYVt8ALj748DjA3mgmc0G7iEYF7zX3e9O+/xbHFpKMRQ4zt1HDOSZ2VSjyS0iIpHWr8QHYGbHAVXp1939jRDfLQfmApcCTcASM1vo7qtSyvnrlPv/CjirvzHmko4mEhGJtlCJz8yGEbTSPgjEe7ktzMzOc4A17r42We4DBEshVvVy/43Al8LEmC+HtizTXp0iIlEUtsU3l2AG5/eAF4GWo3zeBGB9yvsm4NyebjSzE4BJwBNH+aycqIlr5xYRkSgLm/hmA3/j7nNzGUyaG4AH3b3Pg+9aWlpIJBIDfmBzc3Of5exr7QBg9/7WrDwzSsLUT6lTHWWm+umb6iizbNRPf8b4fEBPCmwgWBbRpT55rSc3EHIP0Hg8TkNDwwBDg0Qi0Wc5HR2d8JPXOdDWyVSbRnlZbMDPjYow9VPqVEeZqX76pjrKrD/109jY2OP1sDu3PAC8P+S9mSwBppjZJDOrJEhuC9NvMrNpBOsEn83CM7OqrCzWPcFlX6u6O0VEoiZsi+8x4NtmVgssAran3+DufY7FuXubmd0MLCaYDPN9d19pZncCS929KwneADzg7n0dhVQQNfEK9ra0sbe5jWHJ0xpERCQawia+h5N/TgJuSrneyaGz+kLt1+nuiwiSZ+q129Pe3xEyroKoqaqA3ZrgIiISRWET38V931I6dDSRiEh0hd2y7Km+7yodOppIRCS6wk5ukRQ6mkhEJLrC7tzS18SVTnd/VxbiiYRDRxNp9xYRkagJO8ZXRjCBJdVowICtwMvZDKrY6WgiEZHoCjvGd1FP183sJGAB8M9ZjKno1eowWhGRyBrQGJ+7vwrcDXw9O+FEQ/fRRGrxiYhETjYmt2wFpmahnMjQRtUiItE1oMRnZqOBzwGvZiecaNBhtCIi0RV2VudrHDm5pRKoS76+NptBFTuN8YmIRFfYWZ1PcWTiawbWAfOTY30lQ2N8IiLRFXZW5005jiNSatTiExGJrP6cxweAmdUQHBm0w933Zj+k4qe9OkVEoiv05BYze4+ZLQV2Aq8DO83sj2Z2aa6CK1Zq8YmIRFeoxGdm7wEeBWqALwOfAr4C1AKLSi35Vackvs7OojwyUEREehG2q/MOgsNor3D3jq6LyQNkfwH8E/B41qMrUpUVZcQrymhp66D5YAfHVIY6ilBERIpA2K7OM4C5qUkPIPn+u8CZ2Q6s2HUdTbRHG1WLiERK2MTXAgzr5bPa5OclRUcTiYhEU9jE9yTwZTOblHrRzI4n6Ab9bXbDKn7avUVEJJrCjvF9Efg/wM3sOWATMBY4j2CW5xdzE17xUotPRCSaQrX43P1l4HTg34A4MAOoAu4BznT3V3IWYZHq2qh6j1p8IiKREnoBu7tvAr6Qw1gipVbblomIRNJRn85gZseaWWU2g4kSLWIXEYmmjC0+MzsbuAYYAjzo7s+Z2fXAtwlOZmg1s/8Gbklf6jDYaXKLiEg09Zr4zOwygsXpnUAr8Bkz+yTw38BiYCXB+r5PJ1//Z86jLSLar1NEJJoytfj+jmAZw1UERxB9k2Cx+nfd/bNdN5nZvwOfoMQSX/cYnxawi4hESqYxvlOA77j7AXfvBL5GMJPzkbT7HgZOylF8Raurxbevpb3AkYiISH9kavGNBjanvN+S/HN72n076H1XlyOY2WyCZRDlwL3ufncP91xPsDC+E1jh7h8KW36+qKtTRCSajnpW59Ews3JgLnA5MB240cymp90zBbgNuMDdTwFuyWeMYdWoq1NEJJL6Wsf3fjM7Nfm6jKAFdqWZpW5KPbkfzzsHWOPuawHM7AGCMcRVKfd8gmBD7B0A7r7liFKKQG1yAbtmdYqIREtfie/ve7h2ew/Xwh5KNwFYn/K+CTg37Z6pAGb2fwTdoXe4+69Clp831fHgKCItYBcRiZZMiW9Shs9yqQKYAlwE1ANPm9lp7r6zty+0tLSQSCQG/ODm5ubQ5Ww/ECS8nfuy8+wo6E/9lCrVUWaqn76pjjLLRv30mvjcfd2ASu7ZBmBiyvv65LVUTcAf3P0g8JqZvUyQCJf0Vmg8HqehoWHAwSUSidDlHGhth3lvcKCtMyvPjoL+1E+pUh1lpvrpm+oos/7UT2NjY4/X8zq5hSB5TTGzScntzm4AFqbds4CgtYeZjSHo+lybzyDDqBpSRnlZjJa2DlrbSmrTGhGRSMtr4nP3NuBmgp1fEsA8d19pZnea2ZXJ2xYD28xsFcE5f3/j7tvyGWcYsVgsZS2fxvlERKIi9OkM2eLui4BFadduT3ndCXwu+V9Rq4lXsOvAQfa2tDGyumT36xYRiZR8d3UOKl3blmkRu4hIdIRKfGY23MziuQ4manQ0kYhI9PSZ+MysAtgGXJb7cKJFu7eIiERPn4kvOSFlM6DdmNNov04RkegJO8b3v8DHcxlIFNXqMFoRkcgJO6vzdeBDZraE4BiiTaRtU+bu389uaMWve4xPLT4RkcgIm/jmJv+cAMzs4fNOoAQTnzaqFhGJmrCJr1D7dha1Gi1nEBGJnFCJL0f7dkZerXZuERGJnH7t3JI8m++dwCiCk9ifdPeVuQgsCmo0uUVEJHJCJb7kWr77gBuBWMpHnWb2Y+Amdy+55Q5awC4iEj1hlzN8Cbie4BDaScAxyT9vBz5Iz4fTDnoa4xMRiZ6wXZ0fBr7i7nelXFsH3GVm5cBHCZJjSVGLT0QkesK2+MYDz/Ty2TPJz0uO1vGJiERP2MS3Ebigl8/elvy85Ghyi4hI9ITt6vwR8Pdm1pF8vQkYS3CC+t8D/5Kb8IpbdeWhxNfR0UlZWayPb4iISKGFTXx3AJOBf0q+7hIDfgLcmdWoIqK8LEZ1ZTn7WtvZ19pGbdWQQockIiJ9CLuAvY1gr867gAs5tI7v6VJexwdBd+e+1nb2tijxiYhEQZ+Jz8wqCboyf+zuS4CSTnTpauIVbKYlmOAyvNDRiIhIX8Kcx9cK/DnB2j1JU5Ns5e3RBBcRkUgIO6vzeeC0XAYSVbVa0iAiEilhE9/ngS+Y2RVmpqmLKUZVVwKwZsveAkciIiJhhJ3VOZ9gBOth4KCZbeXwg2g73f2EbAcXBZedUsfCFRt5sLGJP3u7Tm8SESl2YRPfb0g7cV0Cl06vY8TQIazatJuXNuzi1Ama4SIiUszCLme4KcdxRFa8opyrzhjP/c+u48HGJiU+EZEi1+cYn5lVmtkyM7ssHwFF0ZxZEwFYsHwDLW0ldzqTiEikhF3OMAnQtMVenDphONPHDWPn/oP8etWWQocjIiIZhJ3V+TiQlRafmc02MzezNWZ2aw+f32RmW81sefK/j2fjubk2Z1Y9APOWri9wJCIikknYyS3fAf43eRL7AoJNqg+b7OLua/sqJHl231zgUqAJWGJmC919VdqtP3X3m0PGVhSuPnMCX120mt+9spVNuw4wbrjW+4uIFKOwLb6ngInA55KvXwZeSfsvjHOANe6+NtmF+gBwVb8iLlIjqyu5dHodHZ3w0LINhQ5HRER6EbbF99EsPW8CkNoX2ASc28N915rZhQQJ9q/dPRL9h9fNqufRFzcxf+l6PnXRScRiWusvIlJswi5nuD/XgaR4BPiJu7eY2Z8D9wOXZPpCS0sLiURiwA9ubm4eUDnHdnQyemg5r2/bz4NPPc+pdYOru3Og9VMKVEeZqX76pjrKLBv1E7bF1yszKwNGuPv2ELdvIOgy7VKfvNbN3belvL0X+FpfhcbjcRoaGkI8PrNEIjHgcj64vozvPvkqf9hazpyLBh5TMclG/Qx2qqPMVD99Ux1l1p/6aWxs7PF6r2N8ZrbdzGakvI+Z2UIzm5x269nA1lBRwBJgiplNSh53dAOwMO2541LeXglE6p8+XWv6Fr24ib06sUFEpOhkmtwygsNbhGXAFcnrRyV5oO3NwGKChDbP3Vea2Z1mdmXyts+Y2UozWwF8BrjpaJ9XCJPGVHPOiaPY39rOohc2FTocERFJM+Cuzv5y90XAorRrt6e8vg24Ld9xZdN1s+r54+vbmd+4nuvPntj3F0REJG/CLmeQfnjfaeMYWlnOktd3sHarjisSESkmSnw5UB2v4IrTg6HK+Y1NBY5GRERS9dXVOSFlMkt5yrWdKffUZz+s6JszayLzljbx0LImPn/pVCrK9W8MEZFi0Ffie7CHawvS3sfQWX1HmHXCSCaNqea1t/bxu1fe4uJpxxU6JBERIXPiy9ZuLSUpFosxZ1Y9X/uVM2/peiU+EZEi0Wviy/NuLYPStTPq+dfFzq8Tm9m+r5VR1ZWFDklEpORp4CmH6oZV8c6px3KwvZMFz2vjahGRYqDEl2NdO7nMW7qezk4NhYqIFJoSX469q+E4Rg4dwuo397By4+5ChyMiUvKU+HIsXlHO1WdNAGC+TmcXESk4Jb48mDMz6O5csHwjzQfbCxyNiEhpU+LLg+njh3HqhGHsOnCQx1dtLnQ4IiIlTYkvT7pafdrCTESksJT48uSqM8dTWV7G717ZysadBwodjohIyVLiy5MRQyu57JQ6OjvhoWVq9YmIFIoSXx51remb39ikNX0iIgWixJdHbz95DOOGV7Fu237++Nr2QocjIlKSlPjyqLwsxrUzglOc5i1Vd6eISCEo8eXZdTODxLfoxU3sbWkrcDQiIqVHiS/PThxTzTmTRnHgYDuPvrCx0OGIiJQcJb4CuL5742p1d4qI5JsSXwG897SxVFeW07huB69u3VvocERESkqmE9glR4ZWVnDF6eP56dL13PbQizSMre13GSOrK/nLi04iXlGegwhFRAYvJb4C+eA5E/np0vX88bXtR720YXR1JX96/onZDUxEZJBT4iuQGceP5Ps3zeKNbfv7/d1Xt+7jh8+tY35jkxKfiEg/KfEV0CXT6o7qe80H21mwfAMvNO1i9Zu7mTZ2WJYjExEZvDS5JYKqhpRz5RnjAZivmaEiIv2S98RnZrPNzM1sjZndmuG+a82s08xm5TO+qOja93PB8xtobesocDQiItGR18RnZuXAXOByYDpwo5lN7+G+WuCzwB/yGV+UnFE/nKl1NWzb18oTq7cUOhwRkcjId4vvHGCNu69191bgAeCqHu77MvAvQHM+g4uSWCx26HDbpesLHI2ISHTkO/FNAFL/lm5KXutmZjOAie7+aD4Di6Krz5pARVmMJ1/eypbd+jeCiEgYRTWr08zKgG8CN/Xney0tLSQSiQE/v7m5OSvl5NPZE47h2fX7+Y/FzzPn1BE5fVYU6yffVEeZqX76pjrKLBv1k+/EtwGYmPK+PnmtSy1wKvCkmQGMBRaa2ZXuvrS3QuPxOA0NDQMOLpFIZKWcfPqzzlE8+4OlPPVGC/943TRisVjOnhXF+sk31VFmqp++qY4y60/9NDY29ng934lvCTDFzCYRJLwbgA91fejuu4AxXe/N7EngC5mSXqm7yI5lTE2cV7fuY9kbO5l5wshChyQiUtTyOsbn7m3AzcBiIAHMc/eVZnanmV2Zz1gGiyHlZVwzIxgmfbBRk1xERPqS9zE+d18ELEq7dnsv916Uj5iibs7Mev7r6bU8smIT/3jFdIZWFtXQrYhIUdHOLYPAlLpazjp+BHtb2vjli28WOhwRkaKmxDdIdK/pU3eniEhGSnyDxBVnjKNqSBnPrd1+VCc+iIiUCiW+QWJY1RAuP3UcoEkuIiKZKPENInNm1QPwYGMT7R2dBY5GRKQ4afrfIHLepNHUjzyGph0H+Ny85YwcWpnV8rdv386oV1b2+Fm8ooyPXjCJscOrsvpMEZFsU+IbRMrKYnxw1kS+8fjLPLx8Y46esrvXTzbvbubbN5yVo+eKiGSHEt8g84kLJ1M3rIp9rW1ZL3vz5s3U1R15anxLWwd3/3I1v3zpTe5sPsiwqiFZf7aISLYo8Q0yVUPKuf7siX3feBQSiWYaGib1+NnTL2/lmVe38ciKjfzJuSfk5PkiItmgyS2SFV0Ta+YvbSpwJCIimSnxSVbMPmUctfEKlq/fySub9xQ6HBGRXinxSVYcU1nOFWeMB2B+o1p9IlK8lPgka65Pdnc+tKyJg+0dBY5GRKRnSnySNWdOHMGU42p4a28rv129pdDhiIj0SIlPsiYWix2a5KLuThEpUkp8klUfOKue8rIYT6zewpY9zYUOR0TkCEp8klXH1sa52I6jvaOTBc9vKHQ4IiJHUOKTrLs+ZU1fZ6c2yxaR4qLEJ1l38bTjGFNTyStb9rJ8/c5ChyMichglPsm6IeVlfOCsCQDM004uIlJklPgkJ+bMCvYL/cWKjRxobS9wNCIihyjxSU5MravljIkj2NPSxq9Wbip0OCIi3ZT4JGe6JrnMW6LuThEpHkp8kjPvP2M88Yoynl27jfXb9xc6HBERQOfxSQ4NqxrC5aeOZcHyjXzxZy9gY2sLHVLObd++nVFrVhY6jG5nnziK9542LmvldXR08sPn1nHu5FFMGzssa+Vm0t7RyQ+ffZ23nTyGqXWD/2dIck+JT3Lqg2cfz4LlG3nm1W088+q2QoeTJ7sLHUC3+595nWdufRdjh1dlpbyFKzbypYUrmVpXw+JbLiQWi2Wl3EwWPL+BOx5ZxbSxtfzys+/IyzNlcFPik5w6/6TR/MeHZ7BhZ2lsX7Z582bq6uoKHQYAi17cROO6HfxsWROfvvjkrJQ5b+l6AF7evJcVTbs4c+KIrJQb5pmr39zDSxt2c1r98Jw/Uwa3vCc+M5sN3AOUA/e6+91pn/8F8GmgHdgLfNLdV+U7Tsme2admr6ut2CUSzTQ0TCp0GABMHlPNR+9bwvyl6/nURScNuKW0fvv+w1rt85auz3niW7dtH394bfthz1Tik4HK6+QWMysH5gKXA9OBG81setptP3b309z9TOBrwDfzGaPIYPGOKWOoGxbn9W37Wbpux4DLezB54sYZyWT3yPLcr9FMf+bDyzfQfFDrQmVg8j2r8xxgjbuvdfdW4AHgqtQb3D11gKQa0GaPIkehoryMa2Z0LSlZP6CyOjo6u5PQF99j3Ws0F698c8Bx9qY95Zm3XT6N0yYMZ3dzG4+t2pyzZ0ppyHfimwCk/gY2Ja8dxsw+bWavErT4PpOn2EQGnTkzg8T36Iub2NfSdtTlPLt2Gxt2HqB+5DGcN3l0d7ld42+58Ps1b7FpVzPHjxrKuZNGHTrrMYfPlNJQlJNb3H0uMNfMPgT8A/CRTPe3tLSQSCQG/Nzm5uaslDNYqX76Vox1dMpxVazc0sy9i5dx2ZSjWw7w309vAeCi46twX40d005leYxnXt3Gk0teoK5mSKhy+lM/33tqc/KZcVavXs20Y9oZUhbj96+8xVNLXuS4mqL862vAivFnqJhko37y/ZOzAZiY8r4+ea03DwD/3leh8XichoaGAYYGiUQiK+UMVqqfvhVjHX1kbw1/+7MX+P3GNj57Zf9j23XgIM/+6HViMfiL2WdSP3IoAO9NHGTB8o08v7OKvz57aqiywtbPzv2tPLe+65lnMX7EMQDMThzkkRUbWbG7is+cPaXf/y9RUIw/Q8WkP/XT2NjY4/V8d3UuAaaY2SQzqwRuABam3mBmqT/N7wNeyWN8IoPOe08fx9DKcpa8voO1W/f2+/uPrNhIS1sHbztpdHfSg0MbkT/Y2ERHR3aH4h9evpHW9g7efvKY7qQHh7puc/FMKR15TXzu3gbcDCwGEsA8d19pZnea2ZXJ2242s5Vmthz4HH10c4pIZjXxCt6X3L2la7JIf3SNqV0/a+Jh18+fPJoJI45hw84DPLs2u5sTzG/s+ZkXnDyG8cOreGP7/sOWOYj0R947yd19EbAo7drtKa8/m++YRAa7ObMmMr+xiZ8ta+Lzl8Dh16QAAArbSURBVBnlZeHW9Pmbe1jRtIvaqgrec8rYwz4rK4tx3cx67vnNK8xbup4LTh6TlVhXbtzFSxt2M6yqgkunH74ZQHlZjGtn1vOdJ9Ywv3E95580OivPlNKiTapFSsDZJ47kxNFD2by7hadf2Rr6e12tvSvPGE/VkPIjPr8u2fX4q5feZNeBg1mJdX7y8OKrz5qQ8ZmLXtzEnubsPFNKixKfSAmIxWLdY3JhlwMcbO/g588Hc8/Suxy7TBw1lAtOHk1LWwePrNg44Dhb2tp5eHnmZ54wuprzJo+i+WAHv3hBZz1K/ynxiZSIa2ZMoCwGj6/azPZ9rX3e/8TqLWzb18rUuhpOz7BN2JyZ/UuomfwmsYUd+w8ybWwtp4zv/fSHbD5TSo8Sn0iJGDf8GC6ceiwH2zu7W1WZpE5qybTP5+xTx1JbVcGKpl34m3sGFGPYZ15+2lhq4hUse2Mna7YM7JlSepT4RErIoZZS5tmdW/Y081vfSkVZjKvPOmJzpcNUDSnn/WeMT5Z79C2wN3c189TLWxlS3vczh1ZWcMXpwUzV+UcxU1VKmxKfSAl59/TjGDF0CKs27ealDbt6ve/nyzbQ3tHJJdOOY0xNvM9yu8bjfv78Bg62dxxVbD9b1kRHJ7y7oY5R1ZV93t81ZvnQsg20HeUzpTQp8YmUkHhFOVefGbSmelvT19nZ2d2K6m2CSboz6oczta6GbftaeWL1ln7H1dl5aEPqsM+ccfwITjq2mq17Wnjq5fAzVUUG52Z3ItKrObPque+Z13loWVOP6/n2tbSxZstextTEuciODVVmLBZjzsyJ3LUowbcef5k/9rK4fPv27Yx69cjjNfc0H+S1t/ZxXG2cd0wJtx6wa6bq3b9czb8+9vJhZwVGWW91VEoumXZc1taF9kSJT6TEnDJ+OKdNGM6LG3bxvd+/1ut9186cQEV5+E6hq8+awNcXO6vf3MPqjJNceu9ivW5mfb+eec1ZE/jGY05i024Sm3b3/YXI6L2OSsH/rXmLX91yYc7KV+ITKUFzPzSDxxOb6ezseb/LqiHlfKCPCSbpjq2N88OPncOLGcYON2/eTF1dXY+fVQ0p55oZ/XvmccOq+OHHzs04Xhk1meqoVLztpNy19kCJT6QkHT96KB97+6Ssl3vu5NGcO7n3bcQSiRYaGiZn9ZnnTR7NeRmeGTW5qCM5nCa3iIhISVHiExGRkqLEJyIiJUWJT0RESooSn4iIlBQlPhERKSlKfCIiUlKU+EREpKQo8YmISElR4hMRkZIS622vvihpbGzcCqwrdBwiIlJUTpg5c+YRR4wMisQnIiISlro6RUSkpCjxiYhISVHiExGRkqLEJyIiJUWJT0RESopOYAfMbDZwD1AO3Ovudxc4pIIzs+8DVwBb3P3U5LVRwE+BE4HXgevdfUehYiwkM5sI/ACoAzqB/3L3e1RHh5hZFfA0ECf4u+ZBd/+SmU0CHgBGA43An7p7a+EiLSwzKweWAhvc/QrVz+HM7HVgD9AOtLn7rIH+npV8iy/5QzcXuByYDtxoZtMLG1VRuA+YnXbtVuA37j4F+E3yfalqAz7v7tOB84BPJ39uVEeHtACXuPsZwJnAbDM7D/gX4FvufjKwA/hYAWMsBp8FEinvVT9Hutjdz3T3Wcn3A/o9K/nEB5wDrHH3tcl/VT0AXFXgmArO3Z8Gtqddvgq4P/n6fuDqvAZVRNx9k7svS77eQ/AX1wRUR93cvdPd9ybfDkn+1wlcAjyYvF7SdWRm9cD7gHuT72OofsIY0O+ZEl/wl9X6lPdNyWtypDp335R8/SZBN1/JM7MTgbOAP6A6OoyZlZvZcmAL8DjwKrDT3duSt5T679u3gb8FOpLvR6P6SdcJPGZmjWb2yeS1Af2eKfHJUXH3ToIfyJJmZjXAz4Bb3H136meqI3D3dnc/E6gn6F2ZVuCQioaZdY2hNxY6liL3dnefQTAc9WkzuzD1w6P5PVPigw3AxJT39clrcqTNZjYOIPnnlgLHU1BmNoQg6f3I3R9KXlYd9cDddwK/Bc4HRphZ18S6Uv59uwC4Mjl54wGCLs57UP0cxt03JP/cAvyc4B9QA/o9U+KDJcAUM5tkZpXADcDCAsdUrBYCH0m+/gjwcAFjKajkWMz3gIS7fzPlI9VRkpkda2Yjkq+PAS4lGAv9LXBd8raSrSN3v83d6939RIK/d55w9z9B9dPNzKrNrLbrNXAZ8BID/D3TJtWAmb2XoK+9HPi+u99V4JAKzsx+AlwEjAE2A18CFgDzgOMJTsO43t3TJ8CUBDN7O/A74EUOjc/8HcE4n+oIMLPTCSYelBP8I3ueu99pZpMJWjijgOeBD7t7S+EiLTwzuwj4QnI5g+onKVkXP0++rQB+7O53mdloBvB7psQnIiIlRV2dIiJSUpT4RESkpCjxiYhISVHiExGRkqLEJyIiJUWnM4hkkZndBPwPMMXd15jZLcAbKQvc8x3PCOAWYGHX3qIpnz0J4O4X5T8ykcJR4hPJrVuA3wMFSXzACII1mE3AsrTPPpX/cEQKT4lPJGLMLJ6NBc3uviob8YhEjRawi2RRalcn8GvghLRb7nf3m5L3ngF8GXgHUEXQIrvV3X+XUt59wLuBOcA3CE6B+C93/6yZ3QB8EjgNOAZ4Bfi2u9+f/O6JwGs9hPlRd7+vp65OMzPgbuBiggNkVwB3uPuvUu65g6AVOZVgx6N3AtsItnD7irt37WQjUpQ0uUUkdz5AcGTKYoLNmc8nSHSY2QzgGYJtqT4BXEuQPH5tZjPTyhlOsIXVTwh2qP9x8vpkgnPb/oTgPLJHgHvN7C+Sn28Crkm+/mpKDI/2FKyZjSfolj0DuBm4HtgJPGpml/fwlZ8DTySfvQD4Jw7tnyhStNTVKZIj7v68mbUAb7n7c2kffx14g+CE8lYAM1tMsAHvP3L4wZo1BPs1HrYRr7v/c9drMysDngTGAX8J/Ie7t5jZ88lb1vYQQ7rPASOB8919TbLcRcAq4C7gl2n3f8Pd/yf5+tdmdglwI0GLV6RoqcUnkmfJkwreCcwHOsysInkMTYyge/TCtK8cBH7RQzlTzOwnZrYhec9B4OOAHWVoFwLPdSU9CM7TI2hpnmlmw9LuT285vkSwabBIUVPiE8m/UQQnFvwjhxJW1383AyOTLbguW5MJqFvyANzHCbolbyUYJzwb+D7B2NzRxrWph+tvEiTlkWnX03fDbyEYqxQpaurqFMm/nQRHGc0FftDTDWkTRHqagXY+wcSZd7j777suphxgejS2A2N7uD42GcOOAZQtUjSU+ERyq4VgxmU3d99nZr8jaK0tO8pZkEOTfx7sumBmI4Greng+6TH04ingFjM70d1fT5ZZDnwQeN7ddx9FnCJFR4lPJLdWAe8wsysIugzfSiaVzwFPA4vN7HsEXYxjgBlAubvf2ke5zwC7gblm9iWgGvgH4C2CWaBdNhPMFr3BzF4A9gGvufu2Hsr8FnAT8HiyzN0Ei9ynAu/r5/+3SNHSGJ9Ibt0GOMFp0UuAOwCS24edTZCU/g14DLiHYE3e030V6u5bCZZLlBMsafgqcC/wv2n3dRBMeBlJMHFmCfD+XsrcCLwdWAn8e7LcUcD7UtfxiUSdFrCLiEhJUYtPRERKihKfiIiUFCU+EREpKUp8IiJSUpT4RESkpCjxiYhISVHiExGRkqLEJyIiJUWJT0RESsr/B32xW/s/ZP/HAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let's visualize how the true error bound changes with iteration\n",
    "ts = np.linspace(0, 1., 20)\n",
    "error_bounds = []\n",
    "for b_re, acc in zip(b_re_values, train_accs):\n",
    "    kls = [kl_bernoulli(1 - acc, t) for t in ts]\n",
    "    idx = np.argmin(np.abs(np.array(kls) - b_re))\n",
    "    error_bounds.append(ts[idx])\n",
    "\n",
    "plt.figure(figsize=(7,5))\n",
    "plt.plot(error_bounds, lw=2)\n",
    "plt.ylabel(\"Error Bound\", fontsize=16)\n",
    "plt.xlabel(\"Iteration\", fontsize=16)\n",
    "\n",
    "# Note that here we ignore a few things: error estimation for the data term and also quantization of lambda"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Results\n",
    "\n",
    "The authors apply the method to a bunch of small architectures on MNIST and get non-vacuous error bounds.\n",
    "\n",
    "<style type=\"text/css\">\n",
    ".tg  {border-collapse:collapse;border-spacing:0;}\n",
    ".tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:black;}\n",
    ".tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:black;}\n",
    ".tg .tg-0pky{border-color:inherit;text-align:left;vertical-align:top}\n",
    ".tg .tg-0lax{text-align:left;vertical-align:top}\n",
    "</style>\n",
    "<table class=\"tg\">\n",
    "  <tr>\n",
    "    <th class=\"tg-0pky\"></th>\n",
    "    <th class=\"tg-0pky\">600</th>\n",
    "    <th class=\"tg-0pky\">1200</th>\n",
    "    <th class=\"tg-0pky\">600, 600</th>\n",
    "    <th class=\"tg-0pky\">1200, 1200</th>\n",
    "    <th class=\"tg-0lax\">600, 600, 600</th>\n",
    "    <th class=\"tg-0lax\">R-600</th>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-0pky\">Train Error</td>\n",
    "    <td class=\"tg-0pky\">0.001</td>\n",
    "    <td class=\"tg-0pky\">0.002</td>\n",
    "    <td class=\"tg-0pky\">0.</td>\n",
    "    <td class=\"tg-0pky\">0.</td>\n",
    "    <td class=\"tg-0lax\">0.</td>\n",
    "    <td class=\"tg-0lax\">0.007</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-0pky\">Test Error</td>\n",
    "    <td class=\"tg-0pky\">0.018</td>\n",
    "    <td class=\"tg-0pky\">0.018</td>\n",
    "    <td class=\"tg-0pky\">0.016</td>\n",
    "    <td class=\"tg-0pky\">0.016</td>\n",
    "    <td class=\"tg-0lax\">0.15</td>\n",
    "    <td class=\"tg-0lax\">0.508</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-0pky\">Bound</td>\n",
    "    <td class=\"tg-0pky\">0.161</td>\n",
    "    <td class=\"tg-0pky\">0.179</td>\n",
    "    <td class=\"tg-0pky\">0.186</td>\n",
    "    <td class=\"tg-0pky\">0.223</td>\n",
    "    <td class=\"tg-0lax\">0.201</td>\n",
    "    <td class=\"tg-0lax\">1.352</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-0pky\">#Parameters</td>\n",
    "    <td class=\"tg-0pky\">471k</td>\n",
    "    <td class=\"tg-0pky\">943k</td>\n",
    "    <td class=\"tg-0pky\">832k</td>\n",
    "    <td class=\"tg-0pky\">2384k</td>\n",
    "    <td class=\"tg-0lax\">1193k</td>\n",
    "    <td class=\"tg-0lax\">472k</td>\n",
    "  </tr>\n",
    "</table>\n",
    "\n",
    "- Nonvacuous bounds in over-parameterized regime\n",
    "- Still very much not tight\n",
    "- On randomly labeled data (last column) the bound predicts high error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "py37",
   "language": "python",
   "name": "py37"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}