TensorFlow Basics

tensorflow-basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup the Problem\n",
    "\n",
    "I'm going to model the **quadratic** $y = 4x^2 - 3x + 17$ in the shape of $ y = ax_1^2 + bx_2 + c$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PEP-8(-ish) imports\n",
    "For glory and simplicity, we import the minimum of the \"Python for numerical computation\"-universe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# for the numerical compute\n",
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# for graphing the changes in data\n",
    "%matplotlib inline\n",
    "import pylab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data and (brief) Exploration of the Data\n",
    "Sometimes called \"EDA\" for:\n",
    "__E__xploratory\n",
    "__D__ata\n",
    "__A__nalysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# random data\n",
    "def generate_data(count=1000):\n",
    "    x_data = np.random.rand(count).astype(np.float32) # data for ax^2 component\n",
    "#     x_1_data = np.random.rand(10 ** int(np.log10(count))) # data for bx component\n",
    "    noise = np.random.normal(scale=0.01, size=count) # just to make the number jiggle out of place a little\n",
    "    \n",
    "    y_data = 4 * (x_data ** 2) - 3 * (x_data) + 17 + noise # our lovely equation\n",
    "    \n",
    "    return x_data, y_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X, target = generate_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Note about debugging\n",
    "It's often worthwhile it check-in with whatever data you're working with, to be sure that your code is doing \n",
    "  what you think it aught to.\n",
    "* In Software Engineering, this mentality is often implemented with _Test-Driven Development_\n",
    "* In Machine Learning, you can do TDD as well, but ML is more of a _pipelining_ endeavor\n",
    "  * Your model may be just __one__ of many pieces of a system\n",
    "    * For instance, Google Lens (in Google Translate) does something like\n",
    "      1. Object Detection\n",
    "        * Which is itself __segmentation__ and __classification__\n",
    "      2. Optical Character Recognition (Natural Language Understanding)\n",
    "      3. Translation (Natural Language Processing)\n",
    "      4. Font-Duplication (akin to the now-famous StyleTransfer)\n",
    "      5. Augmented Reality (by superimposing the translation, in-place)\n",
    "  * You can simulate those other pieces, to make sure your part of the system does what it does __best__\n",
    "    * Refering back to the Google Lens example, your translation model works against words.\n",
    "      * Heck, it might not even be a model, but a one-to-one look-up (for the sake of prototyping)\n",
    "    * The interface between the teams that construct these would be negotiated, and __that__ is where the \n",
    "      Software Engineering principles come in.\n",
    "      * Your design of your subsystem will be decoupled from the rest of the pipeline\n",
    "\n",
    "Your job is just to produce a broad an output as possible (if the other team is combative) and be able to get your input to whatever shape your model needs it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's plot a chart\n",
    "\n",
    "Just to make sure we made the correct kind of parabola"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0xb28530f60>]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3X141PWZ7/H3PXlAUdSUJ0UEDAoV0VqIinYFq9hW1hYfWhXbXd1aAc9ut+1ud+vaandt7cFuPceeq14FtNRjayg+IHV7YKvuVrGVRDKpLQ+pgikTIwoBRqUi5Ok+f8wMHZLfJJNMMpOZ+byui8vM7/ubmftnkjvfuX/fB3N3RESkeIRyHYCIiGSXEr+ISJFR4hcRKTJK/CIiRUaJX0SkyCjxi4gUGSV+EZEio8QvIlJklPhFRIpMaa4DCDJq1CifNGlSrsMQEckb4XB4j7uPTufcIZn4J02aRF1dXa7DEBHJG2YWSfdclXpERIqMEr+ISJFR4hcRKTJK/CIiRUaJX0SkyCjxi4gUmbQSv5mtMLPdZrY56dg5ZlZjZi+bWZ2ZnZfiuTea2bb4vxsHKnARkUISjkS5/1fbCUeig/5e6Y7jfwj4AfBw0rHvAv/m7uvMbF788cXJTzKzDwDfBKoAB8Jm9pS7D/6ViYjkiXAkymcfrKG1vZPy0hCPfGEWMydWDNr7pdXjd/f1wL6uh4Hj4l8fD+wMeOrHgWfcfV882T8DfKKfsYqIFKTV9c0cbOuk06G1vZOaxr2D+n6ZzNz9MvBLM/sesT8gFwacczLwetLj5vgxEREh1ttftbHp8ONOh4rh5YP6npnc3L0V+Iq7nwJ8BfhRwDkWcMyDXszMFsbvFdS1tLRkEJaISP6oadxLR+eRx6IHWgf1PTNJ/DcCq+NfPwYE3dxtBk5Jejye4JIQ7r7c3avcvWr06LTWGRIRyXuzKkdSkpSJy0tDzKocOajvmUni3wnMiX99CbAt4JxfAh8zswozqwA+Fj8mIiLAM1veOtzjDxn86yfPHNQbu5Bmjd/MVhIbsTPKzJqJjdS5Bfi+mZUCB4GF8XOrgMXu/gV332dm3wI2xl/qLnfvepNYRKQoVdc2sXR94+HHnQ5bdr4z6O+bVuJ39wUpmmYGnFsHfCHp8QpgRb+iExEpYOs2v9ntWOBN0AGmmbsiIjly+fSTjnhcEoJrZowf9PcdkhuxiIgUunAkSvRAK4tnV7KhcS9jjzuKRXMmD3p9H5T4RUSyrrq2iTvWbKLDobzEWLnwgqwk/ASVekREsqi6tomvPxlL+gCtHc4T9c1ZjUGJX0QkS8KRKHes2dTtBu6e/YeyGocSv4hIltQ07j3c088lJX4RkSzZtmt/4PHRI4ZlNQ4lfhGRLAhHojz1u+4r1oQMrs7CEM4j3jOr7yYiUqRW1zfTGVDm+faVZ2V1RA8o8YuIDLpwJMrPkpZeTvjQ+OO54fwJWY9HiV9EZJDds66h29LLANedm/2kD5rAJSIyqMKRKBt3HLnb7LHDSrh93rSc9PZBPX4RkUG1ur6527j9XCZ9UOIXERlU9ZEje/sTPzA8p0kflPhFRAZNdW0TDW8dOXb/hOFlOYrmz5T4RUQGQTgS5RtPbup2/IJB3lYxHb0mfjNbYWa7zWxz0rFVZvZy/N8OM3s5xXO/YmZbzGyzma00s6MGMngRkaFqyboGAgbyMOLo/OjxPwR8IvmAu1/n7ue4+znAE/x50/XDzOxk4O+BKnefDpQA12ccsYjIEFdd29RtJA9AScgGfSP1dPQ6nNPd15vZpKA2MzPgWmKbrad6/aPNrA0YTmyDdhGRgha0pWLI4Fvzp2d9lm6QTGv8FwG73H1b1wZ3fwP4HtAEvAm84+5PZ/h+IiJDngUc+/aVZ+V8NE9Cpol/AbAyqMHMKoD5wKnAOOAYM/tcqhcys4VmVmdmdS0tLRmGJSKSG+FIlBe27el2PHqgNQfRBOt34jezUuBqYFWKU+YCf3T3FndvI3Yf4MJUr+fuy929yt2rRo8e3d+wRERyatnzr3WbsFUaYkjU9hMy6fHPBf7g7qn2DGsCZpnZ8Pi9gEuBhgzeT0RkSAtHovx6+5G9/eHlJaxadOGQqO0npDOccyWwAZhqZs1mdnO86Xq6lHnMbJyZrQVw91rgcaAe2BR/r+UDGLuIyJCxZG0D1/zwRQ60dhxx/COnjRpSSR/SG9WzIMXxmwKO7QTmJT3+JvDNDOITERnyqmubWLq+sdtxAxbPmZz9gHqhmbsiIhlaFbDWPsCi2ZVDrrcPSvwiIhl7+0Bbt2NnnDiC2+adkYNoeqfELyKSgSVrG4jsO9Dt+IeHYE8/QYlfRKSfwpEoy18Iru1fk+UN1PtCiV9EpJ+WPv9a4Abql00bOyRr+wlK/CIi/bBkbQPPbN3V7XhJyFg0BEfyJFPiFxHpo1QlHhg6C7H1RIlfRKSPahr34gElnqGwrWI6lPhFRPpo26793dbjAfiL00dlPZb+UOIXEemDJWsbWPNy961FyktDXD2ER/Ik63XJBhERiQmq7Rtww/kTuHrG+CFf209Q4hcRSVNQbX/R7MohO0M3FZV6RETSVDG8HItvr2XA4jxM+qAev4hIWqprm/j6k5sO39R1YMLIY3IZUr+pxy8i0otwJMo3kpJ+QqpVOYc6JX4RkR6EI1Hu+o8tdAa0jT3uqKzHMxB6LfWY2QrgCmC3u0+PH1sFTI2fcgLwtrufE/DcE4AHgenEPhl93t03DFDsIiKDKhyJsuCBGlrbu6d9gyG/NEMq6dT4HwJ+ADycOODu1yW+NrN7gXdSPPf7wH+6+6fNrBwY3v9QRUSya3V9c2DSB7j7qrPyZvhmV+lsvbjezCYFtcU3Ub8WuCSg7ThgNnBT/HVagdb+hyoikl1Bs3MhtvpmPizNkEqmNf6LgF3uvi2grRJoAX5sZr81swfNLOUtcDNbaGZ1ZlbX0tKSYVgiIpm7ZsZ4SpOypAHlJTYk99Hti0wT/wJgZYq2UmAG8EN3/zDwHnBbqhdy9+XuXuXuVaNHj84wLBGRzIQjUVbXN4MZBpSVGAvOn8DKhRfkbYknod/j+M2sFLgamJnilGag2d1r448fp4fELyIyVATd1O3sdE4+4ei8T/qQWY9/LvAHd28OanT3t4DXzSwx+udSYGsG7ycikhVBN3VLQsasypE5imhg9Zr4zWwlsAGYambNZnZzvOl6upR5zGycma1NOvRF4BEz+z1wDvCdgQlbRGTwBN3U/UzVKQXR24f0RvUsSHH8poBjO4F5SY9fBqoyiE9EJGvCkShP1Dfzm217jjheEiJvllxOh9bqEREhXtdfvoHWju79/enjji+Y3j5oyQYRESC25HJQ0ge47tz8HbMfRIlfRATY/35b4PHFsyvzerJWECV+ESl6QTtrQf6ut98bJX4RKXrLnn+NzoAqz4ijy7IfTBYo8YtIUfvyz37L01t3dTteVlI44/a7UuIXkaJVXdvEmpd3djs+/oSj+FkBLM2QihK/iBStFb/uXtcHmFZgwze7UuIXkaIUjkR5reW9wLbRI4ZlOZrsUuIXkaK0ZF1D4NIMZSVWULN0gyjxi0jRqa5tYuOOaLfjo48tL+jafoISv4gUnRW/+WPg8XEFsuxyb5T4RaSohCNRtu/+U2BboS3NkIoWaRORorLs+de6HTu54mj+9uLTCm5phlTU4xeRotLY0r23f8N5E4om6YMSv4gUkeraJhr3HDmEs5Bn6KbSa6nHzFYAVwC73X16/NgqILGl4gnA2+5+TornlwB1wBvufsWARC0i0geJDVZWbXz9iDV5ThtzLPdcc3ZR3NBNlk6N/yHgB8DDiQPufl3iazO7F3inh+d/CWgAjutfiCIi/Vdd28QdazbRdan90pAVZdKHNEo97r4e2BfUZmYGXEuXvXeT2scDfwk8mEGMIiL9Eo5EUyb9u+ZPL8qkD5nX+C8Cdrn7thTt9wH/DHSmaD/MzBaaWZ2Z1bW0tGQYlohIbFetrknfgLvmTy+qm7ldZZr4F5C6t5+4LxBO54Xcfbm7V7l71ejRozMMS0QEKoaXdzvmQPRAa/aDGUL6PY7fzEqBq4GZKU75CPApM5sHHAUcZ2Y/dffP9fc9RUTSFY5EWbWxqdvx8tJQ0Y3i6SqTHv9c4A/u3hzU6O7/4u7j3X0ScD3w30r6IpIN4UiU65dv4HfNR447OW3Msay8ZVbR1vYTek38ZrYS2ABMNbNmM7s53nQ9Xco8ZjbOzNYOfJgiIulb9vxrtHUp7hsU7Siernot9bj7ghTHbwo4thOYF3D8OeC5PkcnItIP9a93X3nzqLKQkn6cZu6KSEFZsraBPfu737wde9xROYhmaNIibSJSEMKRKKvrm3mktvsNXYALTxuV5YiGLiV+Ecl74UiUzz5Yw8G24ClDBlxT4Ltq9YVKPSKS92oa99Lannqe6KLZlarvJ1GPX0Ty3qzKkZSGjNYuI3mOLivhjiumFfUs3SDq8YtIXkusvDlh5DHd2pT0g6nHLyJ5KxyJct2yF+la5QkZLLyoUkk/BfX4RSRvLVnX0C3pQyzp3zbvjOwHlCeU+EUkL1XXNrFxR/eJWgBb3nw3y9HkFyV+Eck74UiU7z39Ssr2y6eflMVo8o9q/CKSV6prm7jj55vp6PTA9sumjVVtvxfq8YtI3ghHotzZQ9IvKzEWz5mc5ajyjxK/iOSNmsa9KZM+wLSTjtNErTQo8YtI3tj/fhup0z5cd65KPOlQ4heRvFBd28TS9Y0p2688Z5xq+2nSzV0RyQsrfh2c9I3YWjwat5++dHbgWmFmu81sc9KxVWb2cvzfDjN7OeB5p5jZr8yswcy2mNmXBjp4ESkOS9Y2sGPvgW7HzeDuq85S0u+jdHr8DwE/AB5OHHD36xJfm9m9wDvdn0Y78I/uXm9mI4CwmT3j7lszC1lEismStQ2BJZ4PDC/jgRvP1c3cfui1x+/u64F9QW1mZsC1dNl7N/68N929Pv71fqABODmjaEWkqIQjUX5aGwlsO+UDw5X0+ynTm7sXAbvcfVtPJ5nZJODDQG0P5yw0szozq2tpackwLBHJd+FIlGuXbeBPhzoC28doK8V+yzTxLyCgt5/MzI4FngC+7O4pF9Bw9+XuXuXuVaNHj84wLBHJd8uefy3lmP0QaKJWBvo9qsfMSoGrgZk9nFNGLOk/4u6r+/teIlJ86l8PXoBt1LHlLPurKpV5MpBJj38u8Ad3bw5qjNf/fwQ0uPv/yuB9RKTIVNc2sWd/a2Cbkn7m0hnOuRLYAEw1s2YzuznedD1dyjxmNs7M1sYffgT4K+CSpKGf8wYwdhEpQOFIlCXrGgLbFmvv3AHRa6nH3RekOH5TwLGdwLz4178mNrdCRCQtS9Y2sGx9Y+CyDKNGlGu8/gDRzF0RGRJ6W5LhH+ZOzWI0hU2JX0RyLhyJ8r1f/iGwbXhZiG9ccabW4RlASvwiklPhSJTPPljDwbaAzXOBq2aMV9IfYFqdU0RyqqZxb8qkX14a4uoZ47McUeFT4heRnKmubeKB9a8Ftp025lhW3jJLo3gGgUo9IpIT1bVN3P7kppTtcz84Rkl/kKjHLyI5sW7zmz22b3kz5QovkiElfhHJuuraJhpb/tTjOZdPPylL0RQflXpEJKtSra+fcOJxw/j7S6doJM8gUuIXkazpLemXlRj3f3amavuDTIlfRLKit5m5Hxp/PHd+8kwl/SxQjV9EsqKnm7mlIVPSzyIlfhEZdOFIlKPLSgLbDLhr/nQl/SwqqFJPOBKlpnEvsypH6odIZIiorm3ijp9vTrmb1qLZlbqRm2UFk/gT6320tndSXhrikS9oxp9IroUjUe5Ys4mOLjnfgIkjh7Nw9mQl/RxIZyOWFWa228w2Jx1blbS5yg4zeznFcz9hZq+Y2XYzu20gA++qpnEvre2ddDq0tXdS07h3MN9ORNJQ07i3W9IHGFYW4t5rz1HSz5F0evwPAT8AHk4ccPfrEl+b2b3AO12fZGYlwP3AZUAzsNHMnnL3rRnGHGhW5UjKS0O0tXdSVhpiVuXIwXgbEUlDdW0T6za/ychjyru1nTepgq9dfoY+kedQOjtwrTezSUFt8X11rwUuCWg+D9ju7o3xc38GzAcGJfHPnFjBnVecybrNb3L59JP0QyWSI72twVPf9HYWo5Egmdb4LwJ2ufu2gLaTgdeTHjcD52f4fimFI1HufGoz7R3Ohsa9TD1xhJK/SA6s2tjUY3unOzWNe/X7mUOZDudcQJcN15ME7bcbfFsfMLOFZlZnZnUtLS19DmTZ86/RHi8mtnc4y54PXupVRAbX2wfaemwvVyk25/rd4zezUuBqYGaKU5qBU5Iejwd2pno9d18OLAeoqqpK+QcilcY97x3x+PlXW5j1nWe58pyTtUGzyCBLDKXetms/kX0HUp73sWljWTRnsnr7OZZJqWcu8Ad3b07RvhE43cxOBd4ArgduyOD9elQ56hi27/7zan+H2jt5691DLF3fyNY33+XhmwetyiRS1BJDqQ+1dab8SD9JQzeHlHSGc64ENgBTzazZzG6ON11PlzKPmY0zs7UA7t4O/B3wS6ABeNTdtwxk8MkWzZlMaUlQdQnWb9tDdW3PdUcR6Z/EUOpUST9kaOjmEGPufa6qDLqqqiqvq6vr8/PCkShfe+L3R/T8E04+4Sh+c9ulAxGeiCQJR6Jcu2xD4MxcA+6+6iwl/Swws7C7V6VzbkGt1TNzYgXnnfqBwLY33j5IOBLNckQihe+Vt/YHJv3zJlXw+K0XKukPQQWzZEPCNTPGpyzr3LOugTlTx2gtH5EMJW7m7n+/LXCp5dNGH8Ojiy/MQWSSjoJL/DMnVnDepApe2tG9d//Sjigv7YhSGjLumj9dPRGRfghHoix4ILYuViqnjj42ixFJXxVUqSfha5efETiJIKG907nz55tV+hHph9X1zT0m/RKDxXMmZzEi6auCTPwzJ1Zw91Vn9XhxHZ3Ofc++quQv0kev7tqfsi1R4lEpdWgryMQPcMP5E3js1gv50PjjA9sdeGHbHj77YI2Sv0iawpEoGwPKqAClJcY9n/6Qkn4eKNjED/GF2z55JuUpxvcDHGrTEs4ivQlHonxm6Yvc8EBNYLsB11adoqSfJwo68UMs+a9ceEHKnj9AxfDuS8eKSEw4EuUzP3yRjTuiHEpR2x9WFuKaGeOzHJn0V8Enfvhzzz8U0PF34JtPbeb2Jzep5CMS4In6ZlLdyjViZVXteJdfiiLxQyz5P7b4Qs6bVMHw8iM3fW7rcKprm1TvF0kSjkS5/1fbebkp9e/E3VedxXeuOktJP88UTeKHWPJ/dPGF/O1HTwtsP9jWyRP1qdacEykeiYXX7n36Fba+GTyK57xJFZoLk6cKbgJXOmZVjqTECNwLdOVLTUwfdzxTTxxBTeNezfKVopS8h3WQkpDxtcu13Hm+KsrEP3NiBd+68izuWLOpW/J3h68/uYmSEHR6bNMI1S+l2FQML0+Z9D80/nju/OSZ+p3IY0VV6kl2w/kTeHRxbAGprjd9HWjvjCX+tnYN95TiEz3QGng8ZCjpF4Ci7PEnzJxYwcyJFUwfd3zKzaFLQqZt4qRohCNR7lnXQMNb3ev6Bnz7St3ILQRFnfgTbjh/As+9spunt+7q1jZ6xLAcRCSSXeFIlNX1zax8qSmwxFNi8K0rta5+oUhnB64VZrbbzDZ3Of5FM3vFzLaY2XdTPPcr8fbNZrbSzI4aqMAH2qI5kwMXdnvj7YNcu/RFDfOUghWORFmwfAOP1AYn/YtOH3W4LCqFIZ0a/0PAJ5IPmNlHgfnA2e5+JvC9rk8ys5OBvweq3H06UEJsu8YhaebE2KYRk0YO79bW4XDTj2u1faMUnHAkytce/x2tQUPcADP48twpKu8UmF4Tv7uvB/Z1OXwrsMTdD8XP2Z3i6aXA0WZWCgwHdmYQ66CbObGC5/7po3znqrO6te0/2MHtT27iM+r9S4EIR6Jct3wD21veC2wPGdytmn5B6u+oninARWZWa2bPm9m5XU9w9zeIfRJoAt4E3nH3p/sfavbccP4EFs+uDGzbuCP2sVjJX/LdPesaaE/R0//YtLE8pvJOwervzd1SoAKYBZwLPGpmlZ60c7uZVRArB50KvA08Zmafc/efBr2gmS0EFgJMmJD7H7bb5p3BW+8eZM3L3T+ktHY4NY171ROSvJO8ZWLQLnUJHzrlBP18F7D+Jv5mYHU80b9kZp3AKKAl6Zy5wB/dvQXAzFYDFwKBid/dlwPLAaqqqlJMHcmu08eOwIiN6++qujZCxfBy9YgkbySWYWht78R7+A0rK9EQ5kLX31LPGuASADObApQDe7qc0wTMMrPhZmbApUBDfwPNhVmVIxlWFvy/6I23D3L7k5tYsjavLkmKWE3jXg62xZZhSJX3z5tUwc8WXqDefoHrtcdvZiuBi4FRZtYMfBNYAayID/FsBW50dzezccCD7j7P3WvN7HGgHmgHfku8R58vZk6s4JEvzOK+Z1/lhW1d/67FLHuhkcvOPBFAa/vIkBWORPnJizsC2648Zxynjx2hn90iYt7TZ74cqaqq8rq6ulyHcVjiI/LBtuBVyceMKGfvn1pxtLaPDD2J0TtBN3IN+OrHp6ZcsVbyh5mF3b0qnXOLdq2evkj0/D97/gRKA/6P7d7fSofH1vbRVo4y1Kyub045eqesNKR6fhHSkg1pSqzrc/WM8TxR35xyMpcD23YFr18ukk2JETz1KYYenzbmWO655mx9Oi1CSvx9lPgD8NIf97F9958Cz1nz8k7OO3WkRvxIzlTXNnHnzzfTnmJtZQMl/SKmUk8/ff4jp/bY/j/XbuWvfqRlHiT7wpEo31izKWXSh9iWiUr6xUuJv59uOH8Cl00bm7J9/6EOXti2h9uf3MQtD9dppq9kRTgS5Usr61NuomLAd67SKpvFTqWeDCyeM5kXtrWkHO2T8MzWXbywrUWjfWTQVNc2cf9z23kj+n7Kc7S0siSox5+BxGifi04fFbikc7JD2shdBkl1bRO3P7kpZdI3jtxxTkQ9/gzNnFjBl+dOYeOOfRxq60w5I9KBx+pe55oZ49Xrl4wlr7nz09pIj+ferdKOdKHEPwASPf/V9c2s2thEe4rKT1uH8zc/fonbLj9Dv4jSb72N2En2sWlj9bMm3SjxD5Dkcf5Ln3+N/2rYFXiD7d2D7Yf399UvpPRVOBJNO+mXl4ZYNGdyFqKSfKPEP8BmTqzggb+uOryHaapPAPc/t53ogVatjyI9SpR0Ej8nq+ub0+7pL5ozWT9bEkhr9Qyy2H6mNbR2BNd/DDh3UgVXfni8/hDIEZKXUS4vDXHnFWdyx5pNpFh9AYBJI4ezcPZkfZosQn1Zq0c9/kE2c2IFn//IJJaubwxsd+ClHdHDm2IcVaZF3iSmpnEvre2xZZQPtnXyrV9s6THpl4aMe689Rz870isl/iy4bd4ZADxa9zr7DrT1eG5rfJE3/fIWr0SZcPf+Q5SEjM54tn+/h/kipSHjrvnT9XMjaVHiz5Lb5p3BbfPOYNZ3nuWtdw+lPtHQaolFrLq2iTt+vpmONOr4l00by+I5k7UPhPSZEn+WXXrGWB7pYf2eTodFP6nj0zPGM+LoMv1CF5ElaxtYtr4x5VyQZKWh2MzxxGgykb7odeauma0ws93x3baSj3/RzF4xsy1m9t0Uzz3BzB43sz+YWYOZXTBQgeerq2eMpzxoUf8ke/7UytL1jfz7L19hwQM13P7kJq31U+Cqa5tYmmbSB7ju3AlK+NJv6fT4HwJ+ADycOGBmHwXmA2e7+yEzG5Piud8H/tPdP21m5cDwDOPNezMnVrDyllmHZ10+8Os/9vixvrW9k5W1Tayub9ZN3wKUGK759Ja3ejzvxBHD2LU/ViIcVhbi6hnjsxGeFKheE7+7rzezSV0O3woscfdD8XN2d32emR0HzAZuip/TSmx/3qKX/PH8sjNP5K7/2MLvmt9Jeb4TG9Whm76FJRyJct2yF1PO9E52ybSxXDNjvOr5MiD6W+OfAlxkZncDB4GvuvvGLudUAi3Aj83sQ0AY+JK7v9fvaAvQzIkV3PnJM1mwfAOtPY3VA370QiO/e/1tRo0YpjV/8liil/+L3+1MK+lDbL6H6vkyUPqb+EuBCmAWcC7wqJlV+pGzwUqBGcAX3b3WzL4P3AbcEfSCZrYQWAgwYUJxTT6ZObGClQsvYOnzr/Fswy5Szanbd6CNp7fuAmDVS03cclGlbgDnmeRJWWkM3MGI7Yur0o4MpLRm7sZLPb9w9+nxx/9JrNTzXPzxa8Asd29Jes6JQI27T4o/vgi4zd3/srf3K6SZu33VlwW4IJYYhmnS15AWjkR5or6Zl5uiNO07wHuHOnq9ifuB4WV89eMf1GxuSVs2Zu6uAS4BnjOzKUA5sCf5BHd/y8xeN7Op7v4KcCmwtZ/vVzRuOH8CU08cwX3PvsoL2/b0en6i/v9EfbOSwxAUW7Kj9zJeV1/9+Ae17IIMml4Tv5mtBC4GRplZM/BNYAWwIj7EsxW40d3dzMYBD7r7vPjTvwg8Eh/R0wj8zSBcQ8FJXuO/rT22xn9vHwBW1jaxfdd+Ths7QvX/ISIciXLfs6/2KemfNuZYPv+RU5X0ZVBpkbYhLHETcOfb71Nd25T2GG/Qvqq5lqjl97YtZzJ9zyQTWqStQCRGcSRqxK1tnaSbRr7+5CamnjhCPf8sSV4+GeCu/9iipC9Dlnr8eSKRWLbt2s+al3em9ZwRw0q4YPIoLp46RjcJB1F/evcAJx43jLPHn6B182VAqMdfgJLHcB9o7Tg8rLMn+w/FzkucWxIyvjV/unqWGUru3T+z5S1+Vvd6n5P+UWUh7v/sTCV8yQkl/jy0aM5k1m9roS0+Fjzdz2wdnc7Xn9zElp3vcHWXG8Bdd3qSYIlROm0d3qd7LhC7cTv3g2M090JyTqWePJVI1Pvfbztik5dhpSEOpTEdtLw0xMpbYmP/E3MHOt0pL9WcgK6qa5tYtbGJYaUh9r3XyvaWvk8+N+CrH5/K3370tIEPUASVeopCculnwshjWLf5TS6ffhLA4c3ce9La3sl1yzZwzFElvHug/XAbFSCkAAAIlUlEQVTv9VBbJ/c9+ypfnjulqJN/4ob69l37D++O1h8hi/23vDSkfRZkyFCPvwD1ZV33VEpCcP25E7qVhApVYtcrB6aPO547f74p7XV0glx0+ii+PHcKgEpokhV96fEr8ReoRI91T3wp33RuBqeyeHbl4e0jC0XX4ZfXLtuQ1q5X6dC+yZILKvVIt5UcFz5c1+/kv3R9I/VNUb52+Rn8ZMMOnnu1hYunjOa+6z88QNEOvkSirxhezuad7/B4uJn2jk7KS0OcffLx/U76xp9vrhvwF/GevpK+DGXq8ReJ/q4Z05MTRwzj7FNOOLxMNOS+rJGc4BNzFwAWPBBbETPI8LIQB/o4HDNh8exKHtqwg7b2Tsp0Y1xySKUeCdS1vHHND18csNcuCQEYnZ1OKGRc8sExh/eETSeWvv7BCBp+mphIdait83AvvMTguKNLiR5oz/AKu0vMttVQWBkKlPglLYeHcXY6oRBMGnUs23f/acBevyQUu1F6QeXIw2PXAVbXN/NY3etHLD3tHptg9oW/OLXbOPfkETaH2ju5oHIkD23YQWt75+Hhp0CvO5kNlJDBt6/UEgsytCjxS9q69lara5vSGg7aVwaUlRiYpSy5JCsNwapFFwL0WqI6dlhJWmvc90dpyLhr/nQ273yHPfsPafczGbKU+CUj4UiUe9Y1ZDR+fSCcXHE00046jmcyGJGUCS1xIflEiV8GRDgS5WtP/H5Ayz9DTWmJcdenphM90Hp4xM+e/YcYPWJY0cxhkMKg4ZwyIGZOrOCea85mwQM1tLV3YgbHlJdQEjLePdiO92GdoKGmNATXFdEENZFk6ezAtQK4Atid2HM3fvyLwN8B7cD/c/d/TvH8EqAOeMPdrxiQqCVrZk6sYOUtswJH0CTWCnrw13+k0x0z6Owcmn8Mxp9wFNPGHQ+gOr0UvXR6/A8BPwAeThwws48C84Gz3f2QmY3p4flfAhqA4zKIU3Ko62SwrscuO/PEI4ZmJmYMP/fKbjo6Y6tYloZClITg6LISZk8ZzXutHTy7ddfhPxInHF3G2++3DUr85aUhvr9ghhK9SFyvid/d15vZpC6HbwWWuPuh+Dm7g55rZuOBvwTuBv4ho0hlyOr6h6Hrp4JU49uDRhSt+HUj77d38ubb79MZH+IJTkd8IFCJwfSTY0NE9x9qP7y2zr8+tZnWDseA+eeM4/SxI46YxKWkL/Jn/a3xTwEuMrO7gYPAV919Y8B59wH/DIzo5/tIHgv6pNBT+w3nTzg8gqbrBK/EAmqpSjRTTxyhSVQiaepv4i8FKoBZwLnAo2ZW6UlDhMwscV8gbGYX9/aCZrYQWAgwYYKGzxW7VJ8i0j1fRFIL9fN5zcBqj3kJ6ARGdTnnI8CnzGwH8DPgEjP7aaoXdPfl7l7l7lWjR4/uZ1giItKb/ib+NcAlAGY2BSgH9iSf4O7/4u7j3X0ScD3w3+7+uQxiFRGRAdBr4jezlcAGYKqZNZvZzcAKoNLMNhPrzd/o7m5m48xs7eCGLCIimUhnVM+CFE3deu/uvhOYF3D8OeC5PsYmIiKDoL+lHhERyVNK/CIiRWZILtJmZi1ApJ9PH0WXG81FQNdcHHTNxaG/1zzR3dMaEjkkE38mzKwu3RXqCoWuuTjomotDNq5ZpR4RkSKjxC8iUmQKMfEvz3UAOaBrLg665uIw6NdccDV+ERHpWSH2+EVEpAd5mfjN7BNm9oqZbTez2wLah5nZqnh7bcB+AnknjWv+BzPbama/N7P/MrOJuYhzIPV2zUnnfdrM3MzyfvRHOtdsZtfGv9dbzKw62zEOtDR+tieY2a/M7Lfxn+9uqwPkGzNbYWa748veBLWbmf2f+P+T35vZjAENwN3z6h9QArwGVBJbHO53wLQu5/wPYGn86+uBVbmOOwvX/FFgePzrW4vhmuPnjQDWAzVAVa7jzsL3+XTgt0BF/PGYXMedhWteDtwa/3oasCPXcQ/Adc8GZgCbU7TPA9YBRmz5+9qBfP987PGfB2x390Z3byW2SNz8LufMB/5v/OvHgUvNzLIY40Dr9Zrd/VfufiD+sAYYn+UYB1o632eAbwHfJbYhUL5L55pvAe539yik3v0uj6Rzzc6ft249HtiZxfgGhbuvB/b1cMp84GGPqQFOMLOTBur98zHxnwy8nvS4OX4s8Bx3bwfeAUZmJbrBkc41J7uZWG8hn/V6zWb2YeAUd/9FNgMbROl8n6cAU8zsN2ZWY2afyFp0gyOda/5X4HNm1gysBb6YndByqq+/833S3x24cimo5951aFI65+STtK/HzD4HVAFzBjWiwdfjNZtZCPjfwE3ZCigL0vk+lxIr91xM7FPdC2Y23d3fHuTYBks617wAeMjd7zWzC4CfxK+5c/DDy5lBzWH52ONvBk5Jejye7h/9Dp9jZqXEPh729LFqqEvnmjGzucDXgU+5+6EsxTZYervmEcB04Ln4Lm+zgKfy/AZvuj/bP3f3Nnf/I/AKsT8E+Sqda74ZeBTA3TcAR9F9x79Ck9bvfH/lY+LfCJxuZqeaWTmxm7dPdTnnKeDG+NefJrb7Vz73+Hu95njZYxmxpJ/vdV/o5Zrd/R13H+Xukzy2y1sNsWuvy024AyKdn+01xG7kY2ajiJV+GrMa5cBK55qbgEsBzOwMYom/JatRZt9TwF/HR/fMAt5x9zcH6sXzrtTj7u1m9nfAL4mNCFjh7lvM7C6gzt2fAn5E7OPgdmI9/etzF3Hm0rzmfweOBR6L38ducvdP5SzoDKV5zQUlzWv+JfAxM9sKdAD/5O57cxd1ZtK85n8EHjCzrxArd9yU5x25xM6GFwOj4vcuvgmUAbj7UmL3MuYB24EDwN8M6Pvn+f8/ERHpo3ws9YiISAaU+EVEiowSv4hIkVHiFxEpMkr8IiJFRolfRKTIKPGLiBQZJX4RkSLz/wEEPJTTGts0QgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pylab.plot(X, target, '.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the numbers, and test it with a few inputs. Do the numbers make sense?\n",
    "* At $x = 0$, we're touch $17$, so that looks good\n",
    "* At $x \\approx 0.4$ we see a value closer to 16.4. Does this make sense?\n",
    "  1. $4 \\times .4^2 \\approx 4 \\times .15 = .6$\n",
    "  2. $.6 - 3(.4) = .6 - 1.2 = -1.8$\n",
    "  3. $y = -1.8 + 17 + x$ where `x` is rather small should be in the range $(15.2, 17)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def build_model_graph(X, return_all=False):\n",
    "    \"\"\"Create TensorFlow graph that is aware of the training data:\n",
    "    X: some numpy vector-array of numbers\n",
    "    \"\"\"\n",
    "    W_1 = tf.Variable(tf.random_uniform([1], 0.0, 5.0), name='a-coefficient') # scalar, between 0 and 5\n",
    "    W_2 = tf.Variable(tf.random_uniform([1], 5.0, 5.0), name='b-coefficient') # scalar [-5, 5]\n",
    "    c = tf.Variable(tf.zeros([]), name='constant-factor') # scalar\n",
    "    bias = tf.Variable(tf.zeros([]), name='bias-factor') # scalar, in case any noise needs to be counteracted\n",
    "    \n",
    "    y = W_1 * (X ** 2) + W_2 * X + c + bias\n",
    "    \n",
    "    if return_all:\n",
    "        return W_1, W_2, c, bias, y\n",
    "    else:\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, taking a second to make sure you built a graph that is executable (i.e. __did the thing you wanted to do__) is a good idea pretty much always."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"add_2:0\", shape=(1000,), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# DEBUG: making sure we built the graph properly\n",
    "with tf.Graph().as_default():\n",
    "    print(build_model_graph(X))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def build_training_graph(expected, result_tensor, learning_rate = 0.5):\n",
    "    \"\"\"Train a loss by the Root-Mean Squared-Error\n",
    "    expected: numpy data\n",
    "    result_tensor: TF output of the computation of the graph\n",
    "    learning_rate: learning rate for our Gradient-Descent optimization\n",
    "    \"\"\"\n",
    "    # RMSE vs MSE is something that can be debated. Swap them and see how the training changes\n",
    "#     loss = tf.sqrt(tf.reduce_mean(tf.square(expected - result_tensor)), name='RMSE-loss')\n",
    "    loss = tf.reduce_mean(tf.square(expected - result_tensor), name='my-MSE-loss')\n",
    "    optimizer = tf.train.GradientDescentOptimizer(learning_rate)\n",
    "    training_op = optimizer.minimize(loss)\n",
    "    \n",
    "    return training_op"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def main_loop(x_all, y_all, steps = 300, periods = 10):\n",
    "    # 10k examples, should be enough. Called \"_all\" because there's no train-test split\n",
    "    example_count = 10000\n",
    "    period = steps / periods\n",
    "    x_all, y_all = generate_data(example_count)\n",
    "\n",
    "    g = tf.Graph() # make a graph for us to play in\n",
    "    \n",
    "    with g.as_default(): # our graph is in charge of this scope\n",
    "        w_1, w_2, c, bias, y_eval = build_model_graph(x_all, return_all=True)\n",
    "        \n",
    "        train_op = build_training_graph(y_all, y_eval, learning_rate=0.3)\n",
    "        init = tf.initialize_all_variables() # so this can fill it with all the goodies\n",
    "        # one way of hooking up the graph (https://www.tensorflow.org/guide/graphs#programming_with_multiple_graphs)\n",
    "        sess = tf.Session()\n",
    "\n",
    "        sess.run(init)\n",
    "        y_init = sess.run(y_eval)\n",
    "        print(\"Initial stats:\", sess.run([w_1, w_2, c, bias]))\n",
    "\n",
    "        \n",
    "        for step in range(steps + 1):\n",
    "            sess.run(train_op)\n",
    "            if step % period == 0:\n",
    "                print(step // period, sess.run([w_1, w_2, c, bias]))\n",
    "        \n",
    "        print()\n",
    "        print(\"Final output as a quadratic:\")\n",
    "        print(f\"{sess.run(w_1)}x^2 + {sess.run(w_2)}x + {sess.run(c + bias)}\")\n",
    "    \n",
    "    print(\"Performance graph incoming\")\n",
    "    pylab.plot(x_all, y_all, '.', label=\"target_values\")\n",
    "#     pylab.plot(x_all, y_init, \".\", label=\"initial_values\") # they throw the graph off, horribly.\n",
    "    pylab.plot(x_all, sess.run(y_eval), \"g.\", label=\"trained_values\")\n",
    "    pylab.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/stephen/anaconda/envs/insight-prep/lib/python3.6/site-packages/tensorflow/python/util/tf_should_use.py:189: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.\n",
      "Instructions for updating:\n",
      "Use `tf.global_variables_initializer` instead.\n",
      "Initial stats: [array([4.5501237], dtype=float32), array([5.], dtype=float32), 0.0, 0.0]\n",
      "0.0 [array([6.652268], dtype=float32), array([8.393686], dtype=float32), 7.7176204, 7.7176204]\n",
      "1.0 [array([1.0373217], dtype=float32), array([0.0540356], dtype=float32), 8.229093, 8.229093]\n",
      "2.0 [array([2.023973], dtype=float32), array([-0.96313745], dtype=float32), 8.319319, 8.319319]\n",
      "3.0 [array([2.6820521], dtype=float32), array([-1.6415722], dtype=float32), 8.379498, 8.379498]\n",
      "4.0 [array([3.1209767], dtype=float32), array([-2.0940769], dtype=float32), 8.419636, 8.419636]\n",
      "5.0 [array([3.4137332], dtype=float32), array([-2.3958888], dtype=float32), 8.446407, 8.446407]\n",
      "6.0 [array([3.6089962], dtype=float32), array([-2.5971918], dtype=float32), 8.464264, 8.464264]\n",
      "7.0 [array([3.7392316], dtype=float32), array([-2.7314553], dtype=float32), 8.476173, 8.476173]\n",
      "8.0 [array([3.8260968], dtype=float32), array([-2.8210082], dtype=float32), 8.484117, 8.484117]\n",
      "9.0 [array([3.8840349], dtype=float32), array([-2.8807378], dtype=float32), 8.489415, 8.489415]\n",
      "10.0 [array([3.922677], dtype=float32), array([-2.920576], dtype=float32), 8.492949, 8.492949]\n",
      "\n",
      "Final output as a quadratic:\n",
      "[3.922677]x^2 + [-2.920576]x + 16.985897064208984\n",
      "Performance graph incoming\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xt80+XZ+PHPlTQFEYSKeECgxWfiIycRioAMBVREREUZAk4c83zaPEw2TzzqfOZwc9P5k4EoyHScVA7zQZingXUC1QZwIFVQJKUiUCBgEYG0uX5/JK1tmrTpKWmS6/168aK5801yfXu4cue67+99i6pijDEmdTjiHYAxxpjYssRvjDEpxhK/McakGEv8xhiTYizxG2NMirHEb4wxKcYSvzHGpBhL/MYYk2Is8RtjTIpJi3cA4ZxwwgmalZUV7zCMMSZhuN3uParaLppjm2Tiz8rKIi8vL95hGGNMwhART7THWqnHGGNSjCV+Y4xJMZb4jTEmxTTJGn84Pp+PwsJCDh8+HO9QTBSaN29Ohw4dcLlc8Q7FGBMiYRJ/YWEhrVq1IisrCxGJdzimGqrK3r17KSwspHPnzvEOxxgTImFKPYcPH6Zt27aW9BOAiNC2bVv7dGZMExVV4heRWSKyW0Q2VmjrJSJrRGS9iOSJyDkRHvszEdkS/Pez+gRrST9x2M/KmNqZsiyfwX9cwZRl+Y3+WtH2+GcDw0Pa/gA8pqq9gP8J3q5ERI4HHgH6AecAj4hIRp2jNcaYJHT3/HVMz9nKtr2HmJ6ztdGTf1SJX1VzgH2hzcBxwa9bAzvCPPRi4B1V3aeqXuAdqr6BGGNMynJ7vCxZXzl9/vPTnY36mvWp8d8N/FFEtgNPAQ+EOeZUYHuF24XBtoSzf/9+/vrXvzb666xcuZJVq1Y16HM++uijPPXUUw36nMaYhjF5yYYqbSWl/kZ9zfok/tuAe1S1I3APMDPMMeEKvRruyUTk5uBYQV5RUVE9wvqB2+Nl6oovcHu89X6u2iZ+VcXvr/0PrzESvzGmaXJ7vGz6phgAj+syPM1G4nFdxtf7G3diRH0S/8+ARcGvXyNQww9VCHSscLsD4UtCqOoMVc1W1ex27aJaZ6habo+Xn764hj+9/Tk/fXFNvZP//fffz5dffkmvXr245557uOCCC+jduzc9evTgH//4BwDbtm3jzDPP5Pbbb6d3795s376dmTNn0qVLFwYPHsxNN93EnXfeCUBRURGjR4+mb9++9O3blw8//JBt27Yxffp0nn76aXr16sUHH3xQJY4DBw6QlZVV/qZy6NAhOnbsiM/n44UXXqBv376cddZZjB49mkOHDlV5/ODBg8vXQdqzZw9li+GVlpYyadIk+vbtS8+ePXn++ecB+OabbzjvvPPo1asX3bt3DxuTMaZu7l2wHgCPayQ4NdBVdirbXJc16uvWZx7/DuB8YCUwFNgS5pi3gCcqDOgOI3xJqMGt2bqXoyV+/Aq+Ej9rtu6lT2bdx5WnTJnCxo0bWb9+PSUlJRw6dIjjjjuOPXv20L9/fy6//HIAPv/8c1566SX++te/smPHDh5//HHWrl1Lq1atGDp0KGeddRYAd911F/fccw8//vGPKSgo4OKLLyY/P59bb72Vli1bct9994WNo3Xr1px11lm8//77DBkyhP/7v//j4osvxuVycdVVV3HTTTcB8PDDDzNz5kx+8YtfRHV+M2fOpHXr1nz88cccOXKEgQMHMmzYMBYtWsTFF1/MQw89RGlpadg3E2NM3RR4D+FxXQ3OYIMQqIk4whZGGkxUiV9E5gGDgRNEpJDATJ2bgL+ISBpwGLg5eGw2cKuq3qiq+0TkceDj4FP9VlVDB4kbRf/T2pKe5sBX4seV5qD/aW0b7LlVlQcffJCcnBwcDgdff/01u3btAiAzM5P+/fsD8NFHH3H++edz/PHHAzBmzBg2b94MwLvvvsumTZvKn/Pbb7+luLg4qtcfO3YsCxYsYMiQIcyfP5/bb78dgI0bN/Lwww+zf/9+Dh48yMUXXxz1Ob399tv85z//4fXXXwcCnyy2bNlC3759uf766/H5fIwaNYpevXpF/ZzGmPDcHi9PLs9HFXAEO1MVC+ONfIVVVIlfVcdHuKtPmGPzgBsr3J4FzKpTdPXQJzODOTf2Z83WvfQ/rW29evuh5syZQ1FREW63G5fLRVZWVvnFSscee2z5caqR37X9fj+rV6/mmGOOqfXrX3755TzwwAPs27cPt9vN0KFDAZg4cSJLlizhrLPOYvbs2axcubLKY9PS0srLRBUvsFJV/t//+39h3yxycnJ48803mTBhApMmTeK6666rdczGmAC3x8vYGaspKVUKXBMqJ/xgyjjzhDMbNYaEuXK3LvpkZnDHkB81SNJv1apVeY/8wIEDnHjiibhcLlasWIHHE34Z7HPOOYf3338fr9dLSUkJCxcuLL9v2LBhPPfcc+W3169fX+V1ImnZsiXnnHMOd911FyNHjsTpDHxOLC4u5pRTTsHn8zFnzpywj83KysLtdgOU9+4BLr74YqZNm4bP5wNg8+bNfPfdd3g8Hk488URuuukmbrjhBtauXVttbMaY6i1aW0hJaSDDqyM49hgyDWbTHZtoTEmd+BtS27ZtGThwIN27d2f9+vXk5eWRnZ3NnDlz+O///u+wjzn11FN58MEH6devHxdeeCFdu3aldevWADz77LPk5eXRs2dPunbtyvTp0wG47LLLWLx4ccTB3TJjx47l73//O2PHji1ve/zxx+nXrx8XXXRRxJjuu+8+pk2bxrnnnsuePXvK22+88Ua6du1K79696d69O7fccgslJSWsXLmSXr16cfbZZ7Nw4ULuuuuuWn/vjDE/KKsDeFzjw/b2e7Ud2egxSHXliHjJzs7W0B248vPzOfPMxv340xgOHjxIy5YtKSkp4corr+T666/nyiuvjHdYMZGoPzNjGtOUZflMz9mKp9nIQOIvS/4a+KeP1S0ni4hbVbOjOdZ6/I3s0UcfLZ8K2blzZ0aNGhXvkIwxcTI3tyCQ9F1Xhu3td2jZIyZxJMyyzImqPlfM/u53v+O1116r1DZmzBgeeuih+oZljImDBR8XBL5wBMbSQpP/9kn/iUkclvibsIceesiSvDFJ5NMd30as7bdveXrM4rBSjzHGxMCUZfmU+BUcwVl7Icn/60mbYxaLJX5jjImB56up7Z/cvF9MY7HEb4wxjczt8QZyfITa/sP9/x7TeCzxG2NMI3t48YbAQmxhevtt/D9p0CVlomGJP0p1XY9/xIgR7N+/v0FiaNmyZYM8DwRWEu3evXuDPZ8xJrwpy/LJ31n8Q7YNSf7v3jijQZeUiUZSJ/7V21fz+w9+z+rtq+v9XJESf2lpabWPW7ZsGW3atKn36xtjEo/b46123v6jAxbFPOlDEif+1dtXc8HLFzB5xWQuePmCeif/iuvx9+3blyFDhnDNNdfQo0fggotRo0bRp08funXrxowZM8ofl5WVxZ49e8rX6r/pppvo1q0bw4YN4/vvvwfgyy+/ZPjw4fTp04dBgwbx2WefAfDVV18xYMAA+vbty+TJk6uNb+zYsSxbtqz89sSJE1m4cCHbtm1j0KBB9O7dm969e4fd5GX27Nnl+wQAjBw5snyBt7fffpsBAwbQu3dvxowZw8GDB8u/H127dqVnz54Rl5A2JtU9uTy4d27F2n7ZhbkKjwyP01X8qtrk/vXp00dDbdq0qUpbdZ7IeUKdjzmVR1HnY059IueJWj0+1FdffaXdunVTVdUVK1ZoixYtdOvWreX37927V1VVDx06pN26ddM9e/aoqmpmZqYWFRXpV199pU6nU9etW6eqqmPGjNFXXnlFVVWHDh2qmzdvVlXVNWvW6JAhQ1RV9bLLLtO//e1vqqr63HPP6bHHHhsxvkWLFul1112nqqpHjhzRDh066KFDh/S7777T77//XlVVN2/erGXf24rn89JLL+kdd9xR/lyXXnqprlixQouKinTQoEF68OBBVVWdMmWKPvbYY7p3717t0qWL+v1+VVX1er1hY6rtz8yYZJK3bZ9m/map8jDKI8F/jwb//x80b9u+Bn09IE+jzLFJewHX4KzBpDvTOVp6lHRnOoOzBjfo859zzjl07ty5/Pazzz7L4sWLAdi+fTtbtmyhbdvKAzadO3cuX8++T58+bNu2jYMHD7Jq1SrGjBlTftyRI0cA+PDDD8tX9JwwYQK/+c1vIsZzySWX8Mtf/pIjR47wz3/+k/POO49jjjmGAwcOcOedd7J+/XqcTmf5fgDRWLNmDZs2bWLgwIEAHD16lAEDBnDcccfRvHlzbrzxRi699FJGjmz8RaWMSTSTl2ygyPVU1U1WCPwfjxJPmaRN/AM6DuC9695j5baVDM4azICOAxr0+Suuu79y5UreffddVq9eTYsWLRg8eHClte7LNGvWrPxrp9PJ999/j9/vp02bNuXLMocSCbdtcVXNmzdn8ODBvPXWWyxYsIDx4wNbKDz99NOcdNJJfPLJJ/j9fpo3b17lsRXX6Icf1ulXVS666CLmzZtX5TEfffQR7733HvPnz+e5557jX//6V1RxGpMqNn1TzKFmKwM3Qur7N/7o43APiZkaa/wiMktEdovIxgptC0RkffDfNhEJm7VE5B4R+VRENorIPBGpmnUa0YCOA3hg0AMNkvSrWyf/wIEDZGRk0KJFCz777DPWrFkT9fMed9xxdO7cuXxNHlXlk08+AWDgwIHMnz8fIOL6+hWNGzeOl156iQ8++KB8Q5UDBw5wyimn4HA4eOWVV8IORmdlZbF+/Xr8fj/bt2/no48+AqB///58+OGHfPHFF0Bgf9/Nmzdz8OBBDhw4wIgRI3jmmWcivmkZk6qmLMtnh+vesAO6+OHW8/8rHmGVi2ZwdzYwvGKDqo5V1V6q2gtYyA+brpcTkVOBXwLZqtqdwAeecfWOOE4qrsc/adKkSvcNHz6ckpISevbsyeTJk8u3XozWnDlzmDlzJmeddRbdunUr37z9L3/5C1OnTqVv374cOHCgxucZNmwYOTk5XHjhhaSnpwNw++2387e//Y3+/fuzefPmSp9UygwcOJDOnTvTo0cP7rvvPnr37g1Au3btmD17NuPHj6dnz57079+fzz77jOLiYkaOHEnPnj05//zzefrpp2t1vsYks7KZPD5HsKwakvx/P2BTXMs8EOV6/CKSBSwNJvCK7QIUAENVdUvIfacCa4CzgG+BJcCzqvp2Ta+XTOvxpzL7mZlU1HXycjZzC6XO7YGGirV9AX2kcfZAieV6/IOAXaFJH0BVvwaeIvDG8A1wIJqkb4wxieq6mbkc8vkpdVRI+mUUnjuvSqqMi/oO7o4Hqo78ASKSAVwBdAb2A6+JyLWqGnZRChG5GbgZoFOnTvUMK3lt2LCBCRMmVGpr1qwZubm5cYrIGFMmZ8ueiBuoO/0dY740QyR1TvwikgZcBfSJcMiFwFeqWhQ8fhFwLhA28avqDGAGBEo9EY6JepZLsurRo0dCDKZGU0I0JpnMzQ1sshJ2A3WF60+Pz1W64dSn1HMh8JmqFka4vwDoLyItgmMBFwD5dX2x5s2bs3fvXksoCUBV2bt3b9ipo8Ykq8eXforHdXnY3r7Dn8EtcZ7JU1GNPX4RmQcMBk4QkULgEVWdSWCGzryQY9sDL6rqCFXNFZHXgbVACbCOYI++Ljp06EBhYSFFRUV1fQoTQ82bN6dDhw7xDsOYmHB7vHzv80Oz4PUwIcn/oxu/bDK9fYhyVk+shZvVY4wxTZHb4+Un01axzXU5OCsk/mBq7X/qUFbf9F6jxxHLWT3GGJPS1mzdy2FHfuWkX0aJSdKvLUv8xhhTD9NWfMHOtOBFnWVJP9jbH9X50XiEVCNL/MYYU0d3z1/HF/pg5UxafrGWk8UTH4lHWDWyxG+MMXXg9nhZsn4HRxzrAg0hJR59pCQucUXDEr8xxtTBw4s3sNM1OfxCbA5XPEKKmiV+Y4yppbm5BeTvLI7Y2/9930/iEle0LPEbY0wtPfHmpoj76OJvxv0jmvbihJb4jTGmFu6ev46DR0sr76Nbxg8XZLwVl7hqwxK/McbUwpL1O/C4Robt7R9fegfXD+wc9nFNiSV+Y4yJ0nUzc9mX9lLlfXTL+J1cdcZErunX9FcXtsRvjDFRytmyh2LnwsCNkIu1TimZEvctFaNlid8YY6Jw3cxcCl23hR/QLW3BU1dc3aQWYquOJX5jjKmB2+MlZ8ueiDtrzRmZnxAlnjKW+I0xpgZ3/N2NxzU+wsVazoRK+mCJ3xhjqjVlWT47i4+AozjQUOVirQ1xias+akz8IjJLRHaLyMYKbQtEZH3w3zYRCbsXoIi0EZHXReQzEckXkQENGbwxxjS26TlbI07fxN+KVsc07eUZwolmz93ZwHPAy2UNqjq27GsR+RNwIMJj/wL8U1V/IiLpQIu6h2qMMbF13cxcdrjujTB9E073z28yG6jXRo2JX1VzRCQr3H3BvXSvBoaGue844DxgYvB5jgJH6x6qMcbEzpRl+eRs2YOv2eZAQ8j0zR7Nn+Wlnw5ImJk8FdW3xj8I2KWqW8LcdxpQBLwkIutE5EURObaer2eMMY1ubm4B03O2st11fYTpm/DST69NyKQP9U/84wnZcL2CNKA3ME1Vzwa+A+6P9EQicrOI5IlInm2oboyJp6krAn1Zv2N3oCF0QHfApoRN+lCPxC8iacBVwIIIhxQChaqaG7z9OoE3grBUdYaqZqtqdrt27eoaljHG1Mvc3AK+3n844oCuy9+xya++WZP69PgvBD5T1cJwd6rqTmC7iJwRbLoA2FSP1zPGmEb3h7c+C3xRlh1Dkv/skf+OdUgNLprpnPOA1cAZIlIoIjcE7xpHSJlHRNqLyLIKTb8A5ojIf4BewBMNE7YxxjS8ubkF7D/ki7z6pmNowl2sFU40s3rGR2ifGKZtBzCiwu31QHY94jPGmJh5+p3POeLIrzx9s8KA7t7H34tTZA0rmnn8xhiT1NweL8+//yVFB4+yM31SoDGkxz/m1A/jEltjsMRvjElpbo+Xn764hsM+Px7XZZUL4OVX6Lr4zSWJPaBbka3VY4xJaWu27uWwzx+44Qhm+pDe/vC27yb09M1QlviNMSmtbMmFSAO6Tn9H/vfKHrEPrBFZ4jfGpLR3Pt2Jx3VFxAHdX/d+M6l6+2CJ3xiTwqYsy2d6zlZwlAYaQnr8mb6ljO7dIS6xNSZL/MaYlDXrw68CA7phl1yGJ67skXS9fbBZPcaYFHX3/HUcLVVICz+gm3fjvqRM+mA9fmNMCnJ7vCxZvyPigG6a4/ikTfpgid8Yk4LunOOuusFKhQHdST1XxiewGLHEb4xJKXNzC/jm2yP4HCEbrEBSD+hWZInfGJMy3B4v//vmpoglHvFnJO2AbkU2uGuMSQluj5cx01exI21y+D10Fa7NWp4Uq2/WxHr8xpiU8OTyfPwKRxzrAg0he+hmOIby8g394hJbrFniN8akhLUF+/G4RkfcQ/edia/HI6y4sMRvjEl6d89fR4lfwXEk0BCS/M9t+V7S1/UrimYHrlkisltENlZoWyAi64P/tonI+moe7xSRdSKytKGCNsaY2vhHNXP2m/nPpuspx8UlrniJpsc/GxhesUFVx6pqL1XtBSwEFlXz+LuA/DpHaIwx9TBlWT7bXKMjztk/teRxbj3/v+IUXXzUmPhVNQfYF+4+ERHgakL23q1wfwfgUuDFesRojDG15vZ4eXDxhuAibOFLPJm+pbx667kpVeaB+k/nHATsUtUtEe5/Bvg10Kqer2OMMVFze7yMn7Gao6UascSD38Wt552Wckkf6j+4O57Ivf2RwG5VdUfzRCJys4jkiUheUVFRPcMyxqSyhWsLOVqqFLpuizhn/65uudw/Inm2U6yNOid+EUkDrgIWRDhkIHC5iGwD5gNDReTvkZ5PVWeoaraqZrdr166uYRljDF/sKgag1LE90BAyZ/801z08M+7s2AfWRNSnx38h8JmqFoa7U1UfUNUOqpoFjAP+parX1uP1jDGmRm6Pl4+3eSOXeErh8YvujkdoTUY00znnAauBM0SkUERuCN41jpAyj4i0F5FlDR+mMcZE58nl+WyPVOLxw3mt/pUSyzJUp8bBXVUdH6F9Ypi2HcCIMO0rgZW1js4YY2phyrJ8PtrmpbRZ+BLP8aV38OexveISW1NiV+4aY5KC2+Nles7Waks8kwbekZKzeEJZ4jfGJIWHF29gpyvyypvdZHnKzuIJZYnfGJPw3B4v+TuLI6682cI/mHN/dEJcYmuKLPEbYxLe8+9/WW2Jp71/Erek2LIM1bHEb4xJaHNzC3h5y80RSzwXZKxk/s0DrLZfge3AZYxJWHfPX8eS9Ts40ixyief6gZ0t6YewHr8xJiHNzS1gSTXLLVMK7Xz34T10NB7hNWmW+I0xCWnWh19R4JoQscST6VtKulPof1rbeITXpFmpxxiTcNweL1t3H0SbeQMNISWeZv6zuaZfJ0b37mBlnjAs8RtjEorb42XM9FV8VU2JZ9bIJSm/LEN1rNRjjEkYbo+XX85biyft+oglnn7N37GkXwPr8RtjEoLb4+Xq51dR6gd/s92BxjCzeKZe2ycu8SUS6/EbYxLCorWFlPqpdhbP5AF/tZp+FCzxG2MSwnubduFxXR1xueV+zd+xtXiiZInfGNPkTVmWz6bvp4PzUKAhzHLLVuKJXjQbscwSkd0isrFC2wIRWR/8t01E1od5XEcRWSEi+SLyqYjc1dDBG2OSn9vj5fmcrRQ7FwYaQpK+q7QLo8+YaCWeWohmcHc28BzwclmDqo4t+1pE/gQcCPO4EuBXqrpWRFoBbhF5R1U31S9kY0yqcHu8/GTaKrZVU9c/1fdnW4Ctlmrs8atqDrAv3H0iIsDVhGzBGHzcN6q6Nvh1MZAPnFqvaI0xKeXJ5fl87bq32qtzX7/tXOvt11J9a/yDgF2quqW6g0QkCzgbyK3n6xljUoTb4+WjbV58js2BhpASj/gzeOLKHpb066C+iX88YXr7FYlIS2AhcLeqflvNcTeLSJ6I5BUVFdUzLGNMops4K7faqZsXtFlkF2rVUZ0Tv4ikAVcBC6o5xkUg6c9R1UXVPZ+qzlDVbFXNbteuXV3DMsYkgVHP/ZuN/lERSzxZvqW886vBcYgsOdSnx38h8JmqFoa7M1j/nwnkq+qf6/E6xpgUMje3gHd2TQbnkUBDSImnVelorujVPi6xJYtopnPOA1YDZ4hIoYjcELxrHCFlHhFpLyLLgjcHAhOAoRWmfo5owNiNMUlmbm4BDy7ewCHHykBDSNKntBXHl/yc009qFYfokkeN0zlVdXyE9olh2nYAI4Jf/5vKH9CMMSYit8fLg4s3VFvXz/TNI81ha+zXl125a4xpEgIbpo+vduomwG+v6G4zeerJVuc0xsSd2+Pl/z5bCenFgYaQEo/T35F0pzDPNk1vENbjN8bEldvj5erpq9iZNinQUKWuDx180yzpNyDr8Rtj4sbt8TJ62qpAXb9iN7RSXX8pC+3q3AZlPX5jTNz86tX1kZdaDtb1bz3vNEv6DcwSvzEmLubmFrBh/8KISy07/CfSzCm2xn4jsFKPMSbmpizLZ3rOVvY1mxpoCFPX7+ibxdzbBsQjvKRnPX5jTEzNzS1ges7WGubrL2VY15OsxNNILPEbY2Km7Mpcj2tktXX9dKfYGvuNyEo9xpiYcHu8PLxkA9td11dN+sHefgv/YE7NOIZnx51tvf1GZD1+Y0xMTFmej1/B79gdaKhS13fQznefJf0YsMRvjGl0c3ML+Hibt4a6/hu2sUqMWKnHGNNo3B4vC9cW8urHBVXr+iGDubeed5ptrBIjlviNMY3C7fEyfsZqjpYqHteV1Q7mDut6ks3XjyEr9RhjGsWarXs5WqoUuZ4Cpy/QGFrX97cizWbwxFxSJX63x8vUFV/g9njjHYoxKW/LrsBKm5E3VYHOvnkssMXXYq7GUo+IzAJGArtVtXuwbQFwRvCQNsB+Ve0V5rHDgb8Q+JD3oqpOaajAQ7k9Xq5+fhWlfnA64NVbbFEnY+Jlbm4BS9bvqPEirZttHZ64iKbHPxsYXrFBVceqaq9gsl8IVNlIXUScwFTgEqArMF5EutY74gieXJ5PqT/wdakfHl68obFeyhhTjUo7aVUzmNv1lFZW14+TGhO/quYA+8LdF9xQ/WpC9t4NOgf4QlW3qupRYD5wRT1irZa7oHJ5J39nMWc99hZzcwsa6yWNMWE8uTy/xitzReDxUT3iEZ6h/jX+QcAuVd0S5r5Tge0VbhcG2xpFqR+KncvZlT6ZYudyAA58X8KDizdY8jcmRqYsy+e1rwdHvDK3VeloAH43yubrx1N9E/94wvf2IfxG6xqmLXCwyM0ikicieUVFRbUOpNi5nH2uqRx2rGOfayoFrgnl901dEe59yRjTUNweL+f/YQX/s3pUxBk8ztKOHF/ycy7qepLN14+zOid+EUkDrgIWRDikEOhY4XYHYEek51PVGaqararZ7dq1q3U8+9IqL++qTm9g7jDw9f7D3D1/Xa2f0xhTM7fHy0+mrWLzfjc+5+ZAY4TtE9Odwq02dTPu6tPjvxD4TFULI9z/MXC6iHQWkXRgHPBGPV4vemW/dE4fO1z3ArBk/Q6mLMuPycsbk0qef/9LFKrdMzfTt5RJF59h++Y2ETUmfhGZB6wGzhCRQhG5IXjXOELKPCLSXkSWAahqCXAn8BaQD7yqqp82ZPCVlJ1J2S9b8JfP59hcfsj0nK02x9+YBjQ3t4B383fVuGfumSe34o4hP7Kk30SIasSye9xkZ2drXl5erR8nj0nlxB/yywdwUdeTeOG67AaJ05hUVu3a+hr4l3kk8Hdnm6U3PhFxq2pUyS2prtzNm7iv8vBxecmHwC8n8M6mXczNLbArfI2pp7+u2BI56RNYW1+wpN8UJVXi75OZwcD2owI3Qko+FZP/g4s38Me3Pmfs86ss+RtTS3NzC8j+3Tt8+N2FEZM+pc1o57uP39kyy01SUiV+gH/fspi2x5wUuFFN8gco8cP097+MaXzGJLKy8o778PBqkr6DTN9CW2Ze/BcfAAAXMElEQVS5CUu6xA+w5zc7q/4yRkj+Kz7bFcvQjElYc3MLePSNjXhcV4MzuD5K2Bk8b3DreafZcgxNWFImfgjW+0uDN6pJ/iV+uG5mbqzDMyahTFmWz4OLN7DFMQ6chwKNEaZttmuZbkm/iUvaxN8nM4O8G2tK/oELvHK27LFlHYyJwO3xMj1na3Bd/cBSy5GSPsA9F51R5TlM05K0iR8Cyf/3AzZVk/x9gY+tYGv6GBNB2ThYdevqZ/qW0jLdyRNX9rC6fgJI6sQPcP+IM2tI/ofKr+59aPEGbn45z2b6GBPk9nh5Z9OuGtfVP7lVMzb+drgl/QSR9IkfAsl/QtaaiMnf59zMEUc+Cry9aRdjpts0T2PcHi+3vpJX47r6AFOv7ROPEE0dpUTiB3j5hn5ccdIHEZN/+TojgF8D648Yk6rm5hYwetoq8o4Mq3ZdfYAnbK5+wkmZxA+w5M4fhy/7ADgqT/PM3bY3prEZ01RE3EELyv9mji+5A5cjcFWulXcST0olfohQ9gkzzfPAoRJbytmkFLfHy0OLNzD2+VXVJn0pzeB4RrDliUutp5+gUi7xQ6DsM+bUD2tc12fJ+h1c9KeVsQ7PmJhze7yMf2ENc3IL+NIZOelT2opOvleYf/OAOERpGkpKJn6A31xyJscFt4Gr7gKvLUXfWc/fJL01W/dytMRfbU+f0lZk+ubZomtJIGUTf5/MDKZd9mealZ4daKgm+S9Zv4MzHl7OTTbV0yQZt8fLJc/k8Me3Pq8h6TvJ9M3jp/06WdJPAtFsxDJLRHaLyMaQ9l+IyOci8qmI/CHCY+8J3r9RROaJSPOGCrwhXNOvEx/e+B4tSnsHGoJriIdL/kdK/LyzaRdX21RPkyTKtkzM31lcQ9KHTN8/cDqEq3p3iEOkpqFF0+OfDQyv2CAiQ4ArgJ6q2g14KvRBInIq8EsgW1W7E/i1GlffgBtan8wMXhi5mFYlo2us+QOUKjy53LZwNIlv4dpCFKJI+ks5oWU6r95i2yYmixoTv6rmAPtCmm8DpqjqkeAxuyM8PA04Jrgxewuq2Ww9nq7p14kHz32ck31/DDTUsKLnR9u8DHzyX7bEg0lIZdM1X8/bHkV5ZykCPD8h25J+Eqlrjb8LMEhEckXkfRHpG3qAqn5N4JNAAfANcEBV3657qI3r/hFnsvSWG+hz3K8DDeGSf7ORFDuXA/C193tb38ckHLfHy/gZq5mbW8AWx6U1lncA20wlCdU18acBGUB/YBLwqohUvK4PEckgUA7qDLQHjhWRayM9oYjcLCJ5IpJXVFRUx7Dqp09mBnn3Pkm/jPsDDaE1f4F9rqnla/sA/H55vtX8TcJ4cnk+R0s1qvIOYIuuJam6Jv5CYJEGfAT4gRNCjrkQ+EpVi1TVBywCzo30hKo6Q1WzVTW7Xbt2dQyrYay56/dkt3gucFbwQ/KvsLZP2aqexYdLGDtjtSV/0+SNeu7fvLd9Lp706pJ+MzJ9S3GKXZWbzOqa+JcAQwFEpAuQDuwJOaYA6C8iLYKfBi4AEmZU9ONf30G/Zu9Uu6pn+WYupWoDvqZJu3v+OpbuuoZ9rqk//NWXLbhWaZ7+QgAeH2XlnWQWzXTOecBq4AwRKRSRG4BZwGnBKZ7zgZ+pqopIexFZBqCqucDrwFpgQ/C1ZjTSeTSKqdf2CXzkjWInr4+2ee1CL9PkzM0toMtDy/jLp9mUOrcHGss+vZb9Piu0KhlNpm8exx+bbuWdFCCqWvNRMZadna15eXnxDgP4YXPpaGuibY5x8evh/21/OCburpuZS86WPVH/7joEXrvVrspNVCLiVtXsaI5N2St3o3VNv06BS9Sbvx1Vz3//9z4eXLyBKcus9GPiw+3xMuCJd2uV9E9t09ySfgqxxB+FPpkZPH9tdtRlH4DpOVttqqeJObfHy5jpq/jm2yNRJ/30NAfPju9tST+FWOKPUp/MDJ64skfNyb/ZSPalvQTYPr4mttweL3fNW4tfo7saF+CcrAzm3dTfkn6KsRp/Lbk9Xn6z8D+85x0c+Q8LoLQFmb5XgUC5aHTvDvbHZRpN2VjUdtf1+B27K117Atgc/RRgNf5G1CczgydH96RzSYSef5jpnnNzC7j6+dXW+zeN4ocdsy7H74wu6Y/q1d6SfgqzxF8HfTIzeO3Wc+md/lbl5F9N3b/Urzy4eINN+TQNyu3xMuHFNcHSTvCKw4odkDBJ/7zTT+CZcWfHOlTThFjir6M+mRm4Jw/j9wM20bp0dKU50UDEQd8l63cw4i85dqWvqTe3x8voaavIZ0T4smOFrRIrlndevqFfrEM1TYwl/nq6f8SZvHfjC2QdrXnQd7vregA2fVPMT6atstKPqbO5uQVc+OJNeJpVM4jrh+N9d9DJ9wpgNX3zg7R4B5AM+mRm8Pqt5/KrV//F+98ODfwhVlzfJ/i137kbDyPJ9C1FCcz6AeyP0UTF7fGycG0hS9YWks9l4Cz94c5qSjsnH9eMqT/tY5MLTDmb1dPApizL54HVXav2wiDsHyXAreedxv0jzoxViCYBzc0t4KHFGyJvnAJhf7/atUzn44cvimGkJl5sVk8c3T/iTIa3fR9KWwQaIg36NhtJgWsCELjYy670NZG4Pd7qk36lhdYqdyruueiM2AVqEoYl/kbwv1f24LTSVzn56B8rL+0MlabaqdNbPvBryd+EMze3gJte/phtrvHV1/MVHKUnVvkkaWVEE46VehqJ2+Nl0dpC/rHuaz7VS6L+aN6rQ2u6ndoaBbvoK8WFXSAQqq3nCzDeLhhMSbUp9Vjib2RlU+5q/ONVcPo70sE3rfyQdKcw72bb4DoVzc0tYOLSH+NzbK96QRaETfpZbVvwp6t72e9LirIafxPSJzODhbedy1lp/8RZ2jHyfH+BUuf2SnP+j5YqC9cWxjJcE2duj5fz/7CCny7NxOcMk/Qr1fNdlebnr5w0xJK+iYol/hjok5nBzInnkFkyjcwjNS31EBj43emaDMC83AKb758i5uYWMGzGZHK+G1q5NBhmfv7JR/9Ipm8xDtsi0dRBjaUeEZkFjAR2q2r3Cu2/AO4ESoA3VfXXYR7bBngR6E7g1/Z6VV1dU1DJVOqpKOKUPIiw0Fvl2v9F3U6m/2ltrVeXJNweL2u27iWjRTrPvreZ3O8vqtXvhACv32Zr6JuA2pR6ormAazbwHPByhRcYAlwB9FTVIyJyYoTH/gX4p6r+RETSgRbRBJWsrunXiTNObsX097/k/c/fZK9/GfvSpla6yKvSH7sTPI6ROP0doXAa6wsP4HTA41f0wHvoqL0JJKiyC7FedxdytMRPoes2Sh3bq58AoODyd6G9788AHH9sOi9cl20/f1MnUQ3uikgWsLSsxy8irwIzVPXdah5zHPAJcJrWcgQ5WXv8oe6ev44l63fUuvdfdkgzl4M5N9pa6onE7fHy0xfXcNgXmOdbl5892PILpqpYDO52AQaJSK6IvC8ifcMccxpQBLwkIutE5EURObaOr5eUnhl3Nk9c2YPT/W/iKu0SfuA3pPZfNvirwGGfn2fe3WwLviWQRWsLOezz43FdXnVefpiZXpQ6KyX9rLYtLOmbeqtr4k8DMoD+wCTgVRGRMMf0Bqap6tnAd8D9kZ5QRG4WkTwRySsqKqpjWInnmn6dGJPdkfa+P1cd+A0z86fsDaDI9RQAH2zZw7gZqy35N3Fuj5ebX85j1sdv4UkPLqFc7YwdyDyylEzfP8qfo7nLwZ+u7mVJ39RbXUs9/wSmqOrK4O0vgf6qWlThMScDa1Q1K3h7EHC/ql5a0+ulSqmnjNvjZfwLazhaEvj4H3EXJah2oC/jWBdX9+lo6/40IXNzC3j6nc8pOng0clkHKvfy/ZVLOzawb6LR0IO74SwBhgIrRaQLkA7sqXiAqu4Uke0icoaqfg5cAGyq4+sltT6ZgX1Py2Z4LFm3mI+2eX9IFOFKPxUGf8sSxb7vfEzP2crObw/bRhtxVDZbZ8uu4h/GcJpRqzdygDbHpDFz4jmW7E2Di2Y65zxgMHACsAt4BHgFmAX0Ao4C96nqv0SkPfCiqo4IPrYXgemc6cBW4OeqWmNNItV6/OGUXfFb5HqKQ46VlZNFuAHAkJ6iU6Bzu5ZcP7CzlQZioOLUzIeXbMCvYT65QcSfXQv/YNr57itvsqu2TW3Zkg1JomytFgjO/nBQ6zcACOyvap8AGo/b42XsjNWUlAZ+EB7XFeAoDf+zgmrLOgB9szK4/5IzLembWrHEn0TcHi8PL95A/s5iIMwbQKSyQUhSae5y0CGjhX0CaEBlC/G9tWkne4qP4nFdCQ5fzQkfwk7RPK55Gtec08nGaEydWOJPQhWv+oVq5n9DjZ8ATm7VjJNaN2ds3072JlAHVd+MLweHP7qEr9DMfzYn+x4vb3IKPD7Kpmia+rHEn6TcHi/T3/+SdzbtAqDANQF1eKNOOOHKCj9qdyzX/9jWba+J2+Pl+fe/5NNvvuVr7/dADeU3qPL9F39G+f63ZS7qehK3nv9fVtYx9WaJP8mVJaG3g28AHtd4cBRH/wYAOPwn0tE3q7z5zJNb8b9X9rAEFEbFsZYjjnx2pk2qPMMKav2Ga3V809As8acIt8fLvQvW49l3CKB2Neay2yFJqWW6k7Q0B4O7tEvZAeGyGTrF3/v4+xoPB4+Whh+whVon/EGnn8DdF3axhG8anCX+FDNlWT5zcj0UHwlc9ltjCQKi+hTQ5pg02rRIZ3i3k5N+wDHcdMydrskccawLHFCb72WYhA+BAXZbW8k0Fkv8KWpubgFTV35RoQYdpgQENX8KoOoAZLM0Bz8/Nyuh3wDKknvZFbBlt9/5dCfrCw+UHxf2jRNq/L6Fq+EDOB3C2L4dbTtE06gs8ac4t8fLk8vz+Whb4Fq5fWkvUexcGLgz2p5rxdt+F5m+xeXNDoH2rZuT5nQkzKeBslUxj/j8iEDPU1uXJ/tK3x+o9Sel0O9PmTNPbkXvzAyusoRvYsASvwEqD0qWiao3C9W8CVQtY5zcqhkXdD2pUoIL7V2XidReW9E+T+g4SJny7wNUPffQtijGRio+zOkQfntFd5spZWLKEr8pF7puTJnyzT+g7m8CUCUBOgUQodQfOCjNAb07ZXCkxE/nE47ljU924FdwOYX5wSUJokniZZuX7Ck+ggLv5e/Cr+B0wKu3nAvAw4s3sHXPd6Q5hRYuJ75SP/u/LwFCEn19ztnfgkzfq1XiG9b1JAafcaJtkGPixhK/CWtubgELPi5g+75D7DvkK2+vtvdbU0IM1xahNxzK6QCHCL7SH56gVfM00p1C8eES0p0OTm5zDC3TnZVq8DXxuC4DR0hQNfXqoVafcsp0aNOc24ecbr17E3eW+E2NrpuZS86WPVXaa/UmAOHfCGpoP770DlqVXhJdoBFU6cFXFC7OSO21TPZpDuGk1s3pdspx3GIXXpkmxBK/icrc3AJm/XsrXxZ9FzZP16oOXlE0v1L1/bWL9No13V/dJ5YaPqk4BV691TY3N01TLNbjN0ngmn6BtXoqzmH3HjrKkrWFbCn6rlISLJ8aWqZsg/hQoVe0hqNRHBONaF6npna/g0zfGzW+VNl6Opb0TTKwHr+JaMqyfKbnbA17X6WLm8pEk8wbIuGXqcsniwhTLytyCIw7pxPd27fm0x0HULA5+KbJa9Aev4jMAkYCu8u2Xgy2/wK4EygB3lTVX0d4vBPIA75W1ZHRBGWahvtHnEmntsfyP//YSIm/cgateHFXmR2ue/E5Nkd+wkifEuqjuufzNyPTtzDi3aHhjOrVntNPamWzckzSi6bUMxt4Dni5rEFEhgBXAD1V9YiInFjN4+8C8oHj6hGniZNr+nXijJNbsWhtIa/lbedoaeRM297352qfq9oB2bqIcvZQOGWb08zNLWD5xm+4pPspNjPHpIy6brb+KjBDVd+t4XEdgL8BvwPujbbHb6Wepil0LKD/aW35fGcxCz4uYOOOA5T64x1heC1cDq4bkMVF3U5ukIvHjGmKYjG42wUYJCK/Aw4T2HP34zDHPQP8GmhVx9cxTUifzIwqCbNPZkb5APGitYXsLj7C+5uLOFoSeBc4voWL9DQHrZqlsaXouwaPqXmag8MlP7zjjOrVnmUbvuFoqYbd4MQSvjF1T/xpQAbQH+gLvCoip2mFjw8iUjYu4BaRwTU9oYjcDNwM0KmTfeRONBXfFKpbrmHh2kIE6Na+Nd5DRyn+3sfqrXvZUHiAcB8Y2hzj4khJKa2buzipdXM6n3Ase787Wqk0E1qumTAgy3r2xlSjrqWefwJTVHVl8PaXQH9VLarwmN8DEwgM/jYnUONfpKrX1vR6VupJPRXXwH/x31/hVyU9zZYxNiZasSj1LAGGAitFpAuQDlS6DFRVHwAeCAY0mEA5qMakb1JTxU8MVos3pnFFM51zHjAYOEFECoFHgFnALBHZCBwFfqaqKiLtgRdVdUQjxmySXLixBGNMw6kx8avq+Ah3Vem9q+oOoErSD5aEVtYyNmOMMY2gIWdVG2OMSQCW+I0xJsVY4jfGmBRjid8YY1KMJX5jjEkxTXJZZhEpAjx1fPgJhFxTkALsnJNfqp0v2DnXVqaqtovmwCaZ+OtDRPKivXotWdg5J79UO1+wc25MVuoxxpgUY4nfGGNSTDIm/hnxDiAO7JyTX6qdL9g5N5qkq/EbY4ypXjL2+I0xxlQjIRO/iAwXkc9F5AsRuT/M/c1EZEHw/tzgfgIJLYpzvldENonIf0TkPRHJjEecDammc65w3E9EREUk4WeARHPOInJ18Gf9qYjMjXWMDS2K3+1OIrJCRNYFf78TevVfEZklIruDqxuHu19E5Nng9+M/ItK7wYNQ1YT6BziBL4HTCOwD8AnQNeSY24Hpwa/HAQviHXcMznkI0CL49W2pcM7B41oBOcAaIDveccfg53w6sA7ICN4+Md5xx+CcZwC3Bb/uCmyLd9z1POfzgN7Axgj3jwCWA0Jgl8Pcho4hEXv85wBfqOpWVT0KzAeuCDnmCgKbvAO8DlwgIhLDGBtajeesqitU9VDw5hqgQ4xjbGjR/JwBHgf+QGDv50QXzTnfBExVVS+Aqu6OcYwNLZpzVgI7+AG0BnbEML4Gp6o5wL5qDrkCeFkD1gBtROSUhowhERP/qcD2CrcLg21hj1HVEuAA0DYm0TWOaM65ohsI9BgSWY3nLCJnAx1VdWksA2tE0fycuwBdRORDEVkjIsNjFl3jiOacHwWuDW4EtQz4RWxCi5va/r3XWl23XoyncD330KlJ0RyTSKI+HxG5FsgGzm/UiBpftecsIg7gaWBirAKKgWh+zmkEyj2DCXyq+0BEuqvq/kaOrbFEc87jgdmq+icRGQC8Ejxnf+OHFxeNnr8SscdfCHSscLsDVT/6lR8jImkEPh5W99GqqYvmnBGRC4GHgMtV9UiMYmssNZ1zK6A7gX2ftxGohb6R4AO80f5u/0NVfar6FfA5gTeCRBXNOd8AvAqgqquB5gTWtElWUf2910ciJv6PgdNFpLOIpBMYvH0j5Jg3gJ8Fv/4J8C8NjpokqBrPOVj2eJ5A0k/0ui/UcM6qekBVT1DVLFXNIjCucbmq5sUn3AYRze/2EgID+YjICQRKP1tjGmXDiuacC4ALAETkTAKJvyimUcbWG8B1wdk9/YEDqvpNQ75AwpV6VLVERO4E3iIwI2CWqn4qIr8F8lT1DWAmgY+DXxDo6Y+LX8T1F+U5/xFoCbwWHMcuUNXL4xZ0PUV5zkklynN+CxgmIpuAUmCSqu6NX9T1E+U5/wp4QUTuIVDymJjIHTkRmUegVHdCcNziEcAFoKrTCYxjjAC+AA4BP2/wGBL4+2eMMaYOErHUY4wxph4s8RtjTIqxxG+MMSnGEr8xxqQYS/zGGJNiLPEbY0yKscRvjDEpxhK/McakmP8Pn7SxyRXFtG0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "main_loop(X, target,\n",
    "          steps=2500,\n",
    "          periods=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:insight-prep]",
   "language": "python",
   "name": "conda-env-insight-prep-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}