simple_g2p_policy_gradient_2lstm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Week 8: sequence learning\n",
    "\n",
    "\n",
    "This time we'll solve a problem of transribing english words, also known as g2p (grapheme2phoneme)\n",
    "\n",
    " * word (sequence of letters in source language) -> translation (sequence of letters in target language)\n",
    "\n",
    "\n",
    " \n",
    "Some letters correspond to several phonemes and others - to none, so we use encoder-decoder architecture to figure that out.\n",
    "\n",
    "This kind of architectures is about converting anything to anything, including\n",
    " * Machine translation and spoken dialogue systems\n",
    " * [Image captioning](http://mscoco.org/dataset/#captions-challenge2015) and [image2latex](https://openai.com/requests-for-research/#im2latex) (convolutional encoder, recurrent decoder)\n",
    " * Generating [images by captions](https://arxiv.org/abs/1511.02793) (recurrent encoder, convolutional decoder)\n",
    " * Grapheme2phoneme - convert words to transcripts\n",
    " \n",
    " \n",
    "We chose simplified character-level __Hebrew->English__ machine translation, for short phrases, as it is faster to train even without gpu cluster.\n",
    "\n",
    "Since you have already done step1-2 in RNN assignment, we trust you to __read carefully__ through already implemented functions instead of reimplementing them for the third time.\n",
    "\n",
    "__Contributions:__ This notebook is brought to you by\n",
    "* Yandex [MT team](https://tech.yandex.com/translate/)\n",
    "* Oleg Vasilev ([Omrigan](https://github.com/Omrigan/)), Dmitry Emelyanenko ([TixFeniks](https://github.com/tixfeniks)) and Fedor Ratnikov ([justheuristic](https://github.com/justheuristic/))\n",
    "* Dataset is parsed from [Wiktionary](https://en.wiktionary.org), which is under CC-BY-SA and GFDL licenses.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "EASY_MODE = False        #If True, only translates phrases shorter than 20 characters (way easier).\n",
    "                        #Useful for initial coding.\n",
    "                        #If false, works with all phrases (please switch to this mode for homework assignment)\n",
    "\n",
    "MODE = \"he-to-en\"                                #way we translate. Either \"he-to-en\" or \"en-to-he\"\n",
    "END = ';'                                        #end of phrase for both source and target\n",
    "MAX_OUTPUT_LENGTH = 50 if not EASY_MODE else 20  #maximal length of _generated_ output, does not affect training\n",
    "REPORT_FREQ       = 500                          #how often to evaluate validation score\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: preprocessing\n",
    "\n",
    "We shall store dataset as a dictionary\n",
    "`{ word1:[translation1,translation2,...], word2:[...],...}`.\n",
    "\n",
    "This is mostly due to the fact that many words have several correct translations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('size = ', 182390)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from collections import defaultdict\n",
    "word_to_translation = defaultdict(list) #our dictionary\n",
    "\n",
    "with open(\"main_dataset.txt\") as fin:\n",
    "    for line in fin:\n",
    "        \n",
    "        ###\n",
    "        #you may want to cast everything to unicode later during homework phase, just make sure you do it _everywhere_\n",
    "        ###\n",
    "\n",
    "        en,he = line[:-1].lower().replace(END,' ').split('\\t')\n",
    "        \n",
    "        word,trans = (he,en) if MODE=='he-to-en' else (en,he)\n",
    "                \n",
    "        if EASY_MODE:\n",
    "            if max(len(word),len(trans))>20:\n",
    "                continue\n",
    "        \n",
    "        word_to_translation[word+END].append(trans+END)\n",
    "    \n",
    "print (\"size = \",len(word_to_translation))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#get all unique letters in source language (a.k.a. source dictionary)\n",
    "all_words = word_to_translation.keys()\n",
    "\n",
    "source_letters = list(set(''.join(all_words)))\n",
    "source_to_ix = {l:i for i,l in enumerate(source_letters)}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#get all unique translation letters (a.k.a. target dictionary)\n",
    "\n",
    "all_translations = [ts for all_ts in word_to_translation.values() for ts in all_ts]\n",
    "\n",
    "target_letters = list(set([l for ts in all_translations for l in ts]+[\" \"]))\n",
    "target_to_ix = {l:i for i,l in enumerate(target_letters)}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Special tokens\n",
    "PAD_ix=-1\n",
    "EOS_ix_source=source_letters.index(END)\n",
    "EOS_ix_target=target_letters.index(END)\n",
    "BOS_ix_target = target_letters.index(\" \")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Draw word/translation length distributions to estimate the scope of the task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAEICAYAAABCsb3rAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHyBJREFUeJzt3X20XXV95/H3R6JIFSgPaSYGbHBIdYBZYkmRVqvVqOBD\nG2YtpGmnknZlYGakHV11xhWmnVVdM5lCZyottrKGiiXgA6RYF1SKLgyitUuCwaIISIk8lMRAYkDi\nQ6EmfueP/btwcrk39+bpPuzzfq111tn7u/dvn9/Oye989/7tffcvVYUkSeqP50x3BSRJ0v5lcpck\nqWdM7pIk9YzJXZKknjG5S5LUMyZ3SZJ6xuSuKZXkfUk+Ot31kLSrJFck+V/7UP77SV6yP+ukvWdy\nl6QZJMmDSd4w3fXYnSS3JPkPg7GqemFV3T9dddKuTO46INLx/5e0HyWZM9110Ozgj68ASPJbSf5m\nYP6+JH81MP9wkpOT/EKSryR5or3/wsA6tyRZleTvgR8CL0lyXJIvJPlekpuAowfWf36SjybZluS7\nbXvzpmiXpRknyVXAi4G/ad3c701SSVYk+Sfg5rbeXyV5pLXDLyY5cWAbVyT58yQ3tHa3Lsm/bsuS\n5OIkW5JsT3JnkpPGqMcRST6dZGuSx9v0MW3ZKuAXgT9rdfyzFq8kx7fpw5Nc2co/lOT3Rw72k/xm\nki8l+b9t2w8kefPAZ/9mkvtb3R9I8u8P0D93r5ncNeILwC8meU6SFwHPA34eoF1HeyHwT8ANwCXA\nUcAHgBuSHDWwnXcA5wGHAg8BHwdup0vq/xNYPrDucuBw4Ni2vf8E/PMB2j9pxquqd9C1s1+uqhcC\na9qi1wL/Bji9zd8ILAJ+Cvgq8LFRm1oGvB84AtgArGrxNwGvAX6Gru2dDWwboyrPAf4S+Gm6g41/\nBv6s1fH3gL8Dfrt1xf/2GOU/2Lb/klb3c4DfGlj+SuBeut+FPwIubwceL6D7fXlzVR0K/AJwxxjb\n1wRM7gKgXSv7HnAyXeP/LPDtJC+ja5x/B7wVuK+qrqqqHVX1CeCbwC8PbOqKqrqrqnYA84GfA/5H\nVT1VVV8E/mZg3R/RJfXjq2pnVd1eVdsP8K5Ks9H7quoHVfXPAFX1kar6XlU9BbwPeHmSwwfW/1RV\n3dba4cfo2jV0be5Q4GVAquqeqto8+sOqaltVfbKqflhV36M7OHjtZCqa5CC6g4sLWh0fBP6Y7sB/\nxENV9RdVtRNYTfdbMdJr92PgpCSHVNXmqrprMp+rXZncNegLwC/RJfcvALfQNejXtvkX0Z2ND3oI\nWDAw//DA9IuAx6vqB6PWH3EV3UHE1Um+neSPkjx333dD6p2n21WSg5JcmORbSbYDD7ZFRw+s/8jA\n9A/pet6oqpvpzsD/HNiS5LIkh43+sCQ/keT/tS717cAXgZ9siXsiRwPPZde2Pvp34un6VdUP2+QL\n22/Fr9L14m1ulxZeNonP1Cgmdw0aSe6/2Ka/wK7J/dt03XSDXgxsGpgfHGZwM3BE62obXL9bsepH\nVfX+qjqBrvvtbXTdd9IwG2uozsHYrwNLgTfQdX0vbPFMauNVl1TVKcAJdN3z/22M1d4DvBR4ZVUd\nRnfAP/gZuxtO9Dt0PQSDvxWjfyd2V7/PVtUb6c7mvwn8xWTKaVcmdw36AvA64JCq2kjXFX8GXdf5\nPwB/C/xMkl9PMifJr9L9QHx6rI1V1UPAeuD9SZ6X5NUMdOEneV2Sf9vOBrbT/SD8+MDtnjQrPEp3\nrXo8hwJP0V0r/wngf092w0l+LskrWw/ZD4AnGbvNHUp3nf27SY4E/mCydWxd7WuAVUkOTfLTwO8C\nEz7fIsm8JEvbCcFTwPfHqZ8mYHLX06rqH+ka09+1+e3A/cDft2vi2+jOrt9D98PyXuBtVfWd3Wz2\n1+lunnmM7gfiyoFl/wq4li6x30N3cHHV/twnaRb6Q+D3k3wXOGuM5VfSdXNvAu4Gbt2DbR9Gdyb8\neNvGNuD/jLHenwCH0J2F3wp8ZtTyPwXOane7XzJG+d+hO3i4H/gS3Y21H5lE/Z5DdyDwbbrfjNcC\n/3kS5TRKqnbXuyJJkmYbz9wlSeoZk7skST1jcpckqWdM7pIk9cysHYTg6KOProULF053NaQZ7/bb\nb/9OVc2d7nrsju1ZmtietOVZm9wXLlzI+vXrp7sa0oyXZPRTBWcc27M0sT1py3bLS5LUMyZ3SZJ6\nZlLJPcmDbdzfO5Ksb7Ejk9yUbtzvm5IcMbD+BUk2JLk3yekD8VPadjYkuSRJWvzgJNe0+LokC/fv\nbkqSNDz25Mz9dVV1clUtbvMrgbVVtQhY2+ZJcgLdcH8n0j2X/EMDIwldCpxLNw7xorYcYAXd6GHH\nAxcDF+39LkmSNNz2pVt+Kd04vLT3MwfiV7fxux8ANgCnJpkPHFZVt1b3zNsrR5UZ2da1wJKRs3pJ\nkrRnJpvcC/hcktuTnNdi86pqc5t+BJjXphew65jeG1tsQZseHd+lTFXtAJ6gG4lsF0nOS7I+yfqt\nW7dOsuqSJA2Xyf4p3KuralOSnwJuSvLNwYVVVUkO+Ag0VXUZcBnA4sWLHfFGkqQxTOrMvao2tfct\nwKeAU4FHW1c77X1LW30TcOxA8WNabFObHh3fpUySOcDhdEMRSpKkPTRhck/ygiSHjkwDbwK+AVwP\nLG+rLQeua9PXA8vaHfDH0d04d1vrwt+e5LR2Pf2cUWVGtnUWcHM5Fq0kSXtlMt3y84BPtfvb5gAf\nr6rPJPkKsCbJCuAh4GyAqroryRrgbmAHcH5V7WzbeidwBXAIcGN7AVwOXJVkA/AY3d3202rhyhvG\nXfbghW+dwppImsju2ivYZjV8JkzuVXU/8PIx4tuAJeOUWQWsGiO+HjhpjPiTwNsnUV9JkjQBn1An\nSVLPmNwlSeoZk7skST1jcpckqWdM7pIk9YzJXZKknjG5S5LUMyZ3SZJ6xuQuDZEkL01yx8Bre5J3\nJzkyyU1J7mvvRwyUuSDJhiT3Jjl9IH5KkjvbsktGhmluj56+psXXJVk49XsqDTeTuzREqureqjq5\nqk4GTgF+SDcY1EpgbVUtAta2eZKcQPc46BOBM4APJTmobe5S4Fy68SMWteUAK4DHq+p44GLgoqnY\nN0nPMLlLw2sJ8K2qeghYCqxu8dXAmW16KXB1VT1VVQ8AG4BT20iQh1XVrW2QpytHlRnZ1rXAkpGz\neklTw+QuDa9lwCfa9Lw2ciPAI3QDRgEsAB4eKLOxxRa06dHxXcpU1Q7gCeCo/V15SeMzuUtDKMnz\ngF8B/mr0snYmfsCHXE5yXpL1SdZv3br1QH+cNFRM7tJwejPw1ap6tM0/2rraae9bWnwTcOxAuWNa\nbFObHh3fpUySOcDhwLbRFaiqy6pqcVUtnjt37n7ZKUkdk7s0nH6NZ7rkAa4Hlrfp5cB1A/Fl7Q74\n4+hunLutdeFvT3Jau55+zqgyI9s6C7i59QZImiITjucuqV+SvAB4I/AfB8IXAmuSrAAeAs4GqKq7\nkqwB7gZ2AOdX1c5W5p3AFcAhwI3tBXA5cFWSDcBjdNf2JU0hk7s0ZKrqB4y6wa2qttHdPT/W+quA\nVWPE1wMnjRF/Enj7fqmspL1it7wkST1jcpckqWdM7pIk9YzJXZKknjG5S5LUMyZ3SZJ6xuQuSVLP\nmNwlSeoZk7skST1jcpckqWdM7pIk9YzJXZKknjG5S5LUMyZ3SZJ6xuQuSVLPmNwlSeqZSSf3JAcl\n+Yckn27zRya5Kcl97f2IgXUvSLIhyb1JTh+In5LkzrbskiRp8YOTXNPi65Is3H+7KEnScNmTM/d3\nAfcMzK8E1lbVImBtmyfJCcAy4ETgDOBDSQ5qZS4FzgUWtdcZLb4CeLyqjgcuBi7aq72RJEmTS+5J\njgHeCnx4ILwUWN2mVwNnDsSvrqqnquoBYANwapL5wGFVdWtVFXDlqDIj27oWWDJyVi9JkvbMZM/c\n/wR4L/Djgdi8qtrcph8B5rXpBcDDA+ttbLEFbXp0fJcyVbUDeAI4apJ1kzRJSX4yybVJvpnkniQ/\n7yU2qX8mTO5J3gZsqarbx1unnYnX/qzYOHU5L8n6JOu3bt16oD9O6qM/BT5TVS8DXk53qc1LbFLP\nTObM/VXAryR5ELgaeH2SjwKPtq522vuWtv4m4NiB8se02KY2PTq+S5kkc4DDgW2jK1JVl1XV4qpa\nPHfu3EntoKROksOB1wCXA1TVv1TVd/ESm9Q7Eyb3qrqgqo6pqoV0R/E3V9VvANcDy9tqy4Hr2vT1\nwLLWPXcc3VH9ba0Lf3uS01pjP2dUmZFtndU+44D3BEhD5jhgK/CX7S9fPpzkBUzTJTZ74qQDZ1/+\nzv1C4I1J7gPe0OapqruANcDdwGeA86tqZyvzTrqb8jYA3wJubPHLgaOSbAB+l9YtKGm/mgP8LHBp\nVb0C+AGj2tpUXWJrn2VPnHSAzNmTlavqFuCWNr0NWDLOequAVWPE1wMnjRF/Enj7ntRF0h7bCGys\nqnVt/lq65P5okvlVtXk/XmLbuLtLbJIOLJ9QJw2JqnoEeDjJS1toCV0Pm5fYpJ7ZozN3SbPe7wAf\nS/I84H7gt+gO8tckWQE8BJwN3SW2JCOX2Hbw7EtsVwCH0F1eG7zEdlW7xPYY3X06kqaYyV0aIlV1\nB7B4jEUz/hLbwpU3HIjNSr001MndHwtJUh95zV2SpJ4xuUuS1DMmd0mSesbkLklSz5jcJUnqGZO7\nJEk9Y3KXJKlnTO6SJPWMyV2SpJ4xuUuS1DMmd0mSesbkLklSz5jcJUnqGZO7JEk9Y3KXJKlnTO6S\nJPWMyV2SpJ4xuUuS1DMmd2nIJHkwyZ1J7kiyvsWOTHJTkvva+xED61+QZEOSe5OcPhA/pW1nQ5JL\nkqTFD05yTYuvS7JwqvdRGnYmd2k4va6qTq6qxW1+JbC2qhYBa9s8SU4AlgEnAmcAH0pyUCtzKXAu\nsKi9zmjxFcDjVXU8cDFw0RTsj6QBJndJAEuB1W16NXDmQPzqqnqqqh4ANgCnJpkPHFZVt1ZVAVeO\nKjOyrWuBJSNn9ZKmhsldGj4FfC7J7UnOa7F5VbW5TT8CzGvTC4CHB8pubLEFbXp0fJcyVbUDeAI4\nanQlkpyXZH2S9Vu3bt33vZL0tDnTXQFJU+7VVbUpyU8BNyX55uDCqqokdaArUVWXAZcBLF68+IB/\nnjRMPHOXhkxVbWrvW4BPAacCj7audtr7lrb6JuDYgeLHtNimNj06vkuZJHOAw4FtB2JfJI3N5C4N\nkSQvSHLoyDTwJuAbwPXA8rbacuC6Nn09sKzdAX8c3Y1zt7Uu/O1JTmvX088ZVWZkW2cBN7fr8pKm\niN3y0nCZB3yq3d82B/h4VX0myVeANUlWAA8BZwNU1V1J1gB3AzuA86tqZ9vWO4ErgEOAG9sL4HLg\nqiQbgMfo7raXNIVM7tIQqar7gZePEd8GLBmnzCpg1Rjx9cBJY8SfBN6+z5WVtNfslpckqWdM7pIk\n9cyE3fJJng98ETi4rX9tVf1BkiOBa4CFwIPA2VX1eCtzAd1TqnYC/6WqPtvip/DMNbq/Bd7V/uzm\nYLqHYJxCd1ftr1bVg/ttL/ezhStv2O3yBy986xTVRJKkZ5vMmftTwOur6uXAycAZSU7Dx1VKkjQj\nTZjcq/P9Nvvc9ip8XKUkSTPSpK65JzkoyR10D7a4qarWMQ2Pq5QkSRObVHKvqp1VdTLdU6hOTXLS\nqOVFdzZ/QPksakmSJrZHd8tX1XeBz9NdK5/yx1VW1WVVtbiqFs+dO3dPqi5J0tCYzN3yc4EfVdV3\nkxwCvJHuhreRR0xeyLMfV/nxJB8AXsQzj6vcmWR7uxlvHd3jKj84UGY58GV8XKWk/Wx3f+HiX7eo\njybzhLr5wOp2x/tzgDVV9ekkX8bHVUqSNONMmNyr6uvAK8aI+7hKSZJmIJ9QJ0lSz5jcJUnqGZO7\nJEk9Y3KXJKlnTO6SJPWMyV2SpJ4xuUuS1DMmd2nItIGg/iHJp9v8kUluSnJfez9iYN0LkmxIcm+S\n0wfipyS5sy27ZGQUxyQHJ7mmxdclWTjV+yfJ5C4No3cB9wzMrwTWVtUiYG2bJ8kJdE+LPJFuPIkP\ntSdVAlwKnEv3eOlFbTnACuDxqjoeuJjuUdWSppjJXRoiSY4B3gp8eCC8FFjdplcDZw7Er66qp6rq\nAWAD3aiQ84HDqurWNgbElaPKjGzrWmDJyFm9pKljcpeGy58A7wV+PBCbV1Wb2/QjwLw2vQB4eGC9\njS22oE2Pju9Spqp2AE8AR41VEYdwlg4ck7s0JJK8DdhSVbePt047E5+SERkdwlk6cCYzKpykfngV\n8CtJ3gI8HzgsyUeBR5PMr6rNrct9S1t/E3DsQPljWmxTmx4dHyyzMckc4HBg24HaIUlj88xdGhJV\ndUFVHVNVC+lulLu5qn4DuB5Y3lZbDlzXpq8HlrU74I+ju3HuttaFvz3Jae16+jmjyoxs66z2GVPS\nEyDpGZ65S7oQWJNkBfAQcDZAVd2VZA1wN7ADOL+qdrYy7wSuAA4BbmwvgMuBq5JsAB6jO4iQNMVM\n7tIQqqpbgFva9DZgyTjrrQJWjRFfD5w0RvxJ4O37saqS9oLd8pIk9YzJXZKknjG5S5LUMyZ3SZJ6\nxuQuSVLPmNwlSeoZk7skST1jcpckqWdM7pIk9YzJXZKknjG5S5LUMyZ3SZJ6xuQuSVLPmNwlSeoZ\nk7skST1jcpckqWdM7pIk9YzJXZKknpkwuSc5Nsnnk9yd5K4k72rxI5PclOS+9n7EQJkLkmxIcm+S\n0wfipyS5sy27JEla/OAk17T4uiQL9/+uSpI0HCZz5r4DeE9VnQCcBpyf5ARgJbC2qhYBa9s8bdky\n4ETgDOBDSQ5q27oUOBdY1F5ntPgK4PGqOh64GLhoP+ybpAFJnp/ktiRfawfq729xD9SlnpkwuVfV\n5qr6apv+HnAPsABYCqxuq60GzmzTS4Grq+qpqnoA2ACcmmQ+cFhV3VpVBVw5qszItq4Floz8WEja\nb54CXl9VLwdOBs5IchoeqEu9s0fX3NtR+CuAdcC8qtrcFj0CzGvTC4CHB4ptbLEFbXp0fJcyVbUD\neAI4aozPPy/J+iTrt27duidVl4Zedb7fZp/bXoUH6lLvTDq5J3kh8Eng3VW1fXBZa+C1n+v2LFV1\nWVUtrqrFc+fOPdAfJ/VOkoOS3AFsAW6qqmk5UG918WBdOkAmldyTPJcusX+sqv66hR9tR/C09y0t\nvgk4dqD4MS22qU2Pju9SJskc4HBg257ujKTdq6qdVXUyXfs7NclJo5ZPyYF6+ywP1qUDZDJ3ywe4\nHLinqj4wsOh6YHmbXg5cNxBf1m6sOY7uetxt7cxge5LT2jbPGVVmZFtnATe3HxlJB0BVfRf4PN21\ncg/UpZ6ZzJn7q4B3AK9Pckd7vQW4EHhjkvuAN7R5quouYA1wN/AZ4Pyq2tm29U7gw3TX7r4F3Nji\nlwNHJdkA/C7thh5J+0+SuUl+sk0fArwR+CYeqEu9M2eiFarqS8B4N8QsGafMKmDVGPH1wEljxJ8E\n3j5RXSTtk/nA6nbH+3OANVX16SRfBtYkWQE8BJwN3YF6kpED9R08+0D9CuAQuoP0wQP1q9qB+mN0\nd9tLmmITJndJ/VBVX6f7a5fR8W14oC71io+flSSpZ0zukiT1jMldkqSeMblLktQzJndJknrG5C5J\nUs+Y3CVJ6hmTuyRJPeNDbA6AhStvGHfZgxe+dQprIkkaRp65S5LUMyZ3SZJ6xm75Kba7Lnuw216S\ntO88c5ckqWdM7pIk9YzJXZKknjG5S5LUMyZ3SZJ6xuQuSVLPmNylIZHk2CSfT3J3kruSvKvFj0xy\nU5L72vsRA2UuSLIhyb1JTh+In5LkzrbskiRp8YOTXNPi65IsnOr9lGRyl4bJDuA9VXUCcBpwfpIT\ngJXA2qpaBKxt87Rly4ATgTOADyU5qG3rUuBcYFF7ndHiK4DHq+p44GLgoqnYMUm7MrlLQ6KqNlfV\nV9v094B7gAXAUmB1W201cGabXgpcXVVPVdUDwAbg1CTzgcOq6taqKuDKUWVGtnUtsGTkrF7S1DG5\nS0OodZe/AlgHzKuqzW3RI8C8Nr0AeHig2MYWW9CmR8d3KVNVO4AngKP2+w5I2q1eP352oke9SsMo\nyQuBTwLvrqrtgyfWVVVJaorqcR5wHsCLX/ziqfhIaWh45i4NkSTPpUvsH6uqv27hR1tXO+19S4tv\nAo4dKH5Mi21q06Pju5RJMgc4HNg2Vl2q6rKqWlxVi+fOnbuvuyZpQK/P3CU9o137vhy4p6o+MLDo\nemA5cGF7v24g/vEkHwBeRHfj3G1VtTPJ9iSn0XXrnwN8cNS2vgycBdzcrsvPWLvr4XMgJ81WJndp\neLwKeAdwZ5I7Wuy/0yX1NUlWAA8BZwNU1V1J1gB3091pf35V7Wzl3glcARwC3Nhe0B08XJVkA/AY\n3d32kqaYyV0aElX1JWC8O9eXjFNmFbBqjPh64KQx4k8Cb9+HakraD7zmLklSz5jcJUnqGZO7JEk9\nY3KXJKlnTO6SJPXMhMk9yUeSbEnyjYGYo0hJkjRDTebM/QqeGfFphKNISZI0Q02Y3Kvqi3QPoxjk\nKFKSJM1Qe3vNfVpGkUpyXpL1SdZv3bp1L6suSVK/7fMNde1MfEqeHe1AE5IkTWxvk/u0jCIlSZIm\ntrfJfWTkJ3j2KFLL2h3wx/HMKFKbge1JTmvX088ZVWZkW7NiFClJkmayCQeOSfIJ4JeAo5NsBP4A\nR5GSJGnGmjC5V9WvjbPIUaQkSZqBHPJ1hlm48oZxlz144VunsCaSpNnKx89KktQzJndJknrG5C5J\nUs+Y3CVJ6hmTuzREHOVRGg4md2m4XIGjPEq9Z3KXhoijPErDweQuyVEepZ4xuUt6mqM8Sv3gE+ok\nPZpkflVt3o+jPG7swyiPu3tiJPjUSM1cJvdZxEfT6gAZGZnxQp49yuPHk3wAeBHPjPK4M8n2JKcB\n6+hGefzgqG19GUd5lKaNyV0aIo7yKA0Hk7s0RBzlURoO3lAnSVLPmNwlSeoZk7skST1jcpckqWdM\n7pIk9YzJXZKknjG5S5LUMyZ3SZJ6xuQuSVLPmNwlSeoZk7skST3js+V7wqEpJUkjPHOXJKlnPHOX\npL20ux4ze8s0nTxzlySpZ0zukiT1jMldkqSe8Zr7kPDaoCQND8/cJUnqmRlz5p7kDOBPgYOAD1fV\nhdNcpaHhWb32N9uz7UrTa0acuSc5CPhz4M3ACcCvJTlhemslaW/YnqXpN1PO3E8FNlTV/QBJrgaW\nAndPa6004ZPv9oVnL71le57AvrQr240mY6Yk9wXAwwPzG4FXjl4pyXnAeW32+0nuHWd7RwPf2a81\nnHq934dcNIU12Xt9+B5eOsWfZ3se3z7vywxqN34vU++nJ7viTEnuk1JVlwGXTbRekvVVtXgKqnTA\nuA8zQ1/2YbrrMJZhas8j3JeZqU/7MmJGXHMHNgHHDswf02KSZh/bszTNZkpy/wqwKMlxSZ4HLAOu\nn+Y6Sdo7tmdpms2Ibvmq2pHkt4HP0v3pzEeq6q592OSEXX2zgPswM7gPe8j2vFvuy8zUp30BIFU1\n3XWQJEn70UzplpckSfuJyV2SpJ7pXXJPckaSe5NsSLJyuuszWUkeTHJnkjtG/nQpyZFJbkpyX3s/\nYrrrOSjJR5JsSfKNgdi4dU5yQfte7k1y+vTU+hnj1P99STa17+GOJG8ZWDaj6g+Q5Ngkn09yd5K7\nkryrxWfN9zCe2dqWR8zGNj1itrftQX1o53ulqnrzort551vAS4DnAV8DTpjuek2y7g8CR4+K/RGw\nsk2vBC6a7nqOqt9rgJ8FvjFRnekeQ/o14GDguPY9HTQD6/8+4L+Ose6Mq3+r13zgZ9v0ocA/trrO\nmu9hnP2atW15YB9mXZseqOesbtuT2JdZ1c735tW3M/enH3tZVf8CjDz2crZaCqxu06uBM6exLs9S\nVV8EHhsVHq/OS4Grq+qpqnoA2ED3fU2bceo/nhlXf4Cq2lxVX23T3wPuoXtC3Kz5HsbRt7Y8Yka3\n6RGzvW0P6kM73xt9S+5jPfZywTTVZU8V8Lkkt7fHcgLMq6rNbfoRYN70VG2PjFfn2fTd/E6Sr7fu\nvJGuxxlf/yQLgVcA65j938Nsqefu9KVNj5jt/6dGm5XtfLL6ltxns1dX1cl0I2mdn+Q1gwur6zOa\nVX+3OBvrDFxK1xV8MrAZ+OPprc7kJHkh8Eng3VW1fXDZLP0e+qB3bXrEbK57Myvb+Z7oW3KftY+9\nrKpN7X0L8Cm6rqBHk8wHaO9bpq+GkzZenWfFd1NVj1bVzqr6MfAXPNMlN2Prn+S5dIn9Y1X11y08\nq78HZk89x9WjNj1itv+fetpsbOd7qm/JfVY+9jLJC5IcOjINvAn4Bl3dl7fVlgPXTU8N98h4db4e\nWJbk4CTHAYuA26ahfrs18uPV/Du67wFmaP2TBLgcuKeqPjCwaFZ/D8zStjyiZ216xGz/P/W02dbO\n98p039G3v1/AW+juGP4W8HvTXZ9J1vkldHdofg24a6TewFHAWuA+4HPAkdNd11H1/gRdl9aP6K5N\nrdhdnYHfa9/LvcCbZ2j9rwLuBL5O19Dnz9T6tzq9mq579OvAHe31ltn0Pexm32ZdWx6o+6xs0wP1\nn9VtexL7Mqva+d68fPysJEk907dueUmShp7JXZKknjG5S5LUMyZ3SZJ6xuQuSVLPmNwlSeoZk7sk\nST3z/wE4mkZESZz+lQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2fae8b0610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.figure(figsize=[8,4])\n",
    "plt.subplot(1,2,1)\n",
    "plt.title(\"words\")\n",
    "plt.hist(list(map(len,all_words)),bins=25);\n",
    "\n",
    "plt.subplot(1,2,2)\n",
    "plt.title('translations')\n",
    "plt.hist(list(map(len,all_translations)),bins=25);\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: auxiliary functions\n",
    "\n",
    "we need some helper functions that\n",
    "* convert data from strings to integer matrices\n",
    "* sample random minibatches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def as_matrix(sequences,token_to_i, max_len=None,PAX_ix=PAD_ix):\n",
    "    \"\"\"\n",
    "    convert variable length token sequences into  fixed size matrix\n",
    "    example usage: \n",
    "    >>>print( as_matrix(words[:3],source_to_ix))\n",
    "    [[15 22 21 28 27 13 -1 -1 -1 -1 -1]\n",
    "     [30 21 15 15 21 14 28 27 13 -1 -1]\n",
    "     [25 37 31 34 21 20 37 21 28 19 13]]\n",
    "    \"\"\"\n",
    "    max_len = max_len or max(map(len,sequences))\n",
    "    \n",
    "    matrix = np.zeros((len(sequences),max_len),dtype='int32') +PAD_ix\n",
    "    for i,seq in enumerate(sequences):\n",
    "        row_ix = list(map(token_to_i.get,seq))[:max_len]\n",
    "        matrix[i,:len(row_ix)] = row_ix\n",
    "    \n",
    "    return matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import random\n",
    "def sample_batch(words,word_to_translation, batch_size):\n",
    "    \"\"\"\n",
    "    sample random batch of words and random correct translation for each word\n",
    "    example usage:\n",
    "    batch_x,batch_y = sample_batch(train_words, word_to_translations,10)\n",
    "    \"\"\"\n",
    "    \n",
    "    #choose words\n",
    "    batch_words = np.random.choice(words,size=batch_size)\n",
    "    \n",
    "    #choose translations\n",
    "    batch_trans_candidates = list(map(word_to_translation.get,batch_words))\n",
    "    batch_trans = list(map(random.choice,batch_trans_candidates))\n",
    "    \n",
    "    return as_matrix(batch_words,source_to_ix), as_matrix(batch_trans,target_to_ix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### split the dataset\n",
    "\n",
    "We hold out 20% of all words to be used for validation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda2/lib/python2.7/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "\n",
    "train_words,test_words = train_test_split(all_words,test_size=0.1,random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Build encoder-decoder (1 point)\n",
    "\n",
    "__assignment starts here__\n",
    "\n",
    "Our architecture consists of two main blocks:\n",
    "* Encoder reads words character by character and outputs code vector (usually a function of last RNN state)\n",
    "* Decoder takes that code vector and produces translations character by character\n",
    "\n",
    "In this section, we'll implement __encoder__ the same way you did for week6.5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: THEANO_FLAGS=device=gpu4,floatX=float32\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10).  Please switch to the gpuarray backend. You can get more information about how to switch at this URL:\n",
      " https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29\n",
      "\n",
      "Using gpu device 4: GeForce GTX 1080 (CNMeM is enabled with initial size: 22.0% of memory, cuDNN 5110)\n"
     ]
    }
   ],
   "source": [
    "%env THEANO_FLAGS=device=gpu4,floatX=float32\n",
    "import theano\n",
    "import theano.tensor as T\n",
    "\n",
    "import lasagne\n",
    "from lasagne.layers import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One more note: in this assignment, we'll be using classes as namespaces:\n",
    "\n",
    "```\n",
    "class my_pocket:\n",
    "    coin = \"$\"\n",
    "    coins = coin*3\n",
    "    mobile = \"nokia 3310\"\n",
    "    \n",
    ">>>print my_pocket.coins\n",
    "$$$\n",
    ">>>print my_pocket.mobile\n",
    "nokia 3310\n",
    "```\n",
    "\n",
    "\n",
    "Your first assignment is to implement encoder network using lasagne layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_mask_by_eos(is_eos):\n",
    "    \"\"\"takes indicator of \"it ends now\", returns mask.\n",
    "    Ignores everything after first end.\"\"\"\n",
    "    assert is_eos.ndim==2\n",
    "    is_right_after_eos = T.concatenate([T.zeros_like(is_eos[:,:1]),is_eos[:,:-1]],-1)\n",
    "    is_after_eos = T.eq(T.cumsum(is_right_after_eos,axis=-1),0).astype('uint8')\n",
    "    return is_after_eos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class encoder:\n",
    "    \"\"\"encoder rnn\"\"\"\n",
    "    \n",
    "    #input tokens and mask\n",
    "    input_sequence = T.matrix('token sequence','int32')\n",
    "    input_mask = get_mask_by_eos(T.eq(input_sequence,EOS_ix_source))\n",
    "    \n",
    "    inp = InputLayer(shape=(None, None),input_var=input_sequence,name='encoder input')\n",
    "    mask = InputLayer(shape=(None, None),input_var=input_mask,name='encoder mask')#<a layer that takes input_mask>\n",
    "    \n",
    "    #embedding\n",
    "    emb = EmbeddingLayer(inp,#<which layer?>,\n",
    "                         input_size=len(source_letters),#<how many letters?>,\n",
    "                         output_size=64)\n",
    "    \n",
    "    #encoder rnn\n",
    "    rnn1 = LSTMLayer(incoming=emb,#<which layer?>,\n",
    "                     num_units=512,\n",
    "                     mask_input=mask)#=<which layer?>)\n",
    "    \n",
    "    rnn2 = LSTMLayer(incoming=rnn1,#<which layer?>,\n",
    "                     num_units=512,\n",
    "                     mask_input=mask)#=<which layer?>)\n",
    "    \n",
    "    #slice last time-step of encoder rnn\n",
    "    rnn1_last = SliceLayer(rnn1,-1,axis=1,name='last rnn1 time-step')\n",
    "    rnn2_last = SliceLayer(rnn2,-1,axis=1,name='last rnn2 time-step')\n",
    "    \n",
    "    #compute decoder initial state\n",
    "    code1 = DenseLayer(rnn1_last,512,nonlinearity=T.tanh)\n",
    "    code2 = DenseLayer(rnn2_last,512,nonlinearity=T.tanh)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Decoder\n",
    "\n",
    "In this section, we will define __one step__ of decoder (just like we defined one step of agent last week).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from agentnet.memory import RNNCell,GRUCell,LSTMCell\n",
    "from agentnet.resolver import ProbabilisticResolver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class decoder:\n",
    "    \"\"\"single step of decoder rnn\"\"\"\n",
    "    \n",
    "    inp = InputLayer((None,),name=\"prev phoneme\")\n",
    "    \n",
    "    emb = EmbeddingLayer(inp, len(target_letters), 50)\n",
    "    \n",
    "    #decoder memory 1\n",
    "    prev_cell1,prev_out1 = InputLayer((None,512)),InputLayer((None,512))\n",
    "    lstm_cell1,lstm_out1 = LSTMCell(prev_cell1,prev_out1,emb)\n",
    "    \n",
    "    #decoder memory 2\n",
    "    prev_cell2,prev_out2 = InputLayer((None,512)),InputLayer((None,512))\n",
    "    lstm_cell2,lstm_out2 = LSTMCell(prev_cell2,prev_out2,lstm_out1)\n",
    "    \n",
    "    neck = concat([lstm_out1,lstm_out2])\n",
    "    \n",
    "    logits = DenseLayer(neck,len(target_letters),nonlinearity=None)\n",
    "    \n",
    "    #probabilities\n",
    "    probs = NonlinearityLayer(logits,T.nnet.softmax)\n",
    "    \n",
    "    #output phonemes\n",
    "    out = ProbabilisticResolver(probs,assume_normalized=True)\n",
    "    \n",
    "    #log-probabilities\n",
    "    logprobs = NonlinearityLayer(logits,T.nnet.logsoftmax)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wire it all together (1 point)\n",
    "\n",
    "Here we define functions for model _inference_ (both greedy and sampled)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from agentnet import Recurrence\n",
    "from collections import OrderedDict as od\n",
    "class model:\n",
    "    #maximum output length for inference\n",
    "    n_steps = theano.shared(MAX_OUTPUT_LENGTH)\n",
    "    \n",
    "    #initial inputs: indices of \"START\" special phoneme.\n",
    "    l_start = InputLayer((None,),T.zeros_like(encoder.input_sequence[:,0])+BOS_ix_target)\n",
    "\n",
    "    #Here we define recurrence: it's a custom recurrent layer, acting exactly like Agent, \n",
    "    #except it's a lasagne layer.\n",
    "    \n",
    "    rec = Recurrence(\n",
    "        #recurrent states\n",
    "        state_variables=od({decoder.lstm_cell1:decoder.prev_cell1,\n",
    "                            decoder.lstm_out1:decoder.prev_out1,\n",
    "                            decoder.lstm_cell2:decoder.prev_cell2,\n",
    "                            decoder.lstm_out2:decoder.prev_out2,\n",
    "                            decoder.out:decoder.inp}),#<where decode.out goes on next tick?}),\n",
    "        \n",
    "        #initial values for recurrent states\n",
    "        state_init={decoder.out:l_start,\n",
    "                    decoder.lstm_cell1:encoder.code1,\n",
    "                    decoder.lstm_out1:encoder.code1,\n",
    "                    decoder.lstm_cell2:encoder.code2,\n",
    "                    decoder.lstm_out2:encoder.code2,\n",
    "                   },\n",
    "        \n",
    "        tracked_outputs=(decoder.out,decoder.probs,decoder.logprobs),\n",
    "        \n",
    "        unroll_scan=False,\n",
    "        n_steps=n_steps\n",
    "    )\n",
    "    \n",
    "    weights = get_all_params(rec,trainable=True)\n",
    "    \n",
    "    \n",
    "    #sample mode\n",
    "    predicted_translations,probs_seq,logprobs_seq = get_output(rec[decoder.out,decoder.probs,decoder.logprobs])\n",
    "    auto_updates = rec.get_automatic_updates()\n",
    "\n",
    "    #output mask\n",
    "    mask = get_mask_by_eos(T.eq(predicted_translations,EOS_ix_target))\n",
    "    \n",
    "    generate_sample = theano.function([encoder.input_sequence],predicted_translations,\n",
    "                                      updates=auto_updates)\n",
    "    \n",
    "    #greedy mode (picking max-probability actions on each step)\n",
    "    greedy_translations = get_output(rec[decoder.out],recurrence_flags={\"greedy\":True})\n",
    "    greedy_auto_updates = rec.get_automatic_updates()\n",
    "    \n",
    "    greedy_mask = get_mask_by_eos(T.eq(greedy_translations,#<what?>,\n",
    "                                       EOS_ix_target))\n",
    "    \n",
    "    \n",
    "    generate_greedy = theano.function([encoder.input_sequence],greedy_translations,#[<what goes in?>],<what goes out?>,\n",
    "                                       updates=greedy_auto_updates)\n",
    "    \n",
    "\n",
    "    @staticmethod\n",
    "    def translate(word,sample=False):\n",
    "        assert word.endswith(END)\n",
    "        \n",
    "        #convert to matrix\n",
    "        word_ix = as_matrix([word.lower()],source_to_ix)\n",
    "        \n",
    "        #generate output\n",
    "        if sample:\n",
    "            trans_ix = model.generate_sample(word_ix)[0]\n",
    "        else:\n",
    "            trans_ix = model.generate_greedy(word_ix)[0]\n",
    "        \n",
    "        #convert from int32 to string\n",
    "        trans = list(map(target_letters.__getitem__,trans_ix))\n",
    "        \n",
    "        #crop padding\n",
    "        if END in trans:\n",
    "            trans = trans[:trans.index(END)+1]\n",
    "            \n",
    "        return ''.join(trans)\n",
    "        \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "source:סופר;\n",
      "sample:����'x*zm�0j��$���m�x���\"�nۇ�yks΃ƥ������$��v2�\n",
      "greedy:zzz�nss%g~��$y�a-md����!�.���u�$yÉ?-m���u�n��h+�\n"
     ]
    }
   ],
   "source": [
    "#test untrained model\n",
    "#should be random\n",
    "print ('source:'+all_words[0])\n",
    "print ('sample:'+model.translate(all_words[0],sample=True)) #sample mode\n",
    "print ('greedy:'+model.translate(all_words[0]))             #inference mode\n",
    "\n",
    "\n",
    "#praise Cthulhu!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Scoring function\n",
    "\n",
    "LogLikelihood is a poor estimator of model performance.\n",
    "* If we predict zero probability once, it shouldn't ruin entire model.\n",
    "* It is enough to learn just one translation if there are several correct ones.\n",
    "* What matters is which output will we produce if we __take most likely phoneme on each step.__\n",
    "\n",
    "Therefore, we will use minimal Levenshtein distance.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import editdistance #!pip install editdistance\n",
    "\n",
    "def get_distance(word,trans):\n",
    "    \"\"\"\n",
    "    A function that takes word and predicted translation\n",
    "    and evaluates (Levenshtein's) edit distance to closest correct translation\n",
    "    \"\"\"\n",
    "    references = word_to_translation[word]\n",
    "    assert len(references)!=0,\"wrong/unknown word\"\n",
    "    return min(editdistance.eval(trans,ref) for ref in references)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def score(bsize=100):\n",
    "    \"\"\"a function that computes levenshtein distance for bsize random samples\"\"\"\n",
    "    \n",
    "    batch_words = np.random.choice(test_words,size=bsize)#<pick bsize random words from test_words>\n",
    "    \n",
    "    #<for each word, predict using sample=False>\n",
    "    #distances=<compute get_score for each word in batch_words>\n",
    "    \n",
    "    distances = []\n",
    "    for word in batch_words:\n",
    "        predicted = model.translate(word,sample=False)\n",
    "        distances.append(get_distance(word,predicted))\n",
    "        \n",
    "    return np.array(distances,dtype='float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[38.16,\n",
       " 37.830002,\n",
       " 36.029999,\n",
       " 35.68,\n",
       " 34.540001,\n",
       " 40.549999,\n",
       " 35.099998,\n",
       " 38.490002,\n",
       " 36.439999,\n",
       " 39.110001]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#should be around 5-50 and decrease rapidly :)\n",
    "[score(100).mean() for _ in range(10)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Supervised pre-training\n",
    "\n",
    "Here we define a function that trains our model through maximizing log-likelihood a.k.a. minimizing crossentropy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from agentnet.learning.generic import get_values_for_actions\n",
    "\n",
    "class llh_trainer:\n",
    "\n",
    "    #variable for correct answers\n",
    "    reference_answers = T.imatrix(\"reference translations\")\n",
    "    \n",
    "    #shift 1 step to the past to get \"previous answers\"\n",
    "    prev_answers = T.concatenate([T.zeros_like(reference_answers[:,:1])+BOS_ix_target,\n",
    "                                  reference_answers[:,:-1]],axis=1)\n",
    "    \n",
    "    #mask on prev answers\n",
    "    input_mask = get_mask_by_eos(T.eq(prev_answers,EOS_ix_target))\n",
    "    \n",
    "    #create input layers\n",
    "    l_sequence = InputLayer((None,None),prev_answers)\n",
    "    l_mask = InputLayer((None,None),input_mask)\n",
    "    \n",
    "    #teacher-forced trainer\n",
    "    rec = Recurrence(input_sequences={decoder.inp:l_sequence},\n",
    "                     \n",
    "                     #next state:prev state\n",
    "                     state_variables=od({decoder.lstm_cell1:decoder.prev_cell1,\n",
    "                                         decoder.lstm_out1:decoder.prev_out1,\n",
    "                                         decoder.lstm_cell2:decoder.prev_cell2,\n",
    "                                         decoder.lstm_out2:decoder.prev_out2,}),\n",
    "                     \n",
    "                     #layer:initial value\n",
    "                     state_init={decoder.lstm_cell1:encoder.code1,\n",
    "                                 decoder.lstm_out1:encoder.code1,\n",
    "                                 decoder.lstm_cell2:encoder.code2,\n",
    "                                 decoder.lstm_out2:encoder.code2,},\n",
    "                     \n",
    "                     tracked_outputs=(decoder.probs,decoder.logprobs),\n",
    "                     unroll_scan=False,\n",
    "                     mask_input=l_mask)\n",
    "    \n",
    "    \n",
    "    #get log-probabilities\n",
    "    logprobs_seq = get_output(rec[decoder.logprobs])\n",
    "    auto_updates = rec.get_automatic_updates()\n",
    "    \n",
    "    #compute mean crossentropy\n",
    "    crossentropy = -get_values_for_actions(logprobs_seq,reference_answers)\n",
    "    \n",
    "    loss = T.sum(crossentropy*input_mask)/T.sum(input_mask)\n",
    "    \n",
    "    #get all params\n",
    "    weights = get_all_params(encoder.code2,trainable=True)\n",
    "\n",
    "    #weight updates           \n",
    "    grads = T.grad(loss,model.weights)\n",
    "    grads = lasagne.updates.total_norm_constraint(grads,10)\n",
    "    updates = lasagne.updates.adam(grads, model.weights)\n",
    "\n",
    "    train_step = theano.function([encoder.input_sequence,reference_answers],loss,\n",
    "                                 updates=auto_updates+updates)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAegAAAEICAYAAACUFGeOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXl8ZGd15/09tS/aW2r1vnhr4xVwG2xWYWNi4iQwmWQG\n3jADgYknmSSQDBkDmZckb5YJWWYYEpJhPIE4GQjEOCEQSIwXkJeAjRfaa9ttuzf13tpV+/a8f9yl\nbpWqVFVSSSpJ5/v56KOqu55b0q3fPec5zzlijEFRFEVRlM7Ct9oGKIqiKIoyHxVoRVEURelAVKAV\nRVEUpQNRgVYURVGUDkQFWlEURVE6EBVoRVEURelAVKCXiIiMiMiJ1bZjsYjIB0TkYc/7hIhc0KZj\n/7qI/IX9eo+IGBEJtOnYu2xb/e04nrJxWOv37GogIs+JyMgKnetzIvLJlThXp6MCrVRgjOkyxhxe\naJtmv+CMMf/NGPMf2mGXiBwVkbd7jn3ctrXYjuMrSqcjIi+KyCU1lo+KSFvus3oYYy43xoy2+7jV\nDoJ9rp83xvxOu8+1FlGBXsO0yxtdDjrZNkVZLRZ7X4jIhYDfGHNopc6prD4q0ICIfExE7qpa9hkR\n+RP79c+KyEERmRORwyLyH5s8rojIp0XknIjMisgzInKFvS4qIv9dRI6JyIyIPCwiUXvdT9ghpWn7\n6fhVnmMete19GkiKSEBEtonI34nIeRE5IiIfXsCmTSLyDdueHwAXVq03InKR/fpHReR5+7pPisiv\niUgc+Gdgmx1iTtjn/y0RuUtEvigis8AH7GVfrDLhgyJySkROi8ivec57h4j8rue966WLyP8FdgH/\naJ/vtuqQuW3DN0RkUkReFpGf8xzrt0TkThH5a/tanhOR/c38DZXOZCPdsza3AP9Uw97fA94MfNa+\nNz5rLzci8osi8hLwkufzGbOv6wkRebPnOAveI+KJYDWx7WtF5If2uq+KyN96723Pdq8CPgdcb9s+\nbS93vwuc7wH7nj9nf2+82/5uOmTf77/uOaZPRD4uIq+IyIRt50CDz7ZzMcZs+B9gN5ACuu33fuA0\ncJ39/hYsIRPgrfa2r7XXjQAn6hz3R4AngD5731cBW+11fwaMAtvt870BCAOXAEngJiAI3Aa8DITs\n/Y4CB4CdQBTrIesJ4DeAEHABcBj4kTo2fQW4E4gDVwAngYc96w1wkf36NPBm+3X/QtcM/BaQB95t\n2xS1l33RXr/HPvaX7XNfCZwH3m6vvwP4Xc/xKs5hX/fbPe+d4wXs9w8Cfw5EgFfbx77BY1sG+FH7\ns/594JHV/r/TH71nm7ln7WPcvcA9PQr8h6plBrgXGACi9rL3AZuAAPBR4AwQsdcteI9477+FtrWv\n5xjwEfuz+Ekgh+ferrLzA3i+f+xldzjb23+rgv1ZBYGfs+/tvwG6gcuBNLDX3v4jwCPADvtv87+B\nL6/2/+ui/89X24BO+QEeBv69/fom4JUFtv0H4COef6B6N/sNwCHgOsDnWe6z/6murrHPJ4E7q7Y9\nCYzY748CH/Ssfz1wvOoYnwD+ssax/Vgieqln2X+jvkAfB/4j0FN1nHnXbN+0D9ZYVi3Q3nP/IfB5\n+7V7U9Y6BwsINNYXXxH7y9pe//vAHR477vOsuwxIr/b/nP4s7Wcj3LP2uhgwAYTrrB+ltkDf0ODz\nm3Kup9E9wnyBrrkt8Bb72qXq77QUgU5jhffBEmUDvN6z/RPAu+3XB4EbPeu2Yn3nBVb7/3UxPxri\nLvM3wHvt1/+P/R4AEXmniDxih1OmsZ4cBxsd0BjzHeCzWE/e50TkdhHpsfeNAK/U2G0b1hOoc4wS\nMIb11O4w5nm9GyvcPO38AL8ODNc49hCWoHn3P1ZjO4d/jXWtx0TkARG5foFtq+1qZptjWNe7VLYB\nk8aYuapjez+zM57XKSAiOja31tkI9yzAjcD3jDHZRvZXUXE/ijVEddAOz08DvVR+Jq3cI/W23Qac\nNLY61rJjEUyYcjJo2v591rM+DXTZr3cDX/N8rgexHt7rfbYdjQp0ma8CIyKyA/hX2De7iISBvwP+\nGBg2xvRhjQVJMwc1xvyJMeYarKfMS4D/AoxjhYgurLHLKax/MuzzC5aHeNJ7WM/rMeCIMabP89Nt\njPnRGsc+jxUu2ulZtmsB2x8zxrwL2IzlgdxZ4/wVu9Q7lofqc5+yXyexPAWHLS0c+xQwICLdVcc+\nWWd7ZX2wEe5ZsB4u5o0/1zl2zeX2ePNtwL8B+u3PZIYmP5MWOA1stz8Dh531Nqa574xWGAPeWfXZ\nRowxa/K7QAXaxhhzHitU9JdYN89Be1UIayzjPFAQkXcC72jmmCJyrYi8XkSCWAKUAUr2E/YXgP9h\nJ4v4ReR6+4vlTuAWEbnR3u+jQBb4Xp3T/ACYs5NQovaxrhCRa2tcYxH4e+C3RCQmIpcB769je0hE\nfkZEeo0xeWAWKNmrzwKbRKS3mc+hik/a574c+Fngb+3lB4AfFZEBEdkC/ErVfmexxurmYYwZw/p8\nfl9EIiJyFfAhoDpBTVlHbIR71uadwLcWMLvuveGhG+vh/DwQEJHfAHoa7LMYvo/lsf6SWMlw7wJe\nt8D2Z4EdIhJq0/k/B/yeiOwGEJEh24Y1iQp0JX8DvB1PqMwOm34Y6yacwgqlfaPJ4/UA/8fe7xjW\nONIf2et+DXgGeAyYBP4Aa8zrRaxkjj/Femr/ceDHjTG5WiewRffHsBKjjtj7/AVW+KoWv4QVDjqD\nNdbzlwvY/++Ao2JlZf888DP2OV/ASvY6bIeSWglTP4CVQHM/8MfGmHvs5f8XeAprrOseysLt8PvA\n/2uf79eYz3uxxqVPAV8DftMYc18Ldilrk3V9z4qVQZ4wxhxfwObPAD8lIlNiZ7HX4NtYiWaH7OvK\nsPTQ8zzsa/5JrAfkaazP5ZtYDyy1+A7wHHBGRMbbYMJnsP7W94jIHFbC2OvbcNxVQSqHChRFUZRO\nQURuAwaNMbetti2LRUQeBT5njFnIGVBqoEkyiqIonctR4B9X24hWEJG3Ai9iRQZ+BrgKy3tXWkQF\nWlEUpUMxxtzZeKuOYx/lWguHgZ8yxpxeXZPWJhriVhRFUZQORJPEFEVRFKUDWdEQ9+DgoNmzZ89K\nnlJR1iRPPPHEuDFmaLXtqEez93IymSQejy+/QS3SqXZB59qmdrVOMpnkhRdeWPS9vKICvWfPHh5/\n/PGVPKWirElEZKEKb6tOs/fy6OgoIyMjy29Qi3SqXdC5tqldrTM6Osrb3va2Rd/LGuJWFEVRlA5E\nBVpRFEVROhAVaEVRFEXpQFSgFUVRFKUDUYFWFEVRlA5EBVpRFEVROpCGAi0iXxCRcyLybI11HxUR\nIyING6EriqIoitI8zXjQdwA3Vy8UkZ1YPVYXaoOmKMoGxhjDVx8fI5MvrrYpirLmaCjQxpgHsXqf\nVvNp4DZAi3krilKT00nDf7nrab793JnVNkVR1hyLqiQmIu8CThpjnhKRRtveCtwKsGvXrobH3vPx\nbzXc5uinbmnKTkVRVpds0Xp+n0jkVtkSRVl7tJwkJiIx4NeB32hme2PM7caY/caY/UNDHVtaWFGU\nZSBfsn5PpVSgFaVVFpPFfSGwF3hKRI4CO4AnRWRLOw1TFGXtU7AFeiKpAq0ordJyiNsY8wyw2Xlv\ni/R+Y8x4G+1SFGUdkC9ZIe4pFWhFaZlmpll9Gfg+sE9ETojIh5bfLEVR1gNO8vakCrSitExDD9oY\n894G6/e0zRpFUdYVBR2DVpRFo5XEFEVZNpwQt3rQitI6KtCKoiwb5SzuPKWSlkxQlFZQgVYUZdlw\nQtzFkmEuU1hdYxRljaECrSjKspH3eM2TOg6tKC2hAq0oyrLheNAAk8ns6hmiKGsQFWhFUZaNfIVA\n51fPEEVZg6hAK4qybHhD3FqsRFFaQwVaUZRlo1CCUMD6mtExaEVpDRVoRVGWjXwJ+qJBwgGfetCK\n0iKLaje5ntD2loqyfORLhnDQx4AvpA0zFKVFNrxAK4qyfBRKEA746Q6rB60oraIhbkVRlo18EUJ+\nHwPxkI5BK0qLqEArirJsOEliA/GQetCK0iIq0IqiLBv5kiFsC7SOQStKa6hAK4qybORtD7o/FmIu\nUyBfLDXeSVEUQAVaUZRlxEoS8zEQDwLaF1pRWkEFWlGUZaNQMoQDfvrjIQCmtNynojSNCrSiKMuG\nE+IeiFkCPanj0IrSNCrQiqIsG3k7xB0LWyUX0nntCa0ozaICrSjKslEoGUIBH5Gg9VWTyWuSmKI0\nS0OBFpEviMg5EXnWs+yPROQFEXlaRL4mIn3La6aiKGuRfMkqVBIN+gFI54qrbJGirB2a8aDvAG6u\nWnYvcIUx5irgEPCJNtulKMo6oFCCcNAj0HkVaEVploYCbYx5EJisWnaPMcYZTHoE2LEMtimK0mZq\nRcTs5b9sR8WeE5E/bMe5iiVD0UDI7ydsC3RGBVpRmqYdzTI+CPxtvZUicitwK8CuXbvacLrmOlCB\ndqFSlBrcAXwW+GtngYi8DXgXcLUxJisim9txolzBGm/2etDNCvQPjkySK5R408WD7TBFUdYkS0oS\nE5H/ChSAL9XbxhhzuzFmvzFm/9DQ0FJOpyjKEqkVEQN+AfiUMSZrb3OuHedyBDrk9xH0C36fNB3i\n/sO7X+AP7n6hHWYoyppl0R60iHwA+DHgRmOMaZtFiqKsNJcAbxaR3wMywK8ZYx6r3sgbDRseHmZ0\ndHTBg05nLIE+evhlHigcIyiGlw8fY3T0TEODDp1OEQvQ8ByLJZFILNuxl0qn2qZ2tU4ikVjS/osS\naBG5GbgNeKsxJrUkCxRFWW0CwABwHXAtcKeIXFD94G2MuR24HWD//v1mZGRkwYOOTaZg9Ltcedml\njOzfSdfD97JpyxZGRq5ccL9EtsDs3d8mGgnT6ByLZXR0dNmOvVQ61Ta1q3WW+uDQzDSrLwPfB/aJ\nyAkR+RDWGFY3cK+IHBCRzy3JCkVRVpMTwN8bix8AJWDJg785uzFGKGB9zUSCfjJNTLM6NpEEIJXV\nhDJlY9PQgzbGvLfG4s8vgy2KoqwO/wC8DfiuiFwChIDxpR40axclCXsFutBYdI9PWEG5VL6IMQYR\nWaopirIm0UpiirKBqBMR+wJwgT316ivA+9uRV+J40OGAlcEdDfqbKlRy1BboYsmQLWjlMWXj0o5p\nVoqirBHqRMQA3tfuc2XtjG0nxB0N+pvK4j4+mXRfp3JFIvYULUXZaKgHrSjKslD2oK2vmXDQ11Qt\n7qPj5bzTZFabaygbFxVoRVGWBXcetMeDbqZQybGJpFvYJKW1u5UNjAq0oijLQrZaoEONQ9yZfJHT\nsxku3doNQDKnHrSycVGBVhRlWXBLfdpJYpFAYw/6xFQKY+BVW3sAnWqlbGxUoBVFWRbmhbhDjbO4\nj9kZ3JfZAq0etLKRUYFWFGVZyNpznkN+zzzoBklizhQr14NWgVY2MCrQiqIsC1lPNyuASNBHrlii\nWKo/xfrYRJLucIAd/VEAkhriVjYwOg96hWmmVaa2yVTWA26pT385ixusRLB4uPZXz0Qyx1BP2F2v\nHrSykVEPWlGUZaG61Gc0ZAn0QpncqWyBrnDAFXP1oJWNjAq0oijLQq5YIiC4tbQjgbIHXY9krkgs\n5MfvE6JBv3rQyoZGBVpRlGUhmy8R8HzDREKNBTqVKxAPWeHteNhPUguVKBsYFWhFUZaFXLGIt4y2\nE7ZO5+pncqeyRWL2+HMsFGiquYairFdUoBVFWRZyhRJBX7lVZMTO5l6o5WQyVyBue9qxkF9rcSsb\nGhVoRVGWhWyhRNDzDVP2oBdKEisSc0PcAa3FrWxoVKAVRVkWcoWqMejgwmPQxhjLgw57PGhNElM2\nMCrQiqIsC5ZAe0PcC0+zyhZKlAxlDzoU0FrcyoZGBVpRlGVhXoi7QRa3M97setBh9aCVjY0KtKIo\ny0Kuzhh0vXrczniz40HHQn4dg1Y2NCrQiqIsC9lCsSrEbX3d1AtxO96yk8UdDwU0i1vZ0DQUaBH5\ngoicE5FnPcsGROReEXnJ/t2/vGYqirLWqA5xO5XE0rki44ksX3j4CMaUG2c4ZT2986CzhRKF4sId\nsBRlvdKMB30HcHPVso8D9xtjLgbut98riqK45IqVWdw+nxAO+MgUivzDD0/y2998npPTaXd9qtqD\ntseiUwtUHlOU9UxDgTbGPAhMVi1+F/BX9uu/At7dZrsURVnjZPOVhUrA7gmdK3J6JgPAdCrvrnM9\n6FCg4rdmcisblcWOQQ8bY07br88Aw/U2FJFbReRxEXn8/PnzizydoihrjVyxMsQNVqJYOl/kjC3Q\nM+myQLsedLjSg9ZMbmWjsuQkMWMNItXtwG6Mud0Ys98Ys39oaGipp1MUZY1QXagErKlWmXyJUzNW\naLvCg56Xxa0etLKxWaxAnxWRrQD273PtM0lRlPVAtlAk6K8McYcDvvoedNU8aGcsWj1oZaOyWIH+\nBvB++/X7ga+3xxxFUdYL9TzoZLbA2Vl7DDqdc9clc0VEytneTja39oRWNirNTLP6MvB9YJ+InBCR\nDwGfAm4SkZeAt9vvFUVRACgUrbKdtcagj0+mKNmDYjOpSg86FvTjsxPLHA9ai5UoG5VAow2MMe+t\ns+rGNtuiKMo6IVuw5i7XyuL2Tq3yhriTuXIvaPB40DoGrWxQtJKYoihtJ2cL9LwQd9CPU5sk4JOK\nJLGUpxc0QCyoY9DKxkYFWlGUtuMT4R2XDTMcm+9BO1wwFK8cg/b0ggarWQZoiFvZuKhAK4rSdnpj\nQW7/9/u5aqhyFM2pxx0N+tk1EGMmXfaOU55e0AAhv4+AT7Qet7JhUYFWFGXFcDpabe2N0BcLMZOq\nzOL2etAiUrej1TeeOsU5OxNcUdYrKtCKoqwYTk/orX0ReqNBpqvmQXs9aIB4eH5Hq0S2wIe//EP+\nz0OHl99gRVlFVKAVRVkxnDHoLT1R+qJBUrmim1CWqvKgoXZP6Kmk5XUfGJued/xiqW5RQ0VZc6hA\nK4qyYkQqQtxBoDzVKlmVxQ3QFwtxpiqU7Wz/zMmZilaUh87Ocekn/5kj48lls19RVhIVaEXZQNTq\n7+5Z91ERMSIyuFznd8agt/RG6Ik6Am15xKls5TxogNftHeCpsWkSnjD3lD1uncmXePHsnLv8lXMJ\n8kXDsQkVaGV9oAKtKBuLO5jf3x0R2Qm8Azi+nCd3sri39VlJYmB5xLlCiVyxNM+DfvPFgxRKhkde\nmXCXeedOPzU24752POuEZn0r6wQVaEXZQNTp7w7waeA2FuhM1w6GeyKIwN7BLvpsD3o6lSdd1cnK\n4Zrd/USDfh56qdyq1kksC/l9HBibcpe7oXIVaGWd0LDUp6Io6xsReRdw0hjzlIgstN2twK0Aw8PD\njI6ONjx2IpGo2M4Ywx++OcqxZx/jbNIaP37kyaeZPmp5zmNHX2a0cKziGBf3wbefOs7bescB+OEr\nOXf5v7xwktFRS6SfOWQnjz33IsPJhTO8q+3qJDrVNrWrdRKJxJL2V4FWlA2MiMSAX8cKby+IMeZ2\n4HaA/fv3m5GRkYbHHx0dpd5206kcH3voXrbuvoirLxmEBx7ktVdezsjV2yq2Oxw4wm9/83kuvOp1\n7ByI8VDieeLHjnPTay7gM/e/xP7r30RXOMB908/A4eMM79jNyMgli7ZrtelU29Su1lnqg4OGuBVl\nY3MhsBd4SkSOAjuAJ0Vky3KfuDsSRMQKWSfthhjV86AB3nKJlbP28MuWBz2VytEXC3H1zj6MgadP\nWNOtnKpkiYyGuJX1gQq0omxgjDHPGGM2G2P2GGP2ACeA1xpjziz3uf0+oTscYCaVcxtiVI9BA1w4\n1MXm7jCPHbGGzmdSefpiQS7f2gPAy+esMKJ3upairAdUoBVlA1Gnv/uq0RcLMZPOuy0l4zUEWkTY\nvSnGqRmrTeV02hLogbiVBT6esMaeZ22BnlMPWlkn6Bi0omwgFujv7qzfs0KmALjlPl0PukaIG2BL\nb5Rn7FD2VCrHq7b2EPD76I8FmUxmgbJA6zQrZb2gHrSiKKtGXyzIdCrvlvOs5UGDVXns9EwGY4wV\n4ranaG3qCjNhe9DuPGj1oJV1ggq0oiirRm80yGw6785drutB90TIFkpMpfJuiBtgIB5iIpmzhFs9\naGWdoQKtKMqq4YS4HQ86Fqwt0Ft7IwC8dHaOYsnQb1chG+wKMZHIks4XKdiNMlSglfWCCrSiKKtG\nXyzIVCrHV58YIxzwEfDX/kraYgv0C2es2tu90bIHPZnMud5zKOBTgVbWDUsSaBH5VRF5TkSeFZEv\ni0ikXYYpirL+edu+zVyzq5/BrjDvuXZn3e229kYBOHh6FsCt470pHmYqlWfSbkG5vS9KIlPAGG07\nqax9Fp3FLSLbgQ8Dlxlj0iJyJ/AerGL8iqIoDdm/Z4C7fuENDbcb6g7j9wkHbQ+6P+YkiVlCfXQ8\nBVhNOI6MJ8kWSm5rS0VZqyw1xB0AoiISAGLAqaWbpCiKUonfJ2zuDvPiGceDtgU6HgbgyLhVrGSb\n7WlrmFtZDyzagzbGnBSRP8ZqT5cG7jHG3FO9nbfA/q5duxZ7ug3Fno9/q+E2Rz91ywpYoiidwxZ7\nqhWUQ9xOsZLD41YP6O39tkBnCgx2hVfBSkVpH4v2oEWkH3gXVh3fbUBcRN5XvZ0x5nZjzH5jzP6h\noaHFW6ooyobGyeSGcpLYoB3iPuIIdF/rHvTv/9NBPv/wkXaZqShtYykh7rcDR4wx540xeeDvgcaD\nSYqiKItgS48lvl3hAEE723uT7SUfPm8J9NZFhLi/9cxpHjx0vvGGirLCLEWgjwPXiUhMrCayNwIH\n22OWoihKJY4H7Yw/A/RFg/jEqiLWHQ64nnUr1cQmEjnS+WJ7jVWUNrBogTbGPArcBTwJPGMf6/Y2\n2aUoilLBlhoC7fOJOw7dEw3SFbHSapr1oFO5Aul8kXROBVrpPJbULMMY85vAb7bJFkVRlLq4HnQ0\nVLF8IB5iPJGjNxp0+0nPNSnQTh1v9aCVTkQriSmKsiao5UFDeapVbzRId9hal2xSoMcTVics9aCV\nTkQFWlGUNcHm7ggi8wV6oMsJcQeIBH34fUIiU2AymePOx8cWPKbjQadyOm9a6TxUoBVFWROEAj5+\n+W0X8eNXbatYPmiPQfdGg4gIXeEAiWyBv3n0GLfd9TTn5jJ1jzlh95JeKMT9nRfOktEQuLIKqEAr\nirJm+M/v2MfrL9hUscyZauVkcHeFA8xlCm5jjVm7kYYxhmyhUmjHbQ86ky9RKs2v331iKsUH73ic\nu589s6BdX/vhCX7yz/9lEVekKPVRgVYUZU0z4PGgwRLoZLbAobO2QNtTrj77nZf5kU8/WCHETogb\nIFOY7yVPp5rrMf3ksWmePD5NvlhawpUoSiUq0IqirGkGu8rTrAC6IgEmUzm3eInjQb9yPsHRiRQH\nTky7+zohbsDtSe3FEeZGIe6plDOWraFwpX2oQCuKsqYZiM8PcT93coaC7Sk7HrTTM/r+g2fdfb0e\ndK1MbqfgSbawsGdcFmhNNlPahwq0oihrmsu39XDLlVt5/V5rbLorEiDpEdu5jCXMjkDf9/w5d50z\nzQpqJ4o5HnRDgU5ax1YPWmknKtCKoqxp4uEAf/Yzr3XnSXeFrPpLItb62XSlB/3i2TnOpyzBnUjm\n2NxteeC1POg5V6AXFt5px4POqkAr7UMFWlGUdYVT7nPvYJygX5h1PegC19sZ4AfOFymVDJPJHDvs\nFpW1vF+n4Ek23yjEbZ0jqSFupY2oQCuKsq7oClsCfemWbrojQeYyeYwxzKbzXLWzlwuG4hw4V2Am\nnadYMuwciAG1E8GaGYPO5ItueFzHoJV2ogKtKMq6otv2oC8Z7qYnEmA2XSCTL5ErluiNBrlh32Ze\nnCxxYioNsKAH7Y5BL5DF7UzFAkhqiFtpIyrQiqKsK+K2B71vuJueaJDZTN4df+6NBnnjxYMUDHz7\nOav4yM5+y4OulSQ214QHPZksZ4KrB620ExVoRVHWFbsHYoT8Pq7a2UdPJMhcplAh0K/bM4Bf4OtP\nnQRghyPQNcQ12USSmJMgZm2vHrTSPlSgFUVZV1x/4Sae/I2b2N4XpTsSYDZd6UHHwwEu6vMxNmmF\nuHcOWCHuxU6zmvKEuJ1j5Islrd+tLBkVaEVR1hVOwwyAnsj8EDfAZZv89rawtbf+GPRcE1ncUxUe\ntLX9b3z9Od5z+yNLvRRlg6MCrSjKuqUnGpgX4ga43BbogViIUMBHKOCr7UHbU7SaCXFHg35X5I+O\nJzkwNs3YZKp9F6NsOFSgFUVZt3RHgqRyRSbtmtuOQO/t9dEVDrDJruMdC/lrl/p0a3EvHOKOhfz0\nxYKuB+3Mvb7PU1ZUUVpFBVpRlHVLjz3lyplS1R2xBNrvE37qmh1u4ZJosLZAO0lfC3nQU6kc/bEQ\nsVDZg3bLiqpAK0sgsNoGKIqiLBdOh6uxyRTdkQB+n7jrfusnLndfR0N+UlUh7lLJNJcklszRFwvi\n94lbSWw2nUcEHj08yWwmT4/9YKAoraAetKIo6xbHYx6bSrvh7VpEg34yVR60t2xnoyzugbjtQWet\nEqJzWausaKFkePDQ+SVehbJRWZJAi0ifiNwlIi+IyEERub5dhimKoiyVcog7taBAe8PTDo733BUO\nNKgklqMvFiIeCpDMFZjLFjAGRvYNMRAP8Z2D5+ruqygLsVQP+jPA3caYS4GrgYNLN0lRlOVCRL4g\nIudE5FnPsj+yH7KfFpGviUjfatrYTpwQdyZfWlCgI0H/vCxuJ+FrU1eooQfdHwsSCwdI54rM2uPP\nfbEQl27pZmxKM7mVxbFogRaRXuAtwOcBjDE5Y8x0uwxTFGVZuAO4uWrZvcAVxpirgEPAJ1baqOXC\nqcsNNPSgq5PEnDKfm+IhCiVDoThfpIslw2wmb3vQfpK5yild0RrCryjNspQksb3AeeAvReRq4Ang\nI8aYpHcjEbkVuBVg165dSzid0uns+fi3Gm5z9FO3rIAlSj2MMQ+KyJ6qZfd43j4C/NRK2rSc9HhE\nudEYdLUK/uF/AAAgAElEQVSQOiHugbjVLzpbKBHwV/o0M+k8xkB/zOqalcqWPejeaJBInelbitIM\nSxHoAPBa4JeNMY+KyGeAjwOf9G5kjLkduB1g//79ZgnnUxRl+fkg8Le1VngftoeHhxkdHW14sEQi\n0dR2y0XJGAQwwMz4GUZHJ2vaNT2eZTpRrFj22BlLoHNzEwB854GH6A6Vs8ABTiUsr/rMsVcYT5RI\nZAv8y+MHADj07AFmJgpMzxVb+gxW+zOrh9rVOolEYkn7L0WgTwAnjDGP2u/vwhJoRVHWICLyX4EC\n8KVa66sftkdGRhoec3R0lGa2W066Hvg2c5kCV1xyASMjF9W068G553n8/FjFsvOPj8GBp7ny4t08\neOIV9r/+OrcsqMPjRyfh4e/zhv1X89ypGf7x8Its3nURHHiOG9/yBl4afYXnp0+39Bl0wmdWC7Wr\ndZb64LBogTbGnBGRMRHZZ4x5EbgReH5J1iiKsiqIyAeAHwNuNMasq0iX09GqcRZ3AWMMIpaX7IS4\nB7vsELenmpgxhjOzGbdRRn8sSDxkfZ2emrGKovRGg0SrQtwfu+tpxqZS7N/dz/uu383m7kgbr1SZ\nSGQ5PJ7k2j0Dq21KW1hqoZJfBr4kIiHgMPCzSzdJUZSVRERuBm4D3mqMWXcpxz3RICenG8yDDvkp\nGWucORK06nQnMs4YtFUO1JvJ/d0Xz/HBOx53xdupJAZwZiaD3yfEQ343O9wR/n965jQA33tlAgN8\n9B372n69G5m/+t5R/veDh3nhd252H7TWMkuaZmWMOWCM2W+MucoY825jzFS7DFMUpf2IyJeB7wP7\nROSEiHwI+CzQDdwrIgdE5HOramSbceZCN0oSAypaRCayBcIBn9sZy7vu5HQGAL8PusMBBrvCxO3t\nTs9k6IkEEBH3uNlCCWMMqXyR979hD4NdIcYT5S5YSnuYzRTIFkoL1k5fS2ipT0XZQBhj3ltj8edX\n3JAVxKkm1ijEDVbLyb6YtWwuW6A7EiAcKIusQ9quMnbPr7wV8VkeuHOM0zNlbz0atHwgR9yLJUM0\n5KcvFmImrQLdbpzPOZEtELX/HmsZLfWpKMq6pifahAdtf5l7p1olswW6wgHCtsh6G2Y4Vce6IgG3\nznbMHoM+O5N1p3c54fJ0vuiORcdCfvqiQaaS+aVfnFKBI9BOkZm1jgq0oijrmp4mPGgnFO1N6Epk\nCsTDAcIBW6DzXg+6SCjgq2i+4XjQuWK5apkr/Lmi24wjZnvQUyn1oNuNE9pOqEAriqJ0Ppt7wkSC\nvoqqYtXU8qDnbA86Epwf4k7mCsSrQqjOGDSUHwoqPWhLNGKhAP2xoFtxTGkfmcL68qB1DFpRlHXN\nv79+DzdeOjyvCpgX7xi0QyJTYFtfpOxBV4W4nZC2g1ewe9wx6HLyWbFk3HP1xYLqQS8Dbog7tz4E\nWj1oRVHWNV3hAPu2dC+4TaRGiDuZs8egA47IVoa4q5OQYuH5db/LIe4SyWzRXdYXC5HJlyoyw5Wl\n4/yNnDrqax0VaEVRNjyON5zOl7/Y541Bz/OgKwXa8ZahnJgW9Ya4894QtzW3ejo1P8xdLBnuff4s\nK1kvplAscW4us2LnWy7KSWLr48FHBVpRlA1POUms7CXPZQt0RbxZ3FUedLBSoP2+8rzn3hpZ3Clv\nFnfMWl8rzP3MeJGf++vHefbkbFuurRm++Mgx3vSp7/LEsbVdykKzuBVFUdYZUXcM2m6QUSiRK5To\nDgcI+edncafyhXkeNJTHsqtD3Jlc8wKdtrXl5HR6ydfVLOcTWXLFEr/wxSc4N7t2Pen1lsW9rpPE\nmml/uFZpV2vHZj8jbROprGeqK4k54d6BeJiA30fAJ/ND3OH5X5+xsJ+JpCeL2w6Pe7PDvSHumRoh\n7lzRVNiwEqRzJUJ+H3OZAj97x2P88U9fzau29qzY+dvFesviVg9aUZQNTyhgibDj5R4dt0qS7xm0\nyopFgv6KEHcqWyQWnO9BOw0z5nnQ+aKbWRwL+V2Bnqop0NbvsyvoyabzRXpjQf70va/h5HSaW/7k\nIT5976EVO3+70CxuRVGUdUjUbmwBcGQiCcCeTXEAwgFfRcZ1KtdciDsSqKwkJmIda6EQd9aejnV2\nNtuW62qGTN4aU3/7ZcM88Gtv46bLhvnsd19mIrFyNiwVY4wnxK1JYoqiKOuGeDjArD0AfGw8STjg\nY0uP1Q4yHPBVJonli0RD80PcTrESZx60zyeEAz43SSwesppoRIJ+IkFfzWIl+dXwoD1Jb72xIL96\n0yUUS4Z/evbMitlQzfGJlJsT0AwVhWQ0xK0oirJ+uGhzFy+etTKnj04k2bMpjs8u5Rn2hLjzxRL5\nolnQg+7xVC2Lhvxukph37nR/LMRUsoYHvUoh7ojHtku39HDJcBffOHByxWzwUiwZbvnTh/jCw0ea\n3sebxLdeksRUoBVFUYDLt/dw6EyCXKHEkfGkO/4Mtgdtu7bebOxq4qEA8ZC/omqZEzqvDov3RoO1\nx6BXLcRdKQfvevV2Hjs6taLZ5A7n5jLMZQqcaeEhJVOorAK3HlCBVhRFAa7Y1kuuWOLFM3OMTabd\n8WeoDHGXu1LND3G/dnc/b903VLHMEuiS5UEHKz3oWi0nnSSxmXR+xSqNOWPQXn78qm0A/ONTp1bE\nBi+n7IeCVoTW+1k1kyRmjOEXv/QkDxw637qBK4QKtKIoCnD5Nmta0b3PnyFXLLFn0CvQfneaVcqT\njV3N+67bzZ//zDUVyyJBP+mclSTm3ac/XseDLpYriJ1bIS86nS+6RVUcdm2KcfXOPu5ehXHoE1OW\nQLdSstNJEOuLBeeNQRdLxq2F7pDIFvjWM6f53svjS7R2+VCBVhRFwcrYjof8fPPp0+57h3DQ5wqA\nE+KursVdj0jQygBP5QoVHa96oyGma2Rxe8qBc3aF5kKna3jQANfu7ufg6VkKxVKNvZYPJ6zemkBb\nH9ymeGjeGPSHv/JDPnrngYpl4wnrs5/r4PFqFWhFURSsjOvLt/VyeNyaYrV3ngddKdC1POhaREN+\nN4u7MsQdZDqVn1dzO1cydNtCvlKJYulcqSJJzOHy7T1kCyVeOZ9ccP9UrtDW2uEnHQ+6BfF0Bbor\nTCZfqnioeOnsHC+cmavYftyeQtbJ49Uq0IqiKDaXb7fC3JGgj+GesLs8HPQ1FeKuRdQOcVc32OiP\nhSiUzDxvL1eE3XaC2kolitUagwa4bGsvAM+fnqm770w6zzW/cx/3HzzXNntOuR508z2zM/YD1GCX\nVQQm6QlFTKXyrsfs4MzxbuUcK82SBVpE/CLyQxH5ZjsMUhRFWS2u2GYJ0p5NcUTEXW5lcVcmiUWD\nzVVKjgT9ZArONKvyPk6xkuqOVrkibOmJEAr4Vqwudj2BvnAoTjjg47kFGnecn8uSzhd54Uz7mnss\nJsTt/F0Gu6wHK2cc2hjDdCrHZDJLyTMOfd4W7E6ektUOD/ojwME2HEdRFGVVuWJ7WaC91Apxx8PN\ne9CZXJF01TSrvjotJ3NFQzQUYLgn3FSI++xshvNzi/e088UShZKpOaYe8Pu4dEs3z52qL76OMJ5b\ngg1ejDFuiDuRbT507kQ4NsUrBTqZK5IvGkoGpj2FYcbnHA96nQq0iOwAbgH+oj3mKIqirB4XDsUZ\niIe4YntlowhrmpUd4s63liQWDflJ5Yuk8kXiocoxaIDziQz3HzxL3h4zzZUgGvQx3B1pGOLOFUr8\n1Oe+x8f+7unmLrAGTnlTp+91NZdt6+W5UzN1hdKZ0tSujPPZdIFkrshgV5hiyVQ0GqlmPJHl6y/n\nKJWMZwzaevBxPGNvMRhv6dKJZLZiu05kqR70/wRuA1Y2xU9RFGUZCPh93Pef38qtb7mwYrm3WUba\nHYNuLsQdDfqZTecxhqoQtyUkH/nKAT70V4+705myRUMk6Ge4J9Iwi/vOx8cYm0xzdHzhJK6FyDTI\nSr9sWw+zmQITmdoC7YzJt6v71olpq1HJpVu6gYU93PsPnuVrL+c5PJ5ws+wHqwTaW07VOw49Ptf5\nIe5Ft5sUkR8DzhljnhCRkQW2uxW4FWDXrl2LPV3H04mtLTvRppWmXW05lY3DQDw0b1k44CNXKFEq\nGZJZZwy62WlWfpyhT2+Ie7ArhE/AGel2Gmfki9axN/eEeeBQfa80ky/yp995CbDGbI0xFePmzeJ4\nqPWux5kffmy2th/mfB7tCnE74e1Lt3Tz8MvjzGUKDNfpfOmE171FXTZVjUF7G5KM1/Cg5zKFRX92\ny81SPOg3Aj8hIkeBrwA3iMgXqzcyxtxujNlvjNk/NDRUvVpRFKXjCdtlMHPFEul8kXDAh9/X3Be6\ntwBItGoM+s7/eD13/8pbAEsoSiVDrmTts6UnQiJb4De//mzNmtRfevQ4Z2ez/Mjlw2QLpZpFT5qh\nkUC/aksPPoHjdQTaOwbdjqlWToLYpXY/6oWyrNO212wJtPV6U9zxoC27vJ+LN8TteNPFUrkLVqex\naIE2xnzCGLPDGLMHeA/wHWPM+9pmmaIoSocQtttGZvOluq0m6+GtcR2vCovv3zPA1t4IQb+QyBbc\nMHok6Oea3f1siof4ymNj/PY3n59X1OSrj49xze5+/tVrdgDlqUmNqA6HO+JUax40WA8VewbjnEjU\n8aDtEHeuUGK2DQlXp6bTRII+dm+yppotFOJ2vObZdIFMoUjQL24nMceD9n5uE0lviDtLyB53n8u2\nMJ0rX2z6s14qOg9aURSlAU4CVbbgzGdufnTQ6zXXEnYRoSscYC6T93izPvbvGeCJT97EH/301QAV\nmdpnZjK8cGaOmy4bZluf1RKzGdEYffEcI388yhPHJt1l5Wlj9R86NsVDpPL1xqDLSVzn2zAOfXI6\nzba+KN12R7CFxogdgXZC3JGAn65w5X5TSUt8e6NB12vO5IvMZQvsHrAeAlopVvJ73zrIj/7JQytS\nXa0tAm2MGTXG/Fg7jqUoitJplAW6NK+mdiPqhbi9dEeCJDIFV3C82w3ZY6rnPeHZB+0GDyP7htjW\nFwWaE+ivPn4CgKfGyoVHnHNW1+L2EgsFyNRJpvbWvW5HJvfJqTTb+6J0RyxPeOEQt1egS4SDfnf4\nwTsG3R0OsKUn4oa4HU96tz2drtlEsXSuyNd+eJLpVJ5DZxOLu8AWUA9aURSlAWFbvMoedCsh7oU9\naICucIBEtuAKjlcsh7qtMVWvB/3AofMM94TZN9zNpniIUMDH6ZlK7/XRwxMV4d2ZdJ57D54F4NDZ\nctnLRmPQjt3ZYmMPeqmJYqWS4ehEih39MdeDXijE7Xj/s+k82XyRSNDnRiSSnizuvniQwe6QmyTm\nzIHeO9g4jO7ln5897Yr5gbHpRVxha6hAK4qiNCBie9CZvOVBNzsHGhqHuAG6IgFmMwVXcCoEussK\nYTsCXSiWeOil87z1kiFEBBFha2+kom/zsydn+Le3P1KRXPZPz5wmVygx2BXmRa9ANxHijoUC1HMy\nU7mCG1Z2plotNlnslfMJZtJ5XrOrzx2vX1CgvR50odyRy3rgcZLEcvTHQmyKh13P2cngdjqWNSvQ\nX338BLsGYvTHgjylAq0oirL6eD3oZK7Q2hh0hQdde7/ucIBEpuAWQ/Hu0xMNEPL73BD3gbFpZjMF\nRvZtdrfZ1hut8KA/fe8hAF46Vw7Dfu3Jk1wwFOeWK7dw6MycK6Ku1x6qLwcLedDJXJHN3WGiQT/n\nZrM8e3KGSz95N4fPtx4CfuzoFADX7hnA73PG5psdgy4RsRPy4mG/J8Sdpy8WYlNXiAl7DNqZA723\nhRD38YkU3z88wU9fs4Ord/apB60oitIJuGPQi/CgI02EuLsjdog7ZyUeeY8vIgx1h10P+oFD5/H7\nhDdeNOhus7Uv4o5BP3l8ivtfOEfAJ7xii+Sp6TQ/ODrJT75mO5ds6SaZK7oed6bpEHftdalsgVjY\nz1B3mHNzWe59/izZQmnB8qD1eOzoJINdYfbYGdzdkUBTY9CzmTzpnJUkBhC3hwzAyuLuiwYZ7AqT\nyFrj/M7Dzm7bg0400TDjm8+cAuAnr9nBq3f2cejc3LIXOVGBVhRFaYAjXolsgVSusmRnw31DjZPE\nuiKVY9DVYjnYHXYzkJ87NcvFm7votacTAWzvi3J2NkOhWOLT9x5iIB7iPa/bydGJFMWSccOxb754\niH3DVoUuZxy62SSxfMmaM1yNk9W+uTvMubkMjxyeAKgIuTfLY0cnuXZPv1s0pMsjtLUoFyop1Ahx\nl0t99seC7vzoiWSOiUSOeMg/r+rYQjx+dIqLNnexvS/Kq3f2YQw8c6J+l692oAKtKIrSgAuG4ojA\nwdNz9jzo5kPcjmj4fULIX/srtyscrJhmFQlWbjfUVfagj00k5zXz2NobpWTgkcOTPPTSOB96016u\n3N5LrlDi5FSa507N4vcJ+7Z0c7Et0C+esbzrdL5IwCcE69gGZc/fKevpxXlg2dwTZmwyzQ/thwGn\nIlg9PnPfS/zS3zzpvj89k+bEVJpr9wy4yywPeqExaCviMFsd4g5ZSWKFojU3uy8Wcrtcjc9lGU9k\nGewOEw74CQV8DcegjTE8eXyK1+7qA+DqHdbv5Q5zq0AriqI0oDsS5OLNXRwYmyKdbzFJzBboWNBf\nt5xkdyRAvmjcutHV3uxQd4jzc1mKJcPYZNot4uHgzIX+k/tfwu8TfvqaHVww1AVYiVfPnZrhoqEu\nIkE/vdEgW3sjrgedzpUali2NhR2Bnh/nTuYKxMIBNndbiWq5QgmR+R50rlBibDLlvn/y+BSPHinP\nx/aOP5c/l+CCIW7vGHQ2X3RzBeL22LVTOKU/FnSbaEwks0wks65H3RMJMNfAgz48nmQ6leea3f3W\n8eIh9myKcWBsasH9looKtKJsIETkCyJyTkSe9SwbEJF7ReQl+3f/atrYqbx6Zx9PHp8mXzTEmqzD\nDR6BXqA9pTOlyPGSqwVzqCvMZDLLqek0uWLJnb/r4MyF/sHRSUYuGWJzT4QLKwR6lsu2lQtaXzLc\nzYtnbIHOF+tWEXMoe9DzBTqVLRILWmPQAD6B1+0ZmOdB/+qdB7j5fz7o9mSeSeeZTObc948dmSQe\n8vOqrd3uPl0NxNMJcSeyBZK5gjsGfdHmLs7MZtwx+P64x4NO5Bify7nvu+wEvYV48pglxK/dVb41\nXr2zr2I++XKgAq0oG4s7gJurln0cuN8YczFwv/1eqeLqnX2uh9uKB+0kmC0UFnemKTmVuKqPP9Qd\npmQsrxOY50Fv7Y24r396v1X6cyAeoi8W5LGjk5yby7pNLwD2benm5fMJCsUSmXyxsQdt256sIZap\nXIF4OOAK9OXbennV1h63gQfAN58+xbeePk0yV3TDybOZPMVSOWpwYGyaV+/qI+AJtfc0DHEX3WYj\n44mcG+J+3V7LC7/nOatDmJPFDfCDI5Mcm0yyxf7MnPH/hXjy+BQ9kYD70APwCyMX8YUPXNuW+uP1\nWHQ3q42EdoVS1gvGmAdFZE/V4ncBI/brvwJGgY+tmFFrhFfv7HNftzIG7fMJkaBvQREsC7TlQTue\noIMjfo8frS3Q3ZEg3ZEAQb+PGy4ddpdfMBjnuy9aVce8HvTFm7vIFUocn0xZWekNBdpaX92b2Rjj\nFm7ZbNt43QUDDNuNPmbTBXLFEp/8h2cJ+X3kiiWm0zl6Y0FmbWGeSOboj4c4PZPmiu3DFcd3SqDW\nI50v0hMWZrKGYsm413Hl9l4iQR93OwIdDRILBYgG/dz1xAkGu0L83JsvcM/R2IOe5jW7+vF5GqTs\n29K9wB7tQQVaUZRhY8xp+/UZYLjWRt7WscPDw4yOjjY8cCKRaGq7lWYxdhVLhpAPciU49sohRtOH\nm943QIl8uv45X5mwhO/wqXECYnjwwQcq1o9NWetHnxsjIHDowKO8XDWe/brNsCUG33v4QXdZvJgl\nZzfgmDr8DKNj1j7j9vnufvBRTp3Nk8+bBT+Pl+zzf/+xJ0keLctGvmQolAxnTx5nwpwi4IPB7Gmm\nTlpe5T/c9yBPnS8ylcrzby4JcuehEvc//Ah7e3xM2UVD7n/4UY71+ZhI5EhNnGF0tDwuPX46RyZf\n4r7vfJdAVfewkjHkCiWGI4aZrLXu7KkTjI6eA2BvNxyctMLsLz7zJFOv+OgKlCiW4Bev9PHK0z/g\nFSCbyDCern/9qbzh0NkUl3VnWv6fSSSWVg5UBVpRFBdjjBGRmjE7Y8ztwO0A+/fvNyMjIw2PNzo6\nSjPbrTSLtevqQ9/jsaNTXHP1FYxcvqXp/Xq+fz9bN3czMvK6musHT87wB489TFbChAPpebbtnUjy\ne4+OciJRYu9gnBveNt/2WpdzkFd46OQLbO+LcstNb3OX7zqf4A8ee4CtF+wjOj5GFBgZub6u/cOn\nZ+HRh7j40ssZuXKru3wqmYN77uXyfRfzr9+0l3e9o0TA7+PpE9P82YF/YcuFl3Pf+WPsG87yb264\ngjsPfZ+LXnUV1+4ZoPjtuy1bLr6MK3b3Y+65n2uv3MfIdbvd4x8JHuFrLz/PNa9/I/1VfbpTuQJ8\n+9sMxgKMJawHiH0X7WVk5GIAniq8xMH7rIItN9/wZnoiQT41fJa+WJBrdpcT0b5+9gDnj066n3k6\nV+RPvvMSv/S2i4iHAzx46DyGH/Cv3/pa3nTxIK2w1IdTHYNWFOWsiGwFsH+fW2V7OhZnek0rIW6w\nsoq7w82MQWcJ1egz7SQ0GcO8KVYLccGQta13/Blgc481/np2NmuNQS8ySSxlh7zjdgKcM3683U5a\nG5tM8eSxKa7d2+/O255O591xZ4DxZM5tsuGE8h2qO1N5cRLE+sLlz8ub/e6MQwd84n72N75quEKc\noVwkxuGhl87zv0Zf4fuvWPO5j9jtOVcipF2NCrSiKN8A3m+/fj/w9VW0paNxptn0xYINtqzkd959\nBR++8eK6650s7lyxRC2tjIcDbnGUXQOx+RvUwUlqunxbb8XyrnCArnCAMzMZa9pYk0li1fOgU7aw\nVT+wDMRDRII+7jt4lmSuyLV7Bui1P7OZdJ5Zz7jyRCLL+YSVHLe5SqCdjlazNcahnfHw/khZoMOe\n63jNrj5Cfh99sWDd6W1QHoN2kr2c8qjOQ4Tzu9W/eTtQgVaUDYSIfBn4PrBPRE6IyIeATwE3ichL\nwNvt90oNfuTyLfz1B183zyNtxHUXbFrQA+uKlAUu5K8tJo53uWdTKwId5+PvvJR/e+3Oees291iV\nv5oT6NoedDJX6UE7iAg7+mM8ctgaT752z4DrQc+kchUe9KTHg3Y8e4eeqo5WhWKJ//PgYTL5ojsH\nut/rQQfKkhYJ+rl6Zy8DVaHxaroiAQolQ9Yeq3emn3kFOh7yL1jIZbnQMWhF2UAYY95bZ9WNK2rI\nGsXnE95yyVDbjxsO+N0s53o9K4a6wxydSLn1o5tBRPj5t15Yc91wd4Szs1nSuVLDedCOgCerQ9y2\nBx0NzpeS7X1RXj6XYHtf1J2nHQ36mU7lmUl5Peic26bSKb3p4Dy4OFnWB8am+b1/OsiuTTG29VrH\n7AoJQb+QL5p5BV5+991XksguXGfbCX/PZQpEgn63gMu0LdDTqXxFWdWVRD1oRVGUDsARo3pa6XjQ\nu1sIcS/Elt4IZ2czZJvwoH0+IeSHdHWIu44HDbC93xLQa/eUi3v0xYIVIe6h7jDjiSzn57L0xYKE\nq6aXOSHuOVtkp2xhn0mVy6KGfOIKaLVA79vSPW/MuZou10vPUyiWOHzeGnOe9XjQvbGFvfDlQgVa\nURSlA+h2BbpOiLsrjE9gR397BHpzT5hzs1lS+eK82t+1CPvne9DJXO0xaCgnil27tyyQvdFgRZLY\n3sG4FeKey8wbf4byZ+KEuL1hZ1eg/dDjCnTrktYdtvZNZAscm0yRK5YqzjWbztMbXZ1gs4a4FUVR\nOgAnY7meB/2BN+5l/54BQoH2+FXD3RFXjBp50AARv7iZ0w6OB12rjealW7rx+4Q3XFiemtQbtT3o\ntCW4ezfFeflcgq5IYF4GN0BPxBm3rkzYmknnXVtCfup60M3gDaM7LTtDfh/TKWue9nQ6xwWDXXX3\nX05UoBVFUToAV6BrTLMCy9vc28L4cyOGPQlZzQhb2D+/1KfzPl7Dg77h0s38y8ducEtqghXiPjqe\nchOvhnvCTKVyRAI+Xn/BpnnHCAV8xEP+cmjbI9BOkljYL66QV1dgawbnc5/LFjh01srgvmJ7T8W5\ndAxaURRlA+OMt7ZQ5ntJDPeUPdZmaouH/TKv1KfjxdbaX0QqxBmgLxpiOp1zRW9TVxhj4NRM7RA3\nWHW0HW92tk6Iu3cpIW6PB33o7Bw7B6Js7Y1WCvQqTLGCJQi0iOwUke+KyPMi8pyIfKSdhimKomwk\nuhskibUbrwfdTIg7HKjhQeeKhPy+psPuvZ4ksZ5ouQUkzC9S4tAfDzJlC7QjmtPeEPcCSWLN0BcL\n4RP452dP88KZOfYNd9Njh+Kt6VylNelBF4CPGmMuA64DflFELmuPWYqiKBuL8hh0/aIa7cQriE0J\ntF/mVxLLFVrq7NUbDZLJlzg3l6UnGqyYo1xXoGOhmiHuyiQx67NbjED3RoP811su476D53j5XIKL\nh7vdbHPnfGtOoI0xp40xT9qv54CDwPZ2GaYoirKRcJKVwis08BgJ+t3qWM2OQTuimMkXKZYMyWzR\nrXDWDM75xiZT9ESCbglTgM3dkTr7lEPc3lB3Jl9EBII+2N4XIxL0uQ85rfKhN+3lYzdfCsAV23rp\njQbJFw2nZ6wKZ6sl0G1JErPb170GeLTGOrcDzq5du9pxOkUBtA2osr5wQtzBFfKgwcrknk7lmxRo\nIZktYozhxv/+AO+5difpfIFYC6LoCN1kMkdv0x50sG6SWDToR0T46f07ePPFgy1589X8wsiF3HTZ\nZvYOdnHn42MAHJ9MAatT5hPakCQmIl3A3wG/YoyZrV5vjLndGLPfGLN/aKj9FXgURVHWA05Fqxo1\nP4NQdyUAAApPSURBVJaNYTuJqxlhi9iFSmbSeU5Op7n/hXOte9DRsiD3RoP0x0I4ZbI399RPEpvN\n5CmWDDPp8nzolKePddDvY2cbCrhctNmaGtZnP0gcn0i6tq4GSxJoEQliifOXjDF/3x6TFEVRNh5d\nDQqVLAfDttfazBh0KCCk8kVOTFlzhZ89OcP5uWxLXqvXE+2JBvD7hIGY1VijXrev/lgQY+wmG+k8\nQb9QLBnGE9lFjTk3gyPIxyZsDzq6xiqJidUe5PPAQWPM/2ifSYqiKBsPp6JVvVrcy4GTyd1coRKr\n3eUr5625woWS4eCZ2ZpzoOvh9USd1wPxEEPd4bodp/rtMpunZ9LkiiW3ktqZ2dYeDlrBqUzmhLjX\nogf9RuDfATeIyAH750fbZJeiKMqGwi1XuYLlo3ZvihHwiZsFvRBh27N/2W7HCJZgtzQG7fWg7Xnf\nO/qj7B6oX4DF8bodb9Zpt3l2JrOoec/N4Jzz+GQKkXJ+wEqz6LMaYx4GVi4WoyiKso65Znc/v/vu\nK9iSOrxi53z3a7Zz1Y4++ppoBuGMjb90NkEk6OOS4W6ePjHT0hh0d9gKaxdLxvVK/+CnrgJTfx/H\ngz4ybo0HOwJ9PpFl50AUKNbbddE4tp2eydAbDeKrU91tudFKYoqiKB2A3ye877rdBFZQDIJ+34J9\nqr2EA5ZdL52bY1tflNfbTTBaCTOLlIuKON705u7IvD7QXhyBPmYnbO22+2EXS/PbS7aLLvtBAlYv\nvA0q0IqiKEoTOB700YkU2/uivH6vVTu7lTFoKAueE+JuRF886J4XqMjWbmbsfDFUPEioQCuKoiid\nTMQegy6WDNv7oly7Z4BI0Fd3elQ9WhW+7nCAgE/medDQmvfeKo59qzUHGrSblaIoitIE3vnZ2/ui\n9MaC3P/REYa6WhNoR/CaSUwDy5vtiwU5O5sFYGtPlIBPKJTMsnnQln3Bit+rgXrQiqIoSkPCnvnZ\n2/uj1u++aMv9qXujQYJ+aUlcnSQ2J6N6Kc0xmj6n40GrQCuKoiidjHc21ba+6KKPc8lwNxdt7q47\n77kW/bbX3R0O4POJm2C2EiHu1RyD1hC3oiiK0pAKD3oJAv2fRi7k5996YUv7OB60I8yOaC5niLsv\ntvoCrR60oiiK0hBnDNonsKW3/rSoRoiIO4WpWfqrxHIlBLoTksRUoBVFUZSGBHxCyO9juCdC0L+y\n0uHMhXZqYjvjwhENcSvKyrZ27MQ2ku206einbmnbsRRlJYmG/EsKby8WN8Rdy4POLM85y+danUYZ\noB60oiiK0iQD8RC7N9Wvm71c9McqpzyVs7iXT8IuGOoi4BO7nOjqoB60oiiK0hS3/7trKhperBTV\nHnTPCoxBX7O7nwO/+Q66WmgG0m5UoBVFUZSmuHi4ubrd7aY6ScwR7GjQT24Zz7ua4gwa4lYURVE6\nnP54pQe9qct6391kPe+1igq0oiiK0tFcONTFfxq5kLdfthmAN180yP/6mddyxfaeVbZsedEQt6Io\nitLR+H3CbTdf6r4P+H2888qtq2jRyqAetKIoiqJ0ICrQiqIoitKBqEAriqIoSgeiAq0oiqIoHYgK\ntKIoiqJ0IEsSaBG5WUReFJGXReTj7TJKUZSVR0R+VUSeE5FnReTLIrL4lkWKoiyZRQu0iPiBPwPe\nCVwGvFdELmuXYYqirBwish34MLDfGHMF4Afes7pWKcrGZike9OuAl40xh40xOeArwLvaY5aiKKtA\nAIiKSACIAadW2R5F2dAspVDJdmDM8/4E8PrqjUTkVuBW+21CRF5scNxBYHwJdnU6G/r65A9W0JIm\nadGmJf/9mjzf7qWco1WMMSdF5I+B40AauMcYc493m0Xcy9C5/++dahd0rm1qV+sMsoR7edkriRlj\nbgdub3Z7EXncGLN/GU1aVfT61jbr9fpEpB8rArYXmAa+KiLvM8Z80dmm1XvZPm5Hfl6dahd0rm1q\nV+vYtu1Z7P5LCXGfBHZ63u+wlymKsvZ4O3DEGHPeGJMH/h54wyrbpCgbmqUI9GPAxSKyV0RCWAkl\n32iPWYqirDDHgetEJCYiAtwIHFxlmxRlQ7PoELcxpiAivwR8Gyvj8wvGmOfaYFNLIbQ1iF7f2mZd\nXp8x5lERuQt4EigAP6Q919qpn1en2gWda5va1TpLsk2MMe0yRFEURVGUNqGVxBRFURSlA1GBVhRF\nUZQOpKMEer2VDhWRL4jIORF51rNsQETuFZGX7N/9q2njYhGRnSLyXRF53i4P+RF7+Xq5voiI/EBE\nnrKv7/+zl6+L61tuOuVeXgv/pyLiF5Efisg3O8U2EekTkbtE5AUROSgi13eCXbZt80rSroZtrX6/\ni8gn7PvhRRH5kWbO0TECvU5Lh94B3Fy17OPA/caYi4H77fdrkQLwUWPMZcB1wC/af6/1cn1Z4AZj\nzNXAq4GbReQ61s/1LRsddi+vhf/Tj1CZMd8Jtn0GuNsYcylwtW3fqtu1QEna1bDtDpr8frf/594D\nXG7v8+f2fbIwxpiO+AGuB77tef8J4BOrbVcbrmsP8Kzn/YvAVvv1VuDF1baxTdf5deCm9Xh9WGUv\nn8SqlLfurm8ZPq+OvZc77f8Uq37E/cANwDftZatqG9ALHMFOIvYsX/XPjHIFywGsWUjfBN6xWrY1\n+/1efQ9gzX66vtHxO8aDpnbp0O2rZMtyMmyMOW2/PgMMr6Yx7UBE9gCvAR5lHV2fHXo8AJwD7jXG\nrKvrW0Y68l7u0P/T/wncBpQ8y1bbtr3AeeAv7dD7X4hIvAPswhhzEnBK0p4GZoxVknbVbbOpZ8ei\n7olOEugNh7Eepdb0PDcR6QL+DvgVY8ysd91avz5jTNEY82osL+d1InJF1fo1fX0biU78PxWRHwPO\nGWOeqLfNKtkWAF4L/C9jzGuAJFUh41X8zLwlabcBcRF5XyfYVk077Ogkgd4opUPPishWAPv3uVW2\nZ9GISBDrS+9Lxpi/txevm+tzMMZMA9/FGjtad9e3DHTUvdzB/6dvBH5CRI5idQO8QUS+2AG2nQBO\n2BEjgLuwBHu17YL6JWk7wTYWsGNR90QnCfRGKR36DeD99uv3Y42JrTlERIDP8/+3d/coDQRhHMaf\nqYKdH9jZWIidJ7AQrMwJUth5CgkIXsBLWNiKdfAAYiEh+ImVjXeweC1mQBuNimbG5flBIGwg+b/L\nTt5lZzOBm4g4evdSV+pbTinNl+dz5HnLWzpS3x9rZiy3fJxGxH5ErET+M4UBcB4Ru7WzRcQz8JRS\nWi+btoHr2rmKj5akbSEbn+Q4AwYppV5KaRVYAy6mvtusJ/mnTLj3gXvgERjWzvML9ZyQ50leyGel\ne8AS+aaQB2AELNbO+cPaNsmXb8bAVXn0O1TfBnm5yzEwAQ7K9k7UN4P918RY/i/HKbDF201i1bOR\nf7lwWfbbKbDQQq6S7ZB8sjwBjoFejWzf/X4HhmU83AE7X/kMl/qUJKlBLV3iliRJhQ1akqQG2aAl\nSWqQDVqSpAbZoCVJapANWpKkBtmgJUlq0CuMaWRTSDAYHwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2f2c182fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 48%|████▊     | 48500/100000 [3:54:46<18:04:47,  1.26s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "llh=0.581, mean score=9.929\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 49%|████▊     | 48518/100000 [3:54:51<4:08:25,  3.45it/s] "
     ]
    }
   ],
   "source": [
    "from IPython.display import clear_output\n",
    "from tqdm import tqdm,trange #or use tqdm_notebook,tnrange\n",
    "\n",
    "loss_history=[]\n",
    "editdist_history = []\n",
    "\n",
    "for i in trange(100000):\n",
    "    loss_history.append(\n",
    "            llh_trainer.train_step(*sample_batch(train_words,word_to_translation,32)))\n",
    "    \n",
    "    if (i+1)%REPORT_FREQ==0:\n",
    "        clear_output(True)\n",
    "        current_scores = score()\n",
    "        editdist_history.append(current_scores.mean())\n",
    "        plt.figure(figsize=(8,4))\n",
    "        plt.subplot(121)\n",
    "        plt.title('val score distribution')\n",
    "        plt.hist(current_scores, bins = 20)\n",
    "        plt.subplot(122)\n",
    "        plt.title('val score / traning time')\n",
    "        plt.plot(editdist_history)\n",
    "        plt.grid()\n",
    "        plt.show()\n",
    "        print(\"llh=%.3f, mean score=%.3f\"%(np.mean(loss_history[-10:]),np.mean(editdist_history[-10:])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for word in train_words[:10]:\n",
    "    print(\"%s -> %s\"%(word,model.translate(word)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Policy gradient (3 pts)\n",
    "\n",
    "First we need to define loss function as a custom theano operation.\n",
    "\n",
    "The simple way to do so is\n",
    "```\n",
    "@theano.compile.as_op(input_types,output_type(s),infer_shape)\n",
    "def my_super_function(inputs):\n",
    "    return outputs\n",
    "```\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "__Your task__ is to implement `_compute_levenshtein` function that takes matrices of words and translations, along with input masks, then converts those to actual words and phonemes and computes min-levenshtein via __get_distance__ function above.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@theano.compile.as_op([T.imatrix]*4,[T.fvector],lambda _,shapes: [shapes[0][:1]])\n",
    "def _compute_levenshtein(words_ix,words_mask,trans_ix,trans_mask):\n",
    "    \"\"\"\n",
    "    A custom theano operation that computes levenshtein loss for predicted trans.\n",
    "    \n",
    "    Params:\n",
    "    - words_ix - a matrix of letter indices, shape=[batch_size,word_length]\n",
    "    - words_mask - a matrix of zeros/ones, \n",
    "       1 means \"word is still not finished\"\n",
    "       0 means \"word has already finished and this is padding\"\n",
    "    \n",
    "    - trans_mask - a matrix of output letter indices, shape=[batch_size,translation_length]\n",
    "    - trans_mask - a matrix of zeros/ones, similar to words_mask but for trans_ix\n",
    "    \n",
    "    \n",
    "    Please implement the function and make sure it passes tests from the next cell.\n",
    "    \n",
    "    \"\"\"\n",
    "    \n",
    "    #convert words to strings\n",
    "    words_ix = list(map(lambda seq,mask: seq[mask!=0],words_ix,words_mask))\n",
    "    words = list(map(lambda ix: ''.join(map(source_letters.__getitem__,ix)),words_ix)) #<restore words (a list of strings) from words_ix and words_mask>\n",
    "\n",
    "\n",
    "    assert type(words) is list and type(words[0]) is str and len(words)==len(words_ix)\n",
    "    \n",
    "    #convert translations to lists\n",
    "    trans_ix = list(map(lambda seq,mask: seq[mask!=0],trans_ix,trans_mask))\n",
    "    translations = list(map(lambda ix: ''.join(map(target_letters.__getitem__,ix)),trans_ix))#<restore trans (a list of lists of phonemes) from trans_ix and trans_mask\n",
    "\n",
    "    assert type(translations) is list and type(translations[0]) is str and len(translations)==len(trans_ix)\n",
    "\n",
    "    #computes levenstein distances. can be arbitrary python code.\n",
    "    distances = list(map(get_distance,words,translations)) #<apply get_distance to each pair of [words,translations]>\n",
    "    \n",
    "    assert type(distances) in (list,tuple,np.ndarray) and len(distances) == len(words_ix)\n",
    "    \n",
    "    distances = np.array(list(distances),dtype='float32')\n",
    "    return distances\n",
    "\n",
    "#forbid gradient\n",
    "from theano.gradient import disconnected_grad\n",
    "def compute_levenshtein(*args):\n",
    "    return disconnected_grad(_compute_levenshtein(*[arg.astype('int32') for arg in args]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simple test suite to make sure your implementation is correct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#test suite\n",
    "#sample random batch of (words, correct trans, wrong trans)\n",
    "batch_words = np.random.choice(train_words, size=100 )\n",
    "batch_trans = list(map(random.choice,map(word_to_translation.get,batch_words )))\n",
    "batch_trans_wrong = np.random.choice(all_translations,size=100)\n",
    "\n",
    "batch_words_ix = T.constant(as_matrix(batch_words,source_to_ix))\n",
    "batch_trans_ix = T.constant(as_matrix(batch_trans,target_to_ix))\n",
    "batch_trans_wrong_ix = T.constant(as_matrix(batch_trans_wrong,target_to_ix))\n",
    "\n",
    "batch_words_mask = get_mask_by_eos(T.eq(batch_words_ix,EOS_ix_source))\n",
    "batch_trans_mask = get_mask_by_eos(T.eq(batch_trans_ix,EOS_ix_target))\n",
    "batch_trans_wrong_mask = get_mask_by_eos(T.eq(batch_trans_wrong_ix,EOS_ix_target))\n",
    "\n",
    "#assert compute_levenshtein works correctly\n",
    "correct_answers_score = compute_levenshtein(batch_words_ix,batch_words_mask,\n",
    "                                            batch_trans_ix,batch_trans_mask).eval()\n",
    "\n",
    "assert np.all(correct_answers_score==0),\"a perfect translation got nonzero levenshtein score!\"\n",
    "\n",
    "wrong_answers_score = compute_levenshtein(batch_words_ix,batch_words_mask,\n",
    "                                            batch_trans_wrong_ix,batch_trans_wrong_mask).eval()\n",
    "\n",
    "true_wrong_answers_score = np.array(list(map(get_distance,batch_words,batch_trans_wrong)))\n",
    "\n",
    "assert np.all(wrong_answers_score==true_wrong_answers_score),\"for some word symbolic levenshtein is different from actual levenshtein distance\"\n",
    "\n",
    "print(\"Everything seems right!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you got it working...\n",
    "\n",
    "\n",
    "* You may now want to __remove/comment asserts__ from function code for a slight speed-up.\n",
    "\n",
    "* There's a more detailed tutorial on custom theano ops here: [docs](http://deeplearning.net/software/theano/extending/extending_theano.html), [example](https://gist.github.com/justheuristic/9f4ffef6162a8089c3260fc3bbacbf46)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Self-critical policy gradient\n",
    "\n",
    "In this section you'll implement algorithm called self-critical sequence training (here's an [article](https://arxiv.org/abs/1612.00563)).\n",
    "\n",
    "The algorithm is a vanilla policy gradient with a special baseline. \n",
    "\n",
    "$$ \\nabla J = E_{x \\sim p(s)} E_{y \\sim \\pi(y|x)} \\nabla log \\pi(y|x) \\cdot (R(x,y) - b(x)) $$\n",
    "\n",
    "Here reward R(x,y) is a __negative levenshtein distance__ (since we minimize it). The baseline __b(x)__ represents how well model fares on word __x__.\n",
    "\n",
    "In practice, this means that we compute baseline as a score of greedy translation, $b(x) = R(x,y_{greedy}(x)) $.\n",
    "\n",
    "Luckily, we already obtained the required outputs: `model.greedy_translations, model.greedy_mask` and we only need to compute levenshtein using `compute_levenshtein` function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from agentnet.learning.generic import get_values_for_actions\n",
    "\n",
    "class trainer:    \n",
    "    \n",
    "    rewards = -compute_levenshtein(encoder.input_sequence,encoder.input_mask,\n",
    "                                   model.predicted_translations,model.mask,)\n",
    "    \n",
    "    baseline = -compute_levenshtein(encoder.input_sequence,encoder.input_mask,\n",
    "                                    model.greedy_translations,model.greedy_mask,) #<compute __negative__ levenshtein for greedy mode>\n",
    "    \n",
    "    #compute advantage using rewards and baseline\n",
    "    advantage = (rewards - baseline) #<your code - compute advantage>\n",
    "    \n",
    "    \n",
    "    #compute log_pi(a_t|s_t), shape = [batch,seq_length]\n",
    "    phoneme_logprobs = get_values_for_actions(model.logprobs_seq,model.predicted_translations)\n",
    "    \n",
    "    #policy gradient\n",
    "    J = phoneme_logprobs*advantage[:,None]\n",
    "    \n",
    "    loss = -T.sum(J*model.mask) / model.mask.sum()\n",
    "    \n",
    "    \n",
    "    #regularize with negative entropy\n",
    "    entropy = -(model.probs_seq*model.logprobs_seq).sum(axis=-1) #<compute matrix of shape [batch,seq_length] of entropy, H=-sum(p*log_p), don't forget the sign!>\n",
    "\n",
    "    loss -= 0.01*(model.mask*entropy).sum() / model.mask.sum()\n",
    "    \n",
    "    \n",
    "    \n",
    "    # Compute weight updates, clip by norm\n",
    "    \n",
    "    grads = T.grad(loss,model.weights)\n",
    "    grads = lasagne.updates.total_norm_constraint(grads,10)\n",
    "\n",
    "    updates = lasagne.updates.adam(grads, model.weights,learning_rate=1e-5)\n",
    "\n",
    "    train_step = theano.function([encoder.input_sequence],loss,\n",
    "                                 updates = model.auto_updates+model.greedy_auto_updates+updates)\n",
    "    \n",
    "    \n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Policy gradient training\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAEICAYAAACOKIcAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecHHX5B/DPs3s1yaVfLiE9AQIhQAJHb0dvKlh+/MQG\nKAYVFVRAREVUVBRFVPyJEQIiTZqC9JYjxARCEkgnlfR26bmUu9vd5/fHzOzNzs7szN5tvf28X6+8\nsjszO/Odvdt79tuer6gqiIiIKL9C+S4AERERMSATEREVBAZkIiKiAsCATEREVAAYkImIiAoAAzIR\nEVEBYEDuJBFpEJF1+S5HR4nIlSIyzfa8WURGZejct4jIfebjESKiIlKWoXMPM8sazsT5qHQU+2c2\nH0RkoYg05Oha94rIj3NxrULDgEwJVLWHqq5MdUzQP2iq+ktVvToT5RKRVSJyju3ca8yyRjNxfqJC\nJyJLRORQl+2NIpKRz5kXVT1CVRszfV5nhcC81tdU9eeZvlYxYEAuYpmqbWZDIZeNKF86+rkQkdEA\nwqq6NFfXpNxjQAYgIt8Xkacc2/4gIn80H18lIotFZI+IrBSRawKeV0Tk9yKyRUR2i8h8ERln7qsW\nkd+JyGoR2SUi00Sk2tz3CbOJaKf57fdw2zlXmeWdB2CviJSJyEEi8rSINInIRyLy7RRl6iciz5nl\nmQlgtGO/isjB5uOLRGSRed/rReQGEekO4CUAB5lNxs3m9W8TkadE5GER2Q3gSnPbw44ifFlENojI\nRhG5wXbdB0XkdtvzeC1cRP4BYBiA/5jXu8nZBG6W4TkR2S4iy0Xkq7Zz3SYiT4jIQ+a9LBSR+iA/\nQypMpfSZNV0M4EWX8v4CwGkA7jE/G/eY21VErhWRZQCW2d6fteZ9zRaR02znSfkZEVsLVYBjjxGR\n9819T4rIP+2fbdtxhwO4F8BJZtl3mtvjfwusvwPmZ36L+XfjUvNv01Lz836L7ZwhEblZRFaIyDaz\nnH193tvCoaol/w/AcAD7ANSYz8MANgI40Xx+MYzAJQDOMI89xtzXAGCdx3nPBzAbQG/ztYcDGGTu\n+zOARgCDzeudDKASwKEA9gI4F0A5gJsALAdQYb5uFYAPAAwFUA3jS9VsALcCqAAwCsBKAOd7lOlx\nAE8A6A5gHID1AKbZ9iuAg83HGwGcZj7uk+qeAdwGoA3ApWaZqs1tD5v7R5jnfsy89pEAmgCcY+5/\nEMDttvMlXMO873Nsz63zlZnPpwL4PwBVAMab5z7LVrYDAC4y3+tfAXgn3793/MfPbJDPrHmOl1N8\nphsBXO3YpgBeA9AXQLW57QsA+gEoA/A9AJsAVJn7Un5G7J+/VMea97MawHXme/EpAK2wfbYd5bwS\ntr8/5rYHrePNn1XEfK/KAXzV/Gw/CqAGwBEA9gMYaR5/HYB3AAwxfzZ/BfBYvn9fA/9e57sAhfIP\nwDQAXzIfnwtgRYpj/w3gOtsvjNeH+ywASwGcCCBk2x4yf4mOdnnNjwE84Th2PYAG8/kqAF+27T8B\nwBrHOX4A4AGXc4dhBM3DbNt+Ce+AvAbANQB6Os6TdM/mh3SqyzZnQLZf+zcA7jcfxz+EbtdAioAM\n4w9dFOYfZ3P/rwA8aCvH67Z9YwHsz/fvHP917l8pfGbNfd0AbANQ6bG/Ee4B+Syf92+HdT9+nxEk\nB2TXYwGcbt67OH5OnQnI+2E01wNGEFYAJ9iOnw3gUvPxYgBn2/YNgvE3ryzfv69B/rHJut2jAC43\nH3/OfA4AEJELReQds3lkJ4xvhv39TqiqbwK4B8Y36y0iMklEepqvrQKwwuVlB8H4hmmdIwZgLYxv\n5Za1tsfDYTQf77T+AbgFQJ3LuWthBDD761e7HGf5NIx7XS0ib4nISSmOdZYryDGrYdxvZx0EYLuq\n7nGc2/6ebbI93gegSti3VuxK4TMLAGcDmK6qLX7ld0j4PIrR5bTYbG7fCaAXEt+TdD4jXsceBGC9\nmtHQrRwdsE3bB2/uN//fbNu/H0AP8/FwAP+yva+LYXxZ93pvCwoDcrsnATSIyBAAn4T54RaRSgBP\nA/gtgDpV7Q2jL0eCnFRV/6iqx8L4FnkogBsBbIXR5DPa5SUbYPxSwby+wKgBrref1vZ4LYCPVLW3\n7V+Nql7kcu4mGM0/Q23bhqUo+3uqegmAATBqGE+4XD/hJV7nsnFee4P5eC+MmoBlYBrn3gCgr4jU\nOM693uN46hpK4TMLGF8mkvqPPc7tut3sL74JwGUA+pjvyS4EfE/SsBHAYPM9sAz1OhjB/makYy2A\nCx3vbZWqFsXfAgZkk6o2wWj6eQDGh2WxuasCRl9EE4CIiFwI4Lwg5xSR40TkBBEphxFwDgCImd+g\nJwO4yxzcERaRk8w/JE8AuFhEzjZf9z0ALQCme1xmJoA95qCRavNc40TkOJd7jAJ4BsBtItJNRMYC\nuMKj7BUi8nkR6aWqbQB2A4iZuzcD6CcivYK8Dw4/Nq99BICrAPzT3P4BgItEpK+IDARwveN1m2H0\ntSVR1bUw3p9fiUiViBwF4CsAnAPKqAsphc+s6UIAL6Qotudnw6YGxpfxJgBlInIrgJ4+r+mIGTBq\npN8UY/DaJQCOT3H8ZgBDRKQiQ9e/F8AvRGQ4AIhIrVmGosCAnOhRAOfA1vRlNoN+G8aHbgeMprHn\nAp6vJ4C/ma9bDaMf6E5z3w0A5gN4D8B2AL+G0We1BMbgiz/B+Fb+cQAfV9VWtwuYQfZjMAYyfWS+\n5j4YzVFuvgmjeWcTjL6aB1KU/4sAVokxavprAD5vXvNDGIOzVppNQ+k0O78FY8DLGwB+q6qvmtv/\nAWAujL6qV9EeqC2/AvAj83o3INnlMPqVNwD4F4CfqOrraZSLilOX/syKMcK7WVXXpCjzHwB8RkR2\niDnK3MUrMAaGLTXv6wA635ScxLznT8H4QrwTxvvyPIwvKG7eBLAQwCYR2ZqBIvwBxs/6VRHZA2OA\n1wkZOG9OSGJTPxERFQoRuQlAf1W9Kd9l6SgReRfAvaqa6ss/wRjgQ0REhWkVgP/kuxDpEJEzACyB\nUfP/PICjYNTOyQebrIlKiIhMNhMsLLBtG2+OSP5ARGaJSKo+P8ohVX3C1jdeLMbA6H7aCaM//TOq\nujG/RSoObLImKiEicjqAZgAPqaqVgepVAL9X1ZdE5CIAN6lqQx6LSVSSWEMmKiGqOhXGgKSEzWgf\ncdsL7VPRiCiHctqH3L9/fx0xYkQuL0lUlGbPnr1VVWtzdLnrAbwiIr+F8SX9ZLeDRGQigIkAUF1d\nfezQoammlxpisRhCoeL73s9y51ZXLvfSpUsDf5ZzGpBHjBiBWbNm5fKSREVJRFJlUMu0rwP4jqo+\nLSKXAbgfxlSiBKo6CcAkAKivr9cgn+XGxkY0NDRktrQ5wHLnVlcudzqf5eL7SkJEmXYFjIQxgJH9\nioO6iPKAAZmINsBYEQkwFldYlseyEJUszkMmKiEi8hiMFXT6i7He9E9gLGn3B3NxgAMw+4mJKLcY\nkIlKiKpe7rHr2JwWhIiSsMmaiIioADAgExERFQAGZCIiogLAgExEWbGiqRmLt0XzXQyiosGATERZ\ncfbv3sKv3zuQ72IQFY2CG2U94uYXfI9ZdcfFOSgJERFR7rCGTEREVAAYkImIiAoAAzIREVEBYEAm\noqxS1XwXgago+AZkERkqIlNEZJGILBSR68ztt4nIehH5wPx3UfaLS0TFJhpjQCYKIsgo6wiA76nq\nHBGpATBbRF4z9/1eVX+bveIRUbFriyrKwvkuBVHh8w3IqroRwEbz8R4RWQxgcLYLRkRdQ2s0hmow\nIhP5SasPWURGAJgA4F1z07dEZJ6ITBaRPh6vmSgis0RkVlNTU6cKS0TFpy0ay3cRiIpC4IAsIj0A\nPA3gelXdDeAvAEYBGA+jBv07t9ep6iRVrVfV+tra2gwUmYiKSSTKPmSiIAIFZBEphxGMH1HVZwBA\nVTeralRVYwD+BuD47BWTiIoVa8hEwQQZZS0A7gewWFXvsm0fZDvskwAWZL54RFTsWhmQiQIJMsr6\nFABfBDBfRD4wt90C4HIRGQ9AAawCcE1WSkhERY01ZKJggoyyngZAXHa9mPniEFFX0xZhHzJREMzU\nRURZxSZromAYkIkoqyIMyESBMCATUVa1cdoTUSAMyESUFWKOPOGgLqJgGJCJKCvCZkRmHzJRMAzI\nRJQV4ZARkFlDJgqGAZmIssJqsubyi0TBMCATUVaImb5AGY+JAmFAJqKsYg2ZKBgGZKISYi6VukVE\nFji2f0tEPhSRhSLym8xcy/g/yioyUSAMyESl5UEAF9g3iMiZAC4BcLSqHgHgt5m4kJVvNxZTrGhq\nxk1PzWWSEKIUGJCJSoiqTgWw3bH56wDuUNUW85gtmbiWmFXkqCq++ej7eGLWOizZvCcTpybqkoKs\n9kREXduhAE4TkV8AOADgBlV9z3mQiEwEMBEA6urq0NjYmPKkkUgEALBkyVLs3N0GAJg7ZzaalhZ+\nPaC5udn3/goRy51bmS43AzIRlQHoC+BEAMcBeEJERqkmdv6q6iQAkwCgvr5eGxoaUp90yitANILR\nBx+CKZtWAnv34+QTT8CI/t2zchOZ1NjYCL/7K0Qsd25lutyF/1WViLJtHYBn1DATQAxA/86e1OpD\njsYUrRGj7zjGAV5EnhiQiejfAM4EABE5FEAFgK2dPqsZkWPKgEwUBJusiUqIiDwGoAFAfxFZB+An\nACYDmGxOhWoFcIWzubpD1zL/t9eQI5yTTOSJAZmohKjq5R67vpDpa1mjrGPavgQjk4QQeWOTNRFl\nVUw1vuITAzKRNwZkIsoqexBmQCbyxoBMRFnhttoTAzKRNwZkIsoKa1iYfXwYAzKRNwZkIsoKa4qT\nfXEJLjRB5I0BmYiywoq99vUkWEMm8saATERZYdWQY2yyJgqEAZmIsiLeZM1BXUSBMCATUVbEzKZq\n1pCJgmFAJqKsiDdZs4ZMFAgDMhFlBUdZE6WHAZmIsiLGUdZEafENyCIyVESmiMgiEVkoIteZ2/uK\nyGsissz8v0/2i0tExcCeDIRN1kTBBKkhRwB8T1XHAjgRwLUiMhbAzQDeUNVDALxhPicigj3uclAX\nUTC+AVlVN6rqHPPxHgCLAQwGcAmAv5uH/R3ApdkqJBEVF3sQnrduV/wxAzKRt7T6kEVkBIAJAN4F\nUKeqG81dmwDUZbRkRFS07IF3yeY97ds5qIvIU+CALCI9ADwN4HpV3W3fp0aHkesnTUQmisgsEZnV\n1NTUqcISUXHwirusIRN5CxSQRaQcRjB+RFWfMTdvFpFB5v5BALa4vVZVJ6lqvarW19bWZqLMRFTg\nYh4RmQGZyFuQUdYC4H4Ai1X1Ltuu5wBcYT6+AsCzmS8eERUjBmSi9JUFOOYUAF8EMF9EPjC33QLg\nDgBPiMhXAKwGcFl2ikhExcYr7jIgE3nzDciqOg2AeOw+O7PFIaKuQD1qyBEGZCJPzNRFRBnnFXdj\nDMhEnhiQiSjjPPuQOe2JyBMDMhFlXHV5GDeePyZpO/uQibwxIBNRxnWvLMO1Zx6ctH3PgQjW79yf\nhxIRFT4GZCLKmQenr8Ipd7yZ72IQFSQGZCLKuWWb93iOxCYqVQzIRCVERCaLyBYRWeCy73sioiLS\nP9vlOPf3UzH5v6uyfRmiosKATFRaHgRwgXOjiAwFcB6ANbkqyJw1O3J1KaKiwIBMVEJUdSqA7S67\nfg/gJngsEpMNXtmGiEpVkNSZRNSFicglANar6lwjdb3ncRMBTASAuro6NDY2+p77+iMVd883zlkm\nQMQW7pu2bAl0jnxobm4u2LKlwnLnVqbLzYBMVMJEpBuM3PTn+R2rqpMATAKA+vp6bWho8L9AYyMw\nfy8AoK5XdcKUp7q6OjQ0TOhAqbOvsbERge6vwLDcuZXpcrPJmqi0jQYwEsBcEVkFYAiAOSIyMNMX\n6t+jIuF5iso4UUliDZmohKnqfAADrOdmUK5X1a2Zvlaf7o6AnOkLEBU51pCJSoiIPAZgBoAxIrLO\nXD41J7pVhJ1lydWliYoCa8hEJURVL/fZPyJb164uT/xzw3BMlIg1ZCLKie6ViTVkRmSiRAzIRJQT\n3SrYIEeUCgMyEWXVgJpKAECfbuUJ24VVZKIE/MpKRFn1+MQTsWrbXoRDid//OaaLKBFryESUVaNq\ne+Csw+pQVeYIyHkqD1GhYkAmopyoLHdOezL+j0RjiMW4FCMRAzIR5URlmfufm4N/+BKu/+cHOS4N\nUeFhQCainHAGZPugrufmbsh1cYgKDgMyEeWEW5O1KpuqiSwMyESUE84asioQZd8xURwDMhHlhDMg\nR2KKtigDMpGFAZmIcqLCEZCfnrMuYX1kolLHgExEOVFZFsbkK+sTtj00Y1X88bE/fw1TlzZhxopt\neH/NjtwWjqgAMFMXEeXMWYfVJTx/aMbq+ONte1tx5ytLMH/9LgDAqjsuzmnZiPKNNWQiKhhMp0ml\njAGZiAqGMCJTCfMNyCIyWUS2iMgC27bbRGS9iHxg/rsou8Ukoq7isa+e6LkvxHhMJSxIDflBABe4\nbP+9qo43/72Y2WIRUVd10uh+nvsYj6mU+QZkVZ0KYHsOykJEJS7EJmsqYZ3pQ/6WiMwzm7T7eB0k\nIhNFZJaIzGpqaurE5Yioq2NAplLW0YD8FwCjAIwHsBHA77wOVNVJqlqvqvW1tbUdvBwRlQTGYyph\nHQrIqrpZVaOqGgPwNwDHZ7ZYRNSVnXZIf9ftHNRFpaxDAVlEBtmefhLAAq9jiYic7vncMa7b7Usy\nTlu2NVfFISoIQaY9PQZgBoAxIrJORL4C4DciMl9E5gE4E8B3slxOIupCysPuVWFF+2ITX7j/3VwV\nh6gg+KbOVNXLXTbfn4WyEFGJCHu0TXM5RiplzNRFRDlXHnL/0xNhQKYSxoBMRDkXstWQG8a0z75g\nDZlKWVGu9jTi5hcCHcfVYogKX9g29zgSTQzILZEoKsvCuS4SUV6whkxUQjxy098pIh+aiX7+JSK9\nc1kme23ZWUPetb8tl0UhyisGZKLS8iCSc9O/BmCcqh4FYCmAH+SyQAk15FgsYZ/jKVGXxoBMVELc\nctOr6quqGjGfvgNgSC7LFE5RQ44q+5SpdBRlHzIRZc2XAfzTbYeITAQwEQDq6urQ2Njoe7Lm5mbf\n47Y2bWk/ft/+hH3Tp89Abbfc1xuClLsQsdy5lelyMyATEQBARH4IIALgEbf9qjoJwCQAqK+v14aG\nBt9zNjY2wvO4l43BmYMG1gGbNgAAyisqgf0H4odsrByKT59+cEI/c2dFY+o5D9qSstwFjOXOrUyX\nm03WRAQRuRLAxwB8XjW37cRh25zkjbsOJOy767WleGLW2oxda/ryrRh9y4t4f82OjJ2TKFMYkIlK\nnIhcAOAmAJ9Q1X25vn7Y56/Qpt0HUh+QhreWGkvAvrOSS7xT4WFAJiohHrnp7wFQA+A1EflARO7N\nZZn8mo+dc5M7hatJUQFjHzJRCSnE3PQhSR0l2zI498laTcq+iAVRoWANmYjyyq+G3BbJXPC0Yj9n\nU1EhYg2ZiPLi55eOQ7fyMOav35XyOGeykM6wQn+mxq3t2t+GD9buxBmH1vofTOSDNWQiyosvnjgc\nnz52iH8NOYN9yJmuIX/z0Tm4YvJMbG1uycwJqaQxIBNRXvkF5Ja2KO5+fSl2H0jOax2NKV6avxGx\ngKtEtfchZ8byLc0AgNYIc3xS5zEgE1Fe+Q3qen7+Rtz9+jL86sXFSftemL8RX39kDu6btjLgtYz/\n2YdMhYgBmYjyym8eslX73L0/krSvssx48euLtyTtc2UG/1iGInKuAvvUpU1obkm+f+paGJCJKK/C\nPjVki9vgrgozms/8aDvaov7NxvFBXYFLl1oupk+t27EPX5o8Ezc8MTfr16L8YkAmorwKmqfaLUGI\nfXWoJZv2+J5D2odZB7pmUJmqcbvZ2xIFAKxoas7aNagwMCATUV6VBQzIbS4Dt+zLM27Z455ic9Ou\nA3h5wSYAmR/UZclEPD7QFnUdnGZ96fDra6fix4BMRHnVvTJYOoSIS5O0PYBt3u0+9egz907H1x6e\njVhMOz2o685XPkTDnVPiz63zONdxTpeq4rAfv4xRt7yIs3/XmLDPqn3bWxL+/f56fLhpd6euSYWH\nAZmI8qqmqjzQcRGfGvJmj0Uo1u3YH3+9VcmMqWLTrgN4deGmtMr65ykrsGpb8vob0U5Wke23tqJp\nb8I+677tg9+u/+cHuODutzt1TSo8DMhElFc9HDXkTx8zBNefc0jSca41ZFsg86ohx18fi0Gkvcn6\nM/dOx8R/zM5I1q6g86C9pKphW6PM2WTd9TEgE1Fe9axKDMhV5SFcMG5g0nFuNeRYvH8VrolD7OwZ\nv/7SuCJec+5sczPQ+RpyqjK0RIxBXeISkLdkcGlKyj8GZCLKqx6OgBwOietUKLcUmlYg61VdjuYD\nETTtacH8de65saO2Jms7t0AflPXKzgb1VAG9pc2qISfvu+yvMzp1XSosDMhElFfOPuSQiOtUqKjL\nPGQrkPWqLkdzSwQX/fFtfPyeaa7XiURj8VHWiecNFky9+qgBoDPrX6gq9rdGPfe3mE3Wbl9S3Pqz\nqXhxtSciyitnH7JXDdltHnLMpYbsxW3aFBC8hvzpv0yPP1bVhCbkzjRZ/+Od1bj12YVJ2y/76wwc\nNbgXjhjcE0Dw+dpUvBiQiSivatyarF2CTySmeOK9tbjp6XlY8NPz0aOyLB4Ie1aXY2tz++hkZ8AE\njBryv95fl3TeoDVkq8/Zek1ZWDIy7enp2cllAozsYzM/2h5/znjc9bHJmojyqqo8jAevOg5HDu4F\nwAjIbrXBSDSGX75kLDCxY28rgPZR1j2ry7HHNqjLrdb74vxNWLo5OdtVR9Zbdp4/aKauXfvbMH3F\n1oRtLQFXirJGWWdqLWcqPAzIRJR3DWMGoH+PCgBGX6lbbbA1qti5zwi6Vt7qhCZr2+ILbsshbvNY\ns9heu127fR/+9MayeNDb2tyCpj0teGflNs/XuD23PDFrLW58cm78mON/8To+97d3E/qM0wnIO/e1\n4lcvfRjoeCo+vk3WIjIZwMcAbFHVcea2vgD+CWAEgFUALlPVHdkrJhF1dVYNMOTRh7zVFlAPmCOP\n7aOs7TGxNRJD98rE13u1Ktv7pq/++yws2bwHd55eDQCov/1199c4a8geJ7/pqXkAgAuPHIgvPzir\nvXzRGKoRBmCs9+zktlBGKCT4x4zVmDQ12FKTVHyC1JAfBHCBY9vNAN5Q1UMAvGE+JyLqMCsGhz1G\nWdu1RKJ4fOYa/NJcI9k5MKzVNYmI96CuDTv343N/ewfrduxLKIuX9iQlxjn9BnW9smBzwnN7jdqt\nhrzXZalFVcXAXlWpC0ZFzTcgq+pUANsdmy8B8Hfz8d8BXJrhchFRibFiWjjkvyTjgbYYbn5mfrym\nWuFYVNmtydqrWTkai+HPU5Zj+opt2Gs2JfuNnwraZG3556y1nse7BeQ9B5IDciSqqCwP+5QM2LWv\nzTWrWSF7c00bXl6wMd/FyLuOjrKuU1Xr3dsEoM7rQBGZCGAiAAwbNqyDl+s6Rtz8gu8xq+64OAcl\nISosVg02HAr51pBfceSgLg8nHm9lt7LzqsVGYop9jnnAfoOmUw3qOueutzBmYA3+/LljPF+fGJCT\ny9rsUkOOxGK+KTpjqjj6Z6/isvoh+M1njk55bCF5aFErHlo0p+T/9nV6UJcaox88f0tUdZKq1qtq\nfW1tbWcvR0RdXDgE12lPdg9OX5XwvLws8U/Z4o3G2sgHbP2zXsEsEtWkAOgy5Tlxf1INuf3x8i3N\neGFe6tqefWS3WwYytxpyW1R9a+LWqZ6esz7lcVSYOhqQN4vIIAAw/9+SuSIRUSmyQk3IY5R1KuWO\nJutvPfY+AGD3/vapUN5N1op9rYkBMGgNuaPzkP2Ob25JzsvdFo359lVbcd5vGtau/W2467WlGcnj\nTZnT0YD8HIArzMdXAHg2M8UholLV3ocsaa9s5OxDtuy0B+QUTdbO1JV+NWRnH23Qecj2a6bi1Yfs\n12RtlduvOL94YRH++MaytJefpOzyDcgi8hiAGQDGiMg6EfkKgDsAnCsiywCcYz4nIuowqx+4zCNT\nVypux3+0dS8u/fN/48+9glk0pkkB0K/m6AyozmQffvwCq1ve7LZozLfmHrTCaw1e80onSvnhO6hL\nVS/32HV2hstCRFlWyHkFulUYf4685iGn4haQP/V//00YrOVV643EYti1P7GJOKapg7Jz38PvrMHt\nlx4ZuLxWrfznLyxy3f/Iu2sSnh82sAZb9rSkbLK+5h+zEGtuDVYA8zTMxllYmKmLqLQ8iALNK9Dd\nnEusmv5CCm4Bece+xCDb5pERKxrTpLWUo+o++tkS70O2bVu9bS9emh9s6k40pnhq9lo86gi87edq\nX8Vp7KCeuHDcIGzf24q3ljR5nvOVhZvx2urkpm43apY8ze89lGUMyEQlpJDzCvSoNObYuiXF8BOk\nRu0VYPe2ROOZvyxRbV+H2I3bUpBn3NmIrz8yx7ccgBHQK8pS//m1UomWhwWfnDAYALBx1/6k4w60\nRRNGkzut3rY3IcsZ0N7H7LYcJeVPya/2VOrzgoPcP9C13wMKllfAnlOgrq4OjY2Nvidubm4OdBwA\nNG00mlsXLl2BRl3rc3SihQvn+x6zqWmb6/Z33jde27NCsLvViFR79+3HlLfd11UGgPdmv489H4XR\n1urdRDxlyhTPfbNmzca2Az4jpiNGrX1v8x7Mee8dAMC2nXuSjjvx9ldcm9et9/3Kl/eiTID7zu8e\n37elyeijXrRoIbpvX5KyHLkU9HelUKTz+x1EyQdkImqnqioirpFCVScBmAQA9fX12tDQ4Hu+xsZG\nBDkOAJaFVuLfyxejdtAQNDSMBV52/7L4+ndPxzl3TU3YNv7oo4HZM1Oev3tNL2B7ctf41M3lAFpx\n3KhavPGhMYOzorIKx9SfAHj8sT3yqKNxysH9UTb1VaAteYoSACwJDQPgvhDEUeMnGKtTzZ3luh8A\narpXY/s/fuzPAAAe90lEQVSBfejXpzfOPL0eaHwVKKsEkDjga2eLe2CPv+8vv4CIIuHn8NjaWcDm\nzRh3xBFoOHKQZxlyxvxZB/1dKRTp/H4HwSZrIiqIvALdzCZr+5xgt2bdgwfUJG0rC9Dn7ExR+ZvP\nHAUAWLnVWEd5eL/2GqTRh5x4/CcnDMad5mv8pi0BwB0pVmWKxGJJc6edrKlc5eEQKsuNx8750h0V\nb7LOUSfyrc8uwCfu8W5xIAMDMhEVRF6BsYN6AgCOOMhYF/m+L9Xj9e+c4Xpsv+4VCc+DzFt29iE7\n813X9WxfHsptUNcXTxqOMQONLwPRWAw3Pjk3aeBYUNGY+s5dtr6MlIUlHpz3p+grTod15VwN6npo\nxmrMW7crNxcrYgzIRCWkkPMKTBjWB2/d2IDPn2DkvD9nbB2G9evmeuyL152GB648Lv7cPsq6qtz9\nz9rSzc0Jz4f1TTx3Xc/2lZRimpzSsiIcil8nElU8OXud3y15ChKQrRp0mZnbuyIcck2z2REacNrT\nvtYIvvHIbNfBZJR57EMmKiGFnlfA3mycSl3PKvSqLo8/twfkIEmz5v7kvKT81QNqEmvIbgOlykJG\nkAzSZJ2KEZBTH2PVkK2EKRVlIddlJTtC1Zr2lDokvzR/E16cvwlVZWHc9b/jO33dWEzTntLmZt66\nnfjeE3Px72tPiU+X6wpYQyaiomRPl2kPyEGCZa/q8qR+56qK9qUNF22LYue+xBHUvbuVt9eQOxmQ\nIwFqyJXxJutQwvNMiDdZ+xwXC1qV9nD8L17HfW+vjD9vzlAf+G9eXoJlW5rxwdqdGTlfoWBAJqKi\n8JtPH4X/+3z7kob2mpZ9HvJwj2ZuJ2cykfJQCFeePAIAMHVdBN80F6gAgIuPHIQhfbrFa6udXW84\nGmvPS/3I1Se4HhMf1GWWM6MBOV5D9jvO+N+vj/6+t1diwXqjj3jdjn045663sGX3AWzZ04LbX1gc\nP67ZJUd3Z6SZQrzgMSATUVG47LihuMhjio49uB43vC+e/vrJvudz1pDLwoKvnTE6/tw+6KufmaSj\nutwaCe4+uMpK5uEnEtN4Gkx707udfVAXAFSWh12PS9e0ZVuTVqvyYmX08mtlvv2FxfjYn4xR1A/N\nWI3lW5rxzPvJS0C6LZqhXS2qdkLXaXwnopJlD8ihkMRHQwd9DWD01XotamH1HddUGcHTLbDYj/MT\ni2k8ynll7IoP6upEk7V9EYu2aAzvrdqOL9z/bvt+v+UcO5DRy6pMu6Uq3XMgeVQ6l4Bsx4BMREXP\nPqW3LCTxpuVUrMUs2l8X8nyddf6q8hDKQuIaWID22qyfSEzjf3y95lBbgbqiEwHZ3td9yA9fSqrB\n+68eZdaQU1zaWcO1mrftA9AqykJojcRcv8j4rfFcSthkTURFz97HGQ4JygPUVJ214bIUNWTr/CKC\nmqoyzxqyX7IPSzQWi9cM7de82zaS2fpyUG0ONqssS7/JOuLIub3VsRpU4Bpyij5kZ1C3jrQ3+df2\nMEawb3JZVtJeRL9lKePXyGEK7hVNzWjL0Oh2PwzIRFT0nE3FqabWPHvtKfHHb990ZvxxeTjk3eRs\nO11NVXnSco0WZ0C/8fwxrsfZR1nbv0wMsCUniZhzjruZfceVHvOrvaiq72hwv4AMbe9D3n2gzXUR\nC2fQt+7HnunMqpmv3b4PTvYa8lcf8k4l6lKseB93tmzZcwBn/+4t/OS5hVm9joUBmYiKnj2O7vcY\ncGUZN7hX/PFQW3KQspB/DRkAaqrKsH1ve02zfnifhHPYWdnHnGL2gJyQ1KS9FmwFtI7WkB9/by2i\nPolE/Ppv7X3IR932Kj75f9OTjolEnU3Wxv9utcp1O5ITjNjLYOUSLxRWS8g7K9wXJsk0BmQiKnr2\nQLqnJXU6S6/Kc1k45NmfG0qoIScGZHsQdfYhe9XUI7bEIPYpW90q7AE5am4ri183HT94Zr5vDdm/\ngpw4ynrxxt1oi8bwxuLNmDztIwAuc7KtPmRbDXnuuvYpUU5Bm6ldLpH15SOtL2K+LQkZwkFdRFT0\nEgKyz1xXr/7Q8rB4BlB7DblnVTkWbtgdf24fJW1v8j5vbJ1nuIjGNF4ztC45qFdVwheC1ngN2Rrh\nnf6fa2c2MqeO9CFf+uf/xu//y6eOTKplW7fgllVsrUsNuTPBLttN1ta95GogeJcOyEHX+iWiwvXT\nTxyBxRt3u+67/4p6tEZiCbVMZ0B+8msn4X/uneF7nVRTluyB9ZzD6/Dqos3x5/bRz9ZArJ9dcgS+\ncMJwvL18q+v5tja34t63VgAwatFTbzwTfbqXJwy6ijdZlxt/pnt0IEWkW5+tnX+TdXICEfuXEcC7\nD/nZDzYkna9pTwsOtEUTWhU6M8o6psDPn1+EIw7qiQE1VTj1kP4dPpcbqwaeq6lZXTogE1Hxu8LM\nnuXm7MPrACAhzaVzStJxI/oGuo4VTOf8+Fzc/ngjnllmO48tIo0f1jvhdZUuNeTycCjlwLIZK9oD\ndVgkvoiGfbCYVUO2mrF7dKCGvNalidgu4JiulJm6vGrIXtbv3I/RtT3iz2NpDGBWVbw4f1O833rd\njn2432w6B4BVd1wc/GQBWF9IcpW8hH3IRFT07E3Wfs20Xqxm2b7dK1Dp7Au2PXUuZmAfbGX1IYd9\n5uXMtS1FaA929vzcVm1vhLnghpWUxO6175zumXoTALbsbklZDr/aaczRh+zGOajLq0tgcO9qAMDq\nbXvTKoPd8/M24tpH52DGSmOQVbZrrlbZcjVXmgGZiIqePSD/4MLDO38+R0yxDx7q4UgoUpHQZG08\ntmrHQWpW9pZy+zzma888GO/ecna89lzj0mRdURaKBzo3+3wWc/Drv7UGbKWah2wPiv94Z7XnHOHR\nA4xa8UdbE2vt6Qzq2tqc+AXjj28sD/xaALj56XmYurTJc/+Gnftx6q/fxJptRhmtn19MgXvfWpGw\nUEY2MCATUdGz1zIvnTC40+dzBuTEGnLi9KPEJmszgUg617KdvNxxLvsazW6DukIiOChFQG5c4h18\nAP/BStbUpVQ1Ufso61ufXeDZvN2/ewV6dyvHyqbEdamd556/bpfn1DXn9wdngE4lFlM8/t5afGny\nTM9jlm7eg3U79uMjsxZvjUtTVdzx0ocJC2VkAwMyERU9r/nDHT9f4nN7f3CZY6c9YYfVZJ0qzv38\n0nGJ57YFMHvqTmet1G1QVzgknrmwAWDZlmbPfUBy7XTNtn246oGZ8Zq1FZBTZaqyB1RV737pUEhw\nUK9qbHZk63I2B3/8nmm47vH34cavLp2qRSJIs7PVh2+9L9a9uX0faVyyBfsjmW3KZkAmoqLn1mf7\n35vPwns/PKeD5wt+rL0POWy2P6cKDIcM6JHwPCEgpxjp7daHbL12ZP/uwQrrYG+yXr1tL06/cwqm\nLGnClA+NmrXVP/zQjNWurzeygSUG66jHKK2wCCrLQ2iJxDDi5hdwm5n9yq3J2qtm79cFMHXZViza\n4D4iP0h/825zhH57IE7837J2+z5c+cB7uH9+8Bp6EAzIRFT03EY0D+5djdqaSpej/TkDfKpRxvZB\nXtbaxdafb+v/0w+tjR/jXG7RuVKVF9cma/Mv+JQbGnD0ECMD2Y8uDt6Hbo9Rk22jlS1tATJ9OQOd\nVzKSUEhQWRaKT+d6cPqqpDJY3OYwA/6jwq+YPBMX/fFt131+SVIAYLdZQ44P5rICs+O1e80WhE17\nM5vjmgGZiMghqck6RY3Znl0rHlDNv999uhk5nEf2a0/R6WxeD9ra7tZk7fZFIZ1VoeyBxj4Hem9r\nBLFYcu3XKRJLzpftHHVtCYeAirJwQo5rIL2R0p1JBGK/zubdB/DR1r1JxyQ1WcenPbUfE7F9Wcj0\n2GsGZCIiB2eQdMY9exOxPcBaj6zAMX5obzxw5XG4xVZrdQbRVCOY7dzmIbs11QddcQoA3rKNOG6y\nDZC66al5uPuNZb415LZoLCmgeiVxCYtRQ95vG/l97SNzsLwpdT93ptjL+a3H3seZv21MSOX53+Vb\nsWGnkUnMCsSxWHKTdUen1QXBxCBE1CXcdMEYnDSqX9qve+m607BjX+KyhMmjrBM3PPTl43HGnVMQ\n08Sg2J77uP3YMw8bYO4ztodDgp98fCx++p9FaZXTLdC6xfJ0BrhNW74VH23di5H9uyfk5waAd1Zu\nw3Db4htu5qzZmTB3GgDeXOK+QITCqL3vbWkfQf3C/I1o9DjeLhNZF+21fWsu9NvLtuLy44dh064D\n+Px978b3R1MM6tq9P3sBmTVkIuoSvtFwMCYM6+N/oMPhg3ri5NGJKRedrb7OWuzQvt1whtkvnFBD\ntlqsXSqWw80EH62RGK46ZWTa5fTj7LcOarejmdYysGeV7zrAV0yeif/MS0yR6dXPG4kpKspC2O9Y\nwtH6ElPlsbzkWb9rTFkGP++v2YGRP3gB6215tIeZXzQWbjAStDiXlYw5EoLYa8h7WtqylpCEAZmI\nujx7P28Q9hpx/x4VuOjIgUnHWH+S3QKym19/+igcPKBHPBhkmvWlwlp7OCgr6Dhr1iLAPp+lLAHg\nQ1sTtXPAml0spqgsC2Ovo8nXes8OGVCTsN1KHbqyKbmv1+nHHxubtC0aU/zihUW47bmFUAVmrdoR\n33egLfHczp9b+/xjs+y2gByNqWc/eWd1qslaRFYB2AMgCiCiqvWZKBQRUSa9+b0GfLhpd8JayKnY\nm6xn/ejc1McmBDJrlHXyH+zjR/bF6989I9D1/Zx12AC8+eGWhDnIN54/BqN1A047pBbnjU1cACMV\nq2bsDMibdx9IWkjCjX2QVr8eFQn5uO2iMUX3yuRBXdZ1nTXkfa0RlIe9A7zd+KHJP9fpK7bib2+3\njxy398FbtfTpK7Zhws9exaQvJYauVPOQIzHNWm7rTNSQz1TV8QzGRFSoBvaqQsOYAejfI9g0qDTG\nRbmOdM5W6uMnv3YS/vWNk/Gnyyfg+W+dGl8rGTACW223EMrDoYQAY6893upSk3x+3kZ845HZ+HDT\nnoTt76zc7ruUJZAYkO3v75ljahOOi6q6jgBvb7JObMXY1xpNat72cuzwvjhvbF38+d6WCL54f2JG\nLnv2L/vjHfva8NSsdUlltf/vrCFbg904ypqIKMvK0kgMEg4Jnr32FPz2f45u70POTrFw3Ii+mDCs\nD7pXlgWu7R9a156I5KIjByXtf3D6Krw4f1OHy7Tclg2s1haQH7jqeFTbgmzM7EN2svrnncF6X2s0\nYQCYl2nfPxNA+xQzAFjnsu7yn6e0571uiSSed976XQnPnfOP7V+wojHF4zPX+JarIzo7yloBvC4i\nUQB/VdVJzgNEZCKAiQAwbNiwTl4uP4KM8Mv0sl9+CrFMRF1FkIHK1h/pcAg4emhvHD20N95fsyNx\nZwGwVosCjNSeXz1tJJr2tODfLusVd1Y/R/+1vek+qu5zpK3WCOcqWvtbo/Hc4KkM6WP0ydsbKq56\nIDlf9TbbKHJnrmzn4LVfvbgYrZEYhvRJzhN+oC2alfcO6HwN+VRVHQ/gQgDXisjpzgNUdZKq1qtq\nfW1tbfIZiIgKjNWv2aebfx+mfQS2tc7voF7eCz7kyqvfOR3PffOUhOlSZSHBDy8ei7s/OyEr1+zX\n3btLwKuGbA2QGmhbSAMAvvbwbKzZnno9Zy8bdh1Iud/ZFO6sMe9tjeJnzy9yHU1tDQjLhk7VkFV1\nvfn/FhH5F4DjAUzNRMGIKLdE5DsArobR8jUfwFWqmvovWxdl/R3uF7DP2XLlySNw+KCeOGl0+vOh\nM+3QOmPUctOe9oQfzoUxMi2phuxo6rXn/bZYiTaG90vMx71+537c9p+Fga8dML8KgOR0nfs8msbX\n7kj+QuAM3pnU4Z+OiHQXkRrrMYDzACzIVMGIKHdEZDCAbwOoV9VxAMIAPpvfUuVPN7Oq8qljvJdy\ndGuUDoWkIIKxnX0FqSBNwEF86aThrttTDZqLqnsN2RoUNrxf8nSwIFOe2nX83vZ6rBu9eXfy4hEJ\nc5Yz3DPRmRpyHYB/mc01ZQAeVdWXM1IqIsqHMgDVItIGoBuA7HSUFYF+1SHM+tE56Nc9vTm9mTD9\n5rOSElV0hn06kzOrVkdVl7vP6+7b3dmH3M6oIbtfPyxIua5zEEcN6YXHvJc6TsmrGdreuuB3bCZ0\nOCCr6koAR2ewLESUJ6q6XkR+C2ANgP0AXlXVV+3H2Ado1tXVobGx0fe8zc3NgY4rNM3NzVgwa0bK\nY7ZvM1rz58+bB9nY8bqN1/vTkXG8bu93qy2JxdSpb6V1vp4Vgt2tydXATevXuh6/YG77OsaNjY2I\n2QZLNW3dhtXLd7m9DFEFFs7xjqaXHVqOJ5a6z2+27negKo7qH8a8rZn7MrNsbfLo84VLlsYfx2Kx\njP5+M5c1EUFE+gC4BMBIADsBPCkiX1DVh61jzFkUkwCgvr5eGxoafM/b2NiIIMcVmiDlvn/Fu8C2\nrTj66KPjaTTT8rIxUyKT749buduiMeC1l5Kv9bL/TI3PnTQK9761AoCxjvMyc4rT4YeOxrMrPowf\n16OyDM0tERx/XD3wzrT4teS1F41oC6B3nz4486Qx+MOc/7pe68JzGoA3X3Tdd9TYMXhiqXuPqP2e\nZrV+iHlTVqCqPORbk62pKvOdZ91W1g1A4uIXg4eOBMygHAqFMvrz4zxkIgKAcwB8pKpNqtoG4BkA\nJ+e5TJQB6fQbf/a4oQkjy3tUhnHDeYfimjNG4aunj4pvdzZZW5mrUq0bHY1pyrSe4ZDgRxcfjmev\nPSVpn9ta0G7KzAWie1T6j46vcVnO0sm1yTqLg7pYQ86QTKxGQpRHawCcKCLdYDRZnw1gVn6LVBwy\nM0wqe4Iu72h5/9bzcNUDMzFlSRMG9arGp48dEt/365c+xLa9rUlzhq1G7bKw4FMTBqPNI6GG16Cv\nr4wzAvXVp41y3e+2FrQba9BYkDWha6rKAZ/pUW5pQDPZv+/EGjIRQVXfBfAUgDkwpjyFYDZPU+mw\nAugmc3TxMMfIZyuFZI/K9hpyTWWZLUmK4K7/HY8/XZ48z/m0Q/onpce0DKlJHYqqzcVBhvZNHPj1\n9NdPSnhutQaUOdfPdDF6QPekbUG6HuxN4UydSURZoao/UdXDVHWcqn5RVZPb66hLszJrbd5t1ByH\n9unm2G+w15DfueXs+OvCjtq4dfxTXzsJ32g42PO6fs3qVo3XXuMeP7Q3jh3eN/E85ijyIM30E4b2\nwZLbL0hILfrHAAlTWrJYQ2aTNRFRB3Q2O+bsH52TtPJRttz6sbGYMKy373HWPd39v+Px16krMKAm\nsYnZylxlD8jdHTVkNwN7VSGUIkh6LIUcZ/UN29/zp7+ePMShwqwZlweY3lUWFlSWhROycZUHSGLO\nPmQiogJz1JBemLZ8Kwb0TC+blyXdLGCd8eVTR7pu714Rxl5bXmcrNJ1+aC1Od2m+tQKis0/X3oec\neLz70o5OfgHZLcC6ndOqIQcJyNYx9qxdVuAHgJ5VZdjtMgo7ock6w23WbLImIuqA7557KF749qk4\nbGDPfBelQ96+6UxM+/5Zab3GCrDdKhx9wVYN2WMAmdv2v3z+mPhjvybmClvN9dlrT3EdiW0v196W\nxEB6tcsXEiuDWSQWS9oGAEP7JmcOAxIXpohmOCCzhkxE1AFl4RCOOCjYEoiFyCvgpGKtD+xMgRnv\nQw659yG7NVcfOrAm/thvULS95nr0UO+m99MOcR+U1bM6eRpUvIZs6zWwj0j/n2OHYOGGRUmvs+ey\n9huMli4GZCIiAuDfBGs17zpTcFqvswdOO7casr1W7NuHbNZc1aeAfbtX4P4r6jGyf3ec9bv2rGRu\nFXBroQurD3mY4wvKII9UnlaT9Y3nj8ERsi51wdPEJmsiIgKQuH6xxwEAkleNaq8JOw5Xa3t7RLRG\nTNtr0341ZKvmGqSF+OzD6zCqtkfCNrdVpqxkI1at/4lrEqdQeeX9tgZ1nTe2LkBp0sOATEREBt8a\nsnGAs89X49s9asguAdl+rDPD16kH9w+USSuIp79+Mi4+alDS9h5mQO5lNmeXOwak2QeGXWPLUtZi\n1pCzsZQlAzIREQVy/Ehj3q9zFLMVx71GU9ubrCvN5CCpEog9fPUJmP/T8+PPrVW3vnvuoekWGccO\n7+M66rqnGZAfuPI4/OTjY5NGvdsDtD3zl5WpK1NLWdqxD5mIiAD4Nwn/7Uv1WLN9X/LgLZ95yPaK\n87iDemLKkqaUAdlSU1mGwwf1RFV5GKvuuNj/BfZrSnuft1vwrKkyasZD+3bDVae4jMK2BeEhtgQp\nVkAOMrUqXQzIREQEwH/QVHczQDo9evUJ+Oesta6Dp4DEGvIfL5+AOWt2YkBNlW957LXkdL39/bPi\nGcfCLqk0/fJjl9u+RfxP/RD0rC7DDU/Ow75WY0pVkPSc6WKTNRERdcrJB/fHHz47IWkhC2vglL3m\nXFNV3rHlKtM0uHc1jhnWB4B7DTlpLrVD98owfvnJI3HMsN4QEVwwbhDKwxKvdZd79Jd3BgMyEVEJ\ne/Hbp+HEUUbfcKYXS/j3tafgZ5cc4bni1Kja5AUessGtKd1vFaxe1eX43AnD8Mw32pOQhG1BOEia\nzXSxyZqIqISNPagn7vncMTjnrrfwVY/lD91Mv/mshDzQbkbX9sBoxxQku5euOw3RmGLm9GmBr9sR\n9hHdP7jwMLy3aofva9ySiVg17XBIXKdSdRYDcgHi2srBBHmf0h0IQlSK+veoxAe3npfWaw7ySJyR\njmwENTf2CvI1Z4zGNWf4vyZV/uy6mkrf/NwdwSZrIiLq0vyap4OyBnJl4suIGwZkIiKiAKwkIl5p\nNTuLAZmIiCgAa33oIX2yE5DZh0xERGS69wvHJqXRtLREjLSZh9Z5D1TrDAZkIiIqCaP6+0+zumDc\nQM99VgpNe+auTGJAJiKiLm/a98+M9wF31M8uGYcxA1fHE45kGgMyERF1eZmo1R7Uuxo3nn9YBkrj\njgG5C8vkfOZMnSvX84ILcU4350YTkRuOsiYiIioADMhEREQFgAGZiIioADAgExERFYBOBWQRuUBE\nlojIchG5OVOFIiIiKjUdDsgiEgbwZwAXAhgL4HIRGZupghEREZWSztSQjwewXFVXqmorgMcBXJKZ\nYhEREZWWzsxDHgxgre35OgAnOA8SkYkAJppPm0Vkic95+wPY2olyFYtSuM+ke5Rf56kk2ZXWzzLg\nezC8o4XJhdmzZ28VkdUBDi3W33OWO7e6crkDf5aznhhEVScBmBT0eBGZpar1WSxSQSiF+yyFewRK\n5z7tVLU2yHHF+t6w3LnFchs602S9HsBQ2/Mh5jYiIiJKU2cC8nsADhGRkSJSAeCzAJ7LTLGIiIhK\nS4ebrFU1IiLfBPAKgDCAyaq6MANlCty8XeRK4T5L4R6B0rnPjijW94blzi2WG4CoaibPR0RERB3A\nTF1EREQFgAGZiIioABRUQO6qqThFZLKIbBGRBbZtfUXkNRFZZv7fJ59l7CwRGSoiU0RkkYgsFJHr\nzO1d5j5FpEpEZorIXPMef2pu7zL3mCmF/FlO9/MoIj8w72OJiJyfn1J37DNWCGXvyOemEMptK0tY\nRN4XkefN59krt6oWxD8YA8NWABgFoALAXABj812uDN3b6QCOAbDAtu03AG42H98M4Nf5Lmcn73EQ\ngGPMxzUAlsJIqdpl7hOAAOhhPi4H8C6AE7vSPWbofSroz3I6n0fzd3gugEoAI837Cuep3Gl9xgql\n7Ol+bgql3LbyfxfAowCez/bvSiHVkLtsKk5VnQpgu2PzJQD+bj7+O4BLc1qoDFPVjao6x3y8B8Bi\nGNncusx9qqHZfFpu/lN0oXvMkIL+LKf5ebwEwOOq2qKqHwFYDuP+cq4Dn7GCKHsHPjcFUW4AEJEh\nAC4GcJ9tc9bKXUgB2S0V5+A8lSUX6lR1o/l4E4C6fBYmk0RkBIAJML4Jd6n7NJuvPgCwBcBrqtrl\n7jEDivGz7PUzLMh7CfgZK5iyp/m5KZhyA7gbwE0AYrZtWSt3IQXkkqVGe0eXmH8mIj0APA3gelXd\nbd/XFe5TVaOqOh5GZrrjRWScY3/R32OpK/SfYTF+xorxcyMiHwOwRVVnex2T6XIXUkAutVScm0Vk\nEACY/2/Jc3k6TUTKYfyheERVnzE3d7n7BABV3QlgCoAL0EXvsROK8bPs9TMsqHtJ8zNWUGUHAn9u\nCqXcpwD4hIisgtHtcpaIPIwslruQAnKppeJ8DsAV5uMrADybx7J0mogIgPsBLFbVu2y7usx9ikit\niPQ2H1cDOBfAh+hC95ghxfhZ9voZPgfgsyJSKSIjARwCYGYeyteRz1hBlL0Dn5uCKLeq/kBVh6jq\nCBi/w2+q6heQzXLna+Sax2i2i2CMHFwB4If5Lk8G7+sxABsBtMHoV/gKgH4A3gCwDMDrAPrmu5yd\nvMdTYTTdzAPwgfnvoq50nwCOAvC+eY8LANxqbu8y95jB96pgP8vpfh4B/NC8jyUALsxjudP+jBVC\n2TvyuSmEcjvuoQHto6yzVm6mziQiIioAhdRkTUREVLIYkImIiAoAAzIREVEBYEAmIiIqAAzIRERE\nBYABmYiIqAAwIBMRERWA/wcaBNtXxjapoQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2ef128ae10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "J=-1.500, mean score=7.673\n"
     ]
    }
   ],
   "source": [
    "for i in trange(100000):\n",
    "    loss_history.append(\n",
    "        trainer.train_step(sample_batch(train_words,word_to_translation,32)[0])\n",
    "        )\n",
    "    \n",
    "    if (i+1)%REPORT_FREQ==0:\n",
    "        clear_output(True)\n",
    "        current_scores = score()\n",
    "        editdist_history.append(current_scores.mean())\n",
    "        plt.figure(figsize=(8,4))\n",
    "        plt.subplot(121)\n",
    "        plt.title('val score distribution')\n",
    "        plt.hist(current_scores, bins = 20)\n",
    "        plt.subplot(122)\n",
    "        plt.title('val score / traning time')\n",
    "        plt.plot(editdist_history)\n",
    "        plt.grid()\n",
    "        plt.show()\n",
    "        print(\"J=%.3f, mean score=%.3f\"%(np.mean(loss_history[-10:]),np.mean(editdist_history[-10:])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "שנג'ן איירליינס; -> shanghai airlines;\n",
      "קטגוריה:פוליטיקאים צרפתים; -> french politicians;\n",
      "רפואת מין; -> sex marriage;\n",
      "תאוריית שני הגורמים; -> the tallest series;\n",
      "יבגני אריה; -> yevgeny aria;\n",
      "הליודורוס; -> helionorum;\n",
      "לוקיוס טארקוויניוס פריסקוס; -> lucius treekoris;\n",
      "שמחה הולצברג; -> simcha holzeberg;\n",
      "תנור מיקרוגל; -> microleagus;\n",
      "קואורדינטות גאוגרפיות; -> georgian cuisine;\n"
     ]
    }
   ],
   "source": [
    "for word in np.random.choice(test_words,10):\n",
    "    print word,'->',model.translate(word,sample=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from agentnet.utils import save,load\n",
    "save(model.rec,\"2lstm_scst.pkl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "predicted_translations = list(map(str.split,map(model.translate,tqdm(test_words))))\n",
    "distances = map(get_score,test_words,predicted_translations)\n",
    "\n",
    "print \"Mean Levenshtein distance:\",np.mean(distances)\n",
    "print \"Median Levenshtein distance:\",np.median(distances)\n",
    "plt.hist(distances,range=[0,10]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f2eefb39d90>]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYHFW5/7+nlu6efclM9oTsCQkkQELYAiaAGFlExQ1X\nFsUNuer1hygXXPAqKigu1wUVEC4giuhll52AQFbIRjaSTJLJMpM9s/V+fn9UnapTp6u6e9ae7nk/\nz5Mn3dXVVaere77nre95z3sY5xwEQRBE8aMVugEEQRBE30CCThAEUSKQoBMEQZQIJOgEQRAlAgk6\nQRBEiUCCThAEUSLkFHTG2F2MsVbG2Dpp20mMsTcYY28xxlYwxub3bzMJgiCIXOQTod8DYLGy7ScA\nvsc5PwnAzfZzgiAIooDkFHTO+RIAh9TNAKrtxzUA9vRxuwiCIIhuYvTwfV8F8C/G2G2wOoUzg3Zk\njF0D4BoAqKiomDtjxowenpIgCGJosnLlygOc88Zc+7F8pv4zxiYAeJxzfoL9/JcAXuac/50x9hEA\n13DOz891nHnz5vEVK1bkPB9BEAThwhhbyTmfl2u/nma5fAbAI/bjvwGgQVGCIIgC01NB3wPgXfbj\ncwFs6ZvmEARBED0lp4fOGHsQwEIADYyxZgDfAfA5AL9gjBkAorA9coIgCKJw5BR0zvnlAS/N7eO2\nEARBEL2AZooSBEGUCCToBEEQJQIJOkEQRIlQNIK+vOkQNu47VuhmEARBDFp6OlN0wPnw714HADTd\nelGBW0IQBDE4KZoInSAIgsgOCTpBEESJQIJOEARRIpCgEwRBlAgk6ARBECUCCTpBEESJQIJOEARR\nIhSdoOezIAdBEMRQpOgEPZ5KF7oJBEEQg5KiE/RonASdIAjCj6IT9K5EqtBNIAiCGJQUnaBHSdAJ\ngiB8KQpBT6fdgVCK0AmCIPwpCkFPcRJ0giCIXBSHoEsROlkuBEEQ/pCgEwRBlAhFIehJ2UOntEWC\nIAhfikLQ0xShEwRB5KQoBD1JWS4EQRA5KQpBT3OK0AmCIHJRFIIuR+jyY4IgCMKlKARd9tCTVJyL\nIAjCl6IQdDkqT6QoQicIgvAjp6Azxu5ijLUyxtYp27/CGNvIGFvPGPtJ/zXRm4eeIsuFIAjCl3wi\n9HsALJY3MMYWAbgUwBzO+SwAt/V901xkEU+kXctl9a4j+O6j62nRC4IgCOQh6JzzJQAOKZu/COBW\nznnM3qe1H9rmkPJ46O7jD/zm37jntSbEkuSrEwRB9NRDnwbgbMbYUsbYy4yxU4N2ZIxdwxhbwRhb\nsX///h6dLMhyEQ8p84UgCKLngm4AqAdwOoD/B+CvjDHmtyPn/E7O+TzO+bzGxsYenUyutpjwyXJJ\nUIROEATRY0FvBvAIt1gGIA2goe+a5SUl+eZ+g6J+Ik8QBDHU6Kmg/xPAIgBgjE0DEAJwoK8apSLr\ntV/aYoIsF4IgCBi5dmCMPQhgIYAGxlgzgO8AuAvAXXYqYxzAZ3g/ppokpQhdfiwgy4UgCCIPQeec\nXx7w0if7uC2ByBruNwBKlgtBEETRzBSVInRbvOUbgjgJOkEQRHEIulxtUQyKdsTdqotJKgdAEARR\nHIIuBNvQmDMoerQr4bxOlgtBEESRCLqI0MOG5tgvRztdQSfLhSAIokgEXQyEhk3didbboq6gk+VC\nEARRJIIufHMrQrcey1E5WS4EQRBFLOhyVE6CThAEUSSC7lguhu6kLXojdLJcCIIgikLQ046HrjmR\neYIsF4IgCA9FIejJdGaWCwk6QRCEl6IQdJG2GDF1R9wTHg+dLBeCIIiiEHRhs4QNslwIgiCCKApB\ndycW6a7lkiRBJwiCkCkKQfd46HaELlddJMuFIAiiSAQ9JWW5HO1KYNv+dppYRBAEoVBcgm5Yg6Ln\n3v4yEklrG2Mk6ARBEEAeC1wMBuSZooJ4KgWNWSJPlgtBEEQRRegaAwydOds6YimYugZDZxShEwRB\noFgEnXMYmgZDc5t7LJqAqWsI6RoJOkEQBIpF0NMcmmYtcCFojyZh6gymrjl+OkEQxFCmaATd0DQY\nutvc9ljStVzSFKETBEEUjaBrDDAlD10IumW5UIROEARRNIJu6Br0QMuFInSCIIiiSFs87/jhOG5Y\nucdyaYslUVtmwjSYUw6AIAhiKFMUEfrC6cPx2bMnebZZEbqV+RJPccSTaTQd6ChQCwmCIApPUQi6\nIJZIOY+7EimYOrM89GQaP3pqAxbe9hJa26IFbCFBEEThKC5BV7xyU9ccy2XljsMAgD1Hoth+oMNZ\n5YggCGKokFPQGWN3McZaGWPrfF77T8YYZ4w19E/zvESlCB2wBL3MNNAWTaKmzAQALNt+EItuewm/\nfXnrQDSJIAhi0JBPhH4PgMXqRsbYOAAXANjZx20KRBV0Q2cYW1eG5sNdqLYFfdWOIwCAZdsPDVSz\nCIIgBgU5BZ1zvgSAnzr+HMD1AAbM2wgZ3uaGdA3j6svRHks6FsveY5aHLmU4EgRBDAl65KEzxi4F\nsJtzvjqPfa9hjK1gjK3Yv39/T07ncO2iqbju3CnOc1PXML6+HACw42AnAKDlaFSct1fnIgiCKDa6\nLeiMsXIA3wZwcz77c87v5JzP45zPa2xs7O7pPJSFdHz1/GnOc9PQMK6+DADwTms7AGAfRegEQQxR\nehKhTwYwEcBqxlgTgLEAVjHGRvZlw4LQNCtVEQBMjWFcnRWhxzMqLpKiEwQxtOj2TFHO+VoAw8Vz\nW9Tncc4P9GG7shI2NcRTaZi6hoqwgaqIlekiQxE6QRBDjXzSFh8E8DqA6YyxZsbY1f3frOyEDR0A\nYBqWaleGM/slstAJghhq5IzQOeeX53h9Qp+1Jk8iptUPiQUv/ARdI0UnCGKIUVQzRQVibVGRxlhB\nETpBEESxCrptuehZLBcaFCUIYohRlIJeXWYJeHnI+p88dIIgiCKph67yow/OxprmI1g0w0q28bNc\n0pyKcxEEMbQoSkGf2FCBiQ0VzvPKsGXB1JWbONyZAADEaeFogiCGGEVpuahURqx+qa485GxTJxol\nU2mkqKQuQRAlTEkIesQeJI2YurNNXWd0xk1PY/EdSwa0XQRBEANJSQi6WGvUlKoxZkToaY4tdr0X\ngiCIUqQ0BN2e5x/S3dSWREZtF4IgiNKmNATdFnIxcxQA4kkSdIIghhYlIug+lgsJOkEQQ4ySEPTG\nSiu7ZVxdmbNt24EOfO+x9QBAC0YTBDEkKAlBf8+skfj5R+fg+vfM8Gy/+99NAIBo0rsW6fUPr8Y3\nH17jPH963V5s208DpgRBFDclIeiMMXzg5LGoimTOk/rS/Suxcsdhz7a/rmjGQyt2Oc+/8L+rcO7t\nL/d7OwmCIPqTopwpGoTms6rFk2v3OTVfCIIgSpmSiNBzsWlfW+BrlN5IEESpUHKCfsuls3Dm5GGe\nbVtagwU9mkgFvkYQBFFMlJygf+qMCZg5qtqzLZoIjsKzvUYQBFFMlJygA0B7LJl7JxuK0AmCKBVK\nUtDboiToBEEMPUpT0LsVoZPlQhBEaVCSgn71gol576tOOiIIgihWSlLQ3zWtEXd+aq7va0klTVG2\nXH705AZaBIMgiKKlJAUdAMLSYhfDKtyVjLoUz1y2XH6/ZBtW7fTOKiUIgigWSlbQQ7r70aaOqHQe\nd8ZVQfc+33Okq38bRhAE0U+UrKCHTfejnX/8COexKuhqxL7nSLR/G0YQBNFPlKygyxH6FWdOwC2X\nzgIAdCgZMDFF0Hce6uz/xhEEQfQDJSvoYWmxC0PXnEUwMi0X7yBp04GO/m8cQRBEP5BT0BljdzHG\nWhlj66RtP2WMbWSMrWGM/YMxVtu/zew+YUP3PBfrjnbGvRG66qHvPUoeOkEQxUk+Efo9ABYr254F\ncALnfDaAzQC+1cft6jUhw/vRTJ8InXOekYeeSFHaIkEQxUlOQeecLwFwSNn2DOdchLpvABjbD23r\nFWFF0MVC0rKgp9I8w3JJpmnmKEEQxUlfeOhXAXgq6EXG2DWMsRWMsRX79+/vg9PlhxqhG5qI0F3L\nJZnmGZZLkiJ0giCKlF4JOmPsRgBJAPcH7cM5v5NzPo9zPq+xsbE3p+sWaoRu5h2hczyxZi+2DpI1\nRlNpDs6pkyEIIjc9FnTG2BUALgbwCT4IFcfQVcvFjtCltMXnNrTg76uaPfsd7Urgyw+swnm3v4z9\nbbH+b2gWYskUJn/7Sdz+zOaCtoMgiOKgR4LOGFsM4HoA7+OcF0XitqllRuhLtx8K2h0AcKgj3q9t\nykUsad09/Pm1poK2gyCI4iCftMUHAbwOYDpjrJkxdjWAXwOoAvAsY+wtxtjv+rmdvUZE6B3xzOqK\nV5w5wfc9ha6Vzm03KD34boAIghiEGLl24Jxf7rP5T/3Qln7FzXJxLZdYIg1dY5jYUOH7nkILusi4\noQKQBEHkQ8nOFFUxtcw89GgyBVNnTo66SjRZ2BRGUcqXInSCIPIhZ4RezHz7whkYX29F36aRGaFH\n4ymYmuZE7yqFjtATtqAXUs8553hq3T68Z9ZI6Jr/dSIIYnBQ0hH6NedMxuITRgKQ89C9EbqhM08h\nL5lCC3rKzonnKJyiv7XrCL50/yos3X6wYG0gCCI/SlrQZZw89Jgk6Ik0TF0LtFxiiTTaY0nc8dzm\njJWOgli54zB2HuybxB/hoRdyFaUuuwMsdOdGEERuhoygO3noCddy6YqnYOpZLJdkCnc8uxl3PLcF\n/3xrT17nuey3r+Gcn77Y+wbDmuQEFHZQNG53ZFTjhiAGP0NG0EUeekcs03LRWLCHnrAFrT2a6Nb5\ntvXBTNPBUIZAtGEwtCUbqTTHv9bvo1m1xJBmyAi6iNDlyUIx23IJyiKJJdLO2qSxbma8fPPva/K2\naYIYDAtWiw5tsBctu/vf2/H5+1bisTV7C90UgigYQ0jQM6PwrkQKhsYCo7poMuUMmOYr6LXlJgyN\nYXnTYazZfbTnDQaQGAQiKjJtBrvlssteaepge2HLNRBEIRkygu6XyRJLpOwI3f890UTaKfIVS+Y3\nKJhMcSycbhUh29LSlnf72mNJPLzSW1dmMETo4i6jt3cbPSGeTCOeZ0cqxhsMSq0khjBDRtAjpo6K\nkGWflNk2SjSZhqEzRzhVMYgmUs5i07FEvsKSxoRhFQgbGra05O+jf/uRtfjG31ZjTfMR91gDFBW3\nx5KYcMMT+Nf6fRmvCcslUYDO5eyfvIDjb34aj67egwk3PJE100Z8h5qPoLe2RXHXq9vJXydKniEj\n6ADQWBUGANRXhABYImDqGurKreeTGyudfctM3UlrBNxsj1wkUxwhQ8OU4ZXY3Jq/oO+wLYOkJJx+\nvvXB9hieXNu3PrEYwP3l81syXhNWS6oAEXrLsRhSaY6f/muj/TwauG+2CP1f6/bh+4+/jQPthS22\nRhD9zZAU9IbKkLPN1BkWTG3AnZ+ai+sXT3e2V0YMRJMpJ/LLJ0LnnCOZ5jB0DdNGVOGdblguMTv6\nFCUKAK+4C6768wp86f5VONLZd+IkPqPfTFB3ULRw0a0IrBmC7ZR0Fq9fbBvsA7sE0VtKeuq/yrAK\nb4QOuDNIL5g1Eq+9c8DZXhU2EEukHCHLx0MXwmhqDKNrI2hpiyGd5r42gIrwiuWB0JQkTu/79atI\npDh2H86M5HuLOJRf+qawfQo5KJqPUyKuh58tI76XV7YcwLzj6jBJuhMjiFJiSAl6me2h19vCDsAz\nS1SOUKsiBqKJtDMYmE+WixAVXWdojISRSnMc7oxjWGUY2/a3Q2MMEwIqO4rjy4OAckS5ptnKmBHj\nAH3pr4vP6Buhpws3KKqSLcJ27qR8vifxvVz/8BoAQNOtF/VD6wii8Awpy0VM/6+KGBnbAO8qRxVh\nA1FPhJ5b0IQ9YWoaGmx7R/i2597+Mhbe9lLge/0FPVO0xbZ8sz9ysf1AB17ebK31qvtE6ImkHaEP\noOUST6Y9A5jicba7EnH9/CP0wndGBDEQDClBVxeOBrwiLot7xNQRTaacSFiu0hiEky2jMzRUWoKe\n7zJ2wtKRhdovbVFMgoqn+qa2yqLbXsJvXtoKANB8fg3JAY7QU2mOBT9+AQ8t3+VsE1chIbXhI797\nHbc+tdF53mEvLegn6IX0/wliIBlSlsv4+nIAwLAK76CoQLYcIqaGra0dmDnKyqyQSwZ85PevI2Lq\nuPeq+Z7jC5/Z0DVH0A/kOdFFCLnIptl9pAsvbdqfsZ84R3dnruaDn4ceH+BB0aNdCbS2xbDrsFvg\nTHRiso+/rOkQljUdwgljqvH0un3oiAtBz7wugyGfnyAGgiEl6FedNREja8pw8rha3P6stfCynFUi\n++lHOhPoSqTw1xXWZB8hGKk0x7KAtUhFNGtozMmoyVfQVcvlvXcswbFo8F1BfwxS+nno7qBo33Qg\nRzsTWN18BOdMa/R9XZRmkD+fuCbiLkG2Y6594E0AwCR7bIIidGIoM6QsF0PX8L45oz3Wi1wSQM5h\nHlEd8bxX3NJv2Hss8PhC/AyNoTpiIKRreVsuAiFe2cRc3k+Fc45fPb/FmQovs+tQp2MdvfbOgYx9\n/LNchJjmL4p7jnTh0l+/6nx2zjlSaY7OeBJzvv8MPn3XssCOTqRjyp9PRN0Jx/7KFO19do66vMrU\ni5ta8ciqZorQiSHDkBJ0gRyJmrp/hP69S2d58tVbjsVw+zObsLzJPzoH3EjQ1DUwxtBQGcL+btYW\nieUZCQcJevPhLtz+7GZ87t4VGa+d/ZMXccXdywEAH//jUpz9E2+ZX78IPd6DHO67Xt2O1c1H8cgq\n6+7mF89vweRvP4lN+9y8/C4fUQbcCF2eyBW1xxfEXcJhnxz8Tp+67VfevRxf/+tqtOXoHAmiVBiS\ngm54BN3fQ6+OmBn5ysu2H8IWe/anqbtFvdpjSVx59zKndos4Tk15CMe6vGV3/YRY3pZv9oo8KBpN\npPDZPy/Hxn3HHCFUPXZx3CC7CIDvtJ1kD+qhi3OLOjj3vr4DgOWPu+33/5yHfSJ04bCITuVIp3sc\ndWao/LnFa8/4lDQgiFJkSAq6LNxylotakVEI0pjaMiyc3oiuRMqxKRIp7ojHU2v34sVN+/Ez4cvb\nxwnpLEMI/bJl2mPutrwFPeke97WtB/Dchlb89xMbnChWjbY7pHMEZaz4iWyiB8W5RJT8zNsteHhl\ns1sPRroWQZ78YVus/V4Xn1nuGMrtvHz13ABwyvg6AMDBju7Nqj3aleizMYNSpiuewvUPr8bhbl5f\nov8YkoJuBAyEmkrennjN0BnKQzo6YknskJaX23OkC8++3eLYKtUR03N8U9cyhKHDx2rwRK55R+ju\nfs2Hu6zzl5mO36xGrqLTCOkaOgOKXPmd2ymfG+BDf/fR9Tj5+894tgkf+7WtB/GNv612POwu6byJ\npP/xhDj4D256I3RTZygPecf1Y9L7gurc5yrSNed7z+C6B9/Mug8B/HXFLvx1RTPueG5zoZtC2AxJ\nQZd125SET1cidFFy19As4WiLJrH7SBfG1JYBAD70u9fxuXtX4M2dVoVEMatSHMfQWcZg4gsbWzME\nZd9Rt+hUvvnlsviKgVoG9w5AHeAUPnLY1AL9a78IXUTmqQDL5Z7XmpyoWqCKsRhb6JLuToI+p/DQ\n/QY+xbU80mXtUx4yPBH6+PpyT9piUCeUrdCa+G6eWkc2Tb7QmPPgYUgKuhyhG1kidJENY2gaykM6\nWtus6n8zR1cDcMXnja0HAQCtx2Ke45i6liEeN/1zHf751m7Ptr1Hu5zH+VsusqBb3n1rW8wRa9U+\nEhF6xNQ99kvQMQX5FraSOynVv3cidEmk40ERut05tPu0UVxLEaFXhHRPxtK0EVXOACoQbBMFdWhA\n36SDPrhsZ9ZsqFJBxAwcpOiDhSEp6LIb4Z36r0Tohmy5uLf2x4+s8uzXZouPKO8qjmPqGpLpdEba\nXMsxb+bLXjtCD+la3oIuWzlNBzsAAK3Hok5kq3ro7TFLBMOG5hv9AkGCnt+gqHxMNUIXn1+2eoI9\ndDtCjwVH6MKi0jTmafOwipDn3EGpll1Z6qr3hXf+rUfW4r2/eKXXxykWqMz84GFICjpj/mmLqu/s\nCrrmubUfZ884VXHTFplzvGSKZ4hERdjqHNY2H8Xn71uBXYc6UVtuorrMzLvuuhCyaCLlRKwtx6QI\nXfO3XCKmHiho2QZFN7e04e09wVGnHPUHzWKNxvMX9A6fwWNxlyDakUilPeeKmJrHcgm6q8gWofdV\njZyhgPg7Ij0fPOQUdMbYXYyxVsbYOmlbPWPsWcbYFvv/uv5tZt9TZYuqHJUzls1DdwV9bJ2/oAt0\nYbkYluWizlSMJVLYuO8Yrrh7Gf61vgWvbzuIUTVlCBuaI1C5Ku4K8RX++4yRVehKpJy7hMwIXQi6\n1i3LRUS5e49GceEvg6PONlnQAzqMzgBB55zjh09uwDutbc6gqN9dRDyZxp1LtuJVu8yxlWmUQnlI\nxx8+PQ9hU8fRroQTwQfNEFU7tHSa4+p7luOVLfvz7lCDSA8hQ5kW+xt85BOh3wNgsbLtBgDPc86n\nAnjefl5U1NuThlTfXMb10L2Wy9i6sox9p41wc9ZFdGyKCF0Ryr1Ho1h8xytOOt3uw10YXRNByHAt\nFyNLuwA3ChYzJE85zupT19vRq/r+dhGhG3rwoGiWLBdBMpXGrkOdWLXzMHYfcb1/uZMIWipOFlI5\nsj7YEcedS7bhY3cuxZGuYA+9+XAXfvjkRiya3ohPnX4cEsk0ook0PnbqeLx75gjMsK2wb/9jrd3W\nAEFXPn9bLInnN7bi7yubcder233fky+DYWHvgYYsl8FDzlounPMljLEJyuZLASy0H/8ZwEsAvtmH\n7ep36spD2HGwE6YRHGeICN1ULBdRp0Vm2ogqbG5pd/YX/ydS6Qx74dHVezzPk2mO4dVhNB/uckRV\n0wBkSXhJKBH6xbNH4YGlO/HCplYA7oCVQAiuqXfTQ1e2/ezZzU51Rut4Vq59ezS35dLlidAzVUAu\nB+DXFjEIfdncsVi7+yjiqTTSnCNir/v6wVPG4i/Ld2GHPaYQZOuIjuVYNIGqsOFM/vrnW3t895dZ\n23wU00dW+VbuDPpcpYr7Gxs6n3mw01MPfQTnXCxsuQ/AiKAdGWPXMMZWMMZW7N+fWT2wUIiKi3oe\nETpj3gksEXuR6Zmjqp1tU4e7A6XC7jB0DYkUz4hy/eq7VIYNK0IXE4N86qoIykO6I3giQp89thZT\nh1c621VRFZaIqKkCuCs3/fiyE3Hx7FG+ZQdUH3rFjsOe5xHDuhYiouacB3YYXQGDomokber+n128\n39A0hHTLnkqkOMKG+90Mrwo7lTGDarh0xVPYcbADs7/7DO5fuhPHognf/VR2HerEJb9+Fd97bH3g\nPmoHWMqIJQEpQh889HpQlFv5aoFfKef8Ts75PM75vMZG/wp7hUCIWZCfDLiCnuY8YwLLM187Bw99\n/nTn+XHDXF/dO1M07fyRXzBzBKoj/jdFZSHDY7n4LVt38vha/Pyjc2yvPYWHlu9E04EOVIUNVIYN\nnDap3tlXjXBFBJ1Ipx3BnWhXKJw5qgYTGyoyFpYAMiNOufQw4HYcQtA/f99Kz0QpmSAPXe00Gisz\n74AA18oxdeYZzA6b7uPKsOG0JVuEfp9djmDD3mM41pVfrRfxuVYqnZrMULJcnLRFEvRBQ08FvYUx\nNgoA7P9b+65JA4MQ9ENZpi0L0UingfKwd4r5tBFVqLJnhgJAbbn72NDd7JhkKu0I1iVzRmPqCG/K\no6AipHvSFv1Wr7/nyvn4wMljETI0PL2uBd/8+1r8ZfkuDLPHAyY2uD6+GqELkVvTfBQ/sheG+O0n\nT8H1i6fjhDHVjr2kCrgqivWKoIs7iqNdCTy/oQXPvN3i+/kAb4SebSGP4UqlS4HoEExd8wh6xPAK\nuuikgwZFO+MpLNmy3/k8+UbopnONgkW7WC2X1mNRTLjhiay1flTE9aU89MFDT+uhPwrgMwButf//\nvz5r0QBx+qRh+P2SbU4dbcG9V83H6FpLULwRup5xDBmRigi4s09N23IRk2hMnTnCqVJuWy4iXc+v\n8qGoLRMyNOw65A5ICgtovJROGVcWtW6XLBfB8KoIvrRwiuezxlNphAwNj67eg/vf2JFhh1SXmfDj\n9mc2O+coC0iN9EwskkRRFcGROQTd0JnHlgmb7ndTETbQGU8hleaBg6IdsaQTbXfGUxkF1IJwVm/K\nkslSrJbLKnu2851LtmL+xPoce1u49en7rVlEN8knbfFBAK8DmM4Ya2aMXQ1LyN/NGNsC4Hz7eVGx\naMZwvPSNhXjviaM828+Z1ogpth8etsWXc6Ai5N/3vfSNhXj8Kws8r+uOoDMk0u6gqKlrvvnVgDvr\nsT2axCW/etVZixQAPjR3rNUewx1slRGCJts+avpdrhWOHEG397vuwTexdPuhjGg0SLDkrJTPnTPJ\naatMUC0XNUIfUZ3LctE8g5IRxXIBrDz2oDz09ljS8dk746nA2vOq/eQutBGsYN0pMxxENJHClx9Y\nhT1SFlF/U2ffYaplHLKRciJ0YrCQT5bL5QEvndfHbRlwJijRuYoQjRTnKAuI0MUxRGYF4Foupq6B\nc1dMTV1zvGxTqcRYbgv6Pmm2p+DHl83Gd983y8mTV6N8IZ7j6uQI3b98bq7Pqu6ndgRBHZJMXbl/\nFN+Vp4deU+61dQRiMNfQFA/d8EbogDWRKiiQbo8mpSXrgiP0eCrtObYj6FlEO6ikQXd45u0WPLFm\nLxiAX3/8lF4fLx/Eb8uv1nwQ4vfblxH6fa83YXnTYfzy8pP77qBDiCE5UzRf/CyXigBh91guUnEu\nQBIinTnZJhOVzqQ8ZCAckFKoa8yJPAFkRL/iudzpxJNpbGlpw7b97c5zGdXREZ2Eup9qnYgZp6Nr\n/G0RwEoJ9fsbF8eKmFpglstNF8/0fL4q6XMHeejy/hX2WMfRLJHmgfaYI0Kd8WSghx6NK51iHqs3\n9UXpADE5yc9260tO+v4zuO8Na3BY2CfdKYXbHwuH3/R/6zPSeon8IUHPgjMoyi1f+EsLJ+OvXzjD\nd19fy8XYE18UAAAgAElEQVROiRSRaUjXcM5UK9NnfL1X0CvCuidbIxtqDnRE8pBvuXQWxteXI55M\n490/X4Jzb38ZgNeCKTN1rPvee3yPqVZBVO2Q9lgSGgPuURbIlqktN33vw7viKRgac1IOBcKTvuOj\nJ+HqBRM9/vjoWncSl2y5yPvIn7/KziISFRn9kGvpWB66/11HVBmHEJ1d9kHR3oucuObZUlezcd/r\nTVi5I3Nw87uPrsf/vPgOAKudRzoTuOmf1gRwkVrbHctFvCdXOWJi4CBBz4IQOc45GGO4fvEMzBpd\n47uv7OO61RZFhC4G8zT88IMn4JXrF2XYEuUhw3N7n40MD10S+E+dMQEXzR7lqbt+tCuBmCROVREj\nIw2zxh7sPJLjD7o9msSYujJMHV4ZuE9dgGXSlUjB0BlChhqhW4/FhC3ZUhID1IB7i28og8ueCN3+\nXNk+R0ubW664K55CW0CErs4ozVaobNXOwzjameiTLBdRx10tRZEvN/3felz229cztt/zWhN++q9N\nAJBRYqIn0XbKtp6CyhQTAw8JehZCumu55EL+49OkiUWAW2XQ1BnCho5x9eUZ0Xh5SPdEmlnblWG5\neN+neuxvbDvosVL87gSGV1nC2ZpjUev2WBIhe83UoNmSluXif81MTctY+COlWAymdNxRtZllFsQx\nBBEz00MXgq6mf1ZHDKfMMSAGRQMEXbGbYoqHnkhZefvJVBof/M1ruPKeZX1juXBxPXp9qEDE3Y7h\npGK631e+Ebewnoo1s6cUIUHPguOh9/D3KoS1U1otSBBRRLg8ZHii/GxMVtY6VQVaFdqNe9u8gu5z\nJzDczixpPRbNeE2mPZZ0xDQoBbO2wn9QFBAph5pHQNQqlfJxF07LnIxm6Mwj+vLYgRhrEGmJZUon\nWVNuOhk5DZUhdCWCLRdV0F3LxZptO/XGp3DHc1ucjJm1u4/2iaAnB8BDjynzHeSB3qCsH5V8a+UT\nAwcJehbCRv4Ruh9iUFREgPJiGqoIV4Tzj9CvOHOC57naOYh211eEYOoMsWRKEfTMr72+PARDY/lF\n6E76pPX5VN2pChuBmQ+G7X/HfbJcRBkGuUM6bdIwrPyv8zH3OLegp+qhy7V1KhUPPaIMYtdIefSN\nVRF0ZYnQo4rlIrdZDA7fv3Qn2mJiSTy3o+qNGAsHQ111Kq/35ml/OBG6EHSpg32ntS2vY4jvLZbM\nrPnfW4qlauXmlrY+/+y9gQQ9C+KPsqeCLiLZ/3lxq/1cGshTRDhi6J4ZjwCweNZIPPu1czKOO66+\nHPdeNd+J6NXOQQj2iOoIwoaOWDLtESM/q0TTGBoqw76CfuVZE3COHSm3R6UI3T7OqBrLFomYGr5/\n6ays3q9ppxzKHYwQEyEucoRu6gzDKsOea6dO0JIzYZwIvTMgQvcIetjKcglIWwyK0AH3N8E5dyJ+\nQ2NOhN6b4DrlHEPUSuGewmXZyLf8b0ysPesz+3V9lrr3MuJO4pUtB3D8zU/n9Z58KYYSCu+0tuGC\nny/BL57fUuimOJCgZ8EV9J69Xy0yFVR/BLAEVc11P3l8bWCpgHOmNTrWixpxhxxBDyNsaOhKpJBI\ncUdkgkoGD6/2F/Spw6vwH+dZM0qTae6pQgkAo+wUxqnDq/DpMyYAyExyMaRxhbDh76GLOxq5w1HP\nJY4hP5c7kLBhRe+iNHFWQa8MoyOe8tRyl3lmfQv++aa7XKAs6GKgO8W5NLfA/VysF9XCnQJt9jX7\n82tNmPeD57DVTkHNRr6Wjxgkdy0X9xvLtpCJjDyQ2tcLg2RLDR0siLGYFU35l0vob0jQsyAEsLcR\nut9zP3tF3WbkGBUT+6vvi9t/DCOrrRrroraJyAAJKjDZWBn29dCtrBL3HMK/FmIgaslk+6MWJQNc\nD12a+p/2RuhiULkqYrh1caSQV51YJMOYdachygqrnWRNmZuB01AVQirNA+2hh1bswlcfessZJJQ/\nnxh0PdKZwAPLdjqfzRH0XkToYtUlEaG/uMmqO/P2nmOOVcI5x4+e2oC1zUc9781XWMU5WttiOOvW\nF5zjzhhZhXV7jmZ7q0N/im5fjEX0N7pPZ1hoSNCz4N7y9uz9qiDLqyP5DSiqg5VBZWTd/TXP/wIh\nysOrIwgbmuP3igJjQf7uiJoImg93ZZT3Dema544iZLdLRMbHDbNy6kWJAiAzU0J8FifLxTP137uo\nh4iqrzt3qvR+zfM4lKWOfWNV2Fl4Wx1oliP0hgrXe/cbVxCIuxZZZI5IMyofWbXbab/w0HvifwuE\nuDq2jr39Kw++ifN/9jJajkXxwsZW/P7lbfjI773piXlbLlIa6+4jXdh92Lpe8ybUYcPetqxVSAX9\nma7Y25WjBgLnDp4EvTgYV1+OUyfU4dYPnpjX/g9dczp+ctls57kqyLKIiz/40TUR3P7hOQAyxSfX\nwJor6N6OYMZIq077WZOHIWzoTp61E6EHiM3H549HIpXGrXY1RoGa9y3EVRxmWEUIm3/wXnz27ImB\nbRXvERkq4g/2j69sw9ceWu35vKdNrMfjX1ngOZ64K9CYtZ84nmqpANadhqiFk81ykStoNgSU7AVc\nC0KuF3/QZ0alnF/fmwg9pkxgkjvH5sNduORXr+LqP68AkHn3mAgoPSAP3J1yy7NoOtDheb3F7rTO\nmtyAVJrjTbtYVzZS/ehzi+j/gaU78cSavdh1qHPQzSDVBmGE3tNqi0MCU9fwty+cmff+p00ahtMm\nDfO8Xz2eg/0Hf+aUBlxmR7aqdZJteTx5f7UjuPDEkXjjW+dhpL2sXb4R+gljanDK+DpsP+D1ag1N\nidAdcXV9cXWgVf2Ju4KuIaQzxxr4wRMb3PNIkf8JY7wTuIQwi7sece4an+qPsjir1/SEMVZnN21E\npaeCZkNV2LOknlxrZ/2eo1g0Y7jHzmjzSe3zDor2PkIPsh3kcQ71u1Rn+grkYx3qiOMf0tgAAKcQ\n2PyJ9dAYsKzpEBZMbcjaTnUSVTrNfev452L7gQ60RROYPbY2o71iOcGGyhAOtMdx0Ymj+r0kQr6I\nyHwwZbmQoPcj6qQWvx+iHGBleug9i9AZYxhpD1SGDc2p+V6eI0IHLD98RZN3AYeQwTznMHU3Wrae\nZ/9c8j4iy+XtvccyrJ1s66gK8RWlicXzS08enbGvnMaodjSnTqjHph9YS+S+tMldQatBqfNeHjKc\nXPZm247wCnpmZoyuuZ1AX0To+XjU6ncZVFVT3S48dIGwXGrLQ5g5uhrL86iLrs4ujSXTgUXssrHo\ntpcAAE23XuRsUzsLccfVHk2iJqD420Aj2jiYBJ0sl35EjsjlWuWA/4rp3bdcdPv/4K8xbGqS5WLt\nn03QGyrDzrJ2AkPzRuCO5WJ/ilwLWsv7GDpzItBvPbJW2Se4XSJCF9bLsMowln77PHzzPTMy9pUF\nXb1LMjSrcwobOqZI5QvUSF8uwibSEmVB95uMJK8f25sYUlyfuGO5BO+rfpWyEE644Qkn3VGN9tWU\nzH3HomC2nXXqhHqs2nk45wCrajX41cDvKb94foszsC2T72IkKsu2H8LXpQHufEmnOS799at4fE2m\n3SPy8IWg7z7ShSvvXtbjNvYFJOj9iCwmS65f5LuPPEVezU0PyuQQhAPy0D37GLoz809Mi892WHWJ\nOdGOsCeVUFgj1vNsQuwcQ6rl/hl7YpRYNUiQ7Y5E3L3I5xpRHfG9xc8m6HInOXGYWyCtSlkaUF40\noyOWxKZ9bXhug7sak1+Ebq1x2vtFH0T07BwrS8XxDMtFEeHXth70HEvQ6VMGWVh88yfUI5ZMY+3u\n7Nku6h1EVyKF/W0xp8Jnb3hs9R587t4VGduDljfMxXUPvolH3tzt3G3ly8GOOFY3H8XGvZmTrcTn\nT9lf9q+e34IXN+3HYwX0+knQ+5FsWSrj7IhdXmg6w3LJIZRBaYsy8mBmeR4R+jCfwUFTZzA05gh4\npoeeW9BFJ2BoDAunD8cvPnZShvhki/TFrXw+A1Aey0Vpm1/NHcCdYSqQI7n2WBLvuWOJpxKhn4ce\nTaSdP/Jc7UynOb720Fu+VRFFBkoixcF5cFolkFmRUb2mO+06/epgqd+grvgexfhFrhmj6pT/rngK\nF/7yFafCZ2/ZtC/z/D2NfsUd8kafY2ajxb5b9fs+RScpIvQ6OxhSSxC3Hovij69sw65Dnd1rdA8g\nQe9HskXYp08ahseuXYCrznIzOVTLJWeEHpC26NlHOqY6oOiHyCmXMexiXOqKSU6EnqWd1y6agse/\nskCyXKz/z5g8LGPfbBaTsFzy8ZXlQdFc1/DKsyZYVoN0Ta46a6InJm6PZVoJYrq/TCyZcmySbL7q\nX5btxOV/eAP/eHM3rrh7eeZx7Aj9hY2tmPZfT+WwXLzXTI3Edxy0RERNA/Q7pgggxN1KR8wqizDh\nhidwz7+3Z+yv+txRO0IHeh5Jy/ilLh7rSqK1LdrtjBex/sDGvflNmhII28evGqX4jkXHVl8u1in2\nfvZdhzvxgyc2YLuSWdQfkKD3I0JMgoTqxLE1nigx31ougqBBUb99AHdAMVsmgr/lYu0vptWbSsdg\nZjneV86bghPG1EiWi7WvqO4ok4+Hnk8hKI/lkqWzA4DvXDIL23/kDsZ9/d3TcPMlMx3BMzSGdh/x\n9vPQYwnXcsnWzhseWYul9qCj3yeWc8QTKZ61c1D7K3Xwc4cdFeYzUUd8r2LwvCOWdGZD/vn1HRn7\nZ0Tokoeebz2Y7rK5pQ2X3/kGrnvwzW5F66Y9b2HDvu4J+t5sEbq9TVwGUTfoUId3sF900NkCr76C\nBL0fEbewao2WINQvPNdte1DaoveYrtiL6DibQ+JvuVjvq7UjkJCUEy4fV0ZYSU5teDH1X7JV1Nzv\nbB2N+GPJJ0KvCOnuIGoPa9AK33pEdcSppigje+hfXjQZX140GfFUGoft6CzN85twwpg1SByVxFDN\nQNl1OPhWXb3bUqPazS1tnsHabMilF0K6ho54Kuts6QwPPZ5yOv3NLd330fNp48+e3Yyt+4WNlH8e\nvLCc9hyxBHp/WywjF9+PFjtC92tbUum8RXsOKbX4RScb7mbA1hNI0PsR8adW7ZMr7bu/8seZKx1K\n+Mp+k2sEcnaKOrXeDzm6Vd8nFuVwfHnhofsc7/7PnoaHrjndOdfwaisil3cdUxu8jJ2KG6HnJ5Lu\nYhn55ZuIo4q9RdQ1ojrs1GqRkT308pCBqoh1bZ5Yu9fZ/rIy6Au4nqzgaFcCM256Gtc+sMrZpq6U\ntNcn20OgCroqckc6E3hxY2t+gi51tuVhHR0xd13WNLf8/Bv+vgZv7bImHflluVRHhKAHR+hBmSad\nPh1nNqLdEHTR0Ykxhu88ug4Lb3sp0Lp57Z0DaIsmnGvvF0g4g6J2M8Q1Vj10ccdFEXqRU18RwpcW\nTsZ9Vwcv15aNXDVkLj5xNH72kTmOWPoh/4iE9ZNtabPKsIG/feEMZcarN0JXy+b6DYrWVYQ8k6xO\nm1gPAHhHyoD4wfvzm4ELZO+0/GioFG3t3U98RHXE18uVBV3XmO8f65V3L/dMVgKAtwM8XDknPpbI\nX6g0zRp0E6UOPFU1dQ3Dq8L4nxff8V2rVkUexK8IGeiIJx0BTKetzuEvy3fhM3ctA5DpK0cTbqEz\nteMSbG5pw8RvPYk3th3Ehb94Bfe+3uS81p7HAuTq+fLFEXT7fyHUf3xlW8a+Rzrj+Pgfl+LLD7zp\nfA6/6o8JJ23Rm5WkLrTtROgk6MWNWLZuynD/iom5yBWN1pSb+OApY7Pu47FcRISeY9bLqRPqcf7M\nEc5zIYoiQhcdgzhKPnnop02yBH3dblfQThxbk3dnVxbq3k9VROg9tly4a7n4IQunzljg+Iea3hhU\nqnf8MHeeghqhZyOdBub/8Hmc8aMXrHZJUWvE1HDzJTOxuvko/rFqd9AhHGTrrCKsozOW8pQf6LAF\nV1h8iRTH++aMxt+/aK2z2xlPOTn7hzsyP+cX/3clLvj5EgDAb17aik0tbVi5w53EFlQ/piKk47uX\nzMzYnk3QVzQdwqybn3YGacV1Ef8LK3BN89GMTlcI8PrdR52O0i9CdwdF7eJt9j7qAuWOh06Wy9BG\nnYzUE2TL5aLZo9FYFcbVWWquCOSoW0RuIkIXUZjoGPKZij2qpgxnT23AHR89ybM933VUuztgLAR9\n3oS6HHtazBptef7H296/+POtzWNWImPB0ZfqhwetOSrWL02leaDo+6EOgsrWSsjQsHjWSOgaC7wz\nkJE9/4qwHaHbx0tzdyBYfGepNEfE1JwyzgfbY85gshqlAsBT6/Y5j1c0HUIqzdF00B0faA8Q9KU3\nno8rzsr8zaqfvfVYFP/9xNtIptK45fG30RFPOZ87oVgu0WTKKfuslr8VApxMc2dBcT/7U3yX4rqJ\nc8SUO5eBtFxo6v8g49Frz0JNmQnOgQkNFbnfkAP5RzS6NoLlN56f1/s85Wody8USNxGBCEHPt7zw\nfVeflrEtaF1Sle5aLiKLZurwKjTdehEm3PBE1v0XnzAKL35joZPe9ttPzsUfX9nmmU16xZkTcMmc\n0bj8zjc8EXpteSjvCD1o9qVY2/TVLQe6VX8/pkTzssiZugZD1zCyOuKkL2ZDthUqQgY6YpLlwrnz\nWUSEnkynYeia89nlGjO5FhsXFtB2yYIL8tDFrF1Rz0WgRug3/nMdnn27BQumNmKNPSlK/IydCN3+\n3qKJFMbUlmHv0SiOdSVw7QOrMKI6gpsunulk68ipl9kHRb1rq4q1ZsWY2EBaLiTogwy5QFFf4FdU\nKx9kG0VNNRQ/YOHc9GZWZL4/8u7WCPnYqeMwvr7ced+yG8/zzVaRmSh1oCeNq8WvP34Klmx2ve0r\nzpyACQ0VCNvVIsfUluHz75qED5w8xpkaftGJo3DmlGG48R/rAABPrt2HsXXlmNhQgVNuedapq6PS\nFU/h6w+tdmajVoR0dOThe6tRqtxhiO97TF2Zx1a481NzsfdoFLc8/rbH1pNthYqwjv1tMY+gixnH\nQsATKQ7THj9gzF3wYUxtGQ4qqXtBA/zy+qVqhD6poQLbDnQ4wvjqN8/F9Q+vcQYy5bGGGTc95dwN\ndcWTzm9SfOdO9GyLdTSRxpThYacNj6+xBrO/tHCyr5XjZ3+KbSKgkUs1JNPc+btxBb3/LRcS9BJH\n/hEFLersh3fJN+t9l540GjsPduDqsycBkAW954qebydTbnbvpzq8OoL3nzzGfV4VAXowlFEhLW83\nts5aai9kaEDMmoAjVmgSmQ31FSHP3c2Dy3biwWU70XTrRYFiDlhisKbZLVlbU2bmJeiygF9x9zLP\n4Kpox9jaMiyzty35f4scv35TSxseWLrT2V+OQp1BUclycSJ0Q6SQpp1JZ2WmjtY2awBxrN2BRBMp\nR/z9LBgV1UN//LoFntm5EVPHiGo3C0vcnaTT3GNtybNgn1m/D/UVId8IvTpiIqRrnnz25za0OPX9\nZfwidLFNzOj1LNqSSsPUNby+9SBe3NgKxnKvb9AXkIde4gjBrIoYWdf6VGGMOd64EAZT1/D1C6Y7\nhaxEca7eFJvLN0KPdHNQtK8QaXiLpjc61pPfDN0LZo3E8KowrjhrgrPYtUw+GRnyZRSprmNqy/Ju\nqyzmgHuXNabOPYYpLQwivlfxs0ikFA9dsVyEtx92LBfuHKM6YmKXXSdFjP3IIp6tMxN0KFku5SEj\n4/NXht0xjWgijTd3HsYlv37VexypY3jkzd34yO9fdwYsEyludwBWZ1MVMZxcc8DKXlK/q+FVYSRT\nHH96dTtuefxtZ7t8R5NMc8Sl8griul3+hzewYsdh+y6GBJ3oJSIqn9QDP95a6o0F/hDFQGJdRc/L\nmeYboXfn7qIvmTK8Er+8/GT85hNz3bb4zNAdXVuGZTeej8mNlb55+a9tPRB4DvHZ5BsdMf2+O4Ku\nIr42+Rhy1o/osMUi23IaYnnYsnyEMKXS3EnVDBsaookUYsm0086RNREno0QI+mOr9+CO5zYDQF6L\nXAcNisrIg/XRRAqfu3dlxqLWfmUH4tJYQzyVRjSZRtjUUBUxPAW7uuKpjIHssXVlSKatgdY/vbrd\nuZOSbZhoIuWJ0NVU14GwW4BeCjpj7GuMsfWMsXWMsQcZY/nPFCEGBHE72ZMBVlPXsqYkfvO9M/C3\nL5yBWaNrAvfJRb4/9IGIboLO+745oz0evmhzUGfkl/UjR89q5yQ6xC4pQhVWT2O1/0pKP7lsNubb\nuf3Z2g4oEbq82LaIru27ATlCrwwZiCfT6HTWMHV/S2nu5nGPtjuL0dIkMWHp/PDJjbjjuS1IpNI4\n2J47Qs9nYpGcchtNpHw7Cr+7AfmzxZJpxJNpRAwdVRHTI+idyqzdunITFWEDiRTHtBHWALlYclDu\nAOVKm+r5gO6NX/WGHnvojLExAK4DMJNz3sUY+yuAjwG4p4/aRvQBIlq6eHbmQhC50KUKi36YuoZT\nJ2QXlVx0d+S/3qfWzEAT8rFcZPwEXU7PC0lL8AFAVcREy7GYxzMXg4jqwhuCj5w6DoxZdb6D0Hwi\n9JAnQhd2nAmgy5PlUm53KEdscWyPJfHvd6xSvPFk2lnhSAj6qBrr/6qwgWkjvIMVU298CiOzTH4T\n5BOhy7/HpoDMnQM+nYff4iTCcpHLBFsRuvs9jKiOwNQ1pNLcuV6iamLWCD2pRuiDXNCl95cxxhIA\nygEMrkX/CJw+aRiW33i+75T+XJg6A+f9Gxl3x0p5+qtnZ137c6AQf5xBJR38BF0unSoGVQVqLXbA\nFYQpI4JHcnNNVBHiN9pjuWR66OL8suVTaS9XeEjywUVOdzyZdrJmRtcIQbcEe3h1GHXlmZ2QWEDj\nyjMnYlx9Gb732NsZ++SzMPV06XpsC6jFctAnarcGKa0VpYQlE7EtF5nOeNLTsY6qidgrUaWdbBXx\n2WUB74ilHJ9efQ0YuAi9x2fhnO8GcBuAnQD2AjjKOX9G3Y8xdg1jbAVjbMX+/Zm1LYj+pydiDliD\navnUOu8N3VmDcsbI6kEh6OKaCBHLeN3nM+2UBV3pxEQdGAD44Clj8M8vn+UIwoyRVXju6+f4nscv\n6rtAmuEr7ImIqaOhMuysSKR+jqpwZociykn45a8vazqE6x9eAwAYUWN9HyJCry4zfTsocZ6bL5mJ\n0ya6JSHqpIlbf1vZ7Ps+mUUzhuPpr54NU2doDSgv4FfrPZ5MO4XDxAQpK0J3z29oDH9d0ewZ+BxZ\nE4GpMyTT3Jn81Xy4C5xzz6DosWjC69N3o85MX9JjQWeM1QG4FMBEAKMBVDDGPqnuxzm/k3M+j3M+\nr7GxsectJQYcQ2d5TesfaghrIEjQ1Qg9ZN+yO88NDT94/wmOFSIL4OmThuGkcbWOBxvStcDSEX6C\nfuen5+HHl1k1cuRWjKkrg6l7My1Ex1PuI+hiEN1vkQlvG6xIvsKO6EfXlqE8pPvepQhBlauDTuzB\n2M6MkdWIGLqzVOJT/3E2LjpxlPO6n18fT6WdcYljUf8IfaTP9zmiOgJD05BMpZ0JR+2xJI51JT2W\nS1s04fHN1UHRfKqE9gW9+Ws9H8B2zvl+znkCwCMAzuybZhGDAVPXBuxWsZg4ZAvGyBr/DBS5E7z9\nw3N8Fi5h+OTpx+F/PnEKAOCS2a4YiaJit7z/BMw7rg7TR1pi/snTx2ecJ2hAWZxfFu+xtWUZdwbC\nEy73sW7G1JbB1JlnQtLJ42udImsqZ01pwDXnTMJ3L5kFxpgj3jJiVSh5Vu3EhsqM/fIhbGrObNRR\nNRHPXaifFx9PyRG6m08vR+h+d3+jaiIwbKumK5HCcfaA767DnZ568G/uPILmw53O30simfbMz8in\n2mVf0Ju/1p0ATmeMlTPrl3MegA190yxiMKBrLK/1QocawlcOitBFLfWzpgzDZXPHZsxyDdlCfNK4\nWjTdehFOGe/Wm5kxstp57eEvnumI3w/efyLuvvJUz3FEVKwirBR5APFd0xuxYEqDdz/NrX+eeQwt\no5bQ1OGVHj/+ua+/y3ls6hq+feHxjrBWl2UKuoiQ5TIO7z95ND40N3uBOT9EZ8aYZVnlqvXDuXuH\nIM94rZYidL/rOaI6AlOzBrHjyTSm2qUgmg93IpniTjmMX73wDlqOxZxzxFNpT8QeVMOnr+mNh74U\nwMMAVgFYax/rzj5qFzEIsPLQKUJXEXnKQYIuojEREat1aFQBlQU/6JiAWyFQ2BkVShR87ozhntfl\nFL+PzBuH331qrmd/XZlYpAq+Gj1XRUxnUHXK8EpPnRuVqnDmgLFrubifd/aYWvz0Q7M9+31x4WQ8\n8NnMuj8y4q6nOmJC11jGXZDfYLu4Q5AnSE21B1nnHVeHMmU28llThmHO2FoYOnNq4ovibdsOdCCR\nSjvLzglEp2ANosorTw1MhN6rLBfO+XcAfKeP2kIMMkxdAwXowQSlUDolWm1RUaNH9ZqKJd+A7Pn2\nQkxFPXvZ1rj54pn4hG3LCMsl13cnTsUArLrp3RkR6tg6r6VUFTEckSrPUVvHL9oV7ZW9/7CZOYPy\n6gUTcw5+i2sqZi2r17gspCPe5RXRygwPXceZk+uw8ZbFCOkavvrQW5797//s6QCs71H4541VYQyv\nCmPb/g4k0xzlyuessL/LeDLtmaDktyZpf0C1XIhADJ3lrJ3eF7x2w7kDlqfbF/z9i2di9a4jgeIr\n0ttEJK5aLmrpG11juHj2KFwoDez54SzybV8qOUIfVRNxbAhnSr/viqWZMMZ8Oyd1W2XYcDJEcgm6\n3+9GCKqc2aR+7xu+vzivQmzifcLyUO+CykN6xozRqoiS5WKIJRyzr/wlD/BGDB2TGyuxdX87qiJm\nRtKAuC7xFPdG6L2pj9ENiueviBhwaspMJwLqT0bXlvmuZTpYmXtcHa5aEFxTXqTwzbErZ6pC4fen\n/euPn5JT0IWNICJ0eTDTs9Sgj4fuR66aanWKoB8/qto5T0Uoeyzol+WiWkRWG7375VtVMzNCz7Sx\nPmvcIbwAAArQSURBVCAVZwPcNst56DJBCQBy6m4kpGNSY4UVodu57Z5z2J8xoUToRWG5EKXNjy+b\nnXsnIoP5E+vx2LULnFo3QtCrIoZVD6WH1SlFYS0R4cqRrnftWGG5ZFf0XK2Q/eHHrl2AE8fWYKk9\nMzWX8Ipzf/qM4/D2nmNYseNwYH46ANx71XyM68aCLrXK+raq5VIe0nHbh+dg6ohK/OTpTQAkD12y\nXGSCVggzpSi8zNQxY1Q17l+6E69tPYjTJ3mzfsR3HVc89N6UmO4OFKETgYyojgQuwUZk58SxNY7g\niuqEwsLo6d23EGq/zCO/QcBcUwhEWl2Q7stF104ca9XrEVZHrnRWccx3TWvETLtj84vQBedMa+xW\nTvrV9h2SWK4vQ9BNA7rGPJOmMtIWVUEPiKLlCL3M1PHhuWMx0x4cVZMGxFebSHkj9NFZBrv7EorQ\nCaKfEenK9RUh7DjY6aQ1dhfHcvERdLkMgFhwIW8PPWA/P19dWAxmjt5CzrQR0WmkD8dJ5h5Xj99+\n4hTMsIVVFWdRpkDueISgi1ov6sCtiNBNneGr509ztsuiXRayVmhaMLUBb+89lvFdiE4hnnQj9M8u\nmJjVoutLKEIniH4mZSvaMFsge3r7revBi3zLEXo6R+QtEO0I2s9P0MV5cpWEkJcnFB1YX1fMfO+J\no5yo3rSFdY59JyEydGQxrnAEPYaaMjNjYpaY3Xnbh+fgy4umONvlOyLRcYj00vZoEl9aONl53V0w\nOu2sqHTxnNGe/P3+hASdIPoZMe2/vpeCLvCL0OVIVBw+t4cuInl//IpsCbso1/yEj8yzJgvNGFXt\nfN6BSIGNmDpe+sZC3Hv1fABKhC55+MN96huJQeZyZcDX0L0eOuAKemtbDNcvnoH77PM5qxgl+YAu\nDu20dcDORBBDFCHoImukp3ouxGSeT8liWTRyeePufsi6n59oi+nufp2KzOITRqHp1ous84iN0one\ne8JI34UoeoxYDhHe2v/ynYucseVXsO6/LpqJ8fXlzgQtgX+EbkXcYlEP0QlEE2kYGkM85S6UkWsW\na19Cgk4Q/YywQCpDokxtzyS9viKEx7+ywHeGpidCdyLivg+JxRT27lThvPCEUXhg6U7Mlzqi335y\nbpZ3dB8h1hOV9UBN6brUlpkoM3V0JVK+gl5TbuIr503N2K4OigLAKHtBDzHhSNTgGVtXhk372pBI\nUYROECWJiNDDZu//sE8Y4786lCwaZ05uwMLpjfivi47PeizRrWTzth+7doGnLouoGphrUFRmwdQG\nJ1rvL2aNrsGfPjMPZ072li8IKxH69JFVeGvXETR2Y96DJ23RTtdsqPC+/7hhFbj7ylNx6oR6nPmj\n5z0zRUnQCaKEqHCmvFti0B85yXKEXhbScc+V83O+RwhNNktApCsKhOXS33Xye8J5x4/I2CZfl+oy\nE5MbK/HWriOor8x/5Sv5s4prpmkMZ09twLul+vOLpg+3z6l78tDJciGIEuJHHzwRJ46pcdYATfeh\nop8xaRhe33awR4tof/y08TjUEccX3zU59842c4+zKkPO7+XSgwOFPA5g6hrG2FaJmP6fD/KgqHw3\nc9/V/gXEQjrzzBSlCJ0gSoiGyjCuO28qookURtdEcGMOK6Q7/OEz89B0oMMjOvkSNnT85wXTu/We\nhdOH462b341anwyYwYg6Aery08bj8bV78dFTx+V9DJESOSzP9WzFmrGtbVFUhY0efTc9hQSdIAaI\niKnjtW+d16fHrAwbgb56f1EsYg5kCvqomjK88J8Lu3UMkdGT7/KHpq4hkUpj/Z5jON6eJTtQUB46\nQRAlS0+sKBVH0Kvyj9C74ils3Nvm1PMZKChCJwiiZOmLJRRFvvywivwi9FE1Eby4aT9SaY5Zowf2\n7okidIIgSpa+WHGrM25lq4jc81zMHFXtpKrOGTuwgk4ROkEQJUtfROgfmjsWuw514rpzMycd+TFT\nisqzLdPXH5CgEwRRsoiUwfN9ctTzJWLq+NaF+WcmCd+8sSrc5wXJckGCThBEyWLqGl74z3cNWLVD\nwJr+/9Xzp+Zcgao/IEEnCKKkmdQ4sLYHY9566gMJDYoSBEGUCCToBEEQJQIJOkEQRIlAgk4QBFEi\nkKATBEGUCCToBEEQJQIJOkEQRIlAgk4QBFEisJ4uWNujkzG2H8COHr69AcCBPmxOX0Ht6j6DtW3U\nru5B7eoevWnXcZzzxlw7Daig9wbG2ArO+bxCt0OF2tV9BmvbqF3dg9rVPQaiXWS5EARBlAgk6ARB\nECVCMQn6nYVuQADUru4zWNtG7eoe1K7u0e/tKhoPnSAIgshOMUXoBEEQRBZI0AmCIEqEohB0xthi\nxtgmxtg7jLEbCtyWJsbYWsbYW4yxFfa2esbYs4yxLfb/dQPQjrsYY62MsXXStsB2MMa+ZV+/TYyx\n9wxwu77LGNttX7O3GGMXFqBd4xhjLzLG3maMrWeM/Ye9vaDXLEu7CnrNGGMRxtgyxthqu13fs7cX\n+noFtavgvzH7XDpj7E3G2OP284G9XpzzQf0PgA5gK4BJAEIAVgOYWcD2NAFoULb9BMAN9uMbAPx4\nANpxDoBTAKzL1Q4AM+3rFgYw0b6e+gC267sAvuGz70C2axSAU+zHVQA22+cv6DXL0q6CXjMADECl\n/dgEsBTA6YPgegW1q+C/Mft8XwfwAIDH7ecDer2KIUKfD+Adzvk2znkcwF8AXFrgNqlcCuDP9uM/\nA3h/f5+Qc74EwKE823EpgL9wzmOc8+0A3oF1XQeqXUEMZLv2cs5X2Y/bAGwAMAYFvmZZ2hXEQLWL\nc87b7aem/Y+j8NcrqF1BDNhvjDE2FsBFAP6onH/ArlcxCPoYALuk583I/oPvbziA5xhjKxlj19jb\nRnDO99qP9wHo+RLjvSOoHYPhGn6FMbbGtmTEbWdB2sUYmwDgZFjR3aC5Zkq7gAJfM9s+eAtAK4Bn\nOeeD4noFtAso/G/sDgDXA0hL2wb0ehWDoA82FnDOTwLwXgBfZoydI7/IrfupgueCDpZ22PwWlmV2\nEoC9AG4vVEMYY5UA/g7gq5zzY/JrhbxmPu0q+DXjnKfs3/pYAPMZYycorxfkegW0q6DXizF2MYBW\nzvnKoH0G4noVg6DvBjBOej7W3lYQOOe77f9bAfwD1m1SC2NsFADY/7cWqHlB7SjoNeSct9h/hGkA\nf4B7azmg7WKMmbBE837O+SP25oJfM792DZZrZrflCIAXASzGILhefu0aBNfrLADvY4w1wbKFz2WM\n/S8G+HoVg6AvBzCVMTaRMRYC8DEAjxaiIYyxCsZYlXgM4AIA6+z2fMbe7TMA/q8Q7cvSjkcBfIwx\nFmaMTQQwFcCygWqU+EHbfADWNRvQdjHGGIA/AdjAOf+Z9FJBr1lQuwp9zRhjjYyxWvtxGYB3A9iI\nwl8v33YV+npxzr/FOR/LOZ8AS6Ne4Jx/EgN9vfprtLcv/wG4ENbo/1YANxawHZNgjUyvBrBetAXA\nMADPA9gC4DkA9QPQlgdh3VomYPlvV2drB4Ab7eu3CcB7B7hd9wFYC2CN/UMeVYB2LYB1u7sGwFv2\nvwsLfc2ytKug1wzAbABv2udfB+DmXL/1Arer4L8x6XwL4Wa5DOj1oqn/BEEQJUIxWC4EQRBEHpCg\nEwRBlAgk6ARBECUCCTpBEESJQIJOEARRIpCgEwRBlAgk6ARBECXC/wdlfErUNn8MBAAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2ef0372d10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#final learning curve\n",
    "plt.plot(editdist_history)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}