TF Eager JANet model

eager_janet.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From D:\\Users\\Yue\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use the retry module or similar alternatives.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from collections import OrderedDict\n",
    "import tensorflow as tf\n",
    "from tensorflow.contrib.eager.python import tfe\n",
    "\n",
    "# This silent device placement will hurt performance on GPUs, but we are sticking to CPUs only so it\n",
    "# should be fine\n",
    "tf.enable_eager_execution(device_policy=tfe.DEVICE_PLACEMENT_SILENT)\n",
    "tf.set_random_seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Goal\n",
    "We are going to implement the JANet model from the paper [\"The unreasonable effectiveness of the forget gate\"](https://arxiv.org/abs/1804.04849) and try the addition experiment to see if it converges properly. \n",
    "\n",
    "This is not going to be a high performance model, nor will it probably match the performance of the paper. This is going to be a simple exercise in how easy it is to write down research models in Tensorflow Eager without all the baggage of Tensorflow graphs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Components we need\n",
    "For JANet, we need two components :\n",
    "\n",
    "1) Chrono Initializer from the paper [Can recurrent neural networks warp time? ](https://openreview.net/forum?id=SJcKhk-Ab)\n",
    "\n",
    "2) The RNN cell which will do most of the work\n",
    "\n",
    "Lets create the initializer first since it is simpler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chrono Initializer\n",
    "In short, if we know the length of the sequence in the range $[T_{min}, T_{max}]$, then it would be beneficial to initialize the model with a forgetting time in the same range. Therefore, this would be equivalent to initialize the gate $g$ in the range $[\\frac{1}{T_{max}}, \\frac{1}{T_{min}}]$.\n",
    "\n",
    "If the values of both inputs and hidden layers are centered over time, $g(t)$ will typically take values centered around $\\sigma(b_g)$. To obtain values in the desired range above, the biases $b_g$ must be in the range $-log(T_{max} - 1)$ and $-log(T_{min} - 1)$.\n",
    "\n",
    "For the $t$th time step, the forget gate $f_t$ corresponds to $1 - g_t$, where as the input gate $i_t$ corresponds to $g_t$ in an LSTM RNN. Letting the minimum number of timesteps to be 1 and maximum to be $T_{max}$, the paper suggests initializing the forget gate and input gate as follows : \n",
    "\n",
    "\\begin{align}\n",
    "b_f &\\sim log(U([1, T_{max}]) - 1) \\\\\\\n",
    "b_i &= -b_f\n",
    "\\end{align}\n",
    "\n",
    "For a complete explanation of ChronoInitializer, refer to the above paper.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ChronoInitializer(tf.keras.initializers.RandomUniform):\n",
    "    \"\"\"\n",
    "    Chrono Initializer from the paper :\n",
    "    [Can recurrent neural networks warp time?](https://openreview.net/forum?id=SJcKhk-Ab)\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, max_timesteps, seed=None):\n",
    "        super(ChronoInitializer, self).__init__(1., max_timesteps - 1, seed)\n",
    "        self.max_timesteps = max_timesteps\n",
    "\n",
    "    def __call__(self, shape, dtype=None, partition_info=None):\n",
    "        values = super(ChronoInitializer, self).__call__(shape, dtype=dtype, partition_info=partition_info)\n",
    "        return tf.log(values)\n",
    "\n",
    "    def get_config(self):\n",
    "        config = {\n",
    "            'max_timesteps': self.max_timesteps\n",
    "        }\n",
    "        base_config = super(ChronoInitializer, self).get_config()\n",
    "        return dict(list(base_config.items()) + list(config.items()))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RNN Cell\n",
    "\n",
    "The model described in the paper [\"The unreasonable effectiveness of the forget gate\"](https://arxiv.org/abs/1804.04849) is called JANet - a model where the forget $(g_f)$ gate alone is the one of two gates, alongside with the memory $(g_c)$ gate. It doesn't use the input $(g_i)$ and output $(g_o)$ gates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "class JANETModel(tf.keras.Model):\n",
    "\n",
    "    def __init__(self, units, num_outputs, num_timesteps, output_activation='sigmoid', **kwargs):\n",
    "        super(JANETModel, self).__init__(**kwargs)\n",
    "\n",
    "        self.units = units\n",
    "        self.classes = num_outputs\n",
    "        self.num_timesteps = num_timesteps\n",
    "\n",
    "        # Initialize the forget gate with ChronoInitializer\n",
    "        # The memory gate is initialized with zeros\n",
    "        def bias_initializer(_, *args, **kwargs):\n",
    "            forget_gate = ChronoInitializer(self.num_timesteps)((self.units,), *args, **kwargs)\n",
    "\n",
    "            return tf.keras.backend.concatenate([\n",
    "                forget_gate,\n",
    "                tf.keras.initializers.Zeros()((self.units,), *args, **kwargs),\n",
    "            ])\n",
    "        \n",
    "        # Initialize the hidden and recurrent gates\n",
    "        self.kernel = tf.keras.layers.Dense(2 * units, use_bias=False,\n",
    "                                            kernel_initializer='glorot_uniform')\n",
    "        \n",
    "        self.recurrent_kernel = tf.keras.layers.Dense(2 * units,\n",
    "                                                      kernel_initializer='glorot_uniform',\n",
    "                                                      bias_initializer=bias_initializer)\n",
    "        \n",
    "        # Initialize the final layer (for classification or regression depending on the output activation)\n",
    "        self.output_dense = tf.keras.layers.Dense(num_outputs, activation=output_activation)\n",
    "\n",
    "    \n",
    "    \"\"\"\n",
    "    This is required to override a certain issue with how the weights of a model are loaded\n",
    "    by a checkpoint.\n",
    "    \n",
    "    While we could use `model.call(inputs)` directly, the more pythonic way of doing this is\n",
    "    by using `model(inputs)`.\n",
    "    \n",
    "    However, in Eager mode, the ordinary function call does *not* forward the method to the\n",
    "    models `call` method. This bypasses that issue.\n",
    "    \"\"\"\n",
    "    def __call__(self, *args, **kwargs):\n",
    "        if not tf.executing_eagerly():\n",
    "            super(JANETModel, self).__call__(*args, **kwargs)\n",
    "        return self.call(*args, **kwargs)\n",
    "\n",
    "\n",
    "    def call(self, inputs, training=None, mask=None):\n",
    "        # Initialize the hidden memory states\n",
    "        outputs = []\n",
    "        states = []\n",
    "        h_state = tf.zeros((inputs.shape[0], self.units))\n",
    "        c_state = tf.zeros((inputs.shape[0], self.units))\n",
    "        \n",
    "        # Input is in the shape [None, timesteps, input_dim]\n",
    "        for t in range(inputs.shape[1]):\n",
    "            ip = inputs[:, t, :]  # access the T'th timestep\n",
    "            \n",
    "            # Perform the forward pass of the model\n",
    "            z = self.kernel(ip)\n",
    "            z += self.recurrent_kernel(h_state)\n",
    "            \n",
    "            # Split the output into the forget and memory outputs\n",
    "            z0 = z[:, :self.units]\n",
    "            z1 = z[:, self.units: 2 * self.units]\n",
    "\n",
    "            # gate updates\n",
    "            f = tf.keras.activations.sigmoid(z0)\n",
    "            c = f * c_state + (1. - f) * tf.nn.tanh(z1)\n",
    "\n",
    "            # state updates\n",
    "            h = c\n",
    "            \n",
    "            # update our previous state\n",
    "            h_state = h\n",
    "            c_state = c\n",
    "        \n",
    "            # preserve the history of our states\n",
    "            outputs.append(h)\n",
    "            states.append([h])  # here we maintain just 1 state variable, can be more\n",
    "            \n",
    "        # save memory states to global variables so we can access them later\n",
    "        self.cell_outputs = tf.stack(outputs, axis=1)\n",
    "        self.cell_states = states\n",
    "        \n",
    "        # perform forward pass of the classifier / regressor\n",
    "        preds = self.output_dense(outputs[-1])\n",
    "\n",
    "        return preds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup\n",
    "In Eager, if you use only tf.keras layers, then training is a lot simpler. You can simply call the .fit and .predict methods after compiling the models. This is what we will not do for the current model, even though we could, as it is important to explain the limitations of sticking to models with only Keras layers.\n",
    "\n",
    "We will declare some helper methods, along with the dataset loader. Since the addition task is simple enough, we will not be using the canonical way to load data - tf.data pipeline. For this dataset in which we generate synthetic data, the overhead of the tf.data pipeline is more than the benefit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we define some constants \n",
    "import os\n",
    "if not os.path.exists('weights'):\n",
    "    os.makedirs('weights/')\n",
    "\n",
    "# Parameters taken from https://arxiv.org/abs/1804.04849\n",
    "TIME_STEPS = 100\n",
    "NUM_UNITS = 128\n",
    "LEARNING_RATE = 0.001\n",
    "STEPS_PER_EPOCH = 100\n",
    "NUM_EPOCHS = 10\n",
    "BATCH_SIZE = 50\n",
    "\n",
    "CHECKPOINTS_DIR = 'checkpoints_addition/'\n",
    "CHECKPOINT_PATH = CHECKPOINTS_DIR + 'addition_janet.ckpt'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Addition Dataset Generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code reused from https://github.com/batzner/indrnn/blob/master/examples/addition_rnn.py\n",
    "# with some modifications to run on Tensorflow Eager mode\n",
    "def batch_generator():\n",
    "    while True:\n",
    "        \"\"\"Generate the adding problem dataset\"\"\"\n",
    "        # Build the first sequence\n",
    "        add_values = np.random.rand(BATCH_SIZE, TIME_STEPS)\n",
    "\n",
    "        # Build the second sequence with one 1 in each half and 0s otherwise\n",
    "        add_indices = np.zeros_like(add_values, dtype='float32')\n",
    "        half = int(TIME_STEPS / 2)\n",
    "        for i in range(BATCH_SIZE):\n",
    "            first_half = np.random.randint(half)\n",
    "            second_half = np.random.randint(half, TIME_STEPS)\n",
    "            add_indices[i, [first_half, second_half]] = 1.\n",
    "\n",
    "        # Zip the values and indices in a third dimension:\n",
    "        # inputs has the shape (batch_size, time_steps, 2)\n",
    "        inputs = np.dstack((add_values, add_indices))\n",
    "        targets = np.sum(np.multiply(add_values, add_indices), axis=1)\n",
    "        targets = np.expand_dims(targets, -1)\n",
    "\n",
    "        # center at zero mean\n",
    "        inputs -= np.mean(inputs, axis=0, keepdims=True)\n",
    "\n",
    "        inputs = tf.convert_to_tensor(inputs, dtype=tf.float32)\n",
    "        targets = tf.convert_to_tensor(targets, dtype=tf.float32)\n",
    "\n",
    "        yield inputs, targets\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loss and Gradients\n",
    "The following functions simply compute the mean squared error, and compute the gradients of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss(y_true, y_pred):\n",
    "    return tf.losses.mean_squared_error(y_true, y_pred)\n",
    "\n",
    "\n",
    "def grad(model, X, y):\n",
    "    with tfe.GradientTape() as tape:\n",
    "        preds = model(X)\n",
    "        loss_val = loss(y, preds)\n",
    "\n",
    "    vars = model.trainable_variables \n",
    "    grads = tape.gradient(loss_val, vars)\n",
    "    grad_vars = zip(grads, vars)\n",
    "\n",
    "    return grad_vars, loss_val"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training and Checkpointing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch # :  1\n",
      "1 : 1.1137999\n",
      "2 : 1.3734939\n",
      "3 : 1.2632025\n",
      "4 : 1.0217344\n",
      "5 : 1.1007522\n",
      "6 : 1.0640762\n",
      "7 : 0.77063787\n",
      "8 : 0.84670275\n",
      "9 : 0.79458064\n",
      "10 : 0.7586803\n",
      "11 : 0.5092878\n",
      "12 : 0.42632163\n",
      "13 : 0.24331588\n",
      "14 : 0.17305855\n",
      "15 : 0.22396189\n",
      "16 : 0.46569645\n",
      "17 : 0.30956933\n",
      "18 : 0.19394228\n",
      "19 : 0.18876888\n",
      "20 : 0.14756978\n",
      "21 : 0.1357087\n",
      "22 : 0.24186754\n",
      "23 : 0.21103485\n",
      "24 : 0.16359738\n",
      "25 : 0.29180482\n",
      "26 : 0.30941334\n",
      "27 : 0.18169452\n",
      "28 : 0.17933026\n",
      "29 : 0.19284649\n",
      "30 : 0.19533329\n",
      "31 : 0.17026216\n",
      "32 : 0.2019141\n",
      "33 : 0.16495672\n",
      "34 : 0.18687668\n",
      "35 : 0.18738216\n",
      "36 : 0.16143692\n",
      "37 : 0.21796392\n",
      "38 : 0.20093755\n",
      "39 : 0.1940393\n",
      "40 : 0.13964733\n",
      "41 : 0.14817266\n",
      "42 : 0.20116977\n",
      "43 : 0.19677643\n",
      "44 : 0.18117912\n",
      "45 : 0.1921833\n",
      "46 : 0.15685205\n",
      "47 : 0.16924348\n",
      "48 : 0.13538565\n",
      "49 : 0.13495164\n",
      "50 : 0.1701503\n",
      "51 : 0.1312775\n",
      "52 : 0.15106285\n",
      "53 : 0.14762932\n",
      "54 : 0.11621122\n",
      "55 : 0.16249685\n",
      "56 : 0.18139866\n",
      "57 : 0.1195894\n",
      "58 : 0.19839394\n",
      "59 : 0.18842766\n",
      "60 : 0.15654941\n",
      "61 : 0.21005692\n",
      "62 : 0.13576888\n",
      "63 : 0.1602078\n",
      "64 : 0.20151004\n",
      "65 : 0.12443797\n",
      "66 : 0.16430213\n",
      "67 : 0.1604485\n",
      "68 : 0.12348908\n",
      "69 : 0.16183636\n",
      "70 : 0.2118262\n",
      "71 : 0.1975509\n",
      "72 : 0.18941139\n",
      "73 : 0.2185139\n",
      "74 : 0.15553287\n",
      "75 : 0.15934338\n",
      "76 : 0.1537831\n",
      "77 : 0.1566895\n",
      "78 : 0.12518919\n",
      "79 : 0.19793692\n",
      "80 : 0.1583711\n",
      "81 : 0.2284777\n",
      "82 : 0.17029156\n",
      "83 : 0.18526001\n",
      "84 : 0.17859228\n",
      "85 : 0.15501665\n",
      "86 : 0.20986289\n",
      "87 : 0.15498531\n",
      "88 : 0.13561767\n",
      "89 : 0.15113877\n",
      "90 : 0.17337379\n",
      "91 : 0.15633701\n",
      "92 : 0.18732792\n",
      "93 : 0.18975239\n",
      "94 : 0.1320506\n",
      "95 : 0.15652293\n",
      "96 : 0.13723686\n",
      "97 : 0.2221598\n",
      "98 : 0.16244914\n",
      "99 : 0.1635633\n",
      "100 : 0.115488395\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  2\n",
      "101 : 0.15142712\n",
      "102 : 0.19490778\n",
      "103 : 0.15863025\n",
      "104 : 0.16891563\n",
      "105 : 0.1488823\n",
      "106 : 0.19002002\n",
      "107 : 0.13309555\n",
      "108 : 0.1675126\n",
      "109 : 0.11976473\n",
      "110 : 0.13414559\n",
      "111 : 0.17699897\n",
      "112 : 0.15871114\n",
      "113 : 0.18262321\n",
      "114 : 0.15568721\n",
      "115 : 0.14995322\n",
      "116 : 0.22270572\n",
      "117 : 0.18467392\n",
      "118 : 0.19472942\n",
      "119 : 0.13939682\n",
      "120 : 0.15584089\n",
      "121 : 0.15191117\n",
      "122 : 0.20868172\n",
      "123 : 0.19065133\n",
      "124 : 0.20672204\n",
      "125 : 0.15328361\n",
      "126 : 0.15681319\n",
      "127 : 0.17248166\n",
      "128 : 0.15680744\n",
      "129 : 0.1725671\n",
      "130 : 0.15811142\n",
      "131 : 0.1680556\n",
      "132 : 0.14225216\n",
      "133 : 0.14263257\n",
      "134 : 0.14152426\n",
      "135 : 0.12281683\n",
      "136 : 0.15506937\n",
      "137 : 0.25925797\n",
      "138 : 0.12942582\n",
      "139 : 0.17984153\n",
      "140 : 0.15990013\n",
      "141 : 0.103221685\n",
      "142 : 0.14273256\n",
      "143 : 0.16526337\n",
      "144 : 0.19970125\n",
      "145 : 0.1602799\n",
      "146 : 0.18229164\n",
      "147 : 0.1482605\n",
      "148 : 0.16955715\n",
      "149 : 0.15879424\n",
      "150 : 0.15778762\n",
      "151 : 0.18734623\n",
      "152 : 0.10346366\n",
      "153 : 0.17770061\n",
      "154 : 0.13422662\n",
      "155 : 0.0950175\n",
      "156 : 0.15985726\n",
      "157 : 0.16109079\n",
      "158 : 0.19612525\n",
      "159 : 0.26874956\n",
      "160 : 0.10374393\n",
      "161 : 0.14338039\n",
      "162 : 0.17985679\n",
      "163 : 0.18066671\n",
      "164 : 0.11624183\n",
      "165 : 0.15065072\n",
      "166 : 0.1673728\n",
      "167 : 0.17990631\n",
      "168 : 0.17882518\n",
      "169 : 0.19348824\n",
      "170 : 0.1253838\n",
      "171 : 0.20440447\n",
      "172 : 0.17718393\n",
      "173 : 0.12889002\n",
      "174 : 0.20043197\n",
      "175 : 0.12935922\n",
      "176 : 0.16156515\n",
      "177 : 0.1473184\n",
      "178 : 0.20457672\n",
      "179 : 0.19170585\n",
      "180 : 0.12670062\n",
      "181 : 0.11376371\n",
      "182 : 0.12854005\n",
      "183 : 0.1881057\n",
      "184 : 0.19671673\n",
      "185 : 0.1915549\n",
      "186 : 0.123074725\n",
      "187 : 0.14882633\n",
      "188 : 0.15587412\n",
      "189 : 0.1549204\n",
      "190 : 0.15998723\n",
      "191 : 0.15340252\n",
      "192 : 0.13852304\n",
      "193 : 0.1785473\n",
      "194 : 0.18779467\n",
      "195 : 0.14034723\n",
      "196 : 0.1458752\n",
      "197 : 0.16056164\n",
      "198 : 0.13652371\n",
      "199 : 0.19065769\n",
      "200 : 0.20847234\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  3\n",
      "201 : 0.14631604\n",
      "202 : 0.19127764\n",
      "203 : 0.1114826\n",
      "204 : 0.14693926\n",
      "205 : 0.18180187\n",
      "206 : 0.1757907\n",
      "207 : 0.16280074\n",
      "208 : 0.22122839\n",
      "209 : 0.1321277\n",
      "210 : 0.14741613\n",
      "211 : 0.16193426\n",
      "212 : 0.1714394\n",
      "213 : 0.20115525\n",
      "214 : 0.19125421\n",
      "215 : 0.19023961\n",
      "216 : 0.22137111\n",
      "217 : 0.17897464\n",
      "218 : 0.1441868\n",
      "219 : 0.19124608\n",
      "220 : 0.16174869\n",
      "221 : 0.17805603\n",
      "222 : 0.16275558\n",
      "223 : 0.15547869\n",
      "224 : 0.18385968\n",
      "225 : 0.16811806\n",
      "226 : 0.14452225\n",
      "227 : 0.14193898\n",
      "228 : 0.1572187\n",
      "229 : 0.15770416\n",
      "230 : 0.18410131\n",
      "231 : 0.171729\n",
      "232 : 0.113374025\n",
      "233 : 0.17020687\n",
      "234 : 0.21779545\n",
      "235 : 0.13953048\n",
      "236 : 0.13485359\n",
      "237 : 0.11338455\n",
      "238 : 0.1580136\n",
      "239 : 0.15234464\n",
      "240 : 0.16268384\n",
      "241 : 0.18441494\n",
      "242 : 0.14589624\n",
      "243 : 0.13381128\n",
      "244 : 0.14633805\n",
      "245 : 0.14886004\n",
      "246 : 0.16279405\n",
      "247 : 0.19427933\n",
      "248 : 0.16472708\n",
      "249 : 0.15572958\n",
      "250 : 0.14369208\n",
      "251 : 0.20090906\n",
      "252 : 0.14735584\n",
      "253 : 0.16509303\n",
      "254 : 0.15320225\n",
      "255 : 0.16610529\n",
      "256 : 0.15259662\n",
      "257 : 0.15674606\n",
      "258 : 0.16167414\n",
      "259 : 0.15840422\n",
      "260 : 0.17905697\n",
      "261 : 0.16085684\n",
      "262 : 0.1844318\n",
      "263 : 0.17283636\n",
      "264 : 0.18175147\n",
      "265 : 0.19120647\n",
      "266 : 0.11100922\n",
      "267 : 0.16280647\n",
      "268 : 0.2003941\n",
      "269 : 0.16432606\n",
      "270 : 0.16657645\n",
      "271 : 0.12586235\n",
      "272 : 0.1741056\n",
      "273 : 0.15290031\n",
      "274 : 0.18318154\n",
      "275 : 0.15236418\n",
      "276 : 0.12173611\n",
      "277 : 0.16374679\n",
      "278 : 0.17364188\n",
      "279 : 0.15512083\n",
      "280 : 0.13950339\n",
      "281 : 0.16697197\n",
      "282 : 0.17575347\n",
      "283 : 0.10168448\n",
      "284 : 0.15351738\n",
      "285 : 0.1746062\n",
      "286 : 0.20178139\n",
      "287 : 0.13283373\n",
      "288 : 0.18089834\n",
      "289 : 0.15519604\n",
      "290 : 0.14023435\n",
      "291 : 0.12693751\n",
      "292 : 0.1658656\n",
      "293 : 0.13202791\n",
      "294 : 0.14162855\n",
      "295 : 0.2071677\n",
      "296 : 0.19718117\n",
      "297 : 0.17673382\n",
      "298 : 0.16001706\n",
      "299 : 0.14818034\n",
      "300 : 0.17414133\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  4\n",
      "301 : 0.15989183\n",
      "302 : 0.1683334\n",
      "303 : 0.1634808\n",
      "304 : 0.2368403\n",
      "305 : 0.15235831\n",
      "306 : 0.20588325\n",
      "307 : 0.15767582\n",
      "308 : 0.20982075\n",
      "309 : 0.13626595\n",
      "310 : 0.1461687\n",
      "311 : 0.17249995\n",
      "312 : 0.1621354\n",
      "313 : 0.15050727\n",
      "314 : 0.1332962\n",
      "315 : 0.2033216\n",
      "316 : 0.21490146\n",
      "317 : 0.14389195\n",
      "318 : 0.17702124\n",
      "319 : 0.13090472\n",
      "320 : 0.15642883\n",
      "321 : 0.14335191\n",
      "322 : 0.15122734\n",
      "323 : 0.1517923\n",
      "324 : 0.14177383\n",
      "325 : 0.12271037\n",
      "326 : 0.15354306\n",
      "327 : 0.14585589\n",
      "328 : 0.24630192\n",
      "329 : 0.14025758\n",
      "330 : 0.16827598\n",
      "331 : 0.11754834\n",
      "332 : 0.18819757\n",
      "333 : 0.10686378\n",
      "334 : 0.22321922\n",
      "335 : 0.14689402\n",
      "336 : 0.15480909\n",
      "337 : 0.15470141\n",
      "338 : 0.15753229\n",
      "339 : 0.17729716\n",
      "340 : 0.17660692\n",
      "341 : 0.14480473\n",
      "342 : 0.14537503\n",
      "343 : 0.19240357\n",
      "344 : 0.14784722\n",
      "345 : 0.18822044\n",
      "346 : 0.1150868\n",
      "347 : 0.17408083\n",
      "348 : 0.12811284\n",
      "349 : 0.1476301\n",
      "350 : 0.18280677\n",
      "351 : 0.17109857\n",
      "352 : 0.10401657\n",
      "353 : 0.18177532\n",
      "354 : 0.16407819\n",
      "355 : 0.1328129\n",
      "356 : 0.12777561\n",
      "357 : 0.16972435\n",
      "358 : 0.121474735\n",
      "359 : 0.1802315\n",
      "360 : 0.17138006\n",
      "361 : 0.14497444\n",
      "362 : 0.1328586\n",
      "363 : 0.12166094\n",
      "364 : 0.14256704\n",
      "365 : 0.13014755\n",
      "366 : 0.15340823\n",
      "367 : 0.12284045\n",
      "368 : 0.17081615\n",
      "369 : 0.09157196\n",
      "370 : 0.14261661\n",
      "371 : 0.15789641\n",
      "372 : 0.14877114\n",
      "373 : 0.19000767\n",
      "374 : 0.18016705\n",
      "375 : 0.13460127\n",
      "376 : 0.1489218\n",
      "377 : 0.15343726\n",
      "378 : 0.12069669\n",
      "379 : 0.16354252\n",
      "380 : 0.1271183\n",
      "381 : 0.15460804\n",
      "382 : 0.1431959\n",
      "383 : 0.17617328\n",
      "384 : 0.17155463\n",
      "385 : 0.16052125\n",
      "386 : 0.20818532\n",
      "387 : 0.22987455\n",
      "388 : 0.12673472\n",
      "389 : 0.116720706\n",
      "390 : 0.12031011\n",
      "391 : 0.10456814\n",
      "392 : 0.1515617\n",
      "393 : 0.110967904\n",
      "394 : 0.14067063\n",
      "395 : 0.10152199\n",
      "396 : 0.14363636\n",
      "397 : 0.109732\n",
      "398 : 0.15370482\n",
      "399 : 0.23777321\n",
      "400 : 0.17698498\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  5\n",
      "401 : 0.14186129\n",
      "402 : 0.16293369\n",
      "403 : 0.12690492\n",
      "404 : 0.12299286\n",
      "405 : 0.14145467\n",
      "406 : 0.16558231\n",
      "407 : 0.18628284\n",
      "408 : 0.13790326\n",
      "409 : 0.15945606\n",
      "410 : 0.15455613\n",
      "411 : 0.18341199\n",
      "412 : 0.14914699\n",
      "413 : 0.15578014\n",
      "414 : 0.12761003\n",
      "415 : 0.14642216\n",
      "416 : 0.19977523\n",
      "417 : 0.11384835\n",
      "418 : 0.11266821\n",
      "419 : 0.16545223\n",
      "420 : 0.18985954\n",
      "421 : 0.14535917\n",
      "422 : 0.11091176\n",
      "423 : 0.12428767\n",
      "424 : 0.16970477\n",
      "425 : 0.106244504\n",
      "426 : 0.16632122\n",
      "427 : 0.12626189\n",
      "428 : 0.12667525\n",
      "429 : 0.16489224\n",
      "430 : 0.17477669\n",
      "431 : 0.12704808\n",
      "432 : 0.14285174\n",
      "433 : 0.1492689\n",
      "434 : 0.12137743\n",
      "435 : 0.1854474\n",
      "436 : 0.14642225\n",
      "437 : 0.15149617\n",
      "438 : 0.16470455\n",
      "439 : 0.13866177\n",
      "440 : 0.15635641\n",
      "441 : 0.15837139\n",
      "442 : 0.13060589\n",
      "443 : 0.15263985\n",
      "444 : 0.12847354\n",
      "445 : 0.14056405\n",
      "446 : 0.1599973\n",
      "447 : 0.1268467\n",
      "448 : 0.15149932\n",
      "449 : 0.15069152\n",
      "450 : 0.095344715\n",
      "451 : 0.12692823\n",
      "452 : 0.16134982\n",
      "453 : 0.10846744\n",
      "454 : 0.110591486\n",
      "455 : 0.11525818\n",
      "456 : 0.12970857\n",
      "457 : 0.09025608\n",
      "458 : 0.13471705\n",
      "459 : 0.1125316\n",
      "460 : 0.17845346\n",
      "461 : 0.13616502\n",
      "462 : 0.13095608\n",
      "463 : 0.11662094\n",
      "464 : 0.13172145\n",
      "465 : 0.113783605\n",
      "466 : 0.1468534\n",
      "467 : 0.09930524\n",
      "468 : 0.123042375\n",
      "469 : 0.11898151\n",
      "470 : 0.09886933\n",
      "471 : 0.10984399\n",
      "472 : 0.123674296\n",
      "473 : 0.15884851\n",
      "474 : 0.09473106\n",
      "475 : 0.13640635\n",
      "476 : 0.14475779\n",
      "477 : 0.09994569\n",
      "478 : 0.13259643\n",
      "479 : 0.15337387\n",
      "480 : 0.10862705\n",
      "481 : 0.09243172\n",
      "482 : 0.12102955\n",
      "483 : 0.12756248\n",
      "484 : 0.06705673\n",
      "485 : 0.112004735\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "486 : 0.081523485\n",
      "487 : 0.13786794\n",
      "488 : 0.09381305\n",
      "489 : 0.11110353\n",
      "490 : 0.098182514\n",
      "491 : 0.09387477\n",
      "492 : 0.09438832\n",
      "493 : 0.095476516\n",
      "494 : 0.06817464\n",
      "495 : 0.09543871\n",
      "496 : 0.07350923\n",
      "497 : 0.11202486\n",
      "498 : 0.059394952\n",
      "499 : 0.085059434\n",
      "500 : 0.052376606\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  6\n",
      "501 : 0.07366293\n",
      "502 : 0.0977461\n",
      "503 : 0.07575514\n",
      "504 : 0.08485856\n",
      "505 : 0.09932089\n",
      "506 : 0.09266989\n",
      "507 : 0.06556279\n",
      "508 : 0.08859516\n",
      "509 : 0.06656102\n",
      "510 : 0.074160814\n",
      "511 : 0.06465433\n",
      "512 : 0.0768479\n",
      "513 : 0.061056625\n",
      "514 : 0.08437074\n",
      "515 : 0.0702509\n",
      "516 : 0.12187559\n",
      "517 : 0.083756074\n",
      "518 : 0.10581833\n",
      "519 : 0.08884242\n",
      "520 : 0.06761867\n",
      "521 : 0.082420155\n",
      "522 : 0.08711414\n",
      "523 : 0.095563926\n",
      "524 : 0.06527415\n",
      "525 : 0.05995813\n",
      "526 : 0.06321092\n",
      "527 : 0.06498899\n",
      "528 : 0.08993475\n",
      "529 : 0.056264304\n",
      "530 : 0.07335807\n",
      "531 : 0.06862191\n",
      "532 : 0.06770751\n",
      "533 : 0.079412185\n",
      "534 : 0.06537493\n",
      "535 : 0.06720868\n",
      "536 : 0.05963801\n",
      "537 : 0.09473534\n",
      "538 : 0.09055745\n",
      "539 : 0.04787309\n",
      "540 : 0.05338252\n",
      "541 : 0.06361542\n",
      "542 : 0.070608474\n",
      "543 : 0.045933895\n",
      "544 : 0.11329553\n",
      "545 : 0.064792864\n",
      "546 : 0.069107845\n",
      "547 : 0.0710126\n",
      "548 : 0.08020279\n",
      "549 : 0.078129314\n",
      "550 : 0.06880879\n",
      "551 : 0.06652519\n",
      "552 : 0.048686218\n",
      "553 : 0.042917076\n",
      "554 : 0.074027404\n",
      "555 : 0.04995659\n",
      "556 : 0.099824965\n",
      "557 : 0.076344736\n",
      "558 : 0.06369189\n",
      "559 : 0.050895676\n",
      "560 : 0.07711893\n",
      "561 : 0.06045536\n",
      "562 : 0.07851929\n",
      "563 : 0.08319768\n",
      "564 : 0.05253439\n",
      "565 : 0.05752097\n",
      "566 : 0.08168973\n",
      "567 : 0.06884125\n",
      "568 : 0.055582065\n",
      "569 : 0.08008318\n",
      "570 : 0.07677271\n",
      "571 : 0.05553363\n",
      "572 : 0.059638027\n",
      "573 : 0.07149445\n",
      "574 : 0.064126\n",
      "575 : 0.060046457\n",
      "576 : 0.047065362\n",
      "577 : 0.06605486\n",
      "578 : 0.045319017\n",
      "579 : 0.07018263\n",
      "580 : 0.041880213\n",
      "581 : 0.06465956\n",
      "582 : 0.06628401\n",
      "583 : 0.059362903\n",
      "584 : 0.057434283\n",
      "585 : 0.041649923\n",
      "586 : 0.049649864\n",
      "587 : 0.05602722\n",
      "588 : 0.051012743\n",
      "589 : 0.063672274\n",
      "590 : 0.060314998\n",
      "591 : 0.051572323\n",
      "592 : 0.05738103\n",
      "593 : 0.049807936\n",
      "594 : 0.05839544\n",
      "595 : 0.060877696\n",
      "596 : 0.04523378\n",
      "597 : 0.07385907\n",
      "598 : 0.05840019\n",
      "599 : 0.044012066\n",
      "600 : 0.048904534\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  7\n",
      "601 : 0.04777501\n",
      "602 : 0.045295812\n",
      "603 : 0.054420196\n",
      "604 : 0.036967106\n",
      "605 : 0.051850665\n",
      "606 : 0.050646733\n",
      "607 : 0.042383656\n",
      "608 : 0.042954408\n",
      "609 : 0.04449568\n",
      "610 : 0.043360166\n",
      "611 : 0.052302714\n",
      "612 : 0.051783744\n",
      "613 : 0.043744653\n",
      "614 : 0.042835366\n",
      "615 : 0.042453796\n",
      "616 : 0.04722375\n",
      "617 : 0.06010188\n",
      "618 : 0.05015109\n",
      "619 : 0.07052367\n",
      "620 : 0.06358686\n",
      "621 : 0.054715637\n",
      "622 : 0.043309633\n",
      "623 : 0.038239118\n",
      "624 : 0.04742624\n",
      "625 : 0.031015906\n",
      "626 : 0.039885774\n",
      "627 : 0.04358694\n",
      "628 : 0.037214097\n",
      "629 : 0.041405972\n",
      "630 : 0.043837957\n",
      "631 : 0.037334636\n",
      "632 : 0.037288006\n",
      "633 : 0.038870875\n",
      "634 : 0.04596491\n",
      "635 : 0.03999333\n",
      "636 : 0.031630196\n",
      "637 : 0.032454945\n",
      "638 : 0.042408675\n",
      "639 : 0.037228826\n",
      "640 : 0.03534526\n",
      "641 : 0.03495513\n",
      "642 : 0.038005464\n",
      "643 : 0.05133914\n",
      "644 : 0.031021714\n",
      "645 : 0.06480448\n",
      "646 : 0.055641443\n",
      "647 : 0.033429805\n",
      "648 : 0.07204032\n",
      "649 : 0.035686243\n",
      "650 : 0.041231398\n",
      "651 : 0.04310468\n",
      "652 : 0.05581066\n",
      "653 : 0.028777342\n",
      "654 : 0.027452188\n",
      "655 : 0.041834883\n",
      "656 : 0.04338586\n",
      "657 : 0.047400884\n",
      "658 : 0.034692727\n",
      "659 : 0.029945832\n",
      "660 : 0.04812961\n",
      "661 : 0.031002436\n",
      "662 : 0.03977715\n",
      "663 : 0.03498343\n",
      "664 : 0.039574455\n",
      "665 : 0.022422504\n",
      "666 : 0.041366577\n",
      "667 : 0.018463558\n",
      "668 : 0.043331485\n",
      "669 : 0.034572016\n",
      "670 : 0.025236774\n",
      "671 : 0.042498186\n",
      "672 : 0.026631989\n",
      "673 : 0.033520643\n",
      "674 : 0.03707531\n",
      "675 : 0.022538457\n",
      "676 : 0.027253294\n",
      "677 : 0.037255332\n",
      "678 : 0.026199946\n",
      "679 : 0.032597672\n",
      "680 : 0.025695058\n",
      "681 : 0.02061349\n",
      "682 : 0.03616573\n",
      "683 : 0.025639756\n",
      "684 : 0.04018739\n",
      "685 : 0.034272414\n",
      "686 : 0.01547039\n",
      "687 : 0.03216763\n",
      "688 : 0.036220647\n",
      "689 : 0.024898035\n",
      "690 : 0.03058352\n",
      "691 : 0.025642531\n",
      "692 : 0.024973879\n",
      "693 : 0.014782318\n",
      "694 : 0.02827166\n",
      "695 : 0.016179822\n",
      "696 : 0.02347338\n",
      "697 : 0.023594905\n",
      "698 : 0.021632273\n",
      "699 : 0.022306101\n",
      "700 : 0.016289122\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  8\n",
      "701 : 0.020902649\n",
      "702 : 0.025259122\n",
      "703 : 0.025379717\n",
      "704 : 0.022688245\n",
      "705 : 0.016392633\n",
      "706 : 0.01776363\n",
      "707 : 0.017607488\n",
      "708 : 0.033347383\n",
      "709 : 0.021862421\n",
      "710 : 0.015916483\n",
      "711 : 0.030695878\n",
      "712 : 0.031233415\n",
      "713 : 0.016633924\n",
      "714 : 0.033028144\n",
      "715 : 0.021118999\n",
      "716 : 0.023250546\n",
      "717 : 0.02820932\n",
      "718 : 0.017477375\n",
      "719 : 0.021663867\n",
      "720 : 0.03238818\n",
      "721 : 0.018695714\n",
      "722 : 0.025090491\n",
      "723 : 0.022978777\n",
      "724 : 0.025241503\n",
      "725 : 0.023944862\n",
      "726 : 0.019659953\n",
      "727 : 0.02097785\n",
      "728 : 0.01966372\n",
      "729 : 0.023441292\n",
      "730 : 0.020550003\n",
      "731 : 0.018677153\n",
      "732 : 0.020555295\n",
      "733 : 0.0153756095\n",
      "734 : 0.025121372\n",
      "735 : 0.01652127\n",
      "736 : 0.0220072\n",
      "737 : 0.00999939\n",
      "738 : 0.025371652\n",
      "739 : 0.01686145\n",
      "740 : 0.024796648\n",
      "741 : 0.014960512\n",
      "742 : 0.016903607\n",
      "743 : 0.021749733\n",
      "744 : 0.019653985\n",
      "745 : 0.016175628\n",
      "746 : 0.01462836\n",
      "747 : 0.02161517\n",
      "748 : 0.020391822\n",
      "749 : 0.037578925\n",
      "750 : 0.022586687\n",
      "751 : 0.0155431675\n",
      "752 : 0.016204944\n",
      "753 : 0.016131276\n",
      "754 : 0.019290337\n",
      "755 : 0.025681803\n",
      "756 : 0.023121621\n",
      "757 : 0.028586132\n",
      "758 : 0.03171596\n",
      "759 : 0.019623036\n",
      "760 : 0.031834494\n",
      "761 : 0.023419132\n",
      "762 : 0.015691467\n",
      "763 : 0.033500258\n",
      "764 : 0.022730062\n",
      "765 : 0.019922173\n",
      "766 : 0.032994896\n",
      "767 : 0.014091652\n",
      "768 : 0.027716422\n",
      "769 : 0.026400797\n",
      "770 : 0.012892388\n",
      "771 : 0.032470938\n",
      "772 : 0.01726125\n",
      "773 : 0.020152602\n",
      "774 : 0.022208914\n",
      "775 : 0.014932071\n",
      "776 : 0.012937251\n",
      "777 : 0.015766947\n",
      "778 : 0.0132568255\n",
      "779 : 0.0155861\n",
      "780 : 0.017096061\n",
      "781 : 0.013463872\n",
      "782 : 0.018087575\n",
      "783 : 0.024214478\n",
      "784 : 0.010821706\n",
      "785 : 0.010175236\n",
      "786 : 0.026655247\n",
      "787 : 0.011721212\n",
      "788 : 0.02163083\n",
      "789 : 0.014458537\n",
      "790 : 0.011331205\n",
      "791 : 0.015352869\n",
      "792 : 0.014839578\n",
      "793 : 0.012153065\n",
      "794 : 0.020075902\n",
      "795 : 0.021816878\n",
      "796 : 0.022544676\n",
      "797 : 0.018230867\n",
      "798 : 0.016187347\n",
      "799 : 0.013606094\n",
      "800 : 0.024549238\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  9\n",
      "801 : 0.012986608\n",
      "802 : 0.016526671\n",
      "803 : 0.033521313\n",
      "804 : 0.014552559\n",
      "805 : 0.017782925\n",
      "806 : 0.014623911\n",
      "807 : 0.01605843\n",
      "808 : 0.0182314\n",
      "809 : 0.023435459\n",
      "810 : 0.013944712\n",
      "811 : 0.01235519\n",
      "812 : 0.010782383\n",
      "813 : 0.019945942\n",
      "814 : 0.016715635\n",
      "815 : 0.0199111\n",
      "816 : 0.015859725\n",
      "817 : 0.020479776\n",
      "818 : 0.015555834\n",
      "819 : 0.015800904\n",
      "820 : 0.012691012\n",
      "821 : 0.016580176\n",
      "822 : 0.019633785\n",
      "823 : 0.013375631\n",
      "824 : 0.016445559\n",
      "825 : 0.012265226\n",
      "826 : 0.015153025\n",
      "827 : 0.014212745\n",
      "828 : 0.016773904\n",
      "829 : 0.013638898\n",
      "830 : 0.011859362\n",
      "831 : 0.0123380935\n",
      "832 : 0.022636991\n",
      "833 : 0.0189455\n",
      "834 : 0.013149257\n",
      "835 : 0.017037136\n",
      "836 : 0.016768163\n",
      "837 : 0.01088644\n",
      "838 : 0.01632203\n",
      "839 : 0.018193217\n",
      "840 : 0.01501472\n",
      "841 : 0.019435056\n",
      "842 : 0.015862606\n",
      "843 : 0.016691906\n",
      "844 : 0.01857283\n",
      "845 : 0.016344197\n",
      "846 : 0.016242089\n",
      "847 : 0.011747944\n",
      "848 : 0.015716914\n",
      "849 : 0.011858826\n",
      "850 : 0.00779434\n",
      "851 : 0.01895096\n",
      "852 : 0.013304486\n",
      "853 : 0.016563827\n",
      "854 : 0.018470047\n",
      "855 : 0.017850008\n",
      "856 : 0.00965603\n",
      "857 : 0.014101025\n",
      "858 : 0.01428686\n",
      "859 : 0.019248547\n",
      "860 : 0.024916233\n",
      "861 : 0.010775974\n",
      "862 : 0.014567508\n",
      "863 : 0.014941969\n",
      "864 : 0.019449465\n",
      "865 : 0.009467914\n",
      "866 : 0.016918374\n",
      "867 : 0.014448612\n",
      "868 : 0.0090315\n",
      "869 : 0.019191442\n",
      "870 : 0.01305908\n",
      "871 : 0.012039092\n",
      "872 : 0.009064677\n",
      "873 : 0.014514124\n",
      "874 : 0.013841455\n",
      "875 : 0.017515764\n",
      "876 : 0.012291939\n",
      "877 : 0.014437266\n",
      "878 : 0.01352719\n",
      "879 : 0.010692399\n",
      "880 : 0.009278242\n",
      "881 : 0.009847321\n",
      "882 : 0.013665096\n",
      "883 : 0.012355395\n",
      "884 : 0.01276628\n",
      "885 : 0.016411718\n",
      "886 : 0.014977279\n",
      "887 : 0.013307059\n",
      "888 : 0.010135107\n",
      "889 : 0.01614172\n",
      "890 : 0.0103136115\n",
      "891 : 0.010267015\n",
      "892 : 0.013264958\n",
      "893 : 0.025968643\n",
      "894 : 0.011141503\n",
      "895 : 0.012102563\n",
      "896 : 0.018123172\n",
      "897 : 0.023255836\n",
      "898 : 0.009807067\n",
      "899 : 0.01969999\n",
      "900 : 0.013991203\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  10\n",
      "901 : 0.011699881\n",
      "902 : 0.010809604\n",
      "903 : 0.012074033\n",
      "904 : 0.016375568\n",
      "905 : 0.013168372\n",
      "906 : 0.016547907\n",
      "907 : 0.016467936\n",
      "908 : 0.010323469\n",
      "909 : 0.014534895\n",
      "910 : 0.014145259\n",
      "911 : 0.012574308\n",
      "912 : 0.019426407\n",
      "913 : 0.01383512\n",
      "914 : 0.014578431\n",
      "915 : 0.018004967\n",
      "916 : 0.015512292\n",
      "917 : 0.022178264\n",
      "918 : 0.016814796\n",
      "919 : 0.013876684\n",
      "920 : 0.01231194\n",
      "921 : 0.018006697\n",
      "922 : 0.010835016\n",
      "923 : 0.01271348\n",
      "924 : 0.015747424\n",
      "925 : 0.01589224\n",
      "926 : 0.014418066\n",
      "927 : 0.015718684\n",
      "928 : 0.01758476\n",
      "929 : 0.018121056\n",
      "930 : 0.015334339\n",
      "931 : 0.014507584\n",
      "932 : 0.016220575\n",
      "933 : 0.014739513\n",
      "934 : 0.015753098\n",
      "935 : 0.015109981\n",
      "936 : 0.016242303\n",
      "937 : 0.013783309\n",
      "938 : 0.018552832\n",
      "939 : 0.015021067\n",
      "940 : 0.01588847\n",
      "941 : 0.007901014\n",
      "942 : 0.010706764\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "943 : 0.011875818\n",
      "944 : 0.009333177\n",
      "945 : 0.02218783\n",
      "946 : 0.012051043\n",
      "947 : 0.014832691\n",
      "948 : 0.014634831\n",
      "949 : 0.0131747965\n",
      "950 : 0.01689469\n",
      "951 : 0.024831302\n",
      "952 : 0.01531312\n",
      "953 : 0.013977639\n",
      "954 : 0.012559077\n",
      "955 : 0.011145801\n",
      "956 : 0.016348049\n",
      "957 : 0.014868401\n",
      "958 : 0.0115115205\n",
      "959 : 0.0137658\n",
      "960 : 0.0151611185\n",
      "961 : 0.011013189\n",
      "962 : 0.008830121\n",
      "963 : 0.014184743\n",
      "964 : 0.012976608\n",
      "965 : 0.00957146\n",
      "966 : 0.011462526\n",
      "967 : 0.01051829\n",
      "968 : 0.012518496\n",
      "969 : 0.01954012\n",
      "970 : 0.012688293\n",
      "971 : 0.015191065\n",
      "972 : 0.011763955\n",
      "973 : 0.0078075076\n",
      "974 : 0.014996645\n",
      "975 : 0.014906674\n",
      "976 : 0.010484314\n",
      "977 : 0.017806204\n",
      "978 : 0.01658817\n",
      "979 : 0.008808107\n",
      "980 : 0.012971384\n",
      "981 : 0.010011482\n",
      "982 : 0.018633323\n",
      "983 : 0.011638668\n",
      "984 : 0.01172273\n",
      "985 : 0.016627317\n",
      "986 : 0.011784148\n",
      "987 : 0.015244544\n",
      "988 : 0.010136694\n",
      "989 : 0.012415483\n",
      "990 : 0.010214868\n",
      "991 : 0.009691322\n",
      "992 : 0.010831093\n",
      "993 : 0.013265954\n",
      "994 : 0.018168332\n",
      "995 : 0.012348447\n",
      "996 : 0.014728769\n",
      "997 : 0.011506953\n",
      "998 : 0.011384925\n",
      "999 : 0.012256512\n",
      "1000 : 0.011961029\n",
      "\n",
      "Saving weights\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import shutil\n",
    "\n",
    "with tf.device('/gpu:0'):\n",
    "    model = JANETModel(NUM_UNITS, num_outputs=1, num_timesteps=TIME_STEPS, output_activation='linear')\n",
    "\n",
    "    optimizer = tf.train.AdamOptimizer(learning_rate=LEARNING_RATE)\n",
    "    global_step = tf.train.get_or_create_global_step()\n",
    "\n",
    "    # try using different optimizers and different optimizer configs\n",
    "    model.compile(loss='mse', optimizer=optimizer)\n",
    "\n",
    "    best_loss = 100.\n",
    "    generator = batch_generator()\n",
    "\n",
    "    loss_history = []\n",
    "    update_counter = 1\n",
    "\n",
    "    for epoch in range(NUM_EPOCHS):\n",
    "        print(\"Epoch # : \", epoch + 1)\n",
    "\n",
    "        for step in range(STEPS_PER_EPOCH):\n",
    "            # get batch dataset\n",
    "            inputs, targets = next(generator)\n",
    "            \n",
    "            # get gradients and loss at this iteration\n",
    "            gradients, loss_val = grad(model, inputs, targets)\n",
    "            \n",
    "            # apply gradients\n",
    "            optimizer.apply_gradients(gradients, tf.train.get_or_create_global_step())\n",
    "\n",
    "            loss_history.append(loss_val.numpy())\n",
    "            print(update_counter, \":\", loss_history[-1])\n",
    "\n",
    "            update_counter += 1\n",
    "        print()\n",
    "        \n",
    "        # remove old checkpoint that we no longer need\n",
    "        if os.path.exists(CHECKPOINTS_DIR):\n",
    "            shutil.rmtree(CHECKPOINTS_DIR)\n",
    "        \n",
    "        # save the checkpoint weights\n",
    "        checkpoint = tfe.Checkpoint(model=model).save(CHECKPOINT_PATH)\n",
    "        \n",
    "        # Optional : Save the weight matrices in Keras format as well\n",
    "        #model.save_weights('addition_model.h5', overwrite=True)\n",
    "        \n",
    "        print(\"Saving weights\")\n",
    "        print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAEyCAYAAADuoYbuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXecHMWZ/p/qnplNyiiAAgogsohCRJNMBoMzxvY54cP24cM+p5PvsLHhbHMOx2F+GMM54QQGc2djC4zJOYkgEAKhgEBZq7x5Zrrr90d3dVdXV3WYnd0drd7v58NHuz0dqntmmaeffup9GeccBEEQBEEQBEHosYZ6AARBEARBEATRyJBgJgiCIAiCIIgESDATBEEQBEEQRAIkmAmCIAiCIAgiARLMBEEQBEEQBJEACWaCIAiCIAiCSIAEM0EQBEEQBEEkQIKZIAiCIAiCIBIgwUwQBEEQBEEQCRSGegAq48eP5zNmzBjqYRAEQRAEQRDDnOeff34z53xC2noNJ5hnzJiBhQsXDvUwCIIgCIIgiGEOY+ytLOtRJIMgCIIgCIIgEiDBTBAEQRAEQRAJkGAmCIIgCIIgiAQyCWbG2NmMsaWMseWMsfma17/EGFvCGHuZMfYAY2y69JrDGHvJ/++ueg6eIAiCIAiCIAaa1El/jDEbwA0AzgCwBsBzjLG7OOdLpNVeBDCXc97NGPscgO8DuMh/rYdzfnidx00QBEEQBEEQg0IWh3kegOWc85Wc8zKA2wBcKK/AOX+Ic97t//o0gKn1HSZBEARBEARBDA1ZBPMUAKul39f4y0xcAuAe6fdmxthCxtjTjLF31zBGgiAIgiAIghgy6lqHmTH2UQBzAZwsLZ7OOV/LGJsF4EHG2Cuc8xXKdpcCuBQA9t5773oOiSAIgiAIgiD6RRaHeS2AadLvU/1lERhjpwP4dwAXcM77xHLO+Vr/35UAHgZwhLot5/xmzvlczvncCRNSm60QBEEQBEEQxKCRRTA/B2A2Y2wmY6wE4EMAItUuGGNHALgJnljeJC0fyxhr8n8eD+AEAPJkQYIgGoCnVmxBX9UZ6mEQBEEQREOSKpg551UAnwdwL4DXANzOOX+VMXYVY+wCf7UfABgB4A6lfNyBABYyxhYBeAjANUp1DYIghpjX1u/Exf/zNL6z4LWhHgpBEARBNCSZMsyc87sB3K0s+6b08+mG7Z4EMKc/AyQIYmDZ1l0GALyxsWOIR0IQBEEQjQl1+vN5fNlmLF67Y6iHQRCDDx/qARAEQRBEY0OC2efr//cyfvH4m0M9DIIgCIIgCKLBIMHsYzEGh4dW2+PLNuOrdywawhERBEEQBEEQjQAJZh+bMbjSo+mP/vwZ3PH8mqEbEEEQBEEQBNEQkGD2YQxweTzM6bgU8CQIgiAIgtidIcHsYzEGrhHMFccdgtEQBEEQBEEQjQIJZh+LMbiSNmbM+7dKDjNBNBy3P7ca7R196SsSBEEQRB0gwezDGCKT/mxfMVfJYSZ2Ef7w3NtYvbW75u0ZWB1HM3Cs296Dr935Mj7zm4VDPRSCIAhiN4EEs48aybAtTzxUHHKYicant+LgX+98BRfd9NRQD2XAqfp/k+2d5DATBEEQgwMJZh/bilbJEIK56pLDTOw6bOkqD/UQBg3NlAOCIAiCGBBIMPtYSpWMQDCTw0zsAuxO4lHML9idzpkgCIIYWkgw+zCmd5ipSgaxKyBu9tiuEUMmCIIgiF0KEsw+FkM0wywm/VGVDGIXIBDMu8jEvf5ANwUEQRDEYEOC2cdiLNKkhBxmYldid7yv09VNJwiCIIiBgASzj2UxyjATuyzDTTzu6Klg8dodQz0MgiAIggBAgjnAm/QX/k5VMohdieHmMH/kZ0/j/Osf174m7g2G2SkTBEEQDQwJZp9GrsO8tauMA75xDxau2jrUQyEaFLcfDvPQf8LjLF670/iaONdhZqoTBEEQDQwJZh/LUCXDaQDrbuGqreituPjpIyuGeihEg9IfwSzYVSbThQ7z0P9tEgRBELsHJJh9GIuKY1EloxEm/bFdRckQQ4YQkcPto6LLZtfj5oAgCIIg8kCC2ce29JGMRpr0RzqBMDFcRaTuAY9YNkxPmSAIgmhASDD7WIyho6+KzZ19ABpr0t8wMw2JAaABkkMDgs5hHm4VQQiCIIjGhwSzj8WAle1dmPsf9wMACg006Y8g0nD7oZhV/dnRW8HfFq/v54jqQ6LDPLhDIQiCIHZjSDD7qDnhRnKYBWkCgXOOJ5dvzuzALXh5Pfa/4h70Vpz+D44YUsJOf7VvK/jqHS/js799ASvaO+swsv6hm9hHVTIIgiCIwYYEs4+lKI1GKiuXdSLXPYs34MM/ewa/ffqtTOtf87fX0Fd1sWlnXz9G17hs2tmLl9dsjyx7bf1O3L9k4xCNaOBwg0l/+SWz+glfva0bANBTHvobKZ0oDpcN/d8mQRAEsXtQGOoBNAq2ZXCYG0AwC9Kc43XbewAAq7Z0Z9qf5Yur4Tph7NQfPoyusoNV15wXLDvnuscAILJsONCf99C0bSN8LHRjGK6fV4IgCKJxIYfZp5EjGWJoWWVCVj0x3AVzVwM4pINFvybCKZuGn7eh/1zoPptBHeahHx5BEASxm0CC2ceKCWbv0gx2JINzjpsfXYEtnbXHJLIKHXHKw7XCwu5Ef95DVZSyBqrLojutIMM8uEMhCIIgdmNIMPvIiQzOOWz/96qhccmvnngTM+YvqPuEuUVrduC7d7+Or9yxKPe2efOrw91h7i/Pv7UVn/vt8/2qQDFY9Ks1tmHTRvhY6M4rnPTXAAMkCIIgdgtIMPvIDnPV5YHDXDWIpZ8+shIAsK27XNdxCAHe1RcKceH4ZdUHWdcTZ0yCWc+zb27DPYs3oHsXqCLSn+RQzGHOGQEaSPQZZv+1wR0KQRAEsRtDgtlHNmerDoftXxlTa+xiwa+iUa3v17ZwMy35ncloHOd9kC5uEkgv6xFCcle4oehPWTn17BonkGFykRv//SAIgiCGFySYfWxJMVcku85UJaPoK+qyU1/30fEFglq1A0iXCYEzmFHgWf4xnF0gcjAUiOvCh37eZyppb/ntz63GZ36z0LCtqUrG0H8uEh3moR8eQRAEsZtAZeV8IpEMhwdfxhXDs+6SEMx1dpiFSFMnIWZBbJF1RFYw6Y+Uhw7xXuwK1ydtjF+782Xja7FNxZOH/g6qDmgzzHSDRxAEQQwy5DD7yBGIquMGLpbJYS4VvA36qvV1mIU+iDjMWTPJNU76M+W0d3d2xUhGbdtGf2+oSIZmWegwN/77QhAEQQwPyGH2YZFIBof4qjZVyRCRjN5K/Z7X3/zoimB/ckQkb1WA7HWYvX93x0iG6/IgkmJcJxDMgzGi/hEI5hrUrqkM4VDpUflvTl+HmcrKEQRBEIMLCWYfKzLpz5UiGaYMs7dBbx0d5u/e/Xo4HksWzNm2z5viEDcJjdTNcLCouhylFMEsdNuu4GS6tevluMMc7GRozrss36TqWmMnvEYQBEEQAwFFMnwik/4cHjhbjjGSYQMA+mooOfbnl9bist+9kLiOrOVUl+2mR1Zg9dZ4++sww5xx0t9u7DBnOeddqUGGnOtdvqkTf3x+TeZt1RuC4HM0RCdersoOc/z1XSEiQxAEQQwvSDD7sEgdZjcQSY7hy7nkO8w9NQjmL9z2Eha8sj6yTJ3IJGeYg0fQHGjv6MP37nkdH/vFs/EdB41Iso0jzDB7AmXV5i4s29iRbeMGo1x1saO7knl90/saWWeXmvQX/nzWfz+aq/FNkOZokPBynySYdTd/VIeZIAiCGGxIMPuoVTLEl7JpRn69M8zqxDt5PJEn1L666eyrxvaRV+8wxWE+5YcP44xrH825l8bg079eiMOu+rv2NV2kwvTkILJOIJj7N7bBQD7HvE8M4o1LhrZKhuww68vK7QJvCEEQBDGsIMHsI0cgyo4bCBBTBQlRJaNerbFVEWBb8Ul/Wcnc6c9QJeOFt7flOl4j8Ogb7cbXdNejmqE1XjDpr8EU8x0LV+OLt70YWdafIarXZ6gjGfLnMXHSHwlngiAIYpAgwewjT7KT6zCbHt3X22FWXUFtlYwUzy/vI3VThvm9P3mybjcCjYBOdOWJZDSaLvvqH1/Gn15aF1kmznFnb/zJQ9Sx1UUcVIe5HqOsHfnzqHWYxWTMQRoPQRAEQZBg9olGMtxAnJojGX6VjDoJS1XAWZEMc7g8SSSwnK1LkuowD+Vj75292bPIWdC9hXkm/TVqBMDN6MR2SfEd3amYzm6oHFz5XJLG26BvC0EQBDEMIcHsI0cyKi4PXKw0YVWvsnKxSX/aOszJ4i1sjZ3tmEIwO5p4wmBVzrju/mU46Jt/C35/Zc0OHPqtv2PBy+sTtsqHzpnPcn6NPumvq5wihP1lct5d56zHq2QMbYZZLnOon/TXmO8HQRAEMXwhwexjcphNwkos7xugSIauDjPn2YRe9gyz96+uDnOGiG9duPb+N9BdDm86Xt+wEwDw4Oub6nYM3fXIJpi9fxsswhwgxy+0LaSFw1xOXi/eGtu87mAgH1d37cPGJQ36xhAEQRDDjkyCmTF2NmNsKWNsOWNsvub1LzHGljDGXmaMPcAYmy699nHG2DL/v4/Xc/D1JOIwGzLMmzv7sGj1dm+5L6YGLJIh12GWH70nCNn8dZiFw1xbxncgaC569a1rva5JGV35tSztwPN2WBwIVrZ3xo5f8vPzHVJ0RfceikWVakom2HB6Q3Xa0Qyz+bwIgiAIYrBIFcyMMRvADQDOAXAQgIsZYwcpq70IYC7n/FAAfwTwfX/bcQCuBHAMgHkArmSMja3f8OtHrA6zEMzSt/O7rn8cF97wBIDwi7y34mBrVxnX3veGsY22ic/+5nm829+fKoRNVTKyRDKyIg6hE4+D3cxEHK+/glk3bs69GtO/fGJVsCxL5Ys8ZeWWbezAls6+zOPMwvNvbcVpP3oEv3vm7cjytibvGu3skZ3j+Pa6DLbWYVZusFjCuoOBk+IwyxElgiAIghgMsjjM8wAs55yv5JyXAdwG4EJ5Bc75Q5xz0XruaQBT/Z/PAnAf53wr53wbgPsAnF2fodcXtQ5zMOlP+lZev6M3+NkJHnc7+NHfl+K6B5bh3lc35jrm317dgJeEYx1zmPWtsbM4v/kzzLXnRF2X10Vc//DvS/Gvf3wZzUW/+kiN2XCTwProz5/BVX9dEizL4jA7GsFp4oxrH8Xp//VI6npLN3TgfTc+ie5yvJqFyvJNnQC8XLdMa8nraC87zDonVixKiziEbbW9zwNj5nUHg+jnyewwZxlexXGxo6e+k0gJgiCI3Y8sgnkKgNXS72v8ZSYuAXBPnm0ZY5cyxhYyxha2t5vr6Q4ktnQlKo4bfCmnZZi3dpUxoskTMKu2dNV8/CTHUy4rl8UZzapzWILDnFUwn3/949jn3+7OeEQzNz68An9YuDqY7FhruT59lheRnDSQsUpGzkl/2zJ0Gvzu3a/h+be24Zk3t6auKx5Y2Hb00cHIZiGY6+Mwm+6whsxhdpMFfnBzkGF4X7ztJRz2bX1DG4IgCILISqGeO2OMfRTAXAAn59mOc34zgJsBYO7cuUPyLR2NZPDgS1n/iJ8HYmJLZx8mj2kBAKzZ1pN4jGUbO/DLJ1dpX1OPI/ZfddzAaeQ82WGWJwdmgdWhSsaS9TuzHSwj4vxqjWSYyqu1+FGP4Dg5IhlDl+X13hdbydq0+Tdocvm9pEl/8qnqbrjURUGVjKGa9JdShznUy+njU1vQEwRBEEQtZBHMawFMk36f6i+LwBg7HcC/AziZc94nbXuKsu3DtQx0oFGrZCQ5zC4PM8ebO8tB178127pj68pcfttLeM0gMFUhLMTOD+5dil9JIjtJ6OV1BJMyzINVJSN2XH8o9c4wt5SigjnPpL8BcVoz7FKci5xnB4BW/1w6Uqtk+P9mdWx9gkjGEH0GnBRHnMrKEQRBEINNlkjGcwBmM8ZmMsZKAD4E4C55BcbYEQBuAnAB51yuB3YvgDMZY2P9yX5n+ssajliVDP9nTcU1VF03+FLv7KsGj/vXbk92mFsV0SajOn/icbz66D5JK7g5nDdAyjBrTrKeVTKef2srtneXta8VFDEoroOIZPzqiTcxY/6CzBMqTdEE1WHOIrryTPrLSp6JmULUW8pGostkZ1pDEp3DnKPqRCNEMpKqeuQZHrXRJgiCIPpDqmDmnFcBfB6e0H0NwO2c81cZY1cxxi7wV/sBgBEA7mCMvcQYu8vfdiuAq+GJ7ucAXOUvazhkUVJx3FBs+N/OD7wWTuhz3GiWeFOHNxmwp5zsiopH6TpUgSqOL+tJjmSHWc12dperiUIhKcNcryoZnHO878an8JGfPaN93VIEsxMIZu9aXnv/MgDAxg5zBQr5HE2RA1Uw62pP67bz/h2iaIJ/3IKSYRbL5ZbX2vbfQaQk2bFVlzTSpL8khznP8EgvEwRBEP0hUx1mzvndnPP9OOf7cM6/4y/7JudcCOPTOeeTOOeH+/9dIG37C875vv5/vxyY0+g/snDzMszyzxyX3LIw8rr8Rd7uizlZwOgYmSCYVQHnGNzFJOdX1svrtvfgoG/ei1sMmWl5/eseWBbUlxbUSySK83h1nT6KEnOYlQxzm+/Kr09w7ytOmiDkaC7lzzCn1WHu6K3grkXrUvdTK8Gkv9g18v7tkyqJJFW/kD8zSU60YKgzzKnvTeAwZx8f6WWCIAiiP1CnPx9Zk/RVnLCsnMuxaks0m1x1eCSqsbnTixv0pQhmUT9Xhyr0hGaICGaesZoG53h7qzfmu1/ZkOmYf3x+TeS1ejnMYawg/tpdi9bFqleIMfX4gllkj5PiLnIJOn3rZ6BVnfRXh0jG/P99BZff+iKWGG4GksgSmzFN+hOfgTSHOWskQ1001A5zWlWPWm7mKPdMEARB9AcSzD6yJNnU0RdMeHI4x/NvbYusW3VduC4POq6VfcHW2VfFso0dxmOMaCoaXzNVyYjq5eSax/JLWbr+yeu3NuV3YLNgyuECwOW3vhhbprajFjEWuQa2ityeXJ955bFYg64ySHws0ViOyjpfxPdU0msq14K4FmpsRXw25Bu0pI546XWY9eenLn982Wa8vGa7dt16IsfVEzPMOfZJgpkgCILoDySYfeSv07Xbe4LfXZfHOrg5frMOIcLkSMAZ1z5qFJsjcjnMeqGZpGOFaOIIS8Yl6QRZZI0oFZTXzNvlQUwoVEWfCfU6iGu7NqFkn1xR47FlmzX7jEde3t7SnTpJU1eWTSa4Kcl4rRav3ZHrugoXPOYwazLMujmRukhJUlk59Z5GfS8++vNncMH/eyLb4PtB1U12zmsRv6SXCYIgiP5AglnD+h29gcioujwWG6g6XoZZ5G8rilqpGtzLom2+3KrgCSIZlro8SyQjWzUGWdirExLrVSWjaogVmFDPb4dfXUOuOawiZ3m/csciPPJGtPkN5zz2nnzrL0twwjUPJo5FN2lOP+bElwEA9y3ZiPOvfzwYW5aSbcIFj03687ftS4lkBILfjS+TibXGznCzNZDIY9QNgUuf81r2SRAEQRB5IcHsI3+frtveE3zBOi6PtTF2/El/ov5yRXEvTRUYkr6yY5EM/3cmhUU410clbnlyFW56ZEVk0p/MP/z8GcyYvyC2nbyrtiYbY1rDyMhgZJh1qMfd7rc1ThqOev037YzGNziPr5NpLEGkIWW9DNdq8dpoe2vTTVV0Hd9hNkQyyimRDM6j63s/x49j0pIDJTKXb+rAjPkLYjc2gmgkw+yI56HicBz/vQfw15cHbpImQRAEMXwhwewjfwd3l52gzbHLNQ6zH8kQjnHcYc6WCU16TZ9h1ju/V971Kr53z+tSfVouZZj1MQX1mLZvZc+a0JY61jwEgjlzJCP8ua/qBNc+aTyqYGWaCEPFcTF3+lj88bPHZRoHgMytsbNcK7nJCJCxcYoQzIZIRnqVjPj4kyYHAl68pduv72wa4rYufU3trCxa7d08/OnFWP8j77ipnf7yfza3d5exbkcvvvGnxbm3JQiCIAgSzD7iS1iUfuvwIwCOy2P1lVdt7sKyTZ1ShjkqmE2OY5JGEtv88bPH4YA9Rwa/y+LP5TyWQZVzuLIYEpsliQv5Jde/CRATGfM6zHLd6lfXhW6qyDCrLqnxGknLd/bIjTmi62/u7MNlv3sBHb2VmAhUtbnIMBdshvEjmjKekVwlI/laqO+/jg4lUpLl+laDz4AyLn/TtCoZ4aS/cFmaY3vujx/DQn+Sq+m8j7j6vrShJyKqxXT16SdLRjv9xV+v1WEG4p9DgiAIgsgCCWYFIYJFrMJxObqUSManf70QHb3V0GFWysmZutIliVchEiyLwWJMKisXruPyuFiQH/XLk/5M8QwZ1XnkPDz/pPJ12vH76//k4eU478ePB3WdK370QJ282GNofS2LJTm2oArM6+5fhgWvrMedz6+JvaYei3OOiuuiaFu5BJOcCdchbmbSygkCGoc5S+MUw3vAA4dZFsya7bUOs25/4c8r27tix6k3rf4EU9NnQHbf9QJfP65NHb3GayZuanTVWgiCIAgiDRLMCgVfBIsvbUcTyRAU/RhDWc0w1xLJkB6/2xbTVsno7qvGmqOIChHeNuljMI3H4dGYSd5Jf+J4L/mP2zf4OWJTAxY1F64dk2sWesKdbGsqxK5rvNqD7zBbLDaBLok0h1nsKYtg7uxTIxkZytoZBHv2DLMYv7RPbZWM/E9E+oOYLGtymN2E9x3Q38Ds6KngHf/5EP6+RF93XFyrtBum9Tt6sGZbd+I6BEEQxO4HCWYf8SWsdp7TRTIExYLvRrv1i2TYFoPF5LJy4TrLNnXist+/ENlOOJU2C0U2uFzhwXzMSEMLl8PhkmDOqZbCSZJ+ZQelgohaIKS3rBeMpvyqKgg7fLE1srkQqzCiZpi5n2Eu2lbmah2AXrCubO+MPUFI6/C4fFMntiq53yw3NI500xZd7v2bXiXD+zdra2yTMK834txNN6LyZ09XR1w3rh3dFfRVXbR36vPV5YwO83HfexAn/udDiesQBEEQux8kmH3EF3OsIoHL0WX4Yi/4DrP6eN2Uac0y6c9iDEyKZKjiT0WIdcaiVRECwZywLVfcXM45inZtJcVCR977XUzyC24EVIfZ0OxDFktZHOaWUtxhVk1E7o/PFMl4eOkm3Prs28Hvp/7wYdzw0PLYpL+3t3TjtB89gh/+/Y3I9vLkO5X2jj6c/l+PYMn6aDfATBlmR+9wZ41kBII74yQ6U7fJeiNuANSoU3hc+U5J93p8WdnxJ4f6VW3U2umiuQ1lmAmCIIhaIMHsI76j1VrJDufoMXyxB53+Mk76SxKhYhe25UcyMpZjE5OZLMYiwscJ7UXNsTjuX7IRjssxZ8pofxtvubgJyOswi8l9amWHcOKakmE23IREXG9DPAMIBbNuIiSDctPDOaqOi4LNgvOT+cQvn8PX//eV4Pc3N3fhB/cuDYSd2P3GDi9m8tyqrZHt5U6DKqboiXjf5t/5srEedFokI1olw+zEmq6pgBs+KmkZ5re3dOPXT61KXEeH+Kx096VnmLWRDI2KLlfDm4Pzfvw4jvqP+yOvi+gS6WWCIAiiFgrpq+weiK9gNZLRW3Gxaos+0ygiGaqjrKv5W3Fco0gEpK5uFpRIRorDHDxqDsWF4yYL3psfXYn//NvrAIAj9h4DwBeeHDVnmI+4+j68+u2zItESb3x6594smE0Osz6S4bocjhV9bVNHb8Qxdl3vPSlYVqwRTBIiaSOEo8ktT8owq+JdIKIrtz23OuH40ZuQcFvv92iGOb69/MQh2KfWsTU4zCk3TR/7xTNYtaUbB08ehZ09VZx6wMTE9QVCEJsc5rRIhu5cxU2ryzne3NwVe128R1nLGxIEQRCEDAlmnyDDnNCNT0W4leoXuE6snnvdY1i2qdO4r9BR9qpkhGXlksdQlbaTWyGL5TrJs1qa1CQmLgqRXypkr5KhrrOjpyIJ/2i+27YYtnaVsWxjB46ZtYe5Soa0T/nGQ73GwmEWbcplvv2XJdFxBhlmvcMso8ZUvO2jv4tdiPcmKZIhogIqWTLMYp3fP/s23nngJBw0eZQ/RnFc16u5LX1eZLStsZMyzLHtk8fX6TvE77vxKQDAqmvOS97AR4y11+DMp036i7zuclgWC24eTNdVvEd5MuwEQRAEIaBIhoLqMCdRlCoujB/RhNN8h01XASFJLHvbcP/4lh+v8JabHEqBEJVMdpg5TyyJJi9Ta0kHkYwEh/mZlVvw0urtQck4gSz0C0qGGQCOvPo+XHTz0wCAHz+4XLvvaKUP86S2zl4pkpHBDa+6Xh3mNIc5GgeIOq+qey5IcphFVEDFyVBWThxv/Y5enPvjx2LjAsL3P2sdZt2NkDxZVLvcQHNRfzHffcMT+MG9rxu3S3t6Ib+eVjdarCsEs+nJSl/GKhkEQRAEoYMEs49p0l8Sct7ZtoBPnjADQDb3UMWV3Eu5rFyqw+yEQkCcg8shOcy6sYTLgjJ6/n6yVMm46Oan8e4bnoi385acbZFZFpGM9TvCBitdfdWgTrOKTgyqywEEEzG9rovGoQbbiioZSQ5z1XEj8ZqYYFZiMuJmxpRh3tzZh6/csUj7WiXDZ8Q0eVQWlMI5TYpaJE2eBBB8HOKTC5PH11y0tctfWr0dNzy0wridk1JSL63Tn65zoTzpT0dfkGEeWMGct345QRAEsWtAgtknnPSn/0L98hn7xZbJeUhRPxnI1pRCRY4yMBYXZyYqkkANMqtufCKcTMRh9sdcViMZGVxbNavtSMd9bf1OlKtuIKBljaQ28ZCRr51cvs2ksRw37jCPbilGfne5t9+ibcUmfd3wUOh091ZdVCRHWC3NZ+paaIpkfHfBa7HqGOG+0+sw67LwQPRaCGc1qQ5zWiQjeBphWK7bP+c85jBnbXSS9vcRaVyizTDLkQzv38BhNoxhMBzmxWt3YNa/3Y1H32gfsGMQBEEQQwMJZgXdF+rx++yB8SM1LZWl72bLYjXXMJa3sZm505+OqjTZSS6DVlXEnmD5pk787wtrg98LgchXIhnpei52nhXHDZZd8afFWPDJ2b+9AAAgAElEQVTKumC/svBR20RHzkdSg7p4hIquSsYebaXI716m20XB8kr2ybGbH9y7NPi5t+JEKp6ojUuEGFNjO6ZIRtJNR5anECaHWRaM4thZIxnX3b8s9r7pJgeK7Va2d+Lc6x7D8367bIHjcjQVog5zlgYuprFG9q0RxJHxavZVVqq0qAST/gbQYBbVUx58fdPAHYQgCIIYEkgw+wgRopaVE8vSvmdFOTgAsWxvlmMHVTFiZeXS6jCHj93lyWmuRjC7Lse51z0WEYXC0Q4y1HZ2h1nXsEXebmdPVSvcd/oO84im+JxT2VXVxSNUHFfjKsY6/XGvSob/3pq6/fWUHSU3Hf031l45mPSnf7+TamirGWZZBG/tKmP9jp7ESEapYEWOnTQ5Tr52T63cgvuWbIyux8UYlO05x2vrO7Bk/U58+H+eiW2jOsxJ5fVkkm4WrrnndbwgiXPdmvL5iH3pHObojUXYEXOgEHseqJbiBEEQxNBBVTIUdJP+ijbTZoll19SSnMssE7oi++GqwxyKgrTGJUJUuZJYjTjM0roO57Ga0RZjsBkLlova0lmymPGGLTwihuQKFrLIEW2iRzQV4i2jpfFVgxrT5ooNrqZKhqnMX9F/fzwXPS7ueisOmnjomqpjj7VX9g9bNkQykt45VTR6jVW8LY76j/vAOXD4tDHabV0OtJVslKtuUF84q8PsHctV1otHNwDv2op11c+Ny+MOc2/VwWhE4zA6jJ0wXY6fPhLNPiedl7wvXZUM+eekxiUbdvSiuWhhTGsp9loeRESLYswEQRDDD3KYfcT3sq2ZFFawLG21Cll3WEzq/JfzG1NuNGJZ0bJy6ZEMIerk/emznDqhwph3zCCS4Yu2LHWY1f2pJd4clwfiVV5TRDJGNsfv16KCR4zJMjrMVU2GWW1VLX5PdZiVSIYofccVwWwpTxKMUYSE967qurj7lfXh7450rRRHW4VzjtaSd+2WrNuJXz7xZnIHP+V9Mj21UHfhcvO5/fzxN9FUSHaYTTddpgyz7slM2qS/QNBX45P+5M+iOA/dDeix33ugLu2wA4c5sb8mQRAEsStCDrOP+IqTHWbRbrpgcJirLodteeLWtlggxHRl5ZJweSgCbF8wC02QFMm4a9E6/ObptwBES8k5LoejEVs68WH5jrYQMcUcDnPMyXXdiEipSgJaPraY9DdCK5jDfcrOsDGSweNVMtTJckLoCgfXVDqwt+KiaMevmzglIaZtpQKIKYqQVBLQcTn+6XcvBL+vaO/E+dc/jv++6PDYuHXbtpY8d/fLfhWOTxw/wzhu9dqpozI2LuHcKJh/cO/S2OTKXsVp7+itYnRr3HGOVLlweayNuox+MmP4c+AwO6KsXPia/PkUTrypDrP6pKMmWG1t5QmCIIjGhxxmn3MO2RMAcOHhk4NlQliVbEvrTLmcB1/AllQlI++kP1cSfaLaRpYM8eW3vhjuw+WRR+tCM0YacXAeE/4WQySSIQTzDQ+twIz5CzBj/gJs7uzTHj/NYZajITKihvLI5riYilTJkB1mwz2Iq3GYK6rDrJyb6Sakp+JoXV01kiE+F2JdU5WMpDSN6rIuWuOV2fvb4g3BMvNkQgSCWdClEXw3PboC3/jT4lhEQB1XUIaZq8t5UI5Nh3rMXmXdHT36yZ2R2I6hjGAwBs328mdaXMeyZvKjzmEeyAyz2DVFMgiCIIYfJJh9Zk8aiVXXnIdDpowOlokv14Kt9wodlweNMGyLBV3z8paV45LDbFmIlJXLIpzFesGkP861Zcscl8fcVctisCwWiD/hwm7Y2Rus85qhNJoqcCqOG31c7vBIJlkg2lqPTJn0F7re0RsItRuf6ob3KccMIxl+/WSDZuopmwSzv18lkiEEmXHSn/Tz7Z85LvKaeiMhWoUXpZiDyWF2OUeLKpg1baafXrkFD7+xKe4wKxfA1OQmKZKhOwd13e095cjvW7u839XYTrC/hJsVdVzqGHSNS+TPUjmIZET3ZbrGtcCkUAZBEAQxvCDBrCDXVhbfvQXb0naIkx1m22KwpUhGR28Ff37JK9+mEwLqfuRJf7Yl1VTO+N3rcjmzGj6alvUG5zzWuEM442okQ8ak2XVVMqKT/lytw5ycYY6XlVMdd1VwqXlrVfSKx+3ClTVFJfqqjt7lVDPMLDo+c5WM8OfJY5ojr6nXpctvMy2PTJ1oJ3BdjrZS9NqJNtUyFYejs7caizWoZy9ezRPJ0KE6zOImAAD+9OJaHHn1fVi0envsKYRAH8mIHye6jZiU6MeRjA6zyKNH92UqcVhLpQvxflMkgyAIYvhBgllBdmDFF27RYjhlv4mYMqYlsm7VCfOXFmNBFYabHl2Jr9yxCF+47SUsWbfTKHwEEcGsTPqTv7jnzRwX/Kz7Ug4nAIYOs+zyOS6PTXizmDd20QCloBHMJmRBBAD/8PNn0d4Rxjeqrj6S0dFbRdFmsUlj8jl4P4e1oXWuIuBnt5VjqNdGRANE5ja3w6zkZNUycyaXUhbmakUJ9QlAt+8Q90ii0+wwA62KO6+LZHjLnVjGW3WYTeLQ5fkc2Le3dmPG/AXB7/LNx+PLNwMAlm7oMFay0HU/1DnM8qKKGsnQTBoFvHw64P0NfPsvrwbjNDXRqaWWuriqWZ8KEQRBELsOJJgVbJ1gti2MbSvhsa+dGlnX4TzIw8p1mFe2d+HeV71atz0VJ/iyNiEm/THmiRkmlZWTv3xv+eQ8aZv4l7IYr5cl9pZt3BkKWIfzmINs+Y1SRO63qMl4Llq9XSsku8vmfKs4nk54dPZW0VoqRNx8gXbSn53iMKeIm+3d3k3DKD8znT/D7P2rPvYX4l51VsNzCcfVpNQsVmM7O32nU74JMWWjXc7RqrSlVhuLCMqOG5uMp152cWlf39ChLOfGMehYvHZH5He56kXw9rHozUJE4GbplgO9Ky1aY0fEuHSNxXk4Lscvn1gVLN9pcJhraW9PDjNBEMTwhQSzgm4WvXBdVYHn+tUxxHZq3AHwvkTTRAf3hWUQ72BMajoRrleSHFmdYK64ocjWZZi/fPui2ON4xhhsKxSqukjGj+57A/913xux5brcrIzjcG2eu6OvgraSrRWupkl/8ulGOgC64QRHE9u7PVE0qiW5RnBvxUW5Gt+ZOulPHL+aUlZOft+LymdDFWRbOj1R3yE5xcbW2JyjtcnWvqZjpzL5Tr3sSV0U80Qy1Goh8uRLUWqNQXlCYHCbk8aW1LjENexPnIca36mrwyyqZOTekiAIgmh0SDAr2JoavUVD3V7ZYWbMXN83rQOayxGUpgMQaY0tiwOd+y1TlSIDujrKjy3bjC1d0YlYYZUM383VxCQA/cQ/UwwgGI/LYzlnwHOYW0q2tsa0/FheCMaCxWKusnyMtLypiGQIh9kUyag4rtZhFrsPXUo3Mj45tlCuunhjY0dsuXpMtX3yNt8F39yhr0gi43KgpZhDMCuiUD1F0+VzefbufUA0TgIogt//0ZJuBr2x6B3hpLFFbp6UWIwTEdPh2Ps0kwJ/+/RbwWejTZlEWZPD7P9LkQyCIIjhBwlmBV2NXp1zDAiH2ftZjmTIMCRXGgDCDHMgmKVJbkYxoxPMksOc9QvfYn6VDKVkmopOoHVpJppFxmgYR3fZQXPR5DDHO/0VEiIZcv7bxHYhmFu83K9JMJerrlbgxxxmR0Qy4mXlLvv9Czjz2kexqaM38r6ntTgXNzKmiAAA3PiwV+pPVDspGW5uVFSHWX36YLp8bs5IhhpN0bU2l9uwA2pZOXMcxrQscJiduCA2RTIEV/xpMW5fuBoA0KZkwvvjMJPFTBAEMfwgwaygE73FgtlhlqtkmMRmmuh49I32yL7k1thJDTtiy6QMc5bGI+JYFmOBUBSNTFSaNYK5OyWSITcuiW7noKVo6zPMukiGFY1kpFXJUNneXUbBYoHoN1XJqDguKtpIhvevKsrCCEy47n1LvOx6R2814jCnlf/d5gvmpAYaNz0ato1mjGV2mVURHnOYDQqP83yT/tSsvjzZNYgwM7Oozd64RJNhrsbdflmY68rOAcD67V75RLWu9b/84SX8+qlVsWMnYZFeJgiCGLaQYFbQZZjl/Ok9X3gHfvfpYwB4wkOukmFqipDmMH/p9kVYv7032JdXRs17zZwvjS8TAmHNth4sWrMjvoIGppSVMznlOsFsyn8KHKnds0x3ueo7zPFtKhkm/ckusK5Khsr27gpGtRQDB9AkXvuqrraiiSnDnORCVh0e2Veaw7zNz1lnfZpvWwzNxWx/vmoDEfUGwxzJyJdhjkcy4pP+GFg0g845VrZ34uZHV2jd/bSyckGlEv/fXmm8spMuPt/qeya2U6vDPPJGO77551fjB09AvMUUySAIghh+kGBWkMXixfOm4eyD98RF86YFyw7caxT2GFEC4H0hy1Uy9N0As+VA39jUEckwh26xfn1tJENa+dk3t2q3U8WwV8ZOavts6cWdTpyp2VgVU5UML5JhaY+ja2RRsCxjhtlx4q2xVXb0VCJtnHXvE5CUYfaOJ+dgeUoUpLtcjTxZkA/54WP2Th6wAXnUFtPfxOjY2aNmmKPnaC4rly+SoQpe2T2WHWY1w/zBm57Gd+9+PTZObzuz4w8An/jlc3hu1VaU/XHKsZBIExzx95RSs7s/mDomEgRBELs+JJgVmOQUjx/RhJ/+w1HBZDGBcKHViXo6HJcHJa+SeHNzV2QCodzmWrtfzfIsHQZjnf4YIg6zxZj2C1/UEZbHkxQfALxz11V66Ku6aCraWuH6suSMC5HjZZgRWw74olwz4HfMHo/zDt0rOJ7cJMXk9ZaresEsDidKvlVdN7VU4MNL27F4bThRUpQLBKCtP50F+SwZY2hWajvLdbpl0iIZ5gxz+tMRGfXzF3WY4+LVGwsPJo/q3P00hxkA/rpoXeD+90Uc5vjNl3qTI8ZYS8PsJ1dsVvL0/phr2BdBEATR2JBg1iBEsOlLdLLfwOQzJ+0TPN439fuoum6qAwp4wkCIWZvJnf7M+VLdsWZPHBHLY8qoDrOowyxnmPXCJf5I29QlLRwPx5ZOfdWH5oKtjb/IVAKHmWlzqwDQ3tGH3z39Vnz/RRuXnzY7+D3SGc8/7MSRTZFtyo6rFfjiPRARFMflQX5bra4guO6BZbFl4mxLOZrDyMiCVBvJMCg1NYcca/RiOB7nPFeVDPVzU9ZEMuSnJ4B3wyM+BjpxrhXzyjLLCj+zfRGHWYr3GGI0QVQjpy28or0TH/6fZ/DosnZprMnzDgiCIIhdFxLMGgIhZxB0bU0FrLrmPHzw6GlhHWZDMNZrrJFNdMhVMjr7qtjeXc7cGlscy2IsUYiqLiDzBbMQVabzCCa8cVkwJzvMa7Z14wGlfJqgpWSlToQLq2RYgQj53G+fx2W/eyFY568vr4+VygM8kS3vv0UStmLxle86GNPGhd0b08rKiRuEqsuDpi17jm6Ora9y8z8cFfk9a3ULlWgmOh7JME3eU1Erlxhz8q65PbcO9drJEyiDOszKpD/H5cH7IeIUE0c2Ye70sZHtksZrS59fuflKFoc56N6Y4Q/t+O89gO/e/RqAsKRit1QpxiWLmSAIYthCgllDGLNIX1fEKEyRDK9SRLbjijrOYl8X/L8ncrlVFcdz63TVJwSqABKRjIoUydDvWwiLcJmIKDz8lVO027y1pduY820u6KtkRI4ZNFMJXcl7Fm/Ask2didsBnsiWIx9yRYkwd45I3MYUyRDutmgq4rg8mOC21+iW2PoytsVw5sF7Agjd0tod5ugkQvlJwv/90/GZs7MxNzVx0p9jrP6iolYYMU36M+XRhcP8q0/Ow08+eqQ/Bt24on+blsW01TzkpwViPyaH2VSGcWV7Z/A5X7ejFzc/ujKyb/lahnpZv6/eioNP37IQb23p0r5OEARBNC4kmDUEBnOGZKM86U+H46SXPROIfYjH/W9v7Q6ERpbca+AwZxQ4QFiHOYxk6NcTYkg+F+EIThvXqt1GiBidBm8u2sZ6yAIhbkRZubQGJTKqwyy7seK4tmVFBbPDsWlnPELicuDs/340cNSrTnaH+cLDJ8eW6ZrjZEHWdBZjQe3g9x81FUfsPTbR2JQ/E6qbmlSJpa/ixmoUm1AnCOpaYy9etwMLXlkfLHdcHtzYiDhFwWbhjZuh05/s0ss3fJHja25+1L9F8bk3Ocyn/egRfPsv8WoZ4rMpb5dWO/2xZZtx/2sbcfVfl+hXIAiCIBoWEswaQtc4fd20SX/VHJEM4eSt2tIdLHt8+WYcPHkUXvrmmanbL93YAcsKx7TPhDac7098kznr4Ek4eb8J/ri9/ypSWTkdFc0j7d6KEytDd/w+ewQ/l6UMskpz0UrNMAeRDL+Riy56YUKtWqLLddsWIpMB12ztxp0vrIlU1AA8IaQ+6hc3NXslCOYrzjsQ33/fofHjpt0pZMBiwAhfyIpOlEk3FPJ7tGxTh7apiErZcbG9p4JxbaVMY1IzyLpIxo0Pr4is4/J4JKNgMalrXvw4nEdLPdqMafPPOtc5PukvvBE0Xb/V27pjr+kiHmkZ5rB5S//ff4IgCGJwIcGsQWiLLN9rliKYv3LmfpHXHSWSkeT+2r4IeHNzNHLQ1lSIZHCT6C47qc1UCpYViCymZJ7lmMSsCW349afmYdb4trBJhyQQeipOLF7wow8eFvwsRJmprnNabeKwRq5XJePyW19MXF+maJszzHKMZpQkjt/e2g2XA587ZZ/IvlTRVnXd4DF9UiRjv0kjY/V9geTPQFYsK3SYi/4xxDBvUjLTQPSm5bdPv43vLHgt+N2ks5du2Ily1cXh08ZkGlNypz/9NlWHB6FyUXmkYIUlB3Ui1uU80sLdi2TEK9H8+MH4xEtTRMhxzWM8aK9RschGMIlQjmS4yQ6zOJc6vP0EQRDEIEOCWUNQJSODYhZP14UuuuzUfSOvV103IjJnjm/DU18/TbsvIWrOmxN9jJ/0BfvJE2bgP983J/i94rjSREQrEOGR/VksiCiISEZ4rPDnv3z+RJy03wQUbQuVqnce51//ePB6b8WNTWCTtxfCQddavClPJMO20FNx8OSKLckbSHj1pcMD6GoW2xbDaQdMBACMai4Eokh1mOUnBAWL+Q6zJ9AmjYpW2pCZOb5NuzztRiELciRDvN/ieuscbPXG6bFIdQf9Md7Y6N24HeVPwEtD3Y+prJyMLDj/vmSDN1abSU1A4ttwHs2B24bKLm9JT2qC4xlO1uXc6AxX3XgDF0cT5Uib8yder8f7TxAEQQwuJJg1CKGc5XtNrZLBWNTV1bVuNn1hin187az98cXTZ2vX/80l8yICZp8JIzBSyuE6DofQp7alj0MULIbxI5r8fUfL58liK3AwCwwVx0V3xcHa7T3B671VJyaYdaemc1RbMjjMQnAVa7DkVJHeEskwh+/ZuXP2wvNXnI7jpCiJmtmV87FjWkuoujxwmCeMNAtmUX5QpS4OM2MY0WT74/OukxilLiOtOt0VqfPdsk0dsfVlZk3QC/805FbiahdAQVefE5SuEwLdE8z+TYBmG89hDs/RtrK38DYJ5qrjJrzGI+XqANOkP+Ewp0UyEKxXz8YpBEEQxMBBglmD0DNZnCBdlYyCJFiqLo88zuWcG/crxK1lMbzniCnBcllgvWP2BFx0dNh5sFSIdswrO1yKZFha8WQxFgi9zr5q5EtbN7SibaHsuDEHTnX6xL5N5yXTXLRShaNcVk7w7+cemLiNfEzZOW/VlJUT12mPEU1BrAFAIEQF8vVpa7IjGeYkwWw6v7TqIFmwWCjsRVZYCDX5pkdce/U9EOf0k4eWa51YwR8uPbbfdaMvv/VFPLFc/3TgH3+9MOYOFywrIipVXM5jY5K1blInRbPDbI5SvLm5C29ujla2EOeWZ9KfWFXcDPzk4RWY/e/3xBrLEARBEI1Hpm9CxtjZjLGljLHljLH5mtdPYoy9wBirMsber7zmMMZe8v+7q14DH0iCjns51pXFkexuOi6PzcA36SV5H9P3aMMlJ870xqGIULliRlMhKjwjkQxmdphF045NHX0RB3VnTwV3fu54PPjlk4NlRdtCxXHhaCoRxBzmlPMSNBfs9DrMbphhFuy9h74iR+yYaoZZLisXOPDhCvJ5tJaiDrNcu7qt5EU3un3HcWxrdELc2FbP7VebokTGZqrvnTGnDvixGr/TnxCc+04YASAaKRnrT9hTPwc7eyr4w3Nv44W3tyUeZ1xbSZvDzoIQ5XJVjCzIVTI499zfddKTDcflKEldDtW4xIQR5mtvqljjuOZIxuPLN+P9P30q+L2jtxJW15A2SYtkqDc0tz77NgBgexcJZoIgiEYn9ZuQMWYDuAHAOQAOAnAxY+wgZbW3AXwCwO81u+jhnB/u/3dBP8c7KFg1RDJMDvPX//cVrN4aOngcZue6oLjBYV3m6HqyYC7Z0QYg0QyzvsScbTNMHOVVd2jv6Is8zj5k6mgcNX0sZvniSxyj4vBImTD5NZnsDrO+NbaMEPJyRYSpY8OYQ5LgLlpR57054jCHTr6gFHGY1UiGd97HzhqHo6aPheNy9JYdMBYv93fR0XvjtavOxqNfO9U4Nnnc1198RPBz1vJt3j5YIPLF+/ed98zBrz81D/tODN+7cb6gV691V9nBv975Cl5dtxNJjGktZa7DrFJr3KBoWVKVDI7v37sUx1/zIDbt7AXgPbWRb3BEnlyQ1OnSOOnP0GJdx5xv/R3tHX2x/QWl5lIiGeJyivVrLTNIEARBDB5ZrKN5AJZzzldyzssAbgNwobwC53wV5/xlAMMikJdWKk7GksSpQBUYP3v8zeg2xgxz9O0QQlFdv0ly15qL0QYg5aobrF+wLK3YsRkLXLhNHb2BsLnuQ4dH6hIH47C9DLPaJRCIO8xawaxxKJuKVur1FS6efG2njg0d5qSOebYVraLdqqnDLB9fjmSogktURDj74D1hWwxVx0V32UGrRvSXChZaSrZ2kqHAUj4romJJHsFsW+H5C4e1pWTjpP0mRM5rjO94qzWSBWkN7sa0FmM3clkpaz4vWZAnbHIAjyz1JiiKsoIu55HPdUww57iOAtfl4Dn+7/XH59cEYxHwtEiGv39xbiKqRXKZIAii8ckimKcAWC39vsZflpVmxthCxtjTjLF361ZgjF3qr7Owvb1dt8qgksdQC6tk6CMZOpjhZVXcmoSK7Go2F+3II/6qy4OxFGymrZJhWwz7TGzDlDEtmH/OAYHgkoW4TNG2UK5mE8y6c9O53EXbQtqTfrkOs7cNi8QNkrK1BaUOs641tox8HjGHWWobHlTJqDhoKcWFWZYJimoZP/F5aWvKEcmQHWa1e6N0WcSEUF1jjywUbSv182yiknEiXvyYcpUMHnvS40ifcQDoKUdbtLcm3KyYqLocH/n505nX75S6PgrSq2SIEDMi25pcb4IgCKJxGIxJf9M553MBfBjAfzPG9lFX4JzfzDmfyzmfO2HChEEYUjJp7a5lwuoY8WVaeHqVDIEQiuoj3qaiLJjjTq0cyTh3zp7a4zQVbDwx/zScdsCkQHA1F/Ufh2LByzBXNZGMoiLqdeemux6qoNVRcdxIq2/V/S4ZBD7gudqmTn/izZKvq3CYizaL3QQEXRAtBttmqLocG3f0BnllmWKGjozy9bAZC26M2jQC3ARjLLhxqqqCWbquo/zGLKZ4hK5+sYr6HmclKZKR9HSAMSZN+guXi58dl0duOlSHuaVkZy6FJ7N4bXI8RUbEYOQYhxC+pioZYnHgMGuanxAEQRCNSRbBvBbANOn3qf6yTHDO1/r/rgTwMIAjEjdoACyNCDYhvuvkSWVpAsOkp+MOs/f2qF+oshPcUrKhGoCBw2xZOHTqGNzzhXckHkcImxaDMycyzGrzBiAUrdd96HDc8ql52nPTTjy0WYZIhieM7MBhjp5oUrvwglKHOTLpTyPGhIBrLsRjFmXJ6S5YXjxl4VvbcOTecVGmutM65EiGnDPPFclgDEfPGIcPHT0N17w32k1QPm/RydBUdm1nb1W7XKaWSX8FiwVRFpVz5+yJuy9/h/Y1gQjUyOJTdAt0XB55+qIK5qaChSvfpU6zqC/iqYw8oVeMVdyIdfRWsHjtjuB1V5n0FzjMOVq+EwRBEENDlm/C5wDMZozNZIyVAHwIQKZqF4yxsYyxJv/n8QBOALCk1sEOFsJpzeIwd/qCQ66WkOQwJ036y+wwS0KxRYlkAOEXckFympOOIx6dm3K3IsOscwxFLOLCw6fg5P0mgGkCDyaH2XSZrr7wYABeJMOS1lP3k3Rj4rXGDn/XlZWTH56XbDnGEt2vcAIt5kVcXA7s6Kng6JnjIuv9y+n7RUr+GcemRDLEb/km/Xk3ENe879BY5RB5+CN8way72UlDTB6Ub3hu1nQR1NFctI2RjFHNxcSbHUB/UyNw3Ghpxm4lklEq1B4jyUrgMEunKC6xiBJd8quFOP/6x6XJgN7rYujiPVGr6BAEQRCNR+q3Cue8CuDzAO4F8BqA2znnrzLGrmKMXQAAjLGjGWNrAHwAwE2MsVf9zQ8EsJAxtgjAQwCu4Zw3vGAWZcWyOMyihuoY6fG86oSqmParOrFiP2oSIuIwK5P+AGkiYlBlI1kwh5EMc4a5YmjsoAof3bkF9aWZvMwyXocTZ3uxnIrjwmL60n3e69HxyK5i0TZnmAU6h9nWCPmwHjSLZJRnKEL18nfum/ree8eQfmahE67Wf04iKc4ivzampWRcL4lp41pw37+cBCD6uVQjJybh2+THeHSoE1V1iHNwefxcnZRJfyXbQqmQ4Y/XwIF7jcIX3jk7cR0xiTISyeDRTPKzq7ZGfg8blzBs6ewLltdyM0MQBEEMLpksLc753QDuVpZ9U/r5OXhRDXW7JwHMUZc3OsKNzNIau8N3mOXJaIkOc0LjEnWCnnjsrD6yjWSYS/GOearDrIOaA48AACAASURBVA7HJDxNkQwx6U83cSxLhlk80i8VLPRW4pUvVOSyWzYLha96Q7Gtuxz5/ZMnzMQP712KrrID2zJnmC1JjAnCCYTxqEhZcpjlCgyqI5zl8yIfH4g64Wr9Z5mWoh3plpe1W+DEhNbdbSUbXWUncKvVesbifMS4Jo1qitWdbinZse0A73rrlott0krVBQ6z9BSASw6uLLjVLoL9dZg/dcKMYFKfCfHZ0TUuUf9e1Q6Ad7+yPqjBDFCGmSAIYleAOv1pEJOvskgS8cUqC+a0zGdapz+BKCunPrKNRTIMkQshGmKvG45vmvRXKvgZZl0kI1ZWLr69OJ7sviY5seL6eA6zOZKhOovytrEqGZJg/tdzDsCUMS04ePKoYNlMv+705s6+eCRDNFCxrIgLLPLKeVtHRyb9WQzik5bUuEREKwRZK7kkdSIUJfraSoXYOcuar6Vk45GvnoLHvnZa7LNjuska2VxAl0F0tmZoix5xmP1ljssxY/4CvL6hI9lhLljayZdZJy9ajKWW2xPI4lielBhZR4lk7OipaF8nCIIgGhcSzBpaAoc5fd2OIJIROm9J7pmXYda/pjYwMDrMBXmCoRXbn2WFohHQRDJM5eoM4ocxz8X77j2vx15ThW9SlQxZ6Ht1kvXjEOOv+E6iXPUjDXH4gtLpTxb2R88YhyfmnxZxiI/Ye4zxHETraduKusriScSdnz0e//dPx6eOLdh/RDCHn4ekDPPIJlUwZxN/46Wud4uuPDP4+YLDJmO6Hykp+JU/ZNTs8PQ92lAqxGM0JsE8qqWIzr4q/vnWF2OvNWtu8gQn7LtHdIE0EHmMEYdZM+lPV94va4tvy8qeK444zCJioTyJUaMaKjTpjyAIovEhwaxBCCFTZQEZ8R04RnaYEyMZ5kf3pioZ6pd3WrOQYLKfyDBndJhN4ue4WZ6IWbR6e+w1VTBrM8y2zmFmxhsSMdyq62WYxfXKIpjlRjJZRSUQlqybsUdr7AakEjRQsSKl34TAHdtWwhGaihkmIpP+mBzJyO4wZz01+T2Vn4L8x3sOwSj/94JtZXY51Wtqyr2Pai6gr+riL4vWxV7TTawU/O7Tx0rHUltPh79EHWZl0p9ta5/yJJWyk7EYyyxiXU2GWZ2kK/5+jYKZHGaCIIiGhwSzBpHZ1D3yNzEqEslIcpjNX47mKhnJ66m/y7EEIC6QTWLF9Mj6lP0n4tZ/PFb7mirydTcDurJwyRnm0GGWJ+FladEsdznMoZcBeA7sgsvfETkHUSHEG3PUBU6r9GBCjWQIp71oW0ZRp5ary1pRwyRoS7YViOmixbILZmV4JpGv6xgpMLVsV2GMgSNsXPKBnz4V7oMxPP31d2LCyKbY30epYGk/y6bGPCoWy3495JRSkLFWthW/61rLe/sgwUwQBNHokGDWIESA6lwlkafTnwmTU5z2hRqvghEdhzock/BMmrRmyjdnET7ieLIYNF0j1Rm2pCoS6rGufNdBeN+R0bmmgbi2zZEPE6NbijEh6rXC5sFY5G58WSf5qZgm/VkWQ3PB0kYHVMGcJEhlmkzNaGwrEPwF28JVfik/gan5hvpZ01UfAaI3kLp9mJ5yRNcTT2Tir9kWw56jmyORE0FTwdJm5PM4zJkjGdJ1Ej+r2wYOs6HbIpWVIwiCaHyyF37djQgFc7rDfPfl78CK9s7IsrRIhgl1q6CsXMrjYVUbyK2xgXSH+dqLDsPfX92YeAyTO5dNMIsuepJgNrjZXhk5+fdQXKnH+uQJMwEAd76wJlgmV9TIOjEuiXGtJbR39vn7tHJ14zMRd5jh75/hMyfvgxXtnfjfF6K9gdRIxqiWjA5zwvsmxHTBZvjYcTPw2vqduPXZ1QDM7Z3Va5oUyTCR2WGGefJdOLFVk1U2COasTwRsy3xeKvLNrKlMnIhqmMrHUVk5giCIxoccZg1BJKMvXTAfNHkU3nXY5MiypEhGHoJJfzkdZtWRjTvQ0bf9PUdMxY0fTW5IYXIqs8QkRJa6JF0X03YWY3FBycKfAeDOzx2Pz5w8y7B9uG6eDLOJ6Xu0BeX0LCtfcxETah1mOaN92an74qTZ8fbw6qS/rA5zUmUIcRMUrpE98iJImvRnwmZZIxnmCJNuIuicKaMBeDdmuv2bHGbV0WeM4WPHT8e/nL4f5s0Yp91G4EQcZn+ZoUqGrrW8uo+fPrICM+YvCOo8EwRBEI0BOcwaxGP37kptX1pJkYw8E+LFftIcZlnEHDtrXCBGgyoZscxz9jEITI6bKr516CIZpjiDrZSDYyyaSwaAo6aPxVHT9ZPs5DxwHfQypu/RiqdWbgmOn6X1dRqRyInmhkA37hbF2U6aICiTFBsRjqvagS4Jk2BmLPrZThL0Vkb3X+wzqXukfOP1m0vmYUV7l1kYJzRZKUthZJsxNBVsfOH02Xh21ZbEMepaY5sFs2HSnxTVuOmRFQCArj4nc+aaIAiCGHjIYdYw0n+cnCZUTdTqMKtHE+5g2hNbIR72HNWM2y49LiYw45ME87/tpsfZWc5VN+nPuC6LiinbYkEG2+RKXnbqPvj3cw8EEHWYa80Yy0zfI6yx7D2q7/+fTMRBl6pkmFqZA3EnN8u5nTtnz8TXxXuqe4Jh+uibMsyqSzsyMZKRbfwWY+CcGzPMgFyBhWFMa8l4I6UbY7BcrToj/WowhQN0kYxexR0OBLMhw6yryGHKkBMEQRBDAznMGk7ebyI+c/IsfPpE/WP/NPbxm2Ak8cMPHIamgqWtUysQpbF0guZb7zoIY9uiXdeEiBBrCzGh6q9aHGaTYK510h+gF02MRfeZNOlP8NWzDojtM2uTijSmS+2vbcuqiwiPTfpDNDqji5JkdZQFb37v3NR1RN1t8fnKcmbq0MSTh1Ih2ikwbdJfFhiijUtkbOWmMIsbm9V5lt/jpKo2QCh2X1q9HXc872Xpt3dX0Cs9nXLT6jBLy8WxSS4TBEE0FiSYNdgWw9fPObDm7T9/6r6YOb4tUQy//6ipWL6pI3E/SVUyPuFPeJNfLwSOtGi0kS3DnAWTIMmUYdZM+jNhxapkxDPZydtHj9lf5FbQWSo7ZEHNaMuVPQB9YxtTNQoTWYR9GMnwBbO0iUkoqvEecWPSVLAgf5qTHWZvm/ceOQV9FRcLXlmvPxZjRqdb5OLFvrI4/6bPsCqY5fc5zegVkYxP3/JcZPn6Hb3BzyLtYcwwa/6+a326RRAEQQwMFMkYACyLxSYCCuRHrWmiRgiotMezwuUSX/RidVOnP9NErSRMjm0uhzljJEMerjzpL18d5vqIW9nZFRr82osOwx8/e1zN+4xPahTizzuA7nOR12FWmTq2JbasOeYwpwtF9bKKX9X3NrEOs39+//XBw3H2IQmxEWYWjrbyPmdxmE1PSdSxWxGHORmRslD/xtZt7wnXSYlkyOco9pIWBSEIgiAGF3KYh5C0R9PicXNa1zEhBGZNaIssl6MQf77sBFx4wxMAahNfWbsT6rANkQwdlsVinfCEAZfJYRZCShL46nXJg+zsivfjPUdMNa2eCbXOtPhNnLf8+l//+UQsWrO9X4L5tavOjtXiBpIdZhOxz6z/+5jWEtZJrmpz0fabvsQ/u7JLnRSdCY6lGVgwsVVyuNMw5/BVwRz+nHazqj7NEfzyiTeDn9Mm/clCWpwqtcsmCIJoLEgwDzLy12Ca/isGZeWS15s6thU/+9hcHDNrXOQYtiQEDps2Jvg5a43ZLKiP6FXkKheqONJtKUcwvN8ZnKA1dbqiU93ov3z+REwbF3dXsyK78bVkv3Wklc2TT/OQKaNxyJTReOSNdgDAyftNwDfOPyjX8UxxDuHK6jLMJrkW08v+v5PHtGDJ+p3Bcsvymq1s667E9iHfECVVlGG+w6z9nARVMvwMc4bPtOmGTRXF8mc6ayRD/Wze/9qm4GcnLcMcOYj/N29wowmCIIihgSIZA8iL3zgDv//HY4yvpzrMGRuXAMDpB03CSP8xuFjf5P729/G+TJrDbElVL7I4zGokw7LCG4Y8DrOIN8yZOhpjWktJmyTSHBHM9flzkQWjXIc5zDCbIxnHzBqHfSemTyo1ccun5uHGjxwJIKyt3Z8qGQI18uF1RdTfj0duGFIcZtM41Ioi/XGY1fOXz/GyU/dN3KfjcjyzcgvWbAsjGOrfl9h/xXDnG5305y8jh5kgCKKhIME8gIxtK8Va9+b5HixmbI0dIyXCUE/BnCYiLRaKgCyT/k49YCKYJLJtyWHOlmFG5nWzIDdsqdekP/mS2XYYyRBCTVuH2Rfu/T2vk/ebgHPm7AVAV4c5+w2JiloVw2YsqFl95kGTovuQIxlJDjN8h1lzyNBhzi6YTTds6g2p/DE99YCJ+PWn5hn36XKOi25+OrJM/fvKUyUjaRlBEAQxdJBgHmDytMlWf8/jMEf24ytmUz60lkl/Mou+eWYgULI5zPpJf0IIHTtrHD44dyoe/PLJ+NYFBwfbAZ7orxoee5uOB9Sv26IsxJLc0DyodZjViYo6USpiFfXoXihQIxlZiE36MwxHdpgvPHxK5LVIJCPhmjLGjNEQNcOcJWZkjmTEjhz5be9xrcbtdddOjcCIjLKxcYnLsXjtDmztKgdHJsFMEATRWJBgHmDUjKZcritNCAdl4nJ+efLAYda/vXlLlKmMbi2GOekMgjmsjawIZl8ejGsr4fvvPwyzJowI1hHCsKVkB+Ihi1iU20zXA1nk162sXKTTHzQZZnMko17OOSA1LtFO+ksvK3fbpcdGXvv4cdOl9cI24qJzpvyaIOl98jr9GcahRG90DvM+E9qw/6SRwe8l21AaMSbao8ecMb4Ni648Ex+et3dsW13UuLUYjaKIv/NqQiTj/Osfx8U3Px1GMkgwEwRBNBQkmAeYJActVTCLSEZeh1kIZoPA66/D7B3E+yfdYZYm4uVwaMW6LcVQMOeJZCQ96s+DHFOolwiPtMNm8Y6E4jDy4okjm/HhY/bGibPH12UMQBg3cTVl5UxE27DvEXnt2xceEvxsWwwjfKHcqrT1jlZBCZf/9Z9PjKxXsi2Uq/pJf+KzMKbVi4LoPlsPfPkU3PsvJ4X7MzjMXzpjf/zTKfvgwL1GAdBPsh3dUsQlJ87EAXuOjDSz0d3MNhsyzCaHeUePNzFy6caO4D0gwUwQBNFYkGAeYNQvcln7pn0lCoG2R1tTyppRhIttigyrZbRqITxGimCWG3Mo604a5Z3XvhNHqpsFIqulVAjEQ5ZJd4HzWKf4hEy9BLPNGH7y4SNx+LQxfqc/D/HZEAJaPpptMXz3PXO016pWgkiGtjWzfhvjJVArTTCGNv89VN3f6HX0fj582hgcMmV0ZL0RTQV09Fa02Wqxj1njvZKB7R19hoGF6FzoW//xWJx9yJ742tkHYJTfbMUkVqeNa8XfvngSJo4M/x5167YqN6RO4DDr97txp1eOT272QpP+CIIgGgsqKzfAqE6n/DU4wlBFQNBUsPHDDxyGY/1ycVkJv2vrLxoFQiekucbR1tbRazF3xjjc/pnjcNT0sbHt2ppsbO4EWopW4MxlcajzNDnJS90m/TGGc+bsFUy+E7sVNyGhwzxw7x8QCshQqIevmeSaGhcxudJyhll1duVYR3jucUa1FNDRW9XGMsRnaZbfhl6uUmFC5zAft0/okgsRnvbkR35fdMJWvbESJeJMQnyDL5j3aCsF7cUd6lxCEATRUJDDPMAkOcyTRjXjL58/ETd82CvzNXtSvFzY+4+aiqljW2PLk3A1AqjeCBGTliuWHVSdiJ03c5zWuRUOc2vEYc6eYU6q71sr9XKt1aGJayg+G0JQDqxcjjuu8vFM2WHT262ubVsscEyTWk8HP2mON7K5iJ29FW1OWDwk2cdvSiO3ojaRVklDfL7S4hDy+HWRDPUaXfGnxejoraBsyDBv8Mc+rq0kTfpLHAJBEAQxyJDDPMCklVKbM3U05kwdjSljT8BhU0cnrpuXeguudx02GU+v3AIgFEhpwtTLMOefiCcmuTUXbXzsuOl4Y2MHPnPSrNTt6l1WTqZuk/6UsYnduhlvQuqFiOZcPG9aZBxJxBxm4RBzdb1w0l9TwUJbyUZX2QEQPf/gZkFzrFHNBSzfVMXolnibbeEwj2kt4ZxD9sQH5qZ3X0yrAy7GkhaHkMdvyiXLbNjZi+sfXI7eiqN9XUQyxrU1YePOPn+//VPMWzr78MLb23GGUtKPIAiCqA0SzANMVuF2uNSJr/+Iqgf1FV7XX3xEeISM7aqZXDYth0Mr3MCWoo2RzUVc96EjUrbwsPzGJ2kdCGuhbpP+lPflgsMm4+U1OzBlTIv/urd8MHTz61efHVQCkT8v2SMZUQ7YcyRe39ABxhgmj2lBqWBhVEsRr151NmbMXxDbh0lwA57D3NFb0UaX5PvQGz96lGG0UdJuXvcc1QwAaC4kT4qVP1vd5WqmY7+1pQtdhnVFN0T5xqC/iYxP/eo5LFqzA69868ygoRFBEARROySYB5i4yBr4yTxBJnXAj5QuguXOfXkEpxBVeZusWKx+FTJU6jbpT9nPJSfOxIeP2TuIoQQNTAbhHZTrF2c5WtoluPUfj8WK9k4AwHlz9sK8GeMwShFs8vlPHOmJ1FP2nxDb18hmr7W2tr12De9xmnN/5QUH4cjpY1LnDMjXoLM3LoJ1N6prtvWgp6x3mHX012F+c3OXtx9qsU0QBFEXSDAPMOqX56FT6+kk63G1dXU96tnlD8hSh7m2mIQYu1qiK307Vjdhq1Kv3arRDsZYpPRaIOwGJ5mRi7SnFmPbSpjb5glO22LYc3RzbB3Z6N1zdDOe+bd3YsKIeCUYtXtgZBwZxxs9bvJWraUCLjo6Xms5th/pGnT0ZXOY127vQbmaLIK51NUwb7MiFZYQdSEIgiDyQ4J5EPnzZSdg34nxiX31RnxJqtpmyVVn1d21TBPBTKoznMcVDBqX5KwZbbH6dflTqVfEJS0uMmR6Wa6Skbsbe55ugdEzmzQqLqqBaJm1+PHyU4dqigCi719XRsG8vbuSGrFxJMHcX2e4XsKbIAiC8CDBPIgcVtecspkwkhH9hlYbSNSDNNfOlpp01OIw549ksAGZ8DeYBJGMQT4N2eU1VclQqWWMWZ8AJEUYso5Ppl6TKSNVMjTDMB0lbcjyvvrtMPv/UgMUgiCI+kBl5YYhg1CGOSBLlYywDnP+DHMppy1oMVZTvrWREMMfjAyzzCdPmIl/PfuAAT9OVuE6XhPT6A8DlUFXmek3U8mL6/LgPc9SfSMJ8TSkv/shCIIgPHZtZbGLcPqBk3D1uw9JX7FOiJbEecVmLaRnmKN1mD9w1FRc8945qfsV2+T9umcMKA5QJGOwGCqH2bYY3n+UV54t73XPY4hmFa4XHj5ZOxkw7/EE9XKY03Yz/5wD8PHjpufer+OGkYz+OsOBw0yT/giCIOoCRTIGgZ99fO6gHu87756Dg/YaheNm7ZG+cj/J1ho7dJh/8IHDMu231gymNYCT/gaLoRy+lfFO5dMnzgRQW6476/vDGMOJ+47Hw0vbY6/lyUwL6hbJSBl/c9HG2YfshVueeivXfl3O6xalCLLQ1DGQIAiiLpDDPAwZ21bC50+bPSC1iFXS8sJe4xJ/3RzO774TRwLwup/lwWID07RkMBEidCjOIouoXHXNebji/IMG9BiCFkOGvRaHuW6RjAzjHz8i2+dWfhrich689/3PHvd/P5xz/PzxN9He0dfPsRAEQez6kGAm+kWmSEYNVTK+fOZ+uOVT83D0jOSauLrjFeoQRfn2BQfj8tP2BQDc+JEj8akTZvZ7n1kJIxmDL5mTOu/VizzC1dREJI9gfsfs8dlXzkCWG9HZk0bi5xmeLDVJ57e9u4KtXWUAeqG7bGNH9jEGDnPt7+SyTZ24+q9L8M+3vlDzPgiCIIYLJJiJfpHuMLOa6jAXbQsn76fPrybB6uQwf/z4GfjSmfsDAM6Zsxe++a7aHdW8BJ3+Bu2IEjUeNI8sy9Ni3Ogw5zjeTz96FL5x/kE4Yu/6VKlJ6sT4m0vmBT8fv0+6UG+S2nUvfGsbdvR4TVpefHt7pBLIXxatwxnXPoq/v7oh0xjrkYWuOF6cY0dPttJ5BEEQwxkSzES/SM8w11Ylo1beddhkXHT0tAE/Ti1krcE9lI1LwlbVA+cx5yliYqrDnWd8bU0FXHLizNTW2FkRn2N1Uu3Bk0fhHbPDm7ws5ykLZpk/LFyNPzy3Ovh9yfqdADzXNwv1qrYBDOxngSAIYleBJv0R/SKtrJzcGnswssUXHj5lwI9RK3++7IRMjS6GstHfYBwzVyTDF8x7jmrGhp29seV5eeSrp6BUsHDc9x6saXsgvKHp8zv37T9pJJZu7IjFRLI46Unn8fqGMIKRV7OGDnPtk/6CeA7pZYIgCHKYif5hp0zki3b627Un4/WXtqYCJhq62skM5XWqtaVyHlGVZ9Jfc9H7X9SY1rBN9udP3Rfnztkr+wElpu/Rhr1Gt9S0rUB9f049YGKm9XSUDA4zEI1TiKogWS+dWK0/HQODpw3UYJsgCIIcZqJ/ZKmSEWaY6f4sC0M56S+oKpdRIw1kpz8gzDCXndAp/dwp+wzpTYV4f84/dC9s6SxjtiFqk+X9M0UyAK9VdoChe6eJ+lXbIAiCIABymIl+kqk1NshhzkMQyRiCyyXeo4mj6ttlL3KMPA6zX0WiIgnmof4cCcH87sOn4NZLj81VLlGlKSGSocsO5/1M9CfDLP5uKZJBEARBgpnoJ2nih8lVMnbxDnyDReAwD8Gxm4s2rr3oMNx26bEDdow89cGb/EhGpRqqtnpl4U/Yt7bGPmKun4gjiScntehK06RGQI1keGQ9c/Fn+bFfPIvVW7trGJkcySAIgiAokkH0izTx40UyyGGuhaGIZADAe46Ymnubgcq5isoW5To7zIuuPDNRrCYhPvNFXyj3Zzwio61DOuXAbc6cYZbWu+XJVTU1mhHOMlXJIAiCIIeZGEAY88XEIFbJGA7wIK/a+IxrLUX+rTdjWooYP6KEK6U62PW4kRjdUkyccJeEeKpSCBxmEV3ILyyTHeZQMbt5M8zSerWmMkRbepLLBEEQ5DATNXLw5FF4dd1O7WsH7DkSmzv7sKOnAosxHDNzHN59+OR+VyfYXchbEWEo+cDcabAYw3uPHJhyfgXbwsIrzgAAfOG2lwbkGHkRT0yEUE6rFJNEUlk5OX8c3ETV4DC7NTrEwXakmAmCILI5zIyxsxljSxljyxlj8zWvn8QYe4ExVmWMvV957eOMsWX+fx+v18CJoeXWS4/F/V86Wfva3754EhZecUbQFnv6Hm347w8dUbOjt9sRCJTGV8y2xfDBo6fVpR35roKIZIhz7s+TkyTB3FN2gp/zRl7kEdUaqSC9TBAEEZL6LccYswHcAOAcAAcBuJgxpgbi3gbwCQC/V7YdB+BKAMcAmAfgSsbY2P4PmxhqRjUXUzvXFSwGmueXn7FtJYwf0YRvX3DwUA+lofjg3PzZ6oHAVh3mIJKRf19JgrmrHDa5CR3mfGXlgNojGVSSjiAIIiSLLTQPwHLO+UrOeRnAbQAulFfgnK/inL8MQG0rdRaA+zjnWznn2wDcB+DsOoyb2AWwLZarSQXhUbQtLLzidJx3aG3NOYYr33//YVh1zXlDPYygSoaYkJinTJ5KKeGOsltymAXq2n3V+Drqev2NZJgc6mff3Io/v7S2pn0TBEHsamQRzFMArJZ+X+Mvy0KmbRljlzLGFjLGFra3t2fcNdHo2BYbskoPBDFQqJ0rs5ZLlJMbP774CHznPYckRlnkNuq6KhkPL92E/a/4G15Zs0MzyPDH2if9+cc2vP7Bm55qmFw5QRDEQNMQwUPO+c2c87mc87kTJkwY6uEQdcK2LOxG0VZiN0EI5aIthLP5Q37vF0/CM//2Tpx+4CT84hNHB8vPOngSPnLM9MSSdF19cobZg8ETz39+aS3uf20jAODJFZtj29Ynw+xtV6tDTRAEMZzIUiVjLYBp0u9T/WVZWAvgFGXbhzNuS+ziFCiSQQxDwrJy0Ul/uol5++85EgDws4/PjSwXzU6SJgy2d/ah4rgo2lYkw/z0yq0RZ1d2ogXRDHNtgpcyzARBECFZ/L/nAMxmjM1kjJUAfAjAXRn3fy+AMxljY/3Jfmf6y4jdANtiubq6EcSuQFAlQ5n0l2sf/iZJ2zoux5ubu1Bx3EieuKO3ElmvKyXrLBqgrN/RAzeHCA4iGaSbCYIg0h1mznmVMfZ5eELXBvALzvmrjLGrACzknN/FGDsawP8BGAvgXYyxb3POD+acb2WMXQ1PdAPAVZzzrQN0LkSDQZP+iOGIKI/Y5P8rPuN5hCVTKm2YOPPaRyO/OzxeKaO7rHOYw58rjos3N3fh1B8+jPnnHIDPnryP9ljd5SqWbujAEXt7hYx4INKTz4UgCGJ3IFPjEs753QDuVpZ9U/r5OXhxC922vwDwi36MkdhFeffhkzFrQnLpOYLY1bjg0MkY31bCGL+7YZ57wg8cNRV3PL8m+N3OGfJ3XR7LJHf0agSz5DH3Vhys2NQJwKtsYRLMV/75Vdzx/Bo8Mf80TBnTknmyIOecJvcSBDHsoU5/xIDxpTP3H+ohEETdGd1axDlz4iX/sujL/3zfofjue+cEvxcNDvNxs/ZAd8XBotXbI8t1eeStXeXYMlm/9lbdoKbziCbz//JXb+sGAKxs78SUMS1wUsrKCRyXZ64UQhAEsatCNQwIgiD6QR6paFksqN8MIHCpZU7cdzxuvfRY/PmyE3DWwZMirzkZBbNMb8VBpz8xsC1BME8d2woAWL21B4BUhzlx70DZUcvvEwRBDD9IMBMEQdSBWsq3nXXwJHz/fYfiorlhISLZRS4qkQ3X5TGXuVyNC1a5wkVfxQkqaYxoMncW3Gt0M4DQac6aYa5UKeRMEMTwhyIZBEEMCo997VRjZ7pdwFogAgAAIABJREFUmf7Edxlj+ODR0/DqurD5iCyIS4pgdlygTxHIOodXdqJ7Ky46e6MO8/NvbcXsSSMxqrkY23b11u7gWIC+XJ5Mn+MAiO+HIAhiOEGCmSCIQWHauNahHsKAMGv8CFw8b2986oQZddmfPNlOzQY7nMcc5YpGMMvl43qrDjr9JigMDL0VB++78SnMmzkOt3/muGC9qr9Nb8Xxx5HNOa445DATBDH8oUgGQRBEP7Ashu+9dw5mTxpZl/3JYlcXyVAF6sadfZgxf0GkvFzUYXaw06/dXHXdQGAvXhttqS1iHOLfrJEMXSSEIAhiuEGCmSAIYoiRNWlShvn/t3ffcVKV1//AP8/MzmzvS13aIk0QqQKCqBQpNoxKbDHWmG+iPzWaGI0xaowtxthij90oGjVqIlZAEalLlV6XZRd22d536vP745a5907Z2QLbPu/Xy5cz9965c3eHgTNnznOOkmEOXdZSUuMKnMMQw7q9flTWKwGz2+eHVw24rT3StUBayzTrg0uauPZQGW4ioq6GATMRUTszZnGNJRkOS0mG3y/DdqUwnsO06M/rR2mtEkx7fVIPcK211z49ULZmmiNfOzPMRNQdMGAmImpnxoV1kTLM2w5Xo6LePBo71OOMJRlurx8FaucLr8+vB9zWtYpaZlnLQAfOFzliZls5IuoOuOiPiKidmTPMgTsxloB5xd5SrNhbGvIcWm3z93tKTOUZXr9Eaa3Sq9ljqIG2WYameNXA15pZZoaZiIgZZiKidmeqYTbEn85mTNDTAtcrX1kb9hivL7DoL1yGWctORzu4xGMItJfsKG5RP2oioo6OATMRUTsLl2G2lmRYZSQGJgW6febFgCdlp+CySQNM27y+QFs666I/a5cMYx10JNr5nvpmN657Ixc/7C2L6nFERJ0JA2YionYXuobZWpJhlZUUCJitA00m52RiZN8U/b7TboPHsGhQCGtJRujFfk1ljLUM87q8CgDA+oMV+OVbuVEH3EREnQEDZiKidmaMSR9fOFa/3VRJRmZirH7bWktstwnEGgLulPgYpSTDG7pLhtdvrmE2lmRU1rtxsKwu5DVogXpxdSMAYEtBJb7cVoyjNY2oagi9QJGIqLNhwExE1M60gPnhC0djdL9UfXtTJRlZyeEDZpsQcMYEHp8c54DHZ1j0pwbM3+8pQX5ZvR4oW/sw+/0SP391Lc547Ft9YaCRdr4jVUrAXKcOULnmtXUYc/9XrGkmoi6BATMRUQdhzSc3VZKRaaphNgezfilNAXNSbIxp0p9Qn+3KV9Zi5uPf6oGv37L4DwC2FChTATcXVAZdgxaoN6gjtRvcyv93FtUAUCYREhF1dgyYiYjamdaH2VomYR1cYmWsYbZmmF0eH2INAbPDLpRFfz5t0V+gPtnrl0EZZmkoyTixj1ILvdKwoE+7Vo/Pb3ruerd58eGOouqIPwMRUWfAgJmIqJ1pyVxhyTE3WZKRZC7JuOmdDfp9l9dvyjA77DZ4jG3lhECjJxDoBtUwG2Zja8FzWZ1bPz5GrenYVVyDYX/8XN+uBcxayceOIwyYiajzY8BMRNTOwlX5Nq+tnB//23JEv+/y+uG0mwNmrz/QVk4IoNbl1fdbu2QY4mXUNCrHVRsW8dnViHiLpUyjXq1h1n6mslo3iIg6O076IyJqZ3q5cJQlGXabwM9PHYhYh13fpgW1GrclwxxjF/A2BjLMNiFQZwyYgxb9BUoztG4Xxq4XWh9nl8dcCqJlmLWfiZMAiagrYIaZiKiDsIbH4TLMpw/Nwr3njTIF1I99uct0jMvrQ2xMIKCOsdng9km41Uxyrcurt4IDAiUZfsukP49f6pno6kYlYL7kxVV6YNzoNdcsW/tBtyZgbnD78Nt/b0Z5HbPURNS+GDATEbWzwKI/c8is1QlbaZnjSCUbwTXMwtSHubzOjUteWq3v10oytNZx/hAZYi3DvOZAeeB5PJEDYmv3jubYUVSND9YXYF1eedMHExEdQwyYiYjam77oz0wLoMcPSDNtd6qZ44gBs8dv6pIRo9Ywe8IEsNbaZeukvhibQHWD1/qwoK4YQdfh9WHv0ZqQPZybogXxjZ7Iz3GsfLShANe/sa5dnpuIOhYGzERE7UwvYQ7TRc4mrN0zhOn/obi8PnOG2SZMXTKsAjXMfvj8Eqv2lZn2989ICDm5z7hwMJSDZfWY/ffleHDxjojHhbwm9VqbCsqt6lxeHCqvb/bzWd32/mZ8s+Noq89DRJ0fA2YionamtW0LFzBbt8dGUZLRKyXOkmHW+jCH7slhbCv39693YcXeUtP+iQPT0eDxNbsmubRWGVzy/R7lfD/sLcWIez6Pamy2Vs7R0MyA+apX12L6X5c16zFERJEwYCYiamfh2srpw0MsB2jt4sIFzNOHZuGxhWMsXTJs8Pr9YQNerSezzy+x/mCFad+I3sk4KVsZ2a0t/IuWVsbhUhcHPvXNHjR6/FH1Z9ZKMhqaWZKRq16/1uKOiKi1GDATEbWzcINLtNHYCbHmDqCBRX+hU9ILxmYjNd5h7sNsE/D4wtcw16iBsF8iqFZ5Uk4GeqfGAQDySuui+ZF0WrCrLQ7UFzhG8Vgt693SGua26gEtrZ9YiKjbYcBMRNTOwtUwjx+QhtvOGoa/LTzZtF3LLMfYQv8VrgXUWsB99ujeSoY5Qg2zceqftVxiUGYiJg3KgBDA8j2l1odGxdpuztoRJBStfKS5NcxJ6geMsjZqR2ddAElE3Q8DZiKidvb7ecMxc0RPzD6xl2m7EAI3zxqKnslxpu3GUotQnIbM84Z7zsKTl4xDjF3AY+mS8ds5w/DgT04Kery17/GAjASkJzoxoncKVlsWA0bL5fUhN68c6/KUcomPNxXi2tcjd6DQFv01tyQjOU4NmNX66dbyMmAm6vYYMBMRtbN+6Ql49epTkBgb3fBVLWCOVyf9jbO0nTPWNmckOuGMscFhs8Hj85syvXEOe1B9NBAcoPbPSAAAZCU5UV4fCKa1wDQaLq8fF7+wSr//zpp8LN15NGKdsd5WrpkZ5kDA3LIMc6PHh435gTpuBsxExICZiKiT0WqTUxMc+Ozm0/DuL6ZgVN+UiI9JS3BASmCDYUFfnMOOi8b3Q8/k2IiPHaAGzHEOO2oNI7hT4hxRX3O4MuDSmkBQW2rJCLtb2FZOK8koaWGG+U+fbMVPnlup3/eF6SxCRN0HA2Yiok7GWJIxqm8q4hx2vPfLUzGidzKA0DW3fdPiAQB5ZYH+xHEOO+Kddtw6e1jY59r5wDzEO5VMdmyMDXWGvsvWDHO4yYSRlNYpQe2+klpM/Ms3eHv1QX1fS0sytL7VLc0w7yqqMd3XFh8SUffFgJmIqJNxhmgnlxQbo2eC/SHSuVqXCwAYlKlljJXzZCSGzxTHqWUf2u2aCAFztCUlRlpQW1KjBM7vrTuk7/O0sK2clpmubGhZwJzgNP8cXPRHRAyYiYg6mXCL/uxqhjdUI4y+qfH67QGZiQCAOHXEdrSlFVqArUm2PC7JEjDbo8g4awvztHrlQxVKBtzt9WNHkdKrublt5bQWdtVRDEcJJTHWbrrPGmYiYsBMRNTJhAuYr5g8EAAw1rIIEAB6GOqUtb7CMWo3jZT46ALm2BhzIGnNMFtHdieqpRwDMhLC9ozWWr/VqYv/KuuVIPfmdzfiow2FAJpXw7wxvwK7ipWSimimCYZizTB7WcNM1O0xYCYi6mTCTfg7bWgW8h45B9lp8UH7tGzvtCGZeomB1sc5M8kZ1fMGZ5gDgeWDPzlJf460BKe636E/rm+IawICpRjGbhlSSnyxrUi/35zR2MbFei0NmIMzzKxhJuruGDATEXUCr119ChLUjG1TfZjD2fnAPLx+zSQ9Y6oFuH1S4/HilRNw73kjIz7emmFOilUC4h7Jsbhi8kB9AEuamrHWSjSykmKDyjUAwCagt2+rcwWCYuukwZZO+tOy1c3FGmYismLATETUCcwY0RPXTx8MAIgNk2FuSpzDDofdpmdMjWUSc0f1RmZS5PZy1gyz1gdaX4SoxpWpasAcpwb42WnxerBvdM7JfbG5oApFVY2mDPPVr681HdfctnKalmaYraXXrGEmIgbMRESdhEON5FqaYdZoAaB1UZ69iXHVxo4Zyn3lOrTAu7pRCVCz05XyC23RXd+0+KCsLQBMyskAoLSU23a4Wt++Mb/SdFyDx6fXXTeHy+tvUXbaY6lZZg0zETFgJiLqJOz2tgmYf33mEADACT2TzOdvoqtFrOV5tQA6Rs0wl6ot4ob1UvpB1xgCaGtdMACkqDXQD/xvOz7ZdDjicxsnFIbjN2SCs9S67JZkma0lGJUNbgy68zO8uza/2ecioq6BATMRUSfhsGkZ3db91T3vpN7Ie+ScoHZyTQ0esWaYY+wCQgRfz5yRvXDttBy8+4spuPSU/jh/TN+QGWatO8dOy6CQUBrcPuSX1eMtw2ATq0ZvIJusLTxsScBsXeR3UB328sbKvGafi4i6huZ3mScionYR00YZ5nCazjCbA2a/BBKdMXBaWsYNzEzEn9QFhI9cdDKA4PpnIFDrHI16jw8LX1yJ4moXFk7oFxS8A+aFg1o2vKbRG3RcU6wlGFqXjtZ+UCGiziuqd78QYp4QYpcQYq8Q4s4Q+2OFEO+p+9cIIQap2wcJIRqEEJvU/15o28snIuo+tNKHUJP+2kKTAbMl6PX5/EiMtQcFkqECeq2FnVG0A1MAJWgtrlZa0GmDSQClLd13u0v0Y/TnU6+p1tWCgNlSkqFNGgzXS5qIur4m/9YVQtgBPAtgPoCRAC4TQlh7D10HoEJKOQTAEwAeNezbJ6Ucq/73f2103URE3c6wnknIyUpEVhPdLFqqqYA5zpJh9qkZZi1gfuCCk3D9aTkhHxuq3CMlPvovOY2L9yY99I3eju7F7/bhqlfX4u3VB/XhJwD0rHddMwPmQ+X12FpYZdqmneNYZfaJqOOL5t0/CcBeKeV+KaUbwCIACyzHLADwhnr7AwCzhGhiuTURETXL5MGZWPbbMxEfokVbW9AC5pS40IGsNcPs90skxcXo26+cMhB/PDd0L+dQwXioDPNlk/qHfLyxtZzL68f1b+QCCJRcrDlQbmpN52hhhnn6X5dhz9Fa0zat+wdLMoi6r2je/dkADhnuF6jbQh4jpfQCqAKQqe7LEUJsFEJ8J4SY3srrJSKiY0TLAhvHaBtZM8zJcTG4++wTcevsYU2eO1TAbOy6MfvEnvj2t2fi4QtPDvn4Bkt7OG2kdo1LCWYr692mGuYJA9MBALUtqGG20gapHKtSGCLq+I71u/8IgAFSynEAbgPwjhAixXqQEOIGIUSuECK3pKTkGF8SERGF0qjWBocNmA0Z5ptnDsHCif0xeXAmxvZPa/LcoUoyjF9EThmciUFZiQCA1685BU9eMtZ07FWvmoeZAMoIba0LRkW9W88wP75wDG6aqbTOa25JRihahpklGUTdVzTv/kIAxu/I+qnbQh4jhIgBkAqgTErpklKWAYCUcj2AfQCCUhFSypeklBOllBN79OjR/J+CiIhaTR88kpYQcn+yoYTiZ1MGNlnzbGRr4th0tQ0cAJw5vCcuGGf9IjNYjcsbCJjrPHqG+ZRBGYiNscMZY0NJrQtbCiojnaZJ2nNoJRm7i2twoLQOg+78DIvYm5moW4gmYF4HYKgQIkcI4QRwKYBPLcd8CuAq9fbFAJZKKaUQooe6aBBCiMEAhgLY3zaXTkREbWn2ib3wi+k5+FOYOuTMxEBQ25xgGUCTCxXTE6PvmKGprPPowWxlvVsflJKgDklJjo3Bm6sO4vx//KDvawltYmGMXcDj82POE8sx78nlAID3cw9FeigRdRFNBsxqTfJNAL4EsAPA+1LKbUKIPwshzlcPewVAphBiL5TSC6313OkAtgghNkFZDPh/Usrytv4hiIio9ZwxNtx9zkikJoQOXo1Z4phm1vNeNmkA/rxgVNj9aYYMc7QqG9yorFeC2Tq3DwfL6+GMsSFDPVdibGDxYqPHPIzkaE0jFr6wEsXVjU0+T7VaBy0lUFDRACAweZBDs4m6h6h6+kgpFwNYbNn2J8PtRgALQzzuQwAftvIaiYjoOHvlqomoc/vC7m9uT2K7TeDKKQPxp0+2hdyf0YKA+cfCKtQ0epGV5ERprRtbC6vQLy1eD+zNAbP5Z3l3zSGsy6vAm6vy8Lu5IyI+j5bFdvv82F9i7qAhGTETdQtcwUBEREFmndgL54/pG3Z/qEEkTRFCYMntZ4Tclx4iYP7qN6djxe9n6PefvXy8af/d/9kKAMhOV2qutxRUITs9Xt+fbAiYCyoasO1wFb7eXowvth7RpyYah5TIMNGvTz3G4/XjQGkdACBcRYrH5w+9g4g6NY7GJiKiZmvp1LsTeiSF3J4covfzsF7JAIA/nnMiJudkoldq6Drocf3TsKWgEi6vH9lpgYA5yXDOy15ebXrM7+YOB6CMwbZmn8Px+qU+1ESLs40hdlmtC1MfWYrXr5mEvmlxGJiZGNV5iajjY8BMRERRe/u6yfjsxyNo69lUkbpoXD99MIBAFw/r9UzKyUC8047nv92nB9kA0Csl/ELDcrWPc3WDB5Me/AZ2m8DS28+MeI3ldW6s2lcWdn9prRsurx+PfbkTG/Ir8c71kzF1SFbEcxJR58CAmYiIonba0CycNrR1QeC103L0koZJORlYeyC6teDGQScf/XoqBmcl6osFfz9vBC4an43+GYGWeH1T44POoSmpcQEAimtc+qI+42jtULYWVsHrl7DbhF6moRUxL1qbjzs/+hEAsCFfaWO3q7iGATNRF8GAmYiIjqs/nRdoW7foF1Pgj3LlnHHS3vgB6UH7h/RMNt031jNb7VXHXx81dMmoj7DI8YQeidhXotQv906JQ2Gl0i1jc0EV9hTX4Llv9wU9JtyPVVbrwlWvrcVzl0/AgMzQPa+JqGPhoj8iImo3NpuIukVdc8tA+kTIMO8sqgaglFFotCA4lL6G2ugsyyTEs55Yjvzy+qDHhPsYsHhrEbYWVuP57/aGfT4i6lgYMBMRUZeUlRS+VZ1WUVFRHwiY9x2tDXM0EO+w67d7RDivUbiuGynqYkStFKQjavT42PGDyIABMxERdUlDeibh6qmDwu4/e3TvQC0ygHV54WupHYYseFNTCzWFlQ14dtlefLG1CHep9c1AoCWfNkGwIxpxzxe44uU17X0ZRB0Ga5iJiKjTOCk7BRMHZkR1rBACv5s7HK+vzAMAXDNtEM49uQ++21UCCIHstDgs/rFIP/7LbcVhz2Vso5cZZYb5tR/yTPcfvnA09h6twUOLdwAAahq9eD/3EKSUuOSUAVGd83haG+EDBFF3w4CZiIg6jf/9v+nNOj7OUEpxzzkjYbMJTFAD7i+3FQUdL0TwYr3hvZJblGG2cnv9uOrVdXqtdE2jB3d8sAUAOmTATEQBLMkgIqIuy27o72zt9ZwW79Bv/27ucGQkOnHB2GzTMZNyMvDhr6fqCxNjbCLkVMJo1Lm8cBvqgqs6aElGuNprou6MATMREXVLaYbA9/ShPbDhnrNw21nDTMcM6ZmEpNgYONWSjHiHHb1T40zHXDjOHGSHU+vyIsYQtBs7dETrSFUDblm0EQ1qC7zCygZsyK9o9nkicUex2K+oqhHD//g5Nh+qxHe7S9r0+TXf7jqKnLs+67AfLKh7YcBMRETdUnpCIMOsjebWSi+ykpy4eeYQ3Dl/BADoGeZ4px39LP2de1kC6HCm/3UZjqoDU1rqocU78cmmw/hqu1JOcsZfl+HC51bq+5/6Zg++ClFq0hyNnvABs98vUVrrwpKdxXB5/Vjw7A+46tW1WJdXjuLqRsx6/FscCtFiryWeWboXUgK7imra5HxErcEaZiIi6pZSDQGzNiHQZkgj3TZnuH5bC6QTnHb0TjEHyL2So69pNnblMPL7ZcTx4BrtEK9P4r11+fBazvfEN7sBAHmPnBP1NVm5PKEHuHyzvRjXv5kLAJh6QqZpX2W9B2sPFGBfSR3eXnMQd80/scXPr9FKQ9p4CjtRizBgJiKiLm9kn5SgbbExdjxz2TiMH5iu1zqnxClB9KWWRXhaSUacwx40aKVHcnQZ5khm//07LLn9jCaHs2jX+fGmQny/p1Tf7lNHdreFcBnmV1Yc0G8ftgx5sdsC19ZWJdCspKaOhAEzERF1aXsenI9woeR5Y/qa7sc57Nj9l/mmNnJAoCQj1tB1AwB2/HkeVh8oa/U17i+tQ1mdu8kOHFoN9O5ic5lCnduL2Ji2qbJs9IbOMB8sq9NvW4eu2ISAXQ32/WGy6M2lBd5tdT6i1mDATEREXZojytHbGmeIwFMLRt1eJfu6+d45gFRqmuNi7EHHt0RBRQPWHSjHhIHp6JkSOmutZXGrG8wB64aDFXA28+cMpzFMScbhqkb9dnmdecGiX0q9dKKt4lvtNNEsQiQ61rjoj4iIqAmnD+sBANhxpBoAkBrv0Gug4xxt80/p/pJa/OpfG3DFP8NP2NMC5gZLUHv1a+tweYTHNYexJEOruQ5Xe6259vVcfKUOfvG3cVs6V4RFiETHCzPMRERETTixTwquPy0Hw3olB+2Lc7RNhnnP0VoAQJ6h9KE9GDPMDR4fkmJjUBJFdw9tMmBTwfVv3tuEM4f3wIKxTbTjUwNvl7drBsx+v4TXL0N+o0EdD18lIiKiKPzx3JH46Sn9g7ZbA2ZnjA29UmLxt4VjmnX+jWo/ZWuC9qttRXh3bT6klFFnW49UNWDlvtKQ+w6W1UFKia2FVVh/MHj8tSlgdvtQ5/Ji3lPLo/wpms4w/2djIW5ZtKnJ82hncYWpqe7sfv/hFgz74+ftfRkUJQbMRERErWAtyRjSIwlXTB6Iiyf007dtvncOzjcsMLxyysCg86zerwSvxnCzos6NG95aj7s++hEHSuuCSjHCOfXhpbj85TVYvd+8IHH9wXKc8di3eD/3EM59ZgUuen4VZv7tW1NW2JjRbfT4sCG/ApX10Q8PCZdgPlLVEFWmusHtw8OLd6DO5Q26nvbi80vMfPxbfLKpsM3O+e/1BQA6zqLGV1YcwJmPLWvvy+iwGDATERG1gnXR36c3TcNNM4aYtqXGO9A/Qxl4cvXUQZg7qre+79nLx+OdX0zW7/v8End8sBmHKxtw0DAEZEN+ZdgFeeG8vfogGj0+vLMmHz6/xF617CM3LzAdcH9pHWoaAwGxtSSjztW85wwXAJ768FKc8uA3TT7+zVV5eHH5fuwrUUpTtL7Qu4pqsGpf6zuStERJjQv7S+qw8xgMUWn0+rCrqKbZr21be+B/25FXVt9kSU13xYCZiIioFawlGTF2W8ghJH3TlIC5utGDWENW+pyT+6CXpSvG+7kFuO/Tbdh8qFLftiG/IuoMs2ZnUQ1eXr4ff/jPj/hkU6He59lnKZuoMbSJazRkdOvdPpTUNm86YYPH16Jpf0t2FGPJjuKgPtBahnnuk8tx2curm33ew5UNePjzHfC2otvG4Sql77Q2krwtHa124dxnvtczztFo9PjaZGR4cXVj0LZaS8tAUjBgJiIiaoVI/Y8vGt8P152WAwDoqQ44qahzBz1GG81t9NX2Ytz76TYAyuCVvNK6iGOrjeIddlw0vh/yy+r1gDevtE6fnvfRBnNpwUcbCvH3r3fD75d49POd+vbqBo9eRvH1b06P6rk/3XwY0/+6zDTcJFTW8rMtR/TbWwurcN0bubjujdyg41pbkvGXz7bjxe/2Y2UrstNHKpXA8lgEzPtKauHxSZQ144PJhc+txJj7v2rV8369vRiTH1qC7/eUmLZXN7Y+EO+KGDATERG1gs0mwo6ifvynY3DPuSMBABmJShu68noPYi1lHNqEQc2Qnkn67R7JsRiYmYCjNa6oA7bzxvTB+IFpcPv8eHPVQQDAir2leH3lwZDHP/HNbjy9ZA/W5ZWj1hXIMJbXuVFS40JWkhNDQ3QIieSooV45VDB44zsb9NuL1uXrt62LBq2lCtKwf19JLbYWVkW8Du13bZ1O2BxH1Axz/TEomzhQqpSe1DcjGN+utjeUrWjhpy0yNX6LAaBNMtddEQNmIiKi42Bkn1QMyEjAHXOHB2WYrfcXTuiHM4crvZ8HZiSgR3IsSmpcIetcJ+dk4MlLxpq2JcbGoH96gmnbhvxKvY90OB9bFrWV6QFz8ATCHsmRpxKW1wWC5KNNLPb7YmuxfttaJuDy+k0fFI5UNWLek8tx3jMrMOvx73DuMysAADuLqk1Za01mohMAcKjCXCaSV1oXdRB9WM8we1FV78GPBUqQLqWEp5WDVfarAfOH6wuwYk/oziYAUFjZgALLz2AdINMcWk9v6+VHk2Gud3tD/q67MgbMREREx0G8047ld8zAtCFZQXXPWm2xJiPRiZ5qQDogIwE9kmJR1eAJmf0bOyAN2enxpm1JsTEY0z8NY/qlomdyLOaO6hXVNX6pDh/58FdTEWMTKKt1oai6QZ88ePXUQTgpOwUA0DM5Fg8sGBX2XGW1gWDuaE1wrSygTE50eX0orXVhyuAMAMAOy8I6l9en1xADwLJdR7GzqAY/GjLLUkrMe/J7U9Zao9V97ymu1bf5/RJn/u1bTH1kKZbvLjEFoh+uL8CtizaisLIB+WXKdj3D7Pbhpy+uwnn/WAEpJV5ZcQBD7/4cVc3oImJ1QF3cWFbnxs9eCT98ZtojS3Hao+YuFkeqQv9eQzlUXo/cvEAbQZtWz+73mzLV1imSodz36Tbc+M4GbCmoDLm/0ePDW6sPdpgOIG2BATMREVEbiHfY9SC3KZHqngElYE5PUDKj2enx6JminLeszo3UeHP5BqTy3IDSAzo7LR4/ndgfqfEOfHLTaVh792w8dek4JDqbHrCiZSwHZiYgI9GJ4moXdhfXYngvpUTkvvNH4YP/m6r/DFeeOqjJcwHKwrZQKurdesu6cQPB2DlWAAATv0lEQVTSAQSXCLy9Ot8U6NWEWJRmfC5r2Yp2fKGaTa5p9GDwHxbr+3/+6lpc+lJgMeHt/96Mjzcdxk9fWIXTH1uGvUdr9MC03u3DrmIloG/w+PDvXGWhnpa9Lqiox/6SQGAeDa0kI1p3fLBZv72zqAbznlxu+ubgi61HMO2Rpdh+2Pxtwrwnl+PiF1bpwbGWYfb6JeoMv7NoMsx56gcJY3B9sKwODy/eAbfXj2eW7sE9H2/F4q2hs9ATHvgaty7a2OTzdCQMmImIiNrAlvvmYOWdM6M6NraJcdrxTjvc6nflCc4YU/nD5JwM07ES0Esmfj9vBH64cyb6Z5jLMeIcdqy8a1bE59QWHgoBpCc4kZkUi7V5ZXB7/RjVN9V0rniHPeKEQ2eMTQ9i//7VLtz50Y8hjyurdevHjc5ORd/UuJDH/f7DwONDdeDYbggYVx9QFvd9sqkQ6w9W6C3zth2uxsIXVuIfS/cGPb6goiGotEILsFfsKdUzzMaSmOtez9Ufo2X+T3t0GWY+/l3Q+d1eP4rCZIOLQnSqaHD7UFTViMU/Hgmq/34/N9BN4/11h7CzqAZPfbMHgLK48pZFm1BY2YBvdgTKXKQMBMXF6ocXr2HseYXhA0d1gwd+v4xYy6x9H2JsR/jaD0o7wLdXH9Rf04Nl9Vi6s9j0WCklyurc+HjT4bDn74gYMBMREbUBh92GGHt0/6w6mzguLd6JAWrQm5OViD6pSsnFTTOG4Ndqj+cENWM876Te6J0ahy33zcG10waFPWdKiE4cRj8Zp4yqllLJPtY0enCoXAkUR/ZNMR2bnuAImyX//JbpyEx0YmN+Jd5alYenQwSomuLqRj1YS0twYMaInmGP1QL6QxXBdcdXvrJWv33Na+uw7XAVblm0CRc9v9K0iHFdXgVeXL4/5PnPe2YFPt0cHMSV1rr1GmzjwrxV+8v0+uNwJSeaV1YcwIy/fYvSWhdOfXgJ/p17KOyxtS4v5j65HFMeXoJf/2sDHvjfdrjDdArRMssurw/3/3cbbl60Ue8q8sJ3+zDyT1/gz//dbpooePnLq+H1+dHg9urPV2D4nVY3ePD6yjyMuf8r/UNDOKWGQFv7ALVs11E9e/3Yl7tw7eu5pt9PaW3L667bEwNmIiKi40wLrGefGBwgPnnJWIzsm4KfnzoIb147CXNH9cKJfVLwxa3TcfucYTipbwoWTuiHj2+chrxHzsF4tZQhJc4RVAttZNw3YaDymEcvGq1vu2Zajun4M4Ypiw5/N3c4hhq6dgDApJwMjM5ORSgn9klBRqITa/PKcc8n28JeDwBc8/o6XP5PpW43I9GJaUOywh57qTqWvEDNMC+5/Qz957A65+kV+u1QJRyaXimx+PBXUzGidzJ2FtXg5nfNZQLJsTHYVVwDKQGHXSA/TH/po9UuU7cOaxb8x8JKNHh8eGXFARypasTvPtgS9pqKqhpNz7OzqAabQ9QK220CNeqHgWW7SvDaD3n6QrxzRvdBvduHercPr/5wAB5foJZ4f2kd9pfW6Rnnf63Jx9NL9uj7D1c1YpU6IfL9dYew8IWV+M9Gc49oreT5no+3YvGPR3Dt6+vwwnf7ACjlMXbLn8NJDy7BBrUrhzEIX7k3/CLHjoYBMxERUTtYe/csPHvFeP3+L88YjIkD03GBmum12wROH9ZDD3RH9E6BEAIxdhseWzgGw5rZ5g0Avr9jBtb+YZaenY5z2HHHvOF4+rJxaiY7DqcMUoLQ+84fha33z8WNM4YEBeJPXjoOt80ZHvZ5TuiRFHZfOOkJTkwMEwADwFVTBwFQ6oXjHXac0CMJH/5qKv51/eSwjwGA/SV1SE8w132P6ZeKa6YNwso7Z2HCwHQsGJsd+poSnfh6u1JSkJOVGPY5jta49G4dADD9r8uwq6gGLq8PK/aUYre64PD5b/dFvFYguP3dzqIaLHxhlWnbyjtnhv024bwxffGbs4bq9/uEKHOpqHObar21APnC8dlY/OMRvfTkqSV7sC6vAn/4aCvO/8cKLFHLPIzlGr/+1wYs3XlUv19Z7wnZL/y/ava+0JDNvvyfa1rVGu94ivz9DBERER0T2iATzV3zTzzmz6nVNmtfmcc57Pj1mYEx3svvmKF3T3DYbXBEUWLy6tUT0eD2mzpUTBuSGVTekBQbYyqPsEpLcAT1p95y3xzYhYDdJuC025CV5ERprRuZiYEAOFyWee6oXvhyWzHcPj8GpySiwtDJYtqQLNwxb4R+f4Cl5htQss9pCQ7klwMjeifjjGE9sLu4FlNPyAwagrK7OHhk9tNL9mBM/1Q8tFgZBNM7JS5kvbLVV9uLIu5/+rJx6JsWj0k5mXj5+wP69okD03HPuSMxvHcy4hx2rLprJtITnIhz2LGnuAbzn/oef1s4Bre+twkV9R7UhXgtrpmag482FOJ7S3s7j8+PLQXKcJkDD5+NsrrwbQILKxvwXoiSE4fdBillUJa+sLIB/dKDf/8dDTPMRERE3Yz2lbnNkjl22G16MB2tmSN64ZyT+2DWiJ74zexhAID5o/tg6gmZuO2sYfji1un48tbT8cxl4yKeRwuWt90/F09eMha/PH0wUuIcSIyNQZzDDptN4AI1E+w1tCuLc9iR4LRj2pBMfPirU/Xtz10xAU61zvqskb1w04whOG9MXwBAiqXTSP8Mc1u+E3ok4uMbp8GrljJcNmmAnnUdZannBqAHmHfODwThG/Ir9P7NAHDjzCFBjwvl7dXKEJfThmThajWrrll550ycr/4MWsmM5uR+aRjTP02vJe6TGq/fHtorGXsfOhunqAtGtxRU4qvtgcV4T106Fm9cOwmj+6WGzPIbf9/bj1S3qP/zS8v345HPd+oDUzSzHv8Ol760Kmh7R8MMMxERUTcj9B68bfd1+CtXn6LfTolz4J1fTDHttw5dSXTaUef24alLx5oCsMTYGFwwLlsvTTFaOLE//rniAEotnSM23zsHNjUT/d4NU+D1S9htArfMGoqvthXh0kkDkJ0Wj/vUUePWGtuBmeZyi/PG9EWf1Hi9xdqAzAQ9KB7e2xwwZyXF6tczcWA6Hr1oNPLL6/Hssn1Yp7bDe/cXUzBlcAbu+Xir/ji7Tei/f+0c04Zk4oe9Svb62SvG62UMF43vh1tmDUXftEBg74yx4b0bpuAStSXe7JHhF0xqtNKU5wylIb+ZPcxUknLNtBzkHqzApJwMrD1Qjj6pcaZ+z1p9+FvXTcKynSV49YdAlrspLy7fj7QEBy4cn40JA9Nx93+2wuX1Y/X+ctS52n6KYltiwExERNTNaJUW1jHUx5Kxlnb3X+ajqsGD5btLwtYPhzK8dzJS4x04ZZC5tZ6xdGTy4Ez99o0zhuDGGYHM7sUT+uH1lXmYPdI8yCU13oF9D52Nm97ZgM+3FiEpVgmPqtVa3QEZCZh3Um98s6MY04Zkmh6bkejQA+beqXGYOCgDG/Mr8Oyyfdh2uBrnntwHp55gfgwA9EuPx0Xj+2FknxT85bPtKK1VgtdD5Ztx8YR+SI0PdCLJTovDgMzgsoXJgzPxh7NH4KHFOzHJ8jsJJd7SCtAmgFtmDzVtmzOqF66eOgiXTRqAmkYP3F6/vjDTaMrgTD2Qzkh0hsw6L739DORkJSLnrkDf68p6D84Y1gMLxmZjSI8k5JXVYcHY7CZ7k7c3BsxERETdzE0zhmJDfiWmDA4O5I6VnilxeOnKCdh0qBLOGBt6JMfiogn9mn2e9X+c3eyyEc1J2anIe+SckPvsNoHBPZRMsxYwzzqxF/6zsRDZafE4oUcSLhjbN6h1oDEz2kudiDi2fxoyE50oq3NjbP80ff/zV4zX65+nD83CnFG9AQD7Smrx8Oc7Max3Mpb99kz957tgXDbK69z6gsdQbjj9BNxw+glR/fzWxZuhPi457Dbcd35ggqPLG5z5/ezm0+Cw2/TuKTlZiXjhZxNwpKoBtyzaBAB44pIxGKwu/nztmlPwnw2FWLbzKHqmxOKc0X0AKAH/5OP4Z7A1GDATERF1M6P7pWLd3bOP+/POGdVbDxJbKtpe1y2Rk6UEeElqz+eHLxyN2+cM02uBted+/5enomdyLK5/Mxd3zR+B6kYPlu8u1TPdQgg8/7MJWH+wwtSub/7oPpivBotGN5w+GNdPHxz0QcBht+GXZ0QXDDfXE5eMiaqbSWyMHUN7JsHnl3j5qolIjovRF6wOUQPmjEQnJuVk6FMLn7hkDH4yLvBhaMbwnpgxvCdqGj3w+4/ta3isMGAmIiIigjJtUAjoXRviHPaQHRwmqYvnvrntDH2bMUDUjpmU03SZBKAE2PaWJc2bbWz/NFQ3eoKuN5LFt0yHQHCgmxznwNCeSXpmPicrEdvun4vE2NDhZXKcI+T2zkB0tP53EydOlLm5ue19GURERNQNlda69FHjXZWUMuKQm+aodXnhtNv0jiSdjRBivZRyYlPHMcNMREREpOrqwTIQXMvcGklhssldTef8OEBEREREdJwwYCYiIiIiiiCqgFkIMU8IsUsIsVcIcWeI/bFCiPfU/WuEEIMM++5St+8SQsxtu0snIiIiIjr2mgyYhRB2AM8CmA9gJIDLhBAjLYddB6BCSjkEwBMAHlUfOxLApQBGAZgH4Dn1fEREREREnUI0GeZJAPZKKfdLKd0AFgFYYDlmAYA31NsfAJgllIryBQAWSSldUsoDAPaq5yMiIiIi6hSiCZizARwy3C9Qt4U8RkrpBVAFIDPKxxIRERERdVgdYtGfEOIGIUSuECK3pKSkvS+HiIiIiEgXTcBcCKC/4X4/dVvIY4QQMQBSAZRF+VhIKV+SUk6UUk7s0aNH9FdPRERERHSMRRMwrwMwVAiRI4RwQlnE96nlmE8BXKXevhjAUqmMEPwUwKVqF40cAEMBrG2bSyciIiIiOvaaHM8ipfQKIW4C8CUAO4BXpZTbhBB/BpArpfwUwCsA3hJC7AVQDiWohnrc+wC2A/ACuFFK6TtGPwsRERERUZsTSiK44xBClAA42E5PnwWgtJ2em44/vt7dC1/v7oWvd/fC17t7acvXe6CUssl64A4XMLcnIUSulHJie18HHR98vbsXvt7dC1/v7oWvd/fSHq93h+iSQURERETUUTFgJiIiIiKKgAGz2UvtfQF0XPH17l74encvfL27F77e3ctxf71Zw0xEREREFAEzzEREREREETBgJiIiIiKKgAEzACHEPCHELiHEXiHEne19PdR6Qoj+QohlQojtQohtQohb1O0ZQoivhRB71P+nq9uFEOJp9c/AFiHE+Pb9CaglhBB2IcRGIcT/1Ps5Qog16uv6njqtFOr00ffU7WuEEIPa87qpZYQQaUKID4QQO4UQO4QQp/I93nUJIX6j/n2+VQjxrhAiju/xrkMI8aoQ4qgQYqthW7Pfz0KIq9Tj9wghrgr1XC3R7QNmIYQdwLMA5gMYCeAyIcTI9r0qagNeALdLKUcCmALgRvV1vRPAEinlUABL1PuA8voPVf+7AcDzx/+SqQ3cAmCH4f6jAJ6QUg4BUAHgOnX7dQAq1O1PqMdR5/MUgC+klCMAjIHy2vM93gUJIbIB3AxgopTyJCiThy8F3+NdyesA5lm2Nev9LITIAHAvgMkAJgG4VwuyW6vbB8xQfqF7pZT7pZRuAIsALGjna6JWklIekVJuUG/XQPmHNBvKa/uGetgbAC5Qby8A8KZUrAaQJoToc5wvm1pBCNEPwDkA/qneFwBmAvhAPcT6emt/Dj4AMEs9njoJIUQqgNMBvAIAUkq3lLISfI93ZTEA4oUQMQASABwB3+NdhpRyOYByy+bmvp/nAvhaSlkupawA8DWCg/AWYcCsBFGHDPcL1G3URahfxY0DsAZALynlEXVXEYBe6m3+Oej8ngRwBwC/ej8TQKWU0qveN76m+uut7q9Sj6fOIwdACYDX1DKcfwohEsH3eJckpSwE8DcA+VAC5SoA68H3eFfX3PfzMXufM2CmLk0IkQTgQwC3Simrjfuk0lORfRW7ACHEuQCOSinXt/e10HETA2A8gOellOMA1CHwdS0Avse7EvVr9QVQPij1BZCINsocUufQ3u9nBsxAIYD+hvv91G3UyQkhHFCC5X9JKT9SNxdrX8Oq/z+qbuefg85tGoDzhRB5UMqqZkKpb01Tv74FzK+p/nqr+1MBlB3PC6ZWKwBQIKVco97/AEoAzfd41zQbwAEpZYmU0gPgIyjve77Hu7bmvp+P2fucATOwDsBQdaWtE8oigk/b+ZqoldRatVcA7JBS/t2w61MA2qrZqwB8Ytj+c3Xl7RQAVYavgaiDk1LeJaXsJ6UcBOU9vFRKeQWAZQAuVg+zvt7an4OL1eOZiexEpJRFAA4JIYarm2YB2A6+x7uqfABThBAJ6t/v2uvN93jX1tz385cA5ggh0tVvJeao21qNk/4ACCHOhlL/aAfwqpTywXa+JGolIcRpAL4H8CMCNa1/gFLH/D6AAQAOAviplLJc/Qv4H1C+4qsHcI2UMve4Xzi1mhDiTAC/lVKeK4QYDCXjnAFgI4CfSSldQog4AG9BqW0vB3CplHJ/e10ztYwQYiyURZ5OAPsBXAMlEcT3eBckhLgfwCVQuiBtBHA9lPpUvse7ACHEuwDOBJAFoBhKt4uP0cz3sxDiWij/3gPAg1LK19rk+hgwExERERGFx5IMIiIiIqIIGDATEREREUXAgJmIiIiIKAIGzEREREREETBgJiIiIiKKgAEzEREREVEEDJiJiIiIiCL4/5ShKTtmroN+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x20e90f7b9e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# remove noise from initial epochs\n",
    "loss_history_plot = list(filter(lambda x: x < 0.25, loss_history))\n",
    "\n",
    "plt.figure(figsize=(12, 5))\n",
    "plt.plot(loss_history_plot)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Observations\n",
    "The model does train for 100 timesteps quite quickly, but it doesnt match the speed of learning by the paper. In the paper, the loss starts to drop close to the 300th iteration and drops all the way to the low 0.0x by the 425th iteration.\n",
    "\n",
    "Here the loss starts to reduce at the 400th iteration and drops completely by the 600th iteration, much slower than the paper. Also, I had to clip off the first few iterations with extremely high losses as the graph cannot be visualized properly if they were kept."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Loading Checkpoints\n",
    "\n",
    "Tensorflow Eager makes loading of checkpoints that are fully contained in a Keras model easy. However, there is one small idiosynchacy that needs to be dealt with when loading models in TF 1.7.\n",
    "\n",
    "Models, after being build, need to be called at least once prior to the checkpoint restoring their weights. If this is not done, it throws obscure errors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checkpoint path :  checkpoints_addition/addition_janet.ckpt-1\n",
      "Final average predicted error (should be less than 0.03) :  0.013117644\n"
     ]
    }
   ],
   "source": [
    "if os.path.exists(CHECKPOINTS_DIR):\n",
    "    ckpt_path = tf.train.latest_checkpoint(CHECKPOINTS_DIR)\n",
    "    print(\"Checkpoint path : \", ckpt_path)\n",
    "\n",
    "    model = JANETModel(NUM_UNITS, num_outputs=1, num_timesteps=TIME_STEPS, output_activation='linear')\n",
    "\n",
    "    model.compile(tf.train.AdamOptimizer(), loss='mse')\n",
    "    \n",
    "    # this is where you need to call the model at least once,\n",
    "    # so that all of its variables can be properly restored\n",
    "    zeros = tf.zeros((1, TIME_STEPS, 2))\n",
    "    model(zeros)\n",
    "    \n",
    "    # restore the weights\n",
    "    tfe.Checkpoint(model=model, ).restore(ckpt_path)\n",
    "    \n",
    "    # predict one batch to ensure the weights are correctly loaded\n",
    "    generator = batch_generator()\n",
    "\n",
    "    losses = []\n",
    "    for i in range(100):\n",
    "        inputs, outputs = next(generator)\n",
    "\n",
    "        preds = model(inputs)\n",
    "        loss_val = loss(outputs, preds)\n",
    "\n",
    "        losses.append(loss_val.numpy())\n",
    "\n",
    "    print(\"Final average predicted error (should be less than 0.03) : \", np.mean(losses))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stepping off the well trodden path\n",
    "When building keras models that **only** use Keras layers, there is no issue when loading or saving weights.\n",
    "\n",
    "However, not every thing can be represented in layers, and a few models may require working directlty with TF Eager variables. In such cases, a different approach is required to save and restore these models.\n",
    "\n",
    "Lets see the same model without using Keras layers to define them. Here to keep it simple, we will use Eager variables to define the RNN cell, and a Keras Layer to define the classification / regression layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "class EagerJANETModel(tf.keras.Model):\n",
    "    \n",
    "    def __init__(self, input_dim, units, num_outputs, num_timesteps, output_activation='sigmoid', **kwargs):\n",
    "        super(EagerJANETModel, self).__init__(**kwargs)\n",
    "        \n",
    "        self.input_dim = input_dim\n",
    "        self.units = units\n",
    "        self.classes = num_outputs\n",
    "        self.num_timesteps = num_timesteps\n",
    "\n",
    "        # Initialize the forget gate with ChronoInitializer\n",
    "        # The memory gate is initialized with zeros\n",
    "        def bias_initializer(_, *args, **kwargs):\n",
    "            forget_gate = ChronoInitializer(self.num_timesteps)((self.units,), *args, **kwargs)\n",
    "            \n",
    "            return tf.keras.backend.concatenate([\n",
    "                forget_gate,\n",
    "                tf.keras.initializers.Zeros()((self.units,), *args, **kwargs),\n",
    "            ])\n",
    "        \n",
    "        # Initialize the hidden and recurrent gates\n",
    "        self.kernel = tf.get_variable('kernel', shape=[input_dim, units * 2], dtype=tf.float32,\n",
    "                                      initializer=tf.keras.initializers.glorot_uniform())\n",
    "        \n",
    "        self.recurrent_kernel = tf.get_variable('recurrent_kernel', shape=[units, units * 2], dtype=tf.float32,\n",
    "                                                initializer=tf.keras.initializers.glorot_uniform())\n",
    "        \n",
    "        self.recurrent_bias = tf.get_variable('recurrent_bias', shape=[units * 2], dtype=tf.float32,\n",
    "                                              initializer=bias_initializer)\n",
    "        \n",
    "        # Initialize the final layer (for classification or regression depending on the output activation)\n",
    "        self.output_dense = tf.keras.layers.Dense(num_outputs, activation=output_activation)\n",
    "        \n",
    "        # we need to create a dictionary of all of the weights which are not in Keras layers\n",
    "        self.additional_weights = OrderedDict()\n",
    "        self.additional_weights[self.kernel.name] = self.kernel\n",
    "        self.additional_weights[self.recurrent_kernel.name] = self.recurrent_kernel\n",
    "        self.additional_weights[self.recurrent_bias.name] = self.recurrent_bias\n",
    "        \n",
    "    \n",
    "    \"\"\"\n",
    "    This is required to override a certain issue with how the weights of a model are loaded\n",
    "    by a checkpoint.\n",
    "    \n",
    "    While we could use `model.call(inputs)` directly, the more pythonic way of doing this is\n",
    "    by using `model(inputs)`.\n",
    "    \n",
    "    However, in Eager mode, the ordinary function call does *not* forward the method to the\n",
    "    models `call` method. This bypasses that issue.\n",
    "    \"\"\"\n",
    "    def __call__(self, *args, **kwargs):\n",
    "        if not tf.executing_eagerly():\n",
    "            super(EagerJANETModel, self).__call__(*args, **kwargs)\n",
    "        return self.call(*args, **kwargs)\n",
    "\n",
    "\n",
    "    def call(self, inputs, training=None, mask=None):\n",
    "        # Initialize the hidden memory states\n",
    "        outputs = []\n",
    "        states = []\n",
    "        h_state = tf.zeros((inputs.shape[0], self.units))\n",
    "        c_state = tf.zeros((inputs.shape[0], self.units))\n",
    "        \n",
    "        # Input is in the shape [None, timesteps, input_dim]\n",
    "        for t in range(inputs.shape[1]):\n",
    "            ip = inputs[:, t, :]  # access the T'th timestep\n",
    "            \n",
    "            # Perform the forward pass of the model\n",
    "            z = tf.matmul(ip, self.kernel)\n",
    "            z += tf.matmul(h_state, self.recurrent_kernel) + self.recurrent_bias\n",
    "            \n",
    "            # Split the output into the forget and memory outputs\n",
    "            z0 = z[:, :self.units]\n",
    "            z1 = z[:, self.units: 2 * self.units]\n",
    "\n",
    "            # gate updates\n",
    "            f = tf.keras.activations.sigmoid(z0)\n",
    "            c = f * c_state + (1. - f) * tf.nn.tanh(z1)\n",
    "\n",
    "            # state updates\n",
    "            h = c\n",
    "            \n",
    "            # update our previous state\n",
    "            h_state = h\n",
    "            c_state = c\n",
    "        \n",
    "            # preserve the history of our states\n",
    "            outputs.append(h)\n",
    "            states.append([h])  # here we maintain just 1 state variable, can be more\n",
    "            \n",
    "        # save memory states to global variables so we can access them later\n",
    "        self.cell_outputs = tf.stack(outputs, axis=1)\n",
    "        self.cell_states = states\n",
    "        \n",
    "        # perform forward pass of the classifier / regressor\n",
    "        preds = self.output_dense(outputs[-1])\n",
    "\n",
    "        return preds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gradients of a custom model\n",
    "\n",
    "The gradients of such a model are a little more complicated than the earlier simple `model.trainable_variables`. Since tf.keras Models do not track the weights that are added separately (apart from keras layers, no weights or variables are managed by Keras), we need to write a custom grad function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "def grad(model, X, y):\n",
    "    with tfe.GradientTape() as tape:\n",
    "        preds = model(X)\n",
    "        loss_val = loss(y, preds)\n",
    "    \n",
    "    # this is the crucial step : use the dictionary of weights that we manage\n",
    "    # manually to get all of the weights that are not managed by keras and add\n",
    "    # them to the list of weights for which we need gradients\n",
    "    vars = model.trainable_variables + list(model.additional_weights.values())\n",
    "    grads = tape.gradient(loss_val, vars)\n",
    "    grad_vars = zip(grads, vars)\n",
    "\n",
    "    return grad_vars, loss_val"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reason for custom gradient function\n",
    "\n",
    "Keras has two very convenient methods `add_weight` and `add_variable` to add weights to a layer / model easily. However, TF Eager mode has not implemented these methods yet, and therefore we cant use them directly.\n",
    "\n",
    "Therefore, we have to take the roundabout way of mixing Keras layers with custom weights and keep a track of them using a dictionary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training and Saving the weights of a custom model\n",
    "\n",
    "Saving and restoring of custom models is exactly the same as before if you are using tf.get_variable to create variables in Eager mode. However, tfe.Variable() is NOT managed by keras, so don't use it inside a Keras model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch # :  1\n",
      "1 : 1.1621509\n",
      "2 : 1.1512643\n",
      "3 : 1.1079189\n",
      "4 : 0.96773857\n",
      "5 : 1.0441301\n",
      "6 : 0.83432114\n",
      "7 : 0.89465904\n",
      "8 : 0.8143208\n",
      "9 : 0.6209547\n",
      "10 : 0.48464486\n",
      "11 : 0.6025023\n",
      "12 : 0.49791107\n",
      "13 : 0.24446094\n",
      "14 : 0.15533014\n",
      "15 : 0.20884527\n",
      "16 : 0.26446357\n",
      "17 : 0.2826485\n",
      "18 : 0.25964248\n",
      "19 : 0.15775986\n",
      "20 : 0.14534274\n",
      "21 : 0.11215323\n",
      "22 : 0.22895297\n",
      "23 : 0.16580443\n",
      "24 : 0.15820907\n",
      "25 : 0.17097807\n",
      "26 : 0.16325451\n",
      "27 : 0.14715303\n",
      "28 : 0.16494317\n",
      "29 : 0.16491885\n",
      "30 : 0.15860003\n",
      "31 : 0.16606785\n",
      "32 : 0.14829527\n",
      "33 : 0.21525574\n",
      "34 : 0.21541195\n",
      "35 : 0.15567198\n",
      "36 : 0.13704652\n",
      "37 : 0.1394914\n",
      "38 : 0.15442346\n",
      "39 : 0.18070206\n",
      "40 : 0.17316034\n",
      "41 : 0.1432823\n",
      "42 : 0.10533683\n",
      "43 : 0.124987096\n",
      "44 : 0.20354164\n",
      "45 : 0.17832783\n",
      "46 : 0.2143028\n",
      "47 : 0.1466952\n",
      "48 : 0.20074536\n",
      "49 : 0.19540063\n",
      "50 : 0.14514315\n",
      "51 : 0.2010395\n",
      "52 : 0.14188452\n",
      "53 : 0.17658226\n",
      "54 : 0.17075127\n",
      "55 : 0.19622569\n",
      "56 : 0.20320654\n",
      "57 : 0.14674723\n",
      "58 : 0.14215888\n",
      "59 : 0.14486471\n",
      "60 : 0.14302978\n",
      "61 : 0.12705547\n",
      "62 : 0.21612404\n",
      "63 : 0.20794281\n",
      "64 : 0.15753731\n",
      "65 : 0.22443554\n",
      "66 : 0.1892033\n",
      "67 : 0.19395569\n",
      "68 : 0.1959509\n",
      "69 : 0.15914108\n",
      "70 : 0.17604938\n",
      "71 : 0.19786243\n",
      "72 : 0.13813476\n",
      "73 : 0.1283803\n",
      "74 : 0.12265948\n",
      "75 : 0.1276852\n",
      "76 : 0.14710434\n",
      "77 : 0.15749604\n",
      "78 : 0.1914433\n",
      "79 : 0.12846845\n",
      "80 : 0.15534262\n",
      "81 : 0.18991238\n",
      "82 : 0.17339572\n",
      "83 : 0.16792797\n",
      "84 : 0.17180754\n",
      "85 : 0.15204497\n",
      "86 : 0.13000168\n",
      "87 : 0.15898862\n",
      "88 : 0.14421916\n",
      "89 : 0.18815175\n",
      "90 : 0.143107\n",
      "91 : 0.15737225\n",
      "92 : 0.14719802\n",
      "93 : 0.1403911\n",
      "94 : 0.16701642\n",
      "95 : 0.17096764\n",
      "96 : 0.21338543\n",
      "97 : 0.23259921\n",
      "98 : 0.17520979\n",
      "99 : 0.15143956\n",
      "100 : 0.18171377\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  2\n",
      "101 : 0.17452398\n",
      "102 : 0.17398772\n",
      "103 : 0.13863611\n",
      "104 : 0.18441623\n",
      "105 : 0.15283062\n",
      "106 : 0.20629707\n",
      "107 : 0.18370472\n",
      "108 : 0.14907925\n",
      "109 : 0.16725613\n",
      "110 : 0.19135849\n",
      "111 : 0.13866635\n",
      "112 : 0.18205793\n",
      "113 : 0.14009476\n",
      "114 : 0.15662128\n",
      "115 : 0.1792382\n",
      "116 : 0.2281857\n",
      "117 : 0.123989984\n",
      "118 : 0.15692824\n",
      "119 : 0.13195008\n",
      "120 : 0.19298139\n",
      "121 : 0.16617489\n",
      "122 : 0.15262684\n",
      "123 : 0.18068674\n",
      "124 : 0.14233781\n",
      "125 : 0.081774645\n",
      "126 : 0.19077565\n",
      "127 : 0.15923782\n",
      "128 : 0.21413407\n",
      "129 : 0.16872802\n",
      "130 : 0.13426398\n",
      "131 : 0.18610464\n",
      "132 : 0.15890257\n",
      "133 : 0.15925054\n",
      "134 : 0.19630828\n",
      "135 : 0.17393652\n",
      "136 : 0.15345761\n",
      "137 : 0.17018421\n",
      "138 : 0.22589073\n",
      "139 : 0.2513543\n",
      "140 : 0.13429144\n",
      "141 : 0.16034317\n",
      "142 : 0.121847115\n",
      "143 : 0.15360926\n",
      "144 : 0.15041292\n",
      "145 : 0.14269821\n",
      "146 : 0.17539455\n",
      "147 : 0.16700117\n",
      "148 : 0.14943682\n",
      "149 : 0.16891328\n",
      "150 : 0.18500787\n",
      "151 : 0.17866719\n",
      "152 : 0.14627366\n",
      "153 : 0.18164702\n",
      "154 : 0.22840938\n",
      "155 : 0.22099248\n",
      "156 : 0.17002094\n",
      "157 : 0.18574873\n",
      "158 : 0.16562174\n",
      "159 : 0.19159564\n",
      "160 : 0.13033281\n",
      "161 : 0.18171264\n",
      "162 : 0.12429795\n",
      "163 : 0.14281641\n",
      "164 : 0.12133524\n",
      "165 : 0.1362178\n",
      "166 : 0.17824344\n",
      "167 : 0.17547737\n",
      "168 : 0.1661771\n",
      "169 : 0.13565828\n",
      "170 : 0.16238615\n",
      "171 : 0.14953026\n",
      "172 : 0.1436495\n",
      "173 : 0.17112984\n",
      "174 : 0.19373594\n",
      "175 : 0.10645476\n",
      "176 : 0.18847348\n",
      "177 : 0.1684391\n",
      "178 : 0.15854037\n",
      "179 : 0.20787163\n",
      "180 : 0.18528596\n",
      "181 : 0.19434765\n",
      "182 : 0.1733571\n",
      "183 : 0.14257781\n",
      "184 : 0.112505585\n",
      "185 : 0.14741862\n",
      "186 : 0.14854664\n",
      "187 : 0.14292587\n",
      "188 : 0.13989705\n",
      "189 : 0.21581559\n",
      "190 : 0.18300533\n",
      "191 : 0.13146465\n",
      "192 : 0.17924565\n",
      "193 : 0.1587025\n",
      "194 : 0.21900944\n",
      "195 : 0.16573322\n",
      "196 : 0.15893798\n",
      "197 : 0.13374256\n",
      "198 : 0.18655182\n",
      "199 : 0.15289505\n",
      "200 : 0.10996998\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  3\n",
      "201 : 0.22186726\n",
      "202 : 0.13843639\n",
      "203 : 0.12655365\n",
      "204 : 0.17936353\n",
      "205 : 0.13275121\n",
      "206 : 0.21674049\n",
      "207 : 0.1770219\n",
      "208 : 0.19964287\n",
      "209 : 0.19358288\n",
      "210 : 0.16396336\n",
      "211 : 0.16925587\n",
      "212 : 0.13012736\n",
      "213 : 0.18705796\n",
      "214 : 0.19048297\n",
      "215 : 0.16426483\n",
      "216 : 0.14111724\n",
      "217 : 0.18658395\n",
      "218 : 0.10857462\n",
      "219 : 0.14023249\n",
      "220 : 0.17722149\n",
      "221 : 0.16052969\n",
      "222 : 0.19683939\n",
      "223 : 0.17165703\n",
      "224 : 0.15519667\n",
      "225 : 0.14174989\n",
      "226 : 0.15616634\n",
      "227 : 0.1566721\n",
      "228 : 0.10799822\n",
      "229 : 0.19913803\n",
      "230 : 0.16529432\n",
      "231 : 0.15814413\n",
      "232 : 0.17272209\n",
      "233 : 0.13623579\n",
      "234 : 0.12063271\n",
      "235 : 0.1386844\n",
      "236 : 0.1740633\n",
      "237 : 0.18553345\n",
      "238 : 0.19261423\n",
      "239 : 0.1411343\n",
      "240 : 0.1138795\n",
      "241 : 0.1878658\n",
      "242 : 0.14414324\n",
      "243 : 0.16839427\n",
      "244 : 0.15615423\n",
      "245 : 0.17451763\n",
      "246 : 0.16337463\n",
      "247 : 0.13811496\n",
      "248 : 0.15342808\n",
      "249 : 0.18247299\n",
      "250 : 0.1623476\n",
      "251 : 0.13821931\n",
      "252 : 0.15814374\n",
      "253 : 0.18803948\n",
      "254 : 0.14420147\n",
      "255 : 0.24036933\n",
      "256 : 0.14577095\n",
      "257 : 0.11556837\n",
      "258 : 0.21571127\n",
      "259 : 0.20865135\n",
      "260 : 0.15414356\n",
      "261 : 0.16331671\n",
      "262 : 0.1496009\n",
      "263 : 0.16194119\n",
      "264 : 0.17524152\n",
      "265 : 0.15429325\n",
      "266 : 0.106552295\n",
      "267 : 0.12640072\n",
      "268 : 0.17571625\n",
      "269 : 0.15392298\n",
      "270 : 0.1721752\n",
      "271 : 0.19228888\n",
      "272 : 0.12010077\n",
      "273 : 0.13864908\n",
      "274 : 0.22513275\n",
      "275 : 0.17092635\n",
      "276 : 0.12420061\n",
      "277 : 0.2013131\n",
      "278 : 0.15654771\n",
      "279 : 0.19830738\n",
      "280 : 0.15181726\n",
      "281 : 0.136528\n",
      "282 : 0.2045887\n",
      "283 : 0.12790914\n",
      "284 : 0.14993207\n",
      "285 : 0.16619124\n",
      "286 : 0.15152024\n",
      "287 : 0.22683269\n",
      "288 : 0.14017643\n",
      "289 : 0.15678146\n",
      "290 : 0.13380767\n",
      "291 : 0.15817656\n",
      "292 : 0.17862025\n",
      "293 : 0.17340073\n",
      "294 : 0.17777954\n",
      "295 : 0.1471033\n",
      "296 : 0.16436699\n",
      "297 : 0.19461739\n",
      "298 : 0.16554247\n",
      "299 : 0.21333084\n",
      "300 : 0.12700416\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  4\n",
      "301 : 0.16582884\n",
      "302 : 0.20898075\n",
      "303 : 0.1750078\n",
      "304 : 0.09680592\n",
      "305 : 0.1415002\n",
      "306 : 0.16258782\n",
      "307 : 0.14485079\n",
      "308 : 0.14240536\n",
      "309 : 0.18198383\n",
      "310 : 0.12266002\n",
      "311 : 0.1777145\n",
      "312 : 0.18263713\n",
      "313 : 0.18234497\n",
      "314 : 0.19265197\n",
      "315 : 0.16840225\n",
      "316 : 0.16083483\n",
      "317 : 0.18632455\n",
      "318 : 0.19441384\n",
      "319 : 0.14598598\n",
      "320 : 0.16428588\n",
      "321 : 0.13692603\n",
      "322 : 0.17836694\n",
      "323 : 0.15277189\n",
      "324 : 0.20046842\n",
      "325 : 0.15139152\n",
      "326 : 0.14826779\n",
      "327 : 0.17124844\n",
      "328 : 0.16631278\n",
      "329 : 0.16364838\n",
      "330 : 0.14954187\n",
      "331 : 0.15680958\n",
      "332 : 0.15733175\n",
      "333 : 0.13616699\n",
      "334 : 0.15219837\n",
      "335 : 0.16273442\n",
      "336 : 0.17518112\n",
      "337 : 0.15502705\n",
      "338 : 0.1610923\n",
      "339 : 0.18044762\n",
      "340 : 0.12562336\n",
      "341 : 0.16819558\n",
      "342 : 0.11694947\n",
      "343 : 0.1576317\n",
      "344 : 0.16258074\n",
      "345 : 0.13882609\n",
      "346 : 0.13120383\n",
      "347 : 0.15734188\n",
      "348 : 0.15512449\n",
      "349 : 0.116926\n",
      "350 : 0.1577882\n",
      "351 : 0.17788224\n",
      "352 : 0.1452875\n",
      "353 : 0.18948543\n",
      "354 : 0.13448155\n",
      "355 : 0.19038075\n",
      "356 : 0.10368092\n",
      "357 : 0.16536602\n",
      "358 : 0.16239507\n",
      "359 : 0.13405462\n",
      "360 : 0.15833771\n",
      "361 : 0.1340491\n",
      "362 : 0.16511582\n",
      "363 : 0.21111917\n",
      "364 : 0.16407947\n",
      "365 : 0.16947262\n",
      "366 : 0.15729624\n",
      "367 : 0.13916157\n",
      "368 : 0.16625576\n",
      "369 : 0.1313562\n",
      "370 : 0.21306169\n",
      "371 : 0.15813984\n",
      "372 : 0.17548335\n",
      "373 : 0.18683027\n",
      "374 : 0.16156863\n",
      "375 : 0.15792435\n",
      "376 : 0.14098401\n",
      "377 : 0.117002755\n",
      "378 : 0.16242577\n",
      "379 : 0.16778725\n",
      "380 : 0.151508\n",
      "381 : 0.2292648\n",
      "382 : 0.14269243\n",
      "383 : 0.12150015\n",
      "384 : 0.12754649\n",
      "385 : 0.1621392\n",
      "386 : 0.18651927\n",
      "387 : 0.15823685\n",
      "388 : 0.15176812\n",
      "389 : 0.15039288\n",
      "390 : 0.14135158\n",
      "391 : 0.1985403\n",
      "392 : 0.11613184\n",
      "393 : 0.117393866\n",
      "394 : 0.20429797\n",
      "395 : 0.15860136\n",
      "396 : 0.0956112\n",
      "397 : 0.12968332\n",
      "398 : 0.15787499\n",
      "399 : 0.14649217\n",
      "400 : 0.12882891\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  5\n",
      "401 : 0.16940765\n",
      "402 : 0.17282967\n",
      "403 : 0.16527462\n",
      "404 : 0.18525513\n",
      "405 : 0.15709756\n",
      "406 : 0.16736652\n",
      "407 : 0.17818806\n",
      "408 : 0.097625904\n",
      "409 : 0.14921002\n",
      "410 : 0.16864675\n",
      "411 : 0.14301768\n",
      "412 : 0.15432054\n",
      "413 : 0.16860722\n",
      "414 : 0.12987046\n",
      "415 : 0.15092702\n",
      "416 : 0.12399706\n",
      "417 : 0.13464053\n",
      "418 : 0.15270242\n",
      "419 : 0.14676133\n",
      "420 : 0.15388036\n",
      "421 : 0.123324625\n",
      "422 : 0.17676713\n",
      "423 : 0.15827824\n",
      "424 : 0.13404016\n",
      "425 : 0.15690748\n",
      "426 : 0.13800992\n",
      "427 : 0.116831884\n",
      "428 : 0.14902116\n",
      "429 : 0.09627804\n",
      "430 : 0.15291607\n",
      "431 : 0.10853407\n",
      "432 : 0.16284384\n",
      "433 : 0.119379066\n",
      "434 : 0.14874731\n",
      "435 : 0.15419668\n",
      "436 : 0.10974857\n",
      "437 : 0.16883971\n",
      "438 : 0.11674813\n",
      "439 : 0.1646753\n",
      "440 : 0.14486031\n",
      "441 : 0.118678264\n",
      "442 : 0.10875624\n",
      "443 : 0.14434694\n",
      "444 : 0.12476181\n",
      "445 : 0.115751676\n",
      "446 : 0.12217863\n",
      "447 : 0.1143231\n",
      "448 : 0.117766485\n",
      "449 : 0.1638347\n",
      "450 : 0.14917491\n",
      "451 : 0.11963312\n",
      "452 : 0.14899231\n",
      "453 : 0.17762177\n",
      "454 : 0.15127851\n",
      "455 : 0.15726463\n",
      "456 : 0.109702684\n",
      "457 : 0.14533967\n",
      "458 : 0.16777302\n",
      "459 : 0.14042729\n",
      "460 : 0.15578504\n",
      "461 : 0.118589595\n",
      "462 : 0.15657058\n",
      "463 : 0.19722565\n",
      "464 : 0.1421519\n",
      "465 : 0.10931032\n",
      "466 : 0.13820136\n",
      "467 : 0.13129091\n",
      "468 : 0.16731602\n",
      "469 : 0.1720289\n",
      "470 : 0.15954135\n",
      "471 : 0.13708836\n",
      "472 : 0.12804864\n",
      "473 : 0.10557709\n",
      "474 : 0.14164223\n",
      "475 : 0.162804\n",
      "476 : 0.11445169\n",
      "477 : 0.12999205\n",
      "478 : 0.18466061\n",
      "479 : 0.13450158\n",
      "480 : 0.13166414\n",
      "481 : 0.15271242\n",
      "482 : 0.1544477\n",
      "483 : 0.08261827\n",
      "484 : 0.11228782\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "485 : 0.16018173\n",
      "486 : 0.11356828\n",
      "487 : 0.0773872\n",
      "488 : 0.096311375\n",
      "489 : 0.10748297\n",
      "490 : 0.11437002\n",
      "491 : 0.10593601\n",
      "492 : 0.14546043\n",
      "493 : 0.09018659\n",
      "494 : 0.13730581\n",
      "495 : 0.13203639\n",
      "496 : 0.058265384\n",
      "497 : 0.105687656\n",
      "498 : 0.14168152\n",
      "499 : 0.084342234\n",
      "500 : 0.06757977\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  6\n",
      "501 : 0.07952868\n",
      "502 : 0.07739203\n",
      "503 : 0.18215778\n",
      "504 : 0.070043966\n",
      "505 : 0.103158206\n",
      "506 : 0.08989197\n",
      "507 : 0.09987702\n",
      "508 : 0.0744272\n",
      "509 : 0.07997977\n",
      "510 : 0.082261056\n",
      "511 : 0.10555805\n",
      "512 : 0.10141922\n",
      "513 : 0.09325267\n",
      "514 : 0.06462287\n",
      "515 : 0.04948257\n",
      "516 : 0.076712884\n",
      "517 : 0.068328306\n",
      "518 : 0.06573434\n",
      "519 : 0.088249646\n",
      "520 : 0.05284898\n",
      "521 : 0.122354336\n",
      "522 : 0.11334232\n",
      "523 : 0.07055436\n",
      "524 : 0.14312787\n",
      "525 : 0.06019887\n",
      "526 : 0.06437059\n",
      "527 : 0.1180656\n",
      "528 : 0.10588083\n",
      "529 : 0.07076761\n",
      "530 : 0.104888335\n",
      "531 : 0.08440046\n",
      "532 : 0.06800649\n",
      "533 : 0.069036074\n",
      "534 : 0.068309695\n",
      "535 : 0.0881553\n",
      "536 : 0.07721259\n",
      "537 : 0.093113564\n",
      "538 : 0.08860032\n",
      "539 : 0.07878672\n",
      "540 : 0.06988148\n",
      "541 : 0.060777463\n",
      "542 : 0.060478255\n",
      "543 : 0.076639816\n",
      "544 : 0.08247948\n",
      "545 : 0.046171255\n",
      "546 : 0.0806703\n",
      "547 : 0.047576375\n",
      "548 : 0.06681472\n",
      "549 : 0.054443426\n",
      "550 : 0.07487729\n",
      "551 : 0.0554863\n",
      "552 : 0.078380615\n",
      "553 : 0.050209466\n",
      "554 : 0.055210892\n",
      "555 : 0.06537444\n",
      "556 : 0.04991782\n",
      "557 : 0.06462203\n",
      "558 : 0.058467485\n",
      "559 : 0.05718185\n",
      "560 : 0.045877185\n",
      "561 : 0.059709206\n",
      "562 : 0.04973532\n",
      "563 : 0.06375302\n",
      "564 : 0.050100185\n",
      "565 : 0.05931464\n",
      "566 : 0.0660767\n",
      "567 : 0.045827497\n",
      "568 : 0.07988288\n",
      "569 : 0.056597807\n",
      "570 : 0.052850157\n",
      "571 : 0.057490874\n",
      "572 : 0.057829086\n",
      "573 : 0.05794095\n",
      "574 : 0.056026462\n",
      "575 : 0.050000332\n",
      "576 : 0.06738357\n",
      "577 : 0.043959126\n",
      "578 : 0.08911047\n",
      "579 : 0.04238353\n",
      "580 : 0.074922666\n",
      "581 : 0.075716466\n",
      "582 : 0.050799213\n",
      "583 : 0.070965655\n",
      "584 : 0.0860744\n",
      "585 : 0.06125041\n",
      "586 : 0.073660895\n",
      "587 : 0.061381873\n",
      "588 : 0.057871807\n",
      "589 : 0.048660744\n",
      "590 : 0.09694215\n",
      "591 : 0.06389055\n",
      "592 : 0.055375315\n",
      "593 : 0.09445058\n",
      "594 : 0.06687753\n",
      "595 : 0.054516762\n",
      "596 : 0.06662418\n",
      "597 : 0.09523481\n",
      "598 : 0.0533651\n",
      "599 : 0.053067446\n",
      "600 : 0.05575211\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  7\n",
      "601 : 0.07736846\n",
      "602 : 0.0552001\n",
      "603 : 0.04574337\n",
      "604 : 0.050139762\n",
      "605 : 0.048997816\n",
      "606 : 0.059650965\n",
      "607 : 0.06689899\n",
      "608 : 0.09823707\n",
      "609 : 0.069393195\n",
      "610 : 0.036928907\n",
      "611 : 0.06705368\n",
      "612 : 0.055885483\n",
      "613 : 0.0430569\n",
      "614 : 0.07279368\n",
      "615 : 0.061935645\n",
      "616 : 0.07013589\n",
      "617 : 0.03643005\n",
      "618 : 0.063131124\n",
      "619 : 0.07096699\n",
      "620 : 0.042157035\n",
      "621 : 0.03867495\n",
      "622 : 0.08408603\n",
      "623 : 0.07764095\n",
      "624 : 0.05555534\n",
      "625 : 0.060472623\n",
      "626 : 0.05765079\n",
      "627 : 0.06436953\n",
      "628 : 0.05046819\n",
      "629 : 0.04349673\n",
      "630 : 0.1141779\n",
      "631 : 0.07322818\n",
      "632 : 0.028088273\n",
      "633 : 0.09079697\n",
      "634 : 0.0712476\n",
      "635 : 0.050048884\n",
      "636 : 0.050365344\n",
      "637 : 0.08910738\n",
      "638 : 0.058636155\n",
      "639 : 0.039823867\n",
      "640 : 0.05089892\n",
      "641 : 0.07196513\n",
      "642 : 0.044761654\n",
      "643 : 0.06636423\n",
      "644 : 0.07053846\n",
      "645 : 0.07046036\n",
      "646 : 0.043274842\n",
      "647 : 0.043768436\n",
      "648 : 0.043393165\n",
      "649 : 0.052902985\n",
      "650 : 0.04615542\n",
      "651 : 0.03631371\n",
      "652 : 0.06293922\n",
      "653 : 0.04975712\n",
      "654 : 0.06299912\n",
      "655 : 0.036902577\n",
      "656 : 0.0612465\n",
      "657 : 0.05225478\n",
      "658 : 0.038406122\n",
      "659 : 0.038051557\n",
      "660 : 0.035875544\n",
      "661 : 0.038937297\n",
      "662 : 0.026996393\n",
      "663 : 0.03250654\n",
      "664 : 0.04050415\n",
      "665 : 0.031349894\n",
      "666 : 0.039157156\n",
      "667 : 0.04135289\n",
      "668 : 0.02690461\n",
      "669 : 0.039351813\n",
      "670 : 0.022571877\n",
      "671 : 0.040296078\n",
      "672 : 0.03502075\n",
      "673 : 0.029684229\n",
      "674 : 0.044415906\n",
      "675 : 0.020568905\n",
      "676 : 0.029217523\n",
      "677 : 0.025536818\n",
      "678 : 0.030433211\n",
      "679 : 0.033477653\n",
      "680 : 0.052643117\n",
      "681 : 0.03186213\n",
      "682 : 0.01811907\n",
      "683 : 0.037468776\n",
      "684 : 0.03816268\n",
      "685 : 0.02157622\n",
      "686 : 0.03606137\n",
      "687 : 0.035710596\n",
      "688 : 0.02892365\n",
      "689 : 0.021856418\n",
      "690 : 0.033755653\n",
      "691 : 0.026426092\n",
      "692 : 0.026615009\n",
      "693 : 0.035342086\n",
      "694 : 0.023439674\n",
      "695 : 0.03364588\n",
      "696 : 0.02135745\n",
      "697 : 0.031256534\n",
      "698 : 0.028182926\n",
      "699 : 0.022584226\n",
      "700 : 0.019639416\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  8\n",
      "701 : 0.026487103\n",
      "702 : 0.018899292\n",
      "703 : 0.020177234\n",
      "704 : 0.025114961\n",
      "705 : 0.021267634\n",
      "706 : 0.034305304\n",
      "707 : 0.033459406\n",
      "708 : 0.024987824\n",
      "709 : 0.018467478\n",
      "710 : 0.022687536\n",
      "711 : 0.022842783\n",
      "712 : 0.023766942\n",
      "713 : 0.02352619\n",
      "714 : 0.016435266\n",
      "715 : 0.025214477\n",
      "716 : 0.02603365\n",
      "717 : 0.021770844\n",
      "718 : 0.016647704\n",
      "719 : 0.022101106\n",
      "720 : 0.022887666\n",
      "721 : 0.021594457\n",
      "722 : 0.025939822\n",
      "723 : 0.029485822\n",
      "724 : 0.0223991\n",
      "725 : 0.02487504\n",
      "726 : 0.015103959\n",
      "727 : 0.020116957\n",
      "728 : 0.024512038\n",
      "729 : 0.016456312\n",
      "730 : 0.02474121\n",
      "731 : 0.021990923\n",
      "732 : 0.022414463\n",
      "733 : 0.02273764\n",
      "734 : 0.018195996\n",
      "735 : 0.015131655\n",
      "736 : 0.018630486\n",
      "737 : 0.025349317\n",
      "738 : 0.021785831\n",
      "739 : 0.024760304\n",
      "740 : 0.020802412\n",
      "741 : 0.020911641\n",
      "742 : 0.02492898\n",
      "743 : 0.021754032\n",
      "744 : 0.02905923\n",
      "745 : 0.03147601\n",
      "746 : 0.014711869\n",
      "747 : 0.017211068\n",
      "748 : 0.031051988\n",
      "749 : 0.025224261\n",
      "750 : 0.035324324\n",
      "751 : 0.023175893\n",
      "752 : 0.02357918\n",
      "753 : 0.024354227\n",
      "754 : 0.014987478\n",
      "755 : 0.019780124\n",
      "756 : 0.03400721\n",
      "757 : 0.018811047\n",
      "758 : 0.024055524\n",
      "759 : 0.021259477\n",
      "760 : 0.022887683\n",
      "761 : 0.026393939\n",
      "762 : 0.028880611\n",
      "763 : 0.019565068\n",
      "764 : 0.029858215\n",
      "765 : 0.015567216\n",
      "766 : 0.027770605\n",
      "767 : 0.021531155\n",
      "768 : 0.015297752\n",
      "769 : 0.015660807\n",
      "770 : 0.024965491\n",
      "771 : 0.012396178\n",
      "772 : 0.022141272\n",
      "773 : 0.027769739\n",
      "774 : 0.038974945\n",
      "775 : 0.02044248\n",
      "776 : 0.021008821\n",
      "777 : 0.014710438\n",
      "778 : 0.015092858\n",
      "779 : 0.015304186\n",
      "780 : 0.015131209\n",
      "781 : 0.018996174\n",
      "782 : 0.02421886\n",
      "783 : 0.016571632\n",
      "784 : 0.01445453\n",
      "785 : 0.018240837\n",
      "786 : 0.012433418\n",
      "787 : 0.009910008\n",
      "788 : 0.01312879\n",
      "789 : 0.012092981\n",
      "790 : 0.01402854\n",
      "791 : 0.021272555\n",
      "792 : 0.01436323\n",
      "793 : 0.013599489\n",
      "794 : 0.014096321\n",
      "795 : 0.021129727\n",
      "796 : 0.010184152\n",
      "797 : 0.014366507\n",
      "798 : 0.011746323\n",
      "799 : 0.014906851\n",
      "800 : 0.017781395\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  9\n",
      "801 : 0.030030632\n",
      "802 : 0.014365942\n",
      "803 : 0.012845557\n",
      "804 : 0.021550978\n",
      "805 : 0.01814298\n",
      "806 : 0.02284931\n",
      "807 : 0.0237127\n",
      "808 : 0.019042442\n",
      "809 : 0.019559529\n",
      "810 : 0.02465231\n",
      "811 : 0.014367839\n",
      "812 : 0.025342941\n",
      "813 : 0.02344511\n",
      "814 : 0.020890992\n",
      "815 : 0.01936003\n",
      "816 : 0.01787552\n",
      "817 : 0.015061614\n",
      "818 : 0.0149472775\n",
      "819 : 0.016823359\n",
      "820 : 0.014147806\n",
      "821 : 0.018946111\n",
      "822 : 0.015606833\n",
      "823 : 0.017901631\n",
      "824 : 0.014592238\n",
      "825 : 0.01583731\n",
      "826 : 0.019414708\n",
      "827 : 0.01908092\n",
      "828 : 0.012014141\n",
      "829 : 0.014311067\n",
      "830 : 0.010503138\n",
      "831 : 0.010355268\n",
      "832 : 0.01041521\n",
      "833 : 0.018829599\n",
      "834 : 0.020979555\n",
      "835 : 0.013075921\n",
      "836 : 0.014609251\n",
      "837 : 0.0144327665\n",
      "838 : 0.01754508\n",
      "839 : 0.011855249\n",
      "840 : 0.01767914\n",
      "841 : 0.0109841265\n",
      "842 : 0.007160005\n",
      "843 : 0.014209363\n",
      "844 : 0.006966999\n",
      "845 : 0.008116461\n",
      "846 : 0.012841169\n",
      "847 : 0.011620861\n",
      "848 : 0.014408579\n",
      "849 : 0.01348294\n",
      "850 : 0.01201184\n",
      "851 : 0.009347877\n",
      "852 : 0.015454381\n",
      "853 : 0.010721075\n",
      "854 : 0.013120465\n",
      "855 : 0.020850709\n",
      "856 : 0.014667712\n",
      "857 : 0.017732613\n",
      "858 : 0.014344544\n",
      "859 : 0.015992176\n",
      "860 : 0.01417166\n",
      "861 : 0.015028724\n",
      "862 : 0.011597834\n",
      "863 : 0.013647451\n",
      "864 : 0.01674522\n",
      "865 : 0.014058041\n",
      "866 : 0.014591095\n",
      "867 : 0.011233372\n",
      "868 : 0.011675154\n",
      "869 : 0.015009872\n",
      "870 : 0.011655096\n",
      "871 : 0.0137200495\n",
      "872 : 0.017229075\n",
      "873 : 0.012109876\n",
      "874 : 0.014179173\n",
      "875 : 0.0133238025\n",
      "876 : 0.018385964\n",
      "877 : 0.017418642\n",
      "878 : 0.022792019\n",
      "879 : 0.012964742\n",
      "880 : 0.0123944925\n",
      "881 : 0.0120278625\n",
      "882 : 0.015933165\n",
      "883 : 0.014295025\n",
      "884 : 0.014082588\n",
      "885 : 0.011272328\n",
      "886 : 0.010991094\n",
      "887 : 0.011151891\n",
      "888 : 0.01277179\n",
      "889 : 0.01260668\n",
      "890 : 0.012407446\n",
      "891 : 0.012278632\n",
      "892 : 0.008859095\n",
      "893 : 0.01429797\n",
      "894 : 0.014462714\n",
      "895 : 0.009702824\n",
      "896 : 0.011556063\n",
      "897 : 0.013805108\n",
      "898 : 0.02522901\n",
      "899 : 0.00902943\n",
      "900 : 0.0122801205\n",
      "\n",
      "Saving weights\n",
      "\n",
      "Epoch # :  10\n",
      "901 : 0.00901419\n",
      "902 : 0.012053919\n",
      "903 : 0.013461079\n",
      "904 : 0.015934672\n",
      "905 : 0.0075587807\n",
      "906 : 0.01101004\n",
      "907 : 0.010823178\n",
      "908 : 0.011288586\n",
      "909 : 0.010599801\n",
      "910 : 0.011733476\n",
      "911 : 0.013157886\n",
      "912 : 0.017425172\n",
      "913 : 0.008150251\n",
      "914 : 0.014784017\n",
      "915 : 0.015230116\n",
      "916 : 0.012863653\n",
      "917 : 0.011939194\n",
      "918 : 0.014136527\n",
      "919 : 0.014618843\n",
      "920 : 0.0092842\n",
      "921 : 0.013114586\n",
      "922 : 0.015383261\n",
      "923 : 0.0070189848\n",
      "924 : 0.01075149\n",
      "925 : 0.012399299\n",
      "926 : 0.017086044\n",
      "927 : 0.021438591\n",
      "928 : 0.019750623\n",
      "929 : 0.014763551\n",
      "930 : 0.034406144\n",
      "931 : 0.01718176\n",
      "932 : 0.018059038\n",
      "933 : 0.010594039\n",
      "934 : 0.011660888\n",
      "935 : 0.012951542\n",
      "936 : 0.012108057\n",
      "937 : 0.014253226\n",
      "938 : 0.014137793\n",
      "939 : 0.014750669\n",
      "940 : 0.010612033\n",
      "941 : 0.013792441\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "942 : 0.015041292\n",
      "943 : 0.0144982245\n",
      "944 : 0.0101460675\n",
      "945 : 0.011825992\n",
      "946 : 0.019285101\n",
      "947 : 0.009109263\n",
      "948 : 0.0111296605\n",
      "949 : 0.014955539\n",
      "950 : 0.007480563\n",
      "951 : 0.013984419\n",
      "952 : 0.008782578\n",
      "953 : 0.012421405\n",
      "954 : 0.007365109\n",
      "955 : 0.0088006975\n",
      "956 : 0.009729772\n",
      "957 : 0.0069292225\n",
      "958 : 0.010098675\n",
      "959 : 0.010997141\n",
      "960 : 0.012169895\n",
      "961 : 0.011444533\n",
      "962 : 0.01399242\n",
      "963 : 0.0068554753\n",
      "964 : 0.00894353\n",
      "965 : 0.012242434\n",
      "966 : 0.01021545\n",
      "967 : 0.012649863\n",
      "968 : 0.007302314\n",
      "969 : 0.010416012\n",
      "970 : 0.006149518\n",
      "971 : 0.010594614\n",
      "972 : 0.015516937\n",
      "973 : 0.00739812\n",
      "974 : 0.010529751\n",
      "975 : 0.009240895\n",
      "976 : 0.013352713\n",
      "977 : 0.008962644\n",
      "978 : 0.010766876\n",
      "979 : 0.009021389\n",
      "980 : 0.011382189\n",
      "981 : 0.011743964\n",
      "982 : 0.0120130135\n",
      "983 : 0.015135792\n",
      "984 : 0.0071727526\n",
      "985 : 0.0064567816\n",
      "986 : 0.01175874\n",
      "987 : 0.0115865255\n",
      "988 : 0.009993628\n",
      "989 : 0.01362257\n",
      "990 : 0.011902858\n",
      "991 : 0.007314901\n",
      "992 : 0.015340851\n",
      "993 : 0.015841175\n",
      "994 : 0.007128427\n",
      "995 : 0.007353345\n",
      "996 : 0.011006472\n",
      "997 : 0.008887305\n",
      "998 : 0.009906682\n",
      "999 : 0.016090322\n",
      "1000 : 0.009503086\n",
      "\n",
      "Saving weights\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with tf.device('/gpu:0'):\n",
    "    input_dim = 2\n",
    "    model = EagerJANETModel(input_dim, NUM_UNITS, num_outputs=1, num_timesteps=TIME_STEPS, output_activation='linear')\n",
    "\n",
    "    optimizer = tf.train.AdamOptimizer(learning_rate=LEARNING_RATE)\n",
    "    global_step = tf.train.get_or_create_global_step()\n",
    "\n",
    "    # try using different optimizers and different optimizer configs\n",
    "    model.compile(loss='mse', optimizer=optimizer)\n",
    "\n",
    "    best_loss = 100.\n",
    "    generator = batch_generator()\n",
    "\n",
    "    loss_history = []\n",
    "    update_counter = 1\n",
    "\n",
    "    for epoch in range(NUM_EPOCHS):\n",
    "        print(\"Epoch # : \", epoch + 1)\n",
    "\n",
    "        for step in range(STEPS_PER_EPOCH):\n",
    "            # get batch dataset\n",
    "            inputs, targets = next(generator)\n",
    "            \n",
    "            # get gradients and loss at this iteration (uses the new grad method)\n",
    "            gradients, loss_val = grad(model, inputs, targets)\n",
    "            \n",
    "            # apply gradients\n",
    "            optimizer.apply_gradients(gradients, tf.train.get_or_create_global_step())\n",
    "\n",
    "            loss_history.append(loss_val.numpy())\n",
    "            print(update_counter, \":\", loss_history[-1])\n",
    "\n",
    "            update_counter += 1\n",
    "            \n",
    "        print()\n",
    "        \n",
    "        # remove old checkpoint that we no longer need\n",
    "        if os.path.exists(CHECKPOINTS_DIR):\n",
    "            shutil.rmtree(CHECKPOINTS_DIR)\n",
    "        \n",
    "        # save the checkpoint weights\n",
    "        # Note how we add the model's additional weights as name-variable pairs\n",
    "        checkpoint = tfe.Checkpoint(model=model).save(CHECKPOINT_PATH)\n",
    "        \n",
    "        # Optional : Save the weight matrices in Keras format as well\n",
    "        #model.save_weights('addition_model.h5', overwrite=True)\n",
    "        \n",
    "        print(\"Saving weights\")\n",
    "        print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x20e8fef5320>]"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAEyCAYAAADuoYbuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXm8HFWZ93+nqvtuucnNnpAECGGRfZEQBBTZF0EF0WFRB2Z0XEZnXMbXN4yiAiqIjiiC84LCuI4oruyyIwmLSYCQEAiEJGQje3KTu3Z31Xn/qDpVp06dc6q6b981z/fzgdtdXVXnVHV16nee+p3nYZxzEARBEARBEAShxxnsDhAEQRAEQRDEUIYEM0EQBEEQBEFYIMFMEARBEARBEBZIMBMEQRAEQRCEBRLMBEEQBEEQBGGBBDNBEARBEARBWCDBTBAEQRAEQRAWSDATBEEQBEEQhAUSzARBEARBEARhoTDYHVCZOHEinzlz5mB3gyAIgiAIghjhLFq0aCvnfFLWekNOMM+cORMLFy4c7G4QBEEQBEEQIxzG2Jt51iNLBkEQBEEQBEFYyCWYGWPnMMaWM8ZWMMbmaj7/ImNsGWPsJcbYo4yxfaXPPMbYi+F/d9ez8wRBEARBEATR32RaMhhjLoBbAJwJYB2ABYyxuznny6TVXgAwm3PexRj7NIAbAFwcftbNOT+6zv0mCIIgCIIgiAEhT4R5DoAVnPOVnPMSgDsBvF9egXP+OOe8K3z7LIAZ9e0mQRAEQRAEQQwOeQTzdABrpffrwmUmPgbgAel9E2NsIWPsWcbYBTX0kSAIgiAIgiAGjbpmyWCMfQTAbADvlhbvyzlfzxibBeAxxtgSzvkbynafAPAJANhnn33q2SWCIAiCIAiC6BN5IszrAewtvZ8RLkvAGDsDwFcAvI9z3iuWc87Xh39XAngCwDHqtpzz2zjnsznnsydNykyFRxAEQRAEQRADRh7BvADAgYyx/RhjDQAuAZDIdsEYOwbArQjE8mZp+TjGWGP4eiKAkwDIkwUJgiAIgiAIYkiTacngnFcYY58F8FcALoA7OOcvM8auAbCQc343gO8CaAVwF2MMANZwzt8H4BAAtzLGfATi/HoluwZBEARBEARBDGkY53yw+5Bg9uzZfDAq/d29eAMmtjbgxP0nDnjbBEEQBEEQxMDDGFvEOZ+dtR5V+gv53l+X466F6wa7GwRBEARBEMQQgwRziMMAf4hF2wmCIAiCIIjBhwRzCGMMPullgiAIgiAIQoEEcwhjwFDzcxPDE9/n6C55g90NgiAIgiDqBAnmEIcxkF4m6sHV97yMQ772ICqeP9hdIQiCIAiiDpBgDmEgDzNRH+5cEFSSr5DHhyAIgiBGBCSYQyjCTBAEQRAEQeggwRzCKEsGUWfociIIgiCIkQEJ5hCHsmQQdYaDLiiCIAiCGAmQYA6hLBlEvaEBGEEQBEGMDEgwhziMUTyQqCs0ACMIgiCIkQEJ5pCRUumPc05CbYhAEWaCIAiCGBmQYBaMEA/zOT94Ckde/dBgd4MAQI8sCIIgCGJkUBjsDgwVnBHiYV6+afdgd4EIGQlPLAiCIAiCoAhzBOVhJuoNCWaCIAiCGBmQYA5RK/0t27AL3kjwaBCDBl091bF2exd+8reVg90NgiAIgkhBgjlEjjC/vKEd77npKfzosdcHt1MDwOOvbsaabV113y/nHNfeuwxL17fXfd/DBYowV8dHb38O37r/FWzt6B3srhAEQRBEAhLMIXKlv43tPQCAl9aNfLH3Tz9bgNO//0Td99td9nD7vFW48Mfz677vYUOd9LLv8z3iaUdHbwUAVUgkCIIghh4kmEPkCDNjwd+RMAkwD2Wv+uPsKXvWz4W+q2XfI4V6adwLfjwf+//n/fXZGUEQBEEQVUOCOUSOMDMEinkPCOrVxL0vbcDBVz2I5RvNGTn2hIhoFnktGW+1d2PtdrMtZk940gFQZJkgCIIYupBgDpEr/UUR5gFod8Xm3fjVs28OQEv145FlmwAAy94yCzmfBHPu6+fqu5fhy79/qV/7MpwQvz+CIAiCGCpQHuaQRIQ5vGMPhCXjPT+ch5Ln4yPv2Lff26oXec5KhQRz7kFDZ6kS+XcJgiAIghh6UIQ5hA1Spb+S5wOojzivhPvKS61RYLGZYwkFUoaI/BYDzsnCIkOXDkEQBDHUIMEc4jBEd2onmvQ3cO3XIyJbqlIwezUeYB5xLwvAgRSDJ173KL7/0PIBa88Gz2nK8DmnAQbiJxd7ymRbgiAIYvhAgjnEkSLMYtJfXsFTD+ohKnvLVQrmfhSy8r7bu8v91o7KhvYe3PTYigFrz0be0+vzPSNtXF7oTBAEQRBDDRLMIU7CwxwsG8hAV7nK6LCO3kqVloxaI8zhX5slQxaApSr7NVLIGyn1ee3R/pEIRdsJgiCIoQYJ5gg5whwwkDfuukSYK/bcyPVqkysDC+2+pXPXX2Lw5Q3tmDn3Przy1q5+2X9fyX16ycOcgE4FQRAEMdQgwRziMCkiOIARZiE66+FhrjrC3MfAL4Nl0p90PP2VYu7BpRsBAA+Hae6GGvkjzGTJAOLzRSkJCYIgiKEGCeaQRKW/yMM8MO0CQKUOFfGq9jDXPOkv+Js3wtzfkXrT7u9ZvAFPr9jar23byHvUPuckEgmCIAhiCEOCOYRpPMwmxbNySwe27O7tU3t/fXkjZs69L4osVvoa7sXAWTLiiohm5AFAPaOnm3f34Jv3LkPF863tA8C//eYFXPbT5+rWdrUsWdceRcFtkIc5CXmYCYIgiKEGCeaQRKW/8K8uS8bTb2zFaf/1JD71q0XGfeWZ5Pa/z61JvK+Ph3lgsmTkiTD7/RRh/tqfX8ZP563C317fEvdniOZV+I+7FluvEwHnHHWY8zlioGA7QRAEMdQgwRyir/SXXk9Elpes15eFfvzVzTjoqw9gyTpz2WgAGNtSTLwv18OSUW2EOUPI/vLZN3HstQ+nlsdb5cuSUU8xKLze9ThfA0WW3YKjb4OKagvWDFXEGaAIM0EQBDHUIMEcwmQPs5j0p1lPrNPg6k/dY69uBgA8v2aHtb225qRgrnce5jwTzrKE3FV/XoptnaXUvqqNMNfTkhEXlRk+ompnRh7qvkz6W/TmDhzwlQcG1atdDW+1d+O9P5pntTQNo6+WIAiC2EMgwRwi52EWN2xdpCteR39XF587GQbbMU1JwVwPD7OcaSOP/sor0tLrVedhrmfE0A1PrOfDrtiHENs77X533689M8QzbwRCed4wEcw/e3o1lqxvx12L1hrXGU6DIYIgCGLPgARziJwlgyvCWSZLZIqPnQzFrEaY65Elo9py1HknmqnribfWwiX9FWEWglna/1DXV1s7StbPfc77NWPJYPKnF9bhP/+0JHofZaCxHC55mAmCIIihBgnmENnDLG7YNkuGCR5FmO0KpqGQPPX1yMPs+dVFdfNGNdXgdzQ50mbJkLapZwYIJ/KXDx9VtS1DMAN9mIAZ/s263gaLL/x2cWKCq62btic7BEEQBDGYkGAOYWBYt6Mbi97cEWddsFgyTORJuRbsOrmfekRhqxXMeYWsahfJI1YTeZj7wcPsc555jgebohv0MNOSwXnNIlFst6OrhFuffGPIDySiDDSWfpJgJgiCIIYaJJhDhBC76L+fjsJ2utu2Lfosf54V8VM1ZD0yHVRrgxDrZPmtTRFmm67xpI1qGQxs3tWDZ1duSy13w/Mq92moyithu9nemTXpr+8p/n717Bpc98CrmZNNB5toQq3lcEkvEwRBEEMNEswhssCNRLHOw5wzwpzlYVajaPWwZNQ66S9L3KvHnDVoCPZt3j4P7715Hi657dnUcpHyz+M89Xh/6EVXxQRF+2AoiDDX1n91m1JlqJ2DJLZy6oIh9zUSBEEQezwkmENk8SUsGbqCGFmiJp4QV137dbFkSCpVtkH0lD1cetuzWL5xt7bNLPurKYOH7dF5wh5SQ/B80y69jUFk89PZPIaK0BLnUwjlrAFD7N2tvq10/pIhchIysPWSLBkEQRDEUIMEcwiTVGOcLSO9nhBqpnt6qry2AVUUlOtiyZBfx2+eX7MDz6zchm/c/bK2DyzLPqJaMjJS68n7VvtSLapVRaSV04nLoSK0RDfEoCFLCIvzWMugKXXIQ+MUGLFZMsR5GCrfI0EQBEEISDCHyBFhNR+zTFb2t1o9zPWZ9CdFmKXOF5zga1YjxUKLZgXD05P+kn/129Rn0l9JEcyyJUPt0FCTWeKws44/Wk86pjO//yTOu+mpHG0MtaO2Yys7j+gzPau2duIrf1pS1zSFBEEQBJEHEswhssAVt2OdGLFFVX/61Ercs3hDan9iu1uffAObdvWE75Pb1ietXPxa1rgiKqu2kdfDnJ70lx05lUViXwROWfHkxpP+eMoPO9S0ozju7Nzd6fVe39yBlzfsymxjmAWYoxCzfdKf/sN//fXz+PVza/DKW9nnhSAIgiDqSS7BzBg7hzG2nDG2gjE2V/P5FxljyxhjLzHGHmWM7St9djlj7PXwv8vr2fl6ktCMNiFoudN/875XoteqCH1pXTuue+BVzP3DS9r91KPSXyIzRSLCHItMmSz7SOTFNRQuyeth7oslo9fzEu9jS0a8T9sAZzARx53Xw5y9Xvpz9ZiH2CmIENdeHGG2rGv4cKinESQIgiBGLpmCmTHmArgFwLkADgVwKWPsUGW1FwDM5pwfCeD3AG4Itx0P4OsAjgcwB8DXGWPj6tf9+pGMMJstGXGGCLsyEbt7buU2+D7H1o7ecLtwe1Uw16HSn8kGUWuEmUXrGSwZlr549bJkVFRLRrz/IVqrI9FHIPv4xbXwh0Xr8PdV243raXfD1bdDUzF76uDM5n83nK/QWTRkBwUEQRDEyKWQY505AFZwzlcCAGPsTgDvB7BMrMA5f1xa/1kAHwlfnw3gYc759nDbhwGcA+A3fe96fZHFl9CHOvGR117wr79+HifMmoBnVm7D3HMPxsTWRgDA+JaGoI1+8DDLQkMXbVXbyJslQ52PGA8oLBHmPk76c1hwjlTBHFkyNKWxh4qQqnbSn/j86nuCn9Tq688zrMfhKnHWvkaYf/nMaoAxfPQd+2at2ic8n6PoSqWxLeuaPhPbDrUnCQRBEMTIJ48lYzqAtdL7deEyEx8D8EA12zLGPsEYW8gYW7hly5YcXao/sh82jgKn17OJRFV4PhMW3li5pQM7OoPyyGNDwdwfHmZ5H57mda0eZlVoV23JqOHYRFS8rETenRqyZFz44/lVtb1ldy+Wrm+vahsbeXN3Z6H31CttVXmur/rLy7jqz0ur2qYW1MGZ7ZBN50NMzCW5TBAEQQw0dZ30xxj7CIDZAL5bzXac89s457M557MnTZpUzy7lRs6SEaVN06xnK2pSsCRf3tEVCOZxLcVwP6olo76V/mTdFHlpVcGc6WEWhTcUwSz+5hQ9tUQEhWBWI8yOpk8sQ0i9sGZnVbaQs258Euf/aF7+zmaQN0uG4Inlm7XraQdwyvt6DLz6g+hay7GuepzbO0v4xC8WYndPBQBFmAmCIIiBJ49gXg9gb+n9jHBZAsbYGQC+AuB9nPPearYdCsiV+eK0ctVZMlyrYA7KIxcLjnbfQugcdfVDVUdEo755epEqBJvadz8jwhx5mNXzEEWYzX2RPdm1jAVEKrxSatJf2LbUeGzJMHdIDFjyIL6repEd9U1+fsX/LNCvlSOqXo+BV3+gDhq0RYHEusox3fa3lXho2Sas3NoZrDcMBPPa7V14dSNl8yAIghgp5BHMCwAcyBjbjzHWAOASAHfLKzDGjgFwKwKxLIfH/grgLMbYuHCy31nhsiGHrBmF5rBZMvQRZvPp3BkKNiGe1M1Lng/f52jvLuOFNTtz91vGZMkQL1UxJYSwSeerFeuWrm/H65t2S2nlLBO3NIK9GkSfeg0RZl0k1dbM5t36yoEAcPBVD6SKuvQFdfyRbcnIt988lowhG2GuwpKRHp/1zac9GLzrhsdxzg+y82gTBEEQw4NMwcw5rwD4LAKh+wqA33HOX2aMXcMYe1+42ncBtAK4izH2ImPs7nDb7QCuRSC6FwC4RkwAHGrIHmZbijddkQmBLcK8qyeIWgrxqG5/w4PL8a4bHs/dX33f9BHmLA9z1oNyobOv+stSfOfBV2PBYhEufU0rVwhDyaqHWeyq4vtSto7ohRGbYO4p+/jZ06ur7mNesg6/bx5m9UnF0Iwwx4I5e9Jf6ji5+nkdO0YQBEEQOciTJQOc8/sB3K8s+5r0+gzLtncAuKPWDg4UstYVdgK9hzm0N2jEi8nDzMCifcaCO73e+p3d+TusIZlWLl6uK4whL7fo/MR23SUPnQUvV95jTyPYq0EMPu57aQNOPnBiJLREmxWfw3WS0X5bf0TBmMEgs3BJzvOjrvXqxl34+TNvJpZlpSfc2tGLhau345zD98rVZr1Q/fLVRJhVyMNMEARBDDRU6S9E9vF6higwEN/4OU9H92wR5qjqG8+2M9SKb4jqmiPMwV+zhzk5wc7zOUqeHx23TeeZ/NQ27nvpLfzo0dcBxIOP3y1chydeizOnxPYSHvUj+k4s+95iiTCbqDV/dCpzRc7CJZn7VYLHl972bGqdLEvGh3/yHD71q+fRU/as69Wb9MRRnd8p+JNKlaeuRnqZIAiCGGBIMIfImrEchmf1Hub4tapNiq75dJZDdRqJsH646ScizJo8yKoAFN5khwE/f3o1Zs69L5mVQqn05/kcpYqfq3hLLRHmB5a+hbsWrQv7FH8hIjsCEA9SKp6fKgwiD2DUwcyu7uon8pUVe8Md81bhjS0dVe/HJLx395SxdntX7ktBFZKqXQUwC+bb563C4rU7sXzTbgD1yftdDeJURtanajzMKQFNipkgCIIYWEgwhzBNhFl3Y7YVB7FFmEtedoR5wqiG1LL27jKeDfM5Z2GqrudLEeatHb1Yu70rXD/4nDGGH4aRXeG1Tu7Xj7aXBbVNcyWi3T7HPYs34PVQrJkoe350zgtufC6bi67UZnwsqr1F7o96enXiMljPIvqlHXb2VnDNvctw+R1/tx4DoJn0ZzhRP37iDVz6k2dr9jDr+q7LksE5x7X3LsP7b4mzr/SlXHktCG91VgpAILsYC0WYCYIgiIGGBHOI1sOsuTHbIqe2PMxRhNliZ9DpgH/5+UJcctuz6CpVNJ8qfcsoXOL5HHO+9Ug0uVD2lYqIrhC6f35hfSSOhQaLLBmivzk9zD7n+LffvIAzb/ybtf9lj0eRSFdSnbJ4li0ZqjdbHuCko7HBjl9atxPrdnTF/bSofllkbw8Lz9Ri0zBtsr2jhF3d5dyCOc9auuNRM40AfStXbmJ3TxmrwtRvqfaUPMy2a0ftmrrmQEfHCYIgCIIEc4icJSMSYJmWjPwRZtWSodu3zle6dENQce53C9biyj++ZNy/3O+gb+l+ln0/uVxK9SXcJELofv63L6b2W/F9lCqxh9mm85Ip7qzdjihL/mj5XCaj2tKxqF5zi10myKrB8b6b5+PcHz4lLc8XYd7aEXigRaXGajAJ4t6KB877llZORRdJly0tgnqLzje2dOCE6x7Dqd97Qvt5/DTDvA/TQKxaTzhBEARB1BsSzCGOxsOsI5m6LfmZVTAr0Vqd+OkqpQWzWO8b9yzDb/6+NvW5TFIwp/uZEh5S4RKRQ1qXZUGOUAuRZzqGqE2fRxH3vBHUUsVHb8XHknXtiXMpR0jFrjyfpyoYJiwZSlyy7HFs7QiixLKAtAlH2d6wPSptXsx1LDKmNnorfnBucuo/9TQyjfr0NNfubq3Npr6i8/T/ehIdveanIKby6jqyetYf0XGCIAiCsEGCOUSu9OdFlgyNh9lmyXAN2SZY7GHWVRE8cHIrPnDMdO22Nm3wuTtfwMlS7mZT4RKTOJIfk4uaK7qIqxCmFZ+jtxL7jK1ZMvx4EmQ5Z4i55PnY1lnCe2+ehw1Sij1dhLni8VSEWRbJqYIeno/lGwMP9eimOJuiLcIsf7YtFNujmwr4+l+WYrMhTd03712GnnLyeM0R5iDi35c8zCq2CHNTMf65D3SUVn1qY2s9nSUj+Z4sGQRBEMRAQ4I5RA7WVTQRS4Gs/dJp5XJkydB4mA+f3oapbU3a7Wxez7+8uAFrtst+3LSwVF/LyBFm4RnWidto0qDHQ0tGdt98zlEMBxCqgDQht71LigJrLRlebC/xfI6127uwYWdPar1o3z6PMkTMmjgKL67diV09ZWspaTnavrUzsGQsXtuOnz/zJr7656XabX46b1VqmTnC7MHnPLdgzrOari0hmIvS9TnQolNN/fdWezdueXyFYVAa9G/ZhqC0tOnJCEEQBEEMFLkKl+wJ6Cr96bJkcFuEOYeHWTdBjTHztqo2KHu+MX2dx4N9cW6PhAvEwECe9KcTzDc99joeXrYp8DB7smDW7jbct4+C64Ax4JfPrDavKFGu6HdYqsRWFVkkx5YMpKokpjzMnh+llmtucHHBLfMxZ+Z43HzZMdZjEGwPI8zFQjI3dR6MEeayn+lh5jkGPjI6O5GwZPRK3+1AFwRUc5vfv2Qj7l+yEWcfNhUHTG4FIJed5/jBI6/hR4+twIOff1d6X+RhJgiCIAYYijCHOJoIs+6+bPMw58uSodmWx6Wgbe0B+omBAs+PxbTQRpxzrTcaiCPHgYdZCOb0Qa/c0on7lrwFz+fgPBaSOgH3wpod+Pb9r8Dzg/1yDnQa2lcxWTeSHuYwwuzzhMBKHZsmZ7FYrzuMeL+4bmduS8b2LpElI+yH9UiUvhjEaW/Fr6qoScrDrFnf01kyNN7iwbZkCHS/GZ9zvLh2JwBg0650wRl1sPLGlg7MnHsfnl+zo069HRh6yh6ey5kykiAIghhcSDCHJCr9WUpjy5pOFWUmDzPnsRDVFtmAecKgKjBs9gbP52gIBbPo26+fW2O0DwjR5DBmjTALhIAUAlZ3fi77yXO47W8rsbunDEsdFy0lpe1x4QS7hCUjfCkXLtGJP7UqXtmLxWlvOOhgyJr0F3/WoWSaqKZSo0mc9la87LLZVUaYdQMAYclwNbnGBwq1yIxA1wvO4+ueIX3cat+fXB5Ugrz7xQ116etAcfU9y3Dxbc9ixebqi+EQBEEQAwsJ5hCdh1mfVs5sdTB5mOXH5HJpbXmftui0jD3CzNFQCAWzLx59v2VcXxYvrpMtmEWfRR90Aq4xnFi2aVdPlHkjL2rb7zxwUjhhUjPpTypcohN/qp2m4vFosNMd9t9hTCswxVchWzJEBghf8/3p+i5jy5KRRTI9YObqiT4LFqzaDiB5jQ94pT+Ndx8weOZ5/P0Ji5GM2ndfGvgNJ17dGHi027tLg9wTgiAIIgsSzCFyiq5YdNgf9acizAbRK9scUrmDYY8wq/RWzIK54scT7VZv68LWjl6jiLjhwVelyY1cEszZQkpEuXUDCpGB4q32Huj0slw0REVt22VAg+toqwt6fjpLhowpD3PQ/zDCzPSV8cS5kMV0ZyiYdSn1XnlrFw78ygN46OWN2uMyTY7szTEZMnGd5PBkyFHxn81fhceXb8aDYb/kwVZ/Cmbd8aoeZkFJV1SFcynCrLds6N7n/AmNGK69dxkeXGoeEBMEQRD1gwRziKOJvmmzZMgRPz8oJf1We5ACzXS/rmgipImcwdzuf5axWTJ8n0ce5u88+CqO+9YjxkIRP37iDbwWZo2QBfvOrhI2GVKmCYRo1wmjMU2BjWLTrp6EBUDwzu88btxvWRFPruOgseBoPcyen670J6PzMIv1ukuxJUMXYRaDp4QlQ4kwyywO/baPvLJJe1w2S0YW8qb5IszBSts6evGNe5bhn/5ngXb7QJT2j2jWZVGJPczq9yJ9t9H28TqMaZ4WpCLMwV9nCCrm/jrHAHD7vFX41K+e77f9EwRBEDEkmEPkSGwlZx5mn3Ocf9M8nHDdY6nPZOQomtBgqSqBOQ2/3RZLRkWyZACBcNEVt5DXF30R4vZzd76I47/9qLUP4nh0Ak4I5rLHjRMZjfv1VMEMNBTchGCOBxw8Opc6N4R6fit+7GHuCffnMKYV2+Jc6CwZYm15/0KomVwZpuX5LBmWCLO2Lf0g4t0HTUqtlzfKfPu8VZg5977c+bTFXnW5wNOWDI2dhicj+ephqz7oKJ/40NPL1kwyLMc6BEEQxNCA0sqFaD3MmvUSUTPOE3mQTfoj4cGNyu5J+wRQrJeHWRGptr16Ul+qic75kphRaWlwo9d5o+YCVZC5DkNjQW/JeGldO15a127sx8lKmrlyJRZeYn+MJaOVvs/hOCy2ZEhirrM3GVWXmxSDLZOgNVWmq1Yw+zzYlzowklGzsQDBdzJpdGNiPY/zRORbHLuO/3poOYDg2jOlNFT77IIl9h/nYTZHmOPtkwVqVFIeZinby1DD4xyO9VdIEARBDAcowhyS9DCnRZFAnlMlC6TukmeMMMs3fTlCKu8nr4c5K0uGKmhsGkIupqKzT2Sh04Hy4MB2TPNe34qbH3s9el+RCpEIHMbQUHAS1gW9Xzm9TD1PZd9PCS3HYYliL2Jypui2nNmhw+JhFusbC8RolstZPmwkrDvg+NJdi3HQVx/Am9s69WnlNHm+p49tTgls3+e4+NZnrX2M+2rv55Jw4BL3OR3ljv3myW3Vpwry9uJztXWTJWN7RwnX3f/KkCpsYuuL7ekPQRAEMbQgwRyS9DCLSW2a6FbClxkv395VMj5alaNopkfTppR0KlkR5mLO/QT9iiPpeQV7As0ByxPZbJaMj9z+HL730GvoKlUSfZHRRZh15ziPQKp4PCUKfZ/jmntfid6LPqgZQ7rKacGuizCbuqET0nmiy0E7sugE/vjCegDAu7/7RKIaonoMcpN7jW1OPXkoe3GuYyAjvZ7Iu63pcmdvBe+9eZ7S5/Q+ZftPoh+a75bzWO5XvLTX2jTp77cL1+LWv63Ec6vsuY23dfTWTVTPnHsf/vGOvxs/ryb9IEEQBDF0IcEcIs/GL1vyMJvSym3vKMHnHMfNHJfaZuGbcUGFSHDJ+4S9rLaMzsMcTYTj6QizqG6nI/Yimx/H29Bpjh4pGmyzZExsbQAQRyd1kUYRYdallUt2kciKAAAgAElEQVT2wy5KZk5oQcXzU8JrV08lmrAHxOLNjTzJwfqdUuGPyMUiC2axfk5LBuccnaW02NVuy+XX2ekHxWBPvjabCk4qwqxaIWwC0mbB0Q3goii8Jmd5Oh1fep88+p/esmGKMAuaii5MtHeVcew3H8H1D7xiXKda/vbalsT7ZHXGujVDEARBDCIkmEPkR9iyt1dFFTCNoRDZ3hUI5qzHrNr0WjwpLm2aqFcjUESfKl7a27qjyyyYhdipJktHst34GK65ZxmeeWNbQkDZotaHT28DgCjKqRNGrsM0aeWyPa0y3/vQUThh/4ko55jkJvoQFXEJ198tRXJN6dEAs4dZCOk/LFqHJeva8d9PvoE537JPrBQkrTvAhHCgYUKISflYHcZSTx7UdG46sd9VquCOeaus6+iyjMS5sjVPVpT1tR5mn0e/x7JUil3+3PZ+VIN5asbu3uD3cN9L/ZeOTTfZkSAIghje0KS/EPmmbHp8DCi+TM4xuqmA3o4SdnSW4HP7JLtgG9GeJISQ9DCbJnQBeg+z5wfbywJesGprp3Ff3ZJgrmXClHx27pi/CnfMX4V9J7REy2z2kLbmIJvGW+1BCjuTYG4sOolj1ukPU+lpIBh8FF0WeobN6wFxlDuOMAfv5Qiz6CcH8OMnVuDNrV04ftb4cH1ThDn4e+19y3Du4VPx7Mrt9o7I2yrXycTWRm25aIHwG8tCNhh4JKOuakRfNzHx+gdexS+eedO6jimPMqDalwweZu328euKn86SoX6PZeUCsF3K4glMKUe+8VpRJ5JmQZKaIAhi6EOCOUS+aemKWQjUrAWjm4rY2lHCts5SOHnPHrSPJv0pTcji0lYhT2fJEGKk4nPr42iVOMLMqy5jDegndyUjzOadClGxqyeI+JUradngsCDCvKs7FqxZvnIVxoLzWfa4dgAkI+wBcZnw4H1XKT4meTB1w4NB9ohj9w1sOKbJceL8dPV66C37VfnF5S77HBg/yh5h1n0njsNQLCTbzGPJ2LI7Kcx151nnxY4sHNJHxsIliTzM8TpiNd1ASp6oyTlPPAHQtaEjb4q8WkhEmG3XpmZ9giAIYmhClowQWYjZ08rFrz1JoO7qLgeRWssZdZ04769sAeE8KS5tN/ydGouFJ4mk5ioEsxCCfq0RZo2okQWUzebhheJSeKx1HuaCw1B0ncT+ddrCJjiEHaHs+ZlCSkQ7xVfx5d+/hAtuma+1Fsi7EhFO00Q+n3OUKj5Kno9ez6/K/mKrLKlDrCL3WVRMlFH7qhN2qr9Yd561xVc0fYhLwmdbMqRdoOxxqKno5L7e+Mjr+N/n1iQ+tz1xiAaX/SmYlXR9ALBmW1fiSYVMnig0QRAEMbiQYNYgR8PUKJuaDk5Euyp+IMhswpPzOMqZSBemeIhtwuiO+avwgR/PT/bXkwRzQ37BHFkykH7snQe56p5AjjBbBTMXgllkydBM+tMK5vzeYSBI3VVwGSo5PMxCMMsp9l5cuzMaQMnHkxhgheffVLnP5zyqLlhthFnuM+ccvRXfOijSRpjDyZMyqmB+esU2vLktad+pKMpTd5r1Eeb0UxRTWjl9HuY4M0ZZk25Q3uSmR1+Hiu33I85LnhLwteIl0kgGx37ydx/HJ3+5SL8+ZdIgCIIY8pBgDkl4mMM7ck/Zx3HfegQLVseeU3VCj7jxVjweeJgtgjm4eYr2kjdJWUTZImQA8PyanYn3XigweiteIpKYFTSO03jVdtOWJxsKehJp5SyCOdy4PYwwaz3MkthV20zsy9J3J7Rk5KlsV/ICUatmDBECKFFFUfpcROp1flwgOFaRFaNUbYQ5kfc7aGNUY5WCORx4yKhlyD//2xfx7u8+keq37T1QvYc5XbgkvU+fx+dXFwn2Mn4gtktZ9E31PdeC6XqSr1ePc2zpCAbd81ZsrWo/BEEQxNCBBHOIzpIhkFOPcR4LUS981A6EBRY4N2a4OGhKK046YIL0aFpqGzx3hFmH5weR8J6yjxnjmqPlee0ZPs9Xdjm9nV18VONh1gkv1wnErizItR5m66S/OEPEsrd2mVdELPbVIi4i0tqolB0X7OwqATBbMjyfR/mmSxWvqgizHOX1Q8Fse4ogLl1ZhLksLZh1FhiVlM+5Sg+zLGy/99Br8H1unfQXpaOTPMwlzzzpzzjJMkeEuR5BXZOdRC3Ysm5HUA10XEtRuz7laiYIghj6kGAOke+96o24V0lrJsQt5/FNU0SYTZaMj75jXziMGSr9JSPM1UZ7PZ/jjS3B4/RZk1qj5fknAGZHX7VbacSZTNFhWPCVM7SfiUf0Ng9zEBllmZYMmwc0yJIRXOZvbusyrgcgsk2oglaIeznCvGR9XN1O+MpNEWafx6W1SxXfOqlTRc3KUvJ8a9o0MaCQB32Ok7ZkmPoqo05ifGndztSARbcfEUWWv9KO3gpeWLsjl4c5+bvyU1FpcQ20G3KM/33Vdvznn5bg0Vc24adPrdRum8XFtz6Dj/98oXUdk2BWPe9rt3cDAMYpEzajgXf/2akJgiCIOkFZMkLk26gaYZZzH4viIGXPg+fzaN3Yw6zff8F1gtRvGi8nBxIRQB5GfPOWzvU4x8qtHQCAWZNGRcubLOnpZHwO1GLplH2mOlyHYdLoRu1nQlR0ljyUPV9f6U9nydA0ZR9gMGvFQRlRdEUd9HgawSyzIyPC7PPYktFb8dHQlF8wV1TBXPExxpKLWRthdtJPG0zfWdnzo2uxrPwOvvDbxah4HB+avXe0TOfbNg2kHMZS35/oxw0PvhqdP855LJg1g6GKNG9Ax7fuD4qSiMmAH3/XrOizvOPC51Zlp/5TBxRf+8tSNDe4uGzOPlJ7coRZ/72RJYMgCGLoQxHmkOQkruSNWNzIfZ+j4sURZp/zyAtazvAwuw6Dy1gk7tRIlxrV1N1DP3Pq/hjdVMC0tqbkuj7Hmm1daCg4mNYWWzKack4A5DxdfjgPOg+zTB4PMxAUBlE9tUBwToKUcLUXLgk8zHnLjgftqF+hEPONBf35FIK5w5AFwfM5umqMMG/a1RO/CSOvoxotEWZoIsyMYfyopB3AFGHe2B63p/MPv765I/G+V5MXXOejBoDtnaVoYCcQ5/bHT7whbR8vL1X8VLoa0a1abMj1FKeqFekXz7yJW59cmRrkbA4nDtdiISEIgiCGBiSYQxKT/pQb2+5QCP3bb17AkvXtUcRSPCIHEJVeNmmzosvAGNM+flWzZAD6m+vopiLed9S0VCSzEk4qa20sJCasNRkEnorPaxMSOnEGxJUKbcJQbm9Xd1mfJSP0Hyc9zOl92ewFDmPGR/cqciGXZF+D/aup2QSimmKXody1HGEuedVlybjifxZI+wmOtcXmYY78vfE5cRhLRTdN0XC50I3umlB7rrPSzF+xLdheOZEf+/lCLFi9I7FMt72IpANBFFnthXhKY4ow26inYM7Kux28jqPopuujNjsUiWyCIIiBhARziLY0dsg9L27AUVc/hPuWBOV0owizdDMs+9yaVs51HLhOfKNLRpW4JsKcviEWQi9qqqxxGPlWK+s1FfN9vXK6u2qIIsyKcBERUHFMN192TGpb+RyXPV8rnFwnsFMkJ76l+7nLIogdJ12Aw0RvlGYvic7DLCMm/ZlSlQWT/uQIc/U5r4FYSLZYPMxxdDde5josVfDktU27tdu/IGVg0do2lK7rSrV/6a7FeGDJW5lC0GFBtg51Pc5jIV2upJ9+6Mp/56Xa61wuDa5i9DB7yX9LxHvhY+9rn4D6TFokCIIg8kOCWYMqfHb3VhJRSiF4At9ysKwS5os1CeaCwxKFS9TgmBqN1d1E3VAw94bRbIHn8zBdWXIfeXMyc17b4+1ogplyvlpDwSzO0/lHTsP5R+6VWKeSEMxcKzYdh6HosKB4hcHKAsRPAHQwxhJRf1u5brnyYaKvkSXDHmE24XNgw85g4ldvpboIc3I/HL1eMkvGty48PGHRif3DUuESh2FMU9KS8fQb27RtLFoTR4BNUWgZ0zp/e31rZnGQ0U1FlD0fm3f3JJbLHuayLsKsTGz85gWH48gZbZl9BaoX2dfcuyy1bM22Lnz85wtTFQbV/onXos1OJcLMIEqw97/wJwiCIPoGCeYQ+f6TledVIOccFqWXTfP0Ck5oyRBp5dRKf262h7ngOmh0gwizLh+0GgHNa8ng6FuUS7VkRIJZsjAk80wHIkIse3XjLm3J74rnR/uIi8lU10eHMfzH2W+L3tvOifg+1XORFWHOI3h+9NgKAGGE2SLabUR5mCXB3FRwE755n3PMe31rlDUFCDzZam5pE3LxErkkuKC75OFPL6yL3pvsMMs2tEfn7V/etZ92ndbGAsoex1vtScHscyTmBqQtMmLQGfwd21LE6KZ885dt1/ktj6/AQy9vTG+jfL/fun8ZHnllEx59ZbN2P6qHWbwXPvaO3grm/uGlaKBXmx2KIAiCGEgoS0aIfM/SzcyXEZ/LGQLKnh+UxjZFmN1g0l+cazb+jCNIwSbj+Tx1oy44DI1htoMeJdVdxfM1loy8HmZeY+ESEWHWWzJk64F8Xrww6tZUcNBZ8vDF3y3W+nI7eyuRuKz4HAW3emHPAIxpKuKDx87A7xetQ2PR1UakRzW4mR5mU4S5GkoV35qf2oaI5sqWjMaikxik+ZzjI7c/l9hOzSttorHgJESybhLjL555E7945k28bcoYHDptjDHCvHJLZxQlPvuwqfjJU2lrw+imAkqer62mGRcE8tOFZCIPc/BXlFDPgy3o/d2/LgcArL7+vMTykuejyYmvTzH5U40Yx20k08oJS1HJ81Gq+PjpUytx54K10Tq12aFIMhMEQQwkFGEOkSO+psk8AnGzkiPMFU94mPXbBB7m2JJhq/QnPleFe8Fh0cSzbknYVHyeSAcm9FFewcx59ZFbIMiD6/tpO4WoROcaBLN4TN0o9U8XzewseSiG4lKIr2qFgmhXiHeT6G0qupElQ20jK0tGNZQ833iNZCEGaC1KhFk+t7rTk9cC0tZcjEt4VzzrZEqxT1MBlJLnR7+jhoKjHRCNbiqgvauc+r35PM5AIdtxBOI3JP46jOXOPFKL0FQHBeJJQ6c0oJCjxPLxBIPZ+H1XqZIuCFOTHar6bQiCIIjaIcEckrRk2O9GnibCHOdh1ouTosPgMFkwy22nJ/3pSjkX3LgAhSyYAw8zj+wLog95Jv01Fhz4PB3NzsNfX96EHzzyWqqfIuevHPGWD8/3gyhzVp5oOcIsRGu1XmvRrji/Jl93IJhFHuDkZ1l5mPO0r9tftQjhJov+pqKbPLcaJSWuh2vffxi+cMZBxv2PbSmiu+yBc44Ogz9XIKKmukl/oh9inYLjaCcq7j+pFW9s6Ug93fB8P/oOyl56UmBFjTC7DA2FfIOCWq5zNde0GJjKgzxZBKtZMuT3nSUvNRAWEemesoeZc+/Dr597M7NPJJgJgiAGFhLMIcnS2HZVFlkyVA+zb8/D7DB9lgzdvU+OssX7cCKx9J0HX42We36QD7ohFJeiB3kizI0Fp2YPMwDcv3Rjqp8iEitbD9RKhhWPZ/avs9eLBgGVGiPMTIkw6wYRr1xzDpqKTlS4RG1BtG2bMGhCp896DCIzCyFOG6RId1PRSdldVMTnHz1hJj59yv7G/bc1F6MJpKYJbQIRfbaVAy+Fg5yiy9DamP6uD5jcim2dpZQlQ35iUfH9VARWzfPsOo41wiyLZPn8ZE1KFKi5psVvULasyOuYPMwA0NVbgeepgjl4LypG3vTo65l9UqsfEgRBEP0LCeYQWWf43F7sIo4wyzdJex5mUekvLlySbHtsSwNu+OCR+NzpB4af89SNtSiVOBYp7kTbFT+2ZAiBpFZ309FYdAGuj3ru1daEd8waH72/6vxDU+s4LG1haQxFqXwO1YlpPk9PUlQ5ftb4yNtd1kTmAeCDx86w7iOOMAdt6WwVjQUniDCX9JaM3jAVXK3ZLVR09pM8iAi4LNybim40QnKYfhKebO+1HUNbc5B6rqvXyxTMwiphKwf+7795AUBw7esizAdOGQ0AWL5xV2K5/LsqVXytJeP3i9ZhybogBV7BYfYiOUpGGUFPjiwgQNp20qixZHSV9fYMX/IwA8GxpSLMNWhfijATBEEMLCSYQ9T7j8MY7rhiNi45LigDvO+EFtz20WMBxBEkOVJYCSv92dLKOU5cuIRzjr3HB1X5TjpgAgDgH2bvjWljgxRhns9TXkdXEswyl/3kOWzrKMVZKSIPcz5LRsnz8erGdF7esw+bitFhOrJ/P+0AfOyd6WwHDCyVVUQn1GXRFogIc4S54DC8cNWZ+NCxM6wR5iOmt+F7HzrKenxiwpgQVOo5ccIMEk1FFz0VD8s27MLa7V04/eDJGBNmXugNM1uYvttq0WUEycNfFq8HkLSGyBHmguskfPUCedKcTfOPbQm+62OufRhf+N2L1r7c/PgK7Hfl/caJbzIFh0WZUwT/7yPHYnp4ratZMuTrfmdXORU19znHl+5ajG/cE6R8cxgzFpUBFAEr7eu/HlpuLCYio0aYIw+zNPB59w1PRK8rSkRbHlCWPT/1BKsWmwhN+iMIghhYSDCHpO4/DDjt4Ck4dNqY6PMxzYGgUCPMjYWgfLPPOUxPhkVpbHGj4wAOmNSKZ688Hf/yrllxsywuu61mcyi6jnHi2dodXTVbMkw4Ug5jU1oyxtJ5q0Uf5cfGCdtA6M82CXrGgHGjGsDCSn+A5GGuIUsGEEdW1bRyQpA3F13MX7EN77npKfgcaGsp4rOnHQAgGBgVwkmb9aC7xgjz0vVBJFb+zhoLsYe5wXW0YlzOkmGyDAGBJUOwQimBrfK317YAABa9udO6HhBct3KZ9oaCg3MOnxpnm1Cuc9kqsaWjN/X0Q70GCi7DRcfOwMWz99a2nxCwklb9n/mr8cNHsu0PqodZiPMuqd9yFHr+iq1Se8G1Lk67mhIS0NtosiC5TBAEMbCQYA4R0TWBTnQKoSJueEKcNDe4YR5msyApug4cFt/sxQTBqW1NiW2EuPF9pCZemSLMQCAoVUtGHsE8qtGcWZCxeF+m1GQOY6mImU5Xyov80AKSJ+uE8KaKNmRtkSfgq3qY1fNn8jbLmRd6yl7oQU82eMT0Nkwd04RqsXmYrzz34Mzt5WNolCLMRZcZLBn5hP7Y5mL2SmLd8PeytSO7iqLrsITIFxFVcX2qFhUhPqeMacTOrnLKJ61OMXAdhuNmjsflJ87Uti9bm1Sxqkudp6K2LwZZqtAX/Ozp1dFrzgPB3hIea0kzidHzeaLaZh79zGvIrEEQBEHUDgnmkI+8Y19c94EjIpuEKjo5YguB0B9C0LYUXVTC6nsmaeJGlow424NO8IkItc95ykdadB3ro+da0srZ1nFYvC9ThLmrVEGHoeSvCZFWLo9lRESYKzVGmKMIeXggjpN8fC8Ec7PisWWIbRy9lSDHtSo8v/beQ9GimcyWhc3DPGtSa+b2Da486S8uXGLKRWyLKsu0taQF8+fPOBBnHjoltTxvbuegX4pg5kIwpyfPAUCpEnw+tS34LaqTAtWIrPgOTU93ojzIFR9/fmF9Zn+/qNhR1EGI6H9njicFIg+zuL5KGg9zxePY78r7cf0Dr+p2oYUm/REEQQwsJJhDXIfh0jn7RCWEhR4QflzO4wizEF+iXHZrUwFln8OaVi4sXCKiaxx6ISO29zjH7p5kyeWiy6IJdTqEwKsmrZwuP66AMRb1UexzwqiGxDqrt3Xhqj8vVbYL/iZS50mfCxGRR9ALERjnYc7cJIGahxk8OWlORAtblL44LBbIopy1KpizBjAAcNHbZ6SeXogI6oxxzan181QBTHiYC240SDMJ5rzJPdo0EeZPn7I/vixVShTk8S4LCq6T+K7Fd2iKMIvveq8wer9pl9njDMQRdNNvz/M5esoe5v7xJTyoVPITl9MD0iTaPz6fFNXpCDcP+519DoSHubkhvo7VCPPfVwdlyu9evAFAvicnNWYmJAiCIGokl2BmjJ3DGFvOGFvBGJur+fxkxtjzjLEKY+yDymceY+zF8L+769Xx/kJYFGJLRnCKOI9v8OKGtjMUzGOaikGEGeZJVSLCLG50powa4qbPDR5mm0ATn0V9lywPU8Y0areRBbMqmJjUH9Hsff/+LvzmX95h7EMegrzP+SrnyZX+xLbVEPU/3A8HR1Fq15Sf2XFikS08zKoga3CdSLyabA8zJ7Tg+/9wVGKdcsXH0XuPxY0XH51av5ijAIcs+IsuiyKrprR31RQukWkoBJ553dMF3eRCEwXFkiEohpljVGuDEMRT2wLBnLJsVEyCWd9+xef45C8XpYQwEPzOXnlrFz796+eN/Vc9zCLCrXr3dYgy8KMa4gmkalaZ+SsCwXz49Hi+RBZq5hCCIAiif8m8OzPGXAC3ADgXwKEALmWMqfnF1gC4AsD/anbRzTk/OvzvfX3sb78zqkHYLtKp2cRNv+wFKdF2dpUABBXLKl5Q/MP0+FsILk/xMKuIm7/nQ2vJsIlMEWHcK8w+IMRcg+vgmbmna7dpLsZWhLMOnYKbLzsmes/kSX9hX6e2NeGE/ScY+2AikbbPR+4Ic0Gp9Md5fI7yyEBxioX45TwZiRVp60Yp1grGWJSKrqfshVkykvtuKMQlmUVGDVP7QLI6nsjLrVJthDn4juyWDJOdBgB+8o+zo9djW5JPD9TfQq0UlQizTFPBSUVwhSDee3yLdhtVMIvv1vTb+9trW/BkOElRR1akWM2SUU3hGeHXFwOywJKhH2xkPa1Q90sQBEEMHHn+hZ4DYAXnfCXnvATgTgDvl1fgnK/mnL8EYNhPRWkRk+AUHzDnPDUBUBQaGN1URDmsTmZMK+cyuE78ODeYIJheT2zvmywZlolyQjD98mPH48aLj0pYAUyiSTwqFm0fvffY6D1j6SwTeRDCxXRPD6oi5vNY6zzMkb0ih5ATqwjxy3lSmIjIs5onmCEWYjZLhuifzs4QtC9Fg0WE2eNwJbuLjO48f+msZHU+ddAUTWw0CWbLeXpbmAsZSB+DGCxW41fWoU76k5GvgR9degyai240OJo+tgknzEoPzlSBLb5b03HO/eMSY9/yBGrVPMzVCGaPc5R9P3qSo5v0J8gTsRaQh5kgCGJgySOYpwNYK71fFy7LSxNjbCFj7FnG2AW6FRhjnwjXWbhlizkSNBCIqJqaJYMjnc5LZMmIIsy2wiVqWjmuj0aL7T0/XZ64KFkAdAjxNmVMEy48ZkYukSsLRcdhiYppwaQ/Ycnom2iSb/DCXpHPkpHMkuFzboyk6lA9zGrBFGGBUHNHOywuhhFYMtJZMgLBHEaYjYI5FqIzJ46KluuybgTtpvdx4duTxVnkSX9AfK02mCwZFsErR9ZVwSxSweVwiWRiGujJgvncw6eirbkYVQgsOA7eddDE1DbGCHMN/eqt+Hjs1c32dcp6S4bgCkN2DiC0ZHg8ur50aeUEqjfbBjkyCIIgBhZzTrH6sS/nfD1jbBaAxxhjSzjnb8grcM5vA3AbAMyePXtQbwXCwywiskLUcZ6M4MkT6sY0F1HxOQqcGyO5BdcBY4GHmXNujEa7krDb3VNBa2MhyiJQLGQJZkf7XojVvdqacMDkVjz1epwnVhaKjCWFMQOLs2TkiDLefvlsOIzh2ZXbrOsJwWOKMDNJ+ggx9PL6Xbj23lewaVdvahKdDUcR/BxJr69YrloyHJaMMI9pKqYGDQ0FJ1pHTBbVtX/svuPx/z7ydoxpLuKynzwXtatNv5dDRKvXgPi8FkuGPEAa11LEzZcdg4eXbcJfXtwQeeDrkX/aNAFVnsQqzokQjq7LtJF7NeIrjq8W68ifcmTNUCPaagVO28DPC4v0iIFpULiEBDNBEMRwI0/saD0AuSLAjHBZLjjn68O/KwE8AeAY6waDjJico1oRhOj8h9kzcMcVs6WMGXFUulTxjS4BubSyzwNBrFs1tmQAu3vLCcEgl8bWoQomuT0AeObK03H1+w5LrCOLVi7bHSAsGfkjzKcdPBmnHjxZ2h+0r4XgsZUfF4hj+q+HX8OqrZ3hdvpcyjrERyYPc1S4RLVkSB5mUelPZ8kQ2sloyQj/nnP4Xqlovj7CnF7GoLabfJ/lYbYF5F2X4af/OBsfePt0MMZw/pHTMH1skL1D+G77askAzKJSiHLxNIMxFgnHouNoz6sa8Y09zOb2Td9PHlKCWVGrtt+kSKHYInuYDdYLk5DWQZYMgiCIgSWPYF4A4EDG2H6MsQYAlwDIle2CMTaOMdYYvp4I4CQAy2rt7EAg8uqKm+T4MI3aF88MfKQ3fPAonHbwlOgRc0tDIRJdttLYsge2bMmoIaJlns/R2ethtDSZLHPSXyG5w1gkxjdXVfTJ3a14PPL0Bp/Jk/6MzSbWz0M5PLcmES6LAd0kuJRgtHQubSlJWjrEOVLTyjGWzJKhs1A0uE5UFnxMs/5hjbyNLOxdpp8QqM+cknzfUHAwZ7/xmDmhJdGGacKgLfLqMoYzDp2C7/9DnLFDniwK5P9ebRgn/RWTWUYcJ34C4Tr5IszxtuZ+9iWrRCoPsyRsixkl030elLhvymHJMAlp/X5zr0oQBEHUgUxLBue8whj7LIC/AnAB3ME5f5kxdg2AhZzzuxljxwH4E4BxAN7LGLuac34YgEMA3MoY8xGI8+s550NaMIsIs0hl1VR0sfr681LriUfJTUU3IYQcBjz6H+/G7p4KLrhlfrS84DBMHh2kdlu1tdOYJSNKF8w5eiteIt1ZUbIA6FBTkhUiS4a8fzU6Gb/21Agz9DmVs7hkzj64c8FavPeovaJl8uZC8OSJWuvSrAlhGD0FsAiWKEuGG0fbE4VLxKS/lCVDysNc9lF0nFSktqHgRCLHZMmQuyafe3USYcEJitpoI8yqUC84+N0nT5BWCJfXMOlP9x2oqfLqYckw5Q8XQjLOfL0E39gAACAASURBVMJQ9oLfXtFlGNWYPq/q5LiCZbAhyFPRz4SaVk6OBBcc+2/S84PBbyF8OtTrmbNk6CwZf3lxPXrKHi4+bp/EckorRxAEMbDk8jBzzu8HcL+y7GvS6wUIrBrqdk8DOKKPfRxQ1MlfJkSkt7nBSUQsHcawf1itbXRTIUoNxxiL0mSd+8OnAABzZmo8zKJwic/RW/ET/Sm6+swK8ucysg0h2r8aYZYe93s+T3zuSCnLqrk97zdxFBZ//Szj5+UMS0bCw6yLMDt664mOqMKflN9ajsRHEWZLlow4DVyyHdeJKzeaJ/0lRbLcL3l/RTdIr6bPnJJ8rwrjLA+zen6WfOMsHPGNhwDovwOxfxGxzbJkjGpwM6veZU36c6PvKTvCrJLHw+xz4OCpo/Hqxt2Z+1Pprfj45TOrceahUzG1rSmRC7zgMmtk2+OBh9l1gwqT1gizZvnn7gyqDqYFc9WHQRAEQfQBqvSnkDeaJm70zUU3IepkgbTkG2cnttlHySuru7+L7X0eRDYTgjkjXYFr8DDblsl9CCJhTuIz8Xm1BUNUEh7msPSxmyP9gk4wq8tsX1nsYY6j7cnS2GGlv1ThEpaY5KnzMAOxyDEKZum1/PWok/7UKo2JfTD1O1OPvzpLxugme7pB2Z8frKPdbcRZh03FjRcfZV3HNDhKWTIYiyLIBYOH2bTvLOdIa2Ntc5zXbu/CVX95GR//xQIAybRyDWHxFRN+OOlPRJjLnp/yQAvKlf6f9Pfmts4ofzxBEASRHxLMCnkKRwBShLnoJoSsTbxNCUv92taVs2T0VrwotReQFje3Xz478V6tmKYT2Kp4ksVXYAmQPpP+X8+I1qd+tQhAUkDeePFR+PwZB6bW1VoyaogwR5M3U5P+guVNhbSHWbVMyO9XfOtcALF4kguXrL7+POwVVqmTuyYPEBwn+bSgIOUSNmXByDrGvBHmLBqjiXj5LBl5slOY1lEzcTAmTQp1WULcm8gqjS0YVaNgFv3ZujsQmglLRljy3oQXVvorOE4UYS5X9D8m1Ztto9YB7Lu/+wTO+cFTNW1LEASxJ0OCWSFvjl8hKpob3NSjdhk5qqUKD31aueBvIJh96yS/1sYCFn/trEhotncnC524GvGf7kP8uqJUKnSkKKjt9nzhMdPxA02Z5yTpPcgC8sJjZuDt+4xLraMbwIgIWVyUJIeHOcp2Am1pbHXCJANLTtJTSmPH+aGD41IjobEAlPahvNZNCHQY0Khcg1mT7sTHsiXnk++eFb2u1oLcoEaYM9rPk+3E9B01Kh7moutEgxB1kGJuP1245FsXHo6f//OcxHqthmqMWQifelSeXfEw2ywZckaYhkIgmFVPdNROxkw+zjk+8YuFePK1LX3KkbFxV08ftiYIgtgzIcGskDcaJ0eYZVGnbv7wF09OTNB6+eqzcfkJ+wKwWzKEh9lW2a/gOmhrKUaZPHYpglknZNRF8ludtzKe9Ge+RR+zz1hccEw1tWz0/YvOh7RYHsAcMb0NALChPXnDtwm6KErqiggzx6TWRqkPYTnxtmZc/b7D8M4DJobbpSPMuvMpsmSokVBd1gs5MF5QLBlCsDPGEhPkCg7LnODFlAiz6zBcee4hOGBya9hudYo5Fsz5Ireuy1Kp71RMfVAtGXJ0XQxKJkrfl3bfLPkXAKaNbcY4JV93a0NtgrknTGMnorrynMOGjIm4oqy26zIUXYaS5xsr+onfn7lCJsdDyzbhn3+2oM8WKYIgCKI6SDArqBPnTEQe5gZX8f0mt9+rrRlz9hsfvR/VWJCikZoIcDQ5Lcg3a4swCy/uKQcFuY/ff3RStOYR//K9WxXMiUl/lvtznnZ026siKhJm0rqyGDliRlvV7UdRaOk4vnT22yJhLO//8hNnYtpYYaVgiYFQwU16mgUiKqj6Y6M+mSb9KZMIi04c1ZUHSQWXZT71UCf9qaej1jzKeS0ZefZv2oU4VrGPpL88WLbwq2fgoCmtxn2L35ws2l2Wjk7XGmEWFT0rYbTYk7JcFBz7pL+eMJocRJhd66S/LMR2DDTpjyAIYqAhwayg+mNNiJvxhFGNVkuGDvEYuqLxLDqpCLO5P0LQ7TOhBauvPy8hzAF9hFm+0S786hmJx8vqjTwoXBJgi2hlTUY0YYwwS8jndmxzEbd+9FjcdGmy9k2eCHOsxTlaGwuY2BpE5VXLh+ylla+FgsO0gylx/lTfsS7VmWrJkLstT/qTv/Oi42BUYwF//8rpmcco+qdGe6v1MIvrQBy+unlKkOfY/7SwGIqKKOctRGcywlzd74opkyrV33KtHmaRYlKcFy/hYXYS3+uvPnZ8YtvNu3qD9ZygSmfJ45nWC9ORiu0Yo7RyBEEQAw0JZoU8fkwAeGNLBwDguP3Gp/IwZyEyX6gVxIBYfFR8P1MwZ0XDGWN4zxFT8T9XHBcta2su4sJjpuMPnz4RE1sbEzP2I6EkgqOIo3e2+3OeiZK67fN4uhmLhWrBYTj7sKmYPjY5edLWvtilGimPMksY+sBY0oJRdB1tpPeIGWMBxBHmd8wan9iPmns5akfx58o2CJ1onDw6eczJPsd9lI9ZUG2AWQyOmHQuZMa2NCTe5/nN7D+pFY988eTU8qh8e/i9NGj85bo+6JDPtZxHW6BmQsmLsGSI34o8zg181tJ7xZ71h+fXRcsbXQelimfMwyzQ/dRWb+2MSnIzxqjOH0EQxABTW8hlBKN77K5DCNk5M8fjlY27ouV5ImFNkWBOT/4RwmtnV+BHbrTkhc4TDf/xh49NvHcchhulCXqeJsLMghAWHCkKaosw5z1nKuqjfNOZC8QST7Uj1rdZAiIBHL5XxaC6z1joJgVbU9HVCuYbLjoSH3/nfpg0uhHP/efpkd1GLa4CJPfnOslrpaUYlmRnye8837kVEWa9YK7WkiG+atN2E1sbsL0zTk2mm1yq44DJo1PLhEAW34s8QCzkzD6jW8fVeM6bLINPG91hhFnoXNmSEWRTkZ4IJGw8TiKndEPBQWepEgnfajjle0/gqS+fCiA4TgowEwRBDCwkmBXyPr6+/qIjsXjtTkxta8Lrm+NiCHm0SXNDcIPVRZhFieUtu4NHufZS2H1/QJCwZAgxGb4PLBnZB1TM42HWZckwlbhWI6RifUM7uSa1CUtGFGEO/qYjzGL1pIe5seBoI9nNDS6O2juIMstpA11NhFnup2rJEBUdHYfhkyfPwr/++nkA+c5tHGFOtike2+vO273/9k68ua1Luz9RkfLAyXrfsJhkKh9LrQiR70XWlqR/W1Bt6jrXSR+3bfBpoyf8nYrIsKx3GdT82kkbT0l63dpYwKZdPZmWDBPrd3aHbTKa9EcQBDHAkGBWyDvpb2JrI04/ZAqAdAW3LCJLRjktmEWEeUtHDsFcbb4wDbIlQ/ZIBn+ltHJ9nPSno7no4rLj94kKNph2o6aGS7VvOeei30L4i8MwFftgUoRZzs3cWHSMpae1fdKIf1cRz/K1MqpR5D4G3nPEXrj7syfhfTfPzxVhVvMwq2dDN6A4fHobDp+un0R54gET8b8fPx7Hz5qg/Xxsc/WWDBNFJcKsm/QH5Iswy5cBUyZtAnFGjmoRYl7oXHmQyZH8zct9Tr4OirC0d5drnvS3bkcgmCnCTBAEMfCQYFbIO+nPtE2eYFujxZJRcB2ManCxNRLM9rRyfUW+efvRLPzAAsEA7B9GGWeM00/aAvLlrtbd4JuLLr59YVw5PcunajpeW4SZ8+QgQChm8ZWp33fkeXZYFPUFAvGcN0c3oC+m4SoRZvmz5tCSoXqf8/jDxSHocj+rfcjLiWEWER3jRqkp9BwcNm1M1W0Acc5pTzN5MvHdVBth1niY1eI0taJ6kJNWm/i1fL14PkdbSyCYyxkeZhNrtgdPBBxGEWaCIIiBhib9KeSt9CdTc4TZUAp3THMxsmSo2RdkaumriiyYo0fFUYQZuOS4vXHXp07AOYdPTWwnZ22otR/NyiQso4c5/CQqPqJohavOOwTTDVkYlEOKrCGRh9ky8VCO7jcW9ZYME5F41SwTr+WmRe5l9fKR+/fpU/bHVecfmmpLTFCUM3wk+tIHy4SONjXC7DIcOGU0Hvjcu6Jl37zgcHznoiPwl8+cZN2XyD8trsOkh7m6CLM6OFEHQ3Ku7Nsvn41T3zYpe6caVL3rGiLM8vfd3l1GW3MRvRW/5ujwulAw53BJEQRBEHWGIswKtTxervbGLib9lQyCeXRTIZeHuRqLgAnZkuFLeV4BhJP+GI6bOT613eTRTZjW1oQN7T25ovI6jdCseEozK8oZ7BMnHjAR8+eehplz70t8PmFUQ+S3dRSxLb4n1UetHrugqejWZMmQT42rCDpdFUA1m4d8bv/vOQdr22JIWjzSlozc3c7FWKUgiGi3Sfo+P/KOfXPtS0RhxcCmUaoy6FQ5EJV/e7osGXIe5tMPmYI3tnTg8eVbcvVTRv7NBB7m7Ajzzu4SpralB3UFh+X2NK/dQRFmgiCIwYIizArVPHYXVJv+KjPC3FTE1o5gulCjxXfZF++oQJ6xf8uHg/zGsYfZvq3YMq/vW6Upp2BWPcxict0JBo+tYNFVZ0YRerFnITRMaeVMx95Y0KeVMyEmf8mTJh1LhFm1cIjH/nnOrRNGmMX+osGBsu96sVebktZPk3PaxijpyUKDwZJhK+FugqUizMmNRiuFS2odcKoCN1kRUp9HemxzQ6p8OgCcf+Reudvd1V0BIPIw594sgnI3EwRB1A4JZoVaxEW1s/mbLR5mILBkdPQGN0ebh7keQkhEy7563iFR2i8h8rLEfxQFzSE89h7Xklqm5sU1NScWCxG69/gWPPXlU/GFMw/KbFcg+hhlo4gEsz6tnKotmpQS6Fm4BuEdiVolAiqnswNiUZZv0l8yYq32st6WjHfMmoDvXHQEPnTsjKA9caw5vAIPf+FkPPF/To3ei0l/nmbSn0yegaiMmrYPAEYrhUtqzZrhK4I5MRByk6IdCLKN/NNJM7WCeebEUVh9/Xm57CElT0yOrS3CXONcQ4IgCAJkyUhRix+3ekuGOa0ckIyE6SwZv/vkCXhg6VtViwgd4uafjJKHf3PuI0+k+zOn7o/RTQVcc++yaJkaYc46HDnauvf4tAC3cdSMNnzhjINw6Zy9E22lI8zBe1WQNBaqzZIhPMma/XMeRphlcZVsvxyKozznVohvuehKss3c3c6F6zBcfNw+WL6xo+p2DpySzMUsJv2J61A8UUmXaa+uj6Ji4r4TWqL0eWppbJvdyYbaN3lAImeuEcvPO3IvFFxHK5jFU4s8g19h4WLQW5yyIBsHQRBE7VCEWaGWLBluosBC9o2vqcGcVg4ILBkCXYR5zn7j8fX3HlZtN7WIyF5BE+3MtmSIKGj2MRdcB2cdNiWxzFTpT92bEIB9iagzxvC5Mw7E5NDOEWeh0GfJUGksVBlhNpSVjj9P5mEWUUqxrBJaZXLZQBiSloxUm/VVzCYRnysftoIpwqxGQ9XflcMCIXrUDH1qPNdhcByGJ6VotuqZt02oteH5PDF4MnmYBeI71Alm0ac8g18xiGKM1WSvIMFMEARROxRhVuhrhDlPlE2ktzJZMiaFhSOAoMjJo//x7tTj5HohyvzKYke8yhL/uolpNrIEb+akvzrOXssqXKIKksZidR5m0VdVo8jnVk2DJpYDwBHT28AY8K+n7J/Zlhphjqu05O5uVZiycdQynkmXxtbbJFLZQ1wHt1z2duN+TWXWZWx2Jxse52gsOJFFwuRhFognIyLXtox4yiKfO3Eu1GswFsy1epir34YgCIIIIMGsUKwpwlydh7noMpx56BR8+Ph9tJ9PGRML5qltzVExk/4gsmQkqj4k/hipdtJf1rkxFi4J/9ZjkqPal1TmDQhLRnL9xhrzMJuKVKge23jSX/B+3KgGrLruvFxtnTBrAloa3DiHdJ0tGCqmgUst+Z5Vm4uI+o5qsE8IzSrak+dpRC0RZs45fJ+jseggTGSTjDBL15P45sX5anTTgllU/dSdO/XaKUkFfsiSQRAEMbCQYFaoadKfLJjzPEFnDD/5x9nGz+USy/0ploF4clmfIsw5hWRW+ixjc+FyNQWcice/dErmRDdTHmaxWXrSn1NVNpCCQTCLc9DgOqk0aPLfarjo2Bm46NgZ+MOidQCk89hPwlm9xuNqitXTUFCjvsHOx7Ykcz2r0eGsay7P77gWD3PJ81HxeSI6bar0JyLE4ropFtJ9ai4mJ6HKqL+XshdneFEnHuah1gqDBEEQBHmYU9QSxaw2wpzFVCVtV38iok6ywIzESeahpP3P1rYyBbN9P3mj/1PGNGKfCfZJgZElI+VhDj7gSgyv2gizkxFhbiy6qTRoQfu5m9C0Gfxl/WDJmD/3tOi1KcJcyyRU9Zw2RIJZzfUc/BXnKWvwkud3WItg7in58MIIsyCvh1k3abQp8jDHy8RvUo0ICwsIQ60R5ho2IgiCIACQYE5Ry8QlWUDU6ouUmTJ64ASzSA02Z7+4OImTVy+H5NVJM8Y14zOnmj25xjzM4d+80f98RS6Ydp/irSoumopOVYOpKMJseAyuiqeTD5qEDx+/T5+K0aip6QT1yJIhV1I0XR+1WEFUW4QQwqbiKOK8ZvnZ5e919r7jtGnbarFk7Oopw+c8WdFP40UHJEtG+J3qouK6CLMYZJmeyLAa08pRHmaCIIjaIUtGHZCtArZCI3kRYmHWxFF93lcWJx4wEauvT3plWU57QPwoPp9SYozh/5x9MG55/A3t50YPs8Fv3BdEW2qkUhyLEBcOC8RzY8GtKoIqBl4m0aNeJ0fvPRZH7z029/51RHmYw7+HThuDlVs7Maqhvj9z03mo5emKGmHu6A0mwqoZJeRJmr3IvhZk4fr7T58Yvb798tnY3hkWBaphcPvN+5Zhd08l0b/EE6bE7L3gjy0arpv0JwSzXFRIhtUQYv7ls29iW0dvdRsRBEEQESSY64Acbao1t6sMYwx3f/YkzNAU+xgIIgtshv4567Ap+M3f16YKkNTerr3BvBHePLotTlWnFi4J/sYltINoXlOVAyHRV5MNpR5PIlSiyG/497sfPApXnDiz3y0+wr5SSyRbFcxeWOFQjmgHJJ8IZF0LpgD06YfEqQ1r+a3+9eVNiX6ApUtwq9isPFEhHWkfWRHm4JrM3WUAwFV/XlrdBgRBEEQCEsx1wK2zYAaAI2f0LdrYF+JsC3YhcM37D8cXzjgIo6qcmHjV+YemyivL7arNxkVG8p3baiwZpkl/4pH3AZNb8erG3VU/vo9LXBssGXW6TnRtimNobnAxe+Z4yxZ9I/09Va+Y1d/L+UdOw4adPfjnk/bTtmWzN8j016S/1P45h6krcZYMc1+0lozw2jP53xlLe+wJgiCI/oUEcx2QH//2R+Rw4Mk3Aa3oOlEhkGr42Dv30y7P8o/ntWTkWcuUhzme9Bfwy48dj8Vrd1b9vWZHmGOVpYrDWpHLbg8GfcnDLL//zKkHpNZTfeyZEeZck/5q/60Km8vkMU3GtuIsGWZhrrNk+FGEWV/YqJYIM0EQBNE3SDDXAacfIsyDySDprZQdQiC6k9eSkUcsxRFLfYRZ9GHS6EaccWiyQmEehLAzepjD60T1j/eNfN7z/qKWdqtN4yjyL2dlLKk1D/PbpozG9q4StoRJlk1FQg6bNgYfePt0nHXYVOzsKlnbsQ30IkuGdO7ENWOMMKPvE/iWbdiF/SaOitonCIIg7Ax/dTfEGAkRZnHrHmjhZfIwV1sauxoPs2rzUCf91YroqymbQf9YMoK/Ay2Xo8mfA9CwmGBbzaQ/E7rv4I5/Og7jpRzQpgmTrsvwodl7o625mLoub7z4KNz37++UCvtYIsxhH1Q7i+9zuyWjjxHm99z0FP79zhf6thOCIIg9CBLMdaY/hNBAY/IS9zfqpDWVvHmQ83hpTR7mOK1cfQRzxZDpoH8m/eWz0tSK6js/cPJoAMC+E0Yl2q+Ft+9j9+zPX7EVADAzbCsrJ3ee9JCuw1ITVl3GEudv1iR9php5Qq4qmC88ZgYOm9YWiVpblgzhxVa763GzYHYYq4uHWZxTgiAIIpvhr+6GGCPCkjFIj/ZNQlf1r9YDU+ESccx9jeBFpbENO+qP6yQqXNJP39uDnzsZT3351Oj9h2bPwJ8/cxLOOXxq2G5t+/37f56OX3/8HdZ1RHaL847YC0D9Ugz+/J/nJN7LOvyKE2fiOxcdqd3u4Kmjo9dZ0ew8k1XV35rnc0se5r5fnwDQXfb6vhOCIIg9hOGv7oYY9cjDPNjkLvRXZ8x5mIO/tVRhNLelt3nEWTL6tv+PvXM/fPDYGfj4u/QT+vpDMMf5s+u+awBAW0sRe4+PI6uMsUTu6FoHWJPHNGV6ab//D0dh2TVnR09w8pZjz+I4JYtIEGEOjuNDs2dg8ujG6LNfSOL6oMmxYK7HZFX1OvR8jofCFHYqDPWZ9Len1jHhnOPlDe2D3Q2CIIYZw1/dDTH6UqltqNDfj/ZNmCOj9e+IaMr0uLyvj7xHNxXxvQ8dhTFNRe3n/WrJGPChDsJ2+4+C66CloRCXxu6nUYHrxGeP8+B7FJx80CR88uRZaC66aJMqEbqG34u4hnT/Jlz09hk4Rao+qG5b8TlufOQ1bR+DCPMeqnbrwOJ17TjvpnkkmgmCqArKklFn6hX5Ggr016N9E/GkNX3Ut75t6SPMWZaMuz97Ul2sIf3xJCLLA97fDISFpz+qPso4TtLDrM5JuPI9h+DK9xySWCYmIqo9EteQ7t+Ez552APaTKnmq505UI9T2sYa0cq7DjJ7oPY1d3WUAQHv4lyAIIg8jR90RdWOwLBlxDuTkjf3a9x+GqWOaMG5Ug26zmohLYytZMqK0cnpxceSMsThsWluf2++PJxHxk4FBijAPRJaMSDD3zz9dsnDNG8R1M867zuNsmmwq2LSrx9heMCG1OvE7SGOoIYmY0GtIc00QBKGFIswaFn71jEHLZTsUyFvpr96YArfnHL4Xzjl8r/q25WREmOvamrn9ejJYA524/f5v2VRwpl7IWTLy2nLENWSKMOtOi+m6E9gEM+fVe+yD/VOEGYi/F1NhGIIgCB0kmDVMbG3MXmkEw3JW+qt7u31ssKnooKec7yYY52Hun7Ryg4EQXTkriA9LxEAjb5n06vcv5+LOt00taRjTBXPyC2aP86on7O3B4/8UUYR5GP7GCYIYPEbwrZWolcHKw8yiv7U1/NfPn4z//vDbc61rSisnDno42j0He9LfQCCO0ZbbuC+4jOHGi4/GhcdMx2HTxuTapsF1cPZhU3DHFccllp8ZVojUTfxUBX/aktFrbM/3edVijwRzjPBym3KkEwRB6KAIM5EiEq4Dnoe5b9vvO2FUVEQji4OmjMYBk1sTVd0AoKUYZK9obRx+P43BnvQ3EMQDnX6a9McYDpjcihsvPjpadt0HjsC2DrOAZYzh1o/OTi3/6nmH4NOn7J/IqCFQLRnq+40ZHmZZ6nHOM3+rI3kQVS1iMEwRZoIgqmH4qQKi34kjlSOX42aOxyNffHdq+QXHTMfm3b244sSZA9+pPiJE02BN+hsI+tOSsfr687TLL52zT037K7gOpoxp0n+Wyv+dfL/FEmEOLBmx2OM8e5Bk+tz3eb/46Ycy4tyZCsMQBEHoIEsGkSaKVA7sjVRkrHj/0dMGtF0Z12H49Cn7ZxbSqJXjZo6ra8VCHSNZ/vS3JaO/cY2TTZPr7ew2p5Xz/aS/WpZ9S9e3Y2dXelvTBN49sdqf0MmUZo8giGogwUykELfWgQ48FV0Hi792Fr55weED2/AActenTsQb335PP+09EAAjOWDY32nl+htR+VFNZxhN2Ay/u/buMia2NmDGuObUPgJLBk+8F5z/o3k4/0fzUtuYxr5X/Xkp2rv2rHzE4nyRYCYIohqG511nCDLv/56Kp+eeNtjdqAtsECePtbUUh60YGmz8KI3ZyFXMUf7sYToqmHvOwVh13XuMEWZRKGVHVxmzJrbiOxcdmdqH5/NEDmHVirtuR3dqG9PZ+uML63HdA6/k7v9IwCdLBkEQNUDKpE7MGNeCaWPT0aDhyGBFmIm+EeX9Hdxu4JLj9u63fbNhHmFmjGkHNGKZKGhTqvhobSporRTqpL/P//YFALVHTHv2MFuG+J34JJgJgqiCXHcdxtg5jLHljLEVjLG5ms9PZow9zxirMMY+qHx2OWPs9fC/y+vVcaL/iO7Rg628iKoQkbPBLLqz+vrzcL0mKlovRGS2v7JkDBbiO2soxN751saC1u/u86QN4/4lGwEAZS8OO5cqyXzkNmm4p2VXi9LKkWAmCKIKMgUzY8wFcAuAcwEcCuBSxtihymprAFwB4H+VbccD+DqA4wHMAfB1xti4vneb6E/2hHy+I5FIQ43gry22ZAzPCLMJcVyNhfi4RjUWtE95PJ9rFXCvJJLX7ehKfGbLoLanRVqpcAlBELWQ564zB8AKzvlKznkJwJ0A3i+vwDlfzTl/CYBaZu1sAA9zzrdzzncAeBjAOXXoNzEAkCVjeMH3gEl/ppLmwx1xPLJgHt1U0KZ80xUueXDpxkRUeXtnMlOGTRzuaZPfotLYe1ponSCIPpFHME8HsFZ6vy5clodc2zLGPsEYW8gYW7hly5acuyb6iz0hn+9IJPYwj9zvbbinlTMReZglwdzaWIiygsioHmYA+NSvFmH9zniy3w4l84VNFHt7WKSVsmQQBFELQ+K5Juf8Ns75bM757EmTJg12d/Z4aNLf8ETonhHmVkgw3NPKmTBZMnSRdI/rS2PLk/fUXMwj1ZLx/Ydfw4NLN1a1TZSHeQ8bKBAE0Tfy3HXWA5Cnvc8Il+WhL9sSg4QIalGAeXghRNRIjjCLa1KtlDfccTQR5tGG8uxq4RJBd0IwJyPMVkvGEBOO81dszZ0b+qZHX8enfrWoqv1ThJkgsBd4ewAAIABJREFUiFrIUxp7AYADGWP7IRC7lwC4LOf+/wrg29JEv7MAXFl1L4kBJRbKI0uUjHT2Gd8CADjlbSP3KU1sydCP9Z+58jQMRx0kfnMNSoRZN2j1ldLYgs7eSvR6R9fw9DB39lbw4Z8+h9n7jsPvP31iv7RBgpkgiFrIFMyc8wpj7LMIxK8L4A7O+cuMsWsALOSc380YOw7AnwCMA/BextjVnPPDOOfbGWPXIhDdAHAN53x7Px0LUSdEhHKEBfFGPDMnjsKir56B8aMaBrsr/YYoWW4qXb5XW/W50H94ydGJDBODgbBFNEgDgZZG/TF6Gg8zAHT1ShHmbjXCbGl7CEWYRaq35Rt3R8ueWL4Zu3sqeO9R0+rShk9p5QiCqIE8EWZwzu8HcL+y7GvS6wUI7Ba6be8AcEcf+kgMMGTFGL5MaG0c7C78//buPDyq8uwf+PeZmUz2hZAQCFvYRDZRjAiICyCCUsW9WOvS2lpbrb5dX2ytda9Wq7V1qVptbX8qWvWtWFdEECwCsq+BsBMIJGTfZ3t+f5xlzjlzZoMh6/dzXV5kzjlz5kwmE++5cz/3fVINyUvHX28sxnmnJC6LPvf0WNcwnzxa7JZs6MOc7HLYll5IaV933GjIMBtrmKPVKHfGTGtDmw+z/7gMf77uDNz8NyXfkrCAWathDnTshyQi6lq618oZSgjGy9SZXTi6wFS60B1oWV7j8zIGz1Z22dFmj6EkoymYYY6WQTbGjWv3V2NPZWPU6z1ZjMF9yZEGPL24NPGPoZdkJPzURNSNxZRhpp5Fa3HV+fJORN2TluU1B8zhPxTYZYUb1ZKM3HQ3mgzBc7QEsnHR31XPfwVAmdjYEawLEE9GtYhkhpmIjkP3StNQQmglGXYLi4go8bRA0RgkpySF//Vsl2HWFv2luZ3wGoZyRMswd6aSDGv5iH21tuJ4r5sZZiI6HgyYKYRWksF4mah9+P32JRnh3oN2waKWVU53u+A1RIPR3sd2AXVHfVi2ZpgjJYG9xxnxat+6pTsqUPzQIlMpCxFROAyYKQRLMojal9+2htmBgblK14/8TPNizogZ5mQnfIZgMlqfZbvg2zopMJFavX6UHm2w3WcdVx0pw3y8XS60Dwh7jjXhWKMH1U0e7K9qQn3ryXvORNT1MWCmEMwwE7UvrRTB3CXDiZw0N/Y9OgffmjjIdLzPJrva7FFqmJUMc/DNu2pPVcTH1gLmzWV1+rb9VU1xPoPY/eTNDZj51DJT32iNNdsdKSa2+x7Ewlr20eYL4PzHl+LyZ/97XOcjop6BATOFYA0zUfvSM8zOYI+aZEMNs8PS69GYFR7dLwtAsK2cUsOsBJOBgMQtr66J+NgBKVFe14JLn/lS33as0RPhHifmKzWA99j0vrZmuyP9DvL6jzfDbL6tXceeypP3IYGIuj4GzBSCJRlE7UsL4hyGaUHGISbWIULGcgRtcaA2uCQ92aXv98SQhfX6ZUi2t76l/coTWr1+tKpjvePKMB9nlwvrY3T00Boi6hrYVo5CsCSDqH1pZQJOQybZGDw7HOEzzNrUQy3DnGrIMMcSMNtleutOYsCs/V7RnsHpD3wKACh58OKQzhWRMszWeufYH998P7vnT0RkxYCZQuglGcwxE7ULLQB2hplHb52+acyupqh1z8EuGYaAOYZgsM3nD8mytscCOO05t3oDIds0kTPMsf1+8vkDcBmy9da7tfn8ICKKhiUZFEJAj5iJqB1oNczWWmWNdbvXZyzJUALm5jY/XA4Bt8uhZ19jCpi9gZDjTmaGWWPXzi6khjnC/WNZ9LftcD2G//ojLCmpCD6G5XGNATsRUTgMmCmEYLxM1K60QNHlDBcwK/8mqftbDVlRLWD2+ANIcjrgcjjgC0hIKWPqVdxql2FuUbLVxxrbsGL3sfieTIzsMsShk/5ObNHftvJ6AMDCjYf1bdZAnX2YiSgWDJgphBYwR5sQRkSJccOkwchOTcJFo/va7tcyzHo22RMMmI3dNNLcTr2Xs9cvY8owe/0SLR5zWYKWYb782f/iWy+tiuOZxM7a3g2wK8mI1Ic5+nPrne4GAFQ1Bbt+WE/Z7GFJBhFFx4CZQjx8+TjMGdcPE4fkdvSlEPUIIwoysfG3F6Fvdortfq1zTZq6wM8Y4CY5BFxqCnr8wBz9a18gEHMHCGMJRmaKS69hLqtpAQC8tGzPcfc9ttKyxnYDU0K6ZASMX5v3xZJh1j5M1BgCZut5GDATUSwYMFOIorx0PHv9BNMQBSLqOFpJRppbWaf95a5gmYTDIfTyhinDeusL3Lw+GVOXDMAcMBdkpYS0lXv4w+3419qyuK75061H8PKXe8Pu90sZUnJhDaKNJSXWEo5YAnjtfNXGgNkSZ7ewJIOIYsCAmYiok7OWZBgZW9GNKczW65y9gdDFfOGYA+Zk2z7MWq9mjy8QUxeNW/+5Fg/+Z1vIdi1b7g/IkCyxNftrDPit2WdjAB2u1tk+YDYf28QMMxHFgAEzEVEnp2WYU5NCf2U7DQsF+2WnIEnNMPv8sS36AywBc2ZKxC4Zt7z6NU6779OYzmvHWJLR4jUHq9YssjHg9wXMtdbG52ZX3mHcbnwca8Bsrd8mIrLDgJmIqJPTsrLakBIjY4a5b3aKXsO8am8Vdh5tjOn8xgA5PysZTR4/apvtx2MvL01M1wx/QOoT/vRtEYaKrNlXjVH3fqy3iDMGyeF6MsdSJ22dckhEZIeDS4iIOpEv/3da2PUDqUnBX9lD8tKx91iTadhJSpJTzzDftWBDzI+plWD84ZrxernF3mNNcV97PPZXNett3wAl82wtyTAuWly9txqAErBPO7WPqZzD6w/YlqvYddmwxtDNXmaYiSg6BsxERJ3IgF5pYfelGEoyxvbPDgmYAegBczy0DPPc0wv1nsVH61tDjvvSkF2WUuqZ7+Nx++vrTLf9ARmSEfZEKLswtpULNybbLvNsrXduZoaZiGLAkgwiok5OC/FSDVlUrZ7ZaQlaww0/iaSuxQunQ8DldCArJQkAcLS+LeS4b78c7Mkc62jqWJVWNIZkhK01zEbGINkbpiezbUmG5VC2lSOiWDBgJiLq7NRAUhtKAgQ7ZjgcAot+ch6W/3IagOA0wHjUt3iRrJ47O00LmM0ZZmt1Q6wLCsMtyLO6+OnlIUFxm2GiYcSWc2rwvLy00tRuzhiAa9tDJ/0xYCai6BgwExF1clqs6DBkk7UA1+kQGFGQiYG5SinH8ZZkaMF4pAyzUSyDQ5TjYh94Yl2A1+oNLcnQvgXG4Nrnl1i9txo3vLwaTy7aadqu8egBs/kxjaOxKxpa8e66+PpNE1HPwICZiKiT0+pujdUXWuAXUpLhiP/XepPHH8wwpyoBc0VDaA2zUayBsHXaYKQwu74lfD1xpMEl3kBAr8MuOdIAAHh/42FT/2WtvCNShvl7r67BT9/aiKrGtrgCfSLq/rjoj4iok9NCPGNorGVcQxf9xV6SkZLk0LO4WmeOrFTlfwt2i/6Mwi20s4p1eAqAiANRIo3G9vmlviDy85IKfLCpHD9+Y73tdUQKmA/XKqPAn15cin98tR+PX30arikeGPP1E1H3xQwzEVEnp8V4xq4U16qB3MzRBaZjw5VkPD3vdNw4ebB+e+v9s7Dmnpn67eBwFCeSnCKGkozYAuF4MrUNreEzzNaFfcYuGV5/AG7D87Z24ACCme5IJRnaPq2FXWlFbH2siaj7Y8BMRNTJBWxKMkYXZmHfo3P02mVNuC4ZeRnJOHNwL/12erILGcnBPzLuq2pWH0MgKyUpZNqfRPhFd5HElWGOMGFQC3i1Z2fKMAdkyNCTkOsIs+jPeB5t3y41UI7n2omoe2PATETURQhEL7cIl2EWwrxoUDO4txJw33/ZGH2bVsdsZM04x9pWzhhYv/n1gYhZ5PoI+7QR1tqjGrtmLCmpQGVD5Iy4Fvxa+zAbaWUfPv1fBsxEpGANMxFRJxcsyQDeu/2ckLplo3ABs0MI24B5yc8ugBDmco9Mm4D55S/3mm5Hyr6uP1Cjf21c9Pe/72wOex8gcg2zFjBri/2MLeeeXlyKs4p62d7Per2RYmCPJWvu9SW21zQRdV0MmImIOjmtHMIhgPEDcyIe6woTTDsdAnaD+Rw2x9tlmK0+2FyOkiMNuPrMASH7rnhuhf61NQiNJFJJhjbCWjtfdZP52KpGT8h9jLT7RSrdsLbKCzcQhYh6HpZkEBF1cgGbRX/hGDPMP5t5iv61QwQX9kWTlRI9l/L80t34+b824pxHP7cMEbFmaROz6K9FXZynZayrm8wlGLURgm0AaPX68fuPS1Be1xL2mNDhKMwwE5GCGWYiok5OL8mI4VitvRoADOqdhqLeadhX1QwhhB5wu6MMN4klw6w5VNuCyoY2FOak4qrnV4TUEseVYY5UkqFlmPWA2YPMZBca1GEn1kWKVl+WHsMLy/bEfC2AOdhv8/nR5gugodWH/jmpcZ2HiLo+BsxERJ2c3qEihohZG5kNKENMtLplYw3zoN5ptvfVZMURMANKOURhTirW7q8J2ZeotnJaDbMWMFc1eVCQnYIGtaNFtBHclY2RFwXaMS76u+LZFdhWXg8A+Oru6eiXzaCZqCdhSQYRUScXzDBHj5i1iX2AMsREq+JwiGDwOjg3csAcT4YZiDwVMFGt2ZralIDZ6w9mmAuykmO+f11z5Ay0VbrbCY+hJEMLlgHg3XWH2HKOqIdhwExE1EXEUMJsqnNOcpkzzGU1Sq9la+9mq5y4A+bw2VvraOzj1WJY9OfzB1Db7EVBZkrM919cUhHX46Unu8LWXz/+yQ489nFJXOcjoq6NATMRUSen9Q6Ofei1IslSkjG6XzYA4OKxfSPe75LT+uGi0QW4buKgmB6nor4Nt78WOl0PiG3h3L5H52DWmIKoxwFKxrpGzRb3yQoNmEf1y4rpPNGkJ7si9mHecaQhIY9DRF0DA2Yiok7O2Ic5HsaSDAmJqSPysOX+WTh7aO+I98tKScKLNxbj9mnDYnqcw7Ut+GBzue2+WAaGAMDIgsyYHsvjC+gL/PIy3CH7jYseT0R6srkkw8o4UpuIuj8u+iMi6uSKi3IBAJOiBLpWLqdDH3KixavGcdhR7++ILfhctP1o2H0fbSnHc0t3mWqr7YyIMWBu8wXQqHbG6JUWGjCnGhY9Akr/6WgLAu2ku12RFyF6WcNM1JMwYCYi6uQmD+uNLffPiivYBZT2cVqGORAlw2sn0kRBo+qm8ENDlpcei+kc/bJjq0f2+ANoUNvP9UoPrbVOsQTMqUlOPcCOR0ayCzXN4Z9XCzPMRD0KSzKIiLqAeINlAEhyBVvJHUeSNezUwHhdWxw6DdAq1s4cHl8AjWrmN8eSYXYIpQzFyFiiEW18tlGq2xmx/rrZ4w+7j4i6HwbMRETdlMvh0LtmHE+G2eWMHjC7o5RaAMDY/tlRj7H2fh4UppOHNjwEAHItAXOS0wGXZSiL0yFQmJ2Ce+aMQqo79g8dbpcjYus4rWsHEfUMDJiJiLopt9MBLeaNtujOjl0N8+pfzTDdjmWM9sBeaciMcpwxw7zjodlY/LPzAQBzxvXTt/fNSkFjq0+f7peTZg6yk5yOkKy4y+HAirtn4HvnDkWapVwjErfTEbFLRisDZqIehQEzEVE3daIlGdYa5m0PzEJuujmrG0upSL+cFGSlRC65MNYeJ7ucSHI6sO/ROXj0qnH69gtG5qPF60eNWjNtfWyXU4QE+cabaW5zwByp64jLKeD1S6zdX4N739sSsj+WdnlWHl8Ab319EIHjeTGIqEPF9PcpIcRsAE8DcAL4q5TyUcv+ZAD/AHAmgCoA35RS7hNCFAHYDmCHeuhKKeVtibl0IiKKxOVw4PFrxuNPi0tx+sCc47i/OaK0dqAAlKxuOD84byiqmjwYnp+B7NQkHKptifsaMlOS8PZtk7GnsgmtPiWre7i2BalJzpDyC5cjNMPsNETFqe7QBYHhapGTnA54/QFc99LKhE31e+bzUvzp811IdTtx6fjChJyTiNpH1IBZCOEE8CyAmQDKAHwthFgopdxmOOwWADVSyuFCiHkAHgPwTXXfbinl6Qm+biIiCsMhlIyy2+nAkLx0PPXN4/sV7LAEn8ImJRspSzusTwbuLh4IIP5x20bFRbkoLsrFv9cfAgCU1bbYlngkOUVI3bUxS55uyUinRAiY3WrAHGndo5TS9nsSjjYRsb41vjHdRNTxYinJmAhgl5Ryj5TSA2ABgLmWY+YCeFX9+m0AM0Q8v0WIiChhtLKEWBbtnUzG0gNrKQcAjOufjSevHR/z+bJSlYB39d5qNNm0ilNKMsIHzNaSDLuMufFcXr+EO0IGvZW9mIl6jFgC5v4ADhpul6nbbI+RUvoA1AHQOuwPEUKsF0J8IYQ41+4BhBC3CiHWCCHWVFZWxvUEiIjIbJY6+jpSucSJeOXmYiz5+QW4aHQBnrzWnL3+7jlD9DHXPkPA/NvLRmNYfrrp2Pd/PBVXTojeck5jrINusskMJzkdIRlfh+G2NWgf2TcT3z93CF7/3tm25/IHZMTvYbzT/vSJjXEPOSeijnayF/2VAxgkpTwDwE8BvC6EyLIeJKV8UUpZLKUszs/PP8mXRETUvT1xzWn47/zpMbV8Ox7TTy3AkLx0vHhjsall3O5HLsG9l45GfmYyAHMruz6ZKfj91adFPO+EQZHrrI2t5x68fGzI/iSbrh7GLHvv9GTz8U6BX88ZjSnD80LPpQbKxvj7hkmDTcfE24tZQoack4i6hlgW/R0CMNBwe4C6ze6YMiGEC0A2gCqp9DFqAwAp5VohxG4ApwBYc6IXTkRE9pJdTvTPSW33x9XKH7SFdtaR1I4okeLbt01BpP4RxgyzNXgFlODY2j7PuOivd4Y5wxxpkqE2AMV4OmtJB3sxE/UcsaQfvgYwQggxRAjhBjAPwELLMQsB3KR+fTWAz6WUUgiRry4ahBBiKIARAPYk5tKJiKi9rP/NzJiP1RYLWgPmaKO2HQ4R8Rithjkcl9Oht8/TAl7j+fIsAXOkpTZahrmuJbhAL80y+CTuDLNekkFEXU3UgFmtSb4DwCdQWsS9JaXcKoR4QAhxmXrYywB6CyF2QSm9mK9uPw/AJiHEBiiLAW+TUlYn+kkQEdHJcf9lY/DmrZPQy2bRnubmKUUozE7Rb1+iDhs5x1LqEC3DHI22SG/eWQNt9yU5hF72oC3WMwbM1pKMSNejtawz1mEbx2wDx1HDrP7rZR9moi4npj7MUsoPAXxo2Xav4etWANfY3O8dAO+c4DUSEVEHuWlKUdRj7rtsDO67bIx++6yiXOx7dE7IcdEyzNEIIVDy4GzbhXg5aUlqSYZyO8nlADx+U1BsbW0XronIsPx0uG12Wvs+t0TJMAcCEqfc8xF+e9kYUwlJG0s5iLocTvojIqJ2MTA37YTPkZLktA28s1KSkOR06Flcu3Zw1r7SdhnmTfddhI/uOs92LLg1iI5WktHi9cMXkHjkg+0AgiUZHj/b0RF1NQyYiYioXWQku2wzz4lwwan5mDS0tx6Uah1CrMUPb982GU9co/R+tgbQAJCZ7ILb5cDZQ3Nx6fhC3DJ1CIaq7fCsme1oJRnaokAtwNfKRRI1OZCI2k9MJRlERESJ8tz1E9AnMzn6gXG4++JRAID572wCAPRKc6OspgXVTR7TccVFufpiP7sKEW3fgF5p+PN1ZwAAtpfXY09lk03AbM4w765sxO8/LsHT885ASpJTL9nQAmavXwmY2xgwE3U5DJiJiKhdaYsCTwYtwzyybyY2H6rD7srGkGO0aYCx1lRriw2tkxO1gLm8rgUXP70ctc1KR40dRxqwsaxWzyRrj+fx+dV/GTATdTUMmImIqNvQyh5GFmQqt20aUmiBsrGtnNvlCBvIprhDR2gLoSz621XRiM9LjurBMgD85r0t2FRWh75ZKabH0zLLPSFg3nKoDiMKMpDsCj9+nKgrYcBMRETdhhYgWztiGFkHrADA17+6MOxiPC3D3OYN7k9LcuL9TYfxzJJdIaUdm8rqAABH6lsBBDPM2v3bfH4s21mJ3hlujCnMRndzqLYF3/jzl7j+7EF4+IpxHX05RAnBgJmIiLoNvcWxAN790RTbwFkLmI2BbnZa+ABbC5hbfcGa5YvH9cPba8vMjxmG06llmJX7r9xTjbfWlGFkQSY++cl5ke/cBVU1tgEIfnAg6g7YJYOIiLqsXMtAFa0kQwCYMKgXhuVnhNxHD5hjrGHWBpYYF/ndb+g7HY3Wok4ryThQ3QxACcB3VTTEfJ6uQlvcaK35JurKmGEmIqIua8nPLzANAvn5RSNR1+zFxREWFjr1LhmxBXTXTRyEN78+iDnj+uHKM/rD4RBIT479f5/WGmbN/qpmXPjkMjx57XhcOWEAPL4A/r5iL26eMkRvi9eRPtxcjlH9sjAkLz2u+2mvh92AGaKuij/NRETUZWWnJqFPVnAsd2FOKl6++SxkRAhonXF2yRian4FN983CwNw09MlKQV6G0hLvv/Onx3R/vYbZZz/o5Ot9NQCAV1fswyMfluCfK/fbHucPSCxYfQCNbfGN5D5eP3ptHS5+epl+u8bSoi8c7fqSmGHuMAeqmvHaKvufIzo+DJiJiKhHCXbJOLHz9M9Jxal9M2N+POOiQaOD1c046+HP8P6mwwCAB/+zDSv3VIUct/VwHea/uxlXP7/iBK46Nq1qlrhVveYVu4/hjAcX4fOSo1HvqwXMdtMSqX3Me/Er/Pr/tuivI504/jQTEVGP4nLEV5IRSaRMtsbpEGj1+kOGqGi+3leNyoY20yK5G19eHXJcY6sSiJYcOfl1zw2t5iz2+gO1AILZ8Eia9AwzQ4yOckz9WeOQnMThTzMREfUoDpu2cscrllpmf0Bix5EG+MK007ALajz+AI7UtZq2GRcdnoxezlIqJR9H6lpR1dQWsg9QFlNG06AGzG4XSzI6ivadb2OGOWG46I+IiHoUl01bueOVZjPUxMrjC2DLYSV7fMagHD1bG82k3y3Gvkfn6LebPMGsb12LF/kJHi++42gD5r+7GUW907Cvqtm0z24ATDhahjme+1BiaZ8FW8OUAVH8mGEmIqIexRFnW7lItBrRB+aOMQW3APROF9VNHqzbX4uMZBdO6RO95tkoYMhKGzPMdS2RF+CVHKnH6r3VcT3W2v1KucXBmpawxxiT8nuPNeHc33+OinpzJlwrHTmRcoD3NhxC0fwPQs5NsRFqjrk1zEJTih8DZiIi6lESWcOs1foWGDp1aL41cRAAoKrJg3fWlWFAr1SkxpCRNtpX1aR/3WTojmEcxW1n9h+X49oXvorrsbSAeUSf0N7VwXkwyvfsg03lmPbEUhysbsGHm8tNx2olGeG6gsTi9VUHAAC7KhqP+xw9WTDDzIA5URgwExFRj+JyOOB0CH0gyYnQAua+NgHzFWf0x3Vq0AwAA3qlhpRwZKVErozcXh5c4GfMMEcLmI+HVjNd1xJ67oClvuL219fpXz/+yQ7sMCxE1AL7EykH0B4vEX8F6In0GmYu+ksYBsxERNSjuF0OvPqdibjmzIEnfK5MNeDtneEO2ZeS5ITb0Is4KyUpJGDOi1KHXFbTjOWllZBSmgNmm6BWYwx441kcqHXxsOvmoQW/3kDAVCYCAE0ePz4wZJmr9Q4Nx5/d9AdiX2RIoYSaYmaGOXEYMBMRUY8zdUQeeqWHBrnxeuZbE/DwFWMxoFcaAOClG4v1fckuh6m1WqvPjzS3OaOsDUEJ53cfleCGl1fj85IKNHt8+kLF2uZgUFvb7MEVz/0XH285AgAoPRrM9oZrZQcAxxrb4PUHA+qqMK3I/AEZ7Mvs8WPPsdAyicqGYK1xuZqpDtd3OhbqdG202nUQ8QViHqLSU3HRX+IxYCYiIjpOfbNTcP3Zg/XbM0cX6EFtSpLTFJBeNr5/SIZZ63Thdjmw9p4Lwz7O31fswz++2o/s1CQ4hLkk49klu7D+QC3eXnsQALDnWLDu+a4F61HT5MG6AzV4cdlu+NTr8Qckih/6DBMf/gzLSyvxxCc7UNlgbiWnOVTTopdZtHj92HHELmBuw6JtR3Hve1tQUa+c50TKAbQ2di0ec4Z0/YEaXPX8Cpzx4KITymB3Nkt3VKBo/gfYeTQxPba1zDwzzInDtnJEREQnQbLLoZdRPHLFOMwe2xfvbThkOibD7UJKkgMZyS70jpBtXl56DABQ0+xFXoYbVU0e7K5sxEeby/VhIi1qcHTA0BJu1d5q3PPvLXrJRH5mMq44Y4Ceea5p9uIGmyEpAHDVhAF4Z10Zznt8ib7trTVl2FZeH3JsRUMbvv+PNaZtxmCtor4VEx9ZjBdvOBMXjekb9nlqtJIM7Ryby+pQ3ezBTa8Er3Xlnmqcf0p+1HN1Be9vVF6f9QdqcEpBfJ1U7LAkI/EYMBMREZ0EKUlOPWDOUGudrSUZDgfQOz0ZLrXWeUCvVJSpbd36Zafo5Q1GeRnJWLqjAl/uqsTB6mALuH3HmrHuQA2eWbLLdPyGg8G+zx9tPoLUJBcG906LeO1upwMXjMzHO+vKQvZtORQaMFuz0wVZyaZyAO0aFnx9MO6AuabJg0uf+TLkmMXbj3bagHn9gRo0tPpwXozXpy1ydCZonLheksFFfwnDkgwiIqIE0jpjKBlmpZQhLUkpxQgddCKQl5mMdDWQ/uiuczEwNxUAMCjXPqjNz0xGeV2rKVgGgEO1LbjyuRUhxx+qDR736bajuO3/rcVeQ9mGnVS3ExeOKsD9l42x3T91eJ7ptjVgHpybbiqZ0D44xNqZRAsga5q9eHttaNAOxDamu6Nc8dwK3PiKfebejjYF0nWCXUGklKhv9eotEznpL3EYMBMRESXQA3PHYuv9s+BwCD1QTEtWAmVrH+bJw3pj5qg+mH5qHwBAZkoSJg/tDQCYMswclGq0hYLXnDnEmC6QAAAWmElEQVQAF45S7mcNtK4tHoD7Lh0dct8pw5Rzr9tvH2ye2lcpB0hzO5HqduKmKUXIs+kAMm+iucOIdex3HzXD/MlWZSHiEXUASYrL/Pz/vLgUk3+3GB5fADe9shqbypRMtHa6xz4uwcMfbre91pIj9ahvjb293ovLdmPt/viGuXy85Qh2V578XtD+gJIJPtHW4B9vOYLT7vvU0KmEGeZEYcBMRESUQE6HQHqykjGepAa/A9UuGlqGeUheOtbccyEuG1+IO6aPwM9njdTvf/9lY/Hqdyfiygn9TeedNDQXH9w5Felq8F2Ul47e6UrwPLowSz/uzVsn4aHLx+Gc4aEB968uGQUAWK+WSHzxiwuQl5GMs4fkYu/vLtHrZ42BfVZqkukcd80YgUvG9ov4PdD6Uj/56U4AQLma5W7zmwO4PyzaifK6Vmwvr8cXOyvxv+9sBgB9cWI4+ZnJkBL4dOvRiMdpKhva8MiHJbj5b1/HdLzmtv+3FjP+8EVc95HHMRPcp7YFOdEA15p1Zw1z4jBgJiIiOknunDECK+ZPx0C1vCJZzbD6AzJsS7lUtxPnn5KPAb1STdt/cuEpGFOYjeY2JQjKy3DrfaC1zDAAjOybCbfLERLoAsAgtXZ5/YEauJ0ODMpNw5p7LsSbP5gMIYLDXIylI1kpoQFztIEit08bjmSXA8nq+Q6rtdjh2sGV1ykBdbI6TrzZEznQO3d4Hobmp2PB6gP6trfXlmGfWmpyzV9WYPYfl6FR7e6xvLTS9rlEcrzBZn1rcCKjN0rgr9Fqtk+0hKLF6zPdjvYcXv5yL675S2gZD4ViwExERHSSOB0ChTnBwFcrnfAHomchhRDYcO9M3D5tGCYW5WLikFwAQH81kB6YmwanuliwX3bwMbLVQDnTZopgVkoSctKSEJBKllZYagBStVrrJFfINuWagtP3VsyfHvbae6W7cen4Qr22+WC10rlj6+F63PDyqpD+0KVHlbIHtxowG8eAa66aMACj+gUz6fPOGog1+2uw82gDWr1+/PxfG3HBE0vx1e4qfL2vBiVHGvTSk01ldQCAwpwUSCnR5vPrwXU4NYZe18Y68GguMHQVaWgNfR52tJKWlhMMmA9UN5tuv7R8r/6hwc6D/9nWqWvBOxMGzERERO2kf04qvlk8EC/eeGZMx+ekufGLWafirdsm68Htj6ePwCs3F2PKsDw9AHcYAl/tuNQkp+0ismH5Gcq/fTJC9qUkhdZaawsXf3/1adj7uzn6duMHATv5mck41tgGnz+g94aua/FieekxvL/xsOnYHWr/YW2hZJMlw/zyTcX4w7Xj8Z0pRQCURYFXThgAAPhkyxG8teagfux1L63Uv65qUgJ2bdBLXYsXT31WipH3fIwLnlga0ubPyBjUn/Po5/rXgYA0jQIHgB1HGnDXgvVoavOhxtAjuyHGGutgV5DwGenf/HsLLnl6ecTzWANmAFi5uyrq48czEbKnYsBMRETUThwOgceuPg1jCrOP+xxulwPTTy0AAOSrZR1ZqaHZZCEEslKT9NKPXHWy4ah+SvnG8PzQgFmbTGgsB9GypNYSkWjyM5Lh9UtsPVwPjy+A3obJii6nOZDXMszJLge2W/o8zz29EDNGKc/XafgAkJeRjKH56fjDop24972tttdQ1agEvdoo8ZpmL/60uFTfr2We7dQ0mYPdBasPQEqJF5btwaw/LsOWQ8H73vH6Ory34TCW7aw03aeuxYv3Nx6OGpBqpRNahvlQbQtW7zUvUPznyv22PbA1Xn8Ah2tbQxZp1kUYo66JNbBPhLpmL9YdCGa1l+6oCPm+dUYMmImIiLqob08ajAcvH4tvTxqMVb+agdW/nmHan5niwsDcVLxyczEW3nEOgGAdb7JNi7e9VUomePyAHH2b1oliQE7k3s1WfbKUQH3us/8FAFxdPEDfd6zBo2eugWCG2eOX2KwGsVqAn2NTi60VtJw+MCdkn0aI4LhvbTKitRTEGaYW+47X1+FHr601bZv/7mbc+95WLN1RAQDYWFYLnz+Af605qD/OBrXLx50zRgAAPthcjh+/sR6Pf1IS9joB6GUTWuB8zqOf49oXvrI99k+LS/XMuJQSy0srccPLq3Cwuhn+gMTpA3sBACYWKSU8xtKSioZWtHr9eGdtmWnAzcayWpQcCR+Mh/PBpnJc9NQXtiVGK/dU4YNN5aZtHl8A4x/4FFc+t0J/rjf/7eu4WvB1FA4uISIi6qJcTgdumKSM5i5QO1MYzRxVgD5ZyXpGGgBunFyEVXur9fsZDVDLLLR6aUApAfntwq3omx16/uW/nIb6Vi+2Hq7H7opGvLBsj75P6yPtEMCEQb1wy9QheOELZf+R+lY9+2u0bGcllu2sRF6GGwNz01Dd5NFrsoFgjbO2OPCi0QV4b8NhPWD7zTdG48H/bMP5p+RjW3k9nl+6G7PG9NVLMqyB3ZG6Vvz+4xL87KKRKK9rwf8s2IBHrzoN/7EEepp/rtyvf/3r/9uCJSWV+Gx7sFOH9vxGqYswtx1WgtCXlu/FTVOKMKBX8ENHyZF6/OTNjZg5usAUMBu7bPgDEk6HMC3ee3LRTgih1IO/umKf3llj3QElWJ85ug8+234UU4b3xvqDNahq8qDV68cv396EhRsPY0SfDJRWmFvlfffvypTGN74/CZPV1oNG+441oaHVh3EDzH8ZWbmnCjuPNqK22RMyqXLei0ppzJzTlDIeKSUOG2rBa5u96Jtt7UveeTFgJiIi6qbu+UZoL+a+2Sl454dTbI//ycxTcOWEASjKS9e33TSlCDeptcNWWvePMYXZ8PgCpoD5tAE5eP+OqTi1X6Ze6vH0vNNx14IN2Hq4DvctVMooslJcps4S2vly0pKw/kCtafvssX1x63lD8cPzh6m3+6HkwQK8sfoANhyoxS1Th6B/TgrOOyUfo+/9BADw07c26CUZVgvVWuqzinKxYvcxrNlfg++9am4999r3zsaSkgqsP1iLtZb+1cZg2egUNWA2Tll85vNdeOjysahu8qBPVgq+2FGJ7eX1qG/x6gF9qzdY7w0A9S1e9Ep364smNVICLxq+1wCwWc1uTx2Rj2W/mIbCnBS8vuoAKurb8OfPS/Xnag2Wja57aSW23j9Lb4uoueCJpQCAfY/OMW0/WKNcV02zF08vLsVl4wtRXJSLgOWDiZQSZz70GYblB3+u6lq8yM9MNh1jXYTamTBgJiIiIgDKor+RhhZ18dCyv0bWjOTc0/vj3+sPYcmOYM3q0l9Mw5KSCvx7wyEsLz0GABian45LxvXDexsOmxYXJjkdei9p47YbJxfhxsnK7dmWHtFJDgfqWryYcWofLC6pwCf/cx5m/XGZ6ZhHPyrRy0L2GUoV0txOnDM8T+9pXTT/AwDAX759Jv60uNS2pvjGyYNRmJ2K1CQnGlp9GJibiiF5GdhwsBbPLd2NJxftxODeadivPo6xA8f/rT9k6hBSqwbMS9QykEg2ltUhySnQNytFLzXJTXebxpvPOa0fPthUjoG5qSGTIjWX/vlLvHXbZL323bhA84NN5Xjqs52obvJg3W9m6mPcd1c24h9f7ceC1Qex8+GLsa8qGPS3ev041tiG6iaPqSSmttljKhdpbPMhM462f+2NNcxERESUEL+YNRJv3zY54jEjCoIB+bfOHoTcdDeuOnMA6g1Z4OF9MnBWUS6W/3IarikeaHeaqGaP6QsAOFzbAimBKcPzsO/RObYfCLRg2Rr097OUoWSomdepI/JCuoxMG5mPmaML8MDcsUh1O/HSjcVIdjkwfWQfnD4wByVHGvDkop04tW+mqVWf1afbjurdTRasPoD739+KRz4010BbR5MDSjZ7XP9sU1228etvFg/UB8pMH9kn7OPvOdaE11Yq/a1Ljzbgx2+s1/fd/vo67KpoRHWTB20+P8rUDLP2lwCPP4CvdlfhqueDvZ2veG4Fpj62BFZ7jzVh1Z7gwsZjNiU6nQkDZiIiIkqI26cNR3FRbsRjigcri9LyMpLxyBXj9O1awHTvN0bjurMGAVB7TUcZkhLOc9dPwJ3Th6NBzdgaFw+u/tUMXDq+MOQ+798x1XTbOi3xnR9OwV++PQEZyS7MHV+IMwbl4LnrJ+CN70/C374zES/dWKwfO3VEHkoenI3fXjoGF4/ti4xkF5wOgfkXn4obJpvrx91Oczh2odoV5IVle/C3/+4DAPxs5in6/r9/5yysvefCkOu/4gzzdEhtseOHd56Lx64+Te/NbZwMacflFJBS4v73t4U95tI/f6m3wfvLF7v17de9tBJOh8APzhsKACFdTzTz392M219fp98+1tgW8Zo6GksyiIiIqN1oAfWFo8xZzh9PH475727G9ZMGRZ0kGAuHQ2DuGf2xZn8NPL4Aiot66fv6ZKVg/IBsU7nBVRMGYGTfTOSmu1Hd5MGdM0bg9mnDTOcc2TdTz1BfOLoAF44uQCRCCAgBjOqXhZW/mgGvL4Be6W6s2WduGTdrbF/TtZwzIg8fbz0CAHjhhjNR2dCGqcPz8IdFyqhxl9MRssgOQMiHlWevn4B9x5r0APkH5w1DmtuJKycMwDtrDyE5yaGXwRj9ZeluPP7JDgDKQsrC7BT88LV1pmN2Hg1fCz2mMBvnjsg31bRHc6yBATMRERERAKWu9rOfnh/S13nexEGYN3FQQh9rWH4GXv/+JPt9lpIK7XpevOFM/OHTnbh92jB9lHkiZCS7ADXG1RYFThuZj1lj+uLS8YUoKa/XF+SdY+hUMUstLbEbK/6jC4ZhV0UjPt2mLD4cYlisCSht94yt91LdTtx6nvIh4K3bJkNKiSF3f4j+OammWuoGQx31tJH5pnKJBbdO0jtghDNzdAFy0oIZ/UeuGIdZYwpw5kOfhRzbKy0JYwqzkWEzmbIz6dxXR0RERN3OcJspg+1+DZbBLVoAXVyUizdutQ+yEyUrJQnv/mgKhvfJ0PtiL7xjKtp8frR6A6buEfp91JKSW9VSBwD45exTAQQXI6ZEqI22I4TAjodmY3t5Ay5X+2VbDe6dDmPTi3H97Yfu7Hr4Yuyrakaa24l+2Sk4XNdqOEeabUb8uomDcPclp+rfg86MATMRERH1ONbR3rPGRC6vSLQJg3qZbqe6naaR5HdOH45zT8nXbzsdAqUPX2w77hxA2O3RJLucGN0vC3NO64cfTx+OVXuqsWRHBZaqnUycDqFPlBxTmIX0ZJcpI31tsdKG0OV0mD4IGSc7aosnLxtfiPpWL2qavdh4sBY/PH9YlwiWAUAYG2R3BsXFxXLNmjUdfRlERETUzbV4/PpiM62ndFd0pK4VyS4HeqW7ox8co73HmuAQSoYZABZtO4qJRbnIVkstJj78GaqaPNj9yCVhz/Hh5nL886v9ePW7E00dSLTWedZ+zx1BCLFWSlkc9TgGzEREREQUjxaPHwEpO0XQeyJiDZi79rMkIiIionZnLB/pCdiHmYiIiIgogpgCZiHEbCHEDiHELiHEfJv9yUKIN9X9q4QQRYZ9d6vbdwghZiXu0omIiIiITr6oAbMQwgngWQAXAxgN4DohxGjLYbcAqJFSDgfwFIDH1PuOBjAPwBgAswE8p56PiIiIiKhLiCXDPBHALinlHimlB8ACAHMtx8wF8Kr69dsAZgghhLp9gZSyTUq5F8Au9XxERERERF1CLAFzfwAHDbfL1G22x0gpfQDqAPSO8b4QQtwqhFgjhFhTWVkZ+9UTEREREZ1knWLRn5TyRSllsZSyOD8/P/odiIiIiIjaSSwB8yEAAw23B6jbbI8RQrgAZAOoivG+RERERESdViwB89cARgghhggh3FAW8S20HLMQwE3q11cD+FwqE1EWApindtEYAmAEgNWJuXQiIiIiopMv6uASKaVPCHEHgE8AOAG8IqXcKoR4AMAaKeVCAC8D+KcQYheAaihBNdTj3gKwDYAPwO1SSv9Jei5ERERERAnH0dhERERE1CPFOhq70wXMQohKAPs76OHzABzroMem9sfXu2fh691z8LXuWfh69yyJfr0HSymjdpzodAFzRxJCrInlUwZ1D3y9exa+3j0HX+ueha93z9JRr3enaCtHRERERNRZMWAmIiIiIoqAAbPZix19AdSu+Hr3LHy9ew6+1j0LX++epUNeb9YwExERERFFwAwzEREREVEEDJiJiIiIiCJgwAxACDFbCLFDCLFLCDG/o6+HTpwQYqAQYokQYpsQYqsQ4i51e64QYpEQolT9t5e6XQgh/qT+DGwSQkzo2GdAx0MI4RRCrBdC/Ee9PUQIsUp9Xd8UQrjV7cnq7V3q/qKOvG6KnxAiRwjxthCiRAixXQgxme/v7ksI8RP1d/kWIcQbQogUvr+7DyHEK0KICiHEFsO2uN/PQoib1ONLhRA3JfIae3zALIRwAngWwMUARgO4TggxumOvihLAB+BnUsrRACYBuF19XecDWCylHAFgsXobUF7/Eep/twJ4vv0vmRLgLgDbDbcfA/CUlHI4gBoAt6jbbwFQo25/Sj2OupanAXwspTwVwHgorzvf392QEKI/gDsBFEspxwJwApgHvr+7k78DmG3ZFtf7WQiRC+C3AM4GMBHAb7UgOxF6fMAM5Zu6S0q5R0rpAbAAwNwOviY6QVLKcinlOvXrBij/M+0P5bV9VT3sVQCXq1/PBfAPqVgJIEcI0a+dL5tOgBBiAIA5AP6q3hYApgN4Wz3E+nprPwdvA5ihHk9dgBAiG8B5AF4GACmlR0pZC76/uzMXgFQhhAtAGoBy8P3dbUgplwGotmyO9/08C8AiKWW1lLIGwCKEBuHHjQGzEkQdNNwuU7dRN6H+Oe4MAKsAFEgpy9VdRwAUqF/z56Dr+yOAXwIIqLd7A6iVUvrU28bXVH+91f116vHUNQwBUAngb2oJzl+FEOng+7tbklIeAvAEgANQAuU6AGvB93d3F+/7+aS+zxkwU7cmhMgA8A6A/5FS1hv3SaWnIvsqdgNCiG8AqJBSru3oa6F24QIwAcDzUsozADQh+OdaAHx/dyfqn9XnQvmgVAggHQnMHFLn1xnezwyYgUMABhpuD1C3URcnhEiCEiy/JqV8V918VPtTrPpvhbqdPwdd2zkALhNC7INSVjUdSo1rjvonXMD8muqvt7o/G0BVe14wnZAyAGVSylXq7behBNB8f3dPFwLYK6WslFJ6AbwL5T3P93f3Fu/7+aS+zxkwA18DGKGutnVDWUiwsIOviU6QWq/2MoDtUsonDbsWAtBWzt4E4D3D9hvV1beTANQZ/hREnZyU8m4p5QApZRGU9/DnUsrrASwBcLV6mPX11n4OrlaPZzayi5BSHgFwUAgxUt00A8A28P3dXR0AMEkIkab+btdeb76/u7d438+fALhICNFL/avEReq2hOCkPwBCiEug1D86AbwipXy4gy+JTpAQYiqA5QA2I1jT+isodcxvARgEYD+Aa6WU1eov4Weg/JmvGcB3pJRr2v3C6YQJIS4A8HMp5TeEEEOhZJxzAawH8G0pZZsQIgXAP6HUtlcDmCel3NNR10zxE0KcDmWBpxvAHgDfgZIE4vu7GxJC3A/gm1A6IK0H8D0o9al8f3cDQog3AFwAIA/AUSjdLv6NON/PQojvQvl/PQA8LKX8W8KukQEzEREREVF4LMkgIiIiIoqAATMRERERUQQMmImIiIiIImDATEREREQUAQNmIiIiIqIIGDATEREREUXAgJmIiIiIKIL/D1A6vIOkczx3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x20df650dac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# remove noise from initial epochs\n",
    "loss_history_plot = list(filter(lambda x: x < 0.25, loss_history))\n",
    "\n",
    "plt.figure(figsize=(12, 5))\n",
    "plt.plot(loss_history_plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Restoring custom models\n",
    "\n",
    "Because of the added complexity of saving hybrid models, we need to have a slightly different method to load the weights of the custom model from the checkpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checkpoint path :  checkpoints_addition/addition_janet.ckpt-1\n",
      "Final average predicted error (should be less than 0.03) :  0.010060601\n"
     ]
    }
   ],
   "source": [
    "if os.path.exists(CHECKPOINTS_DIR):\n",
    "    ckpt_path = tf.train.latest_checkpoint(CHECKPOINTS_DIR)\n",
    "    print(\"Checkpoint path : \", ckpt_path)\n",
    "\n",
    "    model = EagerJANETModel(input_dim, NUM_UNITS, num_outputs=1, num_timesteps=TIME_STEPS, output_activation='linear')\n",
    "\n",
    "    model.compile(tf.train.AdamOptimizer(), loss='mse')\n",
    "    \n",
    "    # this is where you need to call the model at least once,\n",
    "    # so that all of its variables can be properly restored\n",
    "    zeros = tf.zeros((1, TIME_STEPS, 2))\n",
    "    model(zeros)\n",
    "    \n",
    "    # restore the weights\n",
    "    # see the difference as to how we pass the additional weight matrices to be loaded\n",
    "    tfe.Checkpoint(model=model).restore(ckpt_path)\n",
    "    \n",
    "    # predict one batch to ensure the weights are correctly loaded\n",
    "    generator = batch_generator()\n",
    "\n",
    "    losses = []\n",
    "    for i in range(20):\n",
    "        inputs, outputs = next(generator)\n",
    "\n",
    "        preds = model(inputs)\n",
    "        loss_val = loss(outputs, preds)\n",
    "\n",
    "        losses.append(loss_val.numpy())\n",
    "\n",
    "    print(\"Final average predicted error (should be less than 0.03) : \", np.mean(losses))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:Anaconda3]",
   "language": "python",
   "name": "conda-env-Anaconda3-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}