q-learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Value Iteration / Q Iteration\n",
    "\n",
    "Background: \n",
    "\n",
    "1) Using the Bellman equation the value function can be defined as the maximum Q function over all actions\n",
    "  $$\n",
    "  V(s) = \\max_{a \\in A} Q(s,a)\n",
    "  $$\n",
    "\n",
    "2) Q function is defined recursively as the sum of the reward and the discounted value of the next state\n",
    "  $$\n",
    "  Q(s,a) = r(s,a) + \\gamma \\max_{a' \\in A} Q(s',a')\n",
    "  $$\n",
    "\n",
    "3) **Value iteration**: an algorithm that iteratively updates the Q function until convergence. It's a programming algorithm that uses the Bellman equation to update the value function.\n",
    "  General algorithm:\n",
    "   - Initialize the value function $V(s)$ for all states to some value (usually 0)\n",
    "   - Iterate over all states s and update the value function using the Bellman update\n",
    "        $$\n",
    "        V_s \\leftarrow \\max_a \\sum_{s'} p_{a,s \\rightarrow s'} (r_{s,a,s'} + \\gamma V_{s'})\n",
    "        $$\n",
    "    - Repeat until convergence\n",
    "\n",
    "4) **Q iteration**: this is essentially the same as value iteration but using the Q function instead of the value function in the update step.\n",
    "    \n",
    "    Plug in \n",
    "    $$\n",
    "    V(s) = \\max_{a \\in A} Q(s,a)\n",
    "    $$\n",
    "    gives the update function\n",
    "    $$\n",
    "    V_s \\leftarrow \\max_a \\sum_{s'} p_{a,s \\rightarrow s'} (r_{s,a,s'} + \\gamma\\max_{a' \\in A} Q(s',a'))\n",
    "    $$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Q-Learning\n",
    "\n",
    "Value and Q iteration assume that the MDP is fully known (including the transition probabilities). We don't know these probabilities in the environment so we can't use these algorithms directly. But we can sample the environment and estimate the Q function from data.\n",
    "\n",
    "General algo:\n",
    "1) Start with an empty mapping of states to action values (Q functions)\n",
    "2) Interact with the environment to collect (s, a, r, s') tuples (state, action, reward, next state). We use epsilon-greedy policy to select actions (for exploration/exploitation tradeoff)\n",
    "3) Update the Q function using the Bellman approximation. Same update function as above BUT we blend the current estimate with the new estimate using a learning rate alpha.\n",
    "    $$\n",
    "    Q(s,a) \\leftarrow (1 - \\alpha) Q(s,a) + \\alpha (r + \\gamma \\max_{a'} Q(s',a'))\n",
    "    $$ \n",
    "4) Repeat until convergence\n",
    "\n",
    "Notes:\n",
    "- We need the epsilon-greedy policy to ensure that all states are visited (http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/ProofQlearning.pdf)\n",
    "- Can we implement optimistic & incremental Q learning? https://proceedings.neurips.cc/paper_files/paper/2001/file/6f2688a5fce7d48c8d19762b88c32c3b-Paper.pdf\n",
    "- According to notes23.pdf alpha needs to trend to 0 for convergence. Not sure why, it is also mentioned that in practice a small constant alpha works well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Step 21, solve rate 0 -> 0.01\n",
      "Step 22, solve rate 0.01 -> 0.04\n",
      "Step 33, solve rate 0.04 -> 0.05\n",
      "Step 40, solve rate 0.05 -> 0.06\n",
      "Step 51, solve rate 0.06 -> 0.17\n",
      "Step 145, solve rate 0.17 -> 0.2\n",
      "Step 147, solve rate 0.2 -> 0.26\n",
      "Step 268, solve rate 0.26 -> 0.28\n",
      "Step 328, solve rate 0.28 -> 0.34\n",
      "Step 509, solve rate 0.34 -> 0.36\n",
      "Step 514, solve rate 0.36 -> 0.41\n",
      "Step 1169, solve rate 0.41 -> 0.47\n",
      "Step 1557, solve rate 0.47 -> 0.56\n",
      "Step 1558, solve rate 0.56 -> 0.58\n",
      "Step 1562, solve rate 0.58 -> 0.63\n",
      "Step 1626, solve rate 0.63 -> 0.65\n",
      "Step 1627, solve rate 0.65 -> 0.68\n",
      "Step 1828, solve rate 0.68 -> 0.71\n",
      "Step 1830, solve rate 0.71 -> 0.74\n",
      "Step 1843, solve rate 0.74 -> 0.76\n",
      "Step 2299, solve rate 0.76 -> 0.81\n",
      "Solved in 2300 steps\n",
      "Playing final policy 0.707\n"
     ]
    }
   ],
   "source": [
    "import gymnasium as gym\n",
    "from random import random\n",
    "import numpy as np\n",
    "import typing as tt\n",
    "\n",
    "\n",
    "State = int\n",
    "Action = int\n",
    "ValuesKey = tt.Tuple[State, Action]\n",
    "\n",
    "# Value table for Q(s,a)\n",
    "q_values: tt.Dict[ValuesKey, float] = dict()\n",
    "\n",
    "GAMMA = 0.9\n",
    "ALPHA = 0.2\n",
    "EPSILON = 0.05\n",
    "\n",
    "\n",
    "ENV = \"FrozenLake-v1\"\n",
    "\n",
    "env = gym.make(ENV)\n",
    "\n",
    "def select_best_action(state: State):\n",
    "    best_action = None\n",
    "    best_value = None\n",
    "    for action in range(env.action_space.n):\n",
    "        value = q_values[(state, action)]\n",
    "        if best_value is None or value > best_value:\n",
    "            best_value = value\n",
    "            best_action = action\n",
    "    return best_action\n",
    "\n",
    "def play_episode(env: gym.Env):\n",
    "    state, _ = env.reset()\n",
    "    done = False\n",
    "    while not done:\n",
    "        if random() < EPSILON:\n",
    "            action = env.action_space.sample()\n",
    "        else:\n",
    "            action = select_best_action(state)\n",
    "        action = env.action_space.sample()\n",
    "        new_state, reward, terminated, truncated, _ = env.step(action)\n",
    "        yield state, action, reward, new_state\n",
    "        state = new_state\n",
    "        done = terminated or truncated\n",
    "\n",
    "#                                                       s      a       r      s'\n",
    "def value_iteration(sample_tuples: tt.Iterable[tt.Tuple[State, Action, float, State]]):\n",
    "    for state, action, reward, new_state in sample_tuples:\n",
    "        q_values[(state, action)] = (1 - ALPHA) * q_values[(state, action)] + ALPHA * (reward + GAMMA * max(q_values[(new_state, a_prime)] for a_prime in range(env.action_space.n)))\n",
    "    pass\n",
    "\n",
    "eval_env = gym.make(ENV)\n",
    "# Returns the fraction of episodes that were successful\n",
    "def evaluate_policy(num_episodes = 10):\n",
    "    def play_once():\n",
    "        state, _ = eval_env.reset()\n",
    "        total_reward = 0\n",
    "        done = False\n",
    "        while not done:\n",
    "            new_state, reward, terminated, truncated, _ = eval_env.step(select_best_action(state))\n",
    "            total_reward += reward\n",
    "            state = new_state\n",
    "            done = terminated or truncated\n",
    "        return total_reward\n",
    "    \n",
    "    episode_rewards = [\n",
    "        play_once() for _ in range(num_episodes)\n",
    "    ]\n",
    "    return len([r for r in episode_rewards if r > 0]) / len(episode_rewards)\n",
    "\n",
    "def train():\n",
    "    step = 0\n",
    "    solve_rate = 0\n",
    "\n",
    "    # Initialize the Q-values to 0\n",
    "    for state in range(env.observation_space.n):\n",
    "        for action in range(env.action_space.n):\n",
    "            q_values[(state, action)] = 0\n",
    "\n",
    "    while solve_rate < 0.8:\n",
    "        samples = list(play_episode(env))\n",
    "        value_iteration(samples)\n",
    "        new_solve_rate = evaluate_policy(100)\n",
    "        if new_solve_rate > solve_rate:\n",
    "            print(f\"Step {step}, solve rate {solve_rate} -> {new_solve_rate}\")\n",
    "            solve_rate = new_solve_rate\n",
    "        step += 1\n",
    "    \n",
    "    print(\"Solved in\", step, \"steps\")\n",
    "    print(\"Playing final policy\", evaluate_policy(1000))\n",
    "\n",
    "train()\n",
    "\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<video alt=\"test\" controls><source src=\"data:video/mp4;base64,AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1wNDEAAAAIZnJlZQAAUoFtZGF0AAACrgYF//+q3EXpvebZSLeWLNgg2SPu73gyNjQgLSBjb3JlIDE2NCByMzE5MSA0NjEzYWMzIC0gSC4yNjQvTVBFRy00IEFWQyBjb2RlYyAtIENvcHlsZWZ0IDIwMDMtMjAyNCAtIGh0dHA6Ly93d3cudmlkZW9sYW4ub3JnL3gyNjQuaHRtbCAtIG9wdGlvbnM6IGNhYmFjPTEgcmVmPTMgZGVibG9jaz0xOjA6MCBhbmFseXNlPTB4MzoweDExMyBtZT1oZXggc3VibWU9NyBwc3k9MSBwc3lfcmQ9MS4wMDowLjAwIG1peGVkX3JlZj0xIG1lX3JhbmdlPTE2IGNocm9tYV9tZT0xIHRyZWxsaXM9MSA4eDhkY3Q9MSBjcW09MCBkZWFkem9uZT0yMSwxMSBmYXN0X3Bza2lwPTEgY2hyb21hX3FwX29mZnNldD0tMiB0aHJlYWRzPTggbG9va2FoZWFkX3RocmVhZHM9MSBzbGljZWRfdGhyZWFkcz0wIG5yPTAgZGVjaW1hdGU9MSBpbnRlcmxhY2VkPTAgYmx1cmF5X2NvbXBhdD0wIGNvbnN0cmFpbmVkX2ludHJhPTAgYmZyYW1lcz0zIGJfcHlyYW1pZD0yIGJfYWRhcHQ9MSBiX2JpYXM9MCBkaXJlY3Q9MSB3ZWlnaHRiPTEgb3Blbl9nb3A9MCB3ZWlnaHRwPTIga2V5aW50PTI1MCBrZXlpbnRfbWluPTEwIHNjZW5lY3V0PTQwIGludHJhX3JlZnJlc2g9MCByY19sb29rYWhlYWQ9NDAgcmM9Y3JmIG1idHJlZT0xIGNyZj0yMy4wIHFjb21wPTAuNjAgcXBtaW49MCBxcG1heD02OSBxcHN0ZXA9NCBpcF9yYXRpbz0xLjQwIGFxPTE6MS4wMACAAABBo2WIhAD/cAtfc/9HPAshXfBMaKoT7kh8WOtSm+ae1vltLxypRQ7EIea6TLV6IQ/vCDgRcY6eAjhGmWh5MiMZSIpBsTSay18XE3eeVYF+ToNGVE9Gz7qgleS31Q48XbFch4YdSwhxJWXLmfQobhjuKgshbTSsvEJQFwKsACcHsuIaQb7D3lMdTcPwMJdCnjVAmoabaofvSlg/i7g+8tqcbLwGewlVZ/RGQRI0ZOj14g346IzZQyFPV1CFWNSQGxx6oA4B9PgYW4j5gehV2EbQmBcI0/QsA5IwQCpDR4SbFZHt1Pjy/NkCMhNuCyk01OsmAM/Uu5vEtfgH7yFVO9l9N6H9LGhGYPQZALEW7y9DiUUsm5usneMT4ZWmLlgUd6Srk37uiBUs/sI0OEBGYBfdkwe6uJsYd6l1zUesy4KWtlMS5K7LOxpl8YrWLGHIveKX2pJZRpJ1+8p7M6AAgn5dXOYsg8UKP2d4Tz2B1iYbKLg/qq5CkwN2jloU546r7yCchYW61UsNO1T3I6WaK7THoxx4TichCOxWH+pSk7gfO0UxmzuXHrPRpZCcH7/rnuFChNOKcKf6sk4FZJSy+PFhspBxU66BwoqArfG6O4cRFqzLsDnVu772uKymRKgbRSGG8x9E2ck/kGXcSyjB1XjvUdZr8zW0MY5rS/ALETWuO5CckIoHXve4qBadc9vrMWZO19IIKmnkN3NLdXkUSI6Jj1esG7PS9ecxdYIBwttoHgtvq5zX8mGolBRzhQGjrJ/MPZwgTIhs4JYF+evE/B59Owtw511E0VrTVETcnBjLZ+ShG86sd8odvFpP32WK1J8zlRVSlKk+hA4R/r3FwIeY0LBuynjHdEiT3Zhmngs/x8UcGPLO3DrLiFWnVpD78Wk40pIlM21E5RA88gcyIGqCqHSeCrIsRrGwDoCD6RHehlmTjdSw3CAetpnJWIv/AxbpxRgpD4mI0mAlWA9E+qDozttCCHtbvFokqYdMA1pIM1wA5f2Q3tkdG7/HaoSyIMKUvnGsTdZCMoH0MyaSbpd1VyDV+jUFoJkkTEXHE8r+fi7fc5WyM3HgrrfVjSGZ3F8h3I7gJPiyxpHV9IUxTDJ0S0/d5h0a7I4RzauGdzA4sL5/G4OQhcn4VU8i1+RhnRkraXI+KCtR88iNOm11BYDd+0Y77CMPfZKJH/YQp6KGl7pxraK04YTQzGc2ywJh75pFSVjMdi6YBRqFX5fO0SVB0VYhIrWaqbfiv8+eZL2nJ4Xn2FId4EfGtoHVj+8R9QuN5Usj1R8F6N2cXFvU/a8NBOsBosAHeI6vezuUy4vqM94e74kVv85GsL6k1UIgQmuhcVsZQAd+cKO2WmashdV8KEFtPO+iSjtA061J2ni46euPbhtwhlzHiDZ37xkKqyXmPZH5nBNseUAwTvBvYShjVt61STGxQYGS2alQ0E1DxXH/thaudMz/SgdPu/ZpKLmoUU+4OqIY9IVZPYuUrqzBhd+cL2mlnxvlycPyL0MvmZr8V8+gGNiD3z7KFlw0X/9jY90a+bsgbEx29uGw6Mg0bhr5IGhHLffg4ecMlMGT1qy2YnPgQXtwiNbZmQs/wLvHM2E/qMnJBidDOGhxmUcgONQrldT9LDaZwI6LgyCmOU847Muwf1I5nkFLzFHqaRvPkRmxRfob+9xtFN8GvRjNYGuyt7xN+52XOjeXt+bxKV/73+9C8gr2mXeTZKuMwdLrHLLZUlUBg5TGvDwomoSmYZ6epJoKXl66ZkQkFrqQECWZBeohOnHq0xm3o+D5jjRd4qNhs0QCLurgxv2gcabQPsL8PsF/oJbh8glWhLAhQNG1eSO2v9BbcngNEp83nRGp5WPW/aXbIwEQ8K31uTaHnUSK0+6melT7yz8U6dokagJ/onM/g1lOnaoEBgVpqjtCBlg7yIGltFdrlxn0eCso+V/QsjX9+1PvLVrHge5BtrXNM1nLKCs38Tm1XUfnGxsIHHcwEijGzPtsfcZpMP8prWqvs2gfBHj1nK9SdB6onalJ0AULHLbb1zrm9OrxM+WWC6YNi6/QjFcjmPMLGRcI0oglYTVJBgWuN7vC0vpF62zRugg/sAABTzll/T+GgPdQjmPIBkTVg6eFMN60DSSug3GtSNBOt1tw2IBV7otFHi4AjXURJ/d0kuClHeJImU7nkbdAlGRKgFdZX8RiZHCbirINHDXITmJFv/Sj845mAAd+em218K5Trjl7de0CxSFnY4OK4RyNJ5sa/Vj7IHKgnfHhFxXZXoY1Dtk7jQ8vQaEv6aQ/GOhIUKLYY9cz5sWkcGRHls8IaoSafPtW2W0+tUp/2DlrCWbbIyC/KpBeShpxG6gx9A2/9nOnQAqPx5AKf98Jg7Lqby+GYmwt/Gh/nsc0Z9n1V2lln+Uij87LeEKRTaS6xS9JCdzAQlrUhe0EEdHIrkAPlMYNLz6mJ17p61hYMBNOnMIwP2D4tWW4KDJzrMcGQDeEuWBUdk6n9irkNuzA2xIgeVcUm2vDVdhI72djovNptnzBTIuh4NxgBNBp6vEhspNOynv4b0BlsJNkLPGYve+ahLkHY2t/8+aiWUKGFtEpTlPpAEEdKT8vS8dA0Q89WLMuJ70phcBBRcZ1ck4CPhuM24UC80MYqhY2ka1/oe69jD2LK97NgXZLwHt3UeK1arjFmrPj2tCsNfB2pekhzperSLx5a7iPcR9TAyCMsCNJKTLgR0PEDyr8LU6jH+gvRwhDd2u0hBReQKMTi8fWopM7Q1CZ5rnHYSLnYlWqWqnM8ww3yrvFpC/6dgppa1hHZ54y+sktXa8m2+QR6J9HeXDZwA6t8RM542VvZF9URkSoEK96qx850m78P4pjG6k6evoqe2qIy2QkwgwOJv3xJp9bxOM5UMPgiXbC1cDKwUDiDi6m9kAHcWxaY2z08jJYR6MDmfToCY6ogggqCrW8xLIMwIW3TVJk79K9HIknN1BddXKQ2SYlN1jcCo3Ouvq8ajU+P/hwzPK1XYWPI/DxD0sYT23xtnyYUrIjZYWGcO3j8TatukNn23xCVapArsXBPvzbEOEL90RciWzSY6zs37zzwTlQCY2qwGo/9gTDtgGDdm+gSygWtUozxR5vnUiDS+Mc1nvtMvSQGpWJseauFDCfdcOspavjS+iw4qVcYs3UtuCpVO+Eg7tV7pnKFFDi+ucJTKY2dEIWv33oJIa0RSpKVH2fqyHrO+8YV7v34XU2tk1m44q0O0wT0UJoYyCL1AedOXJTvtT29A9OmQ49rXca0doaTBW5nsh69AylBWrsUN/ixIBCWKsPWof7KmdbpVb6xaOuTQZcCl6YgoQtkCzv3mdy/sDgnws4FHZJLdzhLpXNWV3t6kz2KsSUH/YF02vMySQfezpxn0s75esNqk7p0XIq8wH/nL62GveE/kl9+LGQ4arQGTWUUZYzPuTfBP+VC23QDvGIYgY4+eDiVIHReBB5HNlFNdzijFEufnjGngpyTNb+yn5SZitastYJjdfb1FtBh62LBcJtipky6GZ4YqHLCZj2hYuDZEfckxy3UA9NdbG/1N47RPZabQ2ax4ctQWcKj6L51XRNcuydGlFN6HmDJdFa7LQvnTES2JCryi+JZQw9fDlV+nKr0oSXc4Stflb0pdAb8yG9Z+Ff8DJ0yu1xD33l9t/qmAIXSZsKzMessq3bctkfe5rW93nTKERcTwTwCcOaCofM4f6DxqYw3hR7UYdlRJuEgLOCrg25TUMHAIOp/A1NWbte6UUiIUMBpqUd4LLczfSPLZyLat1Xv6yoLNpLOhl9kUIJ8FT+hyciu1xGBLIJngdqmLUL/Hr+kl0uJC1t+uGzdgx4GTnLxBs2bQYJPnm0LYz6YmWP8PZVmd6J8zoksYMhhPmlTAR+YOnh8utX+ip92Ped10O5lAz9FDXSm15XDoSF0yj3k7mSWC6MTT5PsAERzLzqESxGEqlckIfazym/hYyPoc0u3NXz+ub03MJJs2xBjEqGqNwOmYFPRtBy417skA5izSVOnYjdAChc6bjnatZ8uueGvyQOXGy9imNCg4PMFssaOYbU3HlLnTxrQNT3WszHBI6HgIzTox1YbhS28JAw45bTXIW8nIBO8mNA4WeCUx6wWI80qgW3MYi/3sxCDejqH6UZOJi1+QWAn+bqaSi0Mg3bDKzu12vwc/kutG00ZmjAQye/Gk3KBLhcHD4Q3f5tLaKzXO40uNW+RIoxWB3m8ASDZuB0UNEkBk+ULEYbYMPuE2snIhv1LaOJHIDJIBtBb4afL6HGj/vtqvbvQHU4RCH0yEBODX7nAg2EVIv70p9/N/bzl1X4HZaMYV/K+AakXf+juD3wb9PsAxDGJ8qzSpFxWx+LXKaibS4L5JYm48bBQIr2SihZ1UpKDnoK7JA78coeIg3QSOfqXLpye9lkw3BP/baF/c2QlHmCEhHskMjml5qCSIZdV/oRCxhaI9d14elElGjz9twHerQAS1/mikEPLcNmI5OhEsXhsePDhdYxcJeDeTNu6RkjbTyyG4WhBTSRhxW2Lwe12be/At/TlFLyTJ+lXEKcCXM1gXHgBZgCJhgS3z1UCTo2VFEUV/IrmSePUtc114h8QsZ93+TLgVBMyB1XPKWIVz2mraJtZns4BoMclxDmUsjQ+BpsA99BpUZkLa47T0lVMPLYBYI9kZBJ/oWj5+nQs28H19uH2WSwRE7W1Yc77NkSYdS6fHJWil9vhTF0ZULOdkoEdU0CagQ/YEuRwUePNI0qo4kbpl7Er7SvvrWE2o+2HgWs3JM8yTUg+m8mKxgPud1edOk9LLZMrF1GEowzBahAzqmESQHpghPI30BiovIXTrz+EW9nXSxOcu0AIN/r64L+SwCbC8GuTbnPFuRWkRUiSn996969VGX3jDANKk5/6SOu2yCEKha9oZJ9koSsMgwv2GH5XrwwB6qTMS/uIFJTz3et8KbLFkSUtt9YJ3+7C2RE8n8dPQo3qihfKu6yaShWGfdFJkxlRWUOuheaLoGnFN3OEqxO2d/RDBeSc3s3DiYdViuMqxHM0YMT7Gb27F2VcUjftXAEFSmgrXM0PrxUYLAp50rJBIc6nczHjyNbH0U6R79xwvgQk/G/70LxouMiizWpgwtdeZCL8JyZEwZIIYoTpc5K0TQFErrjUo1dBSNeRkXSZ09ehXLEobhQhPjRdDvizzv6T3zS7Yvp673Lwin8zz7kMTWY4FScnbJTYBslMg2rcdBM5SXQpFaI34UazxeWn6TrL1oAQoxDaXZwOyKmwFIMKRs7aDT+UpjQSKKxY5mMASl2NSSWvOf6L4cCSjMsjPE4QpANlYmY3lHiEtozISWCALzW1ppOB8uOP1jNvf8OvSIWVslXfHJukXw5KWyMAFo3CO4rSsCVAXXFNkF4B+tIh1NkLev9CMrLQ2GP2T7utAy2HUl6Xrt9/8o2QyBUVP0deA0AbR79YT1y07FY75c2utmOonHewM2hN2QF/oXTdVMFSbUU4GlkhHCfcqopiicGF+pdD5iIPon2PuRDPf87ISei74lvslKQJhedJP6jcFON5kubcYJzA2Qch5ipnZwbw40ekgb6jyyAXcbL82FoFyUyV806Nn2CArALr+xNV014DC+K6rbA1mM9mXfWLD5cit26fkW1cMYM+8PjMDQMFXN9cVXvofl/Irei2qW/8WywC2R+RstnG4hQ/0YZahgPS4N++Y83yg0hOzlXA2IkNqklRLPNS3Yrt0hTLSA/64oxNQEaoT1pDvKeAI63AJpmKWAZiOQkPTLBpE0uO+iXKRsJ7Z0DJNtdkrgNMSqyzwedENd7oqMPAJ3knxGHH11ULwDdmmAV3UTjX3q+KrnlWxpsahtiEnCfmP7Dp1MYHn2FGgWMSdUUTsk6KotvDzs+DZGClRjAzh2m7Rs9Qd8QfJQAgQnEgbpeQnFqSaFxOaShOTblElqdtlhwMJzLVsf3T0KNi/5o67nc2ybKLZlBdW/4P+KQN4p75Z+arvYNY4yRO04zmW4dWc9GoSYSzZ9icXZtH9j6gSwW5HvaQyRNXLVabwnMq8hp64Hf9GiQq6nIQxrtUEYaBR55gqI/iRf8m9Yq+2VUqcQk4Jnv3uJd8NEnmFt37YiE0yTyJ4gAZVbO8w9TdsLcZHhOu1CucjfxbaItcfTs1gq3vgQ4SkGn+ZkctxCa/+CHpB4joef7086gN5iFChBEORb5N79B4VAjOlwdVZUtvkoj8jsmFyOAwoXQyCmAAgfc3b3eyeeoJaGOojyBbIHJJm61DG+b49nbDwC40tjohgEYwUR9SHO7vpzNToh+hdYhAC07RKLMwZmsR/1HITgclTRTZ+1WIVDhxUdQXA2mt5y/KjUC85Bn0UdqmVgEtvDbJfGF+ZsBux4lM4A0gJbSOdtJE0pHkyQYKqQAo/BsRhAuKyt83MVzwE8r3WpTZz5dzdJMLIBKIYdsZcAU7fCHtL+ipQ97WsEdmAL5WuedyBDK9LqZy3qLw8inTSDSgNaDE7/2uGrZspFZx21rZV8UNgSBNifvc6DSASxUEL4oXFaJ6j5IjkPVC8DZGAH6rpdJdSR9nNuMJd1rfYVCAG4xtF5wwNSpVhUg5RFZDwHSBXc0c+rS3NZt5IUpigY+R28F9fS5iG6K3m2vjh4yD9knZJZ5oF91ISAsfaSyHJ8H1JeHXXluhGa+C7nybRMr9z2TiqYfVMsZikambYsLxcXjG4zfVTVEcMcq0BelDbhZ2f9WamDR3v7xqYsoofgZXzgPtaPXqA+VuO+7oW9U0D0nJ/VzIJnNGNlW0SZ5h7pyIJP5chCV+pc6QkRnaPJFkiY6rQfZ54lx5RlbLVp48B0Hxm4O4Uq5q62yr+WMt3XSHT/86iSBIzcIcy7tscO/Ycgr/vHO3wH/sjsCA0gbX4PMUBKXlbmF7FtshB5k1/hIoJUBVJ3jvA4CJItOgbRPbri91bReDI8Ta+wBLigNzxmx0vU4KwCVy3IDnO1Wa1htti3d84J99xUHd/UivPdk8lIpvQyfSFou+YzYT64SvfYgkmSjAaFmMGBDsdcTBGhiv47yEwTa1mZRSRQ5jaGcBnd+c1cxYr79FxwTXBVcao8ZDOK0gs5HNVRaAQyPzPJDnVfKPnYBK0D6JuesuV7C+RTc5rGoObh6bnqxhPfxRqTwGN3Q8c7068XR0EhDUKTNJIJknRGNBrHCIk9SfxJA8mb1FOwK5aeyy0qsRr/24M9tRjF51UCvP1dKhDB1H3uC++tJR8tYjF/tTd2STpOPsq+Sdf2B1YnBdCNzx2tFNxPo2BfURRgjmVdwuwHqCZ4BwpOe00xXGeEMTfmcaYug1XxqqBwYpjfd9KZeo7jRkDD8VXCDQSjJP6E23E/8iO1H2zcH39nLw8t0/mhkwom4ZiGmrvFajqFQON8PImSulJWgqlLIeZ+PiEA6PkJAffvVaFWELg0J1Cs9FBsoePbwtcGmhB29lT38UVOGGegHSLU9mHQwC/pKfUPCIrIJljOO1Mz4RjD//QXICRFxgx2IJzaC6ruD12sUsdc0Hz7Nzme+V4t0/HlR071zSEgLH2ksh0/0Zt+luvLSI6o1gQPGyAlN8M6bYZJIF3DLldxltkWN4R8xJKKPc0X5mfA/qUQESHbFuYHGC0h16Fup7A0gB8xv2m+DKvYsyKSSczeHQmyUiWxnpeCqNATN8TZEO5fRU8w0iAGeTPm+TlydjyWCw4Mq8non/W0p8IwrC/lA43pBTlVspS5L/kctA+SJCX9sjGtPsya2Z3KstQEm8aggAQSxmyMJbRfsXpns0Ytn/dk9G1z4bYcXGIbGFy9B51sAnpyOoY2zVSZnEzCeBCshts0uXGoUBCB+iSsVkv7XLMC6YrcthY3dUu3oiyeWQ7+yyJ7KqwsLzSPjUSNrGY74ajSineqTDX50SPz8m/c+JGP2DJlDjD+a41C8zYJ4f02Njgl1Wws+ZziWwXibQtciqkVbV2oCYMCR94v8h4OWwt48NTJcDoN7Rr1lER5favKwJh99i/FYpphLA+naiN7hBMfypQchpLIJoAfYdM7QwNezhqBzJc91GG/Ru1bR2X7saLRNUSIjOXSfv3FbF7CAlr/Fc6wSKflvUSNYZwzkPqjwrAXmav7cx5sMvuo8EHAXPxk8tR7rpEjvixbt0yawXG8LdNpfI4X/BxjMfaubxXvPw/OzS1KUj5T5K/SeMa1GoJt9TRe9dipO5jBPvn8b1v2kOAuEgqNwRvG2rhZlfhcOnCqOzmY5LaElYlJmxHrfzNWUGfURuL2eNa+aw0iqPL4074MocrXmBsu2RO5zfZtunKKa7PnweZz7Do3Mq4VexH+RMDpdccRNAbMVAZhEqushjXCR+lEqOK7VlCUEo0aFVduJljymLkKfFxKwhngWmSk+d5h7h6FEPeL1agQcQ1I3EhodYAFOUVmc91NDDLrx6D/Bm78h+KvQeCdMTlCypCdjT1lihPhe7d8eYBuqSgbdgPu5ayPjq6d0TwcCTqtBsbWjtpmsNEGJyqLdORTww5BFJN3gOKWDMEaXxyQ8aqYYEPNXrW17zhevzAGTguKhWMEI9ahBJ1CMGHD7iiuCAwClfbnW+nFL20E4UwhJ2A4k5F9Df+xyEZHgfT8ae48sCRGSUfSNrEuWICmUhkOTG8XeuYqFhbdDj/SybMlSp1B6PfQfiH+JClHHwG5tBuN/mTyIlEc2wQOxaV2AUw83f5VLVyY1fGB/YS6g3lfTeKW4BBIYS0CTTL3Xv7VN0coosd5hIhN8/w1rb6W05qGBuRQSOajkVcGLgczS//14+HJEaTdoS418Tf9kbxqG8VbEH1xnh5i3P7kcpO2mXngGTUmMxGHhdRenAT6StiEll575VYQ+vvMu2rDO3pMiMwx1DqDOJtiZo79QF4W94Khv0IUMyBnzccFciEOHgfBTIUyobLbVvR2UgxbXP4H77/6hwpzpwDM09DQmZMwwvTDEBvaNEIXr7Dt1R3bF/reFgCPlAOMJ2eG538qFljtihFLS1/SG3y1cJ681S36JvLQ3n1ge0hGc6/pEbvSD6VSq5q9H72PxDULqMWbYmGuvFxfawLfQWFk2gAe793sPx111xfeyBVVJc0p+CfZ0LK4Hd5T+aI06yYCV7IgjZOs47XR3pHJI+CKC+KfTYziczIBzSOaGw7qMBecwMdFoq49ALchrSURf7LGA9wAHvzc8X/O/PUuIOQXUkCuQnVTJX0HDCRituw8HvTxi9pWZOCWoEs+hjcx9Net/WyiM4nVij+WxojSMgG6rITzISnE4f3svN07orA4BjODXr1atJITdojl04v3j5f/qeeqZtSTMyvhQ8t7VPM9NVMX6OPdhBemTXBYCkbQ7GruqB3RZMoBn2FLmb6KqUYVgIngb94huPzIp2ukKn5ZQYGLPWIvlTRuxpRYA+PpqsOEqFDgvyCe/Fh6RvxmsC/C9SVBoztbOAHZNP42kqDADVoUjo3Y8tSlP6iODd1GFOLoa1i9MNfYMZMCBAqsqia3q/vMV1q4aryhqedVwoSMdZv8N2FzrdR6xHV04Aj/u55hN/EhU1HLeZdd9xcTVGmlWTiSSml4e5xF1OtrBAZL83waMy7rOY/hVRKQljmdUACtc0bEgE9pCGnu6wyl5C0udZsIP06QHu0Uq5GE5uafg9mqDPuNn6voW1FJB+mnpgHvHqecIfFR4xMf07FXsn3iF+vOIST21twphowMVFLZdAjQhZxFayAe2iBAHvmHsrYwN3esjMjrh2huQgRWOxowh13BHBFA3tmmqDWQbmDwJlu32Lgg0FUaSC3zl6jf1GI3ZFd2hnUTKCMTEm4WFHR6Y1Kcxypav54srTnglFW3sAVQ/nRzj/QDWlLItssRzSuR9nUvNdqi1gGof4tRxPHY7xtHpI8UFQpFDGyRgRvDkgVDzjtB5T3vUpxMcDpcl9mv0lhYhLSGi2ZE/70gjbuTwboMMAgntZeOOAifuTPJXQ6QGXmb3Cpe6pB0aSv+W4vK6HItCP1fsxQIy0YLCmGtnQGGdPLtjCZwtZMjTFUY8nW0jdKKtTSY7D3sJESz1DE66H0hP0oiQ6rI8f7rJUHbccZ7eYuKjvNu1eQfOHylpypz8wBbMTJ9T8VCgf6gRBNGZti87o6KsP7Ntb47MCCW+tFjgv4vFKDEdMLkEmjk5g1Om/4+4t64/X9qbHh3J9whj0kg4comBiNL4nQoaOjDWYTLV9o1xicEDVUdQ4AicoTONeKEo3QWBqvApGML8/NtwCMXcivuzWLBA5lGu7zEKxP1KhZusuF0ZtuELb124L9VSBTlcKHuRHIR00HY8Ay33apFUP6IQDEcMJttE+8s0tMoFdUGYPuXBxWAAI3TJC257Ekhn0PVodB2iIWkX9Pk4Pg5sK9vTYvmm+9job/rypfpv0hLICMxboRTyP+qsk3kdf9lV7/8saC9+gdSoaRcnXu+m7UxN/Ux/0C2YHUyO9SsxCrccYkAZ0+G2fIbY9OuRWJ9xsumWmRGMZvDxin5KmX0XQhWYrfIxYGT/9tURXlkrSImYVHDoaq+X7qcoWl0p/9if0zyr8NIEVOXkcidr45C6Au8Fi1kbyXj13lOCoTPIzwj4zQwlbrpV0j+ruoT0SK2+cCjx7QuLEXGIXzIRQZ20pLhHPxZDexBNUnA2Gp+pzQo/fM5ne1gprNMoylnuZ/HHk6q08YB9piNLquKICX6PksRxOLIhaQkqsq0SFjb5LgLmrMKnihDTZjWDuxE3Y8lP15B99ZYY0vwQEtv12ncsz1IAFQSBqgcFkWwQjT4b0kKQukHV9gfajPKIbAY/4iLvzzARMCdNKRQ1UJPJH9ZZevJhlGnu2njvpaVg0MxhDF1rM5cfzfc3JwzJAy1GPVTns2oz1feotP5Qb99AsN98+MXQce8GHscms7zUi6hmf/hKqkgaERrNilsajfOHxqNB8i6tCXf0IK0jo56Wropg0G7+piEYYt5/mSIWimm2NXim7NKDN6glDUu2ePBnptfDafnCBbu5KiG9mHJvR35hS0vGD/pxeO5bOoED0M9sZ8KUnn2kiha4iHYQASsF7vL2ELRqLzHhU2BcZDgTW2M38dHDv5WaKZB3sHAhtz2cApLj/yhBPm1keV0PIeLSZuHDq4UESm87gXX+34uAodHK01UqpmWs2b52f3uKa2awxaV212Oga0yrI1DoQjZ98+ejWQ9w3xJKhmvstXlt0WyUWQHKRjw0VTxayNqYYAV9FyRycQ52Mt6SQfio0ACdLTC4dCiLHfSnlZ+wdfwUfIcZMagaVV/c+iVIlQznr5BBCxHkUkeWL9RffLmb4OpHaZyoXLpnOOwMdpUblEN/NllfhrWRh/Gp1/d5m6+B5AgTx8IDLbm1PPHd5lxON8TXyrJjkDkgpqsn8hLmJSXhbm5I2Xtn6ZDn8R20ouY+4yZ8O/sQaVIwv+Z5/67tQ8IEe+dXiHCnV/WuouyeQYA/CrGkWLFlhWiHZz/yRBhY4vW0Quy1k01FFTN2Mo+9oc5gEasS9sGpgvaFy28sTO0MuaIirNu1oyHyqnM6o8Y9hZEh1rp2O4fSMyv9qvrI7OnzbsM/nEAZCJ2awm8Pka05NavfrFWTRyb5cbdEhiSDp0aG7YRsrv75mIHGscLRbbIj7htMxNr/QfMghNgrCZWLO9mFNwsoFnSNxtsQV5dtKIY3QSKXQHku8apKgnqC+A2m1kEuzHF/+sLH5ZzLG316C13YeFPohF0ZX1OvgfQ6sHOplm+Fv1yx0BuQVu9alrW+ckdtYOkbMlSbHUuZN3KYOWaRBobXK+/BGQTPWfM3bVYPcmQnc5UZ0z5yBVEH2JneZnYchkgQqYhI67WRd770x0B7zdTIRN6RIjT3iAqFqq3XXIb/qMtmn4cNX5KLU7XzomzP0Srr7AM+lPVcicb0PT1vi2fzVkqT9FjJyaiJXk7CP1F2rTwW4Tm03jTp9pGdPk7cdVZWOPPkvtOyTDH2NJ5L9OpofJk6kW2ECHkM9pvB3MYBvUqrf+8ILo5q8zH3rmuDBOvWC3hOtgq5LS/rtl0EDuPBQOA+lmNoKokc5FDzC7fcjl4f+auOJGgRqYTU3U6m1lfPj10hx9LHYJqDrdcTGqqCQvqVz7itesXtAU69GW43Bds9biqBviSVqO/xWY1Bhjc2aCnOuTWDGvQ/gcVSa4eBpA5N+ONCUL8j73awwSL/fsz5ct3PVnvlfzIG0LNS1nYFeve+AOrlJrw7XQciTcXPc+0bcfyIHOtNG7wMzGMItq6pdMBKpPNJqsEgWM/+q1cxx8mBknS5/lPegwN7a+9kpKgomLMX8VS8LfJ15Syj4qvpuzJR/VQ7S1241SEu+TV0EVWw5EN/YRWErSddyimKPHVExZZywU0pGrWz7P1j3F4OjRP6eSLzGbd6gcFFyJD3ffbLsX1mQpfpLb3jLAH17PUbX7XThBt+UTlSQpq/3izxh4NCvHZiUfkWonIvlY9VD6nd7R+gHqRlSJhDg9pbeivxy4+IQM8H9m2gNBJQLsUGR2ZCHxartgHKuhMTLiuNPHaLvWfHQ5q1YwlkfOVT2+BZGNO6VuyjrydGoTOuxHsgkwsBIDBG/4lU/i1P1GSmibGyyqzoCyJB65vbS9VW0aiR3Ef7q3z5Rd121o6pKcZGZL1gj5auKWAUoCJdqTC4ei1qEZH1MZLPiSp8/iqrbOr1vQmAAJEXQPgsYT8nt0T4J5zDDlJkGnhP5za5MrN8DiAP7sYOC+20WzbC9+dAZwUed6pG25i2+xZgKNIGMgMEZFpdE4ZFA/EiNwxJiY/+OyJc6/QPExESgJ4aHnUVkA8Zgr6+FctBKiT+EAQwgaDcZwHTEqcaTkQu5WhQqLPgkl0ApXAWaJ/71ICzDRAoDub+mbU1wmNjld/nucZFay55Uu4sDcVu1jczWPv/3zokX6hvO7V0UMdAURImMlKegGBlAE7iUe6Ani7BSHbD0J9nd0zuwvVXOn7xwNFoPYuTRFS8hWdEmWptmE1eS6fzvkGrl8JbC8gnos3HD1QeMoeaviPeuJLaj+Sdc3U2tqVo3iXPB+g1jrfjJbgm/4q6tKgKRuTGQTEeZnAiAGEGni4PbILelhelb56amG2Mso1Hk6/1/F0hquKu6pbDbkZEANBe1DMzRgD/3wxP0XLVxpVAHoQOHAJ8dAzU+VHuaIhuc2weFEJ3+ZLMRwG6n8/lb8ROJ5spFZx21rZWU724++Jg9XcTstV1iRZiUZBYUSTrhDBJgBd+W9a4EKrr7rD2klEjPMaEgP1o6qRbHBzwe3zu8FGCdnD5VLfJWkeo8cUPPVUpgPdMaXto4TntrZCfno/as2lcIQLV9lMFT6S/DiazA1DUbMH6dUKNeI6hunG3t7UuDFz01hdz1Af9Mj/Ah0X16W4A4GXBOBnMN9VNUJg5IBHsQtjFU6srSr6tsISNVRYnN9WrYTtCrHbEjmE/NcYPKYQFvnENA3fLTtJ+JSxe1X+Gz+pL5EEn4sDgRfsBZ+Y6p2juRsAHsH6BKx4lxzgQHfD2wHQKQscdhULyarOId0JT3WIAmfF2mWAIpygcYClCepFi27F0m8e+gVBUF/NYrCPsGJoQYgVrMdT/wQfYH0OhrgfvRAKJQl+8DWg7GOmYAPqczdo/cF93+Ped4iyxekJY/esjKUznRcTBUQ5zW6OMYSVh9yJsWm239jAn4r3q0+Ta+9dJYKsH24VB5ospNSiTdp3ScaX5xB4VzrDu8OIgLjxZwwWLsoYR4UioZDOA6nAsO+2GRCE8Mj4y2tJ9yKUZs+Omh4bl4M1X1pOaHRIx66gww0Z1Br82kEI7pSnnzQFhN/mM8twvrwVRszepsMePdXzlNA17+ZASvv37wN9gN+xL5/AI4SPMqwnuM+6QicvR3Qc11Ugvkc/MHAe+wvfnDtCtB19IQEmPj/1MnshYqhLZXIf2sBTBfOaZc4kd+ewTG8NZ8SrsdrhA6qCcDY1LCARLF6qRTS3MmVQr9np1siB30tYa80andAdqbtV+3+19vnNj+p19Q53x8hhtC+UfMLoowJjDldlc4BXNtL2aELcr3/BnzzNXdJE3qFcDEq7edQsV4IsAoyxKNZEssv9bKSQayRgyCj39bs3ahdrSKqNUgxWkHhy3PwoezhsKukT75/gSyyepIfX9juBBCIEF7I/Q55FfNsM0horHBl/geAXb5gXralpHWLNFb6FJy3F9haWbcoSpWEaZhi2IrZQDcNIzD+lngCX32cPgfNQV4FHLohJD/amwoXFzl7R844p9G/H0dlmAEq9vnOyGeNZ165tSp1byKV5bPJAry4QZU+E6LoqAlhkAWmQU4dDdnP7RryoM23IUiYgZh/py/9cfKq6Y2TQ/pLCD4U1jjpiQZbJpiJet4TiIpNcV1FmZlO7p5cpW8nyqMjjO1ExVUV/RmRv6ZVKl/4EZR3RKXBuvTgoBn4dtDYkhKN4rfXyqQg24PcwdN0tYfF7Yxh5WcHl3h0YrVNA3sfxlkolywPIvQjD6D3QCXMAHD5jGcSD80ns0O6OLnG4263Pt6uaJ5qsfBC7kXPzRqtSsAUacq1RgV6Iytq8wkeIXxsV6CCW1eswvJTQ12dk+y0KDzHxjH+B38oPWRWUuAH7rIKHeUcgvC7sX0tmEBnfxTt7MpW3pFTR9WFm9ndTwYmVa9oqS8h+AnOTPH4QVv3KVuX9sxAt++u+vImC+7tVrc0lY6ABUNQgONDlzG060AsnsYtGw299fV32o5qNRpopMyuc9eN/usGFsC10d5lgckBgsdwU7mB4YfD5axo/7bF7twq1xwGWQdgzRj2tFOoyQ1ta29stXiIZs9quLZ0LS/trMMELe85cCxzYqgXn3bsDfCkLk1lZ23mAs5Y07ncpa7l2JqHUctjDVtpu5x9nHyXXL+55CUfahFJNnMLZclp3QQHFoR5A78dzyxAOaDHGWPUgSnLVgdGHdub/ARHDqkelesRy3AIOp8fwscwDhQfz9pk9dgDkPMl/Sc2xUYUgonafnEILWgoBCcHxQ/VLSeRiI1mpyKl+7fimkztIH1ryxcwwsgohJEXQNVA/D+pzcLNrP4DhKQDtfccfabAGIvo4HxWg5QH8masnakLkobv/TSbMXKdYEOCCVI2CydxTn3PKrbHCRlKwUBB7LVPjL70Xnu6DSEytt/7gTorLWPbdipv9qThAqtw5jCdHCKLyuNDNRswskz9RKaATv+Kc9zqu13hSTvpsY1wByXimADErlLTrOcJsSrkfNKCussie0NzQqRyMEeY25Rp/c8wVGCPdR4BXN827EfoaYuEcpRoE1GPpQRCuHkJgiR6yGirSV99rL0UtdswQhyFqw6ihVxZTcXyB7KRrbe0tKqsGYqiQdseKo/Hn5v6Ih1iNnRBlME1PSo7UEwlNZKYSt1RdK2p8cCpnLn+7bmcqR+k5GlxjOD+IcWnykMfjGIlBjUJZiyVO1NlYKAom/khA56Elt+/BXNfP+quEzSKP4OcZXxyMsAm0Zno9L6LARyTlj4M5BVCiX8Yo7wCwJji7ZBMYF5WlGsHe+jNdrmRtilwPGa5/dYgttH95dtN0Fi3qUKF0QSHBKQu6HPAkiJtZeudKgxBhuRWmMc/APURv77Q412rxxWq9uQ9ywEms5euIP6vBSsbLpL/JXvgjA/dKN5QquSSicjDGKelIKQSISMrXaFOLWgu+6IY5YlUsl8vyUZvkY1C+1z1f6d13j9NcAlQgpTBhM+2BJ3b+G0IuJnV4N8FrR7CyOQCBUJRwA8WR2Vt36YFIhH/D/1VkU5txEAZBlDCZ5jHyQBkswvl1Z6K0DAGw8+nnoMdV64sy4SlaQIYxgbq4F9Jzh2K7V0SHzeKEeV2x6S+2v1/TedUbmYyzUdah/d0SbfUQRkLof341fIVrED8bQU7ClunjgMXo0JEHMi2sl7q2XymQV5AqUuOkJ3neO5whXVdRRrz6XE51U9qbVAGTydREgM/Yw7lA/w9ato1fcI2lpvy4v0yi41kJlSlVym0xYX/ZMyOPquub7J0G7o/vxYWi/pJk/MZwYKystx1S3c1CMI2rWMQiPkVPg7p8YF5bhMwIhhoOH1dz6WX0S5VT2+ZCk6mx/ko+GtOrNOXgGFHXAvpcZPtA7jNdIk9d5qRLQjdr2xUs29dlvlNfN5NeX9eGIJ0wUAjDCUiQJFXigmNAgN1hts0n2ShKO/ukwgL7n9mFt7rFVyc0c+ibACb9JBrsoFLL2E/WHi4YmwpwLIH/w1bs/3DMs02fysz39+jszz2OejddwIUeKKLGiIuJKXpqK7MoT8RH9EzyJn1nQy4sEWd3yhLn6BUdg01VFAFSb8wPppH363VXIXG7sLdwqbUeC2P8IXRsUf3ILOTUVctEHXRWB4J921ooahp/rNU1+wVOT7lEBPqVWD58mOzaiEeNqGwipk0hIBWx0gHMl5m7KuOr/JY9Bv9BnPoDG3qzh/ssRB6qGRLidE4yt0xK/gfWEuseFyQ3G88XI5HdeXVjjMQ/SK/nKdgFVi9K8tiDr2TIOJzbZKzCnqCGFoJOe+yQxGQl2erGNhfexzWSj2+a4k5If+aNqJkLqKsDSY0d6h7YssyAsuCWUWXO/G80yT93PNYQKiS96IThUwIj53TNbx0102V4hL7IA9LLwUiKru3i+Nanl9SagH69zQJ3uCr/kpneBMTooHtZofjTW5Tf/fwZUeAmat5Ej17++MYId8MrPaA9KG7OlO4cJGZjyUA/OZoi3GyVtV8K4YyvU7bus0bEsteDi5dh0/Q7f4zFUGQGMvQ8fYi0t6nbY5duVxSNQcbXViTutYPym+oLv+TMkQFvEgbdcmtrDa000sl0eJLf94sZkaNQzQ7ClL4gGzNwkuxusTr3V+x3iSVU+l/dUZDcRF+rv4v5DecbzerGqizptcGFAvNrdnMOq9P7XKkS3lUOGv1mQlvYCzOk1bxv7yMLhr6Pt2e26c76z2RBuj5KlZJL/3/M5Y6yTgGrABR8dlLE6iDS3bdU3xbJOnbu0Al65wsBz1QLU3+gW5Vv/jf+jfRplUUYsYs/BTBBuApQzemu4C6N6CNqUxvUqe2J1Lb51ul60AI33V1cRwhaP6swDxcuhAr60dSZgAv/9B0FjrGd1pyg4RP740Bb4hBcgH7WkMihh4SW3JQCWUKhAjl0TYQjUpQnvmkSW903Mon1nUQ59YqcOFNEyzFoSbmQ+L1DEvNDo3vHmd8p0LVzySDp0anjvIsP4c0WHHjcQwFqG8bZ/KV3oXl/3ap4QH22giZHm2avGrLabhZQBmYSlOIwWHqK+cqkpqp2N3znqfYRVVvjNpuWzFMvM3niZM8I3+voUGHU3Db+yeezryGhEEp9a+1U9DWoml+SYgyZ1mOmLRSNeaYyc5u/uiZ5Nh3ILnWmp6MgBmOIiiEpHwsuM3pVZ+XEuHr0hWk+DvITWphRC4tvIUnMqHmTkXmec8/HTIg/nVEDVMe1+U3dCd0ymCqvWhCwQGTGCEA1alDAevG3UhY/bkgYsna5K6yLV8lq+TasaraiGoTqYrak7Oj7pJMECzlA5xEH0pQ7I8n1aBkjOsY7bvrZroVZouOuQUs99KwPtuubH1SPAeQ7qdUyaGOyPC8D2DUR8Xydz2GJ/+Ev+1pSJY3NVvwTQsPIhlMpHrLvWm5reSNWJRbhvevnoJxorcJ3HuMZRpi6ORlC0dWOqapTYUpJCRWSDCY+ozIaTV8eEy4wuCF3y7/73CegDiwh1EPUFmI3Jz48MJ4VJ9smFj868Edr3EPNHgerSS0bnrvE7QiDrM+XYUuJ/UUUE0Z+dGQsE+zTlQCiabnviiDhTdmq9pj/8IktCDLp7cPWoUa1r1m4EZzTliDj5sAdFnxEP3jVpBTvZibpr9lOTwhayqg4K4gDBH7TVcdBInEh5mi3BPcU/d+XcfBywapMW2Sodeef8T1LvzaCGxzTJpWywm1iMIL2Hr7jQa1IW31iDuey7C8RgCZb3YhTVuryeee5uR6QHcvFXlGsvJ78/+J3T3AQi0yp7MNd/RyJMzsAUDGMzjn76q7kjYqOrsNm/wZr5pbKmjCGRuGZHkjIa0MB5CIcniiS2ZPCT22UkJDKxEou3lu4xGNd0kRO32idmG0GbF5gqc9DQrT5RXHJhwal3kLgbc68BY/wwUxCEnh811BjE0qjoMpYcdDzuHMlb/3bwebpVJT6hG79wbTUdo8L2+RsPNZAZ6YHOcXziCe68M8ecG7+2Ulb2055kvMcwPJqh5QQBEnORTGbFVfQGmL51oZx1Ho7pOQmqoNdn9HmDOnBCfMzwjlP9LtBwckhoHLiS4EtXs0dzWfy6lKocO1jSk8pl4rtItyXs4ewLEH8KVflPh2My+DkLBu2JDLXiNmZwDpK3fyWcqNmnD7R9o5jZXXelT2038SLcO9CyzL2ZZhj2N987qpGHtViUg1w+/AsKOQViEV3feafMYUFuAn061uOjq07Anw+mjLc5nsbA5BI4FTWFbYImn22283kGvhEI2OufXJvLwUQ7v+iMaDODiuYKlOqS/q004SLepwpOiy0oq8/KFSCBUjdy9VIFGNScCUQQtWU5UjbMB0/39Ex2UxxL4pnOB8Yr92b64nGAnIDRXkdOgysIgo9BOlsB4ELKke9kToZZRnbcwzAaijbTYIgC9NsFiKPbiNOA1lIY5DkA62li5osWQSleuhV3OCrHtfMHj/xxg3KRotsXlC/9FexkqDctPFfjoUoKHomPuiez33b4uEgVPgBjg5EA70yj0oT2fYk9zcCcXihrCJa+9j7wXRCAFqmel9irtLsb3pnVLLc4de7IKH0FDZSpkPu2NWgRysIcw7BkUy7leB2LszFSC/NhcbrskDQ+5/6Vt386viyOml9+AiKu8TlsvF1IJHnjUdWrpNj2H4ID21qPxr06Hc1i9SZqXZbazpwaHiB3KgWgGdkm2lLZ7tpq8+12iIvqwXKotFlAR+VdjigGC6w/pNfnD4tIE/fBpVjEnFJFZjU7k8RAKwEeZXZkDo00bGZU86rmZu32CLUwwTJBIcXOsICZZCILGhiWGDVimbuO/detMhsE3291/tq8II1r0OTuHsCmzDdgFK4rz+vqVBrvfATTwVKE8eUo6lG6QPubfvUBJojwxa1ngzjcVUG3MXZEb3w0Y3T5RWinEGwDtRbZ/SHcUH/vJ1Tzk0ghknZHK4Xh94FOR9wtXBC0gV96kenEsCvFDVRvU2EvV3mB3Hu6W6mtej3zBYQUTsWXsmyPZ2VUNpUOkUTlI9+NG2l6KgpXqkDaF3ptICBhpJz8oqI4PmSr93SdQSjre4gGb6a0hCWeFaBdWR8VXw+6vyCl0ZyDD601Gn+PftU9K1O7Ea9fDyxd2xgKrVZTOgpXIchIE6fd9Hk7wonZcQUFzqtPnVwHlhIhY0VuhEu2XTKiJmYPq3cXJNsI1XJe5RiaPXRT96Sgkfpqj2s1ZHMK/hY9Qean7CJmRKF4mYkYl8rnrH99pY36o1SWL+dEoQ4ytGX8GNz7Gg5QtGR0ThxxaA5kDysQDjyfbkuCQLcg/UPo03ybmTWfrPlNf/RCBn3W7x9CQkE7+oZ6COGGmxhnQzFZ+k7d2HSegeej3Wi0BJhZwJFeCzBmitsuZTzBebdK+Z7Kw93LfmNhreGIddmQ8fTQjmCgsZ1kTBlDmpSaoi65FNbHr7wbr7oBE0U476Orrksh0a7zxCiYmeft49q0tuqwRzaeDK7HKGlXw5feMPsuZXc30BcdmvabhwAtspu8TTo+a2/tPCUn4L6hKuPlfs/oVa3+uDgS9JpzpKAzZ11aOOXx9ARWtnd2mmKx6pL6rqSqm4TNeYVzQdVqa1lV4nh8fJn+QDa3dc65hu7lZnfGwadr2GW+GEhinnVy0Npv6vDqUgPtXxTjdbjPgzp4/0d/Ld8UEi5Dg0Li0Swa0qQyrdXza7NddKbCy1Viv9yQEHOv/blOUP2UqoEhUDasTh+VSSzFdwV+E+ii/3L0eMF+qj80PmTmajL92xLeKds4Zkryre0TXQ+bZdC4MEro+7WzBzt4okXbHo4e8XLko3JdvwEyVI/zyvM9ueiXU7eAjnN+y2pVk2vjhbPDbOOaCYYUMtCiarN1h4nV3KExAQhNFkTN/kLhUrK31OcJhCqe9BfVqAkZGWZugOJedseLdQ1zW7ONqmf8fIyd7/6x9uEgWEnR9433d8rUT7L2TbISEmf/1O2zsjDYBB5c4Aoa1B2Qe1CXk8bAqvM0grth1fGop6OkebvlrYPQn2W7v0wUG0hMzISLkqtMaa7jV5a8KGKqqxhN8qjBqxxeN6Ra6qDQn1Aq5h5L6CI8tzzpWUFqJeAPKcxCJZL/IJaKFPl2H1pR0AMewQFpExxdKLvPjKSd8HNO7s3cTy8UkvSQvoPgU0sKKBoOYMNnXICGsU9byurtHJylXRBZobS4Q9lknpNR984rnwTK04PImmvofvpdbuPX8pP9gYho8tihR9xRpk/jD32+oXqr/enh0G68HpE7tpxmNdt5nVHeeJzl4eiIBBUtTH8ARH/6iHZlH/oSdn0XyGj47fSWu4czYufr8yugfGyzM3KT3WP8Ie9FL5YfajRgYJs/vSzd75kRI8B7bT3ACt4qlPMfub5h2TV1qYY8uw7ZVKvGA5dOEJp8oOUFN9C5Y4v1fpZtfGYT3RQhgnJx7zaeroxzk3KmtIb9Q8BhO7eeU6d/IPOppfTw54ub3N3UxbYvaHIHB/rUEz4MVViktxxeyPWtT2nQvSayfwpYKXIkk9FZBPHVXUfZ7fWzhux3K05zoxHmO5LfTcn7EfmrGbAFq+vWINxbm3/wQcLWRcVbxX+Qt1cYDb7PG6wUCDP58/ch/yCwQYVDyb0fMXjzpAJc/GYZDDYXsb6+54UmiLgBNZYUgUmxJo1SPmaKFP+G1XrUBiRJ67BxXKs/7pTlBcOy53tBfmsC++4u/uQ9uirL9W+b3B0iDARoraFVGiYIJ5ljE9f7R/l2qYNP5mjGXZ4CrTWNeckz44pmvjKumMN7Kl35DhGilMvwl+MrBAC8aYnioYWnJ+d4X4y96x5V/Kl7J8GzypvRLJ5GwK6bTzqI8J/lAhGHK60sRI8S5Y07PbQ9xknvrcppo1Uv6z7gsMQEH+IcV8jQ4sSefJC7SHQrwO7rjAamH9zag9RwqsjxnFi+T4edZ412P2mqgUKG56f7mOLzMSCuCZTnVHoYSAT5/yewYymKD2hZdAmkY/SnbkC0FVzNOsHM/w5nbjvgDOINt5Rc6En5/6y+PF7Vuz0VDTYtLkV4WovZxJ4KdLFjj+XTbriRzfemhK4/w6WLez5S4VjoHXFMXJvd7XyZctI+mkRBk18itlML2mzltFj8JaXsRcD8qJ0bQ0B3Ocj4dg54cydh1fx7dSdBVAtYeJgZkh1K3iQBo31L+G+n25DjmASzGSOH66difMWtmAZ3xmlBvg4I8WBltXEz0pA9l2jbN1nLhsWTtKbBOJMiYqSNic0BxSw8tbSuhb5JAvfrUv6P8H/sIke/dPhXbCAsYBDteZWGZghvRsJGuCTR6DcAxQ1AmFx7qr31wGwVlmF3ct3+/7J8JUXWa+3MeOGXpP4LBxQRxXMNKPIojgGAHQcILeHqNYBmG1K0BZc7h9l9MfDuKdU7+3jpzavNuKDCggM+99zlFR/VDvcFK702zjAYOlRAgaaFSGVzDqurOQHlptgbIgi4vYRnatjTL3gULYcBMxY5Hs5hJWQpeKvKgtoa4laJEQO8LysdvU+L96rEZ5vscn7vato/P6n8z0eX0lKlwbAkVae7KaoXpF50jaSGg4WxJgykOBNJDYhY8DRAmtFdp1HNpEG5fDr86dKIkO1cjqyHRPOpEhlbO+ATSGJnT1/ACphr+qLln5karRxRF9ExflZa5HaoJzXS3rHLuDee29EX9xhefZjkRzkOhR0RJDFxBhYMX2vFR1GeZdNv7J22CmC7fWFHprS4T4L3XoeiuBma40rutxt2af5jHBRQK/zBshL6eV9JM9y9e0cpfZwdGgHdzz6bI1jqNCViQcGPcfJ5xvHqAny8dMnf/eyVLk4Iag5sltgAygLnxIlsdyb5BgzQSvmGWzkdQ6c61NF1Cvg4DJhVK+6I4/wIMUNX6PxuwIuU58P0EgmHLFR+sH2WNid6EQmyu5WFByX5PrG/t7p+oTnAa9Gyk8AZmG+MFqxi0oVN7gcUDy4XGK6BTuQIhi5GR/fBi+Z1yT2PQ6DOaSXf0g+coaC7wlt4WyZRcbeDoAxENIKcoRsvsdNm/Ej1lM3Nk75kkIESnQDyoLtUVY2tYd+EkRgkUawV03W/kDmH1AsSjEVU7n95vOeQK2VPOGzBT0/BcB+wG0SGLBu34QAAAspBmiNsS//EKLkMkqZc7Wnqk1AbjwYp/X1vs7495Pgkn/S4tzSo+Icyy1LW8cD4/WfzeW6bfp+27PPh94nyZbPHLDca+bpVyGavwJiqfifbOqaTj2ic+LK6U7wV5gcZx/o6kKLqB+jbGvF62IIj+8GolCMtWkNcGtbvhPCuwt+n8lQ7istfRb0FfzSvwXME66xdOgy7FD4WI10rR0onZimuQ3rkEmFEPnHrcboX4Ady99hJBetv2nFLZAMJP+BfhGnbqdV1bIsPU7Dz4hSaE+k3YDXI1V2OdXznsA+3mzCH5esvgqH6OpZioDfqyKPsg7CiIFyYDJUiR158Kna4XW6yPCzfA1xCpTWimplTtKgEw6KCEvUO8e5RdDwgKCIfnXYIEF/O1MeXp8kuAWMAUEH8ngK5OLlYmybCsth4zgYqawKa7t/dUgHb7U0a6hWX/4qZ8Axyl11ARUdHrZ77Vfh11XII+QM1UC6rP7g5m1cKNvIa6EtX++EchRVakgv8UKtxbFKgLZIzK9G2Gp0KLjA/c59YTrSvEanlMcyxpe+Fg5G54CIaSzlOp9NDf1w7i+NdL6SWAPA3Mxw4c34woGGpNIBxxjx33JGUT/8JLslJ/ta2tVfGqE/nEow5AV+6q+dOiZxkn43Thv/nYcBGzyHUYxF11nBwAm9bTHJvqz8pVFY0SBAM0TETZ6Mtci/ijjmKwZK4g0YuB4k1SMjsykfUBbqzY7Jev0arJAyxOTejeThauKwrVqRMn49aHgK+rHILYz6nyRe2FzQOpi6bI10XNF/YOa6EfgjGKXYNrPrDaEckkkMYHca73DGIPIXk7DVCeQGompXssRM7WeFe8qX/A5A5ioQSnb3xzUa9Qt5u+QrWoqZe9Kma6ESC0dXGBr16ea8600TXfw4xZfEzeCcmZETGBksbnXm1wj+fBJYLSKbsR6k5ByuKxKgAAAAQQZ5BeJv/0DHAdSfFWhgCIQAAAQ0BnmJqRH/iF065pWgZ8iALA2LAUDkb1uOyJJb5L0NHfdR1Aa1/rDILSwbcmGZ68Jtf4FKEE21Ju3Fth8kaGmwDDVzf0aPJuxxs6F/w2QS/BB7O4Lk4BU5k677uqFEuV5fZwenx3sW19iRU8wHrfZnLhmqyhZ4QK3CHohT4+qp+Ox/3j/B/lVUp1a7kOiVVzPeHV68+GtMbppLdTSAQptKu00hZHByGkKHiJy3pFO5oIIHHpzplZrYSgabOUZPRugsBEjiPSf660UAGIh1s4RWC1sIGBoy4L5lY3/kZ6BRm0Qr6CBTko11jQh/FuTJz25eBZOp+iYU5HjIkgEtKp5vJ7nlboeC0niuE6Bx78AAAAddBmmdJqEFomUwL/wAu2AFYk7xg7pVleXMhLzUiL/j8M8bQsQhxN9ZkKu65fLPLlZ2HFUPx67HDcmkSqTM4SQPw87AWhqc3oSKPfwPp3NX6K1nyXSQarEKwoSDTHF+FpP2z+XtnVvGmcp+4+jvm/uum0k2y7eUVVvAIsu9aj0VeUH7bd6qcdiFvN44WurTPq0t48+882kizjNgFrAb0XhQ7OL3u8pKrab8jBOcN8RH3BihfTIhl4ykch/tbvwdr9y4azzl9f4II8pOBvcU7ytoj8bwG/qIqSmu83OArTmaQOQwg+d04ikcsNDo12zIw28CWOgflmt8t5IkXBDiz8B4H1nCg3HlLk72y9cjEo7B9njqiO1zAL546YhO8Og6uCf0bBcqLj50A7RSgwUSWXJrrg4ppuhosHhcfdpJpUoIzL9Qy5mq47TOwCEVD7Fv+/p791DFKS5THjIHzTe2dTzAaiDlP3Vmx36mrporaoUJeyQK4+Er2aWV54Sy+oLbghLjHdyrh0OEO5Jcw0XS+5OEt8GRr+nAVCl15Db8viPo40BTbOgCejRCTeKz0wtCgWvis7WoT305fCaob8+Bn196eEPkYBiFPPBvUFKzQUplFWKgyAFGRKDEAAAAfQZ6FRREsnzp2OzSqKiBwqknT5kgbFeihFSBld17oOwAAAU8BnqR0RH8dKxUWY0GS2Exa3xw/O9PF9OecrQT1in2F1b1t/gAAcgW6MAwSx0NbX53jvd1qH/yEhncY4T+17YzIDaZMqulpM6LaQigqoY1Mu5G4z4/JdidHbiwwjfBFa3W2uqAxy3QPD3XkPN1arCQDBoYHULp9JhXZSF9LrxxAiLARhFCadohIwVR8SpF46m5b2ooaIvVKL5gaIICu4/X5+SoGNT28pf3QSC0ru/RCbGDqSJ8/+LWh1dkRSKKYCbYxJPY7vL2dvtvrBsybLoarxY7K1AqQLCOwVjkLXx/r+9PVTaBLbxGaAvDhKYHcgL+s0P47iasEHNKmpqas7Vnkjm8bjGG7hIUyuAOJymdTbXrELJHEYRs7KpcBkRipElgDfvTOiAQDQwlo9/teOPAhSgjs8+bIT1DfWH5qz3FJbo2oVi4zktuu9g0Gy7xp3QAAAW8BnqZqRH9DFtyVIx9GVCOni+nPOVoJ6xUrML7pineYIfcykDxdkSUx7TUpUa3Nah5R+mm4xq9QxQ3YG2v+5wchGlhCa1/cA9/shBAo+lzW7P4II8pQMCsMe6ROolMO5nindLBOWJ+8w13TOunwKi8dRWzCxbnhh+KPELcY0OYoXCLK0b3x7S5vSj5iPlRJf2kAIs75/05Vly281gosAAXYqV2k/JQXdiTv0YbsSf+c8WVDCi5UkXNBz/r4TqCbVxs60cVKZ/MbcHDlWFpM8ezqsa/AjCSRoNMEr4iaG22qEeG65tIIFvzZpXNzsIRnuMAq98Bp1ma2WH5pddHVGvgYjaGjz6J0K3X2y0gpKDjtJVySnPWCMmoV6kFbyjnH0gea4l513rCcjAN/AUMWgNgYMoUpWc+c+P0sEhKaiGJSnpVJXoQgVtUjSWjV4ZnOvS70H77xv7XRNKlhucfYCXujQ4VrhVqXX5PifsJov391AAABfkGaq0moQWyZTAiPAAUU0IbZaOkuGz+SEhdfu4ozV72O+B0PAuDFlsRa5c+wk0OYDpf62O/BEmqZtEKwRh5SGHfK0i7tEwQY0+k/LG8G7v86HznwSFnFYu81R06j0y853PY0I2xTX4w5d/dWFDlQFrhfP42qrcOdj4BXXeD/Pn9wiCTbAm5YgUQJiv6TQTQh5TlMwjeHZ+V1Zxh3+ogRl689R7qTThnKpjMsEmwfcQKkCU427ZwHm7LutqINUF9h9l4BEeYR6nbkFkV8e7pUOqdobZ813q+SyKrfrl7P5SAvocwaN8zFNm8HT6VI7aTlwmlZcK4GCBUu0oS3g22RzDvfN4sNcMICU9++/Kkhppc7bOh4FpZwQo3EgcebPiL8wEeUHVlJlsFCRlPs1tdqrmCFACbXckehvd7vBJmEr1tvFx0Pm80McJQqwVgqELhIf8XeU7H2Gfk+QtVlDbLGrBhJQIqsflS4mrnbkhHucOZFdV6tK0Zbie8stQdKDwwAAAFSQZ7JRRUsnwi9ZCJ2BatLYC5FARhmytIPproZQ47j7hGtGe9xj9vp8JdHZCAmXfqTozm5nOZ9ih+O7XLupToaZDiOvcTuB9on5pu8qTaFee5/jaLKsiUByLtrReEYPMo3icYHyot2ZMs7Kt5gIMdZBUWopZcJzdL6a/bbH9n3HdOz5U4A1DUGdP7AVG2RDZCQIszq6+J659BSrVJjj6r37dP6hozEM5QGTdGVp+RMwBmFIKsXdnLkfOjm507bxjk7GEaLCEFzpJj7thWuDxYGvTmAUiURfZxQEHWYRrKywkDqyI7nB0uhfLES9uEnbecVUdl74yw+sgR/gKmrWzLziVj6jZ7jYs3KBvY0ACgWaiojFGopPKgNhK6NI9bK44QGMS8QI73jeCkN883QZYc0hEYQ5peD80o4Y1eQ7AUzU0I2Bmnc2+GMnmeiAmgCMkFb7hgAAAFtAZ7odER/At5c4mj2+7i055ytBPWKem/fdMU7zBD7mUxUaT72mpSo1PJ5fyj9NNxjV6hihuwNtf9zg5CNLCE1r+4B7/ZCCBR9Lmt2fwQR5SgYFYY90idRKYdzPFO6V8sJn9yqwQM66fAqLx1FbMLFueGH4o8QtxjEG4FcIsrRuNaUeAlHa5I+VEl/aQAizvn/TlWXLbzWCiwABdipXaT8lBd2JO/RhuxJ/5zxZUMKFsetx9RfRFZdBNq42daOKlM/mNuWmmJ+2CMezqsa/AjCSRoNMEr4iaG22qEeG65tIIFvzZpXNzsIRnuMAq98Bp1ma2WH5pddHbIIrOslzL6J0K3X2y0gpKDjtJVyTgvWCMmoV6kFbyjnH0gea4l513rCcjAN/AUMWgNgYMoUpWc+c+P0sEhKaiGJSnpVJXoQgX3fd+i+3ehWSQIP6Jp+qz+10TSreDW9EXL3RocK1wqGoWl7D5y9l5adAyl/iuEAAAEcAZ7qakR/Avo1fLuy4QwOlW9i3QfGqtWe39aLaI2tWS4SJURoam2ydNkaqezB3ltSOPz8yDlTBF4EsNnQd9VtkJgS8YhFx2jUXxAy26jtA5plpZytiXX0ckqQ5MVQiEvN6whKxR3a3hV+jfXW4HEQbVQNrhO1pDrizjllytt0YAkXIxQRNr7cL8S9wnDZYQeNGCve5pL8eb93QOzo+0KwkUo0Luue/p9F+b97IC34B3GQlTdSS6OryZl3Wmy3aj9R76IkpYPsJWKmOtMwL/8uHmr8afr/PffaO0qsj1qRi/jW2HozfhRidaLNWQkArM7gMB2xuFBtSNSBvuXeE43QJHuGcowDo8/7cHgQHfXTorqs1aVTdhyt2NRy4toAAAOxbW9vdgAAAGxtdmhkAAAAAAAAAAAAAAAAAAAD6AAABLAAAQAAAQAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAtx0cmFrAAAAXHRraGQAAAADAAAAAAAAAAAAAAABAAAAAAAABLAAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAQAAAAEAAAAAAAAkZWR0cwAAABxlbHN0AAAAAAAAAAEAAASwAAAIAAABAAAAAAJUbWRpYQAAACBtZGhkAAAAAAAAAAAAAAAAAAAoAAAAMABVxAAAAAAALWhkbHIAAAAAAAAAAHZpZGUAAAAAAAAAAAAAAABWaWRlb0hhbmRsZXIAAAAB/21pbmYAAAAUdm1oZAAAAAEAAAAAAAAAAAAAACRkaW5mAAAAHGRyZWYAAAAAAAAAAQAAAAx1cmwgAAAAAQAAAb9zdGJsAAAAr3N0c2QAAAAAAAAAAQAAAJ9hdmMxAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAQABAABIAAAASAAAAAAAAAABFExhdmM2MS4zLjEwMCBsaWJ4MjY0AAAAAAAAAAAAAAAAGP//AAAANWF2Y0MBZAAM/+EAGGdkAAys2UEAhoQAAAMABAAAAwBQPFCmWAEABmjr48siwP34+AAAAAAUYnRydAAAAAAAAiXRAAIl0QAAABhzdHRzAAAAAAAAAAEAAAAMAAAEAAAAABRzdHNzAAAAAAAAAAEAAAABAAAAaGN0dHMAAAAAAAAACwAAAAEAAAgAAAAAAQAAEAAAAAACAAAEAAAAAAEAABQAAAAAAQAACAAAAAABAAAAAAAAAAEAAAQAAAAAAQAAFAAAAAABAAAIAAAAAAEAAAAAAAAAAQAABAAAAAAcc3RzYwAAAAAAAAABAAAAAQAAAAwAAAABAAAARHN0c3oAAAAAAAAAAAAAAAwAAERZAAACzgAAABQAAAERAAAB2wAAACMAAAFTAAABcwAAAYIAAAFWAAABcQAAASAAAAAUc3RjbwAAAAAAAAABAAAAMAAAAGF1ZHRhAAAAWW1ldGEAAAAAAAAAIWhkbHIAAAAAAAAAAG1kaXJhcHBsAAAAAAAAAAAAAAAALGlsc3QAAAAkqXRvbwAAABxkYXRhAAAAAQAAAABMYXZmNjEuMS4xMDA=\" type=\"video/mp4\" /></video>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from gymnasium.utils.save_video import save_video\n",
    "\n",
    "def save_vid():\n",
    "    env_video = gym.make(\"FrozenLake-v1\", render_mode=\"rgb_array_list\")\n",
    "\n",
    "    solved = False\n",
    "    while not solved:\n",
    "        state, _ = env_video.reset()\n",
    "        done = False\n",
    "        while not done:\n",
    "            new_state, reward, terminated, truncated, _ = env_video.step(select_best_action(state))\n",
    "            state = new_state\n",
    "            done = terminated or truncated\n",
    "            solved = reward > 0\n",
    "        \n",
    "    print(save_video(\n",
    "        env_video.render(),\n",
    "        '.',\n",
    "        fps=10\n",
    "    ))\n",
    "save_vid()\n",
    "\n",
    "# Show 'rl-video-episode-0.mp4' inside the notebook (encoded as base64)\n",
    "from base64 import b64encode\n",
    "bytes = open(\"rl-video-episode-0.mp4\", \"rb\").read()\n",
    "base64 = b64encode(bytes).decode()\n",
    "\n",
    "from IPython.display import HTML\n",
    "HTML(f'<video alt=\"test\" controls><source src=\"data:video/mp4;base64,{base64}\" type=\"video/mp4\" /></video>')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}