Skip to content

Instantly share code, notes, and snippets.

@rozeappletree

rozeappletree/train_-mapnet.ipynb

Last active January 23, 2021 13:49
Show Gist options
  • Save rozeappletree/ddb19bfbd98db817335701293e2840b2 to your computer and use it in GitHub Desktop.
Save rozeappletree/ddb19bfbd98db817335701293e2840b2 to your computer and use it in GitHub Desktop.
Download ZIP
Train_ MAPNet.ipynb
Display the source blob
Display the rendered blob
Raw
train_-mapnet.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Train_ MAPNet.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/rakesh4real/ddb19bfbd98db817335701293e2840b2/train_-mapnet.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "code",
"metadata": {
"id": "EYmlIfEdUJMa",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "ff79395e-fe36-45b5-fad9-bacb73e99621"
},
"source": [
"!ln -sf /opt/bin/nvidia-smi /usr/bin/nvidia-smi\n",
"import subprocess\n",
"print(subprocess.getoutput('nvidia-smi'))"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Sat Jan 23 05:41:17 2021 \n",
"+-----------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1 |\n",
"|-------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
"|===============================+======================+======================|\n",
"| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 29C P8 9W / 70W | 0MiB / 15079MiB | 0% Default |\n",
"+-------------------------------+----------------------+----------------------+\n",
" \n",
"+-----------------------------------------------------------------------------+\n",
"| Processes: GPU Memory |\n",
"| GPU PID Type Process name Usage |\n",
"|=============================================================================|\n",
"| No running processes found |\n",
"+-----------------------------------------------------------------------------+\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "2YCApa2aeU9C",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "89f33bc7-4964-48ba-b84d-25aea226c2a5"
},
"source": [
"from tensorflow.python.client import device_lib\n",
"print(device_lib.list_local_devices())"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"[name: \"/device:CPU:0\"\n",
"device_type: \"CPU\"\n",
"memory_limit: 268435456\n",
"locality {\n",
"}\n",
"incarnation: 335395685610852296\n",
", name: \"/device:GPU:0\"\n",
"device_type: \"GPU\"\n",
"memory_limit: 14638920512\n",
"locality {\n",
" bus_id: 1\n",
" links {\n",
" }\n",
"}\n",
"incarnation: 16660750881782153064\n",
"physical_device_desc: \"device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5\"\n",
"]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "MFGVP64SaGsg",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "2c06e85b-815d-4d54-8727-8408db5affd1"
},
"source": [
"# Install required libs\n",
"\n",
"!git clone https://github.com/lehaifeng/MAPNet.git"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Cloning into 'MAPNet'...\n",
"remote: Enumerating objects: 3, done.\u001b[K\n",
"remote: Counting objects: 100% (3/3), done.\u001b[K\n",
"remote: Compressing objects: 100% (3/3), done.\u001b[K\n",
"remote: Total 103 (delta 0), reused 0 (delta 0), pack-reused 100\u001b[K\n",
"Receiving objects: 100% (103/103), 3.94 MiB | 38.11 MiB/s, done.\n",
"Resolving deltas: 100% (29/29), done.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "loZqWTY1ecre",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "3433f585-6712-41ec-e9f2-4add45dc5f24"
},
"source": [
"!pip install tensorflow==1.15.0\n",
"!pip install tensorflow-gpu==1.15.0\n",
"!pip install utils"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting tensorflow==1.15.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/3f/98/5a99af92fb911d7a88a0005ad55005f35b4c1ba8d75fba02df726cd936e6/tensorflow-1.15.0-cp36-cp36m-manylinux2010_x86_64.whl (412.3MB)\n",
"\u001b[K |████████████████████████████████| 412.3MB 38kB/s \n",
"\u001b[?25hRequirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.1.0)\n",
"Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.1.2)\n",
"Collecting keras-applications>=1.0.8\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/71/e3/19762fdfc62877ae9102edf6342d71b28fbfd9dea3d2f96a882ce099b03f/Keras_Applications-1.0.8-py3-none-any.whl (50kB)\n",
"\u001b[K |████████████████████████████████| 51kB 8.9MB/s \n",
"\u001b[?25hRequirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.10.0)\n",
"Requirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.19.5)\n",
"Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.32.0)\n",
"Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.2.0)\n",
"Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.15.0)\n",
"Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (3.12.4)\n",
"Collecting gast==0.2.2\n",
" Downloading https://files.pythonhosted.org/packages/4e/35/11749bf99b2d4e3cceb4d55ca22590b0d7c2c62b9de38ac4a4a7f4687421/gast-0.2.2.tar.gz\n",
"Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.8.1)\n",
"Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.12.1)\n",
"Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.36.2)\n",
"Collecting tensorflow-estimator==1.15.1\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/de/62/2ee9cd74c9fa2fa450877847ba560b260f5d0fb70ee0595203082dafcc9d/tensorflow_estimator-1.15.1-py2.py3-none-any.whl (503kB)\n",
"\u001b[K |████████████████████████████████| 512kB 52.3MB/s \n",
"\u001b[?25hRequirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (3.3.0)\n",
"Collecting tensorboard<1.16.0,>=1.15.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/1e/e9/d3d747a97f7188f48aa5eda486907f3b345cd409f0a0850468ba867db246/tensorboard-1.15.0-py3-none-any.whl (3.8MB)\n",
"\u001b[K |████████████████████████████████| 3.8MB 52.4MB/s \n",
"\u001b[?25hRequirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow==1.15.0) (2.10.0)\n",
"Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.6.1->tensorflow==1.15.0) (51.3.3)\n",
"Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (3.3.3)\n",
"Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (1.0.1)\n",
"Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (3.3.0)\n",
"Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (3.4.0)\n",
"Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (3.7.4.3)\n",
"Building wheels for collected packages: gast\n",
" Building wheel for gast (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for gast: filename=gast-0.2.2-cp36-none-any.whl size=7540 sha256=dc7fa20d763bae4c61fec1c43e469a54c560aaf4f8d47f7bd3913b8af262ba10\n",
" Stored in directory: /root/.cache/pip/wheels/5c/2e/7e/a1d4d4fcebe6c381f378ce7743a3ced3699feb89bcfbdadadd\n",
"Successfully built gast\n",
"\u001b[31mERROR: tensorflow-probability 0.12.1 has requirement gast>=0.3.2, but you'll have gast 0.2.2 which is incompatible.\u001b[0m\n",
"Installing collected packages: keras-applications, gast, tensorflow-estimator, tensorboard, tensorflow\n",
" Found existing installation: gast 0.3.3\n",
" Uninstalling gast-0.3.3:\n",
" Successfully uninstalled gast-0.3.3\n",
" Found existing installation: tensorflow-estimator 2.4.0\n",
" Uninstalling tensorflow-estimator-2.4.0:\n",
" Successfully uninstalled tensorflow-estimator-2.4.0\n",
" Found existing installation: tensorboard 2.4.0\n",
" Uninstalling tensorboard-2.4.0:\n",
" Successfully uninstalled tensorboard-2.4.0\n",
" Found existing installation: tensorflow 2.4.0\n",
" Uninstalling tensorflow-2.4.0:\n",
" Successfully uninstalled tensorflow-2.4.0\n",
"Successfully installed gast-0.2.2 keras-applications-1.0.8 tensorboard-1.15.0 tensorflow-1.15.0 tensorflow-estimator-1.15.1\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"application/vnd.colab-display-data+json": {
"pip_warning": {
"packages": [
"gast",
"tensorboard",
"tensorflow"
]
}
}
},
"metadata": {
"tags": []
}
},
{
"output_type": "stream",
"text": [
"Collecting tensorflow-gpu==1.15.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/a5/ad/933140e74973fb917a194ab814785e7c23680ca5dee6d663a509fe9579b6/tensorflow_gpu-1.15.0-cp36-cp36m-manylinux2010_x86_64.whl (411.5MB)\n",
"\u001b[K |████████████████████████████████| 411.5MB 43kB/s \n",
"\u001b[?25hRequirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (0.36.2)\n",
"Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.0.8)\n",
"Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.1.2)\n",
"Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (0.10.0)\n",
"Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (3.3.0)\n",
"Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (0.2.2)\n",
"Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.15.1)\n",
"Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.15.0)\n",
"Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (3.12.4)\n",
"Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.1.0)\n",
"Requirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.19.5)\n",
"Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.32.0)\n",
"Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (0.8.1)\n",
"Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.12.1)\n",
"Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (0.2.0)\n",
"Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.0) (1.15.0)\n",
"Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow-gpu==1.15.0) (2.10.0)\n",
"Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (51.3.3)\n",
"Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (1.0.1)\n",
"Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (3.3.3)\n",
"Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (3.3.0)\n",
"Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (3.4.0)\n",
"Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.0) (3.7.4.3)\n",
"Installing collected packages: tensorflow-gpu\n",
"Successfully installed tensorflow-gpu-1.15.0\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"application/vnd.colab-display-data+json": {
"pip_warning": {
"packages": [
"tensorflow"
]
}
}
},
"metadata": {
"tags": []
}
},
{
"output_type": "stream",
"text": [
"Collecting utils\n",
" Downloading https://files.pythonhosted.org/packages/55/e6/c2d2b2703e7debc8b501caae0e6f7ead148fd0faa3c8131292a599930029/utils-1.0.1-py2.py3-none-any.whl\n",
"Installing collected packages: utils\n",
"Successfully installed utils-1.0.1\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "R-DalbHfjrZ2"
},
"source": [
"import os\n",
"import time\n",
"import utils\n",
"import tensorflow as tf\n",
"import numpy as np\n",
"import skimage.io as io\n",
"import argparse\n",
"\n",
"from tensorflow.python.framework import ops\n",
"ops.reset_default_graph()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "3y4Isl-6cFn2"
},
"source": [
"#from google.colab import drive\n",
"#drive.mount('/content/drive')"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4YCjOIIQcFk5",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "579c3876-829b-4df2-8413-08d3b1470e18"
},
"source": [
"DATA_DIR = '/content/MAPNet/dataset/'\n",
"import os\n",
"# load repo with data if it is not exists\n",
"if not os.path.exists(DATA_DIR):\n",
" print('no data available')\n",
"else:\n",
" print('Done!!')"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Done!!\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_UmAQg4ASFP2"
},
"source": [
"# **Load data**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "GOCrdX8ybFZ8"
},
"source": [
"#load data\n",
"import numpy as np\n",
"import glob\n",
"import scipy\n",
"import random\n",
"import cv2\n",
"import skimage.io\n",
"\n",
"def load_batch(x, y):\n",
" x1 = []\n",
" y1 = []\n",
" for i in range(len(x)):\n",
" img = skimage.io.imread(x[i])\n",
" lab = skimage.io.imread(y[i])\n",
" #ret, lab = cv2.threshold(_lab,0,1,cv2.THRESH_BINARY)\n",
" img, lab = data_augmentation(img, lab)\n",
" lab = lab.reshape(512, 512, 1)\n",
" x1.append(img / 255.0)\n",
" y1.append(lab)\n",
" y1 = np.array(y1).astype(np.float32)\n",
" return x1, y1\n",
"\n",
"\n",
"def prepare_data():\n",
" \n",
" img = np.array(sorted(glob.glob(rf'{DATA_DIR}train/img/*.png')))\n",
" label = np.array(sorted(glob.glob(rf'{DATA_DIR}train/lab/*.png')))\n",
" test_img = np.array(sorted(glob.glob(rf'{DATA_DIR}test/img/*.png')))\n",
" test_label = np.array(sorted(glob.glob(rf'{DATA_DIR}test/lab/*.png')))\n",
"\n",
" print(f\"[DEBUG] train label {label}\")\n",
"\n",
" return img, label, test_img, test_label\n",
"\n",
"\n",
"def data_augmentation(image, label):\n",
" # Data augmentation\n",
" if random.randint(0, 1):\n",
" image = np.fliplr(image)\n",
" label = np.fliplr(label)\n",
" if random.randint(0, 1):\n",
" image = np.flipud(image)\n",
" label = np.flipud(label)\n",
"\n",
" if random.randint(0,1):\n",
" angle = random.randint(0, 3)*90\n",
" if angle!=0:\n",
" M = cv2.getRotationMatrix2D((image.shape[1] // 2, image.shape[0] // 2), angle, 1.0)\n",
" image = cv2.warpAffine(image, M, (image.shape[1], image.shape[0]), flags=cv2.INTER_NEAREST)\n",
" label = cv2.warpAffine(label, M, (label.shape[1], label.shape[0]), flags=cv2.INTER_NEAREST)\n",
"\n",
" return image, label"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "3IUD9qj4SI9j"
},
"source": [
"# **Load MapNet**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "m1Kfv7WxbjTU"
},
"source": [
"# mapnet\n",
"import tensorflow as tf\n",
"\n",
"# from keras.layers import UpSampling2D\n",
"\n",
"\n",
"def conv2d(input,filters,kernel_size=3,stride=1,padding='SAME'):\n",
" return tf.layers.conv2d(input,filters=filters,kernel_size=kernel_size,\n",
" padding=padding,strides=stride,use_bias=False,\n",
" kernel_initializer=tf.variance_scaling_initializer())\n",
"\n",
"\n",
"def bn(input,is_training=True):\n",
" return tf.layers.batch_normalization(input,momentum=0.99,epsilon=1e-3,training=is_training)\n",
"\n",
"\n",
"def bottleneck(x, size,is_training,downsampe=False):\n",
" residual = x\n",
" out = bn(x, is_training)\n",
" out = tf.nn.relu(out)\n",
" out = conv2d(out, size, 1, padding='VALID')\n",
" out = bn(out, is_training)\n",
" out = tf.nn.relu(out)\n",
" out = conv2d(out, size, 3)\n",
" out = bn(out, is_training)\n",
" out = tf.nn.relu(out)\n",
" out = conv2d(out, size * 4, 1, padding='VALID')\n",
"\n",
" if downsampe:\n",
" residual = bn(x, is_training)\n",
" residual = tf.nn.relu(residual)\n",
" residual = conv2d(residual, size * 4, 1, padding='VALID')\n",
" out = tf.add(out,residual)\n",
" return out\n",
"\n",
"\n",
"def resblock(x, size,is_training):\n",
" residual = x\n",
"\n",
" out = bn(x, is_training)\n",
" out = tf.nn.relu(out)\n",
" out = conv2d(out, size, 3)\n",
" out = bn(out, is_training)\n",
" out = tf.nn.relu(out)\n",
" out = conv2d(out, size, 3)\n",
"\n",
" out = tf.add(out, residual)\n",
" return out\n",
"\n",
"\n",
"def stage0(x,is_training):\n",
" x = bottleneck(x, 64,is_training, downsampe=True)\n",
" x = bottleneck(x, 64,is_training)\n",
" x = bottleneck(x, 64,is_training)\n",
" x = bottleneck(x, 64,is_training)\n",
" return x\n",
"\n",
"\n",
"def translayer(x, in_channels, out_channels,is_training):\n",
" num_in = len(in_channels)\n",
" num_out = len(out_channels)\n",
" out = []\n",
" for i in range(num_out):\n",
" if i < num_in:\n",
" residual = bn(x[i], is_training)\n",
" residual = tf.nn.relu(residual)\n",
" residual = conv2d(residual, out_channels[i], 3)\n",
" out.append(residual)\n",
" else:\n",
" residual = bn(x[-1], is_training)\n",
" residual = tf.nn.relu(residual)\n",
" residual = conv2d(residual, out_channels[i], 3, stride=2)\n",
" out.append(residual)\n",
" return out\n",
"\n",
"\n",
"def convb(x, block_num, channels,is_training):\n",
" out = []\n",
" for i in range(len(channels)):\n",
" residual = x[i]\n",
" for j in range(block_num):\n",
" residual = resblock(residual, channels[i],is_training)\n",
" out.append(residual)\n",
" return out\n",
"\n",
"\n",
"def featfuse(x, channels, is_training, multi_scale_output=True):\n",
" out = []\n",
" for i in range(len(channels) if multi_scale_output else 1):\n",
" residual = x[i]\n",
" for j in range(len(channels)):\n",
" if j > i:\n",
" if multi_scale_output == False:\n",
" y = bn(x[j], is_training)\n",
" y = tf.nn.relu(y)\n",
" y = conv2d(y, channels[j], 1, padding='VALID')\n",
" out.append(tf.keras.layers.UpSampling2D(size=2 ** (j - i))(y))\n",
" else:\n",
" y = bn(x[j], is_training)\n",
" y = tf.nn.relu(y)\n",
" y = conv2d(y, channels[i], 1, padding='VALID')\n",
" y = tf.keras.layers.UpSampling2D(size=2 ** (j - i))(y)\n",
" residual = tf.add(residual, y)\n",
"\n",
" elif j < i:\n",
" y = x[j]\n",
" for k in range(i - j):\n",
" if k == i - j - 1:\n",
" y = bn(y, is_training)\n",
" y = tf.nn.relu(y)\n",
" y = conv2d(y, channels[i], 1)\n",
" y = tf.layers.max_pooling2d(y, 2, 2)\n",
"\n",
" else:\n",
" y = bn(y, is_training)\n",
" y = tf.nn.relu(y)\n",
" y = conv2d(y, channels[j], 1)\n",
" y = tf.layers.max_pooling2d(y, 2, 2)\n",
"\n",
" residual = tf.add(residual, y)\n",
" out.append(residual)\n",
" return out\n",
"\n",
"\n",
"def convblock(x, channels,is_training, multi_scale_output=True):\n",
" residual = convb(x, 4, channels,is_training)\n",
" out = featfuse(residual, channels,is_training, multi_scale_output=multi_scale_output)\n",
" return out\n",
"\n",
"\n",
"def stage(x, num_modules, channels, is_training,multi_scale_output=True):\n",
" out = x\n",
" for i in range(num_modules):\n",
" if i == num_modules - 1 and multi_scale_output == False:\n",
" out = convblock(out, channels,is_training, multi_scale_output=False)\n",
" else:\n",
" out = convblock(out, channels,is_training)\n",
" return out\n",
"\n",
"\n",
"def pyramid_pooling_block(input, bin_sizes):\n",
" pool_list = []\n",
" h = input.shape[1]\n",
" c = input.shape[-1]\n",
" for bin_size in bin_sizes:\n",
" pool1 = tf.layers.average_pooling2d(input, (h // bin_size, h // bin_size), (h // bin_size, h // bin_size))\n",
" pool1 = conv2d(pool1, int(c)//4, 1)\n",
" pool1 = tf.image.resize_bilinear(pool1, (h, h))\n",
" pool_list.append(pool1)\n",
" pool = tf.concat(pool_list, axis=3)\n",
" return tf.add(input, pool)\n",
"\n",
"\n",
"def spatial_pooling(input):\n",
" h,w=input.shape[1],input.shape[2]\n",
" p1=tf.image.resize_bilinear(tf.layers.max_pooling2d(input,2,2),(h,w))\n",
" p2 = tf.image.resize_bilinear(tf.layers.max_pooling2d(input, 3, 3), (h, w))\n",
" p3=tf.image.resize_bilinear(tf.layers.max_pooling2d(input,5,5),(h,w))\n",
" p4 = tf.image.resize_bilinear(tf.layers.max_pooling2d(input, 6, 6), (h, w))\n",
" p=tf.concat([p1,p2,p3,p4,input],axis=-1)\n",
" return p\n",
"\n",
"\n",
"def channel_squeeze(input,filters,name=\" \"):\n",
" with tf.name_scope(name):\n",
" squeeze=tf.reduce_mean(input,axis=[1,2])\n",
" with tf.name_scope(name+\"fc1\"):\n",
" fc1=tf.layers.dense(squeeze,use_bias=True,units=filters)\n",
" fc1=tf.nn.relu(fc1)\n",
" with tf.name_scope(name+\"fc2\"):\n",
" fc2=tf.layers.dense(fc1,use_bias=True,units=filters)\n",
" fc2=tf.nn.sigmoid(fc2)\n",
" result=tf.reshape(fc2,[-1,1,1,filters])\n",
" return input*result\n",
"\n",
"\n",
"def mapnet(input, is_training=True):\n",
" channels_s2 = [64, 128]\n",
" channels_s3 = [64, 128, 256]\n",
" num_modules_s2 = 2\n",
" num_modules_s3 = 3\n",
"\n",
" conv_1 = conv2d(input, 64, stride=2)\n",
" conv_1 = bn(conv_1, is_training)\n",
" conv_1 = tf.nn.relu(conv_1)\n",
" conv_2 = conv2d(conv_1, 64)\n",
" conv_2 = bn(conv_2, is_training)\n",
" conv_2 = tf.nn.relu(conv_2)\n",
" conv_3 = conv2d(conv_2, 64)\n",
" conv_3 = bn(conv_3, is_training)\n",
" conv_3 = tf.nn.relu(conv_3)\n",
" conv_4 = tf.layers.max_pooling2d(conv_3, 2, 2)\n",
"\n",
" stage1 = stage0(conv_4,is_training)\n",
" trans1 = translayer([stage1], [256], channels_s2,is_training)\n",
" stage2 = stage(trans1, num_modules_s2, channels_s2,is_training)\n",
" trans2 = translayer(stage2, channels_s2, channels_s3,is_training)\n",
" stage3 = stage(trans2, num_modules_s3, channels_s3,is_training,multi_scale_output=False)\n",
"\n",
" stg3=tf.concat(stage3,axis=-1)\n",
" squeeze=channel_squeeze(stg3, stg3.shape[-1], name=\"squeeze\")\n",
"\n",
" spatial=tf.concat([stage3[0],stage3[1]],axis=-1)\n",
" # spatial=pyramid_pooling_block(spatial, [1, 2, 4, 8])\n",
" spatial=spatial_pooling(spatial)\n",
"\n",
" new_feature = tf.concat([spatial, squeeze], axis=-1)\n",
" new_feature = bn(new_feature, is_training)\n",
" new_feature = tf.nn.relu(new_feature)\n",
" result=conv2d(new_feature, 128, 1, padding='SAME')\n",
"\n",
" up1=tf.image.resize_bilinear(result,size=(stage3[0].shape[1]*2,stage3[0].shape[2]*2))\n",
" up1 = bn(up1, is_training)\n",
" up1 = tf.nn.relu(up1)\n",
" up1 = conv2d(up1, 64, 3)\n",
"\n",
" up2 = tf.image.resize_bilinear(up1, size=(up1.shape[1]*2, up1.shape[2]*2))\n",
" up2 = bn(up2, is_training)\n",
" up2 = tf.nn.relu(up2)\n",
" up2 = conv2d(up2, 32, 3)\n",
"\n",
" up2 = bn(up2, is_training)\n",
" up2 = tf.nn.relu(up2)\n",
" final = conv2d(up2, 1, 1, padding='valid')\n",
"\n",
" return final"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "UFEOU_1YSW4P"
},
"source": [
"# **Train**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "RF81kkV8by-h",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "4adba4d4-8d04-4509-8730-51cc85a673b3"
},
"source": [
"parser = argparse.ArgumentParser()\n",
"parser.add_argument('--batch_size', type=int, default=4, help='Number of images in each batch')\n",
"parser.add_argument('--learning_rate', type=float, default=0.001, help='Number of images in each batch')\n",
"parser.add_argument('--crop_height', type=int, default=512, help='Height of cropped input image to network')\n",
"parser.add_argument('--crop_width', type=int, default=512, help='Width of cropped input image to network')\n",
"parser.add_argument('--clip_size', type=int, default=450, help='Width of cropped input image to network')\n",
"parser.add_argument('--num_epochs', type=int, default=200, help='Number of epochs to train for')\n",
"parser.add_argument('--h_flip', type=bool, default=True, help='Whether to randomly flip the image horizontally for data augmentation')\n",
"parser.add_argument('--v_flip', type=bool, default=True, help='Whether to randomly flip the image vertically for data augmentation')\n",
"parser.add_argument('--color', type=bool, default=True, help='Whether to randomly flip the image vertically for data augmentation')\n",
"parser.add_argument('--rotation', type=bool, default=True, help='randomly rotate, the imagemax rotation angle in degrees.')\n",
"parser.add_argument('--start_valid', type=int, default=20, help='Number of epoch to valid')\n",
"parser.add_argument('--valid_step', type=int, default=1, help=\"Number of step to validation\")\n",
"parser.add_argument('-f')\n",
"\n",
"args = parser.parse_args()\n",
"num_images=[]\n",
"train_img, train_label,valid_img,valid_lab= prepare_data()\n",
"num_batches=len(train_img)//(args.batch_size)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"[DEBUG] train label ['/content/MAPNet/dataset/train/lab/7000000.png'\n",
" '/content/MAPNet/dataset/train/lab/7000001.png'\n",
" '/content/MAPNet/dataset/train/lab/7000002.png'\n",
" '/content/MAPNet/dataset/train/lab/7000003.png'\n",
" '/content/MAPNet/dataset/train/lab/7000006.png'\n",
" '/content/MAPNet/dataset/train/lab/7000007.png']\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "j0EaxcHnkB7z",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "b6ba7e7f-f467-4618-aefb-e5b493c35b05"
},
"source": [
"import tensorflow.compat.v1 as tf\n",
"tf.disable_v2_behavior()"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/compat/v2_compat.py:68: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"non-resource variables are not supported in the long term\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "OEdHFizTg9KZ",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "8429650b-12a2-40c6-976b-916108bf415e"
},
"source": [
"img=tf.placeholder(tf.float32,[None,args.crop_height,args.crop_width,3])\n",
"is_training=tf.placeholder(tf.bool)\n",
"label=tf.placeholder(tf.float32,[None,args.crop_height,args.crop_height,1])\n",
"\n",
"pred=mapnet(img,is_training)\n",
"pred1=tf.nn.sigmoid(pred)\n",
"\n",
"update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)\n",
"with tf.control_dependencies(update_ops):\n",
"\n",
" sig=tf.nn.sigmoid_cross_entropy_with_logits(labels=label, logits=pred)\n",
" sigmoid_cross_entropy_loss = tf.reduce_mean(sig)\n",
" train_step = tf.train.AdamOptimizer(args.learning_rate).minimize(sigmoid_cross_entropy_loss)\n",
"saver=tf.train.Saver(var_list=tf.global_variables())"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"WARNING:tensorflow:From <ipython-input-5-474ff2aaaabc>:10: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use `tf.keras.layers.Conv2D` instead.\n",
"WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Please use `layer.__call__` method instead.\n",
"WARNING:tensorflow:From <ipython-input-5-474ff2aaaabc>:14: batch_normalization (from tensorflow.python.layers.normalization) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use keras.layers.BatchNormalization instead. In particular, `tf.control_dependencies(tf.GraphKeys.UPDATE_OPS)` should not be used (consult the `tf.keras.layers.batch_normalization` documentation).\n",
"WARNING:tensorflow:From <ipython-input-5-474ff2aaaabc>:192: max_pooling2d (from tensorflow.python.layers.pooling) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use keras.layers.MaxPooling2D instead.\n",
"WARNING:tensorflow:From <ipython-input-5-474ff2aaaabc>:168: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use keras.layers.Dense instead.\n",
"WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/nn_impl.py:183: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use tf.where in 2.0, which has the same broadcast rule as np.where\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "deyZ5-olgNmr"
},
"source": [
"history = {\n",
" 'train': {\n",
" 'loss' : [],\n",
" 'iou' : [],\n",
" 'iter' : []\n",
" },\n",
" 'val': {\n",
" 'loss' : [],\n",
" 'iou' : [],\n",
" 'iter' : []\n",
" }\n",
"}\n",
"\n",
"CHECKPOINTS_DIR = './'"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "fbvtMhxJd1ec"
},
"source": [
"from tqdm import tqdm\n",
"\n",
"def load():\n",
" import re\n",
" print(\"[INFO] Reading checkpoints dir...\")\n",
" checkpoint_dir = CHECKPOINTS_DIR\n",
" \n",
" ckpt = tf.train.get_checkpoint_state(checkpoint_dir)\n",
" if ckpt and ckpt.model_checkpoint_path:\n",
" ckpt_name = os.path.basename(ckpt.model_checkpoint_path)\n",
" saver.restore(sess, os.path.join(checkpoint_dir, ckpt_name))\n",
" counter = int(next(re.finditer(\"(\\d+)(?!.*\\d)\",ckpt_name)).group(0))\n",
" print(\"[INFO] Checkpoint {} read successed\".format(ckpt_name))\n",
" return True, counter\n",
" else:\n",
" print(\"[INFO] Checkpoint not found\")\n",
" return False, 0\n",
"\n",
"def train():\n",
"\n",
" tf.global_variables_initializer().run()\n",
"\n",
" could_load, checkpoint_counter = load()\n",
" if could_load:\n",
" start_epoch = (int)(checkpoint_counter / num_batches)\n",
" start_batch_id = checkpoint_counter - start_epoch * num_batches\n",
" counter = checkpoint_counter\n",
" print(\"[INFO] Checkpoint Load Success!\")\n",
"\n",
" else:\n",
" start_epoch = 0\n",
" start_batch_id = 0\n",
" counter = 1\n",
" print(\"[INFO] Checkpoint load failed. Training from scratch...\")\n",
"\n",
" train_iter=[]\n",
" train_loss=[]\n",
" IOU=0.65\n",
"\n",
" print(\"==================================================================\")\n",
" print(\"[INFO] GENERAL INFORMATION\")\n",
" print(\"==================================================================\")\n",
" # utils.count_params()\n",
" print(\"Total train image:{}\".format(len(train_img)))\n",
" print(\"Total validate image:{}\".format(len(valid_img)))\n",
" print(\"Total epoch:{}\".format(args.num_epochs))\n",
" print(\"Batch size:{}\".format(args.batch_size))\n",
" print(\"Learning rate:{}\".format(args.learning_rate))\n",
" #print(\"Checkpoint step:{}\".format(args.checkpoint_step))\n",
"\n",
" print(\"==================================================================\")\n",
" print(\"[INFO] DATA AUGMENTATION\")\n",
" print(\"==================================================================\")\n",
" print(\"h_flip: {}\".format(args.h_flip))\n",
" print(\"v_flip: {}\".format(args.v_flip))\n",
" print(\"rotate: {}\".format(args.rotation))\n",
" print(\"clip size: {}\".format(args.clip_size))\n",
"\n",
" print(\"==================================================================\")\n",
" print(\"[INFO] TRAINING STARTED\")\n",
" print(\"==================================================================\")\n",
"\n",
" loss_tmp = []\n",
" train_iou_temp = []\n",
" \n",
" # -----------------------------------------------------------------------------------\n",
" # beg: epoch\n",
" # -----------------------------------------------------------------------------------\n",
" # Test with dummy arg values\n",
" # args.num_epochs = 15\n",
" # args.start_valid = 5\n",
" for i in range(start_epoch, args.num_epochs):\n",
" \n",
"\n",
" # -------------------------------------------------------------------------------------------------------\n",
" # beg: batches\n",
" # -------------------------------------------------------------------------------------------------------\n",
" epoch_time=time.time()\n",
" id_list = np.random.permutation(len(train_img))\n",
" batch_pbar = tqdm(range(start_batch_id, num_batches), desc=f\"[TRAIN] Epoch {i}\")\n",
" for j in batch_pbar:\n",
"\n",
" img_d = []\n",
" lab_d = []\n",
" for ind in range(args.batch_size):\n",
" id = id_list[j * args.batch_size + ind]\n",
" img_d.append(train_img[id])\n",
" lab_d.append(train_label[id])\n",
" x_batch, y_batch = load_batch(img_d, lab_d)\n",
" # print(f\"[DEBUG] {x_batch[0].shape} {y_batch[0].shape}\")\n",
" # (512, 512, 3) (512, 512, 1)\n",
"\n",
" feed_dict = {img: x_batch,\n",
" label: y_batch,\n",
" is_training:True}\n",
"\n",
" _, loss, pred1 = sess.run([train_step, sigmoid_cross_entropy_loss, pred], feed_dict=feed_dict)\n",
"\n",
" inter=0\n",
" unin=0\n",
" for idx, _ in enumerate(pred1):\n",
" pred1_ = np.squeeze(pred1[idx], 2)\n",
" pred1_[pred1_ < 0.5] = 0\n",
" pred1_[pred1_ >= 0.5] = 1\n",
" gt_value=io.imread(lab_d[idx])\n",
" #print(gt_value.shape,pred1_.shape)\n",
" intr,unn=f_iou(gt_value,pred1_)\n",
" inter=inter+intr\n",
" unin=unin+unn\n",
"\n",
" train_iou_temp.append((inter*1.0)/(unin+1e-10))\n",
" loss_tmp.append(loss)\n",
" if (j == num_batches-1):\n",
"\n",
" tmp = np.median(loss_tmp)\n",
" iou_tmp = np.median(train_iou_temp)\n",
" \n",
" history['train']['iter'].append(i)\n",
" history['train']['iou'].append(iou_tmp)\n",
" history['train']['loss'].append(tmp)\n",
" #train_iter.append(counter)\n",
" #train_loss.append(tmp)\n",
" #print('Epoch', i, '|Iter', counter, '|Loss', tmp)\n",
" batch_pbar.set_description(f\"[TRAIN] Epoch {i} --- Iter {counter} --- Loss {tmp}\")\n",
" \n",
" loss_tmp.clear()\n",
" train_iou_temp.clear()\n",
" \n",
"\n",
" counter += 1\n",
" start_batch_id = 0\n",
" # print(f'[DEBUG] Time taken for epoch {i}: {time.time() - epoch_time:.3f} seconds')\n",
" # saver.save(sess, './ckeckpoint_10epoch_new/model.ckpt', global_step=counter)\n",
" # -------------------------------------------------------------------------------------------------------\n",
" # end: batches\n",
" # -------------------------------------------------------------------------------------------------------\n",
"\n",
"\n",
" # -------------------------------------------------------------------------------------------------------\n",
" # beg: val for epoch\n",
" # -------------------------------------------------------------------------------------------------------\n",
" if (i>args.start_valid):\n",
" if (i-args.start_valid)%args.valid_step==0:\n",
" val_iou, val_loss = validation()\n",
" #print(f\"[INFO] current val loss: {val_loss}\")\n",
" #print(f\"[INFO] last iou valu: {IOU}\")\n",
" #print(f\"[INFO] new_iou value: {val_iou}\")\n",
" history['val']['iter'].append(i)\n",
" history['val']['iou'].append(val_iou)\n",
" history['val']['loss'].append(val_loss)\n",
" # saving best model based on best IOU score.\n",
" # Can do based on best val_loss instead too!\n",
" if val_iou > IOU:\n",
" print(f\"[INFO] Saving best model as checkpoint... val_iou: {val_iou}\")\n",
" saver.save(sess, f'{CHECKPOINTS_DIR}model.ckpt', global_step=counter, write_meta_graph=True)\n",
" IOU = val_iou\n",
" # -------------------------------------------------------------------------------------------------------\n",
" # end: val for epoch\n",
" # -------------------------------------------------------------------------------------------------------\n",
"\n",
"\n",
" # -----------------------------------------------------------------------------------\n",
" # end: epoch\n",
" # -----------------------------------------------------------------------------------\n",
" saver.save(sess, f'{CHECKPOINTS_DIR}model.ckpt', global_step=counter)\n",
"\n",
"\n",
"\n",
"def f_iou(predict, label):\n",
"\n",
" tp = np.sum(np.logical_and(predict == 1, label == 1))\n",
" fp = np.sum(predict==1)\n",
" fn = np.sum(label == 1)\n",
" return tp,fp+fn-tp\n",
"\n",
"\n",
"\n",
"def validation():\n",
"\n",
" #print(\"[INFO] Validating ...\")\n",
" inter=0\n",
" unin=0\n",
" loss_accumulator = []\n",
"\n",
" batch_pbar = tqdm(range(0,len(valid_img)), desc=f\"Validating -- \")\n",
" for j in batch_pbar:\n",
" x_batch = valid_img[j]\n",
" x_batch = io.imread(x_batch) / 255.0\n",
" x_batch = np.expand_dims(x_batch, axis=0)\n",
" y_actual_batch = np.expand_dims(io.imread(valid_lab[j]), axis=0)\n",
" # print(f\"[DEBUG] {x_batch.shape} {y_actual_batch.shape}\")\n",
" # (1, 512, 512, 3) (1, 512, 512) \n",
" y_actual_batch = np.expand_dims(y_actual_batch, axis=-1) \n",
" # (1, 512, 512) > (1, 512, 512, 1)\n",
"\n",
" feed_dict = {img: x_batch,\n",
" label: y_actual_batch,\n",
" is_training:False}\n",
"\n",
" #predict = sess.run(pred1, feed_dict=feed_dict)\n",
" _, loss, predict = sess.run([train_step, sigmoid_cross_entropy_loss, pred1], feed_dict=feed_dict)\n",
" loss_accumulator.append(loss)\n",
" \n",
" predict[predict < 0.5] = 0\n",
" predict[predict >= 0.5] = 1\n",
" result = np.squeeze(predict)\n",
" gt_value=io.imread(valid_lab[j])\n",
" #print(f'[ACT] {gt_value.shape} and {result.shape}')\n",
" intr,unn=f_iou(gt_value,result)\n",
"\n",
" inter=inter+intr\n",
" unin=unin+unn\n",
"\n",
" if j == (len(valid_img) - 1):\n",
" batch_pbar.set_description(f\"[VALID] --- Loss {np.median(loss_accumulator):.4f} --- IOU {(inter*1.0)/(unin+1e-10):.4f}\")\n",
"\n",
" iou = (inter*1.0)/(unin+1e-10)\n",
" loss = np.median(loss_accumulator)\n",
" return iou, loss\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "qH24Mi-ceC67",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "051751f9-c008-403f-9f38-f006dd1405df"
},
"source": [
"with tf.Session() as sess:\n",
" train()"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"\n",
"[TRAIN] Epoch 0: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"[INFO] Reading checkpoints dir...\n",
"[INFO] Checkpoint not found\n",
"[INFO] Checkpoint load failed. Training from scratch...\n",
"==================================================================\n",
"[INFO] GENERAL INFORMATION\n",
"==================================================================\n",
"Total train image:6\n",
"Total validate image:5\n",
"Total epoch:10\n",
"Batch size:4\n",
"Learning rate:0.001\n",
"==================================================================\n",
"[INFO] DATA AUGMENTATION\n",
"==================================================================\n",
"h_flip: True\n",
"v_flip: True\n",
"rotate: True\n",
"clip size: 450\n",
"==================================================================\n",
"[INFO] TRAINING STARTED\n",
"==================================================================\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"\n",
"[TRAIN] Epoch 0 --- Iter 1 --- Loss 0.9398379921913147: 0%| | 0/1 [00:53<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 0 --- Iter 1 --- Loss 0.9398379921913147: 100%|██████████| 1/1 [00:53<00:00, 53.06s/it]\n",
"\n",
"[TRAIN] Epoch 1: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 1 --- Iter 2 --- Loss 0.8592573404312134: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 1 --- Iter 2 --- Loss 0.8592573404312134: 100%|██████████| 1/1 [00:00<00:00, 1.08it/s]\n",
"\n",
"[TRAIN] Epoch 2: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 2 --- Iter 3 --- Loss 0.6925444006919861: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 2 --- Iter 3 --- Loss 0.6925444006919861: 100%|██████████| 1/1 [00:00<00:00, 1.08it/s]\n",
"\n",
"[TRAIN] Epoch 3: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 3 --- Iter 4 --- Loss 0.6992093324661255: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 3 --- Iter 4 --- Loss 0.6992093324661255: 100%|██████████| 1/1 [00:00<00:00, 1.07it/s]\n",
"\n",
"[TRAIN] Epoch 4: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 4 --- Iter 5 --- Loss 0.6515692472457886: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 4 --- Iter 5 --- Loss 0.6515692472457886: 100%|██████████| 1/1 [00:00<00:00, 1.07it/s]\n",
"\n",
"[TRAIN] Epoch 5: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 5 --- Iter 6 --- Loss 0.607520341873169: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 5 --- Iter 6 --- Loss 0.607520341873169: 100%|██████████| 1/1 [00:00<00:00, 1.07it/s]\n",
"\n",
"[TRAIN] Epoch 6: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 6 --- Iter 7 --- Loss 0.5781429409980774: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 6 --- Iter 7 --- Loss 0.5781429409980774: 100%|██████████| 1/1 [00:00<00:00, 1.09it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:55<03:43, 55.79s/it]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:56<01:57, 39.13s/it]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:56<00:54, 27.47s/it]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:56<00:19, 19.31s/it]\u001b[A\n",
"[VALID] --- Loss 3.4356 --- IOU 0.0467: 80%|████████ | 4/5 [00:56<00:19, 19.31s/it]\u001b[A\n",
"[VALID] --- Loss 3.4356 --- IOU 0.0467: 100%|██████████| 5/5 [00:56<00:00, 11.37s/it]\n",
"\n",
"[TRAIN] Epoch 7: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 7 --- Iter 8 --- Loss 0.8831216096878052: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 7 --- Iter 8 --- Loss 0.8831216096878052: 100%|██████████| 1/1 [00:00<00:00, 1.08it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:00<00:01, 3.66it/s]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:00<00:00, 3.72it/s]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:00<00:00, 3.77it/s]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:01<00:00, 3.79it/s]\u001b[A\n",
"[VALID] --- Loss 0.6726 --- IOU 0.0000: 80%|████████ | 4/5 [00:01<00:00, 3.79it/s]\u001b[A\n",
"[VALID] --- Loss 0.6726 --- IOU 0.0000: 100%|██████████| 5/5 [00:01<00:00, 3.79it/s]\n",
"\n",
"[TRAIN] Epoch 8: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 8 --- Iter 9 --- Loss 0.8098524808883667: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 8 --- Iter 9 --- Loss 0.8098524808883667: 100%|██████████| 1/1 [00:00<00:00, 1.08it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:00<00:01, 3.87it/s]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:00<00:00, 3.88it/s]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:00<00:00, 3.88it/s]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:01<00:00, 3.86it/s]\u001b[A\n",
"[VALID] --- Loss 0.3067 --- IOU 0.0000: 80%|████████ | 4/5 [00:01<00:00, 3.86it/s]\u001b[A\n",
"[VALID] --- Loss 0.3067 --- IOU 0.0000: 100%|██████████| 5/5 [00:01<00:00, 3.80it/s]\n",
"\n",
"[TRAIN] Epoch 9: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 9 --- Iter 10 --- Loss 0.6901130676269531: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 9 --- Iter 10 --- Loss 0.6901130676269531: 100%|██████████| 1/1 [00:00<00:00, 1.07it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:00<00:01, 3.79it/s]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:00<00:00, 3.81it/s]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:00<00:00, 3.82it/s]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:01<00:00, 3.81it/s]\u001b[A\n",
"[VALID] --- Loss 0.3549 --- IOU 0.0000: 80%|████████ | 4/5 [00:01<00:00, 3.81it/s]\u001b[A\n",
"[VALID] --- Loss 0.3549 --- IOU 0.0000: 100%|██████████| 5/5 [00:01<00:00, 3.80it/s]\n",
"\n",
"[TRAIN] Epoch 10: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 10 --- Iter 11 --- Loss 0.6336275935173035: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 10 --- Iter 11 --- Loss 0.6336275935173035: 100%|██████████| 1/1 [00:00<00:00, 1.07it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:00<00:01, 3.81it/s]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:00<00:00, 3.83it/s]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:00<00:00, 3.82it/s]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:01<00:00, 3.82it/s]\u001b[A\n",
"[VALID] --- Loss 0.2850 --- IOU 0.0000: 80%|████████ | 4/5 [00:01<00:00, 3.82it/s]\u001b[A\n",
"[VALID] --- Loss 0.2850 --- IOU 0.0000: 100%|██████████| 5/5 [00:01<00:00, 3.81it/s]\n",
"\n",
"[TRAIN] Epoch 11: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 11 --- Iter 12 --- Loss 0.565162181854248: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 11 --- Iter 12 --- Loss 0.565162181854248: 100%|██████████| 1/1 [00:00<00:00, 1.08it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:00<00:01, 3.81it/s]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:00<00:00, 3.81it/s]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:00<00:00, 3.83it/s]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:01<00:00, 3.82it/s]\u001b[A\n",
"[VALID] --- Loss 0.4215 --- IOU 0.0000: 80%|████████ | 4/5 [00:01<00:00, 3.82it/s]\u001b[A\n",
"[VALID] --- Loss 0.4215 --- IOU 0.0000: 100%|██████████| 5/5 [00:01<00:00, 3.79it/s]\n",
"\n",
"[TRAIN] Epoch 12: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 12 --- Iter 13 --- Loss 0.5476508140563965: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 12 --- Iter 13 --- Loss 0.5476508140563965: 100%|██████████| 1/1 [00:00<00:00, 1.05it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:00<00:01, 3.61it/s]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:00<00:00, 3.67it/s]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:00<00:00, 3.72it/s]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:01<00:00, 3.72it/s]\u001b[A\n",
"[VALID] --- Loss 0.2783 --- IOU 0.0000: 80%|████████ | 4/5 [00:01<00:00, 3.72it/s]\u001b[A\n",
"[VALID] --- Loss 0.2783 --- IOU 0.0000: 100%|██████████| 5/5 [00:01<00:00, 3.70it/s]\n",
"\n",
"[TRAIN] Epoch 13: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 13 --- Iter 14 --- Loss 0.529972493648529: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 13 --- Iter 14 --- Loss 0.529972493648529: 100%|██████████| 1/1 [00:00<00:00, 1.07it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:00<00:01, 3.75it/s]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:00<00:00, 3.79it/s]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:00<00:00, 3.79it/s]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:01<00:00, 3.79it/s]\u001b[A\n",
"[VALID] --- Loss 0.4133 --- IOU 0.0007: 80%|████████ | 4/5 [00:01<00:00, 3.79it/s]\u001b[A\n",
"[VALID] --- Loss 0.4133 --- IOU 0.0007: 100%|██████████| 5/5 [00:01<00:00, 3.76it/s]\n",
"\n",
"[TRAIN] Epoch 14: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 14 --- Iter 15 --- Loss 0.5296635031700134: 0%| | 0/1 [00:00<?, ?it/s]\u001b[A\n",
"[TRAIN] Epoch 14 --- Iter 15 --- Loss 0.5296635031700134: 100%|██████████| 1/1 [00:00<00:00, 1.07it/s]\n",
"\n",
"Validating -- : 0%| | 0/5 [00:00<?, ?it/s]\u001b[A\n",
"Validating -- : 20%|██ | 1/5 [00:00<00:01, 3.74it/s]\u001b[A\n",
"Validating -- : 40%|████ | 2/5 [00:00<00:00, 3.77it/s]\u001b[A\n",
"Validating -- : 60%|██████ | 3/5 [00:00<00:00, 3.77it/s]\u001b[A\n",
"Validating -- : 80%|████████ | 4/5 [00:01<00:00, 3.77it/s]\u001b[A\n",
"[VALID] --- Loss 0.2580 --- IOU 0.0000: 80%|████████ | 4/5 [00:01<00:00, 3.77it/s]\u001b[A\n",
"[VALID] --- Loss 0.2580 --- IOU 0.0000: 100%|██████████| 5/5 [00:01<00:00, 3.72it/s]\n"
],
"name": "stderr"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "zR4gtIDtz6Iq"
},
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"def plot_curves():\n",
" fig = plt.figure(figsize=(10,7))\n",
" \n",
" plt.plot(history['val']['iter'], history['val']['loss'], label='Val Loss')\n",
" plt.plot(history['val']['iter'], history['val']['iou'], label='Val IOU')\n",
"\n",
" plt.plot(history['train']['iter'], history['train']['loss'], label='Train Loss')\n",
" plt.plot(history['train']['iter'], history['train']['iou'], label='Train IOU')\n",
"\n",
" plt.xlabel(\"Loss\")\n",
" plt.ylabel(\"Epoch Number\")\n",
" plt.legend()\n",
" plt.grid()\n",
" plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 442
},
"id": "D8DwQ4HH4b-i",
"outputId": "51d8b36f-560e-45b6-ba60-e0effbfe6317"
},
"source": [
"plot_curves()"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAGpCAYAAADFpuEPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXzU1b3/8dfJZEhmsjGThADJhLAoOwkQQeWqQW/dr9RetRUVkFbrUq16W2vtrXqt3nvbeu9Vq3W5KtSVWr0urfJzJVVrlU1QwibIkgAiJCF7Qpbz+2OSIQkhC2QyM8n7+XjMIzPf+S6fOYTknfM93/M11lpEREREpG9FhboAERERkYFIIUxEREQkBBTCREREREJAIUxEREQkBBTCREREREIgOtQF9FRKSorNysoK+nGqqqqIi4sL+nEildqnc2qfrqmNOqf26ZraqHNqn671RRutWrVqv7U2taP3Ii6EZWVlsXLlyqAfJz8/n7y8vKAfJ1KpfTqn9uma2qhzap+uqY06p/bpWl+0kTFmx5He0+lIERERkRBQCBMREREJAYUwERERkRCIuDFhIiIicuzq6+spKiqitrY21KWETFJSEhs2bOiVfcXGxpKRkYHT6ez2NgphIiIiA1BRUREJCQlkZWVhjAl1OSFRUVFBQkLCMe/HWktxcTFFRUWMHDmy29vpdKSIiMgAVFtbS3Jy8oANYL3JGENycnKPexWDFsKMMbHGmOXGmLXGmAJjzL91sM4CY8w+Y8ya5scPglWPiIiItKUA1nuOpi2DeTqyDjjdWltpjHECHxljllprP2m33h+ttT8KYh0iIiIiYSdoPWHWr7L5pbP5YYN1PBEREYkcs2fP5q233mqz7P777+faa6894jZ5eXkdTth+pOXhLqgD840xDmAVMAZ42Fr7aQer/bMx5lRgM3Cztbawg/1cDVwNkJaWRn5+fvCKblZZWdknx4lUap/OqX26pjbqnNqna2qjznXVPklJSVRUVPRdQe1ceOGFPPPMM5x88smBZc899xy/+tWvjlhXY2MjVVVVh71/pOVdaWxs7NU2qK2t7dn3pLU26A9gMLAMmNRueTIQ0/z8h8D7Xe1r+vTpti8sW7asT44TqdQ+nVP7dE1t1Dm1T9fURp3rqn3Wr1/fN4UcQXFxsU1NTbV1dXXWWmu3bdtmfT6fbWpqstdcc42dPn26nTBhgr3jjjsC25x22ml2xYoVh+2ro+XFxcV2zpw5dvLkyXbmzJl27dq11lpr8/PzbXZ2ts3OzrZTpkyx5eXldvfu3faUU06x2dnZduLEifaDDz44qs/UUZsCK+0RMk2fTFFhrT1gjFkGnA2sa7W8uNVqTwC/6Yt6RERE5JB/+3MB63eX9+o+JwxP5M5/mnjE971eLzNmzGDp0qXMmTOHJUuWcMkll2CM4d5778Xr9dLY2MgZZ5zB559/zpQpU3p0/DvvvJOpU6fy6quv8v777zNv3jzWrFnDfffdx8MPP8ysWbPYs2cPLpeLxx9/nLPOOotf/OIXNDY2Ul1dfawfv1uCeXVkqjFmcPNzF/AtYGO7dYa1enkB0DszpomIiEjYu/TSS1myZAkAS5Ys4dJLLwXgxRdfZNq0aUydOpWCggLWr1/f431/9NFHXHHFFQCcfvrpFBcXU15ezqxZs7jlllt48MEHKSsrIzo6mhNOOIFFixZx11138cUXX/TK3GHdEcyesGHAH5rHhUUBL1pr/2KMuRt/19zrwI3GmAuABqAEWBDEekRERKQDnfVYBdOcOXO4+eabWb16NdXV1UyfPp1t27Zx3333sWLFCjweDwsWLOjVWf1vu+02zjvvPN58803OPPNM3n77bU499VQ++OAD3njjDRYsWMAtt9zCvHnzeu2YRxK0EGat/RyY2sHyO1o9/znw82DVICLhafeBGr6uagp1GSISYvHx8cyePZuFCxcGesHKy8uJi4sjKSmJvXv3snTpUvLy8nq871NOOYXnnnuOX/7yl+Tn55OSkkJiYiJbt25l8uTJTJ48mb///e9s3LgRl8tFRkYGV111FXV1daxevTqyQ5iIyJFc9fRKHAcP8r3zQl2JiITapZdeyoUXXhg4LZmdnc3UqVMZN24cPp+PWbNmdWs/5513XuC+jSeddBKPPfYYCxcuZMqUKbjdbv7whz8A/mkwli1bRlRUFMcffzznnHMOS5Ys4be//S1Op5P4+Hiefvrp4HzYdhTCRKTPZXrdfLYtdJfGi0j4+Pa3v90yY0LA4sWLO1z3SNM/HGn5q6++etiy3/3ud4HnFRUVxMTEMH/+fObPn9+tenuT7h0pIn3O53Wzv8bS1KT5m0Vk4FIIE5E+5/O4aGiCfZV1oS5FRCRkFMJEpM/5vG4Adpb0zVw8IiLhSCFMRPpcSwgrVAgTkQFMIUxE+lz6YBcAhSU1Ia5ERCR0FMJEpM/FOh14YoxOR4rIgKYQJiIhkeo2FJYqhIkMVLNnz+att95qs+z+++/n2muvPeI2eXl5rFy5stPlZWVlzJs3jzFjxjB69GjmzZtHWVkZ4J/K4vzzz2+z7YIFC3jppZeO9eMcFYUwEQmJFFcUReoJExmwWt83skXr+0cere9///uMGjWKLVu2sHXrVkaOHMkPfvCDY9pnsCiEiUhIpLoMe8prqWtoDHUpIhICF110EW+88QYHDx4EYPv27ezevZtTTjmFa6+9ltzcXCZOnMidd97Z7X1u2bKFVatW8ctf/jKw7I477mDlypVs3bq11z/DsdKM+SISEqlug7Ww+0AtI1PiQl2OyMC29Db4+ove3efQyXDOfx7xba/Xy4wZM1i6dClz5sxhyZIlXHLJJRhjuPfee/F6vTQ2NnLGGWfw+eefM2XKlC4PuX79enJycnA4HIFlDoeDnJwcCgoKSExM7JWP1lvUEyYiIZHq8v/40TQVIgNX61OSrU9Fvvjii0ybNo2pU6dSUFDA+vXre+V4xpgeLQ829YSJSEikuv0/9HSFpEgY6KTHKpjmzJnDzTffzOrVq6murmb69Ols27aN++67jxUrVuDxeFiwYAG1tbXd2t+ECRNYs2YNTU1NREX5/9BrampizZo1TJgwgdraWkpLS9tsU1JSQkpKSq9/tu5QT5iIhMTgGMMgR5SukBQZwOLj45k9ezYLFy4M9IKVl5cTFxdHUlISe/fuZenSpd3e35gxY5g6dSr33HNPYNk999zDtGnTGDNmDMcddxy7d+9mw4YNAOzcuZO1a9eSk5PTux+sm9QTJiIhEWUM6R4XRZqwVWRAu/TSS7nwwgsDpyWzs7OZOnUq48aNw+fzMWvWrB7t78knn+SGG25g9OjRAJx00kk8+eSTAMTExPDss89y5ZVXUltbS1RUFE888QRJSUm9+6G6SSFMRELG53XrdKTIAPftb38ba22bZYsXL+5w3fz8/C6Xezwenn322SMeb9asWXzyyScAVFRUkJCQ0KN6e5NOR4pIyPg8Lp2OFJEBSyFMRELG53VzoLqe8tr6UJciItLnFMJEJGQyvW5A01SIyMCkECYiIePztIQwDc4XkYFHIUxEQsbndQFQpHFhIjIAKYSJSMgkuZwkxEbrCkkRGZAUwkQkZIwx+DxujQkTGYCKi4vJyckhJyeHoUOHkp6eHnjdclPvI1m5ciU33nhjj46XlZXF/v37j6XkXqd5wkQkpHxeF1u+qQx1GSLSx5KTk1mzZg0Ad911F/Hx8fzkJz8JvN/Q0EB0dMcxJTc3l9zc3D6pM5jUEyYiIZXpdVNUWnPYZI0iMvAsWLCAa665hpkzZ3LrrbeyfPlyTjrpJKZOncrJJ5/Mpk2bAP/krOeffz7gD3ALFy4kLy+PUaNG8eCDD3b7eDt27OD0009nypQpnHHGGezcuROAP/3pT0yaNIns7GxOPfVUAAoKCpgxYwY5OTlMmTKFL7/88pg/r3rCRCSkfF43dQ1N7KuoY0hibKjLERmQfr3812ws2dir+xznHcfPZvysx9sVFRXx8ccf43A4KC8v58MPPyQ6Opp3332X22+/nZdffvmwbTZu3MiyZcuoqKhg7NixXHvttTidzi6P9dOf/pT58+czf/58nnrqKW688UZeffVV7r77bt566y3S09M5cOAAAI8++ig//vGPueyyyzh48CCNjY09/mztKYSJSEi1TFOxs6RaIUxEuPjii3E4HACUlZUxf/58vvzyS4wx1Nd3PLHzeeedR0xMDDExMQwZMoS9e/eSkZHR5bGWL1/O66+/DsAVV1zBrbfeCvhvbbRgwQIuueQSvvOd7wD+e1Dee++9FBUV8Z3vfIfjjjvumD+rQpiIhJSvZcLW0mpys7whrkZkYDqaHqtgiYuLCzz/5S9/yezZs3nllVfYvn07eXl5HW4TExMTeO5wOGhoaDimGh599FE+/fRT3njjDaZPn86qVauYO3cuM2fO5I033uDcc8/lscce4/TTTz+m42hMmIiEVIbHP1eYJmwVkfbKyspIT08HjnxT72Mxc+ZMlixZAsBzzz3HKaecAsDWrVuZOXMmd999N6mpqRQWFvLVV18xatQobrzxRubMmcPnn39+zMdXCBORkIp1OhiSEKO5wkTkMLfeeis///nPmTp16jH3bgFMmTKFjIwMMjIyuOWWW/jtb3/LokWLmDJlCs888wwPPPAA4B8rNnnyZCZNmsTJJ59MdnY2L774IpMmTSInJ4d169Yxb968Y65HpyNFJOQyvZorTGQgu+uuuzpcftJJJ7F58+bA63vuuQeAvLy8wKnJ9tuuW7euw31t3779sGUVFRW8//77hy3/v//7v8OW3Xbbbdx2220d7vtoqSdMRELO1zxNhYjIQKIQJiIh5/O42F1Ww8GGplCXIiLSZxTCRCTkfF431sLuA+oNE5GBQyFMREKu9TQVIiIDhUKYiIRcpvfQhK0iIgOFQpiIhFxaYixOh9FcYSIyoCiEiUjIOaIM6YNdOh0pMoAUFxeTk5NDTk4OQ4cOJT09PfD64MGDnW67cuVKbrzxxh4dLysri/379wP++1POmTOHnJwcRo8ezY9//OPAMRcvXsyPfvSjNtvm5eWxcuXKHh2vOxTCRCQs+DRXmMiAkpyczJo1a1izZg3XXHMNN998c+D1oEGDOp2cNTc3lwcffPCojmut5Tvf+Q7f/va3WbNmDZs3b6ayspJf/OIXR/tRjppCmIiEBYUwEVmwYAHXXHMNM2fO5NZbb2X58uWcdNJJTJ06lZNPPplNmzYBkJ+fz/nnnw/4J2tduHAheXl5jBo1qstw9v777xMbG8uVV14J+O81+T//8z889dRTVFf37c+goM2Yb4yJBT4AYpqP85K19s5268QATwPTgWLgu9ba7cGqSUTCl8/jprS6noraehJinaEuR2RA+frf/526DRt7dZ8x48cx9Pbbe7xdUVERH3/8MQ6Hg/Lycj788EOio6N59913uf3223n55ZcP22bjxo0sW7aMiooKxo4dy7XXXovT2fHPkYKCAqZPn95mWWJiIpmZmWzZsqXH9R6LYN62qA443VpbaYxxAh8ZY5Zaaz9ptc73gVJr7RhjzPeAXwPfDWJNIhKmWq6QLCypYcJwhTCRgeriiy/G4XAA/ht4z58/ny+//BJjDPX19R1uc9555xETE0NMTAxDhgxh7969ZGRkHNXxjTE9Wn4sghbCrLUWqGx+6Wx+2HarzQHuan7+EvCQMcY0bysiA4jP6wL8c4VNGJ4Y4mpEBpaj6bEKlri4uMDzX/7yl8yePZtXXnmF7du3B+4X2V5MTEzgucPh6HQ82YQJE3jppZfaLCsvL2fnzp2MGTOGnTt3Ulpa2ub9kpISUlJSjuLTdC6oN/A2xjiAVcAY4GFr7aftVkkHCgGstQ3GmDIgGdjfbj9XA1cDpKWlkZ+fH8yyAaisrOyT40QqtU/n1D5da99GlQf9f3vlr/iCmH29e1okEul7qGtqo8511T5JSUlUVFT0XUGdqKurw+l0Ul9fT01NTaCu4uJivF4vFRUVPPbYY1hrqaiooLq6moaGBioqKgLbtmzT1NREZWXlYZ/NWktlZSUzZsygsrKSxx57jO9+97scOHCAm266iblz59LY2Mj48eP56KOP2LJlC2lpaaxevZqamhoGDx7cZXvV1tb26HsyqCHMWtsI5BhjBgOvGGMmWWs7vr155/t5HHgcIDc31x4pCfem/Pz8IyZuUft0Re3TtfZtZK3l5397m0GeYeTlTQpdYWFC30NdUxt1rqv22bBhAwkJCX1XUCdaTiU6nU5cLlegrttvv5358+fzX//1X5x33nkYY0hISMDtdhMdHU1CQkJg25ZtoqKiiI+PP+yzGWOIj48nMTGR1157jeuuu47f/OY3AJx77rncd999gf08+OCDXHLJJTQ1NREfH88f//hHkpKSuvwcsbGxTJ06tdufO6ghrIW19oAxZhlwNtA6hO0CfECRMSYaSMI/QF9EBhhjDBleN4WlmrBVZKC56667Olx+0kknsXnz5sDre+65B/DP29USMNtvu25dx30927dvDzz3+Xz8+c9/pqKiosMgOmfOHObMmdP9D3CUgjZFhTEmtbkHDGOMC/gW0P4cw+vA/ObnFwHvazyYyMDl87g0TYWIDBjBnCdsGLDMGPM5sAJ4x1r7F2PM3caYC5rXeRJINsZsAW4BbgtiPSIS5jK9bgpLq9HfYiIyEATz6sjPgcNOjFpr72j1vBa4OFg1iEhk8Xnd1NY3sa+yjiEJsaEuR6Tfs9YGZeqFgeho/njUjPkiEjYC01ToRt4iQRcbG0txcbF6nnuBtZbi4mJiY3v2x2OfDMwXEemOQxO2VjN9hCfE1Yj0bxkZGRQVFbFv375QlxIytbW1PQ5ORxIbG9vjCWIVwkQkbGR4DoUwEQkup9PJyJEjQ11GSOXn5/doSoneptORIhI2Yp0OUhNiKCxVCBOR/k8hTETCSqbXzU71hInIAKAQJiJhxT9XmAbmi0j/pxAmImHF53Wzp6yG+samUJciIhJUCmEiElZ8XjdNFnYfUG+YiPRvCmEiElZ8gSskFcJEpH9TCBORsBKYsFVXSIpIP6cQJiJhZViSi+gooyskRaTfUwgTkbDiiDKke1yasFVE+j2FMBEJOz6Pm8JSjQkTkf5NIUxEwo7P61ZPmIj0ewphIhJ2fF4XJVUHqaprCHUpIiJBoxAmImEnME2FrpAUkX5MIUxEwk6m1x/CdhYrhIlI/6UQJiJhx+dt6QnT4HwR6b8UwkQk7HjcTuIGOTQ4X0T6NYUwEQk7xhhdISki/Z5CmIiEJZ/XrYH5ItKvKYSJSFjK9LopLKnBWhvqUkREgkIhTETCks/joqa+kf2VB0NdiohIUCiEiUhYOnSFpE5Jikj/pBAmImGpZa4wDc4Xkf5KIUxEwlKGRyFMRPo3hTARCUuuQQ5S4mMoLNGErSLSPymEiUjYyvS62KmeMBHppxTCRCRsaa4wEenPFMJEJGz5PG72lNVS39gU6lJERHqdQpiIhK1Mr5vGJsueA7WhLkVEpNcphIlI2MrwugDNFSYi/ZNCmIiELZ+mqRCRfkwhTETC1rCkWKKjjK6QFJF+SSFMRMJWtCOK4YNdFJZqrjAR6X8UwkQkrPm8Lp2OFJF+SSFMRMJaptetECYi/ZJCmIiEtQyPm+Kqg1TVNYS6FBGRXqUQJiJhzef1XyFZpHFhItLPKISJSFjLbA5hukJSRPobhTARCWs+T/OErQphItLPKISJSFjzxg3CPcihWfNFpN8JWggzxviMMcuMMeuNMQXGmB93sE6eMabMGLOm+XFHsOoRkchkjNEVkiLSL0UHcd8NwL9Ya1cbYxKAVcaYd6y169ut96G19vwg1iEiES7DoxAmIv1P0HrCrLV7rLWrm59XABuA9GAdT0T6L5/XRWFpNdbaUJciItJrTF/8UDPGZAEfAJOsteWtlucBLwNFwG7gJ9bagg62vxq4GiAtLW36kiVLgl5zZWUl8fHxQT9OpFL7dE7t07WetNE72+t5buNBHpztJjHGBLmy8KDvoa6pjTqn9ulaX7TR7NmzV1lrczt6L5inIwEwxsTjD1o3tQ5gzVYDI6y1lcaYc4FXgePa78Na+zjwOEBubq7Ny8sLbtFAfn4+fXGcSKX26Zzap2s9aaOG9Xt5buNKfONzmJrpCW5hYULfQ11TG3VO7dO1ULdRUK+ONMY48Qew56y1/9f+fWttubW2svn5m4DTGJMSzJpEJPK0TNiqG3mLSH8SzKsjDfAksMFa+99HWGdo83oYY2Y011McrJpEJDL5vJorTET6n2CejpwFXAF8YYxZ07zsdiATwFr7KHARcK0xpgGoAb5nNfJWRNpxD4omJX6QQpiI9CtBC2HW2o+ATkfQWmsfAh4KVg0i0n9keNyasFVE+hXNmC8iESHT69b9I0WkX1EIE5GI4PO62H2globGplCXIiLSKxTCRCQi+DxuGpsse8pqQ12KiEivUAgTkYiQ2TJNhU5Jikg/oRAmIhHh0FxhCmEi0j8ohIlIRBiWFIsjylBYoglbRaR/UAgTkYgQ7Yhi+OBYXSEpIv2GQpiIRAyf5goTkX5EIUxEIkam163TkSLSbyiEiUjE8Hnd7K+so/pgQ6hLERE5ZgphIhIxMjz+G3kXlao3TEQin0KYiEQMzRUmIv2JQpiIRIyWucJ0haSI9AcKYSISMZLjBuFyOjQ4X0T6BYUwEYkYxhj/FZKapkJE+gGFMBGJKD6vS2PCRKRfUAgTkYiS4XFTWFKNtTbUpYiIHBOFMBGJKJleN1UHGymtrg91KSIix0QhTEQiiq6QFJH+QiFMRCKKz+ufsFXjwkQk0imEiUhE8XnUEyYi/YNCmIhElLiYaJLjBlGkaSpEJMIphIlIxMnwujVhq4hEPIUwEYk4mV63TkeKSMRTCBORiOPzuNh9oIbGJs0VJiKRSyFMRCKOz+umocmyp0ynJEUkcimEiUjEydRcYSLSDyiEiUjEaZmmokiD80UkgimEiUjEGTY4ligDhZqmQkQimEKYiEQcpyOK4YNdOh0pIhFNIUxEIpLP49ati0QkoimEiUhE8nldFJZqTJiIRC6FMBGJSJleN/sq6qg52BjqUkREjopCmIhEJF/zNBW6h6SIRCqFMBGJSBnN01ToCkkRiVQKYSISkQITthYrhIlIZFIIE5GIlBI/CJfTocH5IhKxOg1hxhiHMebmvipGRKS7jDFkeFyapkJEIlanIcxa2whc2ke1iIj0SKbXrQlbRSRiRXdjnb8ZYx4C/ghUtSy01q4OWlUiIt3g87r5dFsJ1lqMMaEuR0SkR7oTwnKav97dapkFTu/9ckREui/D46KyroED1fV44gaFuhwRkR7pMoRZa2cfzY6NMT7gaSANf2h73Fr7QLt1DPAAcC5QDSxQD5uIdFfgCsmSaoUwEYk4XV4daYxJM8Y8aYxZ2vx6gjHm+93YdwPwL9baCcCJwPXGmAnt1jkHOK75cTXwSI+qF5EBrWXCVs0VJiKRqDtTVCwG3gKGN7/eDNzU1UbW2j0tvVrW2gpgA5DebrU5wNPW7xNgsDFmWDdrF5EBLhDCSjRNhYhEHmOt7XwFY1ZYa08wxnxmrZ3avGyNtTan0w3b7iML+ACYZK0tb7X8L8B/Wms/an79HvAza+3Kdttfjb+njLS0tOlLlizp7qGPWmVlJfHx8UE/TqRS+3RO7dO13mqjG96rYnpaNAsmxfRCVeFD30NdUxt1Tu3Ttb5oo9mzZ6+y1uZ29F53BuZXGWOS8Y/rwhhzIlDW3YMbY+KBl4GbWgewnrDWPg48DpCbm2vz8vKOZjc9kp+fT18cJ1KpfTqn9ulab7XRqHUf0ehykpc389iLCiP6Huqa2qhzap+uhbqNuhPCbgFeB0YbY/4GpAIXdWfnxhgn/gD2nLX2/zpYZRfga/U6o3mZiEi3+Lxu1u3q9t+FIiJho8sxYc3juk4DTgZ+CEy01n7e1XbNVz4+CWyw1v73EVZ7HZhn/E4Eyqy1e7pdvYgMeD6vm10Hamhs6nxohYhIuOmyJ8wYEwtcB/wD/lOSHxpjHrXW1nax6SzgCuALY8ya5mW3A5kA1tpHgTfxT0+xBf8UFVcezYcQkYHL53FT32j5uryW9MGuUJcjItJt3Tkd+TRQAfyu+fVc4Bng4s42ah5s3+kU1tZ/VcD13ahBRKRDmYErJKsVwkQkonQnhE1qnuurxTJjzPpgFSQi0hM+rz947Syp5sRRySGuRkSk+7ozT9jq5vFaABhjZgIrO1lfRKTPDB/sIspAkW7kLSIR5og9YcaYL/CPAXMCHxtjdja/lQls7IPaRES65HREMSzJRWGpJmwVkcjS2enI8/usChGRY+DzutipnjARiTBHDGHW2h0tz40xHvzzebVef8dhG4mIhIDP4+avm/eFugwRkR7pzhQVvwIWAFtpnjW/+evpwStLRKT7Mr1uvqmoo7a+kVinI9TliIh0S3eujrwEGG2tPRjsYkREjkbLjbyLSqsZMyQhxNWIiHRPd66OXAcMDnYhIiJHq2WaisISDc4XkcjRnZ6w/wA+M8asA+paFlprLwhaVSIiPdDSE1ZYqsH5IhI5uhPC/gD8GvgCaApuOSIiPZcaH0OsM4qdxQphIhI5uhPCqq21Dwa9EhGRo2SMIcPjVk+YiESU7oSwD40x/wG8TtvTkauDVpWISA9let0aEyYiEaU7IWxq89cTWy3TFBUiElZ8HhcrtpVgrcUYE+pyRES61GUIs9bO7otCRESOhc/rpqKugbKaega7B4W6HBGRLnVnstY7Olpurb2798sRETk6gSskS2oUwkQkInRnnrCqVo9G4BwgK4g1iYj0mM/jD2G6h6SIRIrunI78r9avjTH3AW8FrSIRkaMQmLBVV0iKSIToTk9Ye24go7cLERE5FgmxTjxuJ4XqCRORCNGdMWFfcOjG3Q4gFdB4MBEJOz6vW6cjRSRidGeKivNbPW8A9lprG4JUj4jIUfN53KzfUx7qMkREuqU7Y8J29EUhIiLHyud18/b6r2lssjiiNFeYiIS3I4YwY0wFh05Dtvw0s83bDLLWdqcXTUSkz/i8LuobLXvLaxk+2BXqcuQlMnUAACAASURBVEREOnXEgfnW2gRrbWLzIwEYBtwLfA080FcFioh0V8s0FRqcLyKRoMurI40xg40xdwGfAwnACdbafwl2YSIiPZXp1VxhIhI5OjsdmQL8C/Bd4ClgqrW2rK8KExHpqeGDXRgDhaW6kbeIhL/OxnXtAPYBi4Bq4Putb4prrf3v4JYmItIzg6KjGJYYS5F6wkQkAnQWwn7LoYH5CX1Qi4jIMdNcYSISKY4Ywqy1d/VhHSIivcLndfPhl/tCXYaISJeO5rZFIiJhy+dxs7e8jtr6xlCXIiLSKYUwEelXMpP984MVaXC+iIQ5hTAR6VcCc4WValyYiIS37tzAOwb4ZyCr9frWWt3EW0TCTstcYbpCUkTCXXduPfQaUAasAuqCW46IyLFJTYghJjpKV0iKSNjrTgjLsNaeHfRKRER6gTGGDI+LwhKNCROR8NadMWEfG2MmB70SEZFekul1a0yYiIS9zm5b9AX+yVqjgSuNMV/hPx1pAGutndI3JYqI9IzP62bljtJQlyEi0qnOTkee32dViIj0Ip/HTUVtA2XV9SS5naEuR0SkQ0c8HWmt3WGt3QEMA0pavS4FhvZVgSIiPeXzapoKEQl/3RkT9ghQ2ep1ZfMyEZGw5PP6J2zVFZIiEs66E8KMtbblRt5Ya5vo3lWVIiIhEegJUwgTkTDWnRD2lTHmRmOMs/nxY+CrYBcmInK0EmOdDHY7dTpSRMJad0LYNcDJwK7mx0zg6q42MsY8ZYz5xhiz7gjv5xljyowxa5ofd/SkcBGRzvg8bnZqrjARCWNdnla01n4DfO8o9r0YeAh4upN1PrTW6ipMEel1Pq+LjXsqQl2GiMgRddkTZozJMMa80tyr9Y0x5mVjTEZX21lrPwBKeqVKEZEe8nndFJXW0NRku15ZRCQETKsx9x2vYMw7wPPAM82LLgcus9Z+q8udG5MF/MVaO6mD9/KAl4EiYDfwE2ttwRH2czXNp0DT0tKmL1mypKtDH7PKykri4+ODfpxIpfbpnNqna8Fuo/d31vP0+oP8d54Lb2x3Rl6EF30PdU1t1Dm1T9f6oo1mz569ylqb29F73bnKMdVau6jV68XGmJt6oa7VwAhrbaUx5lzgVeC4jla01j4OPA6Qm5tr8/LyeuHwncvPz6cvjhOp1D6dU/t0LdhtZDbv4+n1y8kYm8OMkd6gHSdY9D3UNbVR59Q+XQt1G3Xnz8NiY8zlxhhH8+NyoPhYD2ytLbfWVjY/fxNwGmNSjnW/IiLgv38kaJoKEQlf3QlhC4FLgK+bHxcBVx7rgY0xQ40xpvn5jOZajjnciYgADB8cizGasFVEwld3ro7cAVzQ0x0bY14A8oAUY0wRcCfgbN7no/jD3LXGmAagBvie7WqAmohIN8VEOxiaGKu5wkQkbHUZwowxo4AHgBMBC/wduNla2+mErdbaS7t4/yH8U1iIiASFz+umSHOFiUiY6s7pyOeBF/HfyHs48CfghWAWJSLSG/wTtqonTETCU3dCmNta+4y1tqH58SwQG+zCRESOlc/rYm9FLXUNjaEuRUTkMN0JYUuNMbcZY7KMMSOMMbcCbxpjvMaYyLvuW0QGjEyvG2thV6lOSYpI+OnOPGGXNH/9Ybvl38M/RmxUr1YkItJLfM3TVOwsqWZUqiatFJHw0p2rI0f2RSEiIr3N52meK0w9YSISho54OrL5tGPL84vbvffvwSxKRKQ3DEmIYVB0FEUanC8iYaizMWHfa/X85+3eOzsItYiI9KqoKEOGx6UrJEUkLHUWwswRnnf0WkQkLPk8bk3YKiJhqbMQZo/wvKPXIiJhKdPrplATtopIGOpsYH62MaYcf6+Xq/k5za81T5iIRASf10VZTT1lNfUkuZyhLkdEJOCIIcxa6+jLQkREgiFwhWRJNUnpSSGuRkTkkO5M1ioiErFa5gor0rgwEQkzCmEi0q+1nrBVRCScKISJSL+W5HKSGButwfkiEnYUwkSk38tM1jQVIhJ+FMJEpN/zedw6HSkiYUchTET6vUyvm6LSGpqaNMWhiIQPhTAR6fcyvG4ONjTxTUVdqEsREQlQCBORfs/ncQFoXJiIhBWFMBHp9zK9hyZsFREJFwphItLvpXtcGKO5wkQkvCiEiUi/FxPtIC0hVnOFiUhYUQgTkQEh06u5wkQkvCiEiciAkOF1aUyYiIQVhTARGRB8Hjdfl9dS19AY6lJERACFMBEZIDK9bqyF3QdqQ12KiAigECYiA4SveZoKXSEpIuFCIUxEBgSft3nCVoUwEQkTCmEiMiCkJcQyyBGlKyRFJGwohInIgBAVZcjw6ApJEQkfCmEiMmBkeN2asFVEwoZCmIgMGJlel05HikjYUAgTkQHD53FzoLqe8tr6UJciIqIQJiIDR8s0FRoXJiLhQCFMRAaMzEAI07gwEQk9hTARGTB8HvWEiUj4UAgTkQEjye0kITZag/NFJCwohInIgJLpdasnTETCgkKYiAwoPo9b948UkbCgECYiA4rP66KotAZrbahLEZEBTiFMRAaUTK+buoYm9lXUhboUERngFMJEZEDJaJ6mQqckRSTUghbCjDFPGWO+McasO8L7xhjzoDFmizHmc2PMtGDVIiLSIjBNha6QFJEQC2ZP2GLg7E7ePwc4rvlxNfBIEGsREQEgw+MCNGGriIRe0EKYtfYDoKSTVeYAT1u/T4DBxphhwapHRAQg1ukgLTFGpyNFJORMMK8QMsZkAX+x1k7q4L2/AP9prf2o+fV7wM+stSs7WPdq/L1lpKWlTV+yZEnQam5RWVlJfHx80I8TqdQ+nVP7dC2UbXTvJzVEGfj5TFdIjt8d+h7qmtqoc2qfrvVFG82ePXuVtTa3o/eig3rkXmKtfRx4HCA3N9fm5eUF/Zj5+fn0xXEildqnc2qfroWyjV7fu4ZPt5WE9b+Rvoe6pjbqnNqna6Fuo1BeHbkL8LV6ndG8TEQkqDK8bnaX1XCwoSnUpYjIABbKEPY6MK/5KskTgTJr7Z4Q1iMiA4TP48Ja2H1Ag/NFJHSCdjrSGPMCkAekGGOKgDsBJ4C19lHgTeBcYAtQDVwZrFp6QrNoi/R/md5D01RkpcSFuBoRGaiCFsKstZd28b4Frg/W8Y/W7qrdzH1jLukmna1fbCU7NZuJKRNxRYfvAF4R6RmfJmwVkTAQEQPz+1JjUyP/kP4P/H3H37l/9f0ARJtoxnrHkp2aTXZqNjlDchgWNwxjTIirFZGjkZYYyyBHlOYKE5GQUghrJzMxk3v/4V7yG/KZcuIUPt/3OWv3rWXtvrW8suUVnt/4PACprtRAIMtOzWZ88nhiHDEhrl5EusMRZUj3uDRrvoiElEJYJ7yxXvJ8eeT58gBoaGpgc+nmQChb880a3t35LgDOKCfjk8cf6i1LzSEtLi2E1YtIZzI8Lgp1OlJEQkghrAeio6KZkDyBCckTuHScf8jb/pr9/lD2jT+YvbjpRZ5Z/wwAQ+OGBgJZdmo247zjcDqcofwIItIs0+vmzS90QbaIhI5C2DFKcaVwRuYZnJF5BgD1jfVsLNl4qLds3xre2v4WADGOGCYmTwz0lmUPySbFlRLK8kUGLJ/XTWl1PRW19cTFOCiuKSbFlaKxniLSZxTCepnT4WRy6mQmp07mci4HYG/V3kAgW7tvLc9ueJZFBYsASI9PbzO27HjP8URH6Z9FJNh8Hv8Vkl/s2cXDG37B5/s+xxPjYULKBCYmTww8hriHKJiJSFDot30fSItL48y4Mzkz60wA6hrr2FC8IdBbtuLrFby57U0AXNEuJqVMOtRblpqNJ9YTyvJF+qVMr5uomK+57ZP/oaaxjOuyr2NP1R4Kigt4cveTNNpGwN/bHQhlKROZkDxBPdgi0isUwkIgxhFDzpAccobkAP4JYvdU7Wkz4H/xusU02AYARiSO4MwRZzJ3/Fz98BfpJbsPfoZ7xCMcbHSz+JzFTEyeGHivpqGGTSWbKCguYH3xegr2F/BB0QdY/JM5D40b2qa3bELyBAbHDg7VRxGRCKUQFgaMMQyPH87w+OGcM/IcwP9LYH3x+kBP2RNfPMEfCv7AnDFzWDBxAZmJmSGuWiRyPb/heX694teYhqGc4rm9TQADf4906z+UAKrqq9hQvIGC4oJAOHtv53uB99Pj05mUMikQzMYnjydhUEKffSYRiTwKYWHKFe1ietp0pqdNZ+GkhWwv287igsW8uuVVXv7yZf4x8x9ZOGkhE1Mmdr0zEQH808z8ZsVveGHjC+T58vjyiwsoLnN3a9s4Zxy5Q3PJHZobWFZ+sJwNxRtYt38dBcUFrNu/LnAhDkBWYhYTkv1jzCalTGKcdxxuZ/eOJyL9n0JYhMhKyuKuk+/i+pzreW7Dc7y46UXe3vE2M4fOZOGkhZw0/CQNHhbpROXBSn76wU/5aNdHzJ8wn5un38x1uz7jq31VR73PxEGJzBw2k5nDZgaWldaW+k9hFhdQsL+AVXtXBcZ8RpkoRiWNCgSziSkTGesZS2x07DF/PhGJPAphESbVncpN02/iB5N/wEubX+KZ9c/ww3d/yDjvOK6ceCVnZp2pqytF2tlduZvr37ue7WXbufOkO7no+IsA/xWSf928D2ttr/0R44n1MCt9FrPSZwWW7avedyiYFRfw0a6PeH3r64D/tmhjPGMCY8smpkzk+MHH90otIhLe9Ns6QsUPimfBpAXMHT+XN756g0UFi/jZhz/jwc8eZN6EeVx43IW66bgIsHbfWm58/0bqG+t55FuPcOKwEwPvZSa7qa1vYl9lHUMSgtcblepO5TT3aZzmOw3wX4yzt3pvoLesoLiAd3e+y8tfvgz478AxPHo4X37+JWdmncmIxBFBq01EQkchLMINcgziwuMuZM6YOeQX5vPUuqf4j+X/wSNrH2HuuLlcOu5SXbUlA9b/2/b/+MVHv2CIewgPn/0wo5JGtXm/Za6wwpKaoIaw9owxDI0bytC4oYGJnq217KrcFegtW7Z5GQ9+9iAPfvYgYz1jOTPrTM4ccSZZSVl9VqeIBJdCWD8RZaI4PfN0Ts88ndV7V7No3SJ+v/b3LCpYxIVjLmTexHmkx6eHukyRPmGt5bHPH+PhNQ8zbcg07p99f4fz7fm8/t7iwpJqpo8I7Xx8xhgyEjLISMjgrKyzmFYxjXEnjOPt7W/z9o63+d1nv+N3n/1OgUykH1EI64empU1jWto0tpRuYVHBIl7c9CJ/3PRHzso6i4WTFjLWOzbUJYoEzcHGg9z58Z385au/8E+j/om7Tr6LQY5BHa6bEegJC88beQ+NG8q8ifOYN3EeX1d9zTs73uHt7YcC2fGe4zlzhH8i6JFJI0Ndroj0kEJYPzbGM4Z7/+Febph6A8+sf4aXNr/Em9veZFb6LBZOXMgJQ0/QFZXSr5TUlnDTspv47JvPuGHqDVw1+apOv8djnQ6GJMSwM0xDWGtD44ZyxYQruGLCFW0C2UNrHuKhNQ8pkIlEIIWwAWBo3FB+esJPuXrK1by46UWe3fAs33/7+0xKnsTCyQs53Xc6jihHqMsUOSZbD2zl+veuZ3/Nfn572m85O+vsbm3n87opLA3/ENZa+0D27o53eWv7W4FAdpznuEAgaz8OTkTCh0LYAJIUk8RVU67iiglX8PrW11lcsJhb8m9hROII5k+czwWjLyDGERPqMkV67OPdH/OT/J8wyDGIp856iimpU7q9babXzfJtJUGsLriGxg3l8gmXc/mEywOB7O0db/Pwmod5eM3DCmQiYSwq1AVI34uNjuWSsZfw52//mftOu484Zxx3//1uznrpLJ744gnKD5aHukSRbntx04tc9+51DI0fyvPnPd+jAAbg87jYU1ZDfWNTkCrsOy2B7Olznubdi97lthm3Ee+M5+E1DzPn1Tlc+NqFPLr2Ub4q+yrUpYoI6gkb0BxRDs7KOoszR5zJ8q+Xs2jdIh5Y/QD/+/n/cvHxF3PFhCtIi0sLdZkiHWpsauS+lffx7IZnOTXjVH5z6m+Ic8b1eD8ZXjdNFnYfqGFEcs+3D1dpcWlcNv4yLht/GXur9vLuznd5e/uhHrIxg8dwZtaZnDXiLEYNVg+ZSCgohAnGmMCtVzaWbGTRukU8u+FZntv4HOeNPI8rJ13J6MGjQ12mSEBVfRU/++Bn/LXor1w+/nJ+kvuTox7XmOk9NFdYfwphrR0pkD2y5hF+v+b3CmQiIaIQJm2M847j16f+mhum3sDT65/mlS9f4bWtr5GXkcfCyQuZOmRqqEuUAe7rqq/50Xs/YsuBLfzrzH/lu+O+e0z78zWHsEi4QrI3tA5k31R/E7jKsk0gax5Dpj++RIJLIUw6lJGQwe0zb+fa7Gt5YeMLPL/xeeYtncfUIVO5cuKVWGtpsk00NDX4H7bh0POmBhqbGqm39W1et1+nZbvGpkbqm+rbvD7SfhuaGmi0rdZvfh0dFc3w+OFkxGfgS/CRkZBBcmyypuDoZ9btX8cN799AbUMtD5/xcJv7Mx6toYmxOB0m4q6Q7A1D3EM6DmRrH+H3a3/P6KTR/iELCmQiQaEQJp3yxHq4Luc6FkxcwCtbXuHpgqe5cdmNGAz2aduntURHReOMchJtonFEOYiOivY/TDQHGw+yr2YflkM1xTpiSY9PD8xCnhF/6Gt6QrrurRlh3tnxDrd/eDvJrmT+91v/yxjPmF7ZryPKkD7YFbYTtvaV9oGs5SrL1oFsSuoURiaNDDzS49OJjtKvEZGjpf890i1up5vLxl/GJWMv4d0d7/LemvcYPXJ0IAg5zKFQ5Ixy+kOSiT4UlJrDUmD9lhBlWq3fbp3W20WZqC57teoa69hduZuiiiKKKovYVbGLosoiiiqKWPH1Cqob2v6STY5N7jCgZSRkMMQ9hCiji4fDgbWWJ9c9yQOrHyA7NZsHZj9Asiu5V4/h87oHfAhrbYh7CHPHz2Xu+Lnsq97HOzveYVnhMj4o+oBXtrwSWM8Z5WRE4ghGJo0kKzGLkUkjGZU0ipFJI3E73SH8BCKRQSFMesQZ5eSckefg2uEiLycv1OW0EeOICfyF3p61lgN1BwIBraiiiF2VuyiqKGLNN2tYum0pTfbQFAXOKCfp8emkJ6QfOsXZHNDS49OJHxTflx9twKpvrOff/v5vvLb1Nc4ZeQ6/mvWroMxl5/O6eX3NbvaU1TAsST2kraW6UwOBDKCsrozt5dvZVraNr8q+YlvZNr4s/ZL3d75Po20MbJfmTmvTa9YS0FJdqRomINJMIUwGBGMMnlgPnlgPk1MnH/Z+fVM9X1d+TWFl4WFB7Yt9Xxw2d9rgmMGHes9a96QlZJDm1rQeveFA7QFuyr+JVXtXcV32dVyTfU3QfnlfNjOTP6/ZzaWPf8Iff3gSaYmxQTlOf5AUk0R2ajbZqdltltc31lNYURgIZi2P17e+TlV9VWC9OGccIxPbBrORSSPxJfhwOpx9/XFEQkohTAR/z5cv0Ycv0dfh+2V1ZYGes5aAVlRRREFxAe/ueJcG2xBYN9pEM9gxmGffepZh8cMYHjec4fH+x7C4YaTFpeGM0i+bzmwv2871713P11Vf8+tTfs25o84N6vEmDk9i8cIZzHvyUy59/BOWXH0iQxTEesTpcDJq8KjDpriw1vJN9TdsK9/WJpwt/3o5f/7qz4H1HMaBL8FHVlLb05ojk0aSOCixrz+OSJ9QCBPphqSYJJJikpiQPOGw9xqaGvim+ps2AW3V1lXUNtbyt11/Y1/NvjbrR5kohriHBMLZsLhhgZA2PG44w+KHDejbRy3fs5yb828mOiqaJ896kpwhOX1y3OkjPCxeOIP5Ty1n7hOf8sJVJ5KaMHD/HXqLMYa0uDTS4tI4cdiJbd6rqq9ie9n2w3rPPtr1EQ1Nh/6wSY5NPiyYjUwa2WYIgUgkUggTOUYt02MMjx/ODGYAkF+eT15eHgAHGw/yddXX7KrcxZ6qPeyu3O1/VO1m9d7V7K3e22YsDUCKKyUQyFrCWeuv/XXQ8ytfvsLdf7+bEYkjeOiMh8hIyOjT45+Q5WXRghNYsGgFlz3xCS9cdSLJ8QpiwRLnjGNiykQmpkxss7yhqYFdlbvajDvbVraNpduXUnGwIrBerIllwtIJjPOOY5x3HOOTxzM6abROa0rEUAgTCbJBjkFkJmaSmZjZ4fstPWktwWx35W72VO1hV+UuNhRv4P2d71PfVN9mm6SYpDanOVsCW3p8OsPihpE4KDGiBj832SbuX30/i9Yt4uThJ3PfafeRMCghJLXMHJXMkwtyWbh4BZc194h54gaFpJaBKjoqmhGJIxiROII8X15gubWWktqSQDjLX5dPuS3nlS2vUNNQE9h2zOAxh4KZdzzHe47XxTQSlhTCREKsdU9aR5psE/tr9rcJZ3sq97C7ajfbyrbx8e6PA7+AWsQ54w6Fs+bTnSmulMDFCd4YL55YD7HRoR/3VF1fze0f3c57O9/ju2O/y20zbgv53FMnj07hiXkn8P0/+IPY81fNZLBbQSzUjDEku5JJdiWTOzSXIXuGkJeXR2NTIzsrdrKxZCMbSjawqWQTHxR9wKtbXg1sm5mQGQhmLb1mKa6UEH4aEYUwkbDXMoZsiHsIORw+Pqpl+o3WPWktz/dU7mH13tVU1Fd0sGdwRbvwxnrxxHgOBbRYf0Brs6w5tMU543q1h21v1V5ueP8GNpVu4rYZt3HZ+Mt6bd/H6h+OS+Hxeblc9YeVXPHkcp79wUySXDrNFY4cUY7AOLFzRp4DHLogYGPJxsCjoLiAt3e8HdguOTaZccn+3rKWcOZL8GmOQOkzCmEiEa719Bvtx9a0qDhYQXFNMaV1pZTWNj/qSimpLQm83l+zny8PfElpbSl1jXUd7scZ5TwU1GLahrbBMYMPBbjm4JYYk3jEX2iFdYX86o1fUVlfye9O/x2nZpzaa23SW047PpVHr5jGD59ZxbynlvPM92eQGKsgFglaXxBwmu+0wPLyg+VsKtnUJpx9uvvTwBXOcc44xnrGMtY7NhDOxgweo3FmEhQKYSIDQMKgBBIGJZBFVpfrWmupaaihpLaEA3UH2gS1krpDz0trSymsKKS0rrTNPFCtOYyDpJikw3rX3NFuntv7HMnuZJ459xmO9xzfy5+495w+Lo3fXzada59dxfynlvP0whkkKIhFrMRBiZww9AROGHpCYFldYx1bDmxhU8kmNhRvYGPJRl7d8iovNLwA+IcMjE4aHTiNOdYzlnHecRpnJsdMIUxE2jDG4Ha6cTvd3b468WDjwQ5710pqS9r0vm0u3UxpXSlldWWMjBnJU+c9FRHjcr41IY2H5k7j+udXc+WiFfxh4QziYvTjs7+IccQwMXkiE5MnwnH+ZU22iZ3lO9v0mH2460Ne2/paYDtfgq/NOLOsxCzcTjdxzjhiHbERdXGMhIZ+iojIMRvkGBQ49dMdjU2NfPDXDyIigLU4e9JQHvzeVG5c8hlXLl7B4itPwD1IP0L7qygTRVZSFllJWZw98mzA30u8r2Zfm2C2sWQj7+x4p8Pt3dFu/6P5jxp3tD+gtV7W8jrOGYcr2uV/7XQTFx0X2KZlXU3y3P/oJ4iI9DlHlCMiewnOmzKMRmu5aclnfH/xSp5acAKuQY5QlyV9xBgTuEim9RjGioMVbCrZxO6q3VTXV1NVX0V1QzXV9dWBry3Lvqn+huqG5tfN73fXoKhBgeAWCGwtwa5VYGtZXlhRSO322kCgaxMAo924ol0R+f+wP1EIExHpgQuyh9PY1MQtL67lqqdX8sT8XGKdCmIDWcKgBHKH5h7Vtk22idqG2jbBrH2Iax3YquqrqGmoObRuQxX7ava1CXwHmw4G9v/8X58/4rENpm2vWwdBLc4Zd3jvXetQ166HzxGl/ws9oRAmItJDF07NoLEJfvrSWn74zCoeu2K6gpgclSgTFQg3vXV6vr6xnuqGat778D2yc7Opqq9qG+yaw1vrcNe+t67leUvo665YR+zhvXItr48Q0gxd98Z11GPXfruO9tNVT5+rykUeeV0eP1iCGsKMMWcDDwAO4Alr7X+2e38B8FtgV/Oih6y1TwSzJhGR3nDR9Awam5r42ctfcN1zq3nk8mnERCuISeg5HU6SHEl4o72MHjz6mPfXZJuoaagJBLWqhqpAmGsJah2FvJbnZbVl7GnYQ3VD9eH3+7TtX7ZbcKRl1na5TlfbAEyLmdbldsEUtBBmjHEADwPfAoqAFcaY162169ut+kdr7Y+CVYeISLB894RMGposv3hlHT96/jMenjuNQdGa6FP6lygTFTgtmUpqqMvpVfn5+SE9fjB/WswAtlhrv7LWHgSWAHOCeDwRkT532cwR3D1nIu+s38uNL3xGfWNT1xuJiACmo+65XtmxMRcBZ1trf9D8+gpgZuter+bTkf8B7AM2Azdbaws72NfVwNUAaWlp05csWRKUmlurrKwkPl4T8R2J2qdzap+u9bc2ent7Pc9vPMgJQx1cMyUGR9SxXXXW39onGNRGnVP7dK0v2mj27NmrrLUdXrkR6oH5fwZesNbWGWN+CPwBOL39Stbax4HHAXJzc21eXl7QC8vPz6cvjhOp1D6dU/t0rb+1UR4w6sOvuOeNDQxNG8z/XJJNtOPoTzb0t/YJBrVR59Q+XQt1GwUzhO0CfK1eZ3BoAD4A1triVi+fAH4TxHpERILqB6eMoqHJ8p9LNxIdZbjv4uxj7hETkf4rmCFsBXCcMWYk/vD1PWBu6xWMMcOstXuaX14AbAhiPSIiQXfNaaNpaGzivrc344gy/OafpxClICYiHQhaCLPWNhhjfgS8hX+KiqestQXGmLuBldba14EbjTEXAA1ACbAgWPWIiPSVH51+HA1Nlvvf/ZLoKMO/XzhZQUxEDhPUMWHW2jeBN9stu6PV858DPw9mDSIiofDjM46jscnyu/e3EBVluPfbk3SLGBFpI9QD80VE+iVjNfrMmgAAHZ1JREFUDLd863gamiyP5G8lOsrwbxdMVBATkQCFMBGRIDHGcOtZY2lssjz+wVc4ogx3nD9BQayfKSyp5uXVRRyoruc709KZnJ6kf2PpFoUwEZEgMsbw83PGUd/YxKK/bcdhDL84b7x+SUe4moONLF23hz+tLOLvXxVjDDgdUSz+eDsThydy6YxM5uQMJyHWGepSQ25nsT+k5m/ex8yRXq44cQQ+rzvUZYUFhTARkSAzxt8D1thkeeKjbUQ7ovjZ2WMVxCKMtZbVO0v508oi/vL5HirrGsj0urnlW8fzz9MziI+J5rU1u3j+053866vruPeNDVyQPZzvzfCR4xs8oP69q+oaePOLPby0qohPt5VgDEwansSTH23jiQ+/4lsT0rhy1khmjvQOqHZpTyFMRKQPGOMfE9bYZHn0r/4xYv9y5vED+hdQpNhbXsvLq4t4aVURX+2rwuV0cO7kYVycm8GMLG+bK1/nnZTFFSeOYG1RGS98upPX1+7mjysLGTc0gbkzM5mTk06Sq3/2jjU1WT7dVsJLq4pYum4P1QcbGZUSx0/PGsuFU9MZPtjF7gM1PPvJDl5YvpO3CvYyflgiV56cxQU5w4l1OkL9EfqcQpiISB8xxvCrOZNobLI8tGwL0Q7DTf94fKjLkg7UNTTy3oZv+NPKQv66eR9NFk7I8nDNqaM5d8ow4mOO/OvTGPP/27vz+LbOOt/jn9+RLFm2ZTuxEyfe4qxNk7RZmqYphdLSAQothKWFlsKUwlzu5TIwzOU1wMBc7sAdmAJzWeayvxhogTYdmhZamC70AmFm2tK0Sbpka7M5XpI0u+UtXqTn/nGOZXmJQ5LaR0m+79dLLz16zpH085NI+up5jiSW1JWzpK6cv7v+Qh58bi93P9XE5x7YzJce2sr1F1dz84p6ltWfG7NjA8uN921ooeVoN8l4lFVLarjhktoRf2N1eYJPXjufj10zlweebeXHjzfyyfue5x8f3sp7LqvnvStnML0sEeJfM7EUwkREJpAXfG/YwPeIRcz46DVzwy5LApta21izvoVfPtvKsa4+ppUW8uGrZnPDJXXMrCw+5dtLFhZwy2UzuOWyGbzQ0sbd65p48NlW1qxvYV5VCTevqOcdS2spKzq7ZsdGW2589ZxK/uaNF/DGhdNOOqtVWBDh3ZfW867ldfxx1xF+/Phuvrt2J9/7wy7etGgat13RwLL6SedESB2LQpiIyATzPOPL77yYTMbxfx57iUjE+O9XzQm7rPPWkc5efrmxlXvXt7B1X4pYxOP1C6u48ZJaXjN3yiv201MX1Zbxj7UX8dnrLuRXz+1l9bomPv+rLdz+8Dauu2g6N19Wz/IZ+Rs8RltunDlsufFUmRmXz67g8tkVNB/p4idPNnLP0838+vl9XFxbxvtf1cB1F08nHj03lyoVwkREQhDxjK/euJj+jOMrj7xIgefxX66cFWpN3b1pmo920XS4i6YjQ097j3VTXZ5gYXUpi6rLWFhTysLqsrP2+Kb+dIZ/336Qe59p4f9tfZm+tOOimjK+sGohb11cTXlRbNzuuyQe5eYV9dy8op5NrW3c83QTv9y4l/s3tjJnagk3XVrHO5fVMql4/Go4FaMvN1YHy42vXGism1zEZ69bwMf/bB73b2zljsd38z9+/hxfemgbt1xWzy0r65maLHxF7itfKISJiIQk4hlfe9di0hnHFx/aSsQzPvDqmeN2f845Dnb0jAxZweUD7T1D9i+ORaivKGb2lGJePaeSlqPdrNt9hAee3Zvdp25ygkXVZSyqKWNhtR/MpiTj4/Y3nKkdBzq4d30zv9jQyoH2HiqKY/z55Q3cuLyW+dNKJ7yeRTVl/EPNRXzmzRfy6+f2cfe6Jv7h37bylUde5E0XTePmFfWhfIJwrOXGNyyYRiI2fjNTxfEo71s5g/deVs9/bD/EHU808s3fbuc7a3dw/cXV3HZFAxfXlo/b/U8khTARkRBFIx7fuGkJ6YzjC7/eQjRi/PnlDad9e8f70rQc7ab5SBd7DnfSdKQ7CFudNB3p4nhfJruvGUwvLaRuchGvnTeFGRVF1E0uoj44TS6Ojfrif6ijh817U2ze28bm1hSb9rbx8Kb92e1VpfFgtswPZotqyqguKwxtmS11vI9fP7ePe9c3s7HpGBHPuPqCqdy4vJarL5hKLOqFUleuoliUd11ax7surWPrvhT3rGvi/o2tPPDsXmZVFvvHji2roaJk/AJuJuP44+7D/nLjC/vp7jvz5cYzYWZcOW8KV86bwu5Dndz5RCNr1rfwi42tLKsv57YrZnLtomkURML/9ztdCmEiIiEriHj8881L+cjdG/jcA5vxzKg9wb7OOQ539tJ0pCsIWkNntPanjg/ZP1EQYUZFETMqinnN3KFBq6Y8cVpfC1BZEue186bw2nlTsn2p431s2ZtiU2tbNqD9/sUDZJy/fVJRAYtqylgQLGcuqiljxuSicfth80zG8eSuw9z7TDOPbN7P8b4Mc6eW8Jk3z+dtS2vyelnrwumlfH7VIj79pgv5txf2sXpdE198aCtfeXQbb1w4jfesqGflrIpXbOyaDnexZkML961vofWYv9z4tqWv/HLjmZhZWczfv3Uhn3jDPNasb+HOJxr56OqNTCst5H2Xz+CmS+vGNaCOF4UwEZE8EIt6fOs9S/nwzzbwd7/cxHvmx+DFAzQHAWsgbDUf6aKzNz3kulWlcWZMLuaKOZXUTy4aErQqS0afzXqllRYWsHJWBStnVWT7unvTbN2f8kNZaxub9rbxo//cTV/aT2Yl8SgLppeysGYwmM2eUkz0DGY2mo90sWa9/51erce6SRZGeeeyWm5cXsfi2rPr54QSsQg3XFLLDZfU8tLL7axe18R96/0vim2oKOKmFfXccEktlacRPk603PjJa8d/ufFMJAsLuO2Kmdx6eQNrXzrAjx9v5KuPvsg3f7udty2p5v2vmsmC6olfVj5dCmEiInkiHo3wnVuW8V9/up67tx3k7m1PB/1eNlxdPrsi266fXETtpKK8/ZLLRCzCsvpJLKuflO3r7c+w/UB7dhlzU2sb96xrpruvEfD/1vnTS1kULGMurC5lXlVyzL9xtJ8QGggUf8rXJZwN5lUl+V9vWcinrp3Pw5v2sfqpZm5/eBv/9OiLvGFhFTevqOeK2ZVjzo7l23LjmfA843Xzq3jd/Cq2v9zOHU80cv+GVn7+TAuXzZzMbVfM5PULql6xT7aOF4UwEZE8UlgQ4fvvu4Tv/uL3vHrFMmZMLmJKMn5WzeCMJRb1WFhdxsLqMt5FHQDpjGP3oQ42tQ4uZz743F7ueqoJgKhnzK1Ksqi6NHuM2YXTS9lxNM0j9z2f/QmhGRVFfOL183jHJbXUnEWB4lQUFkR4+9Ja3r60lh0H2lm9rpn7NrTw0Av7qZuc4KZL67lxee2Q5dazYbnxTMytSvLFt1/EJ984n399pok7n9jDf/vZemrKE9z6qhm8e3l93n4Pm0KYiEieKSyIsHRqlEsbJoddyoSIeMacqUnmTE3ytqU1gH/sW/ORbjbtbWPz3jY2tab4/YsHuHd9y5DrFsX2+j8hdEktK86z3yGcMzXJ/7x+AX/zxgt4dPN+Vq9r4quPvsjXH3uJay6cSkW6j+98/0nWnUXLjWeirKiAD105mw++ehaPbXmZO57YzZce2sbXH9vOO5bVcNsVDcyZmgy7zCEUwkREJO+YGfUVRdRXFPHmi6YDfjA70N7DptY2tuxNcWxfI39949Vj/oTQ+aCwIMKqJTWsWlLDzoMd/OvTzaxZ38KRzl5mVvaclcuNZyLiGdcumsa1i6axZW+KO59o5N71Ldz1VBOvmVvJbVc0cNW8qeP2oZBTcX7/zxURkbOGmVFVWkhVaSHXXFjF2rWt530AG272lBI+8+YL+cQb5nH/I3/gputee17NDg63oLqUL99wMZ9603xWr2vip0/u4QN3PENDRRG3vqqBaf0u1PrO3i/XEBERkVHFoxGml3jndQDLNbk4xkeunsN/fOpq/u/NS6koifP5X23hJ5t7Tn7lcaS3ECIiInJeKIh4vGVxNW9ZXM3zLcd44dkNodajmTARERE571xcW05NSbgxSCFMREREJAQKYSIiIiIhUAgTERERCYFCmIiIiEgIFMJEREREQqAQJiIiIhIChTARERGRECiEiYiIiIRAIUxEREQkBAphIiIiIiFQCBMREREJgUKYiIiISAgUwkRERERCEA27gHyTTqU4evdq4t3d9NTXE6urwwoKwi5LREREzjEKYcP07t7NwW98g3Jg1/e/DwUFxOrric+aRWz2LOKz5xCfPYvYzJl4iUTY5YqIiMhZSiFsmMTixcx75hmeXHMvi8rL6d25i55du+jZvp323/0O0ml/RzMKqqv9YDZrdhDQZhOfNYtIeXm4f4SIiIjkPYWwUURKiulvaKD8qquG9Gd6e+nbs4eenbvo2bUzG9C6nlqH6+kZvH5FxeDMWU5Ai1ZVYWYT/NeIiIhIPlIIOwVeLEZ87lzic+cO6XeZDH1799KzY0cQzPyAlnroYTKp1OD1i4uJzZoVBLTZxOf4M2cFtbVYNH//KVx/P5nubjJd3bjuLjLd3UQbG+netBnzDDwPzBvZtjG2eR54nh9KPc/f90TbBtpy7shkwq4AAJdOkz56lP7DR8h0dmCRCESiWDQytB2NDu2PRrFITtsL9zNOLp3G9fSQ6enB9fbijh/32z29uN4eMsePZ9uup4fMcf/c35bT7unBHR9o+7fjenrI9PbienvxCgvxykqJlJYRKS0lUlaKVxpcLislUlqKl9O2REKPXZEx5O8r/1nEPI9YbS2x2lrImT1zzpE+dGjYzNlOOp98krYHHhi8fkEBsYYGP5hljz2bTayhAa+w8E+qwfX2+kEpCEuZ7i7caJe7gr6By51dg9fr7sJlt/sn19WF6+sbcX8VQOMZjtspGwhrowQ0i0aDF4NSvNJk8CKRxEuO7PMvl2bPvVhsov8S+clbWblvC+yeD+UzYNKMwfNJDVAyzf/3PQ2ur4/+I0dJHz5E/+HD9B867LcPHab/8LD20aOvTCD0vNHDWXA52x8Ngl0kCHaj9BONYNECyg4dpPneNX5A6ukh0xsEpBHtXhjlMXoqrKAAi8eDUwwvXjikHSkrwwoKyHR3kT54iN6du0inUmTa28G5E99wQYEf1gYeb2WlRJIjw9toQc4rLlKAk3OeQtg4MjOiU6YQnTKF4pWXDdmWbm+nd9cuenbuonfXTnp27uL41i20/+Y3gy8KZhTU1hKfNQuLxUaGpa7BAEV//6nVFo/jJRJYUQIvUYSXSPinKZUUJIrwioK+ogSWyNknuLxp6zYuWrQIXAaXyUDGgctAJoPLOP+Jedi2Ifs5d8Jtg+0/5Tacf5+9vaTb28mkUqRTKXoO7iDTliLd3o47fnzssSgsJJJMDr5ADAlpycEXjWROuBs4LykJfRbkrDT/Oo51R5jmemDXWmjfB+S8mEfiUF6XDWaZklrSVNCfKaG/N066vccPWMND1aFDpNvaRr1LSySIVlQQraigoK6OxJIlRCsriEyuIFpZgVeShEwa15/GpfshHbT7+wbb6X7oT+PSaUj34/r7T9Cf9reN1p8ObnOgv7+fTHdftu1fL020o52+9g7/sRqPE0mWYpV+ezAgxbF44Qna8ex1LdjmxWMj+2MxP/ydBpdOk+noIJ1KkW5LkUm1ZdvpVFvweGz3220p0keO0tu4h0xbG+n29rEDcDRKJBm8aSorGxrkgsdfYu8+Ut3deCUleCUlREpK8JL+49IrKtJjU/LeuIYwM7sW+CYQAX7onLt92PY48BPgEuAw8G7nXON41pQvIskkicWLSSxePKQ/09NDb2MjvTt3Ds6g7dqNS/fjFRXjJRIUTK06YTjyEkV4RX6gskTCD1NFgyHLEkV4icLTftId0FtQQHLYMXP5KtPbm30xyKT8J/90W4pMe/Bi0Z4a8mLRf+gQ6V27/L6TvVCYBeEsZ7YtmaS0s5OXn1qXMxuXzAa7SDJoJ5Pn73LNyg+zrW0Wkxcs8Mf7wH76m3fQv6+R9P5W+g8dov9oG+nUS/R3biHTN/oYeXGPaGkRkUmlxKumEL14HpHp9USnTiNSUUG0opJopR+8vOLiif0bz9DatWu5KM8fYxaJECkrI1JWBnWndl2XyZDp7DxxeBt4bLb5b6zSqRR9zc3ZNuk0pUDrXXedoDjDKy72w1myBK94IKAV+2GtJGgnk8G2nBBXXJLdZoWF5+djVCbEuIUwM4sA3wZeD7QAT5vZg865LTm7fRA46pybY2Y3AV8G3j1eNZ0NvHicwgsuoPCCC8Iu5ZzhxWJ4lZVEKytP+bouk/FnHNtOHt4yqXbSqRS9jbuJHTjI0Y0bcd3dY99BQYH/bj8nmA2eD19WTWYDX3YpNR4/zVE51TEIjgfsGn4KZmS7OrN92Vna4afg+q7TP6/q62PnKPcXKSsjUllJdFoN8YWTKa6oJFpWTCThiMZ6iXodRDlCpP9lvI4mONYE/TnjfATorYKOGZCaAcca4FDOkmdpDUS0CBA287zs/32oOaXrOufIdHbx+GO/4bJFi0h3dJDp6CTT0e632zv8GbqOdr+/vZ1MZwfpY8foa2kJtnWc/PEJ/oxcEOYGQ1ww2zYsxHmFCYgES9MDS9TmYREPvMgJzr3BfUfZZp4HkUj2fHDfkduGnMtZYTyfiVYAO5xzuwDM7B5gFZAbwlYBfx+01wDfMjNzbqyDDEQmjnkekWCZ41S+snft2rVcddVV/jJpR0d2Vi2dGgxumfYgwA282w+WU/v27w8CXvuQT92OWl8sNjK0DbSTOcuppUn/AxYjAlJncKxgl398YDYwDe73J71QZQuywdnXoiIsOI+Ul1NQXZ2zLUHjoUPMu3SFvyRYUUG0spLopEnYqR6j5xx0HIBje+DoHjjaCMca/XbzU7DpfnDpwf29qB/EJjXkHIvWACVTwfLnxavs2CZoVFgcjeEvryStlXjBFJiEfyIZnP40Lp0m03WcdGdwvGxXd9A+Tqazm3TXYHtwWwd9zQeH7E9/+uR3NtE8j6kG2zwPvOCDT2b+4HmGmee3cz8Ulb1s/mUs2Ney/dltNrh/9hhdGH3bkNv0DXmVH7hgNth2/t1n2yOuaLjcDYPdwT42yh2NvI2yOTOHHMs90cbzEV4DNOdcbgEuO9E+zrl+M2vDP+b70DjWJTJhLBYjOnkyTJ58WtfP9PRkA9xJg1yq3X+n39SU3W+sYwWtsHBIYBoIR5HKipzLxcFydxFecU64yi51Fw/2JxKntHSzZe3aEV8Dc1rMIFnln+pWjNye7oNUqx/Oju4ZDGvH9sCLD0PnwTOvYRwsBXg27Cry25mO0UCYO+nBGcXBacromzNpyPR5ZPoNgsNZcTb4Wu/AuZxtmcFtQ/pz9htxnRGXR7mvUS7jwGHZkJJbE8787ux+QZuc287py15O526zwdvN6Rvyt+eOwZCnhxPMt9iQs1PfPmxjTvQb0VneOfYxw+PtrHibZWYfAj4EUFVVxdq1a8f9Pjs6Oibkfs5WGp+xjev4FBf7p+nTx97POejrw+vqwrq6IRLBFcZx8TguFjvtTx8C/qfx2tr802kK5//QDIjMgEr8E+Clj1N4/ACx3mMTXMvYuru7SehXOcakMRqbPz5FALjsm6Ph0SUnoox4AzX0Os5G78818n7GikpuRNtGzFy5EfuaO/G2obcxWsgbej9H04XsCfG1bDxDWCtDD9WsDfpG26fFzKJAGf4B+kM4534A/ABg+fLl7qoJmDocWE6S0Wl8xqbxOTmN0djWrl3L5RqfMWmMxqbxObmwn4fG8wCIp4G5ZjbTzGLATcCDw/Z5ELg1aN8A/E7Hg4mIiMj5YNxmwoJjvP4SeBR/yf1HzrnNZvYF4Bnn3IPAvwA/NbMd+J9pumm86hERERHJJ+N6TJhz7iHgoWF9n8tpHwduHM8aRERERPJR/nweW0REROQ8ohAmIiIiEgKFMBEREZEQKISJiIiIhEAhTERERCQECmEiIiIiIVAIExEREQmBQpiIiIhICBTCREREREKgECYiIiISAoUwERERkRAohImIiIiEwJxzYddwSszsILBnAu6qEjg0AfdzttL4jE3jc3Iao7FpfE5OYzQ2jc/JTcQYzXDOTRltw1kXwiaKmT3jnFsedh35SuMzNo3PyWmMxqbxOTmN0dg0PicX9hhpOVJEREQkBAphIiIiIiFQCDuxH4RdQJ7T+IxN43NyGqOxaXxOTmM0No3PyYU6RjomTERERCQEmgkTERERCYFCmIiIiEgIFMKGMbNrzexFM9thZp8Ou558Y2Z1ZvZ7M9tiZpvN7K/CrikfmVnEzDaa2a/DriUfmVm5ma0xs21mttXMLg+7pnxiZn8dPL42mdlqMysMu6awmdmPzOyAmW3K6ZtsZo+Z2fbgfFKYNYbpBOPz1eAx9ryZ/cLMysOsMWyjjVHOtk+YmTOzyomsSSEsh5lFgG8DbwIWADeb2YJwq8o7/cAnnHMLgJXARzRGo/orYGvYReSxbwKPOOfmA4vRWGWZWQ3wMWC5c24REAFuCreqvHAHcO2wvk8Dv3XOzQV+G1w+X93ByPF5DFjknLsYeAn424kuKs/cwcgxwszqgDcATRNdkELYUCuAHc65Xc65XuAeYFXINeUV59w+59yGoN2O/+JZE25V+cXMaoHrgB+GXUs+MrMy4ErgXwCcc73OuWPhVpV3okDCzKJAEbA35HpC55z7d+DIsO5VwJ1B+07gbRNaVB4ZbXycc79xzvUHF/8I1E54YXnkBP+HAL4OfBKY8E8qKoQNVQM051xuQQHjhMysAVgKPBVuJXnnG/gP6EzYheSpmcBB4MfBku0Pzaw47KLyhXOuFfgn/Hfl+4A259xvwq0qb1U55/YF7f1AVZjF5LkPAA+HXUS+MbNVQKtz7rkw7l8hTE6LmZUA9wEfd86lwq4nX5jZ9cAB59z6sGvJY1FgGfBd59xSoJPzexlpiOC4plX4YbUaKDaz94ZbVf5z/vct6TuXRmFmn8U/lOSusGvJJ2ZWBHwG+FxYNSiEDdUK1OVcrg36JIeZFeAHsLucc/eHXU+euQJ4q5k14i9nv87MfhZuSXmnBWhxzg3MoK7BD2Xi+zNgt3PuoHOuD7gfeFXINeWrl81sOkBwfiDkevKOmb0fuB64xemLQYebjf9m57ngObsW2GBm0yaqAIWwoZ4G5prZTDOL4R8M+2DINeUVMzP8Y3m2Oue+FnY9+cY597fOuVrnXAP+/5/fOec0i5HDObcfaDazC4Kua4AtIZaUb5qAlWZWFDzerkEfXDiRB4Fbg/atwAMh1pJ3zOxa/EMj3uqc6wq7nnzjnHvBOTfVOdcQPGe3AMuC56gJoRCWIziA8S+BR/Gf9H7unNscblV55wrgffgzPM8GpzeHXZScdT4K3GVmzwNLgC+FXE/eCGYI1wAbgBfwn6fP+5+fMbPVwJPABWbWYmYfBG4HXm9m2/FnEG8Ps8YwnWB8vgUkgceC5+rvhVpkyE4wRuHWpNlJERERkYmnmTARERGRECiEiYiIiIRAIUxEREQkBAphIiIiIiFQCBMREREJgUKYiJzTzKwj7BpEREajECYiIiISAoUwETnvmNkSM/ujmT1vZr8Ifq8RM/uYmW0J+u8J+l6b88XEG80sGW71InKu0Je1isg5zcw6nHMlw/qeBz7qnPuDmX0BKHXOfdzM9gIznXM9ZlbunDtmZr8CbnfOPR78cP3x4Nc1RETOiGbCROS8YmZlQLlz7g9B153AlUH7efyfU3ovMBC0Hge+ZmYfC66nACYirwiFMBGRQdcB3waWAU+bWdQ5dzvwF0ACeNzM5odZoIicOxTCROS84pxrA46a2WuCrvcBfzAzD6hzzv0e+BRQBpSY2Wzn3AvOuS8DTwMKYSLyioiGXYCIyDgrMrOWnMtfA24FvmdmRcAu4DYgAvwsWK404J+DY8L+t5ldDWSAzcDDE1u+iJyrdGC+iIiISAi0HCkiIiISAoUwERERkRAohImIiIiEQCFMREREJAQKYSIiIiIhUAgTERERCYFCmIiIiEgI/j/PB9pt28j9mAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 720x504 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment