Last active
December 11, 2023 14:14
-
-
Save iamkucuk/9cbf985c85167261fed3196e86b39e21 to your computer and use it in GitHub Desktop.
Intuition behind the scaled dot product intuition
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"cells": [ | |
{ | |
"cell_type": "markdown", | |
"id": "3eda0e38-97eb-45d6-a5a9-493ab41e0bc8", | |
"metadata": {}, | |
"source": [ | |
"# Proof that scaled dot product actually works" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"id": "31d747ad-24f1-48d4-8836-65850c11a0fb", | |
"metadata": {}, | |
"source": [ | |
"## Intuition\n", | |
"\n", | |
"Lemma 1: Softmax function tends to reward the higher values, and destroys the lower values. It's not a linear scaling, but an \"attempt\" to create a function with some properties (mimic a probability distribution, destroy lower values etc.). These properties comes from the exponential behavior. Such that:\n", | |
"\n", | |
"$$lim_{x\\to\\infty}\\frac{e^x}{x} = \\infty$$\n", | |
"\n", | |
"Using this, let's assume $x_1 >> x_i$ which $x \\in X$. This leads to:\n", | |
"\n", | |
"$$lim_{x_1\\to\\infty}\\frac{x_1}{softmax(x_1, X)} \\equiv 1$$\n", | |
"\n", | |
"That means:\n", | |
"$$lim_{x_1\\to\\infty}{softmax(x_i, X)} \\leq 1 - softmax(x_1, X) \\equiv 0$$\n", | |
"\n", | |
"This forces the rest of the values be close to zero.\n", | |
"\n", | |
"Lemma 2: Each dimension contributes $E[var[x]] = 1$ to the decision boundary change (as per a normal distribution among embeddings was assumed.)\n", | |
"\n", | |
"$$\\sum_{i=1}^{d} var(x_i) = d$$\n", | |
"\n", | |
"And we know that\n", | |
"\n", | |
"$$std(d) = \\sqrt{d}$$\n", | |
"\n", | |
"This leads a scaling factor of $\\sqrt{d}$ for scaling the dot product which makes its standard deviation 1." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 1, | |
"id": "5c1ee818-4809-4a05-8950-576f53c70d2f", | |
"metadata": {}, | |
"outputs": [], | |
"source": [ | |
"import numpy as np\n", | |
"import matplotlib.pyplot as plt" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 2, | |
"id": "2b7bb95f-4ae6-4059-9797-aed19e9f3353", | |
"metadata": {}, | |
"outputs": [], | |
"source": [ | |
"def softmax(x):\n", | |
" \"\"\"Compute softmax values for each set of scores in x.\"\"\"\n", | |
" e_x = np.exp(x - np.max(x))\n", | |
" return e_x / e_x.sum(axis=0)" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 3, | |
"id": "67e016de-f8ce-4b8e-a975-c90ff33f4333", | |
"metadata": {}, | |
"outputs": [], | |
"source": [ | |
"def plot_more_zoomed_histograms(original_scores, scaled_scores, title):\n", | |
" \"\"\"Plot more zoomed-in histograms of the softmax scores before and after scaling.\"\"\"\n", | |
" softmax_original = softmax(original_scores)\n", | |
" softmax_scaled = softmax(scaled_scores)\n", | |
"\n", | |
" plt.figure(figsize=(12, 6))\n", | |
"\n", | |
" # More zoomed histogram for original softmax scores\n", | |
" plt.subplot(1, 2, 1)\n", | |
" plt.hist(softmax_original, bins=100, range=(0, 0.01))\n", | |
" plt.title(\"More Zoomed Histogram of Softmax Scores (Original)\")\n", | |
" plt.xlabel(\"Softmax Score\")\n", | |
" plt.ylabel(\"Frequency\")\n", | |
"\n", | |
" # More zoomed histogram for scaled softmax scores\n", | |
" plt.subplot(1, 2, 2)\n", | |
" plt.hist(softmax_scaled, bins=100, range=(0, 0.01))\n", | |
" plt.title(\"More Zoomed Histogram of Softmax Scores (Scaled)\")\n", | |
" plt.xlabel(\"Softmax Score\")\n", | |
"\n", | |
" plt.suptitle(title)\n", | |
" plt.show()" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 4, | |
"id": "cdefecb8-c4af-44b0-83f7-86d1e2584079", | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"data": { | |
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA/UAAAJJCAYAAADrxBdDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAACLU0lEQVR4nOzde3zP9f//8fvbDu8dbGsHOzlsyKHaRJRDMcz5UA7llFNJyuGTUyGfmD6yotCH8PmWQ3LsgERhGj4JhaSoRBFi1GhjbGyevz/89v5428E2m+2t2/VyeV0uez1fz9fr9Xw+3+/X67nH6/B8W4wxRgAAAAAAwOGUKu4CAAAAAACAgiGoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHisCCBQtksVhksVi0efPmLMuNMbrzzjtlsVjUuHHjW16+nMTExNjKndt0OwsPD1ffvn1vmM9isWjw4MHZLvvwww+zfPaZbZsfFy5cUExMTLbfIRTc559/rjp16sjT01MWi0WrVq3KMe+xY8c0cOBAVa1aVe7u7vLz81NkZKT69++vY8eOFWj/y5cv1z333CN3d3dZLBZ9++23mjVrlhYsWFCwCpUQ69evV4sWLRQaGiqr1arQ0FA1btxYr776anEXrUicOXNG3bp1U2BgoCwWizp06FCk+2vcuLEiIiKKdB+3Ut++fWWxWHTPPfcoIyMjy/LczrFFLfN8/eeffxbL/vMrL+e0N998UxaLRevWrctxO2+//bYsFotWrFiR5303bty4RP0fA/xdORd3AYDbmZeXl+bOnZulw9uyZYt++eUXeXl5FU/BcvDUU0+pVatW2S5bv369YmJiivwf19tVbm2bkwsXLmjChAmSxD9NhcQYoy5duqhq1apavXq1PD09Va1atWzzHj9+XPfdd5/uuOMOjRgxQtWqVVNSUpJ++OEHvf/++/r1119Vvnz5fO3/jz/+UK9evdSqVSvNmjVLVqtVVatWVc+ePRUQEJCnC0ol0Zw5c/Tss8+qc+fOmjlzpvz8/HTs2DFt27ZNH374oUaPHl3cRSx0//rXv7Ry5UrNmzdPlStXlp+fX3EXySH98MMPWrBggfr161fcRXFIeT2n9ezZU6NGjdK8efNy7Ivmz5+vMmXKqH379kVdbACFjKAeKEJdu3bV4sWL9dZbb8nb29uWPnfuXNWvX1/JycmFti9jjFJTU+Xu7l7gbZQrV07lypXLkv7rr7/qzTffVLVq1fTuu+/eTDH/tnJq25Lu4sWLN/WdKmlOnDihM2fOqGPHjoqOjs4179tvv60///xTX3/9tSpWrGhL79Chg1588UVduXIl3/v/+eefdfnyZfXs2VNRUVH5Xr+kio2NVaNGjfThhx/apffq1atA7XQzLly4IA8PjyLfz759+1S5cmU9/vjjhbK9wjiH34yLFy/Kzc3tlj6N5enpqfvuu0/jx49Xjx49bqtzTV4Uxnc1r+c0f39/PfLII1q1apUSExPl7+9vt/ynn37S9u3bNWLECLm4uNxUmQDcejx+DxSh7t27S5KWLl1qS0tKStJHH32kJ598Mtt1zpw5o4EDB6ps2bJydXVVpUqVNHbsWKWlpdnly3w0cc6cObrrrrtktVptAffBgwfVo0cPBQYGymq16q677tJbb71VoDqkpKSoQ4cOunz5slauXGl3cUKSVq9erfr168vDw0NeXl5q3ry5tm/fnmU7W7duVXR0tLy8vOTh4aEGDRpo7dq1dnkyX1uIj49X//795e/vL29vb/Xu3VspKSlKSEhQly5ddMcddygkJEQjR47U5cuX7bZx6dIlTZw4UdWrV5fValWZMmX0xBNP6I8//rDLd/nyZb3wwgsKDg6Wh4eHHnroIX399dcFaqO8yO7x+/j4eDVu3Fj+/v5yd3dXhQoV1LlzZ124cEFHjhxRmTJlJEkTJkywvfpw7Z3cvLRpZr769evLzc1NZcuW1UsvvaR33nlHFotFR44cseULDw9Xu3bttGLFCtWqVUtubm62JwXeeustNWrUSIGBgfL09FRkZKQmT56cpf0zHxHevn27GjRoIHd3d4WHh2v+/PmSpLVr1+q+++6Th4eHIiMjszwK+scff+jpp59W+fLlbZ/fgw8+qI0bN96wjW/UHjExMbYLK6NGjZLFYlF4eHiO20tMTFSpUqUUGBiY7fJSpey70BsdC3379tVDDz0k6eoFv8zXb8LDw7V//35t2bLF9jlnlmvz5s2yWCxasmSJRo0apZCQEJUuXVrt27fXqVOndO7cOT399NMKCAhQQECAnnjiCZ0/f96uXHn57A4ePChvb2899thjduvGx8fLyclJL730Uo7tlNlWISEheWqnK1euaMaMGapZs6bc3d11xx13qF69elq9erVdnsmTJ9uO48DAQPXu3VvHjx+321bm9+2///2vGjRoIA8PD9u5NTk5WSNHjlTFihXl6uqqsmXLaujQoUpJSbHbxgcffKC6devKx8dHHh4eqlSpUo7nZ0k6cuSILBaLNm7cqB9//DHLq1aFcQ4vqF27dqlbt24KDw+3HXvdu3fXb7/9Zpcv81y7YcMGPfnkkypTpow8PDyUlpYmY4wmTZqksLAwubm5qU6dOoqLi8v2Meu8tnFuXnvtNf3+++968803c82XWeZrz1nS/46Ra19TutnzUKZjx46pU6dO8vb2lo+Pj3r27JmlL5GuvlJTv359eXp6qnTp0mrZsqX27Nljl6dv374qXbq0vv/+e7Vo0UJeXl43vLBY2Oe0fv366dKlS1qyZEmWZZltk/ndnzBhgurWrSs/Pz95e3vrvvvu09y5c2WMybXM2X0e0v+Om+tfM9q1a5cefvhh+fn5yc3NTbVq1dL7779vl+fChQu275mbm5v8/PxUp04du/+tgL89A6DQzZ8/30gyO3fuNL169TIPPPCAbdns2bONp6enSU5ONvfcc4+JioqyLbt48aKpUaOG8fT0NK+//rrZsGGDeemll4yzs7Np06aN3T4kmbJly5oaNWqYJUuWmPj4eLNv3z6zf/9+4+PjYyIjI83ChQvNhg0bzIgRI0ypUqVMTExMvuvStWtXI8l8+OGHWZYtXrzYSDItWrQwq1atMsuXLze1a9c2rq6u5osvvrDl27x5s3FxcTG1a9c2y5cvN6tWrTItWrQwFovFLFu2LEu7VaxY0YwYMcJs2LDBvPbaa8bJycl0797d3HfffWbixIkmLi7OjBo1ykgyb7zxhm39jIwM06pVK+Pp6WkmTJhg4uLizDvvvGPKli1r7r77bnPhwgVb3j59+hiLxWKef/55s2HDBjN16lRTtmxZ4+3tbfr06XPDdpFkBg4caC5fvpxlWr58uZFkNm3aZMs/fvx4c+0p9/Dhw8bNzc00b97crFq1ymzevNksXrzY9OrVy5w9e9akpqaadevWGUmmX79+Zvv27Wb79u3m0KFD+WrTvXv3Gjc3N1OjRg2zbNkys3r1atOmTRsTHh5uJJnDhw/b8oaFhZmQkBBTqVIlM2/ePLNp0ybz9ddfG2OMGTZsmJk9e7ZZt26diY+PN9OmTTMBAQHmiSeesGuXqKgo4+/vb6pVq2bmzp1r1q9fb9q1a2ckmQkTJpjIyEizdOlS8+mnn5p69eoZq9Vqfv/9d9v6LVu2NGXKlDH/93//ZzZv3mxWrVplxo0bZ1en7OSlPY4dO2ZWrFhhJJkhQ4aY7du3m2+++SbHbS5atMj2/V63bp1JSkrKMW9ejoVDhw6Zt956y0gykyZNMtu3bzf79+8333zzjalUqZKpVauW7XPOLNemTZuMJBMWFmb69u1r1q1bZ+bMmWNKly5tmjRpYpo3b25Gjhxpd6wMGTLErmx5/eyWLVtmJJk333zTGGPMyZMnTVBQkImKijLp6em5tn+zZs2Ms7OzGT9+vPn2229zzd+rVy9jsVjMU089ZT7++GPz2WefmVdeecW2X2OMefrpp40kM3jwYFudy5QpY8qXL2/++OMPW76oqCjj5+dnypcvb2bMmGE2bdpktmzZYlJSUkzNmjVNQECAmTp1qtm4caN58803jY+Pj2natKm5cuWKMcaYbdu2GYvFYrp162Y+/fRTEx8fb+bPn2969eqVY/lTU1PN9u3bTa1atUylSpVsn1lSUlKhnMNzEhUVZe65555cP4cPPvjAjBs3zqxcudJs2bLFLFu2zERFRZkyZcrYtVvmubZs2bLm6aefNp999pn58MMPTXp6uhkzZoyRZJ5++mmzbt068/bbb5sKFSqYkJAQu/4qr22ckz59+hhPT09jjDEdO3Y0d9xxh0lMTLRrn0GDBmUp87XnLGP+d4xce7692fNQ5vk6LCzMPP/882b9+vVm6tSpxtPT09SqVctcunTJlveVV14xFovFPPnkk2bNmjVmxYoVpn79+sbT09Ps37/frr4uLi4mPDzcxMbGms8//9ysX78+x/YpinNaRkaGCQsLMzVr1rRLT09PNyEhIaZevXq2tL59+5q5c+eauLg4ExcXZ/71r38Zd3d3M2HCBLt1o6Ki7L4X2X0exlzt8ySZ+fPn29Li4+ONq6uradiwoVm+fLlZt26d6du3b5Z8AwYMMB4eHmbq1Klm06ZNZs2aNebVV181M2bMyLGuwN8NQT1QBK4N6jM7uMx/1u6//37Tt29fY4zJEtTPmTPHSDLvv/++3fZee+01I8ls2LDBlibJ+Pj4mDNnztjlbdmypSlXrlyWAGTw4MHGzc0tS/7cTJ482Ugyo0ePzrIsIyPDhIaGmsjISJORkWFLP3funAkMDDQNGjSwpdWrV88EBgaac+fO2dLS09NNRESEKVeunO2fv8x2uz4o6dChg5Fkpk6dapdes2ZNc99999nmly5daiSZjz76yC7fzp07jSQza9YsY4wxP/74o5Fkhg0bZpcvMzDLa1B/oym3oP7DDz80ksy3336b4z7++OMPI8mMHz8+y7K8tuljjz1mPD097f6hz8jIMHfffXe2Qb2Tk5M5cOBArnXPyMgwly9fNgsXLjROTk5236moqCgjyezatcuWlpiYaJycnIy7u7vdP87ffvutkWT+/e9/29JKly5thg4dmuv+s5PX9sj8x3LKlCk33OaVK1fMgAEDTKlSpYwkY7FYzF133WWGDRtm1275ORYyzwcffPCB3b6uPxdcn799+/Z26UOHDjWSzD/+8Q+79A4dOhg/P78c65TbZ2eMMc8++6xxdXU127dvN02bNjWBgYHmxIkTOW4v06FDh0xERITtu+/u7m6io6PNzJkz7QKg//73v0aSGTt2bI7byjw+Bw4caJf+1VdfGUnmxRdftKVlft8+//xzu7yxsbGmVKlSZufOnXbpmcfdp59+aowx5vXXXzeSzF9//XXDOl4vuyC7MM7h+dnfjaSnp5vz588bT09Pu4smmefa3r172+U/c+aMsVqtpmvXrnbp27dvN5LsvqN5beOcXBvU//TTT8bJycmMGDHCtvxmg/qbOQ9lnq9z6iMWLVpkjDHm6NGjxtnZOUufde7cORMcHGy6dOliV19JZt68ebm2S6aiOKddW7drg/9PPvnESDJvv/12tutknjdefvll4+/vb3fB5maC+urVq5tatWqZy5cv2+Vt166dCQkJsZ1PIyIiTIcOHfJUP+DvisfvgSIWFRWlypUra968efr++++1c+fOHB/tjI+Pl6enpx599FG79MxHrj///HO79KZNm8rX19c2n5qaqs8//1wdO3aUh4eH0tPTbVObNm2UmpqqHTt25KncGzdu1JgxY9S8eXO98sorWZYfOHBAJ06cUK9evewery1durQ6d+6sHTt26MKFC0pJSdFXX32lRx99VKVLl7blc3JyUq9evXT8+HEdOHDAbtvt2rWzm7/rrrskSW3bts2Sfu1jpWvWrNEdd9yh9u3b29W9Zs2aCg4Otj0OuGnTJknK8i5sly5d5Oyc96FGunTpop07d2aZXnvttRuuW7NmTbm6uurpp5/Wu+++q19//TXP+81Pm27ZskVNmzZVQECALV+pUqXUpUuXbLddo0YNVa1aNUv6nj179PDDD8vf319OTk5ycXFR7969lZGRoZ9//tkub0hIiGrXrm2b9/PzU2BgoGrWrKnQ0FBbeubneu1n+MADD2jBggWaOHGiduzYkeXx/pttj/ywWCyaM2eOfv31V82aNUtPPPGELl++rGnTpumee+7Rli1bJOX9WLgZ+Tkmzpw5Y/cIfn4+u8y6NWnSRJs3b9aiRYtyfKz+WpUrV9bevXu1ZcsWTZgwQc2aNdPOnTs1ePBg1a9fX6mpqZKkzz77TJI0aNCgHLeVeXxeP2jgAw88oLvuuivLedDX11dNmza1S1uzZo0iIiJUs2ZNu3NBy5Yt7R4Nvv/++yVdPZbff/99/f777zesa25u9hx+s86fP69Ro0bpzjvvlLOzs5ydnVW6dGmlpKToxx9/zJK/c+fOdvM7duxQWlpalvNDvXr1sjzWndc2zotq1aqpX79+mjlzpo4ePZrn9XJzM+ehTDn1EZnf0fXr1ys9PV29e/e2awM3NzdFRUVl2wbXt3l2iuqcJklPPPGESpUqpXnz5tnS5s+fL09PT3Xt2tWWFh8fr2bNmsnHx8d23hg3bpwSExN1+vTpAu37WocOHdJPP/1ka+Pr/185efKkrY4PPPCAPvvsM40ePVqbN2/WxYsXb3r/wO2GoB4oYhaLRU888YQWLVqkOXPmqGrVqmrYsGG2eRMTExUcHJzl3evAwEA5OzsrMTHRLv36f7YTExOVnp6uGTNmyMXFxW5q06aNJOXpJ3qOHDmibt26qVy5clq6dGmWd2Iz95VdGSQpNDRUV65c0dmzZ3X27FkZY3LMd+22Ml0/irSrq2uO6ZnBgiSdOnVKf/31l1xdXbPUPyEhwVb3zP0FBwfbbc/Z2TnL4EG5KVOmjOrUqZNlqlSp0g3XrVy5sjZu3KjAwEANGjRIlStXVuXKlW/4XqmkfLVpYmKigoKCsuTLLk3K/vM8evSoGjZsaHvv9YsvvtDOnTtt4zRc/w9WdqOAu7q65vi5XvsZLl++XH369NE777yj+vXry8/PT71791ZCQkK25ZXy1x4FERYWpmeffVZz587VwYMHtXz5cqWmpur555+32/aNjoWbkZ9jQvpfm+b3s7NarerRo4dSU1NVs2ZNNW/ePM9lLFWqlBo1aqRx48Zp9erVOnHihLp27ardu3fbAog//vhDTk5OWY69a92oPW90HpSungu+++67LOcBLy8vGWNs54JGjRpp1apVtsCsXLlyioiIKPC7ujd7Dr9ZPXr00MyZM/XUU09p/fr1+vrrr7Vz506VKVMm20Aouz5Eyv78cH1aXts4r2JiYvI0fkNe3cx5KFNOfURmO506dUrS1YtD17fD8uXLs7SBh4dHlnFpslOU57SwsDBFR0dryZIlSktL059//qk1a9boscces/0iz9dff60WLVpIujpg6JdffqmdO3dq7NixkrKeNwois+1GjhyZpe0GDhwo6X//r/z73//WqFGjtGrVKjVp0kR+fn7q0KGDDh48eNPlAG4XjH4P3AJ9+/bVuHHjNGfOnGzvemfy9/fXV199JWOM3T+Fp0+fVnp6ut3dVklZ/nH09fW1XcnP6U7YtaN4Z+fixYvq1KmTUlJStGHDhhyD3Mz0kydPZll24sQJlSpVSr6+vjLGqFSpUjnmk5SlXgUVEBAgf3//HAc9yvyHJbPsCQkJKlu2rG15enr6TQV/+dWwYUM1bNhQGRkZ2rVrl2bMmKGhQ4cqKChI3bp1y3E9X1/fPLepv7+/7Z+na+UUJGc38vWqVauUkpKiFStWKCwszJb+7bff5lq/gggICND06dM1ffp0HT16VKtXr9bo0aN1+vTpHD/X/LRHYejSpYtiY2O1b98+SXk/FopDfj+7ffv2ady4cbr//vu1c+dOTZ06VcOHDy/Qvj09PTVmzBgtX77c1lZlypRRRkaGEhIScgxor23P638x4sSJEzc8D0pXP293d3e7u5HXL8/0yCOP6JFHHlFaWpp27Nih2NhY9ejRQ+Hh4apfv37eK6ybP4ffjKSkJK1Zs0bjx4+3+wnBtLQ0nTlzJtt1rt9/ZtvndM649m59fto4L0JCQjR06FC9+uqrGjFiRJblbm5ukpRlwMGi/C35nPqIzHbKrOOHH35od3zlJK+fd1Gf0/r166e4uDh9/PHHOnHihC5dumT3k4LLli2Ti4uL1qxZY2t36er55Eby+jllln/MmDHq1KlTttvK/Gk+T09PTZgwQRMmTNCpU6dsd+3bt2+vn3766cYVBv4GuFMP3AJly5bV888/r/bt26tPnz455ouOjtb58+ezdJwLFy60Lc+Nh4eHmjRpoj179qhGjRrZ3kW+0Z3o/v37a8+ePZozZ47uu+++HPNVq1ZNZcuW1ZIlS+xGw01JSdFHH31kGwXc09NTdevW1YoVK+yu7l+5ckWLFi1SuXLlsn3cuyDatWunxMREZWRkZFv3zH8QMkdwXrx4sd3677//vtLT0wulLPnh5OSkunXr2u6efvPNN5Ku3jWVst4VyU+bRkVFKT4+3u4fqitXruiDDz7Ic/ky/xHNLI909ee33n777fxUM98qVKigwYMHq3nz5rY2yU5Rfcey+4dauvqI87Fjx2x3zPJ6LOTGarUWySOl+fnsUlJS9Nhjjyk8PFybNm3S4MGDNXr0aH311Vc33E9ObZX5yHdmW7Vu3VqSNHv27By3lfko/aJFi+zSd+7cqR9//PGG50Hp6rngl19+kb+/f7bnguxGCLdarYqKirK9PnP96OV5cbPn8JthsVhkjLH7rCXpnXfeUUZGRp62UbduXVmtVi1fvtwufceOHVkeTy9IG9/IqFGj5OfnZ3dRIlPm9r777ju79Gt/NaGw5dRHZPYhLVu2lLOzs3755Zds26BOnToF2m9R95sdOnSQv7+/5s2bp/nz56tq1aq2X+aQrn6XnJ2d5eTkZEu7ePGi3nvvvRtuO6+fU7Vq1VSlShXt3bs3x7bLvBB/raCgIPXt21fdu3fXgQMHbvrVJuB2wZ164BZ59dVXb5ind+/eeuutt9SnTx8dOXJEkZGR2rp1qyZNmqQ2bdqoWbNmN9zGm2++qYceekgNGzbUs88+q/DwcJ07d06HDh3SJ598ovj4+FzXXbx4sZo2bapq1arl+P59rVq1ZLVaNXnyZD3++ONq166dBgwYoLS0NE2ZMkV//fWXXX1jY2PVvHlzNWnSRCNHjpSrq6tmzZqlffv2aenSpYV2t6pbt25avHix2rRpo+eee04PPPCAXFxcdPz4cW3atEmPPPKIOnbsqLvuuks9e/bU9OnT5eLiombNmmnfvn16/fXX8/RoZGGYM2eO4uPj1bZtW1WoUEGpqam2O16Zn7OXl5fCwsL08ccfKzo6Wn5+fgoICFB4eHie23Ts2LH65JNPFB0drbFjx8rd3V1z5syx/eRUdq9WXK958+ZydXVV9+7d9cILLyg1NVWzZ8++6UfKr5eUlKQmTZqoR48eql69ury8vLRz506tW7cuxzs5mYriO/bKK6/oyy+/VNeuXW0/v3b48GHNnDlTiYmJmjJliqSrbZjXYyEnkZGRWrZsmZYvX65KlSrJzc1NkZGR+S7z9fLz2T3zzDM6evSovv76a3l6euqNN97Q9u3b1a1bN+3Zs0d33HFHjvu55557FB0drdatW6ty5cpKTU3VV199pTfeeENBQUG2u4ANGzZUr169NHHiRJ06dUrt2rWT1WrVnj175OHhoSFDhqhatWp6+umnNWPGDJUqVUqtW7fWkSNH9NJLL6l8+fIaNmzYDes9dOhQffTRR2rUqJGGDRumGjVq6MqVKzp69Kg2bNigESNGqG7duho3bpyOHz+u6OholStXTn/99ZfefPNNubi4KCoqKt/tXRjn8NwkJyfrww8/zJJepkwZRUVFqVGjRpoyZYrtPLFlyxbNnTs318/uWn5+fho+fLhiY2Pl6+urjh076vjx45owYYJCQkLszhd5beP88Pb21tixY7P9jO+//35Vq1ZNI0eOVHp6unx9fbVy5Upt3bo1X/vIjxUrVsjZ2VnNmzfX/v379dJLL+nee++1jTkQHh6ul19+WWPHjtWvv/6qVq1aydfXV6dOnbIdR5k/C5pfRdlvWq1WPf7445oxY4aMMVnOUW3bttXUqVPVo0cPPf3000pMTNTrr7+e5YJRdoKDg9WsWTPbdygsLEyff/65VqxYkSXvf/7zH7Vu3VotW7ZU3759VbZsWZ05c0Y//vijvvnmG9vF57p166pdu3aqUaOGfH199eOPP+q9997L0wVT4G+jGAbnA257145+n5vsRrxOTEw0zzzzjAkJCTHOzs4mLCzMjBkzxqSmptrl03UjA1/r8OHD5sknnzRly5Y1Li4upkyZMqZBgwZm4sSJuZYnc8TgG03Xjj68atUqU7duXePm5mY8PT1NdHS0+fLLL7Ns+4svvjBNmzY1np6ext3d3dSrV8988skndnlyarfM0XqvHcHdGPvRkzNdvnzZvP766+bee+81bm5upnTp0qZ69epmwIAB5uDBg7Z8aWlpZsSIESYwMNC4ubmZevXqme3bt5uwsLA8j36fU/t/8MEHNxz9fvv27aZjx44mLCzMWK1W4+/vb6Kioszq1avttrVx40ZTq1YtY7Vas4zMn5c2zcxXt25dY7VaTXBwsHn++edto3FfO+p3WFiYadu2bbZ1+uSTT2xtWrZsWfP888+bzz77LNtRp7MboTunbV/bjqmpqeaZZ54xNWrUMN7e3sbd3d1Uq1bNjB8/3qSkpGRbruvreaP2yM9I0Tt27DCDBg0y9957r/Hz8zNOTk6mTJkyplWrVtmO7J2XYyGn0e+PHDliWrRoYby8vGw/pZVb/vwcK3n57N5+++0sI1Mbc3VUe29v7xuOPP2f//zHdOrUyVSqVMl4eHgYV1dXU7lyZfPMM8+YY8eO2eXNyMgw06ZNMxEREcbV1dX4+PiY+vXr231WGRkZ5rXXXjNVq1Y1Li4uJiAgwPTs2TPLtnIbEf78+fPmn//8p6lWrZptP5GRkWbYsGEmISHBGGPMmjVrTOvWrU3ZsmWNq6urCQwMNG3atLH7Sc6c5LTvwjiH57S/nM7Jmf3I8ePHTefOnY2vr6/x8vIyrVq1Mvv27ctyXsutj7py5YqZOHGiKVeunHF1dTU1atQwa9asMffee6/p2LGjXd68tHFOsjt/G3P13FyxYsVs2+fnn382LVq0MN7e3qZMmTJmyJAhZu3atYV6HjLmf8fR7t27Tfv27U3p0qWNl5eX6d69uzl16lSW9VetWmWaNGlivL29jdVqNWFhYebRRx81GzduvGF9c1PY57Rr7d2710gyTk5O2f7Cxbx580y1atWM1Wo1lSpVMrGxsWbu3LlZ/ge4fvR7Y67+HOajjz5q/Pz8jI+Pj+nZs6fZtWtXtueYvXv3mi5dupjAwEDj4uJigoODTdOmTc2cOXNseUaPHm3q1KljfH19beUZNmyY+fPPP/NVZ+B2ZjHmmmcFAQB/Cy1atNCRI0eyjH4OANc7fPiwqlevrvHjx+vFF18s7uIAAK7D4/cAcJsbPny4atWqpfLly+vMmTNavHix4uLiNHfu3OIuGoASZu/evVq6dKkaNGggb29vHThwQJMnT5a3t7fdYGoAgJKDoB4AbnMZGRkaN26cEhISZLFYdPfdd+u9995Tz549i7toAEoYT09P7dq1S3PnztVff/0lHx8fNW7cWK+88kqOP4UJAChePH4PAAAAAICD4iftAAAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADoqgHgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoG7roH7BggWyWCyyWCzavHlzluXGGN15552yWCxq3LjxLS9fTmJiYmzlzm26nYWHh6tv3743zGexWDR48OBsl3344YdZPvvMts2PCxcuKCYmJtvvELJ35MgRtW3bVn5+frJYLBo6dGiOeVNSUvTaa6/p3nvvlbe3t7y8vFS5cmV16dJFW7ZsKdD+9+zZo6ioKPn4+MhisWj69On69NNPFRMTU7AKlRA//vijevXqpUqVKsnNzU0BAQG67777NHjwYCUnJxd38YpMdHS0nnnmmSzpP/74o/r27asKFSrI1dVVAQEBatOmjT777LN8bT+zrzhy5Ei+y3bkyBFZLBYtWLAg3+vmR9++fRUeHm6bv3z5sipXrqzp06cX6X4dFf2/46L/d2z0/0WD/v9/iqMtNm/enGN/UlDZ/e/RqFGjXI+Z3DgXTrFKNi8vL82dOzdLx71lyxb98ssv8vLyKp6C5eCpp55Sq1atsl22fv16xcTEqEOHDre2ULeJ3No2JxcuXNCECRMkqUT981eSDRs2TF999ZXmzZun4OBghYSEZJsvIyNDLVq00Pfff6/nn39eDzzwgCTp4MGD+uSTT/TFF18oKioq3/t/8sknlZKSomXLlsnX11fh4eGaOHGi3nrrLYft2Pfs2aMHH3xQd911l8aNG6fw8HD9+eef2rt3r5YtW6aRI0fK29u7uItZ6D7++GN9+eWXWrhwoV36ihUr1KNHD1WqVEkvvfSSqlWrplOnTmn+/Plq06aNnn/+eU2ePDlP+2jbtq22b9+e4/c0NyEhIdq+fbsqV66c73VvhouLi8aNG6dhw4apV69e8vf3v6X7dxT0/8hE/39r0P8XPvr///X/t3tb/Otf/1Lz5s317LPPqlq1avlb2dzG5s+fbySZp556yri7u5ukpCS75T179jT169c399xzj4mKiiqUfV65csVcuHChULZ1vV9++cX4+vqaatWqZanL7SYsLMz06dPnhvkkmUGDBmW77IMPPjCSzKZNm26qLH/88YeRZMaPH39T27kZRfWdKip33nmnad269Q3zxcfHG0lm3rx52S7PyMgo0P6dnZ3Ns88+a5c2aNAg48invN69extPT0+TnJyc7fIrV67csrKkpKTcsn098MADplu3bnZphw4dMh4eHqZOnTrm/PnzWdZ55plnjCSzdOnSXLd94cKFW9puN6NPnz4mLCzMLi0tLc34+fmZV155pXgKVYLR/zsu+n979P/5Q/9ftIq7/y+utti0aVOhnFOuldlPHT582C49IiLC9O/fP9/bu60fv8/UvXt3SdLSpUttaUlJSfroo4/05JNPZrvOmTNnNHDgQJUtW1aurq6qVKmSxo4dq7S0NLt8mY9/zZkzR3fddZesVqveffddSVevNvbo0UOBgYGyWq2666679NZbbxWoDikpKerQoYMuX76slStXZrkKtXr1atWvX18eHh7y8vJS8+bNtX379izb2bp1q6Kjo+Xl5SUPDw81aNBAa9eutcuT+ThIfHy8+vfvL39/f3l7e6t3795KSUlRQkKCunTpojvuuEMhISEaOXKkLl++bLeNS5cuaeLEiapevbqsVqvKlCmjJ554Qn/88YddvsuXL+uFF15QcHCwPDw89NBDD+nrr78uUBvlRXaP38XHx6tx48by9/eXu7u7KlSooM6dO+vChQs6cuSIypQpI0maMGGC7dHHax8NzEubZuarX7++3NzcVLZsWb300kt65513sjx6Ex4ernbt2mnFihWqVauW3NzcbHcK3nrrLTVq1EiBgYHy9PRUZGSkJk+enKX9GzdurIiICG3fvl0NGjSQu7u7wsPDNX/+fEnS2rVrdd9998nDw0ORkZFat25dntrv6NGj6tmzp913+o033tCVK1ck/e/xpEOHDumzzz6ztVdOjzUnJiZKUo5X8kuVsj9F7du3T4888oh8fX3l5uammjVr2o436X/f3fT0dM2ePdvu88o89q59hDWzXJnH8fz581WtWjW5u7urTp062rFjh4wxmjJliipWrKjSpUuradOmOnTokF254uLi9Mgjj6hcuXJyc3PTnXfeqQEDBujPP/+05UlNTVWtWrV05513KikpyZaekJCg4OBgNW7cWBkZGTm2fWJiory9vVW6dOlsl1//vV63bp2io6Pl4+MjDw8P3XXXXYqNjbXLk5fzRuYx88033+jRRx+Vr6+v7a60MUazZs1SzZo15e7uLl9fXz366KP69ddf7baxZ88etWvXzva9CQ0NVdu2bXX8+PEc65u53tdff61evXrZpU+bNk0XLlzQjBkz5OnpmWW9N954Q3fccYdeeeUVW1rmd2PDhg168sknVaZMGXl4eCgtLS3bR+CMMZo0aZLCwsLk5uamOnXqKC4uTo0bN7a7Y5fd4/eZbbZ//351795dPj4+CgoK0pNPPmn32Ut5P6az4+rqqq5du+r//u//ZIy5Yf6/I/r//6H/p/+X6P/p/69y1P6/sNti165d6tatm8LDw23HSvfu3fXbb7/lWr5r13/44Yfl5+cnNzc31apVS++//36WfDt27NCDDz4oNzc3hYaGasyYMTn287169dKSJUt07ty5PJXB5qYvM5RgmVdAdu7caXr16mUeeOAB27LZs2fbrvRcf6X+4sWLpkaNGsbT09O8/vrrZsOGDeall14yzs7Opk2bNnb7kGTKli1ratSoYZYsWWLi4+PNvn37zP79+42Pj4+JjIw0CxcuNBs2bDAjRowwpUqVMjExMfmuS9euXY0k8+GHH2ZZtnjxYiPJtGjRwqxatcosX77c1K5d27i6upovvvjClm/z5s3GxcXF1K5d2yxfvtysWrXKtGjRwlgsFrNs2bIs7VaxYkUzYsQIs2HDBvPaa68ZJycn0717d3PfffeZiRMnmri4ODNq1Cgjybzxxhu29TMyMkyrVq2Mp6enmTBhgomLizPvvPOOKVu2rLn77rvtrjr36dPHWCwW8/zzz5sNGzaYqVOnmrJlyxpvb+88X6kfOHCguXz5cpZp+fLlWa6qjR8/3u5q7eHDh42bm5tp3ry5WbVqldm8ebNZvHix6dWrlzl79qxJTU0169atM5JMv379zPbt28327dvNoUOH8tWme/fuNW5ubqZGjRpm2bJlZvXq1aZNmzYmPDw8y1W6sLAwExISYipVqmTmzZtnNm3aZL7++mtjjDHDhg0zs2fPNuvWrTPx8fFm2rRpJiAgwDzxxBN27RIVFWX8/f1NtWrVzNy5c8369etNu3btjCQzYcIEExkZaZYuXWo+/fRTU69ePWO1Ws3vv/+ea1ufPn3alC1b1pQpU8bMmTPHrFu3zgwePNhIsl0VT0pKMtu3bzfBwcHmwQcftLVXampqtts8fPiwcXFxMVWrVjWLFi0yJ06cyHH/P/30k/Hy8jKVK1c2CxcuNGvXrjXdu3c3ksxrr71mK+P27duNJPPoo4/afV6PPvqokWRLu7ZckkxYWJhp0KCBWbFihVm5cqWpWrWq8fPzM8OGDTOPPPKIWbNmjVm8eLEJCgoyNWrUsLsaPHv2bBMbG2tWr15ttmzZYt59911z7733mmrVqplLly7Z8v3888/Gy8vLdOrUyRhz9Vhp2rSpCQwMzLXuxhgzceJEI8l0797dbN68Ode7N++8846xWCymcePGZsmSJWbjxo1m1qxZZuDAgbY8eT1vZB4zYWFhZtSoUSYuLs6sWrXKGGNM//79jYuLixkxYoRZt26dWbJkialevboJCgoyCQkJxhhjzp8/b/z9/U2dOnXM+++/b7Zs2WKWL19unnnmGfPDDz/kWueXX37ZODk5mXPnztmlV61a1QQFBeW6bpcuXYwkc/LkSWPM/85rZcuWNU8//bT57LPPzIcffmjS09OzvVo+ZswYI8k8/fTTZt26debtt982FSpUMCEhIXb9xeHDh40kM3/+/CxtVq1aNTNu3DgTFxdnpk6daqxWa5ZjNa/HdHZ36o0xtvPcd999l2t7/N3Q/9P/0//T/9P/3379f2G3xQcffGDGjRtnVq5cabZs2WKWLVtmoqKiTJkyZcwff/xhy5fdnfr4+Hjj6upqGjZsaJYvX27WrVtn+vbtm+V/gv379xsPDw9z9913m6VLl5qPP/7YtGzZ0lSoUCHbO/VfffWVkWRWr16daxtd728T1Gd+GPv27TPGGHP//febvn37GmNMlk59zpw5RpJ5//337bb32muvGUlmw4YNtjRJxsfHx5w5c8Yub8uWLU25cuWyPCY3ePBg4+bmliV/biZPnmwkmdGjR2dZlpGRYUJDQ01kZKTdo0rnzp0zgYGBpkGDBra0evXqmcDAQLsDJD093URERJhy5crZTlKZ7TZkyBC7fXXo0MFIMlOnTrVLr1mzprnvvvts80uXLjWSzEcffWSXb+fOnUaSmTVrljHGmB9//NFIMsOGDbPLl3myyWunfqMpt079ww8/NJLMt99+m+M+cnv8Lq9t+thjjxlPT0+7E0RGRoa5++67s+3UnZyczIEDB3Kte0ZGhrl8+bJZuHChcXJysvtORUVFGUlm165dtrTExETj5ORk3N3d7Trwb7/91kgy//73v3Pd3+jRo40k89VXX9mlP/vss8ZisdiVNywszLRt2zbX7WWaO3euKV26tO3zCgkJMb179zb//e9/7fJ169bNWK1Wc/ToUbv01q1bGw8PD/PXX3/Z0pTNY5m5PX4nyQQHB9s9yr1q1SojydSsWdOuA58+fXquQdSVK1fM5cuXzW+//WYkmY8//thueeY/m9OnTzfjxo0zpUqVsjun5CQ1NdV2DEoyTk5OplatWmbs2LHm9OnTtnznzp0z3t7e5qGHHsrxMbT8nDcyj5lx48bZbSPzn6dr/6E3xphjx44Zd3d388ILLxhjjNm1a5eRZPtHID9at25tqlevniXdzc3N1KtXL9d1MwOOzO9r5nmtd+/eWfJeH9SfOXPGWK1W07VrV7t8mXXOa1A/efJku/UHDhxo3Nzccv1ccjqmcwrqDx48aCSZ2bNn59Ycfzv0//T/9P/0/5no///H0fv/wmyL7KSnp5vz588bT09P8+abb9rSswvqq1evbmrVqmUuX75st4127dqZkJAQW/t27drVuLu72y52ZO6nevXq2Qb1ly5dMhaLxYwaNSrP5Tbmb/L4vSRFRUWpcuXKmjdvnr7//nvt3Lkzx0fv4uPj5enpqUcffdQuPfORq88//9wuvWnTpvL19bXNp6am6vPPP1fHjh3l4eGh9PR029SmTRulpqZqx44deSr3xo0bNWbMGDVv3tzuUdJMBw4c0IkTJ9SrVy+7R5VKly6tzp07a8eOHbpw4YJSUlL01Vdf6dFHH7V7ZMXJyUm9evXS8ePHdeDAAbttt2vXzm7+rrvuknR1UKnr0699TGXNmjW644471L59e7u616xZU8HBwbaRIzdt2iRJevzxx+2216VLFzk7530Mxy5dumjnzp1Zptdee+2G69asWVOurq56+umn9e6772Z5bCg3+WnTLVu2qGnTpgoICLDlK1WqlLp06ZLttmvUqKGqVatmSd+zZ48efvhh+fv7y8nJSS4uLurdu7cyMjL0888/2+UNCQlR7dq1bfN+fn4KDAxUzZo1FRoaakvP/Fxv9KhRfHy87r77bttgNpn69u0rY4zi4+NzXT8nTz75pI4fP64lS5boH//4h8qXL69FixYpKipKU6ZMsdt/dHS0ypcvn2X/Fy5cyPZx0/xo0qSJ3aPcme3SunVru8e5smuv06dP65lnnlH58uXl7OwsFxcXhYWFSbo6Suu1unTpomeffVbPP/+8Jk6cqBdffFHNmze/YfmsVqtWrlypH374QdOmTVO3bt30xx9/6JVXXtFdd91l+65t27ZNycnJGjhwYI4jPef1vHGtzp07282vWbNGFotFPXv2tDvOg4ODde+999qO8zvvvFO+vr4aNWqU5syZox9++OGGdc104sQJBQYG5jn/tcz/fxz9+ja4vh7Z2bFjh9LS0rIcn/Xq1bMbgf5GHn74Ybv5GjVqKDU1VadPn7al5eeYzk5m+/z+++95LtffDf0//X926P/p/zPR/ztO/1+YbSFJ58+f16hRo3TnnXfK2dlZzs7OKl26tFJSUrJ8ftc6dOiQfvrpJ9s57Ppz/cmTJ21l2bRpk6KjoxUUFGRb38nJSV27ds122y4uLrrjjjvy3a//LUa/l67+Y/fEE0/o3//+t1JTU1W1alU1bNgw27yJiYkKDg7O8iUIDAyUs7Oz7T2gTNe/D5SYmKj09HTNmDFDM2bMyHYf175rk5MjR46oW7duKleunJYuXZrl/aLMfWVXBkkKDQ3VlStXdPbsWZmrT2XkmO/abWXy8/Ozm3d1dc0xPTU11TZ/6tQp/fXXX7b818use+b+goOD7ZY7OzvnayTnMmXKqE6dOlnS8/LzVJUrV9bGjRs1efJkDRo0SCkpKapUqZL+8Y9/6Lnnnst13cx2zUubJiYm2h3MmbJLk7L/PI8ePaqGDRuqWrVqevPNNxUeHi43Nzd9/fXXGjRokC5evGiX//rPSbr6WeX0uV77GWYnMTEx24Amp+9Pfvj4+Kh79+6291/379+vZs2aaezYserfv7/uuOMOJSYm5uv7m1/5+b5L/2uvK1euqEWLFjpx4oReeuklRUZGytPTU1euXFG9evWyfC7S1X9kZs+eLVdXV/3jH//IVznvuusu2z8WxhhNnz5dw4cP10svvaT333/f9t5quXLlctxGXs8bHh4etvTr8546dUrGmBy/w5UqVZJ09bPdsmWLXnnlFb344os6e/asQkJC1L9/f/3zn/+Ui4tLjuW8ePFittuvUKGCDh8+nON60v+O/+v/CczLCPeZ7ZOfYzY715/HrFarJNm+E/k9prPj5uZmt01kRf9P/58d+n/6/0z0/1c5Qv+fqTDaQpJ69Oihzz//XC+99JLuv/9+eXt7y2KxqE2bNrn2q6dOnZIkjRw5UiNHjsw2z7Xnu+vPdVLW89+13Nzc8t2v/22CeunqFb1x48Zpzpw52V71zuTv76+vvvpKxhi7jv306dNKT0+3u9oqZb0T5Ovra7taO2jQoGz3UbFixVzLevHiRXXq1EkpKSnasGFDjp1cZvrJkyezLDtx4oRKlSolX19fGWNUqlSpHPNJylKvggoICJC/v3+Og69k/oRQZtkTEhJUtmxZ2/L09PSbPkHnR8OGDdWwYUNlZGRo165dmjFjhoYOHaqgoCB169Ytx/V8fX3z3Kb+/v62E8C1EhISst12dlcVV61apZSUFK1YscJ2FViSvv3221zrV1j8/f1vyfdHku655x5169ZN06dP188//6wHHnjglu4/P/bt26e9e/dqwYIF6tOnjy39+sF0MqWkpKhXr16qWrWqTp06paeeekoff/xxgfZtsVg0bNgwvfzyy9q3b58k2QZ2ym0QmryeN67f17UCAgJksVj0xRdf2ILVa12bFhkZqWXLlskYo++++04LFizQyy+/LHd3d40ePTrHcgYEBOjMmTNZ0ps3b6633npLO3bsUL169bIsv3DhguLi4hQREZGl08zL71Rntk9Ox2x+7tbnpjCO6cz2Ka7vv6Og/6f/zw79f97Q/2eP/v/W9//ZKWhbJCUlac2aNRo/frxdWdLS0m6478zv3JgxY9SpU6ds82T+JJ2/v3+2x3tO5wDp6oXD/H6v/zaP30tS2bJl9fzzz6t9+/Z2B9/1oqOjdf78ea1atcouPfN3EqOjo3Pdj4eHh5o0aaI9e/aoRo0aqlOnTpbpRlei+/fvrz179mjOnDm67777csxXrVo1lS1bVkuWLLEb/TglJUUfffSRbWRLT09P1a1bVytWrLC78nPlyhUtWrRI5cqVy/Zxr4Jo166dEhMTlZGRkW3dM7/kmSNIL1682G79999/X+np6YVSlvxwcnJS3bp1baOkfvPNN5Ky3l3LlJ82jYqKUnx8vN0dmitXruiDDz7Ic/kyT6rXniyNMXr77bfzU80Ci46O1g8//GBrl0wLFy6UxWJRkyZN8r3NxMREXbp0KdtlP/30k6T/XYmPjo5WfHy8rRO/dv8eHh7ZBnfXyulzvFnZfS6S9J///Cfb/M8884yOHj2qFStWaO7cuVq9erWmTZt2w/1k1wFLVzvh5ORkWzs1aNBAPj4+mjNnTo4jouf1vJGbdu3ayRij33//PdvjPDIyMss6FotF9957r6ZNm6Y77rgjy3fpetWrV8/2kdhhw4bJ3d1dQ4YMUUpKSpblI0eO1NmzZ/XPf/4z1+3npG7durJarVq+fLld+o4dO/I8Im5eFMYxndk+d999d6GV63ZE/0//nxv6/9zR/2eP/v/W9/+F2RYWi0XGmCyf3zvvvJPrrxFIV9uxSpUq2rt3b7ZtUKdOHdtFzCZNmujzzz+3u7iXkZGR5X+Ma+uSmpqa7379b3WnXpJeffXVG+bp3bu33nrrLfXp00dHjhxRZGSktm7dqkmTJqlNmzZq1qzZDbfx5ptv6qGHHlLDhg317LPPKjw8XOfOndOhQ4f0ySef5Pr+0ZtvvqnFixeradOmqlatWo7v39WqVUtWq1WTJ0/W448/rnbt2mnAgAFKS0vTlClT9Ndff9nVNzY2Vs2bN1eTJk00cuRIubq6atasWdq3b5+WLl2apztYedGtWzctXrxYbdq00XPPPacHHnhALi4uOn78uDZt2qRHHnlEHTt21F133aWePXtq+vTpcnFxUbNmzbRv3z69/vrrWX6yp6jMmTNH8fHxatu2rSpUqKDU1FTNmzdPkmyfs5eXl8LCwvTxxx8rOjpafn5+CggIUHh4eJ7bdOzYsfrkk08UHR2tsWPHyt3dXXPmzLEFJNk9Wnm95s2by9XVVd27d9cLL7yg1NRUzZ49W2fPni2i1rE3bNgwLVy4UG3bttXLL7+ssLAwrV27VrNmzdKzzz5boH8KN23apOeee06PP/64GjRoIH9/f50+fVpLly7VunXr1Lt3b9ujU+PHj9eaNWvUpEkTjRs3Tn5+flq8eLHWrl2ryZMny8fHJ9d9ZXYyr732mlq3bi0nJyfVqFEjx8dE86p69eqqXLmyRo8eLWOM/Pz89MknnyguLi5L3nfeeUeLFi3S/Pnzdc899+iee+7R4MGDNWrUKD344INZ3le81tNPP62//vpLnTt3VkREhJycnPTTTz9p2rRpKlWqlEaNGiXp6ntxb7zxhp566ik1a9ZM/fv3V1BQkA4dOqS9e/dq5syZKlWqVJ7PGzl58MEH9fTTT+uJJ57Qrl271KhRI3l6eurkyZPaunWrIiMj9eyzz2rNmjWaNWuWOnTooEqVKskYoxUrVuivv/664buEjRs31rx58/Tzzz/bfb8qV66s9957T48//rjuv/9+DR8+XNWqVdOpU6c0b948ffbZZxo5cmSO76vdiJ+fn4YPH67Y2Fj5+vqqY8eOOn78uCZMmKCQkJA8Ha95URjH9I4dO+Tk5KRGjRoVSpluZ/T/9P/Xov/PO/r/7NH/3/r+vzDbwtvbW40aNdKUKVNsx/WWLVs0d+5c3XHHHTdsh//85z9q3bq1WrZsqb59+6ps2bI6c+aMfvzxR33zzTe2C3f//Oc/tXr1ajVt2lTjxo2Th4eH3nrrrWxvSkiynffzfbEsX8PqOZhrR7/NzfWj3xpzdaTQZ555xoSEhBhnZ2cTFhZmxowZk+WnOZTNKJuZDh8+bJ588klTtmxZ4+LiYsqUKWMaNGhgJk6cmGt5MkcuvdF07WiJq1atMnXr1jVubm7G09PTREdHmy+//DLLtr/44gvTtGlT4+npadzd3U29evXMJ598Ypcnp3bLHAXz2hFcjbk6KrOnp6dd2uXLl83rr79u7r33XuPm5mZKly5tqlevbgYMGGAOHjxoy5eWlmZGjBhhAgMDbSNab9++3YSFheV59Nuc2v+DDz644ei327dvNx07djRhYWHGarUaf39/ExUVleVnJDZu3Ghq1aplrFZrlpF589Kmmfnq1q1rrFarCQ4ONs8//7xtROVrR27NbeTYTz75xNamZcuWNc8//7z57LPPstQzKirK3HPPPVnWz2nbubXjtX777TfTo0cP4+/vb1xcXEy1atXMlClT7EZQvVEdrnXs2DHzz3/+0zz44IMmODjYODs7Gy8vL1O3bl0zY8YMk56ebpf/+++/N+3btzc+Pj7G1dXV3HvvvXajjudWn7S0NPPUU0+ZMmXKGIvFYncMZZc/c1TzKVOm2KVnjoD6wQcf2NJ++OEH07x5c+Pl5WV8fX3NY489Zo4ePWo3avJ3331n3N3ds3yvU1NTTe3atU14eLg5e/Zsjm21fv168+STT5q7777b+Pj4GGdnZxMSEmI6depktm/fniX/p59+aqKiooynp6ftp1Qyf/onU17OGzkd95nmzZtn6tata/v+V65c2fTu3ds28vJPP/1kunfvbipXrmzc3d2Nj4+PeeCBB8yCBQtyrGumpKQkU7p06SyjyGfav3+/6dOnjylXrpxxcXExfn5+plWrVmbt2rVZ8ubWH2T3k3ZXrlwxEydONOXKlTOurq6mRo0aZs2aNebee+81HTt2tOXLbfT769ssu/3k9ZjOafT7hg0bmvbt22fbPn9n9P/0//T/9uj/6f+v5aj9f2G3xfHjx03nzp2Nr6+v8fLyMq1atTL79u3Lch7KbvR7Y67+ZGWXLl1MYGCgcXFxMcHBwaZp06Zmzpw5dvm+/PJL209IZp4D/u///i/b0e979eplIiMjb9hG17MYk8MzCQCKXIsWLXTkyJE8jXIN/B0NGTJEn3/+ufbv319odxML6vDhw6pevbrGjx+vF198sVjLIkm//PKLqlSpovXr1+dpBGUAJQf9P5C7ktT/3yqZrxBMmzZN/fv3z9e6BPXALTJ8+HDVqlVL5cuX15kzZ7R48WLbe1U5/bwS8Hd36tQpVa1aVXPnzs3yM2NFae/evVq6dKkaNGggb29vHThwQJMnT1ZycrL27duXr1Hwi8oTTzyh48ePZ/uoJ4CSg/4fyL/i6v+L04QJE7R8+XJ99913+fp5T+lv+E49UFwyMjI0btw4JSQkyGKx6O6779Z7772nnj17FnfRgBIrKChIixcvvmXvjmby9PTUrl27NHfuXP3111/y8fFR48aN9corr5SIgD49PV2VK1fWmDFjirsoAG6A/h/Iv+Lq/4uTt7e3FixYkO+AXuJOPQAAAAAADutv9ZN2AAAAAADcTgjqAQAAAABwUAT1AAAAAAA4KAbKk3TlyhWdOHFCXl5ef5ufTAAAlGzGGJ07d06hoaEqVYpr8DeLvh4AUNIUVl9PUC/pxIkTKl++fHEXAwCALI4dO6Zy5coVdzEcHn09AKCkutm+nqBekpeXl6Srjent7V3MpQEAQEpOTlb58uVtfRRuDn09AKCkKay+nqBesj2G5+3tTUcPAChReFS8cNDXAwBKqpvt63lJDwAAAAAAB1WsQf3s2bNVo0YN21Xz+vXr67PPPrMtN8YoJiZGoaGhcnd3V+PGjbV//367baSlpWnIkCEKCAiQp6enHn74YR0/fvxWVwUAAAAAgFuuWIP6cuXK6dVXX9WuXbu0a9cuNW3aVI888ogtcJ88ebKmTp2qmTNnaufOnQoODlbz5s117tw52zaGDh2qlStXatmyZdq6davOnz+vdu3aKSMjo7iqBQAA/r/Y2Fjdf//98vLyUmBgoDp06KADBw7Y5enbt68sFovdVK9ePbs8XMQHACB7FmOMKe5CXMvPz09TpkzRk08+qdDQUA0dOlSjRo2SdLVDDwoK0muvvaYBAwYoKSlJZcqU0XvvvaeuXbtK+t/otp9++qlatmyZp30mJyfLx8dHSUlJvGcHACgRbpe+qVWrVurWrZvuv/9+paena+zYsfr+++/1ww8/yNPTU9LVoP7UqVOaP3++bT1XV1f5+fnZ5p999ll98sknWrBggfz9/TVixAidOXNGu3fvlpOT0w3Lcbu0JwDg9lFYfVOJGSgvIyNDH3zwgVJSUlS/fn0dPnxYCQkJatGihS2P1WpVVFSUtm3bpgEDBmj37t26fPmyXZ7Q0FBFRERo27ZtOQb1aWlpSktLs80nJycXXcUAAPgbW7dund38/PnzFRgYqN27d6tRo0a2dKvVquDg4Gy3kZSUpLlz5+q9995Ts2bNJEmLFi1S+fLltXHjxjxfxAcA4HZU7APlff/99ypdurSsVqueeeYZrVy5UnfffbcSEhIkSUFBQXb5g4KCbMsSEhLk6uoqX1/fHPNkJzY2Vj4+PraJ360FAODWSEpKkiS7u/CStHnzZgUGBqpq1arq37+/Tp8+bVt2o4v42UlLS1NycrLdBADA7ajYg/pq1arp22+/1Y4dO/Tss8+qT58++uGHH2zLrx/e3xhzwyH/b5RnzJgxSkpKsk3Hjh27uUoAAIAbMsZo+PDheuihhxQREWFLb926tRYvXqz4+Hi98cYb2rlzp5o2bWp7qq4gF/G5gA8A+Lso9sfvXV1ddeedd0qS6tSpo507d+rNN9+0vUefkJCgkJAQW/7Tp0/b7t4HBwfr0qVLOnv2rF1Hf/r0aTVo0CDHfVqtVlmt1qKoDgAAyMHgwYP13XffaevWrXbpmePiSFJERITq1KmjsLAwrV27Vp06dcpxe7ldxB8zZoyGDx9um09OTiawBwDclor9Tv31jDFKS0tTxYoVFRwcrLi4ONuyS5cuacuWLbaAvXbt2nJxcbHLc/LkSe3bty/XoB4AANxaQ4YM0erVq7Vp0yaVK1cu17whISEKCwvTwYMHJdlfxL/WtRf6r2e1Wm0/mZs5AQBwOyrWO/UvvviiWrdurfLly+vcuXNatmyZNm/erHXr1slisWjo0KGaNGmSqlSpoipVqmjSpEny8PBQjx49JEk+Pj7q16+fRowYIX9/f/n5+WnkyJGKjIy0DaQDAACKjzFGQ4YM0cqVK7V582ZVrFjxhuskJibq2LFjtif1rr2I36VLF0n/u4g/efLkIi0/AAAlXbEG9adOnVKvXr108uRJ+fj4qEaNGlq3bp2aN28uSXrhhRd08eJFDRw4UGfPnlXdunW1YcMGeXl52bYxbdo0OTs7q0uXLrp48aKio6O1YMGCPP28DQAAKFqDBg3SkiVL9PHHH8vLy8v2DryPj4/c3d11/vx5xcTEqHPnzgoJCdGRI0f04osvKiAgQB07drTl5SI+AADZK3G/U18c+O1aAEBJc7v0TTm98z5//nz17dtXFy9eVIcOHbRnzx799ddfCgkJUZMmTfSvf/3L7h341NRUPf/881qyZIntIv6sWbPy/J787dKeAIDbR2H1TQT1oqMHAJQ89E2Fi/YEAJQ0hdU3lbiB8gAAAAAAQN4Q1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADsq5uAtwOwofvdZu/sirbYupJAAA4HbH/x0A8PfGnXoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADoqgHgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADoqgHgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOKhiDepjY2N1//33y8vLS4GBgerQoYMOHDhgl6dv376yWCx2U7169ezypKWlaciQIQoICJCnp6cefvhhHT9+/FZWBQAAAACAW65Yg/otW7Zo0KBB2rFjh+Li4pSenq4WLVooJSXFLl+rVq108uRJ2/Tpp5/aLR86dKhWrlypZcuWaevWrTp//rzatWunjIyMW1kdAAAAAABuKefi3Pm6devs5ufPn6/AwEDt3r1bjRo1sqVbrVYFBwdnu42kpCTNnTtX7733npo1ayZJWrRokcqXL6+NGzeqZcuWRVcBAAAAAACKUYl6pz4pKUmS5OfnZ5e+efNmBQYGqmrVqurfv79Onz5tW7Z7925dvnxZLVq0sKWFhoYqIiJC27ZtuzUFBwAAAACgGBTrnfprGWM0fPhwPfTQQ4qIiLClt27dWo899pjCwsJ0+PBhvfTSS2ratKl2794tq9WqhIQEubq6ytfX1257QUFBSkhIyHZfaWlpSktLs80nJycXTaUAAAAAAChCJSaoHzx4sL777jtt3brVLr1r1662vyMiIlSnTh2FhYVp7dq16tSpU47bM8bIYrFkuyw2NlYTJkwonIIDAAAAAFBMSsTj90OGDNHq1au1adMmlStXLte8ISEhCgsL08GDByVJwcHBunTpks6ePWuX7/Tp0woKCsp2G2PGjFFSUpJtOnbsWOFUBAAAAACAW6hYg3pjjAYPHqwVK1YoPj5eFStWvOE6iYmJOnbsmEJCQiRJtWvXlouLi+Li4mx5Tp48qX379qlBgwbZbsNqtcrb29tuAgAAAADA0RTr4/eDBg3SkiVL9PHHH8vLy8v2DryPj4/c3d11/vx5xcTEqHPnzgoJCdGRI0f04osvKiAgQB07drTl7devn0aMGCF/f3/5+flp5MiRioyMtI2GDwAAAADA7ahYg/rZs2dLkho3bmyXPn/+fPXt21dOTk76/vvvtXDhQv31118KCQlRkyZNtHz5cnl5ednyT5s2Tc7OzurSpYsuXryo6OhoLViwQE5OTreyOgAAAAAA3FLFGtQbY3Jd7u7urvXr199wO25ubpoxY4ZmzJhRWEUDAAAAAKDEKxED5QEAAAAAgPwjqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAUGRiY2N1//33y8vLS4GBgerQoYMOHDhgl8cYo5iYGIWGhsrd3V2NGzfW/v377fKkpaVpyJAhCggIkKenpx5++GEdP378VlYFAIASiaAeAAAUmS1btmjQoEHasWOH4uLilJ6erhYtWiglJcWWZ/LkyZo6dapmzpypnTt3Kjg4WM2bN9e5c+dseYYOHaqVK1dq2bJl2rp1q86fP6927dopIyOjOKoFAECJUay/Uw8AAG5v69ats5ufP3++AgMDtXv3bjVq1EjGGE2fPl1jx45Vp06dJEnvvvuugoKCtGTJEg0YMEBJSUmaO3eu3nvvPTVr1kyStGjRIpUvX14bN25Uy5Ytb3m9AAAoKbhTDwAAbpmkpCRJkp+fnyTp8OHDSkhIUIsWLWx5rFaroqKitG3bNknS7t27dfnyZbs8oaGhioiIsOUBAODvijv1AADgljDGaPjw4XrooYcUEREhSUpISJAkBQUF2eUNCgrSb7/9Zsvj6uoqX1/fLHky179eWlqa0tLSbPPJycmFVg8AAEoS7tQDAIBbYvDgwfruu++0dOnSLMssFovdvDEmS9r1cssTGxsrHx8f21S+fPmCFxwAgBKMoB4AABS5IUOGaPXq1dq0aZPKlStnSw8ODpakLHfcT58+bbt7HxwcrEuXLuns2bM55rnemDFjlJSUZJuOHTtWmNUBAKDEIKgHAABFxhijwYMHa8WKFYqPj1fFihXtllesWFHBwcGKi4uzpV26dElbtmxRgwYNJEm1a9eWi4uLXZ6TJ09q3759tjzXs1qt8vb2tpsAALgd8U49AAAoMoMGDdKSJUv08ccfy8vLy3ZH3sfHR+7u7rJYLBo6dKgmTZqkKlWqqEqVKpo0aZI8PDzUo0cPW95+/fppxIgR8vf3l5+fn0aOHKnIyEjbaPj4n/DRa+3mj7zatphKAgC4FQjqAQBAkZk9e7YkqXHjxnbp8+fPV9++fSVJL7zwgi5evKiBAwfq7Nmzqlu3rjZs2CAvLy9b/mnTpsnZ2VldunTRxYsXFR0drQULFsjJyelWVQUAgBKJoB4AABQZY8wN81gsFsXExCgmJibHPG5ubpoxY4ZmzJhRiKUDAMDx8U49AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADoqgHgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADoqgHgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOqliD+tjYWN1///3y8vJSYGCgOnTooAMHDtjlMcYoJiZGoaGhcnd3V+PGjbV//367PGlpaRoyZIgCAgLk6emphx9+WMePH7+VVQEAAAAA4JYr1qB+y5YtGjRokHbs2KG4uDilp6erRYsWSklJseWZPHmypk6dqpkzZ2rnzp0KDg5W8+bNde7cOVueoUOHauXKlVq2bJm2bt2q8+fPq127dsrIyCiOagEAAAAAcEs4F+fO161bZzc/f/58BQYGavfu3WrUqJGMMZo+fbrGjh2rTp06SZLeffddBQUFacmSJRowYICSkpI0d+5cvffee2rWrJkkadGiRSpfvrw2btyoli1b3vJ6AQAAAABwKxRrUH+9pKQkSZKfn58k6fDhw0pISFCLFi1seaxWq6KiorRt2zYNGDBAu3fv1uXLl+3yhIaGKiIiQtu2bcs2qE9LS1NaWpptPjk5uaiqBAAAUKjCR68t7iIAAEqQEjNQnjFGw4cP10MPPaSIiAhJUkJCgiQpKCjILm9QUJBtWUJCglxdXeXr65tjnuvFxsbKx8fHNpUvX76wqwMAAAAAQJErMUH94MGD9d1332np0qVZllksFrt5Y0yWtOvllmfMmDFKSkqyTceOHSt4wQEAAAAAKCYlIqgfMmSIVq9erU2bNqlcuXK29ODgYEnKcsf99OnTtrv3wcHBunTpks6ePZtjnutZrVZ5e3vbTQAAAAAAOJpiDeqNMRo8eLBWrFih+Ph4VaxY0W55xYoVFRwcrLi4OFvapUuXtGXLFjVo0ECSVLt2bbm4uNjlOXnypPbt22fLAwAAAADA7ahYB8obNGiQlixZoo8//lheXl62O/I+Pj5yd3eXxWLR0KFDNWnSJFWpUkVVqlTRpEmT5OHhoR49etjy9uvXTyNGjJC/v7/8/Pw0cuRIRUZG2kbDBwAAAADgdlSsQf3s2bMlSY0bN7ZLnz9/vvr27StJeuGFF3Tx4kUNHDhQZ8+eVd26dbVhwwZ5eXnZ8k+bNk3Ozs7q0qWLLl68qOjoaC1YsEBOTk63qioAAAAAANxyxRrUG2NumMdisSgmJkYxMTE55nFzc9OMGTM0Y8aMQiwdAAAAAAAlW4kYKA8AAAAAAOQfQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADoqgHgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAABF5r///a/at2+v0NBQWSwWrVq1ym553759ZbFY7KZ69erZ5UlLS9OQIUMUEBAgT09PPfzwwzp+/PgtrAUAACUXQT0AACgyKSkpuvfeezVz5swc87Rq1UonT560TZ9++qnd8qFDh2rlypVatmyZtm7dqvPnz6tdu3bKyMgo6uIDAFDiORd3AQAAwO2rdevWat26da55rFargoODs12WlJSkuXPn6r333lOzZs0kSYsWLVL58uW1ceNGtWzZstDLfLsJH73Wbv7Iq22LqSQAgKLAnXoAAFCsNm/erMDAQFWtWlX9+/fX6dOnbct2796ty5cvq0WLFra00NBQRUREaNu2bTluMy0tTcnJyXYTAAC3I4J6AABQbFq3bq3FixcrPj5eb7zxhnbu3KmmTZsqLS1NkpSQkCBXV1f5+vrarRcUFKSEhIQctxsbGysfHx/bVL58+SKtBwAAxYXH7wEAQLHp2rWr7e+IiAjVqVNHYWFhWrt2rTp16pTjesYYWSyWHJePGTNGw4cPt80nJycT2AMAbkvcqQcAACVGSEiIwsLCdPDgQUlScHCwLl26pLNnz9rlO336tIKCgnLcjtVqlbe3t90EAMDtiKAeAACUGImJiTp27JhCQkIkSbVr15aLi4vi4uJseU6ePKl9+/apQYMGxVVMAABKDB6/BwAAReb8+fM6dOiQbf7w4cP69ttv5efnJz8/P8XExKhz584KCQnRkSNH9OKLLyogIEAdO3aUJPn4+Khfv34aMWKE/P395efnp5EjRyoyMtI2Gj4AAH9nBPUAAKDI7Nq1S02aNLHNZ77n3qdPH82ePVvff/+9Fi5cqL/++kshISFq0qSJli9fLi8vL9s606ZNk7Ozs7p06aKLFy8qOjpaCxYskJOT0y2vDwAAJQ1BPQAAKDKNGzeWMSbH5evXr7/hNtzc3DRjxgzNmDGjMIsGAMBtgXfqAQAAAABwUAUK6g8fPlzY5QAAAAAAAPlUoKD+zjvvVJMmTbRo0SKlpqYWdpkAAAAAAEAeFCio37t3r2rVqqURI0YoODhYAwYM0Ndff13YZQMAAAAAALkoUFAfERGhqVOn6vfff9f8+fOVkJCghx56SPfcc4+mTp2qP/74o7DLCQAAAAAArnNTA+U5OzurY8eOev/99/Xaa6/pl19+0ciRI1WuXDn17t1bJ0+eLKxyAgAAAACA69xUUL9r1y4NHDhQISEhmjp1qkaOHKlffvlF8fHx+v333/XII48UVjkBAAAAAMB1CvQ79VOnTtX8+fN14MABtWnTRgsXLlSbNm1UqtTVawQVK1bUf/7zH1WvXr1QCwsAAAAAAP6nQEH97Nmz9eSTT+qJJ55QcHBwtnkqVKiguXPn3lThAAAAAABAzgoU1B88ePCGeVxdXdWnT5+CbB4AAAAAAORBgd6pnz9/vj744IMs6R988IHefffdmy4UAAAAAAC4sQIF9a+++qoCAgKypAcGBmrSpEk3XSgAAAAAAHBjBQrqf/vtN1WsWDFLelhYmI4ePXrThQIAAAAAADdWoKA+MDBQ3333XZb0vXv3yt/f/6YLBQAAAAAAbqxAQX23bt30j3/8Q5s2bVJGRoYyMjIUHx+v5557Tt26dSvsMgIAAAAAgGwUaPT7iRMn6rffflN0dLScna9u4sqVK+rduzfv1AMAAAAAcIsUKKh3dXXV8uXL9a9//Ut79+6Vu7u7IiMjFRYWVtjlAwAAAAAAOShQUJ+patWqqlq1amGVBQAAAAAA5EOBgvqMjAwtWLBAn3/+uU6fPq0rV67YLY+Pjy+UwgEAAAAAgJwVKKh/7rnntGDBArVt21YRERGyWCyFXS4AAAAAAHADBQrqly1bpvfff19t2rQp7PIAAAAAAIA8KtBP2rm6uurOO+8s7LIAAAAAAIB8KFBQP2LECL355psyxhR2eQAAAAAAQB4V6PH7rVu3atOmTfrss890zz33yMXFxW75ihUrCqVwAAAAAAAgZwUK6u+44w517NixsMsCAAAAAADyoUBB/fz58wu7HAAAAAAAIJ8KFNRLUnp6ujZv3qxffvlFPXr0kJeXl06cOCFvb2+VLl26MMvo8MJHr7WbP/Jq22IqCQAAAADgdlKgoP63335Tq1atdPToUaWlpal58+by8vLS5MmTlZqaqjlz5hR2OQEAAAAAwHUKNPr9c889pzp16ujs2bNyd3e3pXfs2FGff/55oRUOAAAAAADkrMCj33/55ZdydXW1Sw8LC9Pvv/9eKAUDAAAAAAC5K9Cd+itXrigjIyNL+vHjx+Xl5XXThQIAAAAAADdWoKC+efPmmj59um3eYrHo/PnzGj9+vNq0aVNYZQMAAAAAALko0OP306ZNU5MmTXT33XcrNTVVPXr00MGDBxUQEKClS5cWdhkBAAAAAEA2ChTUh4aG6ttvv9XSpUv1zTff6MqVK+rXr58ef/xxu4HzAAAAAABA0SnQ4/eS5O7urieffFIzZ87UrFmz9NRTT+U7oP/vf/+r9u3bKzQ0VBaLRatWrbJb3rdvX1ksFrupXr16dnnS0tI0ZMgQBQQEyNPTUw8//LCOHz9e0GoBAAAAAOAwCnSnfuHChbku7927d562k5KSonvvvVdPPPGEOnfunG2eVq1aaf78+bb560fcHzp0qD755BMtW7ZM/v7+GjFihNq1a6fdu3fLyckpT+UAAAAAAMARFSiof+655+zmL1++rAsXLsjV1VUeHh55Dupbt26t1q1b55rHarUqODg422VJSUmaO3eu3nvvPTVr1kyStGjRIpUvX14bN25Uy5Yt81QOAAAAAAAcUYEevz979qzddP78eR04cEAPPfRQoQ+Ut3nzZgUGBqpq1arq37+/Tp8+bVu2e/duXb58WS1atLClhYaGKiIiQtu2bctxm2lpaUpOTrabAAAAAABwNAV+p/56VapU0auvvprlLv7NaN26tRYvXqz4+Hi98cYb2rlzp5o2baq0tDRJUkJCglxdXeXr62u3XlBQkBISEnLcbmxsrHx8fGxT+fLlC63MAAAAAADcKgV6/D4nTk5OOnHiRKFtr2vXrra/IyIiVKdOHYWFhWnt2rXq1KlTjusZY2SxWHJcPmbMGA0fPtw2n5ycTGAPAAAAAHA4BQrqV69ebTdvjNHJkyc1c+ZMPfjgg4VSsOyEhIQoLCxMBw8elCQFBwfr0qVLOnv2rN3d+tOnT6tBgwY5bsdqtcpqtRZZOQEAAAAAuBUKFNR36NDBbt5isahMmTJq2rSp3njjjcIoV7YSExN17NgxhYSESJJq164tFxcXxcXFqUuXLpKkkydPat++fZo8eXKRlQMAAAAAgJKgQEH9lStXCmXn58+f16FDh2zzhw8f1rfffis/Pz/5+fkpJiZGnTt3VkhIiI4cOaIXX3xRAQEB6tixoyTJx8dH/fr104gRI+Tv7y8/Pz+NHDlSkZGRttHwAQAAAAC4XRXqO/X5tWvXLjVp0sQ2n/mee58+fTR79mx9//33Wrhwof766y+FhISoSZMmWr58uby8vGzrTJs2Tc7OzurSpYsuXryo6OhoLViwgN+oBwAAAADc9goU1F87yNyNTJ06NcdljRs3ljEmx+Xr16+/4fbd3Nw0Y8YMzZgxI89lAgAAAADgdlCgoH7Pnj365ptvlJ6ermrVqkmSfv75Zzk5Oem+++6z5cttBHoAAAAAAHBzChTUt2/fXl5eXnr33Xdto86fPXtWTzzxhBo2bKgRI0YUaiEBAAAAAEBWpQqy0htvvKHY2Fi7n5Hz9fXVxIkTi3T0ewAAAAAA8D8FCuqTk5N16tSpLOmnT5/WuXPnbrpQAAAAAADgxgoU1Hfs2FFPPPGEPvzwQx0/flzHjx/Xhx9+qH79+qlTp06FXUYAAAAAAJCNAr1TP2fOHI0cOVI9e/bU5cuXr27I2Vn9+vXTlClTCrWAAAAAAAAgewUK6j08PDRr1ixNmTJFv/zyi4wxuvPOO+Xp6VnY5QMAAAAAADko0OP3mU6ePKmTJ0+qatWq8vT0zPU35wEAAAAAQOEqUFCfmJio6OhoVa1aVW3atNHJkyclSU899RQ/ZwcAAAAAwC1SoKB+2LBhcnFx0dGjR+Xh4WFL79q1q9atW1dohQMAAAAAADkr0Dv1GzZs0Pr161WuXDm79CpVqui3334rlIIBAAAAAIDcFehOfUpKit0d+kx//vmnrFbrTRcKAAAAAADcWIGC+kaNGmnhwoW2eYvFoitXrmjKlClq0qRJoRUOAAAAAADkrECP30+ZMkWNGzfWrl27dOnSJb3wwgvav3+/zpw5oy+//LKwywgAAAAAALJRoDv1d999t7777js98MADat68uVJSUtSpUyft2bNHlStXLuwyAgAAAACAbOT7Tv3ly5fVokUL/ec//9GECROKokwAAAAAACAP8n2n3sXFRfv27ZPFYimK8gAAAAAAgDwq0OP3vXv31ty5cwu7LAAAAAAAIB8KNFDepUuX9M477yguLk516tSRp6en3fKpU6cWSuEAAAAAAEDO8hXU//rrrwoPD9e+fft03333SZJ+/vlnuzw8lg8AAAAAwK2Rr6C+SpUqOnnypDZt2iRJ6tq1q/79738rKCioSAoHAAAAAABylq936o0xdvOfffaZUlJSCrVAAAAAAAAgbwo0UF6m64N8AAAAAABw6+QrqLdYLFnemecdegAAAAAAike+3qk3xqhv376yWq2SpNTUVD3zzDNZRr9fsWJF4ZUQAAAAAABkK19BfZ8+fezme/bsWaiFAQAAAAAAeZevoH7+/PlFVQ4AAAAAAJBPNzVQHgAAAAAAKD4E9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADoqgHgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQRHUAwCAIvPf//5X7du3V2hoqCwWi1atWmW33BijmJgYhYaGyt3dXY0bN9b+/fvt8qSlpWnIkCEKCAiQp6enHn74YR0/fvwW1gIAgJKLoB4AABSZlJQU3XvvvZo5c2a2yydPnqypU6dq5syZ2rlzp4KDg9W8eXOdO3fOlmfo0KFauXKlli1bpq1bt+r8+fNq166dMjIyblU1AAAosZyLuwAAAOD21bp1a7Vu3TrbZcYYTZ8+XWPHjlWnTp0kSe+++66CgoK0ZMkSDRgwQElJSZo7d67ee+89NWvWTJK0aNEilS9fXhs3blTLli1vWV0AACiJuFMPAACKxeHDh5WQkKAWLVrY0qxWq6KiorRt2zZJ0u7du3X58mW7PKGhoYqIiLDlAQDg74w79QAAoFgkJCRIkoKCguzSg4KC9Ntvv9nyuLq6ytfXN0uezPWzk5aWprS0NNt8cnJyYRUbAIAShTv1AACgWFksFrt5Y0yWtOvdKE9sbKx8fHxsU/ny5QulrAAAlDQE9QAAoFgEBwdLUpY77qdPn7bdvQ8ODtalS5d09uzZHPNkZ8yYMUpKSrJNx44dK+TSAwBQMhDUAwCAYlGxYkUFBwcrLi7Olnbp0iVt2bJFDRo0kCTVrl1bLi4udnlOnjypffv22fJkx2q1ytvb224CAOB2xDv1AACgyJw/f16HDh2yzR8+fFjffvut/Pz8VKFCBQ0dOlSTJk1SlSpVVKVKFU2aNEkeHh7q0aOHJMnHx0f9+vXTiBEj5O/vLz8/P40cOVKRkZG20fABAPg7I6gHAABFZteuXWrSpIltfvjw4ZKkPn36aMGCBXrhhRd08eJFDRw4UGfPnlXdunW1YcMGeXl52daZNm2anJ2d1aVLF128eFHR0dFasGCBnJycbnl9AAAoaSzGGFPchShuycnJ8vHxUVJSUqE8nhc+em2uy4+82vam9wEAuL0Vdt/0d3c7teeN/s+4Ef4PAYCSobD6Jt6pBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAflXNwFAAAAQM7CR68t7iIAAEow7tQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADioYg3q//vf/6p9+/YKDQ2VxWLRqlWr7JYbYxQTE6PQ0FC5u7urcePG2r9/v12etLQ0DRkyRAEBAfL09NTDDz+s48eP38JaAAAAAABQPIo1qE9JSdG9996rmTNnZrt88uTJmjp1qmbOnKmdO3cqODhYzZs317lz52x5hg4dqpUrV2rZsmXaunWrzp8/r3bt2ikjI+NWVQMAAAAAgGJRrKPft27dWq1bt852mTFG06dP19ixY9WpUydJ0rvvvqugoCAtWbJEAwYMUFJSkubOnav33ntPzZo1kyQtWrRI5cuX18aNG9WyZctbVhcAAAAAAG61EvtO/eHDh5WQkKAWLVrY0qxWq6KiorRt2zZJ0u7du3X58mW7PKGhoYqIiLDlAQAAAADgdlVif6c+ISFBkhQUFGSXHhQUpN9++82Wx9XVVb6+vlnyZK6fnbS0NKWlpdnmk5OTC6vYAAAAAADcMiX2Tn0mi8ViN2+MyZJ2vRvliY2NlY+Pj20qX758oZQVAAAAAIBbqcQG9cHBwZKU5Y776dOnbXfvg4ODdenSJZ09ezbHPNkZM2aMkpKSbNOxY8cKufQAAAAAABS9EhvUV6xYUcHBwYqLi7OlXbp0SVu2bFGDBg0kSbVr15aLi4tdnpMnT2rfvn22PNmxWq3y9va2mwAAAAAAcDTF+k79+fPndejQIdv84cOH9e2338rPz08VKlTQ0KFDNWnSJFWpUkVVqlTRpEmT5OHhoR49ekiSfHx81K9fP40YMUL+/v7y8/PTyJEjFRkZaRsNHwAAAACA21WxBvW7du1SkyZNbPPDhw+XJPXp00cLFizQCy+8oIsXL2rgwIE6e/as6tatqw0bNsjLy8u2zrRp0+Ts7KwuXbro4sWLio6O1oIFC+Tk5HTL65NX4aPX2s0febVtMZUEAAAAAODILMYYU9yFKG7Jycny8fFRUlJSoTyKf33QfiME9QCA6xV23/R358jtmd//K26E/zsAoGQorL6pxL5TDwAAAAAAckdQDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAAAAAcFAE9QAAAAAAOCiCegAAAAAAHBRBPQAAAAAADoqgHgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAclHNxFwAAAAC3Tvjotba/j7zathhLAgAoDNypBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEER1AMAAAAA4KAI6gEAQLGKiYmRxWKxm4KDg23LjTGKiYlRaGio3N3d1bhxY+3fv78YSwwAQMlBUA8AAIrdPffco5MnT9qm77//3rZs8uTJmjp1qmbOnKmdO3cqODhYzZs317lz54qxxAAAlAwE9QAAoNg5OzsrODjYNpUpU0bS1bv006dP19ixY9WpUydFRETo3Xff1YULF7RkyZJiLjUAAMWPoB4AABS7gwcPKjQ0VBUrVlS3bt3066+/SpIOHz6shIQEtWjRwpbXarUqKipK27Zty3F7aWlpSk5OtpsAALgdEdQDAIBiVbduXS1cuFDr16/X22+/rYSEBDVo0ECJiYlKSEiQJAUFBdmtExQUZFuWndjYWPn4+Nim8uXLF2kdAAAoLgT1AACgWLVu3VqdO3dWZGSkmjVrprVr10qS3n33XVsei8Vit44xJkvatcaMGaOkpCTbdOzYsaIpPAAAxYygHgAAlCienp6KjIzUwYMHbaPgX39X/vTp01nu3l/LarXK29vbbgIA4HbkXNwFgL3w0Wvt5o+82raYSgIAQPFIS0vTjz/+qIYNG6pixYoKDg5WXFycatWqJUm6dOmStmzZotdee62YSwoAQPEjqAcAAMVq5MiRat++vSpUqKDTp09r4sSJSk5OVp8+fWSxWDR06FBNmjRJVapUUZUqVTRp0iR5eHioR48exV10AACKHUE9AAAoVsePH1f37t31559/qkyZMqpXr5527NihsLAwSdILL7ygixcvauDAgTp79qzq1q2rDRs2yMvLq5hLDgBA8SOoBwAAxWrZsmW5LrdYLIqJiVFMTMytKRAAAA6EgfIAAAAAAHBQ3KkvAa4fHA8AAAAAgLzgTj0AAAAAAA6KoB4AAAAAAAdVooP6mJgYWSwWuyk4ONi23BijmJgYhYaGyt3dXY0bN9b+/fuLscQAAAAAANw6JTqol6R77rlHJ0+etE3ff/+9bdnkyZM1depUzZw5Uzt37lRwcLCaN2+uc+fOFWOJAQAAAAC4NUp8UO/s7Kzg4GDbVKZMGUlX79JPnz5dY8eOVadOnRQREaF3331XFy5c0JIlS4q51AAAAAAAFL0SH9QfPHhQoaGhqlixorp166Zff/1VknT48GElJCSoRYsWtrxWq1VRUVHatm1brttMS0tTcnKy3QQAAAAAgKMp0UF93bp1tXDhQq1fv15vv/22EhIS1KBBAyUmJiohIUGSFBQUZLdOUFCQbVlOYmNj5ePjY5vKly9fZHUAAAAAAKColOigvnXr1urcubMiIyPVrFkzrV179ffc3333XVsei8Vit44xJkva9caMGaOkpCTbdOzYscIvPAAAAAAARaxEB/XX8/T0VGRkpA4ePGgbBf/6u/KnT5/Ocvf+elarVd7e3nYTAAAAAACOxqGC+rS0NP34448KCQlRxYoVFRwcrLi4ONvyS5cuacuWLWrQoEExlhIAAAAAgFvDubgLkJuRI0eqffv2qlChgk6fPq2JEycqOTlZffr0kcVi0dChQzVp0iRVqVJFVapU0aRJk+Th4aEePXoUd9EBAAAAAChyJTqoP378uLp3764///xTZcqUUb169bRjxw6FhYVJkl544QVdvHhRAwcO1NmzZ1W3bl1t2LBBXl5exVxyAAAAAACKXokO6pctW5brcovFopiYGMXExNyaAgEAAAAAUII41Dv1AAAAAADgfwjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOqkSPfg8AAICiEz56rd38kVfbFlNJAAAFxZ16AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBMfo9AABACXL9iPQAAOSGO/UAAAAAADgognoAAAAAABwUj9+XcNc/gnfk1bbFVBIAAAAAQEnDnXoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEEx+r2DYTR8AAAAAEAm7tQDAAAAAOCgCOoBAAAAAHBQBPUAAAAAADgognoAAAAAABwUQT0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIPid+pvc/yuPQAAAADcvrhTDwAAAACAgyKoBwAAAADAQRHUAwAAAADgoAjqAQAAAABwUAT1AAAAAAA4KIJ6AAAAAAAcFEE9AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICDci7uAuDmhI9eazd/5NW2DrFtAAAAAMDN4049AAAAAAAOiqAeAAAAAAAHRVAPAAAAAICD4p36v5lr35PnHXkAAAAAcGwE9beZ6we3AwAAAADcvnj8HgAAAAAAB0VQDwAAAACAgyKoBwAAAADAQfFOPQAAACRlHZuHQXUBoOTjTj0AAAAAAA6KoB4AAAAAAAdFUA8AAAAAgIMiqAcAAAAAwEExUN7fGIPhAAAAAIBj4049AAAAAAAOijv1sLn+zv3N5OeuPwAAAAAUPe7UAwAAAADgoAjqAQAAAABwUDx+jzzLz+P5DMIHAAAAAEWPoB4AAADZ4iI9AJR8PH4PAAAAAICD4k49igVX/gEAAADg5hHU45bI78/l3Uo3c4GBixMAAAAAihOP3wMAAAAA4KC4U48SqTDvgHM3vWjdqH1pfwC4sZL8RBsAoGQjqEeJUJj/zNxoW0X5j1NRPspfnMFxSQnMS0o5AAAAgJLitnn8ftasWapYsaLc3NxUu3ZtffHFF8VdJAAAUIjo6wEAyOq2uFO/fPlyDR06VLNmzdKDDz6o//znP2rdurV++OEHVahQobiLh0JwKx9LvFX7Ks5HLUvSvrnbXrRob9wu6OsBAMjebRHUT506Vf369dNTTz0lSZo+fbrWr1+v2bNnKzY2tphLBxRMfl8jyC1Yux0vihS24nz94UZtVpT7KkmvedwMRy038o6+vmTgWAOAksfhg/pLly5p9+7dGj16tF16ixYttG3btmzXSUtLU1pamm0+KSlJkpScnFwoZbqSdqFQtgPHl9t3qrC/J8W1r5vddn6Ou+v3VVjHbF62fSv3fb2b2Vd+t12U9SxKjlru3GTWwRhTzCUpfiWxry9sjvq/Q0ltTwBwBIXW1xsH9/vvvxtJ5ssvv7RLf+WVV0zVqlWzXWf8+PFGEhMTExMTU4mfjh07diu60xKNvp6JiYmJ6Xaebravd/g79ZksFovdvDEmS1qmMWPGaPjw4bb5K1eu6MyZM/L3989xnbxKTk5W+fLldezYMXl7e9/Utv4uaLP8o83yjzbLP9osfwq7vYwxOnfunEJDQwuhdLcH+nrHRZvlH22Wf7RZ/tFm+VeYbVZYfb3DB/UBAQFycnJSQkKCXfrp06cVFBSU7TpWq1VWq9Uu7Y477ijUcnl7e3Ng5BNtln+0Wf7RZvlHm+VPYbaXj49PoWzH0dHX3z5os/yjzfKPNss/2iz/CqvNCqOvd/iftHN1dVXt2rUVFxdnlx4XF6cGDRoUU6kAAEBhoa8HACBnDn+nXpKGDx+uXr16qU6dOqpfv77+7//+T0ePHtUzzzxT3EUDAACFgL4eAIDs3RZBfdeuXZWYmKiXX35ZJ0+eVEREhD799FOFhYXd8rJYrVaNHz8+yyN/yBltln+0Wf7RZvlHm+UP7VW06OsdG22Wf7RZ/tFm+Ueb5V9JbDOLMfxWDgAAAAAAjsjh36kHAAAAAODviqAeAAAAAAAHRVAPAAAAAICDIqgHAAAAAMBBEdRfZ9asWapYsaLc3NxUu3ZtffHFF7nm37Jli2rXri03NzdVqlRJc+bMyZLno48+0t133y2r1aq7775bK1euvOn9liTF0WaxsbG6//775eXlpcDAQHXo0EEHDhwo1HoVpeL6nmWKjY2VxWLR0KFDb7Yqt0xxtdnvv/+unj17yt/fXx4eHqpZs6Z2795daPUqKsXRXunp6frnP/+pihUryt3dXZUqVdLLL7+sK1euFGrdikpht9n+/fvVuXNnhYeHy2KxaPr06YWyX9w8+vr8o6/PP/r6/KOvzz/6+/y7Lft7A5tly5YZFxcX8/bbb5sffvjBPPfcc8bT09P89ttv2eb/9ddfjYeHh3nuuefMDz/8YN5++23j4uJiPvzwQ1uebdu2GScnJzNp0iTz448/mkmTJhlnZ2ezY8eOAu+3JCmuNmvZsqWZP3++2bdvn/n2229N27ZtTYUKFcz58+eLvM43q7jaLNPXX39twsPDTY0aNcxzzz1XVNUsVMXVZmfOnDFhYWGmb9++5quvvjKHDx82GzduNIcOHSryOt+M4mqviRMnGn9/f7NmzRpz+PBh88EHH5jSpUub6dOnF3mdb1ZRtNnXX39tRo4caZYuXWqCg4PNtGnTbnq/uHn09flHX59/9PX5R1+ff/T3+Xe79vcE9dd44IEHzDPPPGOXVr16dTN69Ohs87/wwgumevXqdmkDBgww9erVs8136dLFtGrVyi5Py5YtTbdu3Qq835KkuNrseqdPnzaSzJYtW/JbhVuuONvs3LlzpkqVKiYuLs5ERUU5TEdfXG02atQo89BDD91s8W+54mqvtm3bmieffNIuT6dOnUzPnj0LVI9bqSja7FphYWHZdvKOfP53VPT1+Udfn3/09flHX59/9Pf5d7v29zx+//9dunRJu3fvVosWLezSW7RooW3btmW7zvbt27Pkb9mypXbt2qXLly/nmidzmwXZb0lRXG2WnaSkJEmSn59fvutxKxV3mw0aNEht27ZVs2bNbrYqt0xxttnq1atVp04dPfbYYwoMDFStWrX09ttvF0a1ikxxttdDDz2kzz//XD///LMkae/evdq6davatGlz0/UqSkXVZkWxX9wc+vr8K+5+61r09fT11/o79/US/X1B3M79PUH9//fnn38qIyNDQUFBdulBQUFKSEjIdp2EhIRs86enp+vPP//MNU/mNguy35KiuNrsesYYDR8+XA899JAiIiIKWp1bojjbbNmyZfrmm28UGxtbGFW5ZYqzzX799VfNnj1bVapU0fr16/XMM8/oH//4hxYuXFgYVSsSxdleo0aNUvfu3VX9/7V3rzFNnX8cwL+AlHIRpOAQJiB44TKRcUtUljHGHFmMksy5OAlWFx0xg4DLhiya6QsnxDfbEheSGoIuaLIhLtFMN7x1qJsSBQYCESEIMplkysSVWFn7+7/YOFvl8rdALSXfT9Kkfc5zznOh4ctzSs+JjISrqyvi4uKQn5+Pd955ZzKGZjO2mjNbtEsTw6y3HrPeesx66zHrrce8t950zvsZk3KUacTJycnitYgMK/t/9Z8sf5pjWtvuVGKvORuSk5ODhoYGXLx40ap+29OznrPbt28jLy8PVVVVUKvVE+q7vdjjfWY2m5GYmIi9e/cCAOLi4tDU1ISSkhJs2LBhfAN5RuwxX19//TXKy8tx5MgRvPDCC6ivr0d+fj6CgoKg1WrHPZZnxRZzZot2aeKY9dZj1luPWW89Zr31mPfWm455z0X9P/z9/eHi4jLsbElvb++wsypD5syZM2L9GTNmwM/Pb8w6Q8ccT7tThb3m7L9yc3Nx/PhxVFdXY+7cuRMZzjNhrzm7du0aent7kZCQoGw3mUyorq7G/v37YTQa4eLiMuHx2YI932eBgYGIjo62qBMVFYXKyspxj8fW7DlfH330EQoLC7Fu3ToAQExMDDo7O1FUVDSlQ95Wc2aLdmlimPXWY9Zbj1lvPWa99Zj31pvOec9/v/+HSqVCQkICTp8+bVF++vRpLF++fMR9li1bNqx+VVUVEhMT4erqOmadoWOOp92pwl5zBvx9ZisnJwfHjh3DuXPnEBYWNhlDsjl7zVlaWhoaGxtRX1+vPBITE5GZmYn6+vopG/KAfd9nycnJw26f1NraitDQ0HGPx9bsOV8DAwNwdraMFRcXlyl/ixtbzZkt2qWJYdZbj1lvPWa99Zj11mPeW29a5/2kXG5vmhi61UBpaak0NzdLfn6+eHp6yq1bt0REpLCwULKyspT6Q7c42LZtmzQ3N0tpaemwWxxcunRJXFxcpLi4WFpaWqS4uHjU29yM1u5UZq8527p1q/j4+Iher5eenh7lMTAw8OwGP072mrMnOdIVce01ZzU1NTJjxgz59NNP5ebNm3L48GHx8PCQ8vLyZzf4cbDXfGm1Wnn++eeVW9wcO3ZM/P39paCg4NkNfpxsMWdGo1Hq6uqkrq5OAgMD5cMPP5S6ujq5efPmU7dLk49Zbz1mvfWY9dZj1luPeW+96Zr3XNQ/4csvv5TQ0FBRqVQSHx9vcdsUrVYrKSkpFvX1er3ExcWJSqWSefPmSUlJybBjVlRUSEREhLi6ukpkZKRUVlZa1e5UZ485AzDio6yszBZDnHT2ep/9lyMFvYj95uzEiROyePFicXNzk8jISNHpdJM+Nluwx3z19/dLXl6ehISEiFqtlvDwcNmxY4cYjUabjHGyTfacdXR0jPh76snjOPLvf0fFrLces956zHrrMeutx7y33nTMeyeRf77pT0REREREREQOhd+pJyIiIiIiInJQXNQTEREREREROSgu6omIiIiIiIgcFBf1RERERERERA6Ki3oiIiIiIiIiB8VFPREREREREZGD4qKeiIiIiIiIyEFxUU80Del0OgQHB8PZ2Rmff/65vbtDREREk4xZT0RDuKgnmkJ6e3uRnZ2NkJAQuLm5Yc6cOUhPT8fPP//81Mfo7+9HTk4Otm/fjl9//RXvvfceXnnlFeTn59uu4xNkMplQVFSEyMhIuLu7Q6PRYOnSpSgrK7N314iIiCYVs55ZTzTZZti7A0T0rzVr1mBwcBCHDh1CeHg47t69i7Nnz+L+/ftPfYyuri4MDg5i5cqVCAwMtGFvJ8/u3buh0+mwf/9+JCYmor+/H1evXkVfX5/N2nz8+DFUKpXNjk9ERDQSZj2znmjSCRFNCX19fQJA9Hr9mPU6Oztl9erV4unpKTNnzpS1a9fKb7/9JiIiZWVlAsDiodVqh5V1dHTI+fPnBYB8//338uKLL4parZbU1FS5e/eunDx5UiIjI2XmzJmybt06MRgMSvunTp2S5ORk8fHxEY1GIytXrpS2tjZl+6FDh8TT01NaW1uVspycHFm4cKH8+eefI44pNjZWdu/ePea4TSaTFBcXy/z580WlUklwcLDs2bNH2d7Q0CCpqamiVqtFo9HIli1b5OHDh8p2rVYrGRkZsnfvXgkMDJTQ0FAREenu7pa3335bZs2aJRqNRlavXi0dHR1j9oWIiGg8mPXMeiJb4KKeaIoYHBwULy8vyc/Pl0ePHo1Yx2w2S1xcnLz00kty9epVuXz5ssTHx0tKSoqIiAwMDMiZM2cEgNTU1EhPT4/88ccfsmzZMtmyZYv09PRIT0+P/PXXX0rQL126VC5evCi1tbWyYMECSUlJkddff11qa2ulurpa/Pz8pLi4WOnD0aNHpbKyUlpbW6Wurk5WrVolMTExYjKZlDpr166VpKQkGRwclFOnTomrq6vU1NSMOvb09HR5+eWXpbe3d9Q6BQUF4uvrKwcPHpS2tja5cOGCHDhwQEREDAaDBAUFyZtvvimNjY1y9uxZCQsLE61Wq+yv1WrFy8tLsrKy5Pr169LY2CgGg0EWLlwo7777rjQ0NEhzc7OsX79eIiIixGg0Ps2PjYiI6Kkx65n1RLbART3RFHL06FHx9fUVtVoty5cvl48//lh++eUXZXtVVZW4uLhIV1eXUtbU1KQEu4hIXV2dcoZ+SEpKiuTl5Vm0NRT0Z86cUcqKiooEgLS3tytl2dnZkp6ePmqfe3t7BYA0NjYqZffv35e5c+fK1q1bJSAgwOIs+0iampokKipKnJ2dJSYmRrKzs+XkyZPK9v7+fnFzc1OC/Uk6nU58fX0tPh347rvvxNnZWflkQ6vVSkBAgEWAl5aWSkREhJjNZqXMaDSKu7u7/PDDD2P2mYiIaDyY9cx6osnGC+URTSFr1qzBnTt3cPz4caSnp0Ov1yM+Ph4HDx4EALS0tCA4OBjBwcHKPtHR0Zg1axZaWlrG1eaSJUuU5wEBAfDw8EB4eLhFWW9vr/K6vb0d69evR3h4OLy9vREWFgbg7+/3DfH19UVpaSlKSkowf/58FBYWjtmH6OhoXL9+HZcvX8amTZtw9+5drFq1Cps3b1bGbTQakZaWNuL+LS0tiI2Nhaenp1KWnJwMs9mMGzduKGUxMTEW3627du0a2traMHPmTHh5ecHLywsajQaPHj1Ce3v7mH0mIiIaD2Y9s55osvFCeURTjFqtxooVK7BixQp88skn2Lx5M3bt2oWNGzdCRODk5DRsn9HKn4arq6vy3MnJyeL1UJnZbFZer1q1CsHBwThw4ACCgoJgNpuxePFiPH782GK/6upquLi44M6dOzAYDPD29h6zH87OzkhKSkJSUhK2bduG8vJyZGVlYceOHXB3dx9z37HG/9/y//4hAABmsxkJCQk4fPjwsP1mz549ZptERETjxaxn1hNNJn5STzTFRUdHw2AwKM+7urpw+/ZtZXtzczMePHiAqKioUY+hUqlgMpkm3Jd79+6hpaUFO3fuRFpaGqKioka8au1PP/2Effv24cSJE/D29kZubq7VbUVHRwMADAYDFi5cCHd3d5w9e3bUuvX19co8AcClS5fg7OyMRYsWjdpGfHw8bt68ieeeew4LFiywePj4+FjdZyIiovFg1jPriSaCi3qiKeLevXt49dVXUV5ejoaGBnR0dKCiogL79u1DRkYGAOC1117DkiVLkJmZidraWtTU1GDDhg1ISUlBYmLiqMeeN28erly5glu3buH333+3OBtvDV9fX/j5+UGn06GtrQ3nzp3DBx98YFHn4cOHyMrKQm5uLt544w0cOXIE33zzDSoqKkY97ltvvYXPPvsMV65cQWdnJ/R6Pd5//30sWrQIkZGRUKvV2L59OwoKCvDVV1+hvb0dly9fRmlpKQAgMzMTarUaWq0W169fx/nz55Gbm4usrCwEBASM2m5mZib8/f2RkZGBCxcuoKOjAz/++CPy8vLQ3d09rjkiIiIaDbOeWU9kE/b8Qj8R/evRo0dSWFgo8fHx4uPjIx4eHhIRESE7d+6UgYEBpd5Yt7kRGfniOTdu3JClS5eKu7v7sNvc9PX1KfXKysrEx8fHol+7du2S2NhY5fXp06clKipK3NzcZMmSJaLX6wWAfPvttyIismnTJomJibG4qu8XX3whGo1Guru7Rxy7TqeT1NRUmT17tqhUKgkJCZGNGzfKrVu3lDomk0n27NkjoaGh4urqKiEhIbJ3715l+9Pe5uZJPT09smHDBvH39xc3NzcJDw+XLVu2yIMHD0bsKxER0Xgx65n1RLbgJCJiv1MKRERERERERDRe/Pd7IiIiIiIiIgfFRT0RERERERGRg+KinoiIiIiIiMhBcVFPRERERERE5KC4qCciIiIiIiJyUFzUExERERERETkoLuqJiIiIiIiIHBQX9UREREREREQOiot6IiIiIiIiIgfFRT0RERERERGRg+KinoiIiIiIiMhBcVFPRERERERE5KD+B+7y3XVU3oMxAAAAAElFTkSuQmCC", | |
"text/plain": [ | |
"<Figure size 1200x600 with 2 Axes>" | |
] | |
}, | |
"metadata": {}, | |
"output_type": "display_data" | |
} | |
], | |
"source": [ | |
"# Main execution\n", | |
"dim = 100 # Dimension of vectors\n", | |
"vec_base = np.random.rand(dim) # Base vector for comparison\n", | |
"num_vectors_large = 1000\n", | |
"large_set_of_vectors = np.random.rand(num_vectors_large, dim)\n", | |
"\n", | |
"# Compute dot products for the larger set\n", | |
"large_dot_products = np.array([np.dot(vec_base, v) for v in large_set_of_vectors])\n", | |
"scaling_factor = np.sqrt(dim)\n", | |
"large_scaled_dot_products = large_dot_products / scaling_factor\n", | |
"\n", | |
"# Plotting more zoomed histograms\n", | |
"plot_more_zoomed_histograms(large_dot_products, large_scaled_dot_products, \"More Zoomed Histograms of Softmax Scores for Large Number of Values\")" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"id": "c7db98ab-a23e-488b-804c-a92287e6f35c", | |
"metadata": {}, | |
"source": [ | |
"As it can be seen, even without the presence of one highly dominant value, there are lots of values hitting to zero directly, which basically destroys them. The scaled embeddings, however, show a gaussian distribution without any of them hitting zero." | |
] | |
} | |
], | |
"metadata": { | |
"kernelspec": { | |
"display_name": "Python 3 (ipykernel)", | |
"language": "python", | |
"name": "python3" | |
}, | |
"language_info": { | |
"codemirror_mode": { | |
"name": "ipython", | |
"version": 3 | |
}, | |
"file_extension": ".py", | |
"mimetype": "text/x-python", | |
"name": "python", | |
"nbconvert_exporter": "python", | |
"pygments_lexer": "ipython3", | |
"version": "3.11.5" | |
} | |
}, | |
"nbformat": 4, | |
"nbformat_minor": 5 | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment