Skip to content

Instantly share code, notes, and snippets.

@joshfp

joshfp/1.tabular.ipynb

Last active October 10, 2020 16:02
Show Gist options
  • Save joshfp/b62b76eae95e6863cb511997b5a63118 to your computer and use it in GitHub Desktop.
Save joshfp/b62b76eae95e6863cb511997b5a63118 to your computer and use it in GitHub Desktop.
Download ZIP
Fast.ai p1v1: class 4
Display the source blob
Display the rendered blob
Raw
1.tabular.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
2.lm-spanish.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
3.nlp.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# NLP model to predict from title (ULMFiT)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"%load_ext autoreload\n",
"%autoreload 2"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from fastai import *\n",
"from fastai.text import *"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"pretrained_path = Path('~/datasets/wikimedia').expanduser()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load data"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"df = pd.read_feather('tabular-df')"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"columns = ['title', 'target']\n",
"df = df[columns]\n",
"N = -10000\n",
"train_df = df[:N]\n",
"valid_df = df[N:]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Languaje model"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [],
"source": [
"data_lm = TextLMDataBunch.from_df('.', train_df, valid_df, tokenizer=Tokenizer(lang='es'), text_cols='title')"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [],
"source": [
"# pretrained model (spanish wiki)\n",
"pretraind_fnames = (pretrained_path/'models/weights-6', pretrained_path/'itos')\n",
"learn = language_model_learner(data_lm, drop_mult=0.3, pretrained_fnames=pretraind_fnames)"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n"
]
}
],
"source": [
"learn.lr_find()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl83FW9//HXZyb73jZputF9b6HQBqEoiFAr4LUsgsIV3FAu3qsIiuL94YLghqLidq/WBe9VRC+bLIqgAqKypqWlC23pQre0TZo0+575/P6YSQghbdM2M9+Z5P18PObRme/3fOf7yXRmPnPO+Z5zzN0REREBCAUdgIiIJA8lBRER6aGkICIiPZQURESkh5KCiIj0UFIQEZEeSgoiItJDSUFERHooKYiISI+0oAM4UsXFxT558uSgwxARSSkrVqzY7+4lhysX16RgZtcBHwEcWAN8yN1b+5R5D3BTrMxqd//XQz3n5MmTKS8vj0/AIiJDlJltH0i5uDUfmdl44BqgzN3nA2Hg0j5lZgD/CbzZ3ecB18YrHhERObx49ymkAdlmlgbkABV99n8U+JG7HwBw98o4xyMiIocQt6Tg7ruB24AdwB6gzt0f61NsJjDTzP5pZs+a2TnxikdERA4vns1HI4DzgSnAOCDXzC7vUywNmAGcCVwG/MzMivp5rqvMrNzMyquqquIVsojIsBfP5qMlwDZ3r3L3DuA+4LQ+ZXYBD7h7h7tvAzYSTRKv4+7L3b3M3ctKSg7beS4iIkcpnklhB3CqmeWYmQFnAy/3KfN74G0AZlZMtDlpaxxjEhGRQ4hnn8JzwD3ASqKXo4aA5WZ2s5ktixV7FKg2s/XAE8Bn3L06XjGJiMihWaotx1lWVuYapyAiw83tf9nEokkjOH3G0TWhm9kKdy87XDlNcyEikuTcnR88vpnnttbE/VxKCiIiSa65vYuuiJOfFf+ZiZQURESSXH1rBwD5WelxP5eSgohIkmto7QSgIFs1BRGRYa9BNQUREelW3xKtKahPQUREevoUClRTEBGRnj4F1RRERKQ7KahPQUREqG/tID1sZKXH/ytbSUFEJMk1tHaQn5VOdG7R+FJSEBFJcg2tnQm58giUFEREkl59S0dCrjwCJQURkaSnmoKIiPRQUhARkR71rWo+EhGRmGhNQUlBRGTY64o4jW1qPhIREaCxNXGT4YGSgohIUuuZDC9bzUciIsNeIifDAyUFEZGklsilOEFJQUQkqb1WU1BSEBEZ9l5binMINB+Z2XVmts7M1prZXWaWdZByF5uZm1lZPOMREUk1DUPl6iMzGw9cA5S5+3wgDFzaT7n8WLnn4hWLiEiqqm8ZWn0KaUC2maUBOUBFP2VuAb4JtMY5FhGRlNPQ1klWeoiMtMS09sftLO6+G7gN2AHsAerc/bHeZczsJOA4d3/4UM9lZleZWbmZlVdVVcUrZBGRpNO9wE6ixLP5aARwPjAFGAfkmtnlvfaHgO8Cnz7cc7n7cncvc/eykpKSeIUsIpJ06ls6EzZGAeLbfLQE2ObuVe7eAdwHnNZrfz4wH3jSzF4FTgUeVGeziMhr6hNcU4hn+tkBnGpmOUALcDZQ3r3T3euA4u7HZvYkcL27lyMiIkBi11KA+PYpPAfcA6wE1sTOtdzMbjazZfE6r4jIUJLItRQgvjUF3P1LwJf6bP7iQcqeGc9YRERSUUNrJwXZQ6CmICIix27IXH0kIiLHpr0zQmtHhPxM1RRERIa9hgSvpQBKCiIiSSvR8x6BkoKISNJK9FoKoKQgIpK0Er3qGigpiIgkrQbVFEREpFu9+hRERKRb91oKuvpIRER6+hTyNE5BREQaWjvJy0wjHLKEnVNJQUQkSUWnzU5cLQGUFEREklZDgmdIBSUFEZGklei1FEBJQUQkaan5SEREekTXUlDzkYiIoOYjERGJcXfqWxK7wA4oKYiIJKXWjgidEdfVRyIi0nsyPDUfiYgMe/VKCiIi0q17hlRdfSQiIoEssANxTgpmdp2ZrTOztWZ2l5ll9dn/KTNbb2YvmdlfzWxSPOMREUkV3dNmD5mrj8xsPHANUObu84EwcGmfYi/G9p8A3AN8M17xiIikkoYAFtiB+DcfpQHZZpYG5AAVvXe6+xPu3hx7+CwwIc7xiIikhO6rj4bMJanuvhu4DdgB7AHq3P2xQxxyJfBIvOIREUklDa2dhENGTkY4oeeNZ/PRCOB8YAowDsg1s8sPUvZyoAz41kH2X2Vm5WZWXlVVFa+QRUSSRn1rB3mZaZglboEdiG/z0RJgm7tXuXsHcB9wWt9CZrYEuBFY5u5t/T2Ruy939zJ3LyspKYljyCIiyaGmqZ2RuRkJP288k8IO4FQzy7FoqjsbeLl3ATM7CfgJ0YRQGcdYRERSypBLCu7+HNErilYCa2LnWm5mN5vZslixbwF5wN1mtsrMHoxXPCIiqSSopBDXa53c/UvAl/ps/mKv/UvieX4RkVS1v7GdkyYWJfy8GtEsIpJkIhHnQPMQaz4SEZGjU9fSQVfEGZWbmfBzKymIiCSZ6qZ2AEblqaYgIjLs1cSSgpqPRESE6sbokC01H4mIiJqPRETkNdWN0aQwIkdJQURk2KtpaqMgK42MtMR/RSd2om6RXhrbOtlc2Ri7/C5CZ5fTFXHMwMwwwIHOLqczEqEr4kwcmcO8cYVkJ3jmSJFEqm5qZ1Re4vsTQElBDqK9M8LaijoKstIoLch6w2yNkYhT19JBdVMbVQ3tNLV1MjIvg9H5mRTnZRJxZ/eBFnbXtlBR28qB5nbqWjqoa+5gX0Mrr+xrZHdty1HFFg4Zs0rzmT++gLGF2YwuyGR0fhalBZmUFmRRnJdJOJTYmSVFBlN1YzujArjyCJQUhqT9jW3UNLVT09RObXN3h1UmI3MzGJmTQXpaiO6vzJAZvWfmfX5bDQ+truDRdXt7Fg4HyMkIk5eZRmtHF22dEdo6I0ccV2ZaiKKcdIrzMimbPIJ/LZ3I9NF5FOdlEA6FSAtZz5d5xB336HHp4RBpYSNkxubKRlbvrGX1rloe31BFdVNbT7luIYOS/GiiGJ2fSUl+JgXZ6bR2dMVuEfKy0pgwIpvxRdmMLczGjJ6aSjhkFGanU5QTvWWnhxM+fbEMbzVN7UwalRPIuZUUkkgk4uyubWHTvgY27WukurGNlo4uWjq66OxyppbkMn9cIfPGFzCmILrcdXtXhNb2CCt21PC3jVU8uamK7dXNhznToeVlprF0XilL5pTS0RVhX30r++rbaGrrJCs9TGZ6iMy0MCNy0hmVl0lxbgY5mWnUNLVR1dBGZX0boZAxviib8SOyGVeUzajcDLLSj73JZ0pxLm+fW9rzuKMrQnVjO/vqW6lsaGNvfSuV9a3srYs+rqhrZfWuOhpaO8jOCJOVFiYrPURdSwcHmjsGdM7s9DAlseQyIiediENXxIm4MzI3gzljC5g7toDZY/IpyE4nPRxSTUWOSXVTOwsnjQjk3EoKCbS9uol99W2xX7GZZKeHWb+nnn9u3s8/Nu9n5fYDNLV39ZTPzQhHv8jSw5jBQy9V9PwqTgsZnZHX/0TOSg9x2rRirjh1EmMKsxiRk0FRTjru9NQcqpva6YpEf+W7Q8TBee1X+fTRebx1ZsmgfIEnQno4xJjCLMYUZh3xsU1tneyubWFffStGtJaSFjY6uiLUt3RQ29xBbUsH1Y2xZNfQxu7aVsIhCJthZmypbOSBVRVveO6QwcjcTN46s4Qlc0bzlhnFCV+AXVJT97xHaj5KETtrmlmx/QAzS/OZPSaf0GF+EW6vbuIPa/bwh5f2sK6i/nX70sNGR1f023hmaR4XLZzAnLEFzBqTx4zS/DeszdrU1smGvfWs3V3P3vpWMsIhMtJCpIeN2WMKeNOUkSnzZZ4McjPTmFmaz8zS/GN6ntrmdtbvqWfT3gaaO7ro6Ix2jG+vbuYvL+/j3pW7SA8bi6cVc978MSydNyaQkaqSGrrnPQrqPTLskkJXxNlb38qO6mZaO7soyk6nKCeD3Iwwa3bX8cyWap7ZWs3OmmZmluYzZ2wBc8YWsLu2mb+sr2Tjvoae5xqZm8HiqaOYN76AzLQwGWEjHAqxo6aZ9XvqWV9Rz/7YyMSTJhbx+XfOYUZpPvsb2qhqbONAUzuzxuTz5unFlBYc/pdubmYaiyaNZNGkkXF7feTIFeVkcNq0Yk6bVvyGfZ1dEVZsP8BfN1Typ7V7+dx9a7jx92s5depILjhxPOcdP5bczGH3MZRDqG6KjWYOYOAagHnfXrokV1ZW5uXl5Ud83BMbK7nlofXsOtBCe9fBO0kz0kIsnFjElOI8tlQ2sn5PPY1t0QW0T548giVzSjllyig27Wvg6S3VPL1lP3vqWl/3HOlhY8bofOaOK2D+uAKWzC1lwohgOo0kebg76/fU88iavTz8UgWvVjeTkxHm3PljufzUiZw0MZg2ZEkuz22t5r3Ln+XXV57CW2a88YfG0TKzFe5edrhyw+YnSlF2OrPG5PP2eaVMHJnDpJG5ZGeEo23HLe00tHYyfXQeCyeOeF0TTHfnb0FWOoU5rzXnHD+hkHcvmoC709oRob0rQkfsNio3M5BBJ5LczIx54wqZN66QTy+dycodB7hnxS4eXr2He1fu4opTJ3HDubPJU81hWKsJcIoLGEZJ4aSJI/jvyxcd8XGhkHHcyIP/yjczsjPCZKO2fBk4M+tpCvz8O+fy7cc2ccfT23h8QyVfv+h4zphZEnSIEpD93UkhoD4F/ZwVCVhuZhpffNdc7rl6MVnpId7/i+f55G9fZF996+EPliGnpnveIyUFkeFt0aSR/OGa07nmrOk8snYvZ932JMuf2kL7UQwUlNRV3dRGYWy8SxCUFESSSFZ6mE8tncWfrzuDxdNG8bU/buCc7z3FExsrgw5NEqS6KbgxCqCkIJKUJo3K5WcfOJlffLCMSMT50B0v8ME7nmdzZWPQoUmc1TS2B9bJDEoKIkntrNmlPHbdW7nxvDmsePUA59z+FF/4/Vp21hzbVCaSvKqb2gId3KikIJLkMtJCfPSMqTzxmTN5z8nH8dsXdnDmbU9y3e9WsXFvw+GfQFJKTVM7IwNYhrNbXJOCmV1nZuvMbK2Z3WVmWX32Z5rZ78xss5k9Z2aT4xmPSCorzsvkaxcez1OffRsfPG0yj67byztuf4pvP7aRSCS1BqFK/yIRp6apneJkbz4ys2lmlhm7f6aZXWNmRYc5ZjxwDVDm7vOBMHBpn2JXAgfcfTrwXeDWI/0DRIabsYXZfOFf5vLPG87i4kUT+MHjm/n4XStp6TWZoqSm2pYOIk5KNB/dC3SZ2XTg58AU4DcDOC4NyDazNCAH6Dud5PnA/8Tu3wOcbZq4XmRARuRm8K2LT+DG8+bwyNq9vOcnz7C3TmMbUllNz7xHyd98FHH3TuBC4HZ3vw4Ye6gD3H03cBuwA9gD1Ln7Y32KjQd2xsp3AnXAqIGHLzK8mRkfPWMqP3t/GVurGrngR/9kc6X6GVLV/sZgRzPDwJNCh5ldBnwAeDi27ZCTw5vZCKI1gSnAOCDXzC7vW6yfQ9/QOGpmV5lZuZmVV1VVDTBkkeHj7Dml3H31aXRGnPf85FnW7q4LOiQ5Ct3zHqVC89GHgMXAV919m5lNAX59mGOWANvcvcrdO4D7gNP6lNkFHAcQa2IqBGr6PpG7L3f3MncvKynRnDAi/Zk7roB7rl5MdnqYy5Y/ywuvvuGjJEmuujHYabNhgEnB3de7+zXuflesBpDv7t84zGE7gFPNLCfWT3A28HKfMg8SrX0AXAw87qk2l7dIEplcnMvdVy+mpCCTK37+HE9tUs06lVTHagojcpI8KZjZk2ZWYGYjgdXAHWb2nUMd4+7PEe08XgmsiZ1ruZndbGbLYsV+Dowys83Ap4DPHeXfISIx44qy+b9/W8yU4jyu+lU5z29TjSFV1DS1U5QT3LxHMPDmo0J3rwcuAu5w90VEm4cOyd2/5O6z3X2+u1/h7m3u/kV3fzC2v9XdL3H36e7+JnffevR/ioh0K87L5NdXvonxRdl8+Jcv8NKu2qBDkgGobmwPfKnWgSaFNDMbC7yH1zqaRSSJjcrL5NcfOYWinHQ+8Ivn2bRPVyUlu+qmtkCvPIKBJ4WbgUeBLe7+gplNBV6JX1giMhjGFmZz50dOIT0c4vKfPUdFbUvQIckhVDe2MyrAKS5g4B3Nd7v7Ce7+sdjjre7+7viGJiKDYdKoXH79kVNobu/iY3eupK1TI5+TVU1TOyMDvPIIBt7RPMHM7jezSjPbZ2b3mtmEeAcnIoNjZmk+t12ygNU7a7n5ofVBhyP96Io4B5rbKU6R5qM7iF4+Oo7oKOSHYttEJEWcM38MV791Gnc+t4N7VuwKOhzpo7a5PfB5j2DgSaHE3e9w987Y7ZeARpGJpJjrl87ktGmjuPH+Nayr0KjnZNIzmjnAeY9g4Elhv5ldbmbh2O1yoDqegYnI4EsLh/j+ZScxIieDf/vVCvbHRtBK8Cobov8XJSmSFD5M9HLUvUQnt7uY6NQXIpJiivMyWf7+RexvbOOj/1tOa4c6npPB7tiVYeOLsgONY6BXH+1w92XuXuLuo939AqID2UQkBZ0woYjb33sSq3bWcv3dq7VITxLYUxud9ry0MDVqCv351KBFISIJd878MXzunNk8/NIevvPnTUGHM+xV1LZQkp9JZlo40DjSjuFYLYYjkuKuOmMq2/Y38cMnNjOjNI/zTxwfdEjDVkVdC+MCbjqCY6spqL4pkuLMjFsumE/ZpBHceP9adlQ3Bx3SsFVR28K4wqzDF4yzQyYFM2sws/p+bg1ExyyISIpLD4e4/dITMYNrf/cinV2RoEMadtyditrW5K8puHu+uxf0c8t392NpehKRJDJhRA5fvfB4Vu6o5fuPbw46nGGnrqWDlo4uxiZ7TUFEho9lC8Zx0cLx/PDxV7QGQ4Ily+WooKQgIr3cfP58JozI4brfraKxrTPocIaN7stRk775SESGl7zMNL773gXsrm3hB3/V7PiJUlEXrSmMLVLzkYgkmUWTRnLJogn8/B/b2FyphXkSYXdtCxnhEMUBr6UASgoi0o8bzp1NTkaYmx5cj7uuPo+3PbWtjCnMIhQKfviXkoKIvEFxXiafXjqLf2zezyNr9wYdzpBXUdvCuCRoOgIlBRE5iPedMpE5Ywv4ysPraW5Xp3M8RZNC8J3MoKQgIgeRFg5xy/nzqKhr5YcauxA3nV0R9jW0Ma5QSUFEklzZ5JFctHA8P/37Vrbtbwo6nCGpsqGNroirpiAiqeFz584mMy3Mlx9ap07nONiTRJejQhyTgpnNMrNVvW71ZnZtnzKFZvaQma02s3VmpoV7RJLM6Pwsrl0ygyc3VvHXlyuDDmfI2R0buJYMo5khjknB3Te6+4nufiKwCGgG7u9T7D+A9e6+ADgT+LaZBbtqtYi8wQdOm8z00Xnc/PB6rdQ2yCpiU1wkw7xHkLjmo7OBLe6+vc92B/LNzIA8oAbQZQ4iSSY9HOLLy+axo6aZnz61NehwhpQ9tS3kZ6WRn5UedChA4pLCpcBd/Wz/ITAHqADWAJ90d83bK5KE3jy9mPOOH8OPntzMrgNad2Gw7K5tTZqmI0hAUog1By0D7u5n9zuAVUTXZjgR+KGZFfTzHFeZWbmZlVdVVcU1XhE5uBvfORfD+PJD64MOZcioqG1JmqYjSExN4Vxgpbvv62ffh4D7PGozsA2Y3beQuy939zJ3LyspKYlzuCJyMOOLsrnm7Bn8ef0+/rK+v4+0HKk9SbIMZ7dEJIXL6L/pCGAH0f4GzKwUmAWowVIkiV35linMGJ3HTQ+to6Vdnc7HoqW9iwPNHcMnKZhZDvB24L5e2642s6tjD28BTjOzNcBfgRvcfX88YxKRY5ORFuIrF8xn14EWfvC4ptc+Ft1TZifLvEcAcV1S092bgVF9tv241/0KYGk8YxCRwXfK1FG8e+EEfvr3rVy0cDzTR+cHHVJK6r4cNVmmuACNaBaRo/T/zptNTkYaN96/lkhEI52PRk9SGC7NRyIydI3Ky+Q/z53Nc9tquPP5HUGHk5Iqalsxg9KC5Gk+UlIQkaP23pOP4/QZxXz9jy+zs0ZjF45URW0LJXmZZKQlz1dx8kQiIinHzPjGu08gZMZn7lmtZqQjtKeuNamajkBJQUSO0fiibD7/zjk8u7WGXz/XdyYbOZRdB5qTajQzKCmIyCB478nHccbMEr7+xw3sqFYz0kC0dXax80ALU0tygw7ldZQUROSYmRm3vvt40kLGjb9fo3UXBmBHdTNdEVdSEJGhaWxhNp9eOpO/v7KfR9buDTqcpLelKrqS3bSSvIAjeT0lBREZNJefOom5Ywu45eH1NLVpFvxD2VLVCMBUJQURGarSwiFuuWAee+pa+b6mwDikLVWNjCnIIi8zrhNLHDElBREZVIsmjeSSRRP4+d+3sbmyIehwktaWqiamjU6u/gRQUhCROLjh3NnkZIT5wu/X0dmldbP6cne2VjYmXX8CKCmISBwU52Xy2XNm88zWapZ852/89vkdtHVqmu1uVY1tNLR1MrVYNQURGSbed8pEfnz5IvKz0vncfWt46zef5IFVu4MOKylsqYxdeTRaNQURGSbMjHPmj+HBj7+ZX135JkoLMrn+7tVsrmwMOrTAdV95pOYjERl2zIzTZ5Tw8w+eTHZ6mC8+sHbYD27bUtVITkaYMUk0O2o3JQURSYjivEw+c85snt5SzYOrK4IOJ1Bbq5qYWpJLKGRBh/IGSgoikjD/+qaJnDChkK/84WXqWzuCDicwW6oamVqcfE1HoKQgIgkUDhlfuWA++xvb+O6fNwUdTiBa2rvYXduSlP0JoKQgIgl2woQi3nfKRP7n6VdZV1EXdDgJt21/E+4k5cA1UFIQkQB8ZulsRuZm8P/uW0PXMFuYJ5mvPAIlBREJQGFOOl981zxW76rjl0+/GnQ4CbW1qgkzmJKEA9dASUFEAvKuE8bytlklfPuxjew6MHwW5tlS1cj4omyy0sNBh9IvJQURCYSZ8ZULjwfg878fPmMXtlQl55xH3eKWFMxslpmt6nWrN7Nr+yl3Zmz/OjP7W7ziEZHkM74om+uXzuLJjVXDYuxCJOJsrWoanknB3Te6+4nufiKwCGgG7u9dxsyKgP8Clrn7POCSeMUjIsnpA6dNZsGEQm5+aD21ze1BhxNXe+tbaenoStorjyBxzUdnA1vcfXuf7f8K3OfuOwDcvTJB8YhIkgiHjK9fdAK1LR1845ENQYcTV8l+5REkLilcCtzVz/aZwAgze9LMVpjZ+/s72MyuMrNyMyuvqqqKa6AiknhzxxVw5Vum8NsXdvLCqzVBhxM33ZMBTi0ZxjUFM8sAlgF397M7jWjT0juBdwBfMLOZfQu5+3J3L3P3spKSkrjGKyLBuHbJDMYXZXPj/Wto7xyaC/Os2lnL6PxMSvIygw7loBJRUzgXWOnu+/rZtwv4k7s3uft+4ClgQQJiEpEkk5ORxpeXzWPTvkZ+/o9tQYcTF+WvHqBs8gjMkm8ivG6JSAqX0X/TEcADwOlmlmZmOcApwMsJiElEktCSuaW8Y14p3/vrJnbWDK2xC3vrWtld28KiSSODDuWQ4poUYl/0bwfu67XtajO7GsDdXwb+BLwEPA/8zN3XxjMmEUluNy2bR9hsyI1dKN8e7SspmzQi4EgOLa5Jwd2b3X2Uu9f12vZjd/9xr8ffcve57j7f3W+PZzwikvzGFmbzmXfM4m+bqnhg1dAZu1D+6gGy08PMHVcQdCiHpBHNIpJ0rlg8mZMmFvHlh9ZR3dgWdDiDYsX2Ayw4rpD0cHJ/7SZ3dCIyLIVDxq3vPoHGtk5ufnh90OEcs6a2TtbvqacsyfsTQElBRJLUzNJ8/uNt03lgVQVPbEjtca2rd9bSFXEWTU7u/gRQUhCRJPaxM6cxY3QeN96/hsa2zqDDOWrl2w9gBgsnKimIiBy1zLQwt158AnvrW7npwXVBh3PUyrcfYObofAqz04MO5bCUFEQkqS2cOIKPv20696zYlZIzqXZFnBe3H0iJpiNQUhCRFHDN2TNYNGkEN963JuUGtW3a10BDW2fSj0/opqQgIkkvLRzi9veeCAaf/O2LdHalztxI5dsPAKTElUegpCAiKeK4kTl87cLjWbmjlu/99ZWgwxmwFa/WUJKfyXEjs4MOZUCUFEQkZbxrwTguWTSBHz6xmWe2VAcdzoCs2HGARROTexK83pQURCSl3LRsHlNG5XLd71ZxoCm5V2rbWdPMzpoWylKkkxmUFEQkxeRmpvH9y06iuqmNG+59KaknzXv4pT0ALJ07JuBIBk5JQURSzvzxhdxwzmweW7+PO5/bEXQ4B/XAqt2cNLGIiaNygg5lwJQURCQlffjNUzhjZgm3PLyejXsbgg7nDV7Z18CGvQ0sWzAu6FCOiJKCiKSkUMj49iULyM9K42N3rqChtSPokF7nwdUVhAzeecLYoEM5IkoKIpKySvIz+cFlC9le3cxn70me/gV358HVFSyeNorR+VlBh3NElBREJKUtnjaKG86ZxSNr9/LTv28NOhwAXtpVx/bqZs5fMD7oUI6YkoKIpLyPnj6V844fwzce2cDTW/YHHQ4Prq4gIxziHfNT56qjbkoKIpLyzIxvXryAKcW5fOI3L7K7tiWwWLoizkOrK3jrrJKUmBW1LyUFERkS8jLT+MkVZbR3Rrjyly8Etv7Cc9uqqWxoS7mrjropKYjIkDF9dB4/et9CXqls5Jq7XqQrkviO5/tX7iYnI8ySOaUJP/dgUFIQkSHljJkl3LRsHo9vqOSrf3g5oefeWtXI/S/u5qKF48nOCCf03IMlLegAREQG2xWnTmJrVSO/+Oc2phTncMXiyQk5761/2kBmWohPnj0zIeeLByUFERmSPv/OueyobuYLD0SX8Yx3Ynh+Ww2PrtvH9UtnUpKfGddzxVPcmo/MbJaZrep1qzezaw9S9mQz6zKzi+MVj4gML+GQ8aP3LWTJnNF84YF1LH9qS9zO5e589Y8vM6YgiyvfMjVu50mEuCUFd9/o7ie6+4nAIqAZuL9vOTMLA7cCj8YrFhEZnrKUWo3NAAAKu0lEQVTSw/z35Yt45wlj+dofN/C9v7wSl1HPD7+0h9U7a/n00pkp25fQLVHNR2cDW9x9ez/7PgHcC5ycoFhEZBhJD4f4/qUnkZ0e5rt/2URTeyf/ee7sQVv0pq2zi1v/tIE5Ywu4aOGEQXnOICUqKVwK3NV3o5mNBy4EzuIQScHMrgKuApg4cWKcQhSRoSocMr757hPIzQiz/Kmt1DV38LWLjiccOrbE4O584fdr2XWghV9deezPlwzifkmqmWUAy4C7+9l9O3CDu3cd6jncfbm7l7l7WUlJSTzCFJEhLhQyblo2j2vOms7vynfy8d+spK3zkF89h+TufP2RDfxf+S6uOWs6p88YGt9NiagpnAusdPd9/ewrA34bq8YVA+eZWae7/z4BcYnIMGNmfGrpLAqy0/nKH16m/pcv8N33nMjogiOfyfTHf9vK8qe28v7Fk7ju7al7CWpfiRi8dhn9NB0BuPsUd5/s7pOBe4B/V0IQkXj7yOlTue2SBZS/eoAl3/kb//fCzgF3QLs7v3rmVW790waWLRjHTe+aN2j9E8kgrknBzHKAtwP39dp2tZldHc/ziogczsWLJvDIJ09n9pgCPnvvS1zx8+fZXNl40PLuzhMbK7ngv57mCw+s48xZJdx2yQJCQ6AfoTdLlkUpBqqsrMzLy8uDDkNEhohIxLnz+R3c+sgGGts6ecv0Yt53ykSWzC2lvTPChr31rN1dz30v7mb1zlrGF2Xz8bOmc/GiCaSHU2emIDNb4e5lhy2npCAiAlUNbfzuhR3c9fxOdte2kJ+ZRmN7J91fkRNH5vDvZ07jooUTyEhLnWTQTUlBROQodEWcJzdW8ti6fYwtymL+uELmjS9gTEFWSvcdDDQpaO4jEZFewiHj7DmlnJ2iU18fq9SrA4mISNwoKYiISA8lBRER6aGkICIiPZQURESkh5KCiIj0UFIQEZEeSgoiItIj5UY0m1kVUAvU9dlVeJhth7vf/W8xsP8oQuvv/APZ33f7oR73jbX3tqOJO5Ex974fxGut94feH4fan4rvjyOJGWCGuxceNhJ3T7kbsPxItx3ufq9/ywcrpoHs77v9UI/7xnqscScy5qBfa70/9P4Yau+PI4l5IOfovqVq89FDR7HtcPf7O/5YYxrI/r7bD/W4v1iPJe5Extz7fhCvtd4fR07vj4HfT/aYB3IOIAWbj+LNzMp9AJNGJZtUjFsxJ04qxq2Yg5GqNYV4Wh50AEcpFeNWzImTinEr5gCopiAiIj1UUxARkR5DOimY2S/MrNLM1h7FsYvMbI2ZbTaz71uv1TXM7BNmttHM1pnZNwc36vjEbWY3mdluM1sVu52X7DH32n+9mbmZFQ9exHF7nW8xs5dir/FjZjYuBWL+lpltiMV9v5kVDWbMcYz7kthnMGJmg9aOfyyxHuT5PmBmr8RuH+i1/ZDv+8AczeVTqXIDzgAWAmuP4tjngcWAAY8A58a2vw34C5AZezw6ReK+Cbg+lV7r2L7jgEeB7UBxsscMFPQqcw3w4xSIeSmQFrt/K3BrKrw/gDnALOBJoCzoWGNxTO6zbSSwNfbviNj9EYf6u4K+Demagrs/BdT03mZm08zsT2a2wsz+bmaz+x5nZmOJfrif8ej/3v8CF8R2fwz4hru3xc5RmSJxx1UcY/4u8Flg0Du/4hGzu9f3Kpo72HHHKebH3L0zVvRZYMJgxhzHuF92943JEutBvAP4s7vXuPsB4M/AOUF+Vg9nSCeFg1gOfMLdFwHXA//VT5nxwK5ej3fFtgHMBE43s+fM7G9mdnJco33NscYN8PFYE8EvzGxE/ELtcUwxm9kyYLe7r453oL0c8+tsZl81s53A+4AvxjHWboPx3uj2YaK/WhNhMOOOt4HE2p/xwM5ej7vjT5a/6w2G1RrNZpYHnAbc3av5LrO/ov1s6/7Fl0a0GngqcDLwf2Y2NZbt42KQ4v5v4JbY41uAbxP9AoiLY43ZzHKAG4k2bSTEIL3OuPuNwI1m9p/Ax4EvDXKorwUySDHHnutGoBO4czBj7M9gxh1vh4rVzD4EfDK2bTrwRzNrB7a5+4UcPP7A/66DGVZJgWjNqNbdT+y90czCwIrYwweJfoH2rkJPACpi93cB98WSwPNmFiE630lVMsft7vt6HfdT4OE4xgvHHvM0YAqwOvZBnACsNLM3ufveJI25r98AfyCOSYFBijnWAfovwNnx/IHTy2C/1vHUb6wA7n4HcAeAmT0JfNDdX+1VZBdwZq/HE4j2Pewi+L+rf0F3asT7BkymV4cR8DRwSey+AQsOctwLRGsD3Z1A58W2Xw3cHLs/k2jV0FIg7rG9ylwH/DbZY+5T5lUGuaM5Tq/zjF5lPgHckwIxnwOsB0oGO9ZEvD8Y5I7mo42Vg3c0byPaujAidn/kQN/3QdwCDyCufxzcBewBOohm5iuJ/vr8E7A69kH44kGOLQPWAluAH/LaQL8M4NexfSuBs1Ik7l8Ba4CXiP4CG5vsMfcp8yqDf/VRPF7ne2PbXyI618z4FIh5M9EfN6tit0G9YiqOcV8Ye642YB/waJCx0k9SiG3/cOw13gx86Eje90HcNKJZRER6DMerj0RE5CCUFEREpIeSgoiI9FBSEBGRHkoKIiLSQ0lBhgQza0zw+X5mZnMH6bm6LDqr6loze+hws5SaWZGZ/ftgnFukL12SKkOCmTW6e94gPl+avzZJXFz1jt3M/gfY5O5fPUT5ycDD7j4/EfHJ8KKaggxZZlZiZvea2Qux25tj299kZk+b2Yuxf2fFtn/QzO42s4eAx8zsTDN70szuseh6A3d2z3kf214Wu98YmwRvtZk9a2alse3TYo9fMLObB1ibeYbXJgTMM7O/mtlKi867f36szDeAabHaxbdiZT8TO89LZvblQXwZZZhRUpCh7HvAd939ZODdwM9i2zcAZ7j7SURnMf1ar2MWAx9w97Nij08CrgXmAlOBN/dznlzgWXdfADwFfLTX+b8XO/9h57WJzftzNtER5wCtwIXuvpDoOh7fjiWlzwFb3P1Ed/+MmS0FZgBvAk4EFpnZGYc7n0h/htuEeDK8LAHm9prZssDM8oFC4H/MbAbRmSnTex3zZ3fvPZf+8+6+C8DMVhGdE+cffc7TzmsTDK4A3h67v5jX5sj/DXDbQeLM7vXcK4jOuQ/ROXG+FvuCjxCtQZT2c/zS2O3F2OM8okniqYOcT+SglBRkKAsBi929pfdGM/sB8IS7Xxhrn3+y1+6mPs/R1ut+F/1/Zjr8tc65g5U5lBZ3P9HMCokml/8Avk90PYYSYJG7d5jZq0BWP8cb8HV3/8kRnlfkDdR8JEPZY0TXMwDAzLqnPi4EdsfufzCO53+WaLMVwKWHK+zudUSX8LzezNKJxlkZSwhvAybFijYA+b0OfRT4cGzef8xsvJmNHqS/QYYZJQUZKnLMbFev26eIfsGWxTpf1xOd9hzgm8DXzeyfQDiOMV0LfMrMngfGAnWHO8DdXyQ6E+elRBe7KTOzcqK1hg2xMtXAP2OXsH7L3R8j2jz1jJmtAe7h9UlDZMB0SapInMRWj2txdzezS4HL3P38wx0nEiT1KYjEzyLgh7ErhmqJ4/KnIoNFNQUREemhPgUREemhpCAiIj2UFEREpIeSgoiI9FBSEBGRHkoKIiLS4/8D3n9wlGQ5JFMAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.recorder.plot()"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total time: 00:23\n",
"epoch train_loss valid_loss accuracy\n",
"1 5.177172 4.802299 0.309109 (00:23)\n",
"\n"
]
}
],
"source": [
"learn.fit_one_cycle(1, 3e-2, moms=(0.8, 0.7))"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"learn.save('fit-head')"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"learn.load('fit-head');"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total time: 02:52\n",
"epoch train_loss valid_loss accuracy\n",
"1 4.552735 4.594989 0.326132 (00:27)\n",
"2 4.448153 5.098676 0.257221 (00:28)\n",
"3 4.126873 4.391115 0.337241 (00:29)\n",
"4 3.803120 4.354942 0.343525 (00:29)\n",
"5 3.485168 4.389339 0.344239 (00:29)\n",
"6 3.294815 4.425426 0.342907 (00:29)\n",
"\n"
]
}
],
"source": [
"learn.unfreeze()\n",
"learn.fit_one_cycle(6, 3e-3, moms=(0.8, 0.7))"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"learn.save('fine-tuned')"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"learn.load('fine-tuned');"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total time: 00:00\n",
"\n"
]
},
{
"data": {
"text/plain": [
"'samsung galaxy tab , lo mejor'"
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn.predict('samsung', 5, temperature=1.1, min_p=0.001)"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [],
"source": [
"learn.save_encoder('fine-tuned-enc')"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [],
"source": [
"pickle.dump(data_lm.vocab, open('itos', 'wb'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Title classifier"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"vocab = pickle.load(open('itos', 'rb'))"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"data_clf = TextClasDataBunch.from_df('.', train_df, valid_df, df, tokenizer=Tokenizer(lang='es'), \n",
" vocab=vocab, text_cols='title', label_cols='condition')"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"learn = text_classifier_learner(data_clf, max_len=100, drop_mult=0.5)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"learn.load_encoder('fine-tuned-enc')\n",
"learn.freeze()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n"
]
}
],
"source": [
"learn.lr_find()"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEKCAYAAAA1qaOTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xd81dX9x/HXJxtImAlh75nIkqGAKKIMq4LaqqC2ov1pa+uuVq1Vq9Zq67aOOrF1Ua1WwYIMZSiyZYe9w0oIKwGyz++Pe8EQsgj35t6bvJ+Px314v9/vufd+jjfhne86x5xziIiInK6wQBcgIiLVgwJFRER8QoEiIiI+oUARERGfUKCIiIhPKFBERMQnFCgiIuITChQREfEJBYqIiPhERKAL8JX4+HjXpk2bQJchIhJSFi9evNc5l+CL96o2gdKmTRsWLVoU6DJEREKKmW311XvpkJeIiPiEAkVERHxCgSIiIj7h10AxsxFmttbMNpjZ/SVsf97Mlnof68zsQLHtdc1sh5m97M86RUTk9PntpLyZhQOvAEOBVGChmU1wzqUca+Ocu6tI+9uAXsXe5nFglr9qFBER3/HnHko/YINzbpNzLhcYD4wqo/0Y4KNjC2bWG0gEpvqxRhER8RF/BkpzYHuR5VTvupOYWWugLfCNdzkMeBa4t6wPMLObzWyRmS1KT0/3SdEiIlI5/gwUK2FdafMNjwb+45wr8C7/BpjknNteSnvPmzn3hnOuj3OuT0KCT+7LOUlhoePTxalsSMv0y/uLiFQX/ryxMRVoWWS5BbCzlLajgd8WWe4PDDKz3wCxQJSZZTnnTjqx709HcvP53cfLmLxyN3ExEbx9fV/6tW1YlSWIiIQMf+6hLAQ6mllbM4vCExoTijcys85AA2DusXXOuWudc62cc22Ae4B/VXWY7DhwlJ+9Npcpq3Zz+wUdSYiL5udvz+fr1XuqsgwRkZDht0BxzuUDtwJTgNXAx865VWb2mJmNLNJ0DDDeOVfa4bAqt2jLPka9/B3b9x3h7bF9uXtoJz75VX86N4nj5vcW8+ni1ECXeILCQseGtKxAlyEiNZwF0b/jp6VPnz7OF2N5ZWTlcM5fZ5BYN5q3ru9Dh8Zxx7dl5eTzq/cWMWdDBi9c3ZPLepV4jUGVe+nr9Tw3bR3ndUrgoUu6nlCziEhZzGyxc66PL95Ld8oX8+9F2zmaV8Cbv+hz0j/MsdERvDO2L12b1mXcnM0BqvBE6Zk5/GPWRro0ieOHbfsZ/sK3/GnCKg4cyQ10aSJSwyhQiigodHwwbxv92zWiY2LJf+VHR4RzaY+mLEs9yK6DR6u4wpO99PV6cvILeeXaM5l5z2BG923Jv+ZuYcizs1i3R1emiUjVUaAUMWNNGjsOHOXn/VuX2W5YUhMApqcE9gT95r2H+WjBNkb3bUn7hFgaxUbzxOXd+N/tgwgz49fvLyYrJz+gNYpIzaFAKeK9eVtJrBvN0KTEMtt1aBxLu4Q6TA1woDw9ZQ1REWHccWHHE9Z3bVqXv4/pxZa9h7nv0+VUl/NkIhLcFCheW/YeZta6dMb0a0VkePn/W4YlNWHuxgwOHs07adt787by+ZId/ijzuCXb9jNpxW5uGtSOxnExJ23v374R9w7vwv+W72LcnC1+rUVEBBQox30wfysRYcaYfq0q1H5YciL5hY6Za9NOWL814zCPTljFQ1+sJDP75LDxBeccT05eQ3xsFDed267Udr8+rx1DkxL5y6TVLN66zy+1iIgco0ABjuYW8PGiVIYnNyGx7sl/7ZekZ4v6JMRFM3XViYe9Xvx6PWaQmZ3P+AVljhxTYYdz8pmwbCevzdzIQ5+v5BfvLGDB5n3ccUFHYqNLH+zAzHjmyh40b1CL33zwAyt3HPRJPSIiJVGgABOX7+Tg0bxyT8YXFRZmDE1KZObaNLLzPEOQbUjL4vMlOxg7oA1nt2vIO3M2k1dQeNJrD2XnkXYou0KfczS3gOvens/tHy3hr1+tYcKynezNymVMv1aMrsDeVL1akbx2bW8KCh2XvvwdD3y2gn2HdUmxiPieP8fyCgnOOd6bu5VOibGcdYrjdA1LSuTD+duYuzGD87s05oXp64iJDOfX57VneepBbnh3IROX7eSKM1scf83hnHyuePV7MrJymHTHIJrWq1Xq+xcUOm4fv4Sl2w/w/NU9GJbUhDpl7JGUJqlZXb7+3WBenL6ef87dwv+W7+R3wzrz87NbExZW0hieIiKnrsbvoWzNOELKrkP8/OzWmJ3aP6792zciNjqCqSm7Wb3rEF8u38UNA9vQKDaawZ0T6JwYxxuzNx2/yso5x0Ofr2RjehbZeYXc/tES8kvYgznW9pEJK5mWsoc/XZrM5b1aVCpMjqlXK5KHL03iqzsG0a1FPR6ZsIrnpq2r9PuJiBRX4wOlTXwd5tw35IS9iIqKjghncOcEpqXs4dmp64iLieDmQe0Bz/mLm85tx5rdmcxa55mr5ZPFqXy2ZAd3XNCRJ6/oxsIt+3lh+voS3/vVmRt5f942fnVeO64f0KbS/SuuY2Ic7//yLK7u05KXZ2xgyqrdPntvEanZanygADSpF1Ppv/6HJTdhb1Yu01fv4f/OaUe92pHHt43s0YwmdWN4fdYm1u7O5OEvVjKgfSNuG9KRy3o156o+LXhl5ga+Xf/j5GAHjuTy9JQ1PD1lLaN6NuO+4V1Ou3/FmRmPjkqmR4t6/O7jZWxM18CSInL6FCinaXDnBCLDjfq1I7nxnDYnbIuKCOPGc9owd1MGY8ctIDY6khdG9yTce97i0ZFn0CEhlrv+vZTlqQf485cpDHjqG16ZsZFLezTjbz/r7rdzHDGR4bx2XW+iI8L41Xu6o15ETp9GG/aBt77dRLP6tfhJt6YnbcvMzmPAk9+QlZvP+788i4Ed4k/Yvm5PJiNf/o7svELCw4yRPZrxq/Pa0aVJ3Sqp/fuNe7nurfkMT27Cq9eeecrnkUQktPlytGEFShWYumo32fmFjOzRrMTt01P2sGDLPn5+dmtaNqxdxdXBm7M38cSk1Ywd0IaHL0nSlV8iNYgvA6XGXzZcFYYlNylz+4VJiVxYzvhh/vR/g9qy+1A2b3+3mcM5+Tz10+7HD8sB7D6YzTtzNjOkS2PObtcoYHWKSHBToAhmxh8v7kpsdAQvfr2eI7kFPH91Twqd461vN/HKjI0czSvgrW83ceeFnfjt+R1OCBwREVCgiJeZcdfQTsRGR/DEpNXszcphx4GjpO4/yojkJtxxYUden7WR56atY+GWfTx/dU/iY6MDXbaIBBGdQ5GTfDh/Gw9+voLOiXE8fGkSA9p7LiRwzjF+4XYembCK+rUiee263vRu3SDA1YrI6dBJ+RIoUHxr18GjJMRGE1HCUP4pOw9xyweL2XMom9eu7c35XRoHoEIR8QXNKS9+17RerRLDBDxjg316ywA6NI7lpn8t8vvcLyISGhQoUinxsdF8dNPZ9GnTgDv/vZRxczYHuiQRCTAFilRaXEwk797Qj+HJiTw6MYXnp63TdMMiNZgCRU5LTGQ4r1xzJlf2bsGLX6/nqa/WKFREaihdNiynLSI8jL/+tDvRkWG8PmsTOXmFPHJpkoZxEalhFCjiE2FhxuOjziA6Ipy3v9tMTn4BT1zWTcO4iNQgChTxmWN33EdHhPHqzI18u34v7RNiaRtfhzaNatO8QW0S4qJJiIsmPjaK6IjwQJcsIj7k10AxsxHAi0A48JZz7qli258Hzvcu1gYaO+fqm1lP4DWgLlAAPOGc+7c/axXfMDPuHd6Z1o1qM3vdXrZkHGbRln0czi04qW10RBhR4WFEhBsR4WH0aFGPJ6/oTkJc+Xfgf7VyNxOW7WBYUhOGJiWe1myWIuIbfrux0czCgXXAUCAVWAiMcc6llNL+NqCXc+5GM+sEOOfcejNrBiwGujrnDpT2ebqxMXg559iblcvug9mkZWaTnplDWmYOWTn55BUUkl/gyM4rYMKyndSrFcnL15xJv7YNS32/ySt2cetHS4gIM3LyC4mJDOOCron89MzmnN+5sc7diJyCUBltuB+wwTm3CcDMxgOjgBIDBRgDPALgnDs+2blzbqeZpQEJQKmBIsHLzI4f6oJ6pba78Zy23PL+Ysa8OY/7RnTmpkHtTgqHaSl7uO2jJfRsWZ93b+jL2t2ZfLF0J5NW7OJ/y3dxQZfG/PnyM2har5afeyUixfnzsuHmwPYiy6nedScxs9ZAW+CbErb1A6KAjX6oUYJI16Z1mXDbOQztmshfJq1h9BvzeG/eVrbvOwLAjDVp/OaDxSQ3r8e4G/oSFxNJnzYNefyyM5j/hwt46JIk5mzcy7DnZvPh/G26fFmkivnzkNeVwHDn3P95l38O9HPO3VZC2/uAFsW3mVlTYCZwvXNuXgmvuxm4GaBVq1a9t27d6vN+SNVzzvHu91t4Z85mtu87CkCHxrFs23eETomxfPDLs6lXO7LE127NOMz9n65g7qYMBrRvxHNX9aRJvZiqLF8kpITE4JBm1h/4k3NuuHf5AQDn3JMltF0C/NY5932RdXXxhMmTzrlPyvs8nUOpfpxzbNp7mBlr0pi5Np3oiDCeubIHDepElfu6jxZs58//S6FWZDgvju7FOR3jy3yNSE0VKoESgeek/AXADjwn5a9xzq0q1q4zMAVo67zFmFkUMBmY6Jx7oSKfp0CR4jakZXLL+z+wIT2LOy/oxG1DOui+GJFiQmK0YedcPnArnrBYDXzsnFtlZo+Z2cgiTccA492JyXYVcC4w1syWeh89/VWrVE8dGsfxxa0Duaxnc56fvo6x7y4kMzsv0GWJVFuaD0WqvWOHwB7+YiU9Wtbnnzf2I1b3rYgAIbKHIhIszIxrzmrF38f0Yun2A4x9ZwFZOfmBLkuk2lGgSI1xUbem/H1ML5ZsP8AN4xZwWKEi4lMKFKlRftKtKS+N7sUP2w5ww7iFpGVmB7okkWpDgSI1zsXdm/Li6J4s2b6fIc/M4o3ZG8nNLwx0WSIhT4EiNdIl3Zsx9a7z6Ne2IX+ZtIYRL8xmxtq0QJclEtIUKFJjtY2vwztj+zJubF8ccMO4hTz0+UrtrYhUkq6dlBrv/C6NGdghnmemruWN2ZtYs/sQr1x7Jo3jNGSLyKnQHooIEBURxh9+0pWXxvRixY6DXPr37/hh2/5AlyUSUhQoIkWM7NGMz24ZSFREGKNfn8fcjRmBLkkkZChQRIpJalaXibeeQ7P6MTzw2XKy806ebVJETqZAESlB/dpRPHF5N7ZkHOHlbzYEuhyRkKBAESnFwA7xXHFmc/4xayPr9mQGuhyRoKdAESnDHy9OIi4mggc+W0FhYfUYSFXEXxQoImVoWCeKBy9OYvHW/Xy0cFugyxEJaroPRaQcPz2zOZ/9kMpTk9ewNzOXo3kFHM3NJzuvkJYNa9G9RX26t6hH/dplzyQpUt0pUETKYWY8cXk3rvzH9zw/fR1REWHUjgonMjyM9Myc4+3aNKrNn0YmM7hz4wBWKxI4mmBLpILyCzxDskSE/3ik+ODRPFbuOMiy1AN8siiVI7n5zLhnMLWj9LeahAZNsCUSABHhYSeECUC9WpEM7BDPbwZ34Jkru7PnUA7/mLUpQBWKBJYCRcRHerduyKU9mvH6rI3sOHA00OWIVDkFiogP3TeiMwB/nbwmwJWIVD0FiogPtWhQm5vPbceEZTtZvHVfoMsRqVIKFBEf+/V57WkcF81jX67WzZBSoyhQRHysTnQEvx/RhWXbD/DBAt0MKTWHAkXED67o1ZxzOsTz0OcrGTdnc6DLEakSChQRPwgLM966vg/DkxN5dGIKf/1qDdXlni+R0ihQRPwkJjKcV6/tzTVnteK1mRu555Pl5BVovnqpvnQ7r4gfhYcZT1x2Bo3jonlh+nqiIsJ48opugS5LxC+0hyLiZ2bGnRd24qZBbflowTbNVS/Vll8DxcxGmNlaM9tgZveXsP15M1vqfawzswNFtl1vZuu9j+v9WadIVbjjwk4k1o3mkS9WUaDLiaUa8lugmFk48ApwEZAEjDGzpKJtnHN3Oed6Oud6An8HPvO+tiHwCHAW0A94xMwa+KtWkaoQGx3BH37SlRU7DvLvhdsDXY6Iz/lzD6UfsME5t8k5lwuMB0aV0X4M8JH3+XBgmnNun3NuPzANGOHHWkWqxMgezTirbUP+NmUN+w/nBrocEZ/yZ6A0B4r+GZbqXXcSM2sNtAW+OZXXmtnNZrbIzBalp6f7pGgRfzIzHh2VTGZ2Ps9MXRvockR8yp+BYiWsK+3A8WjgP865glN5rXPuDedcH+dcn4SEhEqWKVK1ujSpyy/6t+bDBdtYueNgoMsR8Rl/Bkoq0LLIcgtgZyltR/Pj4a5Tfa1IyLlraCca1Ynm9/9ZTk5+QfkvEAkB/gyUhUBHM2trZlF4QmNC8UZm1hloAMwtsnoKMMzMGnhPxg/zrhOpFurGRPLUFd1I2XWIpzTUvVQTfgsU51w+cCueIFgNfOycW2Vmj5nZyCJNxwDjXZFxKZxz+4DH8YTSQuAx7zqRauPCpETGDmjDuDlb+Hr1nkCXI3LaNKe8SADl5Bdw+Svfs+vgUSbfcS5N6sUEuiSpYTSnvEg1ER0Rzt+v6UV2XiF3/nuJbniUkKaxvEQCrH1CLI+OSub3/1nO2HELaFovhojwMKLCwxjcOYHBnRsHukSRClGgiASBK3u3YENaFpNX7mJDWhZ5BYUczingowXb+OaewTSvXyvQJYqUS+dQRIJU6v4jDHl2Fhd3a8rzV/cMdDlSTekcikgN0KJBbX55Tlv+u2QHK1J1A6QEPwWKSBC7ZXB7GtaJ4olJKZrxUYKeAkUkiNWNieSuCzsyb9M+pq9OC3Q5ImVSoIgEudH9WtEuoQ5PTl6tKYQlqClQRIJcZHgYf7ioK5vSDzN+wbZAlyNSKgWKSAi4oGtjzmrbkFdmbKRQNz9KkFKgiIQAM+Pqvi3ZfSib5RryXoKUAkUkRAzp0pjwMGPqqt2BLkWkRAoUkRBRv3YUZ7VtyNQUjUwswalCgWJm7c0s2vt8sJndbmb1/VuaiBQ3LCmRDWlZbEzPCnQpIiep6B7Kp0CBmXUA3sYz//uHfqtKREo0NLkJANO0lyJBqKKBUuidMOty4AXn3F1AU/+VJSIlaV6/Fmc0r6vzKBKUKhooeWY2Brge+NK7LtI/JYlIWYYlNWHJ9gOkZWYHuhSRE1Q0UG4A+gNPOOc2m1lb4H3/lSUipRmWnIhz8LWGYpEgU6FAcc6lOOdud859ZGYNgDjn3FN+rk1EStA5MY5WDWvrsJcEnYpe5TXTzOqaWUNgGTDOzJ7zb2kiUhIzY1hSInM2ZJCVkx/ockSOq+ghr3rOuUPAFcA451xv4EL/lSUiZRmW3ITcgkJmrU0PdCkix1U0UCLMrClwFT+elBeRAOndugEN60QxRYe9JIhUNFAeA6YAG51zC82sHbDef2WJSFnCw4wRZzRh4vKd3P3xUnYdPBrokkQ0p7xIqDqck8/LMzbw9nebCTO4eVA7fnVee+pERwS6NAkhVT6nvJm1MLP/mlmame0xs0/NrIUvChCRyqkTHcF9I7rw9d3nMSypCS99s4GLX/qWnPyCQJcmNVRFD3mNAyYAzYDmwETvOhEJsJYNa/PSmF7847oz2ZJxhInLdgW6JKmhKhooCc65cc65fO/jXSDBj3WJyCkantyEzolxvP3dZqrLoWwJLRUNlL1mdp2ZhXsf1wEZ5b3IzEaY2Voz22Bm95fS5iozSzGzVWb2YZH1f/OuW21mL5mZVbBWkRrJzLjxnDas3nWIuZvK/fUU8bmKBsqNeC4Z3g3sAn6GZziWUplZOPAKcBGQBIwxs6RibToCDwADnXPJwJ3e9QOAgUB34AygL3BeBWsVqbFG9WxOwzpRvPPdlkCXIjVQRYde2eacG+mcS3DONXbOXYbnJsey9AM2OOc2OedygfHAqGJtbgJecc7t937OscGJHBADRAHReAai1HjdIuWIiQznurNa8fWaPWzZezjQ5UgNczozNt5dzvbmwPYiy6nedUV1AjqZ2Rwzm2dmIwCcc3OBGXj2hnYBU5xzq0+jVpEa47r+rYkMC2PcnM2BLkVqmNMJlPLOaZS0vfiZwgigIzAYGAO8ZWb1vRN5dQVa4AmhIWZ27kkfYHazmS0ys0Xp6RqCQgSgcVwMl/ZoxieLUzl4NC/Q5UgNcjqBUt5lJKlAyyLLLYCdJbT5wjmX55zbDKzFEzCXA/Occ1nOuSxgMnD2SQU494Zzro9zrk9Cgi46EznmxnPacCS3gH8v3BboUqQGKTNQzCzTzA6V8MjEc09KWRYCHc2srZlFAaPx3MtS1OfA+d7PisdzCGwTsA04z8wizCwSzwl5HfISqaDkZvU4u11D3pi9iRenr2f+pgyy83TDo/hXmWM0OOfiKvvGzrl8M7sVzxhg4cA7zrlVZvYYsMg5N8G7bZiZpQAFwL3OuQwz+w8wBFiBZ0/oK+fcxMrWIlITPfiTJO7/bDkvfL0ONx2iIsIY2L4RT1/Zg/jY6ECXJ9WQxvISqeYOHsljwZZ9zNuUwQfzt9KqYW0+vOlshYoAARjLS0RCV73akQxNSuShS5J4Z2xftu07wrVvzicjK+d4m4ysHB74bAW9Hpuqy42l0hQoIjXIgPbxvHN9X7buO8w1b85nz6Fs3v5uM4OfmcnHi7az/0geX6/RXPVSOQoUkRpmQId43r6+L1syDjPwqW94/MsUerVqwJQ7B9GmUW3mbtwb6BIlRGniBJEaaGCHeN4Z25dXZ27gxoFtGdKlMWZG//bxfLlsJ/kFhUSE6+9NOTUKFJEaamCHeAZ2iD9h3YD2jfhowTZW7jxEz5b1A1SZhCr9CSIix53drhEA3+uwl1SCAkVEjkuIi6ZzYhxzN2r4+1AxcdlOJi7bGRRz4ChQROQE/ds3YuGWfZpKOAQUFjr+NmUN78/bSjBMGaVAEZETDOwQT3ZeIUu3HQh0KVKO2evT2b7vKNed3TrQpQAKFBEppl/bhoQZfK/DXkHvg/nbiI+NYnhyk0CXAihQRKSYerUi6da8ns6jBLmdB47y9eo9XNWnJVERwfFPeXBUISJBpX/7eJZs38+R3PxAlyKlGL9wOw4Y069VoEs5ToEiIicZ0L4ReQWOhVv2B7oUKUFeQSHjF2xjcKcEWjasHehyjlOgiMhJ+rRpQGS46X6UIPX16j2kZeZw7VnBcTL+GAWKiJykdlQEvVo20HmUIPX+vG00qxfD+V0aB7qUEyhQRKREAzo0YuWOgxw8onnpg8nmvYf5bsNexvRrRXhY4O89KUqBIiIlGtQxgUIH78/fGuhSpIgP528lIsy4um/LQJdyEgWKiJSod+sGXNy9KS9MX0fKzkOBLke85mzIoH/7RjSuGxPoUk6iQBGRUv151BnUrx3F3R8v1VAsQSI9K4fm9WsFuowSKVBEpFQN6kTxt592Z83uTJ6ftj7Q5dR4BYWOjKwc4mOjA11KiRQoIlKm87s0Zky/lrw+eyMLt+wLdDk12v4juRQ6z6jQwUiBIiLlevDiJFo0qMXvPl7GitSDFBQGfqj0mig9MwcI3kDRjI0iUq7Y6AievbIn1701n0tf/o66MRH0a9uIAe0bcc1ZrYiJDA90iTWCAkVEqoV+bRvy3f3nM3djhuexKYPpq/ewMT2LJy7vFujyaoTjgRKk51AUKCJSYY3jYhjVszmjejYH4NGJq3j3+y1c1aclPTQHvd+lZwX3HorOoYhIpd01tBPxsdE89MVKnVepAumZOdSKDKdOdHDuCyhQRKTS6sZE8seLu7I89SAfLdgW6HKqvb1ZOUG7dwJ+DhQzG2Fma81sg5ndX0qbq8wsxcxWmdmHRda3MrOpZrbau72NP2sVkcoZ2aMZ/ds14ukpa8nwHpIR/0jPrKGBYmbhwCvARUASMMbMkoq16Qg8AAx0ziUDdxbZ/C/gaedcV6AfkOavWkWk8syMx0Ylczgnn6cmrwl0OdVaemZO0J6QB//uofQDNjjnNjnncoHxwKhibW4CXnHO7QdwzqUBeIMnwjk3zbs+yzl3xI+1ishp6JgYxy8HteWTxaks2KybH/0lvQYf8moObC+ynOpdV1QnoJOZzTGzeWY2osj6A2b2mZktMbOnvXs8IhKkbh/SkVYNa3PXv5dy4EhuoMupdnLzCzlwJC9oh10B/wZKSQP1F78MJALoCAwGxgBvmVl97/pBwD1AX6AdMPakDzC72cwWmdmi9PR031UuIqesTnQEL1/Ti7TMbO75ZDnO6aovX8o4HNyXDIN/AyUVKDpgfwtgZwltvnDO5TnnNgNr8QRMKrDEe7gsH/gcOLP4Bzjn3nDO9XHO9UlISPBLJ0Sk4rq3qM8DF3Vl+uo9vDNnS6DLqVaC/S558G+gLAQ6mllbM4sCRgMTirX5HDgfwMzi8Rzq2uR9bQMzO5YSQ4AUP9YqIj5yw8A2DE1K5KnJq1m2/UCgy6k2anSgePcsbgWmAKuBj51zq8zsMTMb6W02BcgwsxRgBnCvcy7DOVeA53DX12a2As/hszf9VauI+I6Z8fTPutM4LobffvgDB49qCmFfCIVA8evtls65ScCkYuseLvLcAXd7H8VfOw3o7s/6RMQ/6teO4u/X9OLKf8zl9Vkb+f2ILoEuKeQdC5T42KgAV1I63SkvIn5xZqsGDGjfiMkrd+sEvQ/szcqhbkwE0RHBe8GrAkVE/GZYchM27z3M+rSsQJcS8oL9HhRQoIiIHw1LSgRgysrdAa4k9AX7sCugQBERP0qsG8OZreozJUWBcro8gRIT6DLKpEAREb8antyElTsOsX2fRk86HcE+jhcoUETEz4YnNwFgasqeAFcSuo7k5nM4t4D4uOC9wgsUKCLiZ23i69A5MY4pq3TYq7L2ZnrGRtMeiojUeMPPaMLCLfvYq/lSKiU9KxsI7psaQYEiIlVgeHIizsF0HfaqlFC4Sx4UKCJSBZKa1qVFg1o67FVJChQRES8zY3hyE+ZsyCAzW2N7nar0zBzCDBrVUaCIiDDijCbkFhQyY63mLjpV6Vm5NKwTRXhYSdNMBQ8FiohUiTNbNSA+NoqpOux1ytJU9bhrAAAR4klEQVQzc4J6psZjFCgiUiXCw4wLuiQya206ufmFgS4npITCOF6gQBGRKjQ0KZHMnHzmbcoIdCkhZW8IjOMFChQRqULndIynVmQ403T5cIU550JiYEhQoIhIFYqJDGdQx3imr96jOVIq6NDRfHILCoP+LnlQoIhIFRualMiug9ms3HEo0KWEhPSs0LgHBRQoIlLFLuiaSJjBNA1pXyHHb2rUHoqIyIka1omiT+uGGn24grSHIiJShqFJiazZnak5UiogVIZdAQWKiATAUO/UwLraq3zpmTlEhhv1akUGupRyKVBEpMq1ia9Dx8axCpQK2JvluUveLLiHXQEFiogEyNCkRBZs2ceBI7mBLiWoOOcoLPzxkupQuQcFICLQBYhIzTQ0KZFXZ25kxto0Lu/VItDlBI0r/zGXRVv3ExFmRIQbufmFnN+5caDLqhAFiogERI8W9WlSN4Z352zh0u7NiAjXAZND2Xks2rqf8zolcEbzuuQXOHILCrm4W9NAl1YhChQRCYiwMOMPF3fl9o+W8NZ3m/n1ee1PapOTXwBAdER4VZcXECk7PTd7jh3YJmT2SorSnwQiEjCXdm/K8OREnpu2jg1pmSds277vCEOfm80Vr37P0dyCAFVYtVZ5AyW5Wd0AV1I5fg0UMxthZmvNbIOZ3V9Km6vMLMXMVpnZh8W21TWzHWb2sj/rFJHAMDP+fFk3akeF87tPlpNf4BnWfmvGYUa/MY99h3NJ2XWIP36+skaM/bVq50ES4qJpHBcT6FIqxW+BYmbhwCvARUASMMbMkoq16Qg8AAx0ziUDdxZ7m8eBWf6qUUQCLyEumkdHJrNs+wHe+m4zG9OzuOr1uRzJzWf8zWdz+5COfPpDKuMXbg90qX6XsvNQyO6dgH/3UPoBG5xzm5xzucB4YFSxNjcBrzjn9gM459KObTCz3kAiMNWPNYpIEBjZo9nxQ19Xvz6PgkLH+Jv7c0bzetx+QUcGdYznkQmrWJF6MNCl+k12XgHr07IUKKVoDhT9kyLVu66oTkAnM5tjZvPMbASAmYUBzwL3lvUBZnazmS0ys0Xp6ZqnWiRUHTv0VScqnDCD8Tf3p3OTOMAz0+OLo3sRXyeKWz5YfFr3reTkF7B2d2ZQHj5btyeTgkJHcrN6gS6l0vwZKCXd1ln8W4wAOgKDgTHAW2ZWH/gNMMk5V+Y+rnPuDedcH+dcn4SEBB+ULCKBkhAXzZe3D2LyHYPo0Dj2hG0N60TxyrVnsudQNg/+d2WlP+PlbzYw/IXZjHlzHou37jvdkn0q1E/Ig38DJRVoWWS5BbCzhDZfOOfynHObgbV4AqY/cKuZbQGeAX5hZk/5sVYRCQLN69eiUSnDtPdq1YCbz23HpJW7KjWopHOOL5fvonWj2mxIO8xPX5vL2HELWLmj9MNouw4e5c3ZmziSm3/Kn3eqVu08SFx0BC0b1Pb7Z/mLPwNlIdDRzNqaWRQwGphQrM3nwPkAZhaP5xDYJufctc65Vs65NsA9wL+ccyVeJSYiNce1Z7XGgA8XbDvl167bk8XmvYe5aVA7Zv9+MPeN6MKSbQe49OXvePiLlWRm553Qfsqq3Vz04rc8MWk1t3+05PgVaP6yauchujarS1hY8I/ZVRq/BYpzLh+4FZgCrAY+ds6tMrPHzGykt9kUIMPMUoAZwL3OuQx/1SQioa1Z/Vpc0DWRjxduP37TY0V9tXI3ZjAsOZHaURHcMrg93953Ptf3b8N787Yy9LnZTEvZQ3ZeAQ99vpJfvbeYFg1qcfuQDkxfncafJq7y27mXgkLHml2ZIX24C/x8p7xzbhIwqdi6h4s8d8Dd3kdp7/Eu8K5/KhSRUHPd2a2ZlrKHr1buZlTP4tf5lG7yyl30ad3ghHs86sZE8qeRyYzq2Yz7P13BTf9aRHxsFHuzcvm/c9ry+xFdiIoIIye/kNdnb6JFg9ol3tF/ujbvzeJoXkFIn5AH3SkvIiFmUId4Wjeqzfvztlb4NVv2HmbN7kyGJzcpcXuvVg2YeNs53DOsE43qRDPuhr788ZIkoiI8/0TeN6ILl/ZoxlOT1/DF0h0+6UdRx07IJzUN7T0UBYqIhJSwMOPas1qxcMt+1uw+VKHXfLXKM3/9iDNKDhSAqIgwbh3SkSl3nXvSOFphYcYzV3anX9uG3PvJciat2FX5DpRg1c5DRIWH0TExtvzGQUyBIiIh58reLYmKCKvwXsrklbvp3qIeLU7jCqroiHDe/Hkfzmhel9988AMvfb3eZ+dUVu08SKcmsUSG+IjLoV29iNRIDepEcUm3pvz3hx1k5ZR9Se/OA0dZtv1AqYe7TkW92pF8eNPZXNGrOc9NW8ft45eSnVeAc46UnYd4Yfo6xrwx75Tu6HfOsWrnIZKbhvb5E9Dw9SISoq7r35rPluzg8yU7uO7s1qW2m+I93HVRGYe7TkVMZDjPXtWDjolx/G3KGtbsOsTRvAJS9x/FDCLDw3j8fyn8++azKzRt766D2Rw4kkdy89A+fwLaQxGRENWrZX2Smtbl9dkbSTuUXWq7ySt30zkxjnYJvjs/YWbcMrg9r1/Xm7yCQjolxvHXn3Zj4YMX8oeLurBg8z7mbqzYHRDV4Q75YxQoIhKSzIzHL0smIyuX0W/MY08JoZKemcPCLfsY7qO9k+KGJTdh5r3n887YvlzdtxXxsdGM7teKJnVjeHbaugqdY1m18yBm0KWJAkVEJGB6t27IP2/sx55D2Yx+Yx67D/4YKhlZObwwfR3O+e5wV0XERIbz2yEdWLx1P7PX7y23/aqdh2gbX4c60aF/BkKBIiIhrW+bhvzrl/1Iz8xh9BtzmbE2jbs/Xkr/p77hg/nbuKR7U7p4Ry6uKlf3aUnz+rV4roy9FOcc32/cy+Kt+0P+hsZjQj8SRaTG693aEyrXv72AG8YtpE5UOFf3ackv+remY2LVhgl47mm5bUgH7v9sBTPWpjGkS+LxbYWFjmmr9/DazI0s3X6A+NhorjurVZXX6A8WjPMCVEafPn3cokWLAl2GiARQys5DLEs9wCXdmxIXExnQWvIKCrng2VnUrRXBP2/ox3zvifpZ69LZtu8ILRvW4lfntudnvVsQExkesDrNbLFzro9P3kuBIiLiH/9ZnMo9nyw7vlw7Kpy+bRpyxZnNubhbUyKC4EZGXwaKDnmJiPjJZT2bsWbXIerXjqR/+0Z0b1E/5O+GL4sCRUTETyLCw/jjJUmBLqPKVN+oFBGRKqVAERERn1CgiIiITyhQRETEJxQoIiLiEwoUERHxCQWKiIj4hAJFRER8otoMvWJm6UBJE0zXA4rPx1l8XdHlkp4XXRcPlD8mdclKqqWibU61H8WXQ7kv5T0/nX6UVWdFtgdTX07nOylpW035+Sq+XLwv/v75KqtNVfx8tXbOJZRTX8U456r1A3ijvHVFl0t6XmzdIl/WUtE2p9qP6tSXCnw/le5HRfpS1vZg6svpfCen+vNUnX6+yuuLv3++fNkXf/+ulPeoCYe8JlZg3cRynpf0Hr6qpaJtTrUfxZdDuS8VeX46ynufsrYHU19O5zspaVtN+fkqvhzKffH370qZqs0hr6piZoucj0bmDLTq0pfq0g9QX4JRdekH+L8vNWEPxdfeCHQBPlRd+lJd+gHqSzCqLv0AP/dFeygiIuIT2kMRERGfqNGBYmbvmFmama2sxGt7m9kKM9tgZi+ZmRXZdpuZrTWzVWb2N99WXWItPu+Hmf3JzHaY2VLv4ye+r7zEevzynXi332NmzszifVdxmfX443t53MyWe7+TqWbWzPeVn1SLP/rxtJmt8fblv2ZW3/eVl1iPP/pypfd3vdDM/H6u5XT6UMr7XW9m672P64usL/P3qUT+vIQs2B/AucCZwMpKvHYB0B8wYDJwkXf9+cB0INq73DhE+/En4J7q8J14t7UEpuC5Vyk+VPsC1C3S5nbgHyHaj2FAhPf5X4G/hvB30hXoDMwE+gRrH7z1tSm2riGwyfvfBt7nDcrqb1mPGr2H4pybDewrus7M2pvZV2a22My+NbMuxV9nZk3x/GLPdZ7/8/8CLvNuvgV4yjmX4/2MNP/2wm/9CAg/9uV54PdAlZ009EdfnHOHijStQxX0x0/9mOqcy/c2nQe08G8vPPzUl9XOubVVUb/38yrVh1IMB6Y55/Y55/YD04ARlf23oUYHSineAG5zzvUG7gFeLaFNcyC1yHKqdx1AJ2CQmc03s1lm1tev1ZbudPsBcKv3kMQ7ZtbAf6WW67T6YmYjgR3OuWX+LrQCTvt7MbMnzGw7cC3wsB9rLYsvfr6OuRHPX8CB4su+BEpF+lCS5sD2IsvH+lWp/mpO+SLMLBYYAHxS5HBhdElNS1h37C/FCDy7jmcDfYGPzaydN+WrhI/68RrwuHf5ceBZPL/4Vep0+2JmtYEH8RxiCSgffS845x4EHjSzB4BbgUd8XGqZfNUP73s9COQDH/iyxoryZV8Cpaw+mNkNwB3edR2ASWaWC2x2zl1O6f2qVH8VKCcKAw4453oWXWlm4cBi7+IEPP/YFt1FbwHs9D5PBT7zBsgCMyvEM35Ouj8LL+a0++Gc21PkdW8CX/qz4DKcbl/aA22BZd5fthbAD2bWzzm328+1F+eLn6+iPgT+RxUHCj7qh/cE8CXABVX5B1cxvv5OAqHEPgA458YB4wDMbCYw1jm3pUiTVGBwkeUWeM61pFKZ/vr7BFKwP4A2FDm5BXwPXOl9bkCPUl63EM9eyLETVj/xrv818Jj3eSc8u5MWgv1oWqTNXcD4UP1OirXZQhWdlPfT99KxSJvbgP+EaD9GAClAQlV9F/7++aKKTspXtg+UflJ+M56jKg28zxtWpL8l1lXVX2YwPYCPgF1AHp5E/iWev2a/ApZ5f+AfLuW1fYCVwEbgZX68STQKeN+77QdgSIj24z1gBbAcz19oTf3dD3/1pVibLVTdVV7++F4+9a5fjmd8puYh2o8NeP7YWup9+P1qNT/25XLve+UAe4ApwdgHSggU7/obvd/HBuCG8vpb1kN3youIiE/oKi8REfEJBYqIiPiEAkVERHxCgSIiIj6hQBEREZ9QoEi1ZmZZVfx5b5lZko/eq8A8owqvNLOJ5Y3Ia2b1zew3vvhskcrQZcNSrZlZlnMu1ofvF+F+HNTQr4rWbmb/BNY5554oo30b4Evn3BlVUZ9IcdpDkRrHzBLM7FMzW+h9DPSu72dm35vZEu9/O3vXjzWzT8xsIjDVzAab2Uwz+4955vT44NhcEd71fbzPs7wDOS4zs3lmluhd3967vNDMHqvgXtRcfhzsMtbMvjazH8wzX8Uob5ungPbevZqnvW3v9X7OcjN71If/G0VOokCRmuhF4HnnXF/gp8Bb3vVrgHOdc73wjOL7lyKv6Q9c75wb4l3uBdwJJAHtgIElfE4dYJ5zrgcwG7ipyOe/6P38csdH8o4rdQGeEQsAsoHLnXNn4pl/51lvoN0PbHTO9XTO3Wtmw4COQD+gJ9DbzM4t7/NEKkuDQ0pNdCGQVGRk1rpmFgfUA/5pZh3xjKwaWeQ105xzReegWOCcSwUws6V4xlb6rtjn5PLjoJqLgaHe5/35cW6JD4FnSqmzVpH3XoxnrgrwjK30F284FOLZc0ks4fXDvI8l3uVYPAEzu5TPEzktChSpicKA/s65o0VXmtnfgRnOucu95yNmFtl8uNh75BR5XkDJv0t57seTlKW1KctR51xPM6uHJ5h+C7yEZx6UBKC3cy7PzLYAMSW83oAnnXOvn+LnilSKDnlJTTQVzzwiAJjZsWG/6wE7vM/H+vHz5+E51AYwurzGzrmDeKb7vcfMIvHUmeYNk/OB1t6mmUBckZdOAW70zpeBmTU3s8Y+6oPISRQoUt3VNrPUIo+78fzj3Md7ojoFz5QDAH8DnjSzOUC4H2u6E7jbzBYATYGD5b3AObcEz0iyo/FMRtXHzBbh2VtZ422TAczxXmb8tHNuKp5DanPNbAXwH04MHBGf0mXDIlXMO4vkUeecM7PRwBjn3KjyXicS7HQORaTq9QZe9l6ZdYAATK0s4g/aQxEREZ/QORQREfEJBYqIiPiEAkVERHxCgSIiIj6hQBEREZ9QoIiIiE/8P0peZCe0Q1ZoAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.recorder.plot(skip_end=8)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total time: 00:25\n",
"epoch train_loss valid_loss accuracy\n",
"1 0.502706 0.497329 0.769300 (00:25)\n",
"\n"
]
}
],
"source": [
"learn.fit_one_cycle(1, 3e-3, moms=(0.8, 0.7))"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"learn.save('first')"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total time: 00:29\n",
"epoch train_loss valid_loss accuracy\n",
"1 0.478203 0.451679 0.795000 (00:29)\n",
"\n"
]
}
],
"source": [
"learn.freeze_to(-2)\n",
"learn.fit_one_cycle(1, slice(1e-3/(2.6**4), 1e-3), moms=(0.8, 0.7))"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"learn.save('second')"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total time: 00:42\n",
"epoch train_loss valid_loss accuracy\n",
"1 0.472015 0.437852 0.802800 (00:42)\n",
"\n"
]
}
],
"source": [
"learn.freeze_to(-3)\n",
"learn.fit_one_cycle(1, slice(5e-4/(2.6**4), 5e-4), moms=(0.8, 0.7))"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"learn.save('third')"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total time: 10:32\n",
"epoch train_loss valid_loss accuracy\n",
"1 0.447866 0.431794 0.805600 (01:02)\n",
"2 0.434827 0.429153 0.805000 (01:02)\n",
"3 0.433098 0.418664 0.811900 (01:04)\n",
"4 0.440408 0.415776 0.811500 (01:03)\n",
"5 0.440411 0.414932 0.812300 (01:02)\n",
"6 0.431286 0.417159 0.811300 (01:02)\n",
"7 0.421277 0.405521 0.819000 (01:02)\n",
"8 0.422464 0.410475 0.816700 (01:05)\n",
"9 0.415283 0.423141 0.808500 (01:02)\n",
"10 0.433402 0.412326 0.815200 (01:03)\n",
"\n"
]
}
],
"source": [
"learn.unfreeze()\n",
"learn.fit_one_cycle(10, slice(1e-4/(2.6**4), 1e-4), moms=(0.8, 0.7))"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"learn.save('final')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Predict"
]
},
{
"cell_type": "code",
"execution_count": 142,
"metadata": {},
"outputs": [],
"source": [
"learn.load('final');"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" <div>\n",
" <progress value='0' max='1', style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
" 0.00% [0/1 00:00<00:00]\n",
" </div>\n",
" "
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"('used', tensor(1), tensor([0.0055, 0.9945]))"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn.predict('iphone 7 como nuevo') # 'iphone 7 as new'"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"preds, _ = learn.get_preds(DatasetType.Test, ordered=True)"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>title_isnew_prob</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>0.972612</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>0.840828</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>0.000209</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>0.890611</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>0.145972</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" title_isnew_prob\n",
"0 0.972612\n",
"1 0.840828\n",
"2 0.000209\n",
"3 0.890611\n",
"4 0.145972"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"preds_df = pd.DataFrame({'title_isnew_prob': preds[:,0]})\n",
"preds_df.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# save predictions\n",
"preds_df.to_feather(PATH/'title-df')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python [conda env:fastai]",
"language": "python",
"name": "conda-env-fastai-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.6"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Display the source blob
Display the rendered blob
Raw
4.tabular-nlp.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
5.full-deep-learning.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@joshfp
Copy link
Author

joshfp commented Jun 29, 2019

make gist public

@kontrabas380
Copy link

kontrabas380 commented Jan 29, 2020 •
edited
Loading

Hello! You've done nice job and I've got a question. When you finish training this model, how can you predict one example? It's not working with .predict(example).
I've done this with two AWD_LSTM networks, but in the end I've met an issue with this error while making prediction:
AttributeError: 'ConcatDataset' object has no attribute 'set_item'

Best regards

@ascientist
Copy link

Hello! You've done nice job and I've got a question. When you finish training this model, how can you predict one example? It's not working with .predict(example).
I've done this with two AWD_LSTM networks, but in the end I've met an issue with this error while making prediction:
AttributeError: 'ConcatDataset' object has no attribute 'set_item'

Best regards

Same problem here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment