Skip to content

Instantly share code, notes, and snippets.

@fformenti

fformenti/Ensemble_Methods_Boosting.ipynb

Last active February 9, 2018 13:22
Show Gist options
  • Save fformenti/d3012b443e16c27d80fa915aa59b645a to your computer and use it in GitHub Desktop.
Save fformenti/d3012b443e16c27d80fa915aa59b645a to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
Ensemble_Methods_Boosting.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Pré Requisitos"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 0.1 Loss Function"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- Mede a distância entre o valor estimado e seu valor real. Um exemplo de Loss Function é o Mean Squared Error, que é usado em problemas de Regressão Multivariada"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"MSE.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"regression.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_PERGUNTA: Alguém sabe uma outra forma de \"fit\" uma reta sem usar Ordinary Least Squares (Mínimos Quadrados)?_ "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 0.2 Gradient Descent"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Se ao invés de usar o OLS para encontrar a reta, nós chutamos um reta qualquer e ao pouco tentar chegar perto da reta estimada pelo OLS. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Chutar uma reta qualquer significa chutar um valor para Beta0 (intercept) e Beta1 (inclinação).\n",
"\n",
"Então teoriacemnte se chutasseos todas as retas possíveis seriamos capaz de construir um gráfico do tipo Heat Map onde Beta0 e Beta1 são os eixos x/y e Erro produzido pelo par (Beta0,Beta1) é verde se for baixo e vermelho se for alto."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"gradient_descent.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_PERGUNTA: Porque usar o Gradient Descent se eu posso usar OLS e resolver analiticamente?_"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"https://www.youtube.com/watch?v=umAeJ7LMCfU"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 0.3 Regularization"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 0.3.1 Ridge Regression"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"ridge.gif\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 0.3.2 Lasso Regression"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"lasso.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Ensemble Methods"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1.2 Boosting"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![alt text](https://www.kdnuggets.com/wp-content/uploads/bias-and-variance.jpg \"Bulls Eye\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__Motivação:__\n",
"- Reduzir o bias criando múltiplos modelos especialistas partes diferentes do domínio das variáveis explanatórias\n",
"\n",
"__Como:__\n",
"\n",
"- Treinando modelos sequencialmente e após cada iteração identificando as observações que não obtiveram uma previsão acurada e dá maior enfoque à elas no modelo subsequente.\n",
"- Usaremos os modelos de todas as iterações para criar um modelo único.\n",
"- Each classifier specializes on a particular subset of examples"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Prós:**\n",
"- Como é pensado para reduzir o viés, geralmente os algoritmos de Boosting nos dão previsões melhores.\n",
"- Em geral produz resultados com baixa variância mesmo sem ser este o seu foco, mas como nem sempre isso é verdade deve-se ficar atento para overfitting.\n",
"\n",
"**Contras**\n",
"- O Método não funciona tão bem para amostras pequenas\n",
"- Não é paralelizável"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1.2.1 AdaBoost (Adaptive Boosting)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"Boosting_idea.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- The weight of each learner is learned by whether it predicts a sample correctly or not."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__Pseudo Algoritmo__\n",
"\n",
"- adaptive boosting changes the sample distribution by modifying the weights attached to each of the instances.\n",
"\n",
"- It increases the weights of the wrongly predicted instances and decreases the ones of the correctly predicted instances.\n",
"\n",
"- The weak learner thus focuses more on the difficult instances\n",
"\n",
"- After being trained, the weak learner is added to the strong one according to his performance (so-called alpha weight). The higher it performs, the more it contributes to the strong learner\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- We have T different classifiers ht\n",
"- weight wt of the classifier is proportional to its accuracy on the training set\n",
"\n",
"- wt = log( 1 / β ) = log (1 − ε ) / ε"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2.1972245773362196"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import math\n",
"e = 0.1\n",
"math.log((1 - e)/e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Para quem quiser ver uma implementação de Adaboost Classifier usando Decision Trees: __[adaboost_from_scratch](https://github.com/fformenti/Adaboost/blob/master/adaboost.py)__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2.1.1 AdaBoost Regression"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"https://alliance.seas.upenn.edu/~cis520/wiki/index.php?n=lectures.boosting"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"AdaBoost training process selects only those features known to improve the predictive power of the model, reducing dimensionality and potentially improving execution time as irrelevant features do not need to be computed."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__Principais argumentos que o Adaboost Regression recebe__\n",
"- n_estimador: Número de modelos. (Tipicamente quanto maior melhor)\n",
"- learning rate: \n",
"- loss: Função que determina a atualização no peso das amostras"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Boosting na prática\n",
"\n",
"Vamos usar o dataset de housing prices que está disponivel neste __[link](https://www.kaggle.com/uciml/breast-cancer-wisconsin-data/data)__"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/usr/local/lib/python2.7/site-packages/sklearn/grid_search.py:43: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.\n",
" DeprecationWarning)\n"
]
}
],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"\n",
"from sklearn import metrics\n",
"\n",
"from sklearn.model_selection import train_test_split, cross_val_score\n",
"from sklearn.tree import DecisionTreeRegressor\n",
"from sklearn.ensemble import AdaBoostRegressor, GradientBoostingRegressor\n",
"from sklearn.grid_search import GridSearchCV \n",
"\n",
"pd.options.mode.chained_assignment = None # default='warn'\n",
"%matplotlib inline "
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"hp = pd.read_csv('../data/house_prices.csv')"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Id</th>\n",
" <th>MSSubClass</th>\n",
" <th>MSZoning</th>\n",
" <th>LotFrontage</th>\n",
" <th>LotArea</th>\n",
" <th>Street</th>\n",
" <th>Alley</th>\n",
" <th>LotShape</th>\n",
" <th>LandContour</th>\n",
" <th>Utilities</th>\n",
" <th>...</th>\n",
" <th>PoolArea</th>\n",
" <th>PoolQC</th>\n",
" <th>Fence</th>\n",
" <th>MiscFeature</th>\n",
" <th>MiscVal</th>\n",
" <th>MoSold</th>\n",
" <th>YrSold</th>\n",
" <th>SaleType</th>\n",
" <th>SaleCondition</th>\n",
" <th>SalePrice</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>60</td>\n",
" <td>RL</td>\n",
" <td>65.0</td>\n",
" <td>8450</td>\n",
" <td>Pave</td>\n",
" <td>NaN</td>\n",
" <td>Reg</td>\n",
" <td>Lvl</td>\n",
" <td>AllPub</td>\n",
" <td>...</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>2</td>\n",
" <td>2008</td>\n",
" <td>WD</td>\n",
" <td>Normal</td>\n",
" <td>208500</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2</td>\n",
" <td>20</td>\n",
" <td>RL</td>\n",
" <td>80.0</td>\n",
" <td>9600</td>\n",
" <td>Pave</td>\n",
" <td>NaN</td>\n",
" <td>Reg</td>\n",
" <td>Lvl</td>\n",
" <td>AllPub</td>\n",
" <td>...</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>5</td>\n",
" <td>2007</td>\n",
" <td>WD</td>\n",
" <td>Normal</td>\n",
" <td>181500</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3</td>\n",
" <td>60</td>\n",
" <td>RL</td>\n",
" <td>68.0</td>\n",
" <td>11250</td>\n",
" <td>Pave</td>\n",
" <td>NaN</td>\n",
" <td>IR1</td>\n",
" <td>Lvl</td>\n",
" <td>AllPub</td>\n",
" <td>...</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>9</td>\n",
" <td>2008</td>\n",
" <td>WD</td>\n",
" <td>Normal</td>\n",
" <td>223500</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>3 rows × 81 columns</p>\n",
"</div>"
],
"text/plain": [
" Id MSSubClass MSZoning LotFrontage LotArea Street Alley LotShape \\\n",
"0 1 60 RL 65.0 8450 Pave NaN Reg \n",
"1 2 20 RL 80.0 9600 Pave NaN Reg \n",
"2 3 60 RL 68.0 11250 Pave NaN IR1 \n",
"\n",
" LandContour Utilities ... PoolArea PoolQC Fence MiscFeature MiscVal \\\n",
"0 Lvl AllPub ... 0 NaN NaN NaN 0 \n",
"1 Lvl AllPub ... 0 NaN NaN NaN 0 \n",
"2 Lvl AllPub ... 0 NaN NaN NaN 0 \n",
"\n",
" MoSold YrSold SaleType SaleCondition SalePrice \n",
"0 2 2008 WD Normal 208500 \n",
"1 5 2007 WD Normal 181500 \n",
"2 9 2008 WD Normal 223500 \n",
"\n",
"[3 rows x 81 columns]"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"hp.head(3)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'pandas.core.frame.DataFrame'>\n",
"RangeIndex: 1460 entries, 0 to 1459\n",
"Data columns (total 81 columns):\n",
"Id 1460 non-null int64\n",
"MSSubClass 1460 non-null int64\n",
"MSZoning 1460 non-null object\n",
"LotFrontage 1201 non-null float64\n",
"LotArea 1460 non-null int64\n",
"Street 1460 non-null object\n",
"Alley 91 non-null object\n",
"LotShape 1460 non-null object\n",
"LandContour 1460 non-null object\n",
"Utilities 1460 non-null object\n",
"LotConfig 1460 non-null object\n",
"LandSlope 1460 non-null object\n",
"Neighborhood 1460 non-null object\n",
"Condition1 1460 non-null object\n",
"Condition2 1460 non-null object\n",
"BldgType 1460 non-null object\n",
"HouseStyle 1460 non-null object\n",
"OverallQual 1460 non-null int64\n",
"OverallCond 1460 non-null int64\n",
"YearBuilt 1460 non-null int64\n",
"YearRemodAdd 1460 non-null int64\n",
"RoofStyle 1460 non-null object\n",
"RoofMatl 1460 non-null object\n",
"Exterior1st 1460 non-null object\n",
"Exterior2nd 1460 non-null object\n",
"MasVnrType 1452 non-null object\n",
"MasVnrArea 1452 non-null float64\n",
"ExterQual 1460 non-null object\n",
"ExterCond 1460 non-null object\n",
"Foundation 1460 non-null object\n",
"BsmtQual 1423 non-null object\n",
"BsmtCond 1423 non-null object\n",
"BsmtExposure 1422 non-null object\n",
"BsmtFinType1 1423 non-null object\n",
"BsmtFinSF1 1460 non-null int64\n",
"BsmtFinType2 1422 non-null object\n",
"BsmtFinSF2 1460 non-null int64\n",
"BsmtUnfSF 1460 non-null int64\n",
"TotalBsmtSF 1460 non-null int64\n",
"Heating 1460 non-null object\n",
"HeatingQC 1460 non-null object\n",
"CentralAir 1460 non-null object\n",
"Electrical 1459 non-null object\n",
"1stFlrSF 1460 non-null int64\n",
"2ndFlrSF 1460 non-null int64\n",
"LowQualFinSF 1460 non-null int64\n",
"GrLivArea 1460 non-null int64\n",
"BsmtFullBath 1460 non-null int64\n",
"BsmtHalfBath 1460 non-null int64\n",
"FullBath 1460 non-null int64\n",
"HalfBath 1460 non-null int64\n",
"BedroomAbvGr 1460 non-null int64\n",
"KitchenAbvGr 1460 non-null int64\n",
"KitchenQual 1460 non-null object\n",
"TotRmsAbvGrd 1460 non-null int64\n",
"Functional 1460 non-null object\n",
"Fireplaces 1460 non-null int64\n",
"FireplaceQu 770 non-null object\n",
"GarageType 1379 non-null object\n",
"GarageYrBlt 1379 non-null float64\n",
"GarageFinish 1379 non-null object\n",
"GarageCars 1460 non-null int64\n",
"GarageArea 1460 non-null int64\n",
"GarageQual 1379 non-null object\n",
"GarageCond 1379 non-null object\n",
"PavedDrive 1460 non-null object\n",
"WoodDeckSF 1460 non-null int64\n",
"OpenPorchSF 1460 non-null int64\n",
"EnclosedPorch 1460 non-null int64\n",
"3SsnPorch 1460 non-null int64\n",
"ScreenPorch 1460 non-null int64\n",
"PoolArea 1460 non-null int64\n",
"PoolQC 7 non-null object\n",
"Fence 281 non-null object\n",
"MiscFeature 54 non-null object\n",
"MiscVal 1460 non-null int64\n",
"MoSold 1460 non-null int64\n",
"YrSold 1460 non-null int64\n",
"SaleType 1460 non-null object\n",
"SaleCondition 1460 non-null object\n",
"SalePrice 1460 non-null int64\n",
"dtypes: float64(3), int64(35), object(43)\n",
"memory usage: 924.0+ KB\n"
]
}
],
"source": [
"hp.info()"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEACAYAAABcXmojAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFdBJREFUeJzt3X+wXGd93/H3xxbCWKaWBR1Zkk2tJlZsZUKAxjJp+LG0\njmIyie2/bDGTRGNc/nESyHRKLDHT+jLTcYxnMiSdjpNpEugNgwUKIR65JUay8Q60aSUDdjCWhSRA\nDXLQtTHBJYQUafztH3t0tL6W7w979+6u9H7N7OjZc55znu/dq7ufPc85u5uqQpIkgHNGXYAkaXwY\nCpKklqEgSWoZCpKklqEgSWoZCpKk1ryhkGR7kseTPJbkniSvTLIqyZ4kB5PsTrJyVv9DSQ4k2Tzc\n8iVJg5S53qeQ5DLgc8CVVfX/knwS+Azwk8B3ququJLcBF1XVtiQbgXuAq4B1wAPAhqp6brg/hiRp\nEOY7Uvi/wHHg/CTLgPOBvwWuA6abPtPADU37emBHVR2vqiPAYWDToIuWJA3HnKFQVd8Ffhf4G3ph\n8L2q2gOsrqqZptsMsLpprwWO9u3iKL0jBknSBJgzFJL8GPBbwGX0nvAvSPIr/X2qN/8012dl+Dka\nkjQhls2z/meAv6qqZwCSfBr4WeBYkour6liSNcBTTf8ngUv7tr+kWfY8SQwKSXoJqirD3P985xQO\nAG9O8qokAa4B9gP3AVubPluBe5v2LmBLkuVJ1gOXA/tOt+OqGvvb7bffPvIarNM6J7VG6xz8bSnM\neaRQVX+d5E+BLwLPAV8G/gvwamBnkluAI8CNTf/9SXbSC44TwK21VD+JJOllm2/6iKq6C7hr1uLv\n0jtqOF3/O4A7Xn5pkqSl5jua59DpdEZdwoJY52BNQp2TUCNY5ySa881rQxs0cVZJkhYpCTXiE82S\npLOIoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSW\noSBJahkKkqTWvN+8pp7eV1SPht89IWmpGAqLMoon59GFkaSzz7zTR0l+Iskjfbdnk7w3yaoke5Ic\nTLI7ycq+bbYnOZTkQJLNw/0RJEmDsqiv40xyDvAksAn4TeA7VXVXktuAi6pqW5KNwD3AVcA64AFg\nQ1U917efifs6zt700WiOFCbtsZI0HOP4dZzXAIer6lvAdcB0s3wauKFpXw/sqKrjVXUEOEwvRCRJ\nY26xobAF2NG0V1fVTNOeAVY37bXA0b5tjtI7YpAkjbkFh0KS5cAvA382e10zFzTXHIfzH5I0ARZz\n9dE7gS9V1dPN/ZkkF1fVsSRrgKea5U8Cl/Ztd0mz7HmmpqbadqfTodPpLKIUSTrzdbtdut3uko65\n4BPNST4B/GVVTTf37wKeqaoPJdkGrJx1onkTp040/3j/mWVPNC9qZE80SwKW5kTzgkIhyQrg/wDr\nq+r7zbJVwE7gdcAR4Maq+l6z7gPAu4ETwPuq6rOz9mcoLHxkQ0ESMEahMPBBDYXFjGwoSALG85JU\nSdIZzFCQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlS\ny1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSa0GhkGRlkk8leSLJ/iRXJ1mVZE+Sg0l2J1nZ1397\nkkNJDiTZPLzyJUmDtNAjhd8HPlNVVwKvBw4A24A9VbUBeLC5T5KNwE3ARuBa4O4kHpFI0gSY98k6\nyYXAW6vqIwBVdaKqngWuA6abbtPADU37emBHVR2vqiPAYWDToAuXJA3eQl7BrweeTvLRJF9O8kdJ\nVgCrq2qm6TMDrG7aa4GjfdsfBdYNrGJJ0tAsW2CfNwG/UVUPJ/k9mqmik6qqktQc+3jBuqmpqbbd\n6XTodDoLqVeSzhrdbpdut7ukY6ZqrudySHIx8L+qan1z/y3AduCfA++oqmNJ1gAPVdUVSbYBVNWd\nTf/7gduram/fPmu+ccdNEk6TbUsxMpP2WEkajiRUVYY5xrzTR1V1DPhWkg3NomuAx4H7gK3Nsq3A\nvU17F7AlyfIk64HLgX0DrVqSNBQLmT4C+E3g40mWA18HbgbOBXYmuQU4AtwIUFX7k+wE9gMngFsn\n7rBAks5S804fDWVQp48WM7LTR5KAMZk+kiSdPQwFSVLLUJAktQwFSVLLUJAktQwFSVLLUJAktQwF\nSVLLUJAktQwFSVLLUJAktQwFSVLLUJAktQwFSVLLUJAktQwFSVLLUJAktQwFSVLLUJAktRYUCkmO\nJPlKkkeS7GuWrUqyJ8nBJLuTrOzrvz3JoSQHkmweVvGSpMFa6JFCAZ2qemNVbWqWbQP2VNUG4MHm\nPkk2AjcBG4FrgbuTeEQiSRNgMU/WmXX/OmC6aU8DNzTt64EdVXW8qo4Ah4FNSJLG3mKOFB5I8sUk\n72mWra6qmaY9A6xu2muBo33bHgXWvexKJUlDt2yB/X6uqr6d5J8Ce5Ic6F9ZVZWk5tj+Beumpqba\ndqfTodPpLLAUSTo7dLtdut3uko6Zqrmey0+zQXI78PfAe+idZziWZA3wUFVdkWQbQFXd2fS/H7i9\nqvb27aMWO+6oJeE02bYUIzNpj5Wk4UhCVc2eyh+oeaePkpyf5NVNewWwGXgM2AVsbbptBe5t2ruA\nLUmWJ1kPXA7sG3ThkqTBW8j00WrgL3qvlFkGfLyqdif5IrAzyS3AEeBGgKran2QnsB84Adw6cYcF\nknSWWvT00UAGdfpoMSM7fSQJGJPpI0nS2cNQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1\nDAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUmtBoZDk3CSP\nJLmvub8qyZ4kB5PsTrKyr+/2JIeSHEiyeViFS5IGb6FHCu8D9nPqm+u3AXuqagPwYHOfJBuBm4CN\nwLXA3Uk8GpGkCTHvE3aSS4BfBP4YSLP4OmC6aU8DNzTt64EdVXW8qo4Ah4FNgyxYkjQ8C3kV/2Hg\n/cBzfctWV9VM054BVjfttcDRvn5HgXUvt0hJ0tJYNtfKJL8EPFVVjyTpnK5PVVWSOt26k11Ot3Bq\naqptdzodOp3T7l6Szlrdbpdut7ukY6bqxZ/Pk9wB/CpwAjgP+CfAp4GrgE5VHUuyBnioqq5Isg2g\nqu5str8fuL2q9s7ab8017jhKwovk27BHZtIeK0nDkYSqyvw9X7o5p4+q6gNVdWlVrQe2AJ+rql8F\ndgFbm25bgXub9i5gS5LlSdYDlwP7hlO6JGnQ5pw+Oo2TL1nvBHYmuQU4AtwIUFX7k+ykd6XSCeDW\niTskkKSz2JzTR0Mb1OmjxYzs9JEkYAymjyRJZxdDQZLUMhQkSa3FnmjWCPTOZyw9z2VIZx9DYSKM\n5gS3pLOP00eSpJahIElqGQqSpJahIElqGQqSpJahIElqGQqSpJahIElqGQqSpJahIElqGQqSpJah\nIElqGQqSpJahIElqzRkKSc5LsjfJo0n2J/mdZvmqJHuSHEyyO8nKvm22JzmU5ECSzcP+ASRJg5P5\nvkglyflV9Q9JlgH/A/h3wHXAd6rqriS3ARdV1bYkG4F7gKuAdcADwIaqem7WPmvSvsCl90U3o/pe\ng9GMO2m/I+lMl4SqGuqXncw7fVRV/9A0lwPnAn9HLxSmm+XTwA1N+3pgR1Udr6ojwGFg0yALliQN\nz7yhkOScJI8CM8BDVfU4sLqqZpouM8Dqpr0WONq3+VF6RwySpAkw79dxNlM/b0hyIfDZJO+Ytb6S\nzDXPcNp1U1NTbbvT6dDpdBZSrySdNbrdLt1ud0nHnPecwvM6J/8e+CHwb4BOVR1LsobeEcQVSbYB\nVNWdTf/7gdurau+s/XhOYeEjj2zcSfsdSWe6kZ9TSPLak1cWJXkV8PPAI8AuYGvTbStwb9PeBWxJ\nsjzJeuByYN8wCpckDd5800drgOkk59ALkI9V1YNJHgF2JrkFOALcCFBV+5PsBPYDJ4BbJ+6QQJLO\nYouaPhrYoE4fLWbkkY07ab8j6Uw38ukjSdLZxVCQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQ\nJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSa95QSHJp\nkoeSPJ7kq0ne2yxflWRPkoNJdidZ2bfN9iSHkhxIsnmYP4AkaXAy35ezJ7kYuLiqHk1yAfAl4Abg\nZuA7VXVXktuAi6pqW5KNwD3AVcA64AFgQ1U917fPmrQvhU8CjKLm0Y07ab8j6UyXhKrKMMeY90ih\nqo5V1aNN+++BJ+g92V8HTDfdpukFBcD1wI6qOl5VR4DDwKYB1y1JGoJFnVNIchnwRmAvsLqqZppV\nM8Dqpr0WONq32VF6ISJJGnPLFtqxmTr6c+B9VfX93nRKT1VVkrnmGl6wbmpqqm13Oh06nc5CS5Gk\ns0K326Xb7S7pmPOeUwBI8grgvwF/WVW/1yw7AHSq6liSNcBDVXVFkm0AVXVn0+9+4Paq2tu3P88p\nLHzkkY07ab8j6Uw3FucU0ns2/BNg/8lAaOwCtjbtrcC9fcu3JFmeZD1wObBvcCVLkoZlIVcfvQX4\nPPAVTr1k3U7viX4n8DrgCHBjVX2v2eYDwLuBE/Smmz47a58eKSx85JGNO2m/I+lMtxRHCguaPhr4\noIbCYkYe2biT9juSznRjMX0kSTp7GAqSpJahIElqGQqSpJahIElqGQqSpJahIElqGQqSpJahIElq\nLfhTUsfB1Ve/nYcf/sKoy5CkM9ZEHSn88IdF1eeoOrGkt/PP/7VR/+iStCQm6kih5xyWPsuG+lEj\nkjQ2JupIQZI0XBN4pKCl0v/tekvJT2eVRsdQ0BxG9VHhkkbF6SNJUstQkCS1DAVJUmveUEjykSQz\nSR7rW7YqyZ4kB5PsTrKyb932JIeSHEiyeViFS5IGbyFHCh8Frp21bBuwp6o2AA8290myEbgJ2Nhs\nc3cSj0YkaULM+4RdVV8A/m7W4uuA6aY9DdzQtK8HdlTV8ao6AhwGNg2mVEnSsL3UV/Grq2qmac8A\nq5v2WuBoX7+jwLqXOIYkaYm97Kmd6r3TaK4L2n0nkiRNiJf65rWZJBdX1bEka4CnmuVPApf29buk\nWfYCU1NTbbvT6dDpdF5iKZJ0Zup2u3S73SUdMwv5SIEklwH3VdVPNffvAp6pqg8l2QasrKptzYnm\ne+idR1gHPAD8eM0aJMnsRQvy+te/jcce+4/A2xa97cuxYsXN/OAH/5XRvcP37BrXj7mQTi8JVTXU\nt/3Pe6SQZAfwduC1Sb4F/AfgTmBnkluAI8CNAFW1P8lOYD9wArj1JT37S5JGYt5QqKp3vciqa16k\n/x3AHS+nKEnSaPgeAklSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLUMBUlSy1CQJLVe\n6qekSkOTDPXzvl6UH9MlGQoaS6P6VFhJTh9JklqGgiSpZShIklqGgiSpZShIklqGgiSp5SWpUmNU\n748A3yOh8TGUI4Uk1yY5kORQktuGMYY0eDWimzQ+Bh4KSc4F/jNwLbAReFeSKwc9ztLojrqABeqO\nuoAF6o66gAXqjrqAeXW73VGXsCDWOXmGcaSwCThcVUeq6jjwCeD6IYyzBLqjLmCBuqMuYIG6oy5g\ngbqjLmBek/IkZp2TZxihsA74Vt/9o80ySdKYG8aJ5qFNki5bBitWbOPcc18zrCGe5x//8Wucd96X\n+NGPHl2S8XT2eiknuT/4wQ8OoZLBs87JupAggy42yZuBqaq6trm/HXiuqj7U12dyHiFJGiNVNdTL\n5IYRCsuArwH/GvhbYB/wrqp6YqADSZIGbuDTR1V1IslvAJ8FzgX+xECQpMkw8CMFSdIEq6olvdF7\n/8IB4BBw25DG+AgwAzzWt2wVsAc4COwGVvat297UcwDY3Lf8XwCPNet+v2/5K4FPNsv/N/DP+tZt\nbcY4CPzaHDVeCjwEPA58FXjvmNZ5HrAXeBTYD/zOONbZ1/9c4BHgvnGtEzgCfKWpc98Y17kS+BTw\nRPO7v3rc6gR+onkcT96eBd47bnX2jft4M8Y9zX7Hr875/sgGeaP3B3sYuAx4Bb0nmiuHMM5bgTfy\n/FC4C/jtpn0bcGfT3tjU8YqmrsOcOoLaB2xq2p8Brm3atwJ3N+2bgE/0/WF/nd4f08qT7Rep8WLg\nDU37AnrnYa4ctzqb/uc3/y5r/rO9ZRzrbLb5t8DHgV3j+Htv+n8TWDVr2TjWOQ28u+93f+E41tlX\n7znAt+m94BqrOpuxvgG8srn/SXpP1GNVZ9XSh8LPAvf33d8GbBvSWJfx/FA4AKxu2hcDB5r2dvqO\nWID7gTcDa4An+pZvAf6wr8/VfX8sTzftdwF/0LfNHwJbFljvvcA141wncD7wMPCT41gncAnwAPAO\nTh0pjGOd3wReM2vZWNVJLwC+cZrlY1XnrNo2A18YxzrpPTF/Dbio2cd9wM+PW51VteSfkjrKN7at\nrqqZpj0DrG7aa5s6Ztc0e/mTnKq1/Tmq6gTwbJLXzLGvOSW5jN6Rzd5xrDPJOUkebep5qKoeH8c6\ngQ8D7wee61s2jnUW8ECSLyZ5z5jWuR54OslHk3w5yR8lWTGGdfbbAuxo2mNVZ1V9F/hd4G/oXZX5\nvaraM251wtJ/dHYt8XinVb24HItaklwA/Dnwvqr6fv+6camzqp6rqjfQeyX+tiTvmLV+5HUm+SXg\nqap6BDjtddzjUGfj56rqjcA7gV9P8tb+lWNS5zLgTfSmI94E/IDekX1rTOoEIMly4JeBP5u9bhzq\nTPJjwG/Rm8FYC1yQ5Ff6+4xDnbD0ofAkvfm+ky7l+Qk2TDNJLgZIsgZ46kVquqSp6cmmPXv5yW1e\n1+xrGXBhVT1zmn3N+fMleQW9QPhYVd07rnWeVFXPAv+d3omucavzXwLXJfkmvVeL/yrJx8awTqrq\n282/TwN/Qe/zwsatzqPA0ap6uLn/KXohcWzM6jzpncCXmscUxu/x/Bngr6rqmeZV/KfpTaeP3+M5\n3zzdIG/0Xn18nV5aLmdIJ5qbsS7jhSeab2va23jhCZ3l9A6Zv86pEzp76V1xEV54QucP+ub0+k/o\nfIPeyZyLTrZfpL4Afwp8eNbycavztSfXAa8CPk/vjYljVeesmt/OqXMKY1UnvfMyr27aK4D/SW8u\nfKzqbPp/HtjQtKeaGseuzmabTwBbx/jv6KfpXWX4qmb/08Cvj1udVTWSS1LfSe+Ey2Fg+5DG2EFv\n3u5H9ObYbm4emAc4/aVfH2jqOQD8Qt/yk5d+HQb+U9/yVwI7OXXp12V9625ulh/q/096mhrfQm/u\n+1FOXU537RjW+VPAl5s6vwK8v+8/2tjUOavmt3Pq6qOxqpPeH/ijze2rNH8D41Zn0/en6V1Y8Nf0\nXtleOKZ1rgC+QxO2Y/x4/janLkmdpndl0djV6ZvXJEktv6NZktQyFCRJLUNBktQyFCRJLUNBktQy\nFCRJLUNBktQyFCRJrf8PRkUgCwEjnWAAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x116711a50>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.hist(hp.SalePrice, color='blue')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEKdJREFUeJzt3VusXFd9x/HvLzeFkBRjUZybUdKSFCwhCFVMJYo6Vds0\noCrOS5OgCqUQ8ZJeUNVCbJAaV1VpSKXyUuWJ0lqoSWUVEgVRiJ00o9KXREExCRhjW8UtTskJodBS\nEYrd/Psw28lwOJc5PnPzWd+PZHnNmrVn/70085s9a89sp6qQJLXhrFkXIEmaHkNfkhpi6EtSQwx9\nSWqIoS9JDTH0JakhI4V+kmNJnkryZJLHu77NSfYnOZxkX5JNQ+N3JTmS5FCS6yZVvCRpbUY90i+g\nV1XXVNX2rm8nsL+qrgYe6W6TZBtwM7ANuB64J4mfKCRpDqwljLPo9g3Anq69B7ixa+8A7quqE1V1\nDDgKbEeSNHNrOdJ/OMkTSd7f9W2pqoWuvQBs6dqXAseHtj0OXLbuSiVJ63bOiOPeXlXfSvLTwP4k\nh4bvrKpKstL1HLzWgyTNgZFCv6q+1f397ST3M1iuWUhycVU9m+QS4Llu+DPA1qHNL+/6XrLKG4Qk\naRlVtXipfU1WXd5JckGSi7r2K4HrgKeBB4Fbu2G3Ag907QeBW5Kcl+RK4Crg8SUKn/s/d95558xr\nsE7rPFNrtM7x/xmHUY70twD3Jzk1/u+qal+SJ4C9SW4DjgE3dWF+MMle4CBwEri9xlWtJGldVg39\nqvoG8JYl+v8T+NVltvko8NF1VydJGiu/P7+CXq836xJGYp3jdSbUeSbUCNY5jzKLlZckrvhI0hol\noSZ9IleStHEY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakho15lU5q57lIgE+dvSLSRGfo6w0w6\nkKfzxiLNiss7ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtS\nQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE\n0Jekhhj6ktSQkUI/ydlJnkzy2e725iT7kxxOsi/JpqGxu5IcSXIoyXWTKlyStHajHul/ADgIVHd7\nJ7C/qq4GHuluk2QbcDOwDbgeuCeJnyYkaU6sGshJLgfeBXwCSNd9A7Cna+8BbuzaO4D7qupEVR0D\njgLbx1mwJOn0jXIU/nHgg8CLQ31bqmqhay8AW7r2pcDxoXHHgcvWW6QkaTzOWenOJL8BPFdVTybp\nLTWmqipJLXXfqSFLde7evfuldq/Xo9db8uGlqUuy+qAxqFrpZSNBv9+n3++P9TGz0hMvyUeB9wAn\ngfOBnwI+A1wL9Krq2SSXAI9W1RuS7ASoqru67b8A3FlVjy163PIJr7UahPGknzfT2MdgP74GtFZJ\nqKp1HZWsuLxTVR+uqq1VdSVwC/BPVfUe4EHg1m7YrcADXftB4JYk5yW5ErgKeHw9BUqSxmfF5Z0l\nnDo0uQvYm+Q24BhwE0BVHUyyl8E3fU4Ct3tIL0nzY8XlnYnt1OUdnQaXd9S6iS/vSJI2FkNfkhpi\n6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+\nJDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtS\nQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkNWDP0k5yd5LMmBJAeT/HnXvznJ\n/iSHk+xLsmlom11JjiQ5lOS6Sf8DJEmjS1WtPCC5oKp+kOQc4F+APwJuAJ6vqruT3AG8uqp2JtkG\n3AtcC1wGPAxcXVUvLnrMWm2/0mJJgEk/b6axj8F+fA1orZJQVVnPY6y6vFNVP+ia5wFnA99lEPp7\nuv49wI1dewdwX1WdqKpjwFFg+3oKlCSNz6qhn+SsJAeABeDRqvoqsKWqFrohC8CWrn0pcHxo8+MM\njvglSXPgnNUGdEszb0nyKuChJL+86P5KstLnVD/DStKcWDX0T6mq/0ryOeDngYUkF1fVs0kuAZ7r\nhj0DbB3a7PKu7yfs3r37pXav16PX662tckna4Pr9Pv1+f6yPueKJ3CSvAU5W1feSvAJ4CPgT4NeB\n71TVx5LsBDYtOpG7nZdP5L5+8VlbT+TqdHgiV60bx4nc1Y70LwH2JDmLwfr/p6rqkSRPAnuT3AYc\nA24CqKqDSfYCB4GTwO2muyTNj1W/sjmRnXqkr9Pgkb5aN5WvbEqSNg5DX5IaYuhLUkMMfUlqiKEv\nSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLU\nEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x\n9CWpIYa+JDXE0Jekhpwz6wKkViWZyn6qair70ZnB0JdmZhphPJ03Fp05XN6RpIYY+pLUkFVDP8nW\nJI8m+WqSryT5/a5/c5L9SQ4n2Zdk09A2u5IcSXIoyXWT/AdIkkaX1U7yJLkYuLiqDiS5EPgScCPw\nXuD5qro7yR3Aq6tqZ5JtwL3AtcBlwMPA1VX14tBjlieXtFaDE5+Tft5MYx/T3Y+vtY0jCVW1rhM1\nqx7pV9WzVXWga/8P8DUGYX4DsKcbtofBGwHADuC+qjpRVceAo8D29RQpSRqPNa3pJ7kCuAZ4DNhS\nVQvdXQvAlq59KXB8aLPjDN4kJEkzNnLod0s7nwY+UFXfH76vW6tZ6TOkny8laQ6M9D39JOcyCPxP\nVdUDXfdCkour6tkklwDPdf3PAFuHNr+86/sxu3fvfqnd6/Xo9XprLl6SNrJ+v0+/3x/rY45yIjcM\n1uy/U1V/MNR/d9f3sSQ7gU2LTuRu5+UTua8fPnPriVydDk/knt5+fK1tHOM4kTtK6P8i8M/AU7z8\nLN0FPA7sBV4HHANuqqrvddt8GHgfcJLBctBDix7T0NeaGfqntx9faxvHVEJ/Egx9nQ5D//T242tt\n45jKVzYlSRuHoS9JDTH0JakhXlpZ6zat68JLWj9DX2PiteGlM4HLO5LUEENfkhpi6EtSQwx9SWqI\noS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6\nktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9J\nDTH0Jakhhr4kNWTV0E/yySQLSZ4e6tucZH+Sw0n2Jdk0dN+uJEeSHEpy3aQKlySt3ShH+n8DXL+o\nbyewv6quBh7pbpNkG3AzsK3b5p4kfpqQpDmxaiBX1ReB7y7qvgHY07X3ADd27R3AfVV1oqqOAUeB\n7eMpVZK0Xqd7FL6lqha69gKwpWtfChwfGnccuOw09yFJGrNz1vsAVVVJaqUhS3Xu3r37pXav16PX\n6623FEnaUPr9Pv1+f6yPmaqV8roblFwBfLaq3tTdPgT0qurZJJcAj1bVG5LsBKiqu7pxXwDurKrH\nFj1ejbJfnRmSsMx7+7j3NIX9bKR/y2A/vtY2jiRUVdbzGKe7vPMgcGvXvhV4YKj/liTnJbkSuAp4\nfD0FSpLGZ9XlnST3Ab8EvCbJN4E/Bu4C9ia5DTgG3ARQVQeT7AUOAieB2z2kl6T5MdLyzth36vLO\nhuLyznzvx9faxjGO5Z11n8iVNN8Gb8qT55vLmcHQlza8aX1y0ZnAX8tKUkMMfUlqiKEvSQ0x9CWp\nIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIZ4aeUNbFrXUZd0\n5jD0NzyvpS7pZS7vSFJDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqI\noS9JDTH0Jakhhr4kNcSrbEoai2lcyrtqGleN3dgMfUljMulA9hLe4+DyjiQ1xCP9GfB/tJI0KxM5\n0k9yfZJDSY4kuWMS+zjz1RT+SNKPG3voJzkb+CvgemAb8O4kbxz3fqah3+/PuoQR9WddwIj6sy5g\nRP1ZFzCC/qwLGFF/1gWM5Mx5ra/fJJZ3tgNHq+oYQJK/B3YAX5vAvsbqzW9+G0eOHH7p9okTL3Du\nua+YYUWj6gO9Gdcwij7WOS595r9GGHed01oa3cjfEppE6F8GfHPo9nHgbRPYz9g9//x/88ILnwd+\nruu5i5Mnd455L88AbxrzY0qtmFQY7+7+wEb/ltAkQv+MfYs8+2y48MKdnHXWRQD88Idf5/zzD451\nH1Uv8P3vj/UhJWlkGffHmCS/AOyuquu727uAF6vqY0Njztg3Bkmapapa10eRSYT+OcDXgV8B/gN4\nHHh3Vc39mr4kbXRjX96pqpNJfhd4CDgb+GsDX5Lmw9iP9CVJ82us39NP8skkC0meHurbnGR/ksNJ\n9iXZtMy2U/tB1zrrPJbkqSRPJnl8BnX+ZpKvJvm/JG9dYdtZz+eodU5lPpep8S+SfC3Jl5N8Jsmr\nltl21nM5ap2zfm7+aVfjgSSPJNm6zLazns9R65zpfA7d94dJXkyyeZlt1zafVTW2P8A7gGuAp4f6\n7gY+1LXvAO5aYruzgaPAFcC5wAHgjeOsbRx1dvd9A9g8qdpGqPMNwNXAo8Bbl9luHuZz1TqnOZ/L\n1PhrwFld+645fm6uWuecPDcvGmr/HvCJOZ3PVeuch/ns+rcCX1iultOZz7Ee6VfVF4HvLuq+AdjT\ntfcANy6x6Us/6KqqE8CpH3RNxDrqPGUqX+Rdqs6qOlRVh5fZ5JSZz+eIdZ4y8flcpsb9VfVid/Mx\n4PIlNp2HuRylzlNm+dwc/jLyhcDzS2w6D/M5Sp2nzGw+O38JfGiFTdc8n9O4yuaWqlro2gvAliXG\nLPWDrssmXdgio9QJg98hPJzkiSTvn05pazYP8zmqeZnP9wH/uET/vM3lcnXCHMxlkj9L8u/ArQw+\nlSw2F/M5Qp0w4/lMsgM4XlVPrTBszfM51Usr1+DzyFJnjufqbPIKdQK8vaquAd4J/E6Sd0yvspHN\n1XyuYubzmeQjwI+q6t4l7p6buVylTpiDuayqj1TV64C/BT6+1JDpVrS0EeqEGc5nkguADwN3Dncv\nMXTN8zmN0F9IcjFAkkuA55YY8wyDtatTtjJ4x5qmUeqkqr7V/f1t4H4GH6/mzTzM50hmPZ9Jfht4\nF/BbywyZi7kcoc6Zz+Ui9wLXLtE/F/M5ZLk6Zz2fP8tgnf7LSb7BYEnvS0leu2jcmudzGqH/IIOP\nUHR/P7DEmCeAq5JckeQ84OZuu2latc4kFyS5qGu/ErgO+Imz7VO03HrjPMznsCXrnPV8Jrke+CCw\no6p+uMywmc/lKHXOei67/V41dHMH8OQSw+ZhPletc9bzWVVPV9WWqrqyqq5kEORvrarFB6Nrn88x\nn4G+j8GvcH/EYJ3pvcBm4GHgMLAP2NSNvRT43NC272TwS96jwK4Jnyk/rTqBn2FwdvwA8JUZ1Pk+\nBieYvwm8ADwLfH4O53OkOqc5n8vUeAT4NwYv+ieBe+Z0Lletc06em//AIBgPAJ8GXjun87lqnTOc\nz//t6nzvovv/le7bO+udT3+cJUkN8f/IlaSGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+\nJDXk/wEPqp9RVjqaLAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11722bed0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.hist(np.log(hp.SalePrice), color='blue')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"#hp.LotFrontage.unique()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__Data Treatment__"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"# Removing Nulls\n",
"# df = hp[pd.notnull(hp['GarageQual'])]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__Feature Engineering__"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"# Has Pool\n",
"hp['has_pool'] = np.where(hp['PoolArea'] > 0, 1,0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__Imputation__"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"#filling NA's with the mean of the column:\n",
"hp['LotFrontage'] = hp['LotFrontage'].fillna(hp['LotFrontage'].mean())"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"# # Watch out for this!!!\n",
"# a = DataFrame({'x': [1,2]})\n",
"# b = a[0:1]\n",
"# b.x = -1\n",
"# print(b)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"df = hp.copy()"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"# Isolate Response variable and Drop unwanted columns\n",
"X_keep = ['LotArea','has_pool','YrSold','OverallQual','OverallCond']\n",
"response_variable = 'SalePrice'\n",
"\n",
"X = df[X_keep]\n",
"y = df[response_variable]"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"y = np.log(y)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=28)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"AdaBoostRegressor(base_estimator=DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,\n",
" max_leaf_nodes=None, min_impurity_split=1e-07,\n",
" min_samples_leaf=1, min_samples_split=2,\n",
" min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
" splitter='best'),\n",
" learning_rate=1.0, loss='linear', n_estimators=300,\n",
" random_state=28)"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"adr = AdaBoostRegressor(DecisionTreeRegressor(max_depth=2), n_estimators=300, random_state=28)\n",
"adr.fit(X_train, y_train)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MSE: 0.0558\n",
"MAE: 0.1765\n"
]
}
],
"source": [
"# Predict\n",
"y_pred = adr.predict(X_test)\n",
"mse = metrics.mean_squared_error(y_pred, y_test)\n",
"mae = metrics.mean_absolute_error(y_pred, y_test)\n",
"print(\"MSE: %.4f\" % mse)\n",
"print(\"MAE: %.4f\" % mae)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/felipeformentiferreira/Library/Python/2.7/lib/python/site-packages/matplotlib/collections.py:590: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n",
" if self._edgecolors == str('face'):\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEZCAYAAACAZ8KHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXuYXWV56H/vzCRkEhKSEBhuwUBFklgURibeipmj1Qbw\nEa31QEoUFAHbqnloar1gD6EYKrVp5NRWKYLGEy5ahVYMIAEZQKV1EKJoCKAQSEKY3MllJmEu7/nj\nXV/WmjVr79l7Zu+Zvfe8v+dZz6z1rdu3Vna+d33vVVQVx3EcxymUutHugOM4jlNduOBwHMdxisIF\nh+M4jlMULjgcx3GconDB4TiO4xSFCw7HcRynKFxwOCVFRM4SkfWj3Y9aQER+IyLvGO1+FIqI9InI\nyQUc1yoiG0eiT055cMHhDAkR2SAi70q3q+ojqjp7NPqURkSWiki3iOwVkd0i8t8ictZo96tQVPUP\nVfXhUl9XRNqiQf4NqfY7o/aqEVbO6OCCwxkqGi0VgYjUZzQrcJuqTgaOBO4Hvl+Ge4uISKmvW0YU\neBr4SGgQkSOBtwJbR6tTTvXggsMpKWk1RDQzWSIiv4q++m8XkcMS+98rImtFZJeI/ExETkvs+5yI\n/E5E9ojIb0Xk/Yl9F0fH/7OIbAeuyupOtKCqvcCtwFEiMiO6xhEicpOIvCQim0TkGhGpi/bVichy\nEdkmIs+JyCejr/Gwv01EviQiPwP2AyeJyGwRWSMiO0RkvYh8KNHfc6Jn2BPda0nUPkNEfhQ9/w4R\neThxzqFZnYgcJiJfFZHN0bJCRMYn3vkmEflrEemInufiQf6pbgXOTwi8hcAdQHfi/jnvGe3/TOLd\nfazfi7dz/0lEXhCRl0Xk6yIyYZA+OVWCCw6n3CjwIeBPgJOANwAXA4jIGcBNwKXAdOAG4IciMi46\n93fAH6nqFOBqYJWINCWuPQ/4PXA0cG2+TkQD3keA36vq9qj528CrwB8AZwDvAT4e7bsMWAC8EWgG\n3s/AGdai6PjDgR3AGmAVcBRwAfBvIhLUdjcBl0XP8nrgJ1H7EmAjMCN6js8nrp+c1V0ZPe8bo2Ue\n8MXEsU3AFOA44BLgX0XkiDyv5CVgHfbvAvBh4DupY3LeU0QWRH3/Y+B10d8kXwZeG533WuB44P/k\n6Y9TTaiqL74UvQDPA+/MaG8FNqaO+/PE9nXA16P1rwN/nzp/PfCOHPd8AnhftH4x8MIgfVwKHAR2\nAT3AduCUaF8TcACYkDh+IfCTaP0nwKWJfe8C+oC6aPtBYGli//nAw6n73wD8n2j9BUwYTUkdczXw\nn8Af5HvHmBBdkNj3HuD5xDvvDH2L2jqAeTney4OYcLkQm3nMBp6O9m0M73+Qe94MXJvYd0r0fk7G\nZnn7gJMT+98KPJf1G/Gl+hafcTgjwcuJ9S7sCx3gNcCSSE2zS0R2AScAxwKIyEdE5InEvj/EbBWB\nQjxzvquq0zBB8RvgU4l7jwO2JK7/DWy2QNSH5PU3ZVw7uf81wJtTz/Ln0X0BPgicA2yI1Fxvidq/\ngg3Q94nI70Xkszme4zhM+ARejNoCO1S1L7HdSfyes1BMNfVO4K8YONsY7J7p9/NiYv0oYCLwy8S7\nuAebVTk1QMNod8AZkwT1y4vAMlUdoGYSkdcA/44NbI+qqorIE0Q2i9R18t0n2Dh2iMhlwK9FZAU2\n6B0EjkwNuIEtwMzE9syMY5L3fxF4SFXfk9kR1ceA90dG/E8B3wNOVNV9wN8AfyMirwd+IiK/UNUH\nU5d4CZgFPBVtnxi1DRlV7RKRe4BPYOq6NFn33Bytb4m2SewLbMc+EOaq6pbh9NGpTHzG4QyH8SIy\nIbFkeTZlEQb/G4FPiMg8MSaJyLkicjgwCRuYtwN1IvJRbMZRDP08nVT1GeAu4G+jAe0+4J9FZHJk\nDP8DiV1RvwcsFpHjRGQq8FkGCqrk9X8EvE5EFonIuGhpiQzm40TkQhE5Qs1IvxfohUPOAa+NjNR7\novYsQXYb8MXImD4Dsxf8vyLfRxZfAOar6osZ+7LuuSra9z3gYhGZIyITSTgnRIL4RuCrInJU9JzH\ni0imUHWqDxccznC4G1OJhOUqBnfTPbRfVX+JGca/BuwEniVyEVXVdcBy4FFM1fWHwE+zrlPIvRJ8\nBfiIiBwd3Ws8ZiTeCfwHcEx03I2YYPk18EtgNdCbmp0cunY0c3gPZhTfjH2R/0N0fTBD+vMi8gpm\n67gwan8tZlTfC/wc+FdVfSjjWb4EPBb159fR+pey+lIMqrpFVX+eY3fOe6rqvcBXMVvQM8ADqT58\nFlPB/Xf0zGswI/qw+utUBqJann8/EbkZOBfYqqqnpfYtwf4Dz1DVnRnnLsB+lPXAN1X1urJ00nEK\nRETOxoz6s0a7L44z2pRzxvEtzJ2xHyIyE3g3/Y1uyf312BfoAmAusFBE5pSxn44zgEj1do6INIjI\n8dhs6o7R7pfjVAJlExyq+gjmBpnmn4G/zXPqPOB3qrpBVbuB24HzytBFx8mHYO68O4HHgd/icQiO\nA4ywV5WInAdsUtVfS+4MDccz0A3yzeXum+MkUdUu7CPGcZwUIyY4Is+LL2BqqkPNGYe60cxxHKeC\nGckZxx9gPuG/imYbJ2ABQvNUNZlYbTMD/eezgq8QERcyjuM4Q0BVh5yYc8TccVX1SVVtUtWTVPUk\nTBg0p4QGmMvfKSIyK8ovdD7wwzzX9UWVq666atT7UCmLvwt/F/4u8i/DpWyCQ0Ruw/zSXyciG6MA\nriSaOPY4EVkNoKo9wCeBH2P+9d9V1adwHMdxKoKyqapUdeEg+09OrL+ExXyE7Xuw3DaO4zhOheGR\n4zVCa2vraHehYvB3EePvIsbfRekoW+T4SCAiWs39dxzHGQ1EBK0G47jjOI5TG7jgcBzHcYrCBYfj\nOI5TFC44HMdxnKJwweE4juMUhQsOx3EcpyhccDiO4zhF4YLDcRzHKQoXHI7jOE5RuOBwHMdxisIF\nh+M4jlMULjgcx3GconDB4TiO4xSFCw7HcRynKFxwOI7jOEXhgsNxHMcpChccjuM4TlGUrea4iNyM\n1RHfqqqnRW3XAO8DFNgBXKyqGzPO3QDsAXqBblWdV65+Oo7jlIP2dli1ytYXLYKWltHtTykpW+lY\nETkL2Ad8JyE4Jqvq3mj9U8AbVfXjGec+D7xJVXcOcg8vHes4TsXR3g6XXAISFWdVhZtuqhzhUbGl\nY1X1EWBXqm1vYvNwYHueSwz5oRzHcUaTVatMaEyfbotIPPuoBcqmqsqFiCwDPgx0Am/JcZgC94tI\nL3CDqt44Uv1zHMdx8jPixnFVvVJVTwS+DazIcdjbVfUM4GzgryK1l+M4TlWwaJGpp3butEXV2mqF\nEZ9xJLgVuDtrh6puif5uE5E7gXnAI1nHLl269NB6a2srra2tpe6n4zhOUbS0mE2jUozjbW1ttLW1\nlex6ZTOOA4jILOCuhHH8FFV9Nlr/FDBPVT+cOmciUK+qe0VkEnAfcLWq3pdxfTeOO47jFMlwjePl\ndMe9DZgPzBCRjcBVwDkicirmZvt74C+iY48DblTVc4FjgDvE3BEagFuyhIbjOI4zOpR1xlFufMbh\nOI5TPBXrjus4juPUJqNpHHccx6lZPHK8QnFVleM4lYhHjjuO4zhFUeuR4y44HMdxnKJwweE4jlNi\naj1y3G0cjuM4ZaCSjePDtXG44HAcxxljVGzkuOM4zlgna9ZRyTORQvEZh+M4ThnIcsldsgSWLx99\nN113x3Ucx6lAslxyV6yoDTddFxyO4zhOUbjgcBzHKQNZLrlXXFEbbrpu43AcxykTlWocd3fcKu6/\n4zjOaODGccdxHGdEccHhOI7jFIULDsdxHKcoPHLccRynTFSCIbwclM04LiI3A+cCW1X1tKjtGuB9\ngAI7gItVdWPGuQuArwL1wDdV9boc93DjuOM4FUklF3OqZOP4t4AFqbZ/VNU3qurpwH8CV6VPEpF6\n4GvRuXOBhSIyp4z9dBzHKTm1XMypbKoqVX1ERGal2vYmNg8HtmecOg/4napuABCR24HzgKfK0lHH\ncZwy0dkJO3bYemPj6PallIy4jUNElgEfBjqBt2QccjyQVF9tAt48Al1zHMfJSzE2i+ZmuOGGgW21\nwIgLDlW9ErhSRD4HrAA+mj6kmOstXbr00Hprayutra3D7KHjOM5A0jaLBx/Mb7N4/HGYORO6umy7\nsdHaLrpoZPqbpK2tjba2tpJdbzS9qm4F7s5o3wzMTGzPxGYdmSQFh+M4TrlI2izAck2tWlUZxu7B\nSH9UX3311cO63ojGcYjIKYnN84AnMg57DDhFRGaJyHjgfOCHI9E/x3GcUtHcDBs3wvbttmzcWDuq\nqrIJDhG5Dfg5cKqIbBSRjwH/ICJPishaoBVYEh17nIisBlDVHuCTwI+BdcB3VdUN447jjCqLFsH+\n/fDkk7bs358/s21QVc2YYcvMmdZWC5TTq2phRvPNOY59CYv5CNv3APeUqWuO4zgjwsSJcMIJtr5z\n5+j2pZR45LjjOMOiVqOj06xaBZMm2cwBBrdxLFoEq1fD5s22PXVqddbeyMIFh+M4Q6ZYT6NaJJ/g\nPHAA9u2z9QkTRr5v5cKTHDqOM2RqOTo6TVZFv+ZmE5xtbbZccokJEoDly80o3tBgy/bt1lYLuOBw\nHMcpgJYWm021ttpy001m7M4lONevt7/jxtmSbKt2XFXlOM6QWbTI1FPB8FutNbQLpaWlvyoqPbvq\n7ISHHoLFi+Goo+Dpp6G72/aJwOzZI9fXcuKlYx3HGRaVYBwfbh+Gen7SxtPZabEaM2eaN9X+/Wbj\nOHDAjp06FW67rTLsP15zvIr77zhO4eQa3IebvrwU569aZTON/fv7u9/OmQNNTQP7PNoMV3C4qspx\nnGExEjOOfN5bWalAli8vfMAuJpVI1rOGZfFiM5AnaWqC668f8mNXLC44HMcZMiPljlvM4N7ZCfff\nD8cfX1yf9uyBjg5TLXV0DNw/2LMWau+pBNXecHGvKsdxhkwp3XHb2+2rffHi2KW1ENJusrt2wbRp\nhfcppBJZt85cZvftg8ceG9iHwZ41y+sqLRSC8Mly360mfMbhOM6oM5yv+TBgh0G8owOeKiK7XUsL\nnHkm7N5tQXrHHAOvvAKXXgrz5xc3K0h7XaVZtcrSrCdTrVdLht0kLjgcxxkypXLHHUwVlRYO6cE8\nOWC3t8PChcWl+mhqMtXW9Omwd695Rx1+uM0KghArxbN2dMCLL0Jfn23X1WWrxSodFxyO4wyZwQb0\nYhiszOpgX/PDYdEiuOMOi7vo6TEh9prXwJQpsRC7/vrhP+uOHfDqq/HMqqcnfuZqwgWH4zjDohQD\neinLrIbZS8gNFWwR+fq4bh1s2QK9vXFbZ6cJjiTDfdZt22D8+IFt1YYLDsdxhkUpvIRKWWa1owM2\nbYL6etvesWNwddCyZSY06upMBaUKzz1ng3wpo+Fnz7a0Iw3RyNvTU53R5O5V5TjOkCmVl1BHh3lD\ngRmnJ07MvlehXlchLrinx9xr770XVq7MfXy4d11dLHDq6nJ7Rw2VJUssQHDCBFtOOMHaqg2fcTiO\nM2RKUYe7vd3cX/fvN1fY7dttQE1+5RcTL9LUZLOXHTvsmmAG78svt/WsWcwZZ8CaNf1VVWedVfrg\nvZYWSztS7XEcLjgcxxkxstRaoUDS3LlxAN6ZZw5MJliogAreT52dtl1XZzOY7m5YsSJbcCxbZi68\nwd5w1FHWNtRnykc5jfwjhQsOx3GGTDEuqrlmDWCDfLBvTJsWpwsJdHSYe+2OHQP3hWsnB++bboJz\nzjFhMXGi2RRCltosWlrMqypco7nZ1letyi8Mxmohq7IJDhG5GasjvlVVT4vavgK8F3gV+D3wUVV9\nJePcDcAeoBfoVtV55eqn44xlhmvYbmkxHf2KFbb913+d+xq5Zg3NzfD1r8dqovr6/h5VQZW1Z4/Z\nIl5+2ewgQUCFuI3du2179WpTB/3TP5l6qrs7FhpXXJH/WVpacl8v67lKoaqrRsppHP8WsCDVdh/w\nelV9I/AM8Pkc5yrQqqpnuNBwnPJQCsN2e7slFFQ1e8KSJXDBBcVd5557bPAVsev09vZP5REG5/r6\n2Otp7954//LlNhsJKcw3b7a2iy4yF9/Zs2254Yb8XlorV8Lpp8O73w0bNsTX27Spdir3lYqyCQ5V\nfQTYlWpbo6pRzCT/A5yQ5xJDTvnrOM7glCLPVLjGuHEWB7FvnyUYTAuh9vZY3bRpU1x6ddGiuCqe\nql2rrw8eeaT/+bt22b66OlvSlfZU40p7qvE1L7oI1q61ZTChcfnldt4rr5jw6usbvHJfVjnZtFF/\nKPm3Kp3RdMf9GHB3jn0K3C8ij4nIpSPYJ8cZUWphYOnstIH14MF4cH/pJcv11N4ez2yeesrsF7t2\nWZ2KYAuYPdtUSX19/VNxJFVoPT1mA+nutkG9szOOzQhxEEmVVLGxEcuWxecHe8WBA/F2ruvlS2xY\nKwkNsxjUxiEidcCFwEmq+vciciJwjKr+Yqg3FZErgVdV9dYch7xdVbeIyFHAGhFZH81gBrB06dJD\n662trbS2tg61W44zooy2YbUUuZdCxHcY+MOXuojNLC65xLylkjObiRPNwB2e89RT47iLQFdXnKgw\n2B4efNCu05AatZYsMRtIsElMnRrHRmTZcJJtzc2mKvv972OhFQjR58nrFUMl2T/a2tpoSxcLGQaF\nGMf/DegD3gn8PbAvajtzKDcUkYuBc4B35TpGVbdEf7eJyJ3APGBQweE41cRoDyylyDP1+ONwxBFx\nviVVmx00Nlqupz174IEHrL2hYWAKDzDDcxYbNsTrc+fadjKyPHhXZcVGgNla7r3X1hsazMj9d39n\n9opQ6vUb37D9aaEBlvTwAx+oDa+q9Ef11VdfPazrFaKqerOq/iXQBaCqO4FxQ7mZiCwAPgOcp6oH\nchwzUUQmR+uTgPcATw7lfo7jlJd162Dr1v6Bc3V1NosAm3V0d5vtY906207PbHbtYlAWLTJhceSR\ntjQ25p4drVtng/m995q94pVXbDaycaOppIKw3rOnv3orIGJxJR/4gAUAFlo9MG0nGsz+Uc0UMuN4\nVUTqw0akPsqQz/0RkduA+cAMEdkIXIV5UY3H1E8Aj6rqX4rIccCNqnoucAxwR7S/AbhFVe8r7rEc\np/IpVUryoVKM22kunn56YJuIzTo2bzaBcthhthw8aAPyjTf2v8dJJ1m0eJpZs+L1fLOj9Ff/LbeY\nLeXgwfj8oEbbutVmQmBeYGEm1Nsbq8vGj+/v7jtUSpk5uNIoRHD8C3AncLSIXAv8GfDFwU5S1YUZ\nzTfnOPYlLOYDVX0OOL2AfjlOVTPaA0twYw25mYIb6+23F36NPXsGto0fb4bie+6xr/0gGJNG5sWL\n7e+iRXDyyWajSNo5DjvMDOhJckVcp1V+mzdbBHhyJhGuPWVKPAsItpKGBlsOHrT9CxaYTaOQf4vB\nhH8tRIlnMajgUNVVIvJLYpvEeapaRH0tx3FyMZoDS9KNFWyWkMvtNBeHHTawbdIkU/GsWwfPPhvP\nBFTh+ecH2gTmzrUUH1u3xtfo7h56WvXGRnMNzrJbgN0PTDD99KfmPdXTYwKvGKEBxQVA1hKD2jhE\n5C3AZlX9mqp+DdgsIm8uf9ccxxkOg7n6zp5tA2awA5QqxXd3t91z7Vr7kg/xFePGWarytE0ATFU0\nfnz89X/ccWZ4L+R50raEri449tiBdS/ABMr999sMZ8kSuPNO+OM/tnsefbR5chXjNpsMgFS19Vpx\nuc1HIaqqbwBnJLb3Z7Q5jlNBFOLtc+qp/Y3avb2xUTvfdZOqtZBIMMnu3Ra30N1twmjCBFOH9faa\n7SFNU5MN3j/6kamL0u62gz1Prprju3ZZtb2kCqy3t380+O239y8bC8V5t422Z9xoUVAAoGr86lW1\nF6jPc7jjOMNkuIGBaW+fri4LyEteL8sNNpdrbOhTOqAtKXgCquaeG2YQvb2xKmjhwmxPo8MPN2N1\nqJ+xaZPFT+R6nnSUe0uLqceuv95mEsHoDbGwARNgg0WDO4NTiOB4XkQ+LSLjRGS8iCwGnit3xxxn\nLJEUFCtX2gB7yy22LFw4PPXH3r3mirppU/8I5qRNIZDVFsgavOtzfELu3h3PRkRMMBx9tKmGlizp\nH2kN8J3vDLxGPiGWjzADWbAAJk82g3dQW4UsuUlD/XDcZmvZ5TYfhaiqPgH8X2JPqgeAy8rWI8cZ\nY6TVMCtX2gwhfBkPxdtp0SJzr9282WIoVM0NdcqUWJ0yZYrZNpJkBejlY+JEu34u+vosrmPCBAsU\n7O4220WyQNLixdkzl2R8R7Guyy0t9r6Caq2jIzaEQ/9o8OF4t422Z9xoUYhXVQdw/gj0xXFqmlwp\nzFetMkERoqI7O20gDeVTh+LtVAhve5vVoOjpse2GBmvLRdbgHWJA0kydakKiq8uuv3u3LdOmwUMP\nmbAI76CjI9sD6qST4vWhDtBJr7V8KeSH491Wqy63+cgpOETks6p6nYj8S8ZuVdVPl7FfjlO15MqP\nlMu429FhaqSg9gnBaAcPxjmgjj66uD6EqnozZ5qq6je/gRdeMCNw8mv9Zz/rX/UuX06mLNfTiy/O\nPvbII22209BgMw6wZ9myxWYfbW39CznV1Q0UHiefPPD+wxmgx+IAXy7yzTjWRX8fS7ULlr3WcZwU\nuQTEYN43wYAMdlyItAYbfJ97zq49lIFv8mQTIJMmwfz5/YXZYYeZ/QGyYzLSz3bNNfEs45prch/b\n2mozi/37Tf3V0WHR4YcfbvXEk++gqcn6GFRqwSaSVenPqQxyCg5VvStKNfIGVR1CbkjHGXukBcSm\nTebNBKaCCu1pQhEjsJnH5Mn2Bd7QYAPtYYfld/NMz3KSNg4w1VE61UdyVgKDu5IuXw4vvhj3Mytq\nPHD99bEQ7emxGciBA9nuuKGvBw/G154xY2wYmauVvDYOVe0VkbeLiCRdch3HGZw9e0xwTJpkA+aL\nL5paqKHBBvLkwBhmGWCxB7t22Sykt9fOO/743PfJmuUUmga8szPObNvYmP/Yxx/PX7c7MGmS/U3b\nJZqbbZbyZJSyNLyDkN12+XKz5cyeXVz0tjPyFOJVtRb4LxH5DyCE+6iq3lG+bjlOdZI0IG/ebF/Q\nIVlfb695MYUqdoGmJrMvBC+i+noTHuGcnh5zk831BZ6lBluxov9sIsx8kqqq5mZLK540judL85H2\nwAp9TXtEvf718XraOJ2L4AXlVAeFCI4JwE6sHkcSFxyOkyJpQBYxgTB5MvzqV/EAq2ozjy98Adas\nGTiA9/XF9bV7e+06ycJHhdDZaQJky5Y4rcjkyXbv1avhzDNtBtHbGwsxVUtMmKvEajLbbCBLD5Fs\nS6rQOjri4kgQB/GF51q5Mja8X3FF/lKvzuiSV3BEKdT/FfidqhaQNd9xxjYhd5GIqWI2brSBMtgD\n6uvjtONr19rxy5bZtkhs6whqqkBIBZ7lsZV2k92/3wb5UG8icPCgLZs3m4F7377+gqO+Pr/b79FH\nD5x1ZLnRrl0b9zWpQnvhBRNoIRhv+/a4/OvKlXDZZbHwvCyKFHPhUZnkjBwXkY8Dv8WC/9aLyHkj\n1ivHqVKSaqMTToi9mYLnkmpcWrWx0QbWF1+0NtX4azyLlSvh7LMtmvzuu+MI8HTd6zPPtPtPnRq7\n+IbBOxRSmjAhHsBDre/u7uzEgIGFWYUSMgjCKsSn7NhhS09Pf0GTTAWybFmsngNbX7assPs5I0++\nlCNXAK9X1bcCb8WKMDmOMwg7d9qA3t5unkTz55uXUUNDPEjX15sRuKsrHkD7+uLiQml+/WtTge3b\nZ9fcssXODbOPZK6mpib7st+7t796LBkX0t2dO0FhLp5+2p4hzIyykhEmCfEpIfjvwIG4hvfUqSZY\ng8ttsO8k7T+FVAZ0Rod8guNVVd0Gh4orDeLl7TjO1Kk2WO7fb8umTfDoo2Y7SH7NNzbaYPr88/2/\ntHP5LgY1VEhTDrkH1uZmU5ElbRL19Xbtujr7+8orsVARsfZx4+Jo9SzWr4+PGzeuv4E/F8nnqauz\nfmSVfz39dOtHUNGJWJtTmeT7ZjhBRP4vFvAHcHxi2yPHHSeD//qvgZ5Ga9fGhujA3r3wy18OtBFk\neSkFpk2zmUZIVw7ZnlaPP25G+RDDAXbNIDxELCjvwAHrR6gl0dcH5+VRSB99tBn5k4WZsgiCranJ\nVHUhlUpjI7zpTfEsI5n249prrcZ3KCE7Y4a1OZVJPsHxGfpHiP8y2vbIccfJwc6dAwf+XLEPwdZR\nSITU5Mk28B57bDzTWL48t6fV3r3Z7VOmmNF8/Ph4QE9yzz1w9dXZ506fHqdJh4G1ugNBsASjfZjF\nqOaOz2hpscy5lZ4sMF++q7GElCuuT0RuxuqIb1XV06K2rwDvBV4Ffg98VFUHeIeLyALgq1jdj2+q\n6nU57uFxiRXIWP7PNWVK7kE7i0IFx8yZ8IMfFPZe29vhHe+IM8EGJkywsqnr1pnBPgT+pY/JEigA\nF1wA3/9+PEsKqqq0oJw8OfYiqyUX27SXmOrA4ljVgoigqjL4kdkUVMhpiHwLWJBquw8zuL8ReIYM\ng3uU5uRr0blzgYUiMid9nFOZZBX7GQulNANZBud8pA3MkuO/cvC26uiwHFD5SpS2tMAb3pC9r6fH\notBDudTB+pNkx454hhFchrPcccMMq9bKqg5WTGosUUgA4JBQ1UdEZFaqbU1i83+AD2acOg+LG9kA\nICK3A+cBT5Wlo05JGaulNAMTJphRvNBjg00jqK0mTLCZQnoWMnWqucNu2mTbTz9tdbNvuy373WbV\n1ejthTlzYnXRvHkDB/I5eT7Rnn56YFvWbKm7O551pn8Ly5dn2zic3FTiDL6cM47B+Bhwd0b78cDG\nxPamqM1xKp4LLshuz5pJNDT093wSMTVPyPWU5MUXzQurocFsHfX1tp3ri/eFFwbev66ufwT6EUcM\nPC+rLZCV1DDruerrbabZ0WEzsGeftWXnTrj//uqdjY5Gtb9KncHnq8eRVYcjMCyvKhG5EnP3vTXr\n2sVca+nz57gCAAAcV0lEQVTSpYfWW1tbaW1tHWq3nBJQbKW2aif9NZivGl6S4FabPL6vLzutB9gg\nHIIIC+lTugRsVnBh1gwiqy2QFTk+aZLZRJJ2jrq6OPBvY+ITsKfHjPvVOhsdjWp/pZrBt7W10dbW\nVrJ+5VNVBS8qiF1yA0O2SIvIxcA5wLtyHLIZmJnYnonNOjJJCg5n9Km1Upr51AQrV8ZZaKdNM4GZ\nK4AurdLp7e3/BZ80kk+bli2AXn3VjglBg+kMu4FVq2IDdfK+hx/e//isGUS+VOlXXmmJEoOQqK+H\nt74VnnrK3IRDe0+PqdRE+rvj7tuX2/BeLVRrMaj0R/XVuVznCiRfPY5vD+vKGUTeUp8B5qvqgRyH\nPQacEtlHXsLK1haY7MAZbZIDbXOzra9aVZ0CpL29f2zBD35gLqOhCFKI5G5osIHz2GPjmVYukh45\nya/05ACfK+1IU5MN7CECO196kq4u29/TY8u4cfBHf9T/36DYmuPBIyrpJQVw+eX9+x/iRcBccUPh\npk2bzJV4rMxGS0GlzuAHNY6LyNHA32IeTiFjv6pqOltu+rzbgPnADBHZCFyFeVGNB9aI/Q96VFX/\nUkSOA25U1XNVtUdEPgn8GHPHvUlV3TBeBSTdFTs74YYb7Itz4sT+pVKrgfZ2+OAHLYguDPYvvRRn\ntA3CMaicurttUMzllRRmFIO53vb02LtLBwI2NFiKko6OwdUWixZZPqu+PovXGD/ehFq6ot7s2f1V\nSaEtHxdd1N+ldvFi+zfesiV2/21osPs1N5vrbxj0Ghvhi1+0AMXQz2r5PYwWlTqDL8Sr6hbgu1j8\nxeXAxcC2wU5S1axZws05jn0Ji/kI2/cA9xTQN6eCSOpjQ4xAV5d9cVaTPjsIwBB5rRoP5CHzK1hb\ncL+tr7cv/FNPhV/8YuA1Cw03GjfO0nFMmWIG5Z4eU12tWGEDbsgmm4+WFvNe+vSnzWYyYUL2l+r0\n6SZUknEZuSoU5mPiRHvup56y+/X0xMF+MHDQq+ZYjtGgEtVjhQiOI1X1myLyaVV9CHhIRNJ1yB2n\nZggC8LDDYp18GFxD6dOpU/sboHt64P3vh5//fHj37u42IfvSS7ZdVxfbO4pRW8yda2lH8iUtbGqC\nE0/snxKk2DrfoU979sSp4ZMqtEoc9JzhU4jgCCnYXhaR92J2h4zKwc5YJzmwhTKkjY0DXRfTBmeo\nvKk4WA2Mp5+OVUzjx5uBGCx+Is1dd+UfqAulo6N/YF1fn6UYf+aZwtUWhdQTz0oJUqz+PBSu+pu/\nMSF38smxradaZphO8RQiOJaJyFRgCfAvwBQs5brj9COrxnRan51O27B6tf0NsQuVYAsJA2pPjw2q\nXV0WbX3NNbGaJSsz7dathWWMzcekSfZu0vmttmwZWiDY3r3w8stmf0iruUqhP09Gh0NcY92pbQYV\nHKp6V7S6G2gta2+cqietmkjrs9N+6cGOkO/LeKQJX9FLlpja5dhj4xnUBRfE6cXThDoVw2HPnvhe\nSfbvt8hxERNat9xiA3Yue8GiRSaUQ6S5iEWah8JPpSL8e86aZe+lp8eCD487rjK8f5zyUIhX1bdS\nTQqgqh8rS48cpwJ4/HGbZQQBt2mTeRAFz6GQ1jxJqOpXaBBgFqE+eBpVs3uEmUhdnQm2uXNzZ5s9\n80xTnU2YYLaLnp7+Qjk9+xvObG/yZPPI2rDBnCFuvNHVVLVMIRPr1cCPouUB4AigwGw8jtOfdNqG\nUA1uJNM4FEJHh305t7fDb38L27aZ0AgpP7JqZvT1mbF5OIRCS1mECO3eXgsGPHgwf5K9piYTfqec\nkh2fUYqkfcl/z+5um2m40Kh9ClFVfT+5LSK3Aj8rW4+cmiZLrw6VZRxvb4ef/CT+8t+/3wbVfNXx\nAuPGFZ4qPYtQezwXyX2dnfndc0cieKxS4wyc8jKU7LivA44qdUecsUOWi2YlDTbLl9sMI0mI9H71\n1fzZb8ePL1xohMSDQb0V6nino8rBjlGNVWWD2VOCIX3uXNtuaho4qJdKsLjL7dijEBvHPuLcVAp0\nAJ8tZ6ccp5QU640UPMHSvPpqdv2JJMW4444bZ4kDGxvNIytEeTc2mrH5hRfMEH7GGXDhhWbTSFbc\nO+aY7LiLrIJDWZX3fLbgDJVCVFUF5uR0nMqjlAbgIDSGo4oK50+YENsW5syx1BzJgf7aa7P7mEyq\n2NiYO8lhoRlVfbbgDIVCZhwPqOq7BmtznEok1yAa9kH/L+1ctQ6SwiKf0JhWQGjsuHGmojrmGLOj\nrF+fX6UUuOgiO67QGUK+GA7HGQ756nE0AhOBo0QkmcFmCl5YyaliOjqyZyFg7apx4sKAqg32+VRV\n6cJMaURsdtHRYQLmlVcsyeDMmZbnKZdKKUkhM4SRiuFIU4mV6pzykG/GcTmwGDgOq80R2IvVBHec\niifLAAy5ZyEiFoewZ4+lUx8/Pq5tMWGCuQ4n4ymSTJpksQy/+lV2X44/Hr79bVNLrVhhwY9HHGFu\ntl1dpnoqRfBjITEcpaaUKkGn8slXj+OrwFdF5FOqmq8aoFOljIUvxCwD8KpV9oWfRWenZfbdu9eE\nTAj0O+ooG9yPOMKW554bGOg3b57NGL773YHXra83QbV8eWzP6O62WUdjo81mtm8vnUopxHAkhWM5\nGeu15scahbjjqohMU9VdACIyDVioqv9W3q455WQ0vxBHWmBlqXey3FDXrbMaIqrmQQUmOLZts5nH\n5z4Xe029//3wpS/1T0l+4YW2nq6lEe7R2Wn2jDDDCGqtAwfsnOHmuUpSqKvtWPh4cEpPIT/VS4PQ\nAIjWLytfl5yRoBRRw0OhvR3+9E9tgL7hBlsPBumVK+H0021ZubJ8fQizkNZWW4LAfPxxszeEID6w\nwTxEi+/eDddfb8ujjw7MYHvLLfYOs2qDq5o9Y/x4sz3s3h2ru0LcxmCuvqV4xiTh46GtzZZLLsnt\nHDAY6YwAlZIBwCkPhcw46kSkTlX7AESkHhhX3m45tUDW1+yVV8b1qMHWr7zSvtYvvzw+N6yXq+hP\nLiPzxIlmxzh40AbzUOEvXdnviSfsb2jv6bG2OXOsjkea+vq4PkYu76yenrgAVikYzJBeSvWSx4SM\nLQoRHD8GbheRGwDBjOb3lrVXTtkpdzqKXKqwMODW19vfMOCGokghM2xXlxmQR7JaXHgnjY39Yyp6\nekyYJN/PtGlmk0jOEqZNs2NuuGHgtXt77fiZM02A7NoVC44wq4GBEevVhMeEjB0KUVV9FngQ+Avg\nE8D9WA1yp4opRJUxHHKpwkKcQ1/fwKp6o014J+ecYyq0M880Q/iJJ8Lf/V3/93PllTYTCYwbZ20t\nLXZ8Xd1Am4WqPeu2bf2z644bF6vHBqv5XUpcveQMFdEiQ2BF5CzgAlX9q0GOuxmrI75VVU+L2j4E\nLAVmAy2qmpncQUQ2AHuAXqBbVeflOE6L7b8zMixebHrzpBqktdWKO116aWw8rq+3bKrQX1UF9uU+\nWvWpw4ypqysu2pSuf7Fypc2KAK64It738Y/HcSGBadOslvikSZbrqqvLDPC7dtk7OPxwm9XcdtvI\nfrW7cXxsIiKo6pCrxxSU5FBEmoGFwIeADcAPCjjtW1jFwO8k2p4EPgBkTOb7oUCrqpbZidApJclB\nqLk5WxUWBqasATdf+0izapUN7lu22HZPz8D6FxddNLCPK1fCd77Tv62uzoRGY6PNKJ56ymJFIK6Y\nN3/+6Azcrl5yhkK+yPFTMWFxPrAN+A+gTlVbC7mwqj4iIrNSbeujaxdyiWHWUnNGkiybxpIlA0vH\nQvaAm699tAgzjaRKKl0IKf21vmJFrIZKpimZNMmE4T33WOBfZ6cZ4hsbvX6FU33km3E8hRVv+hNV\nfRFARP56RHplM477RaQXuEFVbxyh+zpDJMtD5/HHzXW1Glm0yNxrk7aIxkZ46CFTwzU3Ww3yENex\nerWpmTo7B3pN1dfbTGP5cntH06aZUHrTmwZPMeI4lUg+wfGn2IzjYRG5F5txjNQs4O2qukVEjgLW\niMh6VX0k68ClS5ceWm9tbaW1tXVkelhBVJqees8e+6p+6KHy50cqFy0tNtCHbLSNjWbUnjDBbDcr\nV5qQGD/e9m/aZMfPmgXPPtv/WuGYpLPAxIkW3V2N78apPtra2mhrayvZ9QY1jovI4cB5mBD5X5jN\n4k5VvW/Qi5uq6q5gHE+0PwgsyWUcTx17FbBPVZdn7BvzxvGs2gujkSMoaUzetMn6MXOmDbjp/lSa\noMtH6OtDD5lRO9gmHn3UAvhCSdauLptVzJ8Pd9xh8Rg9PRbTce65JiSynAWqdUbmVDdlN46r6j7g\nFuCWKEvunwGfAwYVHIOQ2WkRmQjUq+peEZkEvAe4epj3qlkqJUdQcGW99FLT58+aZckB0/2p9GR4\naQN/sNEEo3ZgwgQTDCH6O7jShliQ8O8RMt5C+cu4Os5IUVTp2MjL6d+jJS8ichswH5ghIhuBq4Cd\nmKfVDGC1iDyhqmeLyHHAjap6LnAMcEdkQG8AbilkduOMPi0t9sXd1mZCI4tKEXRZJIVaZ6e5A8+c\naWqldLnYGTPMhTaUcp06NbZX5IqgzmqvptmX4wSGUnO8IFR1YY5d/5lx7EtYzAeq+hxwern6VWuU\nOwK8WJqbzai8eXP+KnWVSFKohdQfXV2xemrOnLhUa0iKuGyZGbpPOim+Thj8V62KhUJwe02r7Cp5\n9uU4uSib4HBGhkrKEdTebgbi4DW0axd88Yv9+1Npgm4wurvN2H3ggAmOYJNobzevqlAs6ac/hYUL\nzbMKChMIlTz7cpx8uOCoASoliCsMhCecYEtwyU3GZuQTdKOttkkKtcZGi27fts2eqaGhfxW9VavM\nFTckQezutu1kQahCBEKo/wFxni7HqXRccDglo6PDVFQ7dlg97VxkCbpKUNskhVpHh0WNd3XFeadE\nbEbV1GReVskYj0JIC8bm5oEJEZubS/MslcJofww45cEFh1MS2tvti3z/fquMt327zToKVUNVitom\nCLXFi81zCuIZxbZtcO+9Nsvo6bFnFbF1kf4ZdNPquObmgYJx7lwzvnd1WVtj48AZWjVTCR8DTnlw\nweGUhDDwT55sZVf7+kpb0W4kCDaa9etNhdTYaLaN7m4TDqEqYAjoUzXhMmGCFZ+69trcHlRZgnH9\n+pF9vpGmUj4GnNLjgsMpCR0dsaE4lETdutW+OAv5yhxto3l7uxm3wzOo2tLUFM8IJk60Zxo3zgRJ\nb68JlZNPHlgrPK2Oy6quePTR8PDD/dtqTVXl1CZV9k3oVDKh6FHgsMMKL0lb7vogg5E0djc2mnCY\nONGy2l54oSUnfNvb7Hm6u20mImL1OgopvZtV+2L6dFNVzZhhy8yZccBhLeD1PmoXn3E4JaGpyQa+\nLVtsxhGKExVDJXiH9fWZmqqvz4TI/PmxC+6SJWbH2b07tmvMnFnYdbO8yVatMuEU4kTCbKtWqCRX\ncae0uOBwSkJQNR17rKl7+vrsy71avjIXLYpzTAU6O/urjlpaLE4jeF099pjNPgr9ms4SjNUU0zIU\nKuFjwCk9RVcArCQ8yWFlEVwvg76/qam6vjIvuMC8pnp6bCZwxBFWRjZXIsJSuJq6u6ozGgw3yaEL\nDseJyFXu1jPYOrXGcAWHG8cdJ8KNuY5TGD7jcJwEudKquxrJqSVcVVXF/a9VakFvXykFshynHJS9\nkJPjFEOtpJnwqGfHyY3bOJySkhxwCwmMcxyn+nDB4TgZuKHccXLjNg6npNSSbaAWbDWOk4Ubx6u4\n/7VKrQ64tfpcztijYgWHiNyM1RHfqqqnRW0fApYCs4EWVc1M6SYiC4CvAvXAN1X1uhzHueBwRoRa\nmkk5TiUHAH4LWJBqexL4APDwwMMNEakHvhadOxdYKCJzytVJxykEN/o7TkzZBIeqPgLsSrWtV9Vn\nBjl1HvA7Vd2gqt3A7cB5Zeqm4ziOUySV6FV1PLAxsb0panOcUcO9rBwnphIDAIsyWixduvTQemtr\nK62trSXujuN4bQmnumlra6Otra1k1yurV5WIzALuCsbxRPuDwJIs47iIvAVYqqoLou3PA31ZBnI3\njjuO4xRPNaccydXpx4BTIqHzEnA+sHCE+uSUAHdbdZzappzuuLcB84EZQAdwFbAT+Jeo7RXgCVU9\nW0SOA25U1XOjc88mdse9SVX/Icc9fMZRYbjbquNUPhUbxzESuOCoPLwYkuNUPpUcx+E4juPUIC44\nnJLibquOU/u4qsopOW4cd5zKxm0cVdx/x3Gc0cBtHI7jOM6I4oLDcRzHKQoXHI7jOE5RuOBwHMdx\nisIFh+M4jlMULjgcx3GconDB4TiO4xSFCw7HcRynKFxwOI7jOEXhgsNxHMcpChccjuM4TlFUYs1x\npwLwRIWO4+TCkxw6A/Aqfo5T23iSQ6fkrFplQmP6dFtE4tmH4zhO2VRVInIzcC6wVVVPi9qmA98F\nXgNsAP63qu7OOHcDsAfoBbpVdV65+ulUDq4ec5zqoJwzjm8BC1JtnwPWqOrrgAei7SwUaFXVM1xo\njDyjUcUvqMfa2my55BJrcxyn8iib4FDVR4Bdqeb3ASuj9ZXA+/NcYsj6N2d4tLSYTaO11ZaRsG+4\nesxxqoeR9qpqUtWOaL0DaMpxnAL3i0gvcIOq3jgivXMO0dLiqiLHcbIZNXdcVVURyeUS9XZV3SIi\nRwFrRGR9NINxapRFi+DBB001BiOjHsvC7SyOMzgjLTg6ROQYVX1ZRI4FtmYdpKpbor/bROROYB6Q\nKTiWLl16aL21tZXW1tZS99kZAYJ6bDQH7bQb8oMPuhuyUxu0tbXR1tZWsuuVNY5DRGYBdyW8qv4R\n2KGq14nI54Cpqvq51DkTgXpV3Ssik4D7gKtV9b6M63sch1MyFi82w/z06ba9c6fZeK6/fjR75Til\np2LjOETkNuDnwKkislFEPgp8GXi3iDwDvDPaRkSOE5HV0anHAI+IyFrgf4AfZQkNx3EcZ3TwyHHH\nifCIeWesMNwZhwsOx0ngxnFnLOCCo4r77ziOMxpUrI3DcRzHqU1ccDiO4zhF4YLDcRzHKQoXHI7j\nOE5RuOBwHMdxisIFh+M4jlMULjgcx3GconDB4TiO4xSFCw7HcRynKFxwOI7jOEXhgsNxHMcpChcc\njuM4TlG44HAcx3GKwgWH4ziOUxQuOBzHcZyicMHhOI7jFIULDsdxHKcoyiY4RORmEekQkScTbdNF\nZI2IPCMi94nI1BznLhCR9SLyrIh8tlx9dBzHcYqnnDOObwELUm2fA9ao6uuAB6LtfohIPfC16Ny5\nwEIRmVPGftYEbW1to92FisHfRYy/ixh/F6WjbIJDVR8BdqWa3wesjNZXAu/POHUe8DtV3aCq3cDt\nwHnl6met4P8pYvxdxPi7iPF3UTpG2sbRpKod0XoH0JRxzPHAxsT2pqjNcRzHqQBGzTiuqgpo1q6R\n7ovjOI5TOGLjd5kuLjILuEtVT4u21wOtqvqyiBwLPKiqs1PnvAVYqqoLou3PA32qel3G9V3IOI7j\nDAFVlaGe21DKjhTAD4GLgOuiv/+ZccxjwCmR0HkJOB9YmHWx4Ty44ziOMzTK6Y57G/Bz4FQR2Sgi\nHwW+DLxbRJ4B3hltIyLHichqAFXtAT4J/BhYB3xXVZ8qVz8dx3Gc4iirqspxHMepPSoycnyYwYMb\nROTXIvKEiPxi5HpdenK8hw+JyG9FpFdEmvOcW1NBlMN8FzXzm4Cc7+IrIvKUiPxKRO4QkSNynDsW\nfheFvoux8Lu4JnoPa0XkARGZmePc4n4XqlpxC3AWcAbwZKLtH4G/jdY/C3w5x7nPA9NH+xnK+B5m\nA68DHgSac5xXD/wOmAWMA9YCc0b7eUbjXdTabyLPu3g3UBetfznr/8cY+l0M+i7G0O9icmL9U8A3\nS/G7qMgZhw49eDBQE0bzrPegqutV9ZlBTq25IMphvItATfwmIOe7WKOqfdHm/wAnZJw6Vn4XhbyL\nQK3/LvYmNg8HtmecWvTvoiIFRw4KCR4EiwO5X0QeE5FLR6ZrFYcHUfZnrP0mPgbcndE+Fn8Xud4F\njJHfhYgsE5EXMU/WL2ccUvTvopoExyHU5le5rPpvV9UzgLOBvxKRs0auZxWDezz0Z8z8JkTkSuBV\nVb01Y/eY+l0M8i5gjPwuVPVKVT0R+DawIuuQYq9ZTYKjQ0SOAYiCB7dmHaSqW6K/24A7sWnYWGMz\nkDSCzcS+IsYkY+U3ISIXA+cAF+Y4ZMz8Lgp4F2Pmd5HgVqAlo73o30U1CY4QPAg5ggdFZKKITI7W\nJwHvAZ5MH1dD5NLPHgqiFJHxWBDlD0euW6NC5rsYK78JEVkAfAY4T1UP5DhsTPwuCnkXY+h3cUpi\n8zzgiYzDiv9djLYnQA7vgNuwqPFXMd3bR4HpwP3AM8B9wNTo2OOA1dH6yZhHwFrgN8DnR/tZSvwe\nPoY5BWwEuoCXgXvS7yHaPht4GvOWqOr3MJx3UWu/iTzv4lnghWhgeAL4tzH8uxj0XYyh38X3MYG4\nFvgBcHQpfhceAOg4juMURTWpqhzHcZwKwAWH4ziOUxQuOBzHcZyicMHhOI7jFIULDsdxHKcoXHA4\njuM4ReGCw6kJotTqT4jIkyLyPRFpHMa1vi0iH4zWbxSROXmOnS8ibx3CPTaIyPQc7b+OUmH/WEQy\nc7KJyGoRmVLsfR2nFLjgcGqFTlU9Q62+/avAJ5I7RaSYMsmHcqGp6qWavwLl/wLeVmxnyZ0fSIFW\nVX0jFtH7heROiVDVc1V1zxDu6zjDxgWHU4s8Arw2mg08IiL/BfxGROqiIj+/iL7oL4NDg/HXokI2\na4Cjw4VEpE1E3hStLxCRX0ZFcdaIyGuAy4ErotnO20XkKBH5fnSPX4jI26JzjxQrQPYbEbmRwtJ5\nh+d4jYg8LSIrsSjgmckZi4h8JFGs5ztRW2Y/HKcUFPMV5jgVTzSzOIc4lfYZwOtV9YVIUOxW1Xki\nchjwUxG5D2jGCkLNAY7Bat3fFJ2vgIrIUcC/A2dF15qqqrtF5BvAXlX95+j+twIrVPVnInIicC8w\nF7gKeFhVvyQi5wCX5HuM6O97gV9H668FPqyqv4juo9Hf1wNXAm9V1Z0SV8a8Pkc/HGfYuOBwaoVG\nEQkJ3B4GbgbeDvxCVV+I2t8DnCYifxZtTwFOwSqn3aqWf2eLiPwkdW0B3oIN/C8AqOru1P7AHwNz\nRA41TY6S6J0FfCA6924RSRcqS/KgiPQCv8JUVdOBF4LQSN33ncD3VHVnql9Z/Zioqp157us4BeGC\nw6kVutRqKxwiGjT3p477pKquSR13DoOrjgpN6ibAm1X11Yy+FFptrjUIgujc6Qx8jmS/sq6b2Q/H\nKQVu43DGEj8G/jIYykXkdSIyEZuhnB/ZQI7FDN5JFPhv4B0iMis6N3hE7QUmJ469D/h02BCRN0ar\nDwN/HrWdDUwrwfMo8BPgQwl7R7huuh+nl+B+jgO44HBqh6wZQbpS5Dcx+8XjIvIk8HWgXlXvxFJx\nr8Pq2f98wIVUtwOXAXeIyFoshTXAXcAHgnEcG6zPjIzVv8WM5wBXY4LnN5jK6gWyyedtNWBbVdcB\ny4CHon4tj/an+3FZjus6TtF4WnXHcRynKHzG4TiO4xSFCw7HcRynKFxwOI7jOEXhgsNxHMcpChcc\njuM4TlG44HAcx3GKwgWH4ziOUxQuOBzHcZyi+P90q8hWXZgeXwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x117257b10>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"actual_values = y_test\n",
"plt.scatter(y_pred, actual_values, alpha=.75,\n",
" color='b') #alpha helps to show overlapping data\n",
"plt.xlabel('Predicted Price')\n",
"plt.ylabel('Actual Price')\n",
"plt.title('Linear Regression Model')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"importances = adr.feature_importances_\n",
"feature_names = X_test.columns\n",
"n = 5"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"def plot_feature_importance(importances, feature_names, n):\n",
" indices = np.argsort(importances)[::-1][:n]\n",
" fig, ax = plt.subplots(1,1)\n",
" fig.set_size_inches(10,6)\n",
" plt.title(\"Feature importances\", fontsize = 16)\n",
" xlabels = [feature_names[int(i)] for i in indices]\n",
" plt.bar(range(n), importances[indices],\n",
" color=\"#799DBB\", align=\"center\")\n",
" plt.grid()\n",
" plt.xticks(range(n), xlabels, rotation=90)\n",
" plt.xlim([-1, n])\n",
" plt.ylim([0, min(1, max(importances[indices]+0.0005))])\n",
" plt.xlabel('Features', fontsize = 14)\n",
" plt.ylabel('Feature Importance', fontsize = 14)\n",
" plt.title('Variable Importance')\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmIAAAG/CAYAAAD7HruKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu4ZnVZ//H3hwEDxBzxlCI4eSRSGCUhz6Om4pHMQ1Ee\nJrX8ZZr+yiItE+zAzzRDJQsVD3iITC1RQzNzDA8FguABMLh0OCYnAZUQQe7fH8/awzObPTMLZu/v\nWvvZ79d17Yu91rP2s++52bO5WeuzvitVhSRJktrbYegCJEmSVioHMUmSpIE4iEmSJA3EQUySJGkg\nDmKSJEkDcRCTJEkaiIOYpGaSfD/Jmh7HrUlyQ5IFf0clOSzJexe7PklqzUFM0oKSfDLJ4QvsPzjJ\n/2xpSNqaqrpNVW1chPJu8QKISTYmecwi1LDdkmxI8oKh65A0HAcxSVvybuDZC+x/DvC+qrqh7xsl\n2XGxipp7y+342mI7BrnFkIkdhq5D0vAcxCRtyUeB2yd5+NyOJLcDngQcm+SAJF9KckWSi5K8JclO\nU8fekOTFSc4Gvjm17x7d509K8pUkVyU5L8lrFqjhBUku7N7/97ZUaJKfT/LFrpbTkjyyzx8wyfok\nX0jyxu5rz0nykCS/3tV0cZLnTh3/7iR/l+Rfk3yvO6O119TrD0lycpIrk5yU5MFTr21I8mdJPg9c\nDRwLPBw4qrtk++buuDd13/uqJF9O8rCp9zgsyQeTvKf7/l9Psv/U63sm+UiSS5JcluQtU689P8kZ\nSb7bne3cVLek4TiISVpQVV0DfBB47tTuZwFnVtXXgOuBlwG3Bx4MPAZ48by3ORh4ELDPAt/iB8Cz\nq+q2TIa730py8Lxj1gH3Ah4HHLrQJcUkewAfB15bVbcDXgF8OMkdev5RDwBOB3YH/r77Mz8QuCeT\nM4JHJdl16vhfBV4L3AE4DXh/V8fuwCeAI7v3eiPwiW54nfNs4DeA3YD1wInAb3eXbH+nO+YkYD/g\ndsAHgH9Mcqup93hKV+dtgeOBo7rvv6rrw7eBuwN7AMd1rx0MvBJ4Wlf3id17SBqYg5ikrXkP8Iyp\nQeC53T6q6tSqOqmqbqiqc4G3AfPPRB1RVVdW1bXz37iqPldV3+g+/xqToWH+1x9eVddU1deBdwGH\nLFDjs4F/qapPdu/1b8CXgSf2/DN+u6reU5MH734QuCuToe66qvo08CMmw+Ccj1fV56vqR8AfAQ9O\ncjcmw+Q3q+r9XU+OA84Cnjr3RwbeXVVndq9f3+3f7DJr9/VXdMe8EfgJ4L5Th5xYVZ/s6n0fk6EN\nJgPlXYDf73p2bVV9oXvt/zD5d/HN7pLyEcDaJHv27JGkJeIgJmmLuv+QXwY8Lck9mZzd+gBAkvsk\n+XgX3L8K+HMmZ8emnb+l905yYJLPdpfRrgRetI2vP4/JkDTf3YFndpcWr0hyBfBQ4Kd6/jEvnvr8\nGoCqunTevt26zwu4YO6Fqroa+G5X1126GqedO6/mhfqxWU4sySu6S4hXdn+W2zI5i7VQvf8L7Nzl\nzfYEzt1Cdu/uwJum+nN5t3+PBY6V1JCDmKRtOZbJmbBnA5+cGlL+FjgDuFd3efGPuOnvlK2F0T8A\n/DNwt6paDfzdAl+/17zPL1zgfc4D3ltVt5v6uE1V/WWPP9vNFSYDz2Qj2Y3JZcgLgYuYDDzT7j6v\n5vn9mD+EPRz4feCZVbW6u9R6Ff1uTjgf2Ku7RDnfecBvzuvRravqP3u8r6Ql5CAmaVuOBR4LvJDu\nsmRnN+D7wP8m2Rv4rZv5vrsBV1TVj5IcwCR7NX9Q+eMkuyT5WSaZqn9Y4H3eBzwlyeOSrEqyc5J1\nXXZsKTwxyUO7y7V/Cnypqi4ETgDuk+SQJDsm+WVgbya5rTnzB6qLmWTR5tyGSfbusiS3SvInwE/2\nrOsk4H+A/5dk164PD+le+zvgVUn2AUhy2yTP7P9HlrRUHMQkbVWX//oCsCuTcPicVzAZnr7HJB92\nHJsPUgudDZve92LgtUm+B7yamw5ZBXwOOAf4N+D1Xf5r7rXq6ruAyU0BrwIuYXL25/fo9/ttoaUs\ntnYWr5icyXsNk8t7D6Bb4qOqLgee3H3vy5j058lV9d2tvPebmGTwvpvkSOCT3cd/AxuZXBY9b97X\nL1hvVf2YSZD/Xt3XnM/k5gqq6p+B1wHHdZeRvwY8fit/TkmNZJL3lCRtS5J3ARdU1auHrkXSbPCM\nmCT1tz0LyUrSTTiISVJ/g6/KL2m2eGlSkiRpIJ4RkyRJGshiP4i3iSSexpMkSctGVS2YMV2WgxjA\nLF9STcIrj/n00GVs5sSPHsvDD37utg9s6IgXPHamfw4OO+wwDjvssKHLWFHseXv2vD173l6y5ft8\nvDSpXq687DtDl7DibNy4cegSVhx73p49b8+ej4uDmCRJ0kAcxNTLvg91Ee7W1q9fP3QJK449b8+e\nt2fPx2VZLl+RpJZj3X2NMSM2RrOeEZMkzYYkWwzre0ZMvZx71ulDl7DibNiwYegSVhx73p49b8+e\nj4uDmCRJ0kC8NDlCXprsx0uTkqTlYDSXJpMclOSsJGcnOXQLx6xL8pUkX0+yoWV9kiRJLTUbxJKs\nAo4CDgL2AQ5J8jPzjlkN/A3wlKq6H/CMVvVp68yItWeOoz173p49b8+ej0vLM2IHAOdU1caqug44\nDjh43jG/Cny4qi4AqKrLGtYnSZLUVMtBbA/g/KntC7p90+4N7J7ks0m+nOQ5zarTVt197/2GLmHF\nWbdu3dAlrDj2vD173p49H5eWz5rsk6reCXgg8BhgV+BLSf6zqs6ef+D69etZs2YNAKtXr2bt2rWb\nfrjmTrsu122YXAqcG37mLgu6vfn2nKH/fbnttttuu+329Pbc530eJ9XsrskkPw8cVlUHdduvBG6o\nqtdNHXMosEtVHdZtvwP4ZFV9aN57eddkY9OD4VjM+l2TGzZs2PSXW23Y8/bseXv2vL2x3DX5ZeDe\nSdYkuRXwy8Dx8475KPCwJKuS7AocCJzRsEZJkqRmmq4jluQJwJHAKuCYqjoiyYsAquro7phXAL8O\n3AC8varevMD7eEZMM39GTJI0G7Z2RqxlRoyqOgE4Yd6+o+dtvwF4Q8u6JEmShtDy0qSWMdcRa286\n9Kk27Hl79rw9ez4uDmKSJEkD8VmTI2RGrB8zYpKk5WAsd01KkiRpioOYejEj1p45jvbseXv2vD17\nPi4OYpIkSQMxIzZCZsT6MSMmSVoOzIhJkiSNkIOYejEj1p45jvbseXv2vD17Pi4OYpIkSQMxIzZC\nZsT6MSMmSVoOzIhJkiSNkIOYejEj1p45jvbseXv2vD17Pi4OYpIkSQMxIzZCZsT6MSMmSVoOzIhJ\nkiSNkIOYejEj1p45jvbseXv2vD17Pi4OYpIkSQMxIzZCZsT6MSMmSVoOzIhJkiSNkIOYejEj1p45\njvbseXv2vD17Pi4OYpIkSQMxIzZCZsT6MSMmSVoOzIhJkiSNkIOYejEj1p45jvbseXv2vD17Pi4O\nYpIkSQMxIzZCZsT6MSMmSVoOzIhJkiSNkIOYejEj1p45jvbseXv2vD17Pi4OYpIkSQMxIzZCZsT6\nMSMmSVoOzIhJkiSNkIOYejEj1p45jvbseXv2vD17Pi4OYpIkSQMxIzZCZsT6MSMmSVoOzIhJkiSN\nkIOYejEj1p45jvbseXv2vD17Pi4OYpIkSQMxIzZCZsT6MSMmSVoOzIhJkiSNkIOYejEj1p45jvbs\neXv2vD17Pi4OYpIkSQMxIzZCZsT6MSMmSVoOzIhJkiSNUNNBLMlBSc5KcnaSQxd4fV2Sq5J8pfv4\n45b1acvMiLVnjqM9e96ePW/Pno/Ljq2+UZJVwFHALwAXAicnOb6qzpx36Oeq6qmt6pIkSRpKyzNi\nBwDnVNXGqroOOA44eIHjFryGqmHdfe/9hi5hxVm3bt3QJaw49rw9e96ePR+XloPYHsD5U9sXdPum\nFfCQJKcn+Zck+zSrTpIkqbGWg1if29tOBfasqv2AtwD/vLQlqS8zYu2Z42jPnrdnz9uz5+PSLCPG\nJBe259T2nkzOim1SVd+f+vyEJG9NsntVfXf+m61fv541a9YAsHr1atauXbvpdOvcD9ly3YbJ4DN3\nOXBuCBpy++LzzhlVPdOG/ve1VNuz/udz222A0047bVT1rITt0047bVT1zOL23OcbN25kW5qtI5Zk\nR+CbwGOAi4CTgEOmw/pJ7gxcUlWV5ADgg1W1ZoH3ch0xuY6YJGlZ2No6Ys3OiFXV9UleAnwKWAUc\nU1VnJnlR9/rRwDOA30pyPfC/wK+0qk+SJKm1puuIVdUJVXXfqrpXVR3R7Tu6G8Koqr+pqvtV1dqq\nekhV/WfL+rRlZsTamz7FrTbseXv2vD17Pi6urC9JkjQQnzU5QmbE+jEjJklaDnzWpCRJ0gg5iKkX\nM2LtmeNoz563Z8/bs+fj4iAmSZI0EDNiI2RGrB8zYpKk5cCMmCRJ0gg5iKkXM2LtmeNoz563Z8/b\ns+fj4iAmSZI0EDNiI2RGrB8zYpKk5cCMmCRJ0gg5iKkXM2LtmeNoz563Z8/bs+fj4iAmSZI0EDNi\nI2RGrB8zYpKk5cCMmCRJ0gg5iKkXM2LtmeNoz563Z8/bs+fj4iAmSZI0EDNiI2RGrB8zYpKk5cCM\nmCRJ0gg5iKkXM2LtmeNoz563Z8/bs+fj4iAmSZI0EDNiI2RGrB8zYpKk5cCMmCRJ0gg5iKkXM2Lt\nmeNoz563Z8/bs+fj4iAmSZI0EDNiI2RGrB8zYpKk5cCMmCRJ0gg5iKkXM2LtmeNoz563Z8/bs+fj\n4iAmSZI0kF4ZsSQBXtx93AP42ar6VpI/BL5VVR9c2jJvUo8ZMZkRkyQtC4uREXsZ8MfA2+ftvwh4\nyXbUJkmStGL1HcR+C/iNqjoSuH5q/6nA/Ra9Ko2OGbH2zHG0Z8/bs+ft2fNx6TuI7QV8bYH91wG7\nLF45kiRJK0ffQezbwP4L7H8CcMbilaOxuvve+w1dwoqzbt26oUtYcex5e/a8PXs+Ljv2PO71wFFJ\ndmEyvD0kyXOBPwCev1TFSZIkzbJeZ8Sq6l3Aa4AjmFyKPBZ4IfDSqjpu6crTWJgRa88cR3v2vD17\n3p49H5e+Z8SoqrcDb09yR2CHqrp46cqSJEmafX3XEbsfsKqqTp+3fz/guqpqmhNzHTGB64hJkpaH\nxVhH7G3A3gvs36d7TZIkSTdT30Hs/sDJC+w/Gdh38crRWJkRa88cR3v2vD173p49H5e+g9iPgd0X\n2L8aWPBUmyRJkraub0bseCbD2DOr6vpu307AB4FbVdWTlrTKm9ZjRkxmxCRJy8LWMmJ975r8A+Dz\nwNlJPs/kLNjDgN2ARyxKlZIkSStM33XEzmKSBfsAcHvgdsD7gH1b3zGpYZgRa88cR3v2vD173p49\nH5ebs47YRcAfLWEtkiRJK0qvjBhAklsD+wF3Yt6ZtKr6SM/3OAg4ElgFvKOqXreF4x4EfAl41kLv\nbUZMYEZMkrQ8bHdGLMkvAMex8J2T0OMSZ5JVwFHALwAXAicnOb6qzlzguNcBn8Q7MiVJ0gzru3zF\nm4CPA3djssL+DtMfPd/jAOCcqtpYVdcxGewOXuC4lwIfAi7t+b5qwIxYe+Y42rPn7dnz9uz5uPTN\niK0BntrlxG6pPYDzp7YvAA6cPiDJHkyGs0cDDwK87iRJkmZW37NZX2ThRxzdHH2GqiOBP+wCYMFL\nk6Nx9733G7qEFWfdunVDl7Di2PP27Hl79nxc+p4R+1vgDUnuCnwVuG76xao6tcd7XAjsObW9J5Oz\nYtP2B45LAnAH4AlJrquq4+e/2fr161mzZg0Aq1evZu3atZt+uOZOuy7XbZhcCpwbfuYuC7q9+fac\nof99ue2222677fb09tznGzduZFv6rqx/w1Zerqpa1eM9dgS+CTwGuAg4CThkflh/6vh3AR/zrslx\nmB4Mx2LW75rcsGHDpr/casOet2fP27Pn7S3Gyvr32N4iqur6JC8BPsVk+YpjqurMJC/qXj96e7+H\nJEnSctJ7HbEx8YyYYPbPiEmSZsNinBGbe8j3g4C9gFtNv1ZVx25XhZIkSSvQDn0OSrI3cAbwH0ye\nN/kO4N3dP/9mqYrTeLiOWHvToU+1Yc/bs+ft2fNx6TWIMVlW4lTgtsDVwD7AzwGnAU9fmtIkSZJm\nW9+7Ji8HHllVX09yFXBAVX0zySOBt1TVvktd6Lx6zIjJjJgkaVnYWkas7xmxANd0n1/KZJV8mKwN\ndu/tK0+SJGll6juIfQOYO+t1EnBodzbscOCcpShM42JGrD1zHO3Z8/bseXv2fFz63jX558Cu3eev\nZvIA8M8ClwG/vAR1SZIkzbxbvI5YktsDV1TV1lbdXxJmxARmxCRJy8N2Z8SSvDPJbab3VdXlwC5J\n3rkINUqSJK04fTNi64FdFti/K/C8RatGo2VGrD1zHO3Z8/bseXv2fFy2mhFLsjuTOyYBdk9y/dTL\nq4AnAxcvUW2SJEkzbasZsSTbyn8V8Jqq+rNFrWobzIgJzIhJkpaH7XnW5KO7f/47kxX0r5h67UfA\nuVV14faXKEmStPJsNSNWVRuALwIfBk6vqg1TH190CFs5zIi1Z46jPXvenj1vz56PyzbD+lX1I+AJ\nQPNlKiRJkmZZ32dNfgT4eFWNYqkKM2ICM2KSpOVhezJic/4NOCLJfsCXgaunX6yqj2xfiZIkSStP\n33XEjgLuCLwUeA/woXkfmnFmxNozx9GePW/Pnrdnz8el1xmxquo7sEmSJKmnW/ysySGZEROYEZMk\nLQ/b/azJ7k2enOTEJJcnuSzJ55I8afHKlCRJWln6PvT7hcBHgHOAQ4E/BL4N/FOSFyxdeRoLM2Lt\nmeNoz563Z8/bs+fj0veuyUOB362qo6b2vSPJKd1rxyx6ZZIkSTOu7zpi1wI/W1XnzNt/b+AbVXWr\nJapvS/WYEZMZMUnSsrAYGbHzgcctsP+xwLm3tDBJkqSVrO8g9nrgyCTvSPLr3ccxwJHAG5auPI2F\nGbH2zHG0Z8/bs+ft2fNx6buO2NFJLgFeATyt230m8Myq+uhSFSdJkjTLXEdshMyI9WNGTJK0HCzG\nsybn3ujRwD7d5plV9ZntLU6SJGml6ruO2E8n+Qrwr8AfdB+fSnJaknssZYEaBzNi7ZnjaM+et2fP\n27Pn49I3rH8M8D3gHlW1V1XtBdwDuALXEJMkSbpF+q4jdg3w4Ko6bd7+tcB/VtXOS1TfluoxIyYz\nYpKkZWGx1hHbZYH9OwPn3dLCJEmSVrK+g9jvAm9K8uAkq5LskOTBwJuA31u68jQWZsTaM8fRnj1v\nz563Z8/Hpe9dk38P/ATwBeCGbt8OwPXAB5JNZ9uqqn5yUSuUJEmaUX0zYuv7vmFVvXs76unFjJjA\njJgkaXnY7nXEWgxXkiRJK03fjBgASXZPsneSfaY/lqo4jYcZsfbMcbRnz9uz5+3Z83HpdUYsyb7A\ne4D9Fni5gFWLWZQkSdJK0DcjdirwP8BfApcwGb42qaqzlqS6LddjRkxmxCRJy8JiPGvyPsAvV9XZ\ni1eWJEnSytY3I/ZFYO+lLETjZkasPXMc7dnz9ux5e/Z8XPqeEXsh8M4k9wS+Blw3/WJV/cdiFyZJ\nkjTr+mbE1jFZ1PXOC7xcVdU0rG9GTGBGTJK0PCzGsyaPBv4duD+TYexOUx8LDWdbKuSgJGclOTvJ\noQu8fnCS05N8JckpSR7d970lSZKWm76D2N2A11TVN6rq0qq6bPqjzxskWQUcBRwE7AMckuRn5h32\nb1W1X1U9AFgPvK1nfVpiZsTaM8fRnj1vz563Z8/Hpe8g9hlg/+38XgcA51TVxqq6DjgOOHj6gKq6\nempzN6DXkCdJkrQc9Q3r/wvwV93Crl/lpmH9j/R4jz2A86e2LwAOnH9Qkl8EjgDuAjyuZ31aYnff\ne6G1fLWU1q1bN3QJK449b8+et2fPx6XvIPbW7p+v3MLrfc6s9UpVV9U/A/+c5OHAe4H79vk6SZKk\n5abvQ79v1jMpt+BCYM+p7T2ZnBXb0vc8McmOSW5fVZfPf339+vWsWbMGgNWrV7N27dpNU/7c9e/l\nug2TTNbcWai5fNaQ2xefdw4HPO7po6ln2tD/vpZqe27fWOpZCdvzez90PSth+8gjj5yp39/LYfu0\n007j5S9/+WjqmcXtuc83btzItvRavmIxJNkR+CbwGOAi4CTgkKo6c+qYewLfqqpK8kDgH6vqngu8\nl8tXNDY9GI7FrC9fsWHDhk1/udWGPW/Pnrdnz9vb2vIVWx3EkvxSn2/QMyNGkicARzJ5SPgxVXVE\nkhd173F0kj8Ansskg/YD4Her6uQF3sdBTDM/iEmSZsP2PGvyQz2/R69Ll1V1AnDCvH1HT33+l0we\nLC5JkjTztjpAVdUOfT5aFavhuI5Ye9NZA7Vhz9uz5+3Z83FxiJIkSRpIs7D+YjIjJjAjJklaHhbj\nWZOSJElaZA5i6sWMWHvmONqz5+3Z8/bs+bg4iEmSJA2kd0YsSYCfA+4BfKKqfpBkN+Da7iHezZgR\nE5gRkyQtD9uzjtjcG9wZ+ChwAJNnRt6byYKrfwX8EHjZ4pQqSZK0cvS9NPnXwCXA7YH/ndr/j8Dj\nF7sojY8ZsfbMcbRnz9uz5+3Z83HpdUaMyfMhH1NVV0yuUG7yLWCvRa9KkiRpBeh7RmwXJs9/nO8O\nTC5NasaN7YHfK4EP5W3Pnrdnz9uz5+PSdxA7EVg/vSPJjsChwGcWuSZJkqQVoe8g9vvAbyT5N+An\ngDcAZwAPA165RLVpRMyItWeOoz173p49b8+ej0uvQayqzgDuD3wR+DSwM/BBYG1VnbN05UmSJM2u\nba4jluRWTC5NPreqvtmkqm1wHTGB64hJkpaH7XrWZFX9CPhpJuuHSZIkaZH0zYgdC/zGUhaicTMj\n1p45jvbseXv2vD17Pi591xHbFXh2kscCpwBXd/sDVFX9zlIUJ0mSNMt6PWsyyYapzekvmBvEHrXI\ndW2rHjNiWtSM2LyFirUVs/x3T5KWwnY/a7Kq1i1qRdIIOfxu2xEveOzQJUjSTOmbEdMKZ0asPXve\nntmZ9ux5e/Z8XHqdEUvyMSaXJKdPq81tV1U9dQlqkyRJmml9w/qXs/kgthOwH3A34J+WoC6NjM+a\nbM+et+cz+Nqz5+3Z83HpmxFbv9D+JG8ErlrMgiRJklaK7c2IHQ28ZDEK0biZV2rPnrdndqY9e96e\nPR+X7R3E7rMoVUiSJK1AfcP6b+Gm64fdFXgC8M4lqEsjY16pPXventmZ9ux5e/Z8XPqG9e/P5oPY\nDcClwMtxEJMkSbpFel2arKp1VfWoqY/HVNWvVNXbqur6pS5SwzOv1J49b8/sTHv2vD17Pi69BrEk\n70xymwX23zqJZ8QkSZJugb5h/fXALgvs3xV43qJVo9Eyr9SePW/P7Ex79rw9ez4uW82IJdmdGxdx\n3T3J9GXIVcCTgYuXqDZJkqSZtq0zYpcxCeUDnNFtz31cDLwDeOuSVafRMK/Unj1vz+xMe/a8PXs+\nLtu6a/LR3T//HXg6cMXUaz8Czq2qC5eiMEmSpFm31UGsqjYAJLkHcF5V3dCiKI2PeaX27Hl7Zmfa\ns+ft2fNx6fusyY0ASe4K7AXcat7r/7HolUmSJM24vstX3DXJ54ALgC8CG6Y+PrtEtWlEzCu1Z8/b\nMzvTnj1vz56PS9/lK44EfgzsA1wNPBx4BnAmk8ccSZIk6Wbq+4ijRwJPrqqzkhRwaVV9Icm1wGuB\nf12yCjUK5pXas+ftmZ1pz563Z8/Hpe8ZsV24cRmL7wJ36j4/E/C/FpIkSbdA30Hsm8De3eenA7+V\n5O7AiwGXr1gBzCu1Z8/bMzvTnj1vz56PS99Lk28C7tJ9fjjwKeAQ4Fp8xJEkSdIt0nf5ivdNfX5q\nkjVMzpCdV1WXbunrNDvMK7Vnz9szO9OePW/Pno9L3zNimyS5M3BZVZ2yBPVIkiStGH3XEdspyeuT\nfB+4CLh7t/91SV68lAVqHMwrtWfP2zM70549b8+ej0vfsP5rgKcAzwF+OLX/ZGD9zfmGSQ5KclaS\ns5McusDrv5bk9CRfTfKFJPvenPeXJElaLvpemvxV4PlVtSHJ9PMmvw7ct+83S7IKOAr4BSZ3W56c\n5PiqOnPqsG8Bj6iqq5IcBLwN+Pm+30NLw7xSe/a8PbMz7dnz9uz5uPQ9I3YX4NwF9u/IzcuZHQCc\nU1Ubq+o64Djg4OkDqupLVXVVt/lfwN1uxvtLkiQtG30HsTOARyyw/5nAzQnt7wGcP7V9QbdvS14A\n/MvNeH8tEfNK7dnz9szOtGfP27Pn49L3bNZhwPuS3K37mmcm+RkmlyyfdDO+X/U9MMmjgOcDD70Z\n7y9JkrRs9F1H7GNJngX8EXADk/D+qUyeP/npm/H9LgT2nNrek8lZsc10Af23AwdV1RULvdH69etZ\ns2YNAKtXr2bt2rWbrnvPTfvLdRsmZ0PmMkJzZ0aG3p6ubUz1LFb/x/bnG+v20H8/lnJ73bp1o6pn\nJWzP7RtLPStle85Y6pm17bnPN27cyLakassnqbqB6BtV9eNtvlMPSXZk8rikxzBZBuMk4JDpsH6S\nvYB/B55dVf+5hfeprdW93CXhlcfcnPl2ZTriBY9lsX4O7Hk/i9lzSVopklBVWei1bWXETgNuP/VG\nn0hyl60cv1VVdT3wEiaPSDoD+IeqOjPJi5K8qDvsT4DbAX+b5CtJTrql30+Lx7xSe/a8vflnC7T0\n7Hl79nxcbu7K+o8Adtmeb1hVJwAnzNt39NTnLwReuD3fQ5IkaTnoe9ekVjjXtGrPnrc3nVtSG/a8\nPXs+Lg5ikiRJA+kziL03yfFJPgbsDLwtycemPo5f4ho1AuaV2rPn7Zmdac+et2fPx2VbGbFjmaz9\nNZf0f/8Cx3gLlSRJ0i2w1UGsqtY3qkMjZ16pPXventmZ9ux5e/Z8XMyISZIkDcRBTL2YV2rPnrdn\ndqY9e96ePR8XBzFJkqSBOIipF/NK7dnz9szOtGfP27Pn4+IgJkmSNBAHMfViXqk9e96e2Zn27Hl7\n9nxcHMRMr2qrAAAX0UlEQVQkSZIG4iCmXswrtWfP2zM70549b8+ej4uDmCRJ0kAcxNSLeaX27Hl7\nZmfas+ft2fNxcRCTJEkaiIOYejGv1J49b8/sTHv2vD17Pi4OYpIkSQNxEFMv5pXas+ftmZ1pz563\nZ8/HxUFMkiRpIA5i6sW8Unv2vD2zM+3Z8/bs+bg4iEmSJA3EQUy9mFdqz563Z3amPXvenj0fFwcx\nSZKkgTiIqRfzSu3Z8/bMzrRnz9uz5+PiICZJkjQQBzH1Yl6pPXventmZ9ux5e/Z8XBzEJEmSBuIg\npl7MK7Vnz9szO9OePW/Pno+Lg5gkSdJAHMTUi3ml9ux5e2Zn2rPn7dnzcXEQkyRJGoiDmHoxr9Se\nPW/P7Ex79rw9ez4uDmKSJEkDcRBTL+aV2rPn7Zmdac+et2fPx8VBTJIkaSAOYurFvFJ79rw9szPt\n2fP27Pm4OIhJkiQNxEFMvZhXas+et2d2pj173p49HxcHMUmSpIE4iKkX80rt2fP2zM60Z8/bs+fj\n4iAmSZI0EAcx9WJeqT173p7ZmfbseXv2fFwcxCRJkgbSfBBLclCSs5KcneTQBV7fO8mXkvwwye+1\nrk8LM6/Unj1vz+xMe/a8PXs+Lju2/GZJVgFHAb8AXAicnOT4qjpz6rDLgZcCv9iyNkmSpNZanxE7\nADinqjZW1XXAccDB0wdU1aVV9WXgusa1aSvMK7Vnz9szO9OePW/Pno9L60FsD+D8qe0Lun2SJEkr\nTutBrBp/Py0S80rt2fP2zM60Z8/bs+fj0jQjxiQXtufU9p5MzordbOvXr2fNmjUArF69mrVr1276\n4Zo77bpct2FyWWruP8Rzl6jc3nx7zmL1f85Y/nxj3R7674fbbrvt9ti35z7fuHEj25KqdiepkuwI\nfBN4DHARcBJwyLyw/tyxhwHfr6q/WuC1all3a0l45TGfHrqMzUwPhmNxxAsey2L9HNjzfhaz52O0\nYcOGTb9Q1YY9b8+et5eEqspCrzU9I1ZV1yd5CfApYBVwTFWdmeRF3etHJ/kp4GTgJ4EbkrwM2Keq\nftCyVklLK1nwd5IWMMvDr7TStb40SVWdAJwwb9/RU59/h80vX2oExnZmZiVYCT0f21nIMTriBY8d\nuoQl5ZmZ9uz5uLiyviRJ0kAcxNSLa1q1Z8/bs+ftTYeb1YY9HxcHMUmSpIE4iKmXlZBXGht73p49\nb8+8Unv2fFwcxCRJkgbiIKZezM60Z8/bs+ftmVdqz56Pi4OYJEnSQBzE1IvZmfbseXv2vD3zSu3Z\n83FxEJMkSRqIg5h6MTvTnj1vz563Z16pPXs+Lg5ikiRJA3EQUy9mZ9qz5+3Z8/bMK7Vnz8fFQUyS\nJGkgDmLqxexMe/a8PXvennml9uz5uDiISZIkDcRBTL2YnWnPnrdnz9szr9SePR8XBzFJkqSBOIip\nF7Mz7dnz9ux5e+aV2rPn4+IgJkmSNBAHMfVidqY9e96ePW/PvFJ79nxcHMQkSZIG4iCmXszOtGfP\n27Pn7ZlXas+ej4uDmCRJ0kAcxNSL2Zn27Hl79rw980rt2fNxcRCTJEkaiIOYejE70549b8+et2de\nqT17Pi4OYpIkSQNxEFMvZmfas+ft2fP2zCu1Z8/HxUFMkiRpIA5i6sXsTHv2vD173p55pfbs+bg4\niEmSJA3EQUy9mJ1pz563Z8/bM6/Unj0fFwcxSZKkgTiIqRezM+3Z8/bseXvmldqz5+PiICZJkjQQ\nBzH1YnamPXvenj1vz7xSe/Z8XBzEJEmSBuIgpl7MzrRnz9uz5+2ZV2rPno+Lg5gkSdJAHMTUi9mZ\n9ux5e/a8PfNK7dnzcXEQkyRJGoiDmHoxO9OePW9v1nuexI+eH7PMjNi47Dh0AZKkdl55zKeHLmEz\n5551+uguCR/xgscOXYJWEM+IqZex/aJcCex5e/a8PXvenhmxcXEQkyRJGkjTQSzJQUnOSnJ2kkO3\ncMybu9dPT/KAlvVpy2Y9OzNG9rw9e96ePW/PjNi4NBvEkqwCjgIOAvYBDknyM/OOeSJwr6q6N/Cb\nwN+2qk9bd/F55wxdwopjz9uz5+3Z8/ZOO+20oUvQlJZnxA4AzqmqjVV1HXAccPC8Y54KvAegqv4L\nWJ3kzg1r1BZce83VQ5ew4tjz9ux5e/a8vSuvvHLoEjSl5SC2B3D+1PYF3b5tHXO3Ja5LkiRpEC2X\nr6iex81fwKXv12kJXXnZd4YuYcWx5+3Z8/ZmvedjXZPs8MMPH7qEm6hamf+5T6s/eJKfBw6rqoO6\n7VcCN1TV66aO+TtgQ1Ud122fBTyyqi6e914r89+WJElalqpqwam85RmxLwP3TrIGuAj4ZeCQeccc\nD7wEOK4b3K6cP4TBlv8wkiRJy0mzQayqrk/yEuBTwCrgmKo6M8mLutePrqp/SfLEJOcAVwO/3qo+\nSZKk1ppdmpQkSdLmfNakNCJJ7s9knb2d6W5UqapjBy1KkrRkPCOmzST5AVu+U7Wq6idb1rOSJDkM\neCTws8AngCcAn6+qZwxZ1yxK8ntTm8WNd2vPDb9vbF7UjPN3S3tJPja1Of1zDpOeP7VxSVqAZ8S0\nmarabegaVrBnAPsBp1bVr3eLGb9/4Jpm1W2Y/IfpvsCDmNwoFODJwEkD1jWz/N0yiL/q/vk04KeA\n9zH5OT8EuMmNcBqGZ8S0VUnuxOQyGQBVdd6A5cy0JCdX1YOSnAI8GvgecFZV3Xfg0mZWkhOBJ1bV\n97vt2wD/UlUPH7ay2ZZkP+ARTIbhE6vKB04uoSSnVNX+29qnYTR96LeWjyRPTXI28G3gc8BG4IRB\ni5p9Jye5HfB2Jsu9fAX44rAlzbw7AddNbV/X7dMSSfIyJmd67wjcGXhfkt8ZtqqZt2uSe85tJLkH\nsOuA9WiKZ8S0oCRfZXJW5tNV9YAkjwKeU1XPH7i0FSHJTwO3qaqvDl3LLEvyR0zWNPwIk0s2vwj8\nQ1X9xaCFzbAkXwN+vqqu7rZvDfxnVd1/2MpmV5KDgLcx+R9rgDXAb1bVpwYrSps4iGlBc6etk5wO\nPLCqfpzkq1W179C1zaokOwC/Bvx0Vb02yV7AT1WVmaUllGR/4OFMLpP9R1V9ZeCSZlo3iB1QVdd0\n27sAJzmILa0kOwN7M/k5P6uqrh24JHUM62tLrujyMicC709yCfCDgWuadW8FbmByJvK1TPr9VuDn\nhixqBTgN+A6T34eVZC+zkEvqXcB/JZk+C/nOYUuaTUmezo13S07fNXnPJFTVRwYrTpt4RkwLSrIb\ncA2THOGvAT8JvL+qLh+0sBmW5CvdZeCvVNUDun2nV9V+Q9c2q5K8FHgNcAnw47n9np1ZWt1ZyId2\nmyd6FnJpJHk3W14yhKry6TUj4BkxLaiq5s5+/Rh494ClrCQ/SrJqbiPJHZmcIdPSeTlwX/8Ho7kf\nc+OA4M/4Eqmq9UPXoG3zrkktKMkPkny/+7g2yQ1Jvjd0XTPuLcA/AXdK8hfAF4Ajhi1p5p3HZJkQ\nNdLdNfk+JndN3gnvmlxySVYn+eskp3Qff5XktkPXpQkvTWqbuhD5U5nc6fSHQ9czi7oePxj4LvCY\nbvdnqurM4aqafUneCdyHyZMMftTtLlfWXzreNdlel8f7GvAeJjmx5wD7VtUvDVqYAAcx3QxJTquq\ntUPXMavsb3vdY6XgxstkYTKIHT5MRbPPuybbWyhrav50PMyIaUHd3TZzdgD2ZxLe19L5tyTPAD5c\n/h9SE1V1GGxaUZ+5Ffa1pLxrsr1rkjy8qk4ESPIw4H8Hrkkdz4hpQfPutrmeycr6b6+qS4aqadZ1\nD0XelUmQ+Yfdbh+GvISS3B84Frh9t+tS4HlV9fXhqpp93jXZVpK1TH7O53JhVzD5OffRUiPgGTEt\nyLtt2vOhyIN4G/C7VfVZgCTrun0PGbKoFWL+2lZaIlV1GrBvF9CvqvIGlRHxrkndRJInJvlcksuS\nXJXkxCRP6167zdD1rQRJ7pnk1Um+MXQtM27XuSEMoKo2ALcerpzZl+RPmCyJsztwB+BdSV49aFEz\nqntm8JqpXS8HPp/k+O4xahoBL01qM0l+A3gR8AfAKd3u/YH/x+RBvb9YVY8aqLyZlmQPJs89PAS4\nP5Oef7iqvjZoYTMsyT8z+Tl/L5MzM78G7F9VTxu0sBmW5L+Z3LH3w257F+D0qrrPsJXNnu7GiAOr\n6n+TPBn4a+BXgAcAz6yqxw9aoAAvTeqmfhd42LwFLv89yROAC4H/O0xZsyvJi5gMX3cCPgQ8Hzh+\nLkiuJfV84HAmD/2GySO9fLD90roQ2IUbc5A7AxcMV85Mu6Gq5kL5vwQcU1WnAKck+e0B69IUBzHd\nxEKrjFfV5Uk2VtXfDlHTjDsK+CTwsrnwbGJsZil1Z2Fu09188tKp/XfixgFBS+N7wDeS/Gu3/Vjg\npCRvYZJfcnHXxZMuTnI1k/UJp39/7zxMSZrPQUzzfS/J2i7cuUmS/YCrBqpp1t0FeCbw5m4Q+BCw\n07Alzbw3Mxl+Pzxv/0OBxwG/1byileOfug+YhPU3sPmDqbV4jgS+AnwfOLOqTgZI8kDgoiEL043M\niGkz3foy72ey1s8pTH457g+sB549tw6NlkaSPbkxJ3Zr4CNV9aphq5o9SU6tqgdu4bUzqmqf1jVp\nIsmHq+rp2z5SfSTZC1gDfL6qbuj23QXYqarOG7I2TXjXpDZTVZ8HDgRWMRm+nsfk5+RAh7ClleQn\nqur8qnpDVe3P5LFSXiZbGrtu5TV/Lw7rHkMXMGPOB946N4QBVNX/OISNh5cmdRNV9R3A28nb+xKw\n6SxNVf13t2zIa4craWZdkuTAqvqv6Z1JDgBctFgzo6qqe9D3AVV10tD16KYcxLSZ7nbnLamq2rdZ\nMStEd5ngrsCuXXZjLivzk0zuLtPiewXwwe4JEtOX4J/H5PZ+aZb8PPDsJOcyCe6Dv89Hw0FM8z1l\n6AJWoMcxuQy8B/BXU/u/D5gPWwJVdVKSA4HfZtJ7gG8weRi1Z8Q0E5KcALyYye8Yb8UeKcP60kgk\neUZVfWjoOqRWkuwO3K2qvjq17/FV9akBy5oZSZ4J/DnwHuAvq+q6gUvSAhzEtJnuwdNb+qHwAdRL\nKMlq4DXAI7pdG4DXVpXLhiwyL8EPJ8nnmJx535HJZeFLgS9UlYtFL4EkuwF/AjyeyRMk5n6/V1W9\ncbDCtImXJrUZHzw9qHcCX2OypliA5zBZRuSXhixqRnkJfji3rarvJXkhcGxVvWYbg7G2z3XAD5gs\n4Hob4IatH67WHMS0Vd0Co5tWYPaW5yV1z6qaHroOS3L6YNXMsKraOHQNK9iq7gaVZwF/3O3z0swS\nSHIQ8EbgY8ADph53pBFxENOCkjyVSXD8rkxu5787cCbws0PWNeOuSfLwufXausV1/cW5BLwEP6jX\nAp9icjnypCT3BM4euKZZ9UdMHu79jaEL0ZaZEdOCknwVeDTw6ap6QJJHAc+pKh+IvESSrAWOBW7b\n7boCeN7c8ycl6eZIkvI/8qPnGTFtyXVVdVmSHZKsqqrPJnnT0EXNsu75nvsmuW23fVWSlwMOYkvM\nS/DtdA9cfwGwDzeuk1f+T97icwhbHnyUh7bkiiS3AU4E3p/kzUwCn1piVXXV1J2SvzdoMTMuyVOT\nnA18G/gcsBE4YdCiZt97gTsDBzG5M/hu+LtFK5iXJrWgJLdm8pzDHYBfY7LK+/ur6vJBC1thkpxf\nVXsOXces8hJ8e0lOq6q1Sb5aVfsm2YnJA6kPHLo2aQhemtRNJNkR+HhVPQr4MfDuYSuSloyX4Nv7\nUffPq5LcH/gOcMcB65EG5SCmm6iq65PckGR1VV05dD2zbht38O3aspYVaP4l+EvwMtlSe3u3ov4f\nAx8F5hYclVYkL01qQUmOBx4AfJrNHxL7O8NVJS0uL8G3l2Rn4OnAGqZOBlTV4UPVJA3JM2Lako90\nH3OTenDRRc0QL8EP5qPAlUweb/TDgWuRBucgpgVV1buT7ArsVVVnDV2PtNi8BD+YParq8UMXIY2F\ng5gW1K2s/3rgJ4A1SR4AHF5VTx22MmlRXQ18LYmX4Nv5YpJ9q+qrQxcijYEZMS0oyalMbuv/bFU9\noNv39aq637CVSYsnyfru080uwVfVe4apaHZNPdh7FXBvJmu3Xdvtq6rad5DCpIF5Rkxbcl1VXZlk\net8NQxUjLQUvwTf1lKELkMbIQUxb8o0kvwbsmOTewO8AXxy4JmlReQm+naraOHQN0hj5iCNtyUuB\nn2Vy6eDvge8BLx+0ImnxHQYcyOQB61TVV4B7DFmQpJXFM2LakvtW1auAVw1diLSEvAQvaVCeEdOW\nvDHJWUn+NIkBfc2qzS7BJ3kLXoKX1JCDmBZUVeuARwGXAUcn+VqSVw9blbTovAQvaVAuX6Ft6h7M\neyjwy1W109D1SIslyQOr6tSh65C0cjmIaUFJ9gGeBTwDuBz4B+BDVXXJoIVJiyjJBuCngH8E/qGq\nvj5sRZJWGgcxLSjJfwEfBz4HnFxV1wxckrQkktyFyf90PIvJQ78/WFV/OmxVklYKBzFtJslOwJ8D\nLwDO7XbvBbwLeFVVXTdUbdJS8hK8pCEY1td8rwd2B9ZU1QOr6oFM1lVaDbxh0MqkRZZknySHJfk6\ncBSTOyb3GLgsSSuIZ8S0mSTnAPepqhvm7V8FfLOq7jVMZdLi8xK8pKF5Rkzz3TB/CAOoqh/jQpea\nEUl2SvKXwL2ApwFHAucneX13eV6SmnAQ03xnJnne/J1JngP4UGTNCi/BSxoFL01qM0nuBnwEuAY4\npdu9P7Ar8LSqumCo2qTF4iV4SWPhsya1maq6IMmBwKOZrDhewCeq6jPDViYtqi1egk/iJXhJzTiI\n6SZqcpr0M92HNIvOTPK8qnrP9E4vwUtqzUuTklYcL8FLGgsHMUkrUpKw+SX4M7wEL6k1BzFJkqSB\nuHyFJEnSQBzEJEmSBuIgJkmSNBAHMUmSpIE4iEkapSTvTnLDAh/7LsJ7r+vea/fFqFWSbikXdJU0\nVgV8GnjOvP2XL+L3yHa/QXKrqvrRYhQjaeXxjJiksQpwbVVdMu/jx0mekuSUJNck+VaSP0uy06Yv\nTJ6d5OQk30tycZIPJrlr99oa4N+7Qy/tzoy9s3ttQ5K3bFbE5Mzcx6a2NyR5a5I3JLkEOLHbv0+S\nT0x9zw8kufPU190/yWeSXJXk+0lOS7JuSTonadlwEJM0Zjc5Y5Xk8cD7gDcD+wDPB54B/MXUYTsB\nrwb2BZ4M3AH4++6184Cnd5/vA/wU8LJuu7qPaQvte3a372HAc5PcBfgP4KvAg4DHALsBH536mg8A\nF3av7we8BvjhVv7sklYAL01KGrODknx/avtE4NbA66eeE/ntJH8IvBf4fYCqetfU12xM8mLgjCR3\nraqLklzRvXZJVX13GzWEmw6E36qq3990QPJa4LSqeuXUvucBlyf5uar6MrBXV/d/z73HNr6vpBXA\nQUzSmH0O+M2p7R8CZwMHJDl0av8OwM5J7lxVFyd5IJMzTvsBu3PjILUXcNF21lTc+HzKOfsDj5g3\nNM4de0/gy8AbgXd0A9pngA9X1Te3sxZJy5yDmKQxu6aqNjtz1D0j8jDgHxc4/rIktwY+Bfwrk0uI\nlwB3ZHI27Vbb+H43cNOzXzstcNzV87YDfBx4xQLHXgJQVYcneT/wBODxwGuS/J95Z+8krTAOYpKW\nm1OBn5k/oM1Jsjdwe+BVVXVut+9+8w6bu8tx1bz9lwJ3nbdvP7Z9GfFU4FnAeVV1/ZYOqqpzgLcA\nb0nyVuCFgIOYtIIZ1pe03LwW+NUkhye5X5K9kzwjyeu6188DrgVemuQeSZ4E/Om89ziXyWXDJye5\nY3cWDSZ3Uz6huyvzvkneCNyNzc+SLZQZ+xvgtsA/JDmg+76/kOToJLsl2TnJ3yR5ZJI1SQ5kEvT/\nxiL1RNIy5SAmaawWuluRqvpX4EnAo4D/6j7+gMlwRVVdCjwP+EUmg86rgf87/V5VdSGTDNmfA99h\ncpYK4J1TH58HrgL+aV4dN6mrqv4HeCiTS5ufBL4OHMUk03Yt8GNgNfBu4CzgI8AXgd+9WR2RNHNS\ndZPfc5IkSWrAM2KSJEkDcRCTJEkaiIOYJEnSQBzEJEmSBuIgJkmSNBAHMUmSpIE4iEmSJA3EQUyS\nJGkgDmKSJEkD+f/kYuFMxxXd2wAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x117257f50>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_feature_importance(importances, feature_names, n)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__MAKING A SUBMISSION__"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"test = pd.read_csv('test.csv')"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"# Has Pool\n",
"test['has_pool'] = np.where(test['PoolArea'] > 0, 1,0)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"X_pred = test[X_keep]"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"predictions = adr.predict(X_pred)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"final_predictions = np.exp(predictions)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [],
"source": [
"submission = pd.DataFrame()\n",
"submission['Id'] = test.Id\n",
"submission['SalePrice'] = final_predictions"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"submission.to_csv('submission_adr.csv', index=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2.2. Gradient Boosting"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"http://blog.kaggle.com/2017/01/23/a-kaggle-master-explains-gradient-boosting/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"https://www.youtube.com/watch?v=sRktKszFmSk&t=71s"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"https://www.analyticsvidhya.com/blog/2015/11/quick-introduction-boosting-algorithms-machine-learning/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"http://www.ccs.neu.edu/home/vip/teach/MLcourse/4_boosting/slides/gradient_boosting.pdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__Como difere do Adaboost:__\n",
"- Ao invés de atribuir pesos baseados nos erros de cada amostras o Gradient Boosting usa o erro resultante do strong learner como dado de entrada para o modelo seguinte (weak learned).\n",
"- O multiplicar referente ao weak learner não é função do seu poder de previsão da amostra. Ele é calculado pelo processo de otimização do Gradient Descent."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Idéia**\n",
"\n",
"- A idéia é fazer com que cada novo modelo tente capturar os erros dos modelos anteriores a ele"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Esse video trás uma compreenssão gráfica de como cada modelo tenta capturar o error modelo principal (Strong Learner), e como é feita o split de cada árvore na tentativa de minimizar o erros dos residuos (o que é basicamente a descrição verbal do processo de Gradient Descent)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"https://www.youtube.com/watch?v=sRktKszFmSk&t=236s"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MSE: 0.0490\n",
"MAE: 0.1582\n"
]
}
],
"source": [
"# Fit regression model\n",
"params = {'n_estimators': 500, 'max_depth': 4, 'min_samples_split': 2,\n",
" 'learning_rate': 0.01, 'loss': 'ls', 'random_state': 28}\n",
"\n",
"gbr = GradientBoostingRegressor(**params)\n",
"\n",
"gbr.fit(X_train, y_train)\n",
"y_pred = gbr.predict(X_test)\n",
"mse = metrics.mean_squared_error(y_pred, y_test)\n",
"mae = metrics.mean_absolute_error(y_pred, y_test)\n",
"print(\"MSE: %.4f\" % mse)\n",
"print(\"MAE: %.4f\" % mae)"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"([mean: 0.38320, std: 0.02495, params: {'n_estimators': 50, 'max_depth': 2},\n",
" mean: 0.54869, std: 0.03491, params: {'n_estimators': 100, 'max_depth': 2},\n",
" mean: 0.72030, std: 0.03428, params: {'n_estimators': 300, 'max_depth': 2},\n",
" mean: 0.74325, std: 0.03461, params: {'n_estimators': 500, 'max_depth': 2},\n",
" mean: 0.42821, std: 0.02394, params: {'n_estimators': 50, 'max_depth': 3},\n",
" mean: 0.60224, std: 0.02995, params: {'n_estimators': 100, 'max_depth': 3},\n",
" mean: 0.73390, std: 0.03615, params: {'n_estimators': 300, 'max_depth': 3},\n",
" mean: 0.73905, std: 0.03716, params: {'n_estimators': 500, 'max_depth': 3},\n",
" mean: 0.44862, std: 0.02390, params: {'n_estimators': 50, 'max_depth': 4},\n",
" mean: 0.62202, std: 0.02972, params: {'n_estimators': 100, 'max_depth': 4},\n",
" mean: 0.73060, std: 0.04038, params: {'n_estimators': 300, 'max_depth': 4},\n",
" mean: 0.72808, std: 0.04481, params: {'n_estimators': 500, 'max_depth': 4}],\n",
" {'max_depth': 2, 'n_estimators': 500},\n",
" 0.7432489200534269)"
]
},
"execution_count": 70,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"params_grid = {'n_estimators': [50, 100, 300, 500], 'max_depth': [2,3,4]}\n",
"gsearch = GridSearchCV(estimator = gbr, param_grid = params_grid, n_jobs=4,iid=False, cv=5)\n",
"gsearch.fit(X_train, y_train)\n",
"gsearch.grid_scores_, gsearch.best_params_, gsearch.best_score_"
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {},
"outputs": [],
"source": [
"best_params = {'n_estimators': 500, 'max_depth': 2, 'min_samples_split': 2,\n",
" 'learning_rate': 0.01, 'loss': 'ls', 'random_state': 28}"
]
},
{
"cell_type": "code",
"execution_count": 72,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"GradientBoostingRegressor(alpha=0.9, criterion='friedman_mse', init=None,\n",
" learning_rate=0.01, loss='ls', max_depth=2, max_features=None,\n",
" max_leaf_nodes=None, min_impurity_split=1e-07,\n",
" min_samples_leaf=1, min_samples_split=2,\n",
" min_weight_fraction_leaf=0.0, n_estimators=500,\n",
" presort='auto', random_state=28, subsample=1.0, verbose=0,\n",
" warm_start=False)"
]
},
"execution_count": 72,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gbr = GradientBoostingRegressor(**best_params)\n",
"gbr.fit(X_train, y_train)"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MSE: 0.0463\n",
"MAE: 0.1561\n"
]
}
],
"source": [
"y_pred = gbr.predict(X_test)\n",
"mse = metrics.mean_squared_error(y_pred, y_test)\n",
"mae = metrics.mean_absolute_error(y_pred, y_test)\n",
"print(\"MSE: %.4f\" % mse)\n",
"print(\"MAE: %.4f\" % mae)"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEZCAYAAACAZ8KHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXucXWV56P995pJMMknIhZAQCAQkkIAojCTaWpqptxPA\nn2h7PEqNilDQttqcmCoo9hBKaaU2RVpbiwgaf9y0HlAREIJmIFarwQByGy6ahCTAhJDE3CbJXJ7z\nx7PerDV71t6z18zee/aeeb6fz/rstd691rveNTt5n/U+V1FVHMdxHKdY6oZ7AI7jOE5t4YLDcRzH\nyYQLDsdxHCcTLjgcx3GcTLjgcBzHcTLhgsNxHMfJhAsOp6SIyNki0j7c4xgJiMiTIvKHwz2OYhGR\nXhE5sYjzWkVkcyXG5JQHFxzOoBCRjSLy9tx2VV2rqvOGY0y5iMgKEekSkT0isktE/ltEzh7ucRWL\nqr5eVR8udb8i0hZN8m/Iab8raq8ZYeUMDy44nMGi0VYViEh9SrMCt6vqRGAa8CDw3TLcW0RESt1v\nGVHgWeAjoUFEpgG/B2wbrkE5tYMLDqek5KohopXJchF5PHrrv0NExia+f7eIPCYiO0Xkv0Tk9MR3\nl4vICyKyW0SeEpH3Jr67MDr/n0VkO3Bl2nCiDVXtAW4DpovIkVEfR4jITSLykohsEZGrRaQu+q5O\nRFaKyKsi8lsR+WT0Nh6+bxORvxOR/wL2ASeIyDwRWS0ir4lIu4i8PzHec6Nn2B3da3nUfqSI/DB6\n/tdE5OHENYdXdSIyVkS+LCJbo+06ERmT+JtvEZFPi0hH9DwXDvBT3QZ8ICHwLgDuBLoS9897z+j7\nzyT+dhf1+cPbtf8kIptE5BUR+aqINA0wJqdGcMHhlBsF3g/8D+AE4A3AhQAiciZwE3AJMBW4AfiB\niDRG174A/IGqTgKuAm4RkRmJvhcCvwGOAv6+0CCiCe8jwG9UdXvU/E3gEPA64EzgXcCfRd9dCiwG\n3gi0AO+l/wprSXT+BOA1YDVwCzAd+CDw7yIS1HY3AZdGz3Ia8JOofTmwGTgyeo7PJfpPruquiJ73\njdG2EPhC4twZwCRgFnAx8G8ickSBP8lLwNPY7wLwYeBbOefkvaeILI7G/g7g5OgzyReBk6LrTgKO\nAf5PgfE4tYSq+uZb5g3YALwtpb0V2Jxz3p8mjq8FvhrtfxX425zr24E/zHPPR4H3RPsXApsGGOMK\n4CCwE+gGtgNzo+9mAAeApsT5FwA/ifZ/AlyS+O7tQC9QFx2vAVYkvv8A8HDO/W8A/k+0vwkTRpNy\nzrkK+B7wukJ/Y0yILk589y5gQ+Jvvj+MLWrrABbm+buswYTLh7CVxzzg2ei7zeHvP8A9bwb+PvHd\n3OjvcyK2ytsLnJj4/veA36b9G/Gt9jZfcTiV4JXEfif2hg5wPLA8UtPsFJGdwLHA0QAi8hEReTTx\n3esxW0WgGM+cb6vqFExQPAl8KnHvRuDlRP//ga0WiMaQ7H9LSt/J748H3pzzLH8a3RfgT4BzgY2R\nmustUfuXsAn6ARH5jYhcluc5ZmHCJ/Bi1BZ4TVV7E8f7if/OaSimmnob8Jf0X20MdM/cv8+Lif3p\nwHjgV4m/xX3YqsoZATQM9wCcUUlQv7wIXKOq/dRMInI88DVsYvu5qqqIPEpks8jpp9B9go3jNRG5\nFPi1iFyHTXoHgWk5E27gZWB24nh2yjnJ+78IPKSq70odiOojwHsjI/6ngO8Ax6nqXuCvgb8WkdOA\nn4jIL1V1TU4XLwFzgGei4+OitkGjqp0ich/wCUxdl0vaPbdG+y9HxyS+C2zHXhBOVdWXhzJGpzrx\nFYczFMaISFNiS/NsSiNM/jcCnxCRhWI0i8h5IjIBaMYm5u1AnYh8DFtxZKGPp5OqPgfcDXw2mtAe\nAP5ZRCZGxvDXSeyK+h1gqYjMEpHJwGX0F1TJ/n8InCwiS0SkMdoWRAbzRhH5kIgcoWak3wP0wGHn\ngJMiI/XuqD1NkN0OfCEyph+J2Qv+/4x/jzQ+DyxS1RdTvku75y3Rd98BLhSR+SIynoRzQiSIbwS+\nLCLTo+c8RkRShapTe7jgcIbCvZhKJGxXMrCb7uHvVfVXmGH8K8AO4HkiF1FVfRpYCfwcU3W9Hvhp\nWj/F3CvBl4CPiMhR0b3GYEbiHcB/AjOj827EBMuvgV8B9wA9OauTw31HK4d3YUbxrdgb+T9E/YMZ\n0jeIyO8wW8eHovaTMKP6HuBnwL+p6kMpz/J3wCPReH4d7f9d2liyoKovq+rP8nyd956q+iPgy5gt\n6DngxzljuAxTwf139MyrMSP6kMbrVAeiWp7fT0RuBs4Dtqnq6TnfLcf+Ax+pqjtSrl2M/aOsB76u\nqteWZZCOUyQicg5m1J8z3GNxnOGmnCuOb2DujH0QkdnAO+lrdEt+X4+9gS4GTgUuEJH5ZRyn4/Qj\nUr2dKyINInIMtpq6c7jH5TjVQNkEh6quxdwgc/ln4LMFLl0IvKCqG1W1C7gDOL8MQ3ScQgjmzrsD\nWA88hcchOA5QYa8qETkf2KKqv5b8GRqOob8b5JvLPTbHSaKqndhLjOM4OVRMcESeF5/H1FSHm1NO\ndaOZ4zhOFVPJFcfrMJ/wx6PVxrFYgNBCVU0mVttKf//5tOArRMSFjOM4ziBQ1UEn5qyYO66qPqGq\nM1T1BFU9ARMGLTlCA8zlb66IzInyC30A+EGBfkfsduWVVw77GPzZ/Pn8+UbeNlTKJjhE5HbML/1k\nEdkcBXAl0cS5s0TkHgBV7QY+CdyP+dd/W1WfwXEcx6kKyqaqUtULBvj+xMT+S1jMRzi+D8tt4ziO\n41QZHjlexbS2tg73EMrGSH428OerdUb68w2VskWOVwIR0Voev+M4znAgImgtGMcdx3GckYELDsdx\nHCcTLjgcx3GcTLjgcBzHcTLhgsNxHMfJhAsOx3EcJxMuOBzHcZxMuOBwHMdxMuGCw3Ecx8mECw7H\ncRwnEy44HMdxnEy44HAcx3Ey4YLDcRzHyYQLDsdxHCcTLjgcx3GcTLjgcBzHcTLhgsNxHMfJRNlq\njovIzVgd8W2qenrUdjXwHkCB14ALVXVzyrUbgd1AD9ClqgvLNU7HcWqTdevglltsf8kSWLBgeMcz\nmihb6VgRORvYC3wrITgmquqeaP9TwBtV9c9Srt0AvElVdwxwDy8d6zijkHXr4OKLQaLip6pw000u\nPIqlakvHqupaYGdO257E4QRge4EuBv1QjuOMbG65xYTG1Km2icSrD6f8lE1VlQ8RuQb4MLAfeEue\n0xR4UER6gBtU9cZKjc9xHMcpTMWN46p6haoeB3wTuC7PaW9V1TOBc4C/jNRejuM4gNk0VGHHDttU\nrc2pDBVfcSS4Dbg37QtVfTn6fFVE7gIWAmvTzl2xYsXh/dbWVlpbW0s9TsdxqowFC8ym4cbx4mhr\na6Otra1k/ZXNOA4gInOAuxPG8bmq+ny0/ylgoap+OOea8UC9qu4RkWbgAeAqVX0gpX83jjuO42Rk\nqMbxcrrj3g4sAo4Ukc3AlcC5InIK5mb7G+DPo3NnATeq6nnATOBOMXeJBuDWNKHhOI7jDA9lXXGU\nG19xOI7jZKdq3XEdx3GckclwGscdx3GKwqPEqwtXVTmOU9V4lHjpcVWV4zgjGo8Srz5ccDiO4ziZ\ncMHhOE5V41Hi1YfbOBzHqXrcOF5ahmrjcMHhOI4zynDjuOM4o5JVq+CMM2xbtWq4RzO68DgOx3Fq\njlWr4OMfj4/D/kc/OjzjGW24qspxnJrjjDOgvR3GjbPjzk6YNw8ee2x4x1UruKrKcRzHqSguOBzH\nqTmWLbPPzk7bkm1O+XEbh+M4NUewZVwX1RBdtsztG5XEbRyO4zijDLdxOI7jOBXFBYfjOI6TCRcc\njuM4TibcOO44Tk3i+auGj7IZx0XkZuA8YJuqnh61XQ28B1DgNeBCVd2ccu1i4MtAPfB1Vb02zz3c\nOO44oxAv7jQ0qtk4/g1gcU7bP6rqG1X1DOB7wJW5F4lIPfCV6NpTgQtEZH4Zx+k4To3hxZ2Gl7Kp\nqlR1rYjMyWnbkzicAGxPuXQh8IKqbgQQkTuA84FnyjJQx3Fqkv374bXXbD+kHnEqQ8VtHCJyDfBh\nYD/wlpRTjgGS6qstwJsrMDTHcUpIOW0QLS1www3925zKUHHBoapXAFeIyOXAdcDHck/J0t+KFSsO\n77e2ttLa2jrEETqOM1RybRBr1pTWBrF+PcyeHacbGTfO2jx6PJ22tjba2tpK1t9welXdBtyb0r4V\nmJ04no2tOlJJCg7HcaqDpA0CrOTrLbe48Xq4yH2pvuqqq4bUX0XjOERkbuLwfODRlNMeAeaKyBwR\nGQN8APhBJcbnOE5t0NICmzfD9u22bd7sqqpKUjbBISK3Az8DThGRzSJyEfAPIvKEiDwGtALLo3Nn\nicg9AKraDXwSuB94Gvi2qrph3HFqiCVLzEV2xw7bVK2tEOvWwdKltq1bV/jcoKo68kjbZs+2Nqcy\neJJDx3HKQhbjeNa4jKVLoa2tryqstRWuv75Uox/ZDDWOwyPHHafGqJWI6QULih9bVpvIkiVwzz2w\ndasdT5488IrGKR0uOBynhii3t1IpKbeAO3AA9u61/aam0vbtFMaTHDpODVErEdNBwLW12XbxxYXt\nFlltIitXmlG8ocG27dutzakMLjgcxyk5WQXcggW2cmpttW2gVVR7u302NtqWbHPKj6uqHKeGWLLE\n1FM7dthxMd5KtUIWm8i8efDMM7Gqqq7O2pzK4CsOx6khsr6Zl5t8LrS5qqd9+6CjozhX22I45xz7\n7O21LdnmlB93x3Ucpw/FGrUHcqEN/XR0wCOPQHNz+nmDYelSuPfevilHzj3X3XGLpZrTqjuOUway\nBMoNpu9g1L73XnuL/+AH0+9TrB2jvb08Bv3x42HuXNvGjx96f07xuI3DcWqIcrvjBmHQ0AAvvwzd\n3fDgg/D009nus24dXHAB7NpldojeXjjiCJg4sTTj9DiO4cVXHI5TQwzWHTfrKqWjwz4bGixGIu0+\nhVxoV660Sf3AAbu2uxteeKH49CNOdeMrDscZ4WRZpQSvrQMHbLJvaICZM6Grq/+5wVCfZg9pbzcB\nEdxle3vts7U1WzBgPntLEKAh8C8ItmoMhByJuOBwnBpiMO64WdJ5BGGwciX86EfWtnFjflVQPhfa\nefNMeASBIwJvf3s243UhgdfRAS++GHtU1dXFqySn/LiqynFqiFK44+7fDw89lF9ttWABLF9uWWcb\nBvlquXw5HHusrQiammx/+fL+561aBWecYduqVX2/K6SWe+01OHQIenpsO3QoLiPrlB9fcThOjZEl\nUA76rlL277faFbNnm+dUPrXVUFVBCxbA3/wNXHedHS9b1v/aVavg4x+Pj8N+MVX8Xn0Vxozp3+ZU\nBl9xOE6NkdXQnVylNDeb0Dj22MLG9Y4O2LLFvKJ27bL9LKqgdetM3aVq28qV/ccahMq4cbYl26Cw\n8X3ePBt7sKGIeOR4JfEVh+PUEIN1xw2rlBA49/zz1h4m7Nx7rF9vBnKwe9VlfMUsRenYQsb35cst\nqHDXLjuePDldFeaUBxccjlNDDHVCbmmBG27o3xYIgumll+I2VbMjlNqGsGyZqadC9HdoS5JPLbdg\nAdx+e23UJRmJuOBwnBFKmitrKLmaTNWxfn1sVwiCKXgrQby62bSp+HsXE6AX7pm0gxRj3whktfU4\npcMFh+PUEMW64yYjt8Em8dtvt/3x483GAXE/Sfbv7xu3ESLJi03rEewbr7xi/SSN7Ll89KPZhIVT\nHZTNOC4iN4tIh4g8kWj7kog8IyKPi8idInJEnms3isivReRREfllucboONVCMQbvsII49VSYP7+w\nO+7KlRbn8Lvf2fbii9Y2UMGklhaL2+jujtvq623yz1Uj5RvjxRdbDMi+feYm291tHk9eaGnkUM4V\nxzeAfwW+lWh7ALhMVXtF5IvA54DLU65VoFVVU96HHGdkUYzBO3nO/v2wcye84x35+1y/3t72g1G7\nt9fa0gzOYAIL4Oc/7ys0AMaOha98JX1lsGpVX1XT+vU2xkOH4nNC6vP164v7ezjVT9kEh6quFZE5\nOW2rE4e/AP6kQBeDTvnrOLVEMQbvcE5jY/7kg0mbRsgRFUjuJ20DuULrqafsMwT+dXebmimf0MiN\nwzj77PRnVIVt20xAlcqQXe6a5k5+htPGcRFwe57vFHhQRHqAG1T1xsoNy3H6Uk0T1Cuv2Nt7T48J\nh87OeGxJm8aBA6ZiymXpUjNUf//7dnzUUdZHMJbX1VnfSeP4lCnpY7nuungsYPfbtMkEzZgxcZ8i\nJjigcNBhFsqdJdgpzICCQ0TqgA8BJ6jq34rIccBMVR207UFErgAOqepteU55q6q+LCLTgdUi0q6q\na9NOXLFixeH91tZWWltbBzssx+lHJSaoYgze4Zy9e21CFrEJe/NmC8wL2WiDsOjqMhVTU5OtGvbt\nszf+m26y/TFjrI8nnjBhMXZsfO/k6qSuDo4/3v4Ouc8cjN+B3l4TWLfeauP54Q/t+5As8XWvg0mT\nBhfTkUsp4kRGE21tbbS1tZWsv2JWHP8O9AJvA/4W2Bu1nTWYG4rIhcC5wNvznaOqL0efr4rIXcBC\nYEDB4TilphITVKFAt+Rq533vi5MENjaakAhv++3tfVcKqjBrlhViuu++2OC9f79939UVZ63t7Y0F\nR0MDnHmmrVw2bLB+1q61e991V99xpaX4OHgwznUVAvT27o1XHM7wkPtSfdVVVw2pv2IEx5tV9UwR\neRRAVXeISONgbiYii4HPAItU9UCec8YD9aq6R0SagXcBQ3tKx6ki0lRfaTEJuQbxF1+09rDaOOII\n22bMsOC84PoKNlEff7wJmocespVDY+J/rWpsBK+vN/UVWFzHEUfAM8/E3/f02Grm85+H1ZGVMl8N\nkIMH4+9DepM9e+DJJ02NdcwxpanHMZgswU7pKMYd95CIHNaWRuqj3gLnh/NuB34GnCIim0XkIszL\nagKmfnpURP49OneWiNwTXToTWCsij2EG9B+q6gOZnspxSsRA7qtZyVqatbPThMLWrXE22CA4Dh2y\niX7Jkr5pQYKQmDbNjkNep6RggXgV0NRk506bZvd7+GFbJaSNPYmkuK9MmtS/beLEOD/WYDP65lKK\nLMHO4BEdYA0pIkuA/wW8CVgF/E/gC6r6nfIPrzAiogON33GGSimN40uXmtBoaIBnn7W3+gkTTK2U\nO/l98INw5519VweNjaZW2r/f1Gf33mvt55wDu3fH502fDn/0R7Ya6eiAn/7U7A9798YCpK7ODN/h\nPDC11qZNfd1pA0ccERvf162Dd77TYkQCdXXw/vfDHXf0tw2p+uReTYgIqjpoz9UBVVWqeouI/IrY\nJnG+qj4z2Bs6Tq1RjtQW+UqzJu8T1E9JenpsEh4zxgojhcSF48b1Vdts327CIqw6mprgD/7A+nz4\n4XhC/93vTOgEd9v77kuv9gdw8snxfrjv3/99bPyeMSNONBjsHCHG49OfdqExkijGq+otwNOq+pXo\neJKIvFlVf1H20TnOCCPkcNq+3d7q6+rsc8eOuLhSWNU8+2z/61VNCORmg92zx/pStc+6OlthBKM+\n2MQ+Y4YZypO5qu67Lw7OmzIlvyE72EHAVhQ33xwb43tzlNch7UgQUCtXWsS7C4+RQTHG8f8Azkwc\n70tpcxwnAw0NZkju6TH1T1eXrQba2y2w7vrrLTo8l/r6OGI8qM9aWuDf/i22fwThkkZHh/Xb1GRC\nZPduSw+SDPg76ijzmAoCpK7Ovt+2Le5n5UrLoBsElaqpwcKqyd1lRzZFBQAmDQmq2pM0ljvOaGEw\nto7ca4K30eTJNtF3d8dv62EVcPBgnAIkl7FjLVpcxDyf1qyxN/mjj7aI8uSbf1OTxWmATe533WWC\no6fHBMFrr5kgCq65EKupJk0yNZZIXGkvWSipvd0+g60kBCQ6o4NiBMcGEfkr4KtYGpA/B35b1lE5\nTpUQJv6ODotLaG629nyBgElB0dLSV10TJvlAfb1taRPu/v0wc2Z/76ZQtS/5Jt/ebkKisbGvIAoc\nOGBBf3V19p2IZbqtq7Pzu7tjwRFWHC0tthIBEzK5qrF580xwdXXFdpemptjjzN1lRzbFCI5PAP8C\nfCE6/jFwadlG5DhVQtIzaOtWm3xPPTV/9HOuJ9Gtt5rNIDeFuarZFl591SbqNJuCiAmOzZv7th88\naNX7gn0keEU9+GAsMOrq7Pq9e81jK9fzSdVsIkccYc9y4EBfg3hLS+wZlW+FFQL8tm+365uaTL0W\nzikU1OjUPgO641Yz7o7rlJPgOjt1qk3W27fDkUfC3LlWg7u5GRYtiifFUJY1qJz27rW39dNPt+Md\nOyzmYMkSW4mEN/rOzv7ur5Mm2WSctCvk4+KLLTjwxz+OYzgOHjRB0tAQu+g2NMSR4hDHb4jEY548\n2ep2DEYN54KhdiibO66IXKaq14rIv6Z8rar6V4O9qePUGjNnxm/XW7bYRD1unO2HIkkdHXYcUoEc\nOmT7W7aYaiioaxYsMMP09Ok2YY8ZYwbrUHmvvh7+9/82V9di+P73zTV3zJg4V1V4nwopScJ+WA1N\nngwnnmhCZf78OI4jiwDwCnyjl0Kqqqejz0dy2gXLXus4I5pcPf2xx8JZZ5nragjK6+42wRCKFPX0\nxHp/MOGycye86U2m3gkTbVLIBFTjuI5bb+076Q/EjBk2vrByCFHmwVuqp8dWIhMmWP8nnWTtO3bY\ntSEHluMUQ17Boap3R95Tb1DV5fnOc5yRRK76Jamnb2kxoRHcZJMG5fb2OLVH0jjd02N2iBkz+r+d\nB6+qsB/Yt8+M48VqYc8/PxZyobzrnj0mRMIYRey85ctNteVGa2coFDSOR663bxU3JjijgHwp1K+/\n3mIrglfRwYM24YeaFyKxq2ru/5Kurjj1eWDVKrNvhAp9wWANsQBR7Zv9NtDQYO0HD9p9J06Mvb2W\nL48D+Vpa4Oqr4xQhwSvKjdZOKSgmV9V/ALOA/wSipMyoqt5Z5rENiMszp5QkjeEQG7NbWqy6XfBk\ngniib2w0g/ldd5m66s47+3pKBQ+nWbPg938/TvnR1RWfM3ZsnFU2yZQp5hGV9JY66yx49FG7VjXO\nknv88f3zQbnx2slH2XNVAU3ADqweR5JhFxyOUyrWrbOUH1u32lt9yPLa0WFv6ocOxRN1IGkzABMK\nIaYhECb9zZvhu9+NA+6Ct1PoM2110djYN7FgCOjLzSVVV2fCbssWuOSS2NMrydNP91e5gQsUZ3AU\nXHFEKdTnAC+oakoChOHFVxxOKQgqqs5Om+BFzNA8bpzFbTz4oE3euRP79Olw2mm2Mpk/3yre7dtX\n3D0bG+PCSyFwL/efcpowSaO5GU45xYRDc7PVvAjjaG42e8nmzZbaHGKPsBDYV6z7rTNyGOqKI289\nDhH5M+ApLPivXUTOH+xNHKeaCXmVjj0WXv96m2ybm03tA+nR2I2NcSoOsDf4YoUGxKsMsAk8rbZF\nFq+qTZvs/Lo6W/ls327CburU2NOqs9NUX4cO2ViDa3HwCHOcYimkqloGnBaVbz0RuA34fmWG5Til\np5DOf/9+m3DBbAuLFtn+I4/EEy/YBD99elwS9Ykn7K29Lu8rWH+Cy21XlwmrV1/tL5iy9DVrlu3v\n2WPPsX+/CYUJE/qfH0rHhkDB4BHmOFkoJDgOqeqrAKr6WxEZW6ExOU7JyecxBabi2bAh9pACswOE\nlUhdnX0XIrHf8Ab42c/iVB6HDsGb32zR5QNRX2/BhNOmmWpq6lQzjKfV7x6IKVOsn3POMVtMKC0L\nsafWjh2mlgL73Ls3fqZQ0CmZvNBxiqGQ4DhWRP4FC/gDOCZx7JHjTk2RluZ75UpbUWzZEr/xT51q\nBulgPN650yb7pqY4jcfatX29oPbsgd/8xs4ZKENsT49lsQ2p1E84wWwPaYIjpEhPo77ehEYoHXvL\nLdZPss7Gm94UR4QHg3iyGiD0T17oOMVQSHB8hr4R4r+Kjj1y3BkRrF9vRuPgGtvTEyf/A5uQb701\ntnEcONA/wC+wYYNllS0mtXhPT2wPaW+363KFRLI+Ri4zZ1oOqy1b4LOfjVVuyQBA1b6R6hBX+auU\nm667A49cypbkUERuBs4Dtqnq6VHbl4B3A4eA3wAfU9XfpVy7GPgyUA98XVWvzXMP96oaBZRiAkqr\ngb1rV+xFlfxnNHYs3HCDTbQh8G/37vwlVQNjxqTX6k6jrs4m+f374xxTBw7YOETMPjF2bP8kh2PH\nxq7B4Z5f+1o81lCqddmyWFAMB15zvLopm1dVCfgGsDin7QHM4P5G4Dngc7kXRWlOvhJdeypwgYjM\nL+M4nSomTEBtbbZdfLG1ZSVETLe22nbTTXGt7yT19abyCaqqU0+FM88cWGhANgN58r69vXE1wN5e\n+zx40GwpuR5XSaEhYsfXXBOXag2BgStXDu7vVCqSqsFQQyQIf6f2KaoC4GBQ1bUiMienbXXi8BfA\nn6RcuhCLG9kIICJ3AOcDz5RloE5VU84SpC0t5saaFAohtxPYxHvBBf1rYqRRV1ecjSPQ2xsXaerp\n6e96G/qZMye2W3R1padZ37kzvw1nMFlvnb64yq0/ZRMcRXARcHtK+zFA8r/qFuDNFRmRM2JJ86oK\nxYheeSW2ORw4ABs3xtX7tmwpbrURckiFKnv5CB5NIma/yD1fJB7jpk1m5A4qre3b+6rDwsrjhBPs\nM+lSLGKBi8ccEz9vJVVFI6UCYD5vvNEuPArV40irwxEYkleViFyBufveltZ3lr5WrFhxeL+1tZXW\n1tbBDsupQko1AYXyr6GfqVNNHfU3fwOf/GTfc7u7zSge3u4bGoqzb5x44sDqoSA4xo5NDxgMqqb6\nehMYN95oY3/oIVOh/e53fT2wGhrsvi0tZpdJPsPRR5dnpVYMIyWZYjlXvJWkra2Ntra2kvVXaMUR\nvKggdskNDNoiLSIXAucCb89zylZgduJ4NrbqSCUpOJyRR5YJqJBK4ec/76vm2bYNVq82oRSC4pL8\n8peweDGKCGxCAAAcAElEQVQ8+2x6VHcuqnHhpkIR3yEWJAiIsMLI9fHo6bE06KFYUkjA2NkZG8gb\nGkwVNWOGCcGkO+7evX0DF4cDL/RUPeS+VF911VVD6q9QPY5vDqnnFCJvqc8Ai1Q1nzb4EWBuZB95\nCfgAcEGpx+Jkp9y63nz9FzMBDaRSeCbFQvb882ZDSFMt9fSYKuunP7W4C7CVwrRppg7KvSbkhBoo\nTUgQEkFgJAVHU1NsYJ8yJU5uCPHKKxjEQx+vvhrHaIwfH9c337LFbB+1rioabkaKyq3UDGjjEJGj\ngM9iHk5RDCqqqrnZcnOvux1YBBwpIpuBKzEvqjHAarH/4T9X1b8QkVnAjap6nqp2i8gngfsxd9yb\nVNUN48NMuXW9Q+1/IJVCmtG6u9viL9KYMsU+m5rMYN7VZeqo446zCTlXcIwda0Lo+ecLB+6NGROX\nhw0pPyZMsPOPOiqe+MNEFQgrr0susViTMWNsGzfOhEbuBDduHHzhC54Fd6iMFJVbqSnGOH4r8G0s\n/uLjwIXAgAkSVDVtlXBznnNfwmI+wvF9wH1FjM2pEOXW9Za7//r69GC6QiuElSttJXHaaaay6uqC\nJ5/Mf838+fD44yZYgmst2AQfEhCGCPTXvc6+27TJhMWyZXa/3Dfb3FXYokVxqhKIz883wQ1nLMdI\nwVVu/SlGcExT1a+LyF+p6kPAQyKSW4fccYaVgVQKDQ3pxZLy8fLL5pU0bpz11dgYB+vV1aULj2Cg\nTkaXjxtn1yxYYGqu9nYziod6H8ccY8Lgox+1mJHkxA/pnmD5ntMnOKdSFCM4QizsKyLybszuMKV8\nQ3KqkXLretP6b2kxo3D4HvKrDAZSKTQ3Z0t73t1tk/5LL8WqKlVr2707/ZpgoN6ScOU4cMCEzLRp\ncMcdsUqumIl/6dL+q7D16014hAjxT3/ahYVTeYopHfv/AWsx76Z/BSYBK1T1B+UfXmE85UhlqaRx\nPMRRhLftZGEiiFNYQOxqC+ZhlDa2GTP6B88N5AEVclZNmGCTdmdnfvvFUUfBBz9opWOTgiOcP3cu\nPPdc/+cMY01rSytlO3++ZfP1VB7OUCh76VhVvTva3QW0DvZGTu1TSlVI2kSZ7D/3bXvrVvsMVexC\nZPTTT/ev3JdmWE9TUw30zrFnj9W6mDHDgu8KXXPwoGWaTQqNYu4B+R0D0lZhMDLiCpzaphivqm/k\nNCmAql5UlhE5I55SeWi1t1sfnZ1xNtnOTnNLzZ1M01xuByqeFKrpvfRS/nPDiuKoo6wEaz4mT7bP\ntGc/9dR0YXD99f3Vb7fcku5a7DiVpBgbxz3EAX/jgPdhdg7HGRTFeFDlvm2HiTf59j1/fjyJ9vaa\nHWLXrrhwUZIs9g0woVFXN7BBXdWE1hVXwF//df7zQkxG2rMXqsCXtsrzuAJnuClGVfXd5LGI3Ab8\nV9lG5IwKknmV0ib6NGM39D1++mnzfDp4MI6SFjG1VUtL3/6ylmYNWWrr680r6uDB2J03aeeoq7OY\nj1NPtZxRQaWVhXnz7FmKEQYeV+BUA4NJcngyML3UA3FGD5MnW+BdeFsX6T/RQ/+37VWrLGdT6OOu\nu2zS3rzZJvBJk8x4HoLihhrDMGECTJxowiBpqwgxIaHU7K5dZm858US7b5rB/cABU1Ol2S1CBb5i\nhYG73TrDTTFeVXuJVVUKdACXq+r/LfPYBsS9qmqPdeusRvaePfEqYOZM+OM/Np1+Platgksv7RvE\nN2sWnHSSRWu/+qq5zU6e3L9s6pIlsHBher/B1TaXcePgq1+Fq682wRTuO368Tfb79sXpQXp7zWvq\nnHPg3nstBqSzM36+hgbz0Jo1q68nWBibCwGn0lTCq2rCYDt3RhfFuOuG7xsb40m7mGR811xjOZqS\nk/W2bSY4Jk40r6vubttUrc9p0+zcNWvy93v88fDb3/ZXZR15pK1Y7rvPVhRNTSbguros2juMIUlY\nTRx9tHlXhUSEjY12n+7u2OjtwsKpZYrxqvqxqr59oDZndJPFU2rKFHsrP3DAJtfu7nRVVZKdO+0z\nucA8dAieeMKuHzPG3urHjLHMsAcO9E/LkUa+BWtTk33OmGHR3cm+Jk2y9OZJjj++r/2ho8PUVtu2\n2Zg6OtJtOY5Ti+Qtdiki40RkGjBdRKYmtjlYsSXHOUyxpUKXLLEJdMqUWEU0ffrApU5DsaKQjhzs\nTX7vXtsmT7acUnPnxq65xZJ7fn19LMiWLLH77dhhmyqccordu6EhXlGE1c2CBbaiuOMO87Q6cMAi\nzbdvj43269ZZnMrSpcNb3tVxBkuh/2IfB5YCs7DaHIE9WE1wx8lMMstrZ6dllJ04ceBAtmB47u2N\nBUdvr73Nq5qH1pYtZoNIc93NF/F98GB63fFzzuk73txYio0bYxXbuHGxPSVJbo2MceNM9ZWMiPeK\nck4tUqgex5eBL4vIp1S1UDVAZ5RQyIYxUC6rtCyvbW0mNIphxgxbdXR2ms0hpP8INcLHjLH25mar\n6pebMHDRov62FJH+kd4hdXrSKyvNi+mee+JU7U1N+d1nkzUyQsyGR347tU5eVVUCFZHDSQ1FZIqI\n/EUZx+RUIcGG0dZm28UX91WzhDfz1lbbkm/Rade2tPRXAeWbfNetMxvBzp321n700dZeV2fqrp4e\nWzkENdbKlfb99dfHhuiZM/v3W1/fv6272zy0Hnpo6GqkNDXXvHnx97t3m1G/FPdynEpSjOC4RFV3\nhoNo/9LyDcmpRgayYRRajaRdu359uqDJ1f8HofPMMzbRb9hgE/u7322rlaYmK6JUX28G6nz2lTQD\neVp9jt5eM3y/9lp/4Zh8nuZmOP1025qb0+05acJ0+XITIFu2WNDf3r3m2pvvXo5TjRRjRqwTkTpV\n7QUQkXqgsbzDcmqJweaeSgvwC8FwU6b0zeN06JAJDFWLAbn/frjsMlNPPfRQ3xoXaWSpxSES2z7y\nqZEGinzP94wQ23iam4u38ThONVHMiuN+4A4RebuIvAO4A/hReYflVBtpapdkGpBCq5FC1wbWrTOh\nEVxpQxBdyOO0ebN91tXFXlDf/76pom680SbvQv2neVqJpKurwARDPlpabDzbt/f1liqWBQvM5nLM\nMcXbeBynmihGcFwGrAH+HPgE8CBWg9wZRRSyYZTi2iBognsrmE1j3jwTBCGwD8yAndv/8uUmCERs\nP7f/YBdJMnlyenqQkAoln90leEsdeaRts2fHtb2LpRhh6jjVSjGR4z3AV6MNETkb+BfgLwtdJyI3\nY3XEt6nq6VHb+4EVwDxggaqm/ncTkY3AbqAH6FLVPAkjnEqSL0dSMdUBi8mvFAIDu7pi+0NQXX3+\n83aP+vo4/mPZMvtct85Sg4QMtFdfbSqu5P2uuMLUaUFQ1NdbWpL2dls1BA+pujrzhFq8OF0ABXK9\npbLiyQqdWqaoUCkRaQEuAN4PbASKyVP1Daxi4LcSbU9gadlvGOBaBVpVdRD/JZ1KU2gSLLZqYDJd\nR4gSX7kyPn/1arOBhJKpy5bF7rIrV5qxOaijtmyxtjvu6HuPZLxGWJ1MnWquvk89ZXaUxkY7LiQ0\nSlVG15MVOrVK3iSHInIKJiw+ALwK/CfwGVU9rujOLcr87rDiSLSvAZYXWHFsAM5S1dcG6N+THFYx\nuUbzgcqcZi1NG86/7TbzhAplZTs7TcX12GPxuWecYauLYMju7LR4jaYmK9QUki5OmWKpS8491+wn\n+cZU7jK6jlNOypnk8Bngh8D/UNUXo5t9erA3yogCD4pID3CDqt5Yofs6JaSYgk1JsryBJ4VSV5dt\ne/bYcV1d33iJgGpfo/f48WZY/9CHTM3V2Gjf79ljcSOFvMV8teCMZgoJjj/GVhwPi8iPsBXHoCVU\nRt6qqi+LyHRgtYi0q+ratBNXrFhxeL+1tZXW1tbKjHCUE964OzrseMaMyr55J4XSoUO24ujpiSv3\nhZQhgfPPh8cf79+2YIF5RL34YuxhFRaxWQWf41QrbW1ttLW1lay/QilHvgd8T0QmAOcDy7CEh18F\n7lLVB0o2iv73fjn6fFVE7gIWAgMKDqcyhDfxzk6zJ6iaZ1Fu/MaSJZaaY+tWO548uTyeQ3v2mKdV\nsh5HbiGnXbvMmB3sJ1OmxMb0GTP655RKyz3lOLVK7kv1VVddNaT+BnTHVdW9qnqrqr4bmA08Clw+\npLsaqasXERkvIhOj/WbgXZhR3akSwpt4Z6e9pTc0xLmj0iKoS0FuRHnSnfXAAdsPNo58TJ0aq5jC\nKgLijL3Tptk2bpy1ucus46STKQF15OX0tWgriIjcDiwCjhSRzcCVwA7M0+pI4B4ReVRVzxGRWcCN\nqnoeMBO4U0yx3ADcWs7VjVM+QmqO2bPteLCqnny2huDJ9fTT8PDDfWtk5AbkFfKEKuQVNlRvMccZ\niQym5nhRqOoFeb76Xsq5L2ExH6jqb4EzyjUuZ+iESXjcOEu7oWr7qjZhL11q5wX7x1DJZ2toabF0\nI1u3mooqBA6mqaoGGzeRZgQfbIoVxxkplE1wOCOX3Ep3YDaBlpa+tSb27et73VBVPbt32/0OHICf\n/xxuiKKBurrMlXbePMuCmy8gL58nVFZB4EZzZ7TjgsMZFGmT8NKlfSdUgPnzTagEARPe+LOofJJG\n9uDx9PjjsV0jVALcsCEu7JRFQBUrCMJYH3rI3HaTz+k4owkXHE5ZCSuRZNbbe+6Bs86y40ceiY3a\n+d70Fyyw83ftsoC9mTPhySfjtCQhv9XEiZYLK4vNYd26WN3V0JA/w25yVbJ/f5x0cfz46jGau93F\nqRQuOJySkWaAnjwZPv5xUyeNHWvqq56e2BV23z7LKzVpUmGVz4wZlk02vOVPn24R38GFtq4O/umf\n+to1BiLpVrx3rxnZjz029qpKkpsBGEzgLVpUHZO0212cSuKCwxmQYt9kcw3QwVAeamGEGheqNtHv\n32/Be5s3w2mnFR5DrlCaMgUuushSq0Pf3FX5xt7SEmexDbXDOzttO+IIG19zs0WTDzThjh9vQuP6\n6wufVync7uJUEhccTkGyvskmbR8f/KAF5wV6e+2zri5OD9Lba5Pcli3pb/rJftO8ogrFMeWql264\nwVyDx4+355g504RWSI7Y3W0G9sFmAHac0YILDqcgQ3mTbW+3STnU0lC1vsaNs7b6etsfM6a4N/2s\n+aGSYw/V+jo7TR21Ywds3Ng/Y26he1dzGnQXbE4lccHhlI158+DZZ00wdHWZbePYY+GUU+DRR83Q\nPWOGCZFFiyo/EYeaGsWmGqnmxIbVLtickYULDqcgQ3mTXb7cvKa2bzfB0dhoBvING8xo3twcr0bK\n8XacHHtIpx5KzKrCpz9tcSfjx2d/tmqkmgWbM7LIW4+jFvB6HJVhKG6e69bBJZeYDWPOHHOZ3bEj\nju8YTJ9ZKGQcX7DAXVid0clQ63G44HDKztKl0NbW107S2lo9HkmOM9oYquAYMDuu4wwVzzLrOCML\nX3E4FSFNJeRqIscZHlxVVcPjHy3kExpZ6pE7jlM6yllz3HGGTL4AQo90dpzaxQWHU1byCQjHcWoX\nN447w4IbzB2ndnEbh1NWCtky3DjuOMODG8drePyjhWoRENUyDscZbqpWcIjIzVgd8W2qenrU9n5g\nBTAPWKCq6/Ncuxj4MlAPfF1Vr81zngsOpyjci8txYqo5APAbwOKctieA9wEP57tIROqBr0TXngpc\nICLzyzVIZ3SQW4hJxI30jjNYyiY4VHUtsDOnrV1Vnxvg0oXAC6q6UVW7gDuA88s0TMdxHCcj1ehV\ndQywOXG8JWpznEHjXlyOUzqqMY4jk9FixYoVh/dbW1tpbW0t8XCckYDXq3BGM21tbbS1tZWsv7J6\nVYnIHODuYBxPtK8BlqcZx0XkLcAKVV0cHX8O6E0zkLtx3HEcJzu1nHIk36AfAeZGQucl4APABRUa\nk1ME7tbqOKObcrrj3g4sAo4EOoArgR3Av0ZtvwMeVdVzRGQWcKOqnhddew6xO+5NqvoPee7hK44K\n426tjlP7VG0cRyVwwVF5vCiT49Q+1RzH4TiO44xAXHA4mXC3VsdxXFXlZMaN445T27iNo4bH7ziO\nMxy4jcNxHMepKC44HMdxnEy44HAcx3Ey4YLDcRzHyYQLDsdxHCcTLjgcx3GcTLjgcBzHcTLhgsNx\nHMfJhAsOx3EcJxMuOBzHcZxMuOBwHMdxMlGNNcdHBZ4o0HGcWsWTHA4DXkXPcZzhxJMc1iC33GJC\nY+pU20Ti1YfjOE61UzZVlYjcDJwHbFPV06O2qcC3geOBjcD/UtVdKdduBHYDPUCXqi4s1zid0uHq\nN8cZHZRzxfENYHFO2+XAalU9GfhxdJyGAq2qeuZIFBojsYpeUL+1tdl28cXW5jjOyKNsgkNV1wI7\nc5rfA6yK9lcB7y3QxaD1b9XOggVm02httW0k2Ddc/eY4o4dKe1XNUNWOaL8DmJHnPAUeFJEe4AZV\nvbEio6sgCxbUvrBwHGd0MmzuuKqqIpLPJeqtqvqyiEwHVotIe7SCcaqUJUtgzRpTvUF51G9uQ3Gc\n6qDSgqNDRGaq6isicjSwLe0kVX05+nxVRO4CFgKpgmPFihWH91tbW2ltbS31mJ0iCOq3ck3suS7M\na9aMDBWf41SCtrY22traStZfWeM4RGQOcHfCq+ofgddU9VoRuRyYrKqX51wzHqhX1T0i0gw8AFyl\nqg+k9F+TcRxOdpYuNaP71Kl2vGOH2Yeuv344R+U4tUnVxnGIyO3Az4BTRGSziHwM+CLwThF5Dnhb\ndIyIzBKRe6JLZwJrReQx4BfAD9OEhuM4jjM8eOS4UxN4tL3jlI6hrjhccDg1gxvHHac0uOCo4fE7\njuMMB1Vr43Acx3FGJi44HMdxnEy44HAcx3Ey4YLDcRzHyYQLDsdxHCcTLjgcx3GcTLjgcBzHcTLh\ngsNxHMfJhAsOx3EcJxMuOBzHcZxMuOBwHMdxMuGCw3Ecx8mECw7HcRwnEy44HMdxnEy44HAcx3Ey\n4YLDcRzHyYQLDsdxHCcTZRMcInKziHSIyBOJtqkislpEnhORB0Rkcp5rF4tIu4g8LyKXlWuMjuM4\nTnbKueL4BrA4p+1yYLWqngz8ODrug4jUA1+Jrj0VuEBE5pdxnFVLW1vbcA+hbIzkZwN/vlpnpD/f\nUCmb4FDVtcDOnOb3AKui/VXAe1MuXQi8oKobVbULuAM4v1zjrGZG8j/ekfxs4M9X64z05xsqlbZx\nzFDVjmi/A5iRcs4xwObE8ZaozXEcx6kChs04rqoKaNpXlR6L4ziOUzxi83eZOheZA9ytqqdHx+1A\nq6q+IiJHA2tUdV7ONW8BVqjq4uj4c0Cvql6b0r8LGcdxnEGgqjLYaxtKOZAi+AHwUeDa6PN7Kec8\nAsyNhM5LwAeAC9I6G8qDO47jOIOjnO64twM/A04Rkc0i8jHgi8A7ReQ54G3RMSIyS0TuAVDVbuCT\nwP3A08C3VfWZco3TcRzHyUZZVVWO4zjOyKMqI8eHGDy4UUR+LSKPisgvKzfq4sjzbO8XkadEpEdE\nWgpcW/WBkUN8vqr+7SDv831JRJ4RkcdF5E4ROSLPtbX6+xX7fFX9++V5tquj53pMRH4sIrPzXFur\nv12xz5ftt1PVqtuAs4EzgScSbf8IfDbavwz4Yp5rNwBTh/sZMj7bPOBkYA3Qkue6euAFYA7QCDwG\nzB/u5ynV89XCb1fg+d4J1EX7X0z7t1njv9+Az1cLv1+eZ5uY2P8U8PUR9tsN+HyD+e2qcsWhgw8e\nDFSt0Tzt2VS1XVWfG+DSmgiMHMLzBar2t4O8z7daVXujw18Ax6ZcWsu/XzHPF6ja3y/Ps+1JHE4A\ntqdcWsu/XTHPFyj6t6tKwZGHYoIHweJAHhSRR0TkksoMrSKMhsDIkfDbXQTcm9I+Un6/fM8HNfr7\nicg1IvIi5un5xZRTavq3K+L5IONvV0uC4zBqa6t8Vv23quqZwDnAX4rI2ZUbWVkZDV4MNf3bicgV\nwCFVvS3l65r//QZ4PqjR309Vr1DV44BvAtelnVLZEZWWIp4PMv52tSQ4OkRkJkAUPLgt7SRVfTn6\nfBW4C1tmjgS2AknD1mzszWfEUMu/nYhcCJwLfCjPKTX9+xXxfDX9+0XcBixIaa/p3y5BvufL/NvV\nkuAIwYOQJ3hQRMaLyMRovxl4F/BE7nlVTj494+HASBEZgwVG/qBywyoZqc9Xy7+diCwGPgOcr6oH\n8pxWs79fMc9Xq7+fiMxNHJ4PPJpyWi3/dgM+36B+u+H2BMhj4b8dixo/hOkWPwZMBR4EngMeACZH\n584C7on2T8Q8Hh4DngQ+N9zPUsSzXYQZ+jcDncArwH25zxYdnwM8i3l4VN2zDeX5auG3K/B8zwOb\nov+UjwL/PsJ+vwGfrxZ+vzzP9l1sknwM+L/AUSPstxvw+Qbz23kAoOM4jpOJWlJVOY7jOFWACw7H\ncRwnEy44HMdxnEy44HAcx3Ey4YLDcRzHyYQLDsdxHCcTLjicEUGUsv1REXlCRL4jIuOG0Nc3ReRP\nov0bRWR+gXMXicjvDeIeG0Vkap72X0epsO8XkdScbCJyj4hMynpfxykFLjickcJ+VT1Trb79IeAT\nyS9FJEuZ5MO50FT1Ei1cgfKPgN/POljy5z9SoFVV34hFLH8++aVEqOp5qrp7EPd1nCHjgsMZiawF\nTopWA2tF5PvAkyJSFxUl+mX0Rn8pHJ6MvxIV6lkNHBU6EpE2EXlTtL9YRH4VFcVZLSLHAx8HlkWr\nnbeKyHQR+W50j1+KyO9H104TK0D2pIjcSHEprMNzHC8iz4rIKiwKeHZyxSIiH0kU6/lW1JY6Dscp\nBVnewhyn6olWFucSp/4+EzhNVTdFgmKXqi4UkbHAT0XkAaAFKzQ1H5iJ1bq/KbpeARWR6cDXgLOj\nviar6i4R+Q9gj6r+c3T/24DrVPW/ROQ44EfAqcCVwMOq+ncici5wcaHHiD7fDfw62j8J+LCq/jK6\nj0afpwFXAL+nqjskrox5fZ5xOM6QccHhjBTGiUhI4PYwcDPwVuCXqropan8XcLqI/M/oeBIwF6uc\ndpta/p2XReQnOX0L8BZs4t8EoKq7cr4PvAOYL3K4aWKUOO5s4H3RtfeKSG6hsiRrRKQHeBxTVU0F\nNgWhkXPftwHfUdUdOeNKG8d4Vd1f4L6OUxQuOJyRQqdaPYHDRJPmvpzzPqmqq3POO5eBVUfFJnUT\n4M2qeihlLMVWWGsNgiC6dir9nyM5rrR+U8fhOKXAbRzOaOJ+4C+CoVxEThaR8dgK5QORDeRozOCd\nRIH/Bv5QROZE1waPqD3AxMS5DwB/FQ5E5I3R7sPAn0Zt5wBTSvA8CvwEeH/C3hH6zR3HGSW4n+MA\nLjickUPaiiC3UuTXMfvFehF5AvgqUK+qd2Gpw5/G6tn/rF9HqtuBS4E7ReQxLIU1wN3A+4JxHJus\nz4qM1U9hxnOAqzDB8ySmstpEOoW8rfodq+rTwDXAQ9G4Vkbf547j0jz9Ok5mPK264ziOkwlfcTiO\n4ziZcMHhOI7jZMIFh+M4jpMJFxyO4zhOJlxwOI7jOJlwweE4juNkwgWH4ziOkwkXHI7jOE4m/h/C\nW1xAS0ZzFgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1174afb50>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"actual_values = y_test\n",
"plt.scatter(y_pred, actual_values, alpha=.75,\n",
" color='b') #alpha helps to show overlapping data\n",
"plt.xlabel('Predicted Price')\n",
"plt.ylabel('Actual Price')\n",
"plt.title('Linear Regression Model')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__MAKING A SUBMISSION__"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [],
"source": [
"X_pred = test[X_keep]\n",
"predictions = adr.predict(X_pred)\n",
"final_predictions = np.exp(predictions)\n",
"submission = pd.DataFrame()\n",
"submission['Id'] = test.Id\n",
"submission['SalePrice'] = predictions\n",
"submission.to_csv('submission_gbr_gridsearch.csv', index=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2.3 XGBoost (Extreme Gradient Boosting ou Regularized Boosting)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__Como difere do Gradient Descent:__\n",
"\n",
"- Introduz regularização na Loss Function.\n",
"- Não é um algoritmo guloso.\n",
"- Possui cross validation em cada iteração.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Prós**\n",
"- Alto poder de previsão (Maior que Adaboost e Gradient Boosting)\n",
"- Reduz overfiiting por utilizar regularização na Loss Function.\n",
"- Processamento Paralelo"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parâmetros mais importantes**\n",
"\n",
"- lambda: Responsável por atuar na regularização da Loss Function. A regularização é análoga a de Ridge\n",
"- alpha: Responsável por atuar na regularização da Loss Function. A regularização é análoga a de Lasso\n",
"- objective: Determina qual a sua Loss Function\n",
"- eta: É analogo ao learning rate, ou seja, o passo que o algoritmo caminha pela superfície da Loss Function\n",
"- max_depth: Mesmo que GBM\n",
"- booster: Tipo de modelo que será rodado em cada iteração (opções: gbtree: tree-based models, gblinear: linear models)"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [],
"source": [
"#!pip install xgboost"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [],
"source": [
"# import xgboost as xgb"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2.3.1 XGBoost Regression"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [],
"source": [
"from xgboost.sklearn import XGBRegressor"
]
},
{
"cell_type": "code",
"execution_count": 84,
"metadata": {},
"outputs": [],
"source": [
"model_xgb = XGBRegressor(learning_rate=0.05, max_depth=3, n_estimators=100,\n",
" reg_alpha=0.4, reg_lambda=0.4,subsample=0.5213)"
]
},
{
"cell_type": "code",
"execution_count": 85,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"XGBRegressor(base_score=0.5, colsample_bylevel=1, colsample_bytree=1, gamma=0,\n",
" learning_rate=0.05, max_delta_step=0, max_depth=3,\n",
" min_child_weight=1, missing=None, n_estimators=100, nthread=-1,\n",
" objective='reg:linear', reg_alpha=0.4, reg_lambda=0.4,\n",
" scale_pos_weight=1, seed=0, silent=True, subsample=0.5213)"
]
},
"execution_count": 85,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model_xgb.fit(X_train, y_train)"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {},
"outputs": [],
"source": [
"y_pred = model_xgb.predict(X_test)"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MSE: 0.0497\n",
"MAE: 0.1674\n"
]
}
],
"source": [
"mse = metrics.mean_squared_error(y_pred, y_test)\n",
"mae = metrics.mean_absolute_error(y_pred, y_test)\n",
"print(\"MSE: %.4f\" % mse)\n",
"print(\"MAE: %.4f\" % mae)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"params_grid = {'n_estimators': [50, 100, 300, 500], 'max_depth': [2,3,4], 'learning_rate': [0.05,0.03]}\n",
"gsearch = GridSearchCV(estimator = model_xgb, param_grid = params_grid, n_jobs=4,iid=False, cv=5)\n",
"gsearch.fit(X_train, y_train)\n",
"gsearch.grid_scores_, gsearch.best_params_, gsearch.best_score_"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"best_params = {'n_estimators': 500, 'max_depth': 2, 'learning_rate': 2,\n",
" 'reg_alpha': 0.01, 'reg_lambda': 'ls', 'random_state': 28}"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model_xgb = XGBRegressor(**best_params)\n",
"model_xgb.fit(X_train, y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__SUBMIT TO KAGGLE__"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# ALUNOS"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2.3.2 XGBoost Classifier"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/felipeformentiferreira/.virtualenvs/tera_py2/lib/python2.7/site-packages/sklearn/grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.\n",
" DeprecationWarning)\n"
]
}
],
"source": [
"from xgboost.sklearn import XGBClassifier\n",
"from sklearn.grid_search import GridSearchCV #Perforing grid search\n",
"from matplotlib.pylab import rcParams"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"kobe = pd.read_csv('../data/kobe.csv')\n",
"df = kobe[pd.notnull(kobe['shot_made_flag'])]\n",
"df[\"shot_zone_range\"] = df[\"shot_zone_range\"].astype('category')\n",
"df[\"shot_zone_range_code\"] = df[\"shot_zone_range\"].cat.codes\n",
"df = pd.get_dummies(df, columns=[\"combined_shot_type\"], prefix = '', prefix_sep='')"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"def get_minutes_remaing(period, minutes):\n",
" if period < 5:\n",
" return 12*(4-period) + minutes\n",
" else:\n",
" return minutes"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"df['game_minutes_remaing'] = df.apply(lambda x: get_minutes_remaing(x['period'], x['minutes_remaining']), axis=1)\n",
"# Isolate Response variable and Drop unwanted columns\n",
"X_keep = ['shot_distance','seconds_remaining','game_minutes_remaing','playoffs','shot_zone_range_code','Bank Shot','Dunk', 'Hook Shot', 'Jump Shot', 'Layup', 'Tip Shot']\n",
"response_variable = 'shot_made_flag'\n",
"\n",
"X = df[X_keep]\n",
"y = df[response_variable]\n",
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"model_xgb = XGBClassifier(learning_rate =0.1, n_estimators=1000, max_depth=5, min_child_weight=1, gamma=0,subsample=0.8,\n",
" colsample_bytree=0.8,objective= 'binary:logistic',nthread=4,scale_pos_weight=1,random_state=28)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"xgb_param = model_xgb.get_xgb_params()\n",
"xgtrain = xgb.DMatrix(X_train.values, label=y_train.values)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"cvresult = xgb.cv(xgb_param, xgtrain, num_boost_round=model_xgb.get_params()['n_estimators'], nfold=5,\n",
" metrics='auc', early_stopping_rounds=50)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"XGBClassifier(base_score=0.5, colsample_bylevel=1, colsample_bytree=0.8,\n",
" gamma=0, learning_rate=0.1, max_delta_step=0, max_depth=5,\n",
" min_child_weight=1, missing=None, n_estimators=20, nthread=4,\n",
" objective='binary:logistic', reg_alpha=0, reg_lambda=1,\n",
" scale_pos_weight=1, seed=28, silent=True, subsample=0.8)"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model_xgb.set_params(n_estimators=cvresult.shape[0])\n",
"\n",
"#Fit the algorithm on the data\n",
"model_xgb.fit(X_train, y_train,eval_metric='auc')\n",
"\n",
"#Predict training set:\n",
"y_pred = model_xgb.predict(X_train)\n",
"y_pred_prob = model_xgb.predict_proba(X_train)[:,1]\n",
"\n",
"print(\"Accuracy : %.4g\" % metrics.accuracy_score(y_train.values, y_pred))\n",
"print(\"AUC Score (Train): %f\" % metrics.roc_auc_score(y_train, y_pred_prob))\n",
"\n",
"feat_imp = pd.Series(model_xgb.booster().get_fscore()).sort_values(ascending=False)\n",
"feat_imp.plot(kind='bar', title='Feature Importances')\n",
"plt.ylabel('Feature Importance Score')"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"param_test1 = {\n",
" 'max_depth': [2,4],\n",
" 'learning_rate':[0.1, 0.07]\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"model_xgb = XGBClassifier(learning_rate =0.1, n_estimators=140, max_depth=5, min_child_weight=1, gamma=0, subsample=0.8, \n",
" colsample_bytree=0.8, objective= 'binary:logistic', scale_pos_weight=1, seed=27)"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"gsearch1 = GridSearchCV(estimator = model_xgb, param_grid = param_test1, scoring='roc_auc',n_jobs=4,iid=False, cv=5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"gsearch1.fit(X_train, y_train)\n",
"gsearch1.grid_scores_, gsearch1.best_params_, gsearch1.best_score_"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"https://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.10"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment