Skip to content

Instantly share code, notes, and snippets.

@tamnd
Last active August 29, 2015 14:21
Show Gist options
  • Select an option

  • Save tamnd/f0f9dd6a88936d0ca3b1 to your computer and use it in GitHub Desktop.

Select an option

Save tamnd/f0f9dd6a88936d0ca3b1 to your computer and use it in GitHub Desktop.
1.1 Polynomial Curve Fitting.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
"%pylab inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Support that we are given a training set:\n",
"- $x = (x_1, ...., x_N)$\n",
"- $t = (t_1, ..., t_N)$\n",
"\n",
"**Goal** Make predictions of the value $\\hat{t}$ of the target variable for some new value $\\hat{x}$.\n",
"\n",
"In this section, we will fit the data using a polynomial function of the form:\n",
"\n",
"$y(x, w) = \\sum_{j=0}^{M}w_jx^j = w_0 + w_1x + w_2x^2 + ... + w_Mx^M $"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"[<matplotlib.lines.Line2D at 0x1119b7438>]"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcHVW57vHfY5gCiQwyyJCgEJRAIzKFMHhpDiihURBB\nBhE4DBEREr2iTAGJyuxxIIHDwSiIooJwkUECiEJkEAQOY4BAEsAE0DAJBAzze/+oCmmS7s7uPdTa\ne9fz/XzySbp2ZdeT6u6331SttUoRgZmZlccHUgcwM7NiufCbmZWMC7+ZWcm48JuZlYwLv5lZybjw\nm5mVTM2FX9L5kuZIerCPfSZImi7pfkmb1HpMMzOrXj06/guAUb29KKkLGBYR6wFfAc6twzHNzKxK\nNRf+iLgF+Fcfu+wKXJjv+zdgBUmr1XpcMzOrThHX+NcEZnf7+ClgrQKOa2ZmPSjq5q4W+tjrRJiZ\nJbJEAcd4GhjS7eO18m3vI8k/DMzMqhARCzfXfSqi8F8FHAlcLGkk8FJEzOlpx/6Gb1eSxkfE+AUf\nsw7wTeDLwF3AZcAVEfR4Hvt+b9YF9gD2BNYmu9l+TgTP1SF63S18LsrM52IBn4sFqmmaay78kn4L\nbAesLGk2cBKwJEBEnBcRkyV1SZoBvAYcVOsx21WH1DUExg6D9XeWRj7MZ6+axdWdwH8APwWGR/CP\nWo4RwUzgTOBMifXJfqA8KnExcNqGaKMhMHYwLDMXXp8NE6ZGTK7xn2ZmTaTmwh8R+1awz5G1Hqfd\ndUhdW8FZk2DYeGA8rL0PU3f8Nyef/zwnfDSCufU+ZgTTgK9InAh8YzBXPLQpK731S15caf4+o2Hd\nDgkXf7P24Zm7TWIIjJ0EwwA6820X8+SALThxSCOKfncRzInguC358r3diz7AJBg2FMY08viLMSXh\nsZvNlNQBmsiU1AFamQt/k1iWpT84/8+d3bYPgoFFZViR13q8VlhkhoVFxJRUx242PhcL+FzUxoW/\nCUjsOputN+3ptVdhXlE55sLrPW1/kZWWKiqDmTWeC39CEh+QOB046x+sfsJomNH99UNh5iyYWFSe\n2TBh4QwHMnjOnfx8A4lDisphZo2lZnnmrqQo03BOicHARcAKwB4RPN8hdQ2FMYNg4KswbxZMLPqm\nak8ZHiJmAlcDfwC+HcE7RWYys95VUztd+BOQWJuskP4NOCKCNxNHWiyJFYHfAW8De0fwSuJIZkZ1\ntdOXegqWj52/BfgF8JVWKPoAEfwL6AKeBG6UWDltIjOrlgt/gSQ2Bm4ETozgRxGttWZRBG8BXwNu\nAKZIrJ44kplVoYglGwyQGAlcCRwZwaWp81Qr/2F1nMRc4GaJHSP4e+pcZlY5F/4CSGxOtmbRQRFc\nkzpPPURwqsRrwE0Sn4pYdOE9M2tOLvwNJrEB2WiY0e1S9OeL4CyJpYEbJLZr1oXezOz9fI2/gfJV\nNf8IfCuCK1PnaYQIzgSuAK6TWD51HjNbPA/nbBCJDwO3AT+M4L9T52kkCQFnAx3AThE9zwA2s/rz\nOP4mIbEs2SJS10Tw3cRxCiHxAeA3wLvAfq02YsmsVbnwN4G8AF5Ctu7NAWUqgBIDyYarXh/B+MRx\nzEqhmtrpm7v1dzKwOrBDmYo+QATzJD4P3CExPYJfp85kZoty4a8jiQOBvYGREbyROk8KEcyR+BzZ\n7N4nIvhr6kxm9n6+1FMnEpsC1wOdETyUOk9qErsA5wGbR/DP1HnM2pXX6klEYiWyB6Af4aKfyecs\nnA9cImXPYDaz5uCOv0b5zdw/ANMi+GbqPM1EYgBwDTA1gm+lzmPWjtzxp3EiMBg4JnWQZpOv278f\nsKfEnqnzmFnGHX8NJLYHfg1sFsE/UudpVvlaRdcCIyJ4InUes3bijr9AEh8Cfkm28JqLfh8iuBs4\nDfiN5JFkZqm5469CvkTB74GZERyVOk8ryO+FTAbuiuDE1HnM2oUncBXnq8BQsjH7VoEI3s3nOdwr\n8acI/pI6k1lZuePvp3yZ5b8A20bwaOo8rUZiFPBTYOP8cY5mVgOv1dNg+fXp24GfRXBe6jytSmIi\nsHwEB6TOYtbqfHO38Y4FXiTrWK16xwJbSeyWOohZGbnjr1D+oPQbyIZuzk6dp9VJbANcCnwigudT\n5zFrVe74G0RiKbKhm0e76NdHBLeRrd9/TuosZmXjwl+ZccAs4MLUQdrMicAnJPZKHcSsTHypZzEk\nOoCbyEahPJM6T7uR2Aq4HNgwghdT5zFrNR7VU2f5ImO3Ar/wKJ7GkTgbWCaCQ1NnMWs1vsZff4cD\nbwOTUgdpc8cDn5HoTB3ErAzc8fdCYghwL/CpCB5JnafdSewK/BfZJbV5qfOYtQp3/HWSr8VzDjDR\nRb8YEVwF3A+ckDqLWbtzx9+DfGLRGWTdZymfnZuCxOrAA2T/y5qWOo9ZK3DHXwcSywJnkT1G0UW/\nQPny1icD5+T/6zKzBnDhX9TxwB0R/Dl1kJI6B/gQXvnUrGF8qQfokLqGwNhlWG7Fp9lykxd4+ZCZ\ncfevUmQxkNh6MJdfuTX73P9B3lpiLrw+GyZMjZicOptZs/F6/FXokLq2grMmwTB4DbiR0fCdDukF\nF5o0NkQrfJJVl76It3aYv200rNsh4c+JWe1Kf6lnCIzNiv4Ck2DYUBiTKlPZDYGxF/Hs4O7b/Dkx\nq5/SF/7BsExP2wfBwKKzWMafE7PGKn3hnwuv97T9VTyJKBV/Tswaq/SF/zE2+/U+fPTd7tsOhZmz\nYGKqTGU3GyaMhhndtx0Kj/tzYlYfpR/VI/HLlfjRUlty1PKDYOCrMG8WTPRNxLQ6pK6hMGYQDHyK\nEcOns9s1z8XxB6fOZdZsvDpnv4/JSOAyYP0IXi3y2FY5ifXInnXcEcE/U+cxayZJZu5KGiVpmqTp\nko7p4fVOSS9Lujf/1RRrseQzQ38MjHPRb24RTAcuIJvVa2Y1qmkcv6QBwNnAjsDTwF2SroqIhRc2\n+0tE7FrLsRpgT2BpwBO1WsMpwKMSG0XwYOowZq2s1o5/BDAjIp6MiLeAi4HdetivqdZdkVgaOB34\nVgTvLm5/Sy+Cl8g6/h+kzmLW6mot/GvC+x4+/lS+rbsAtpZ0v6TJkjao8Zj1cATwcAQ3pg5i/XIe\nsI7ETqmDmLWyWpdsqOTO8D3AkIj4t6SdgSuAj/W0o6Tx3T6cEhFTaszXwzFYCTgW2K7e722NFcGb\nEscAP5D4UwTvpM5kVjRJnVDb0+pqGtUjaSQwPiJG5R8fB7wbEWf08XeeADaLiBcX2l7IqB6JH5E9\n3/VrjT6W1V9+U/5m4IIIzk+dxyy1wodzSloCeBTYAXgGuBPYt/vNXUmrAc9GREgaAfwuIj5Sj/D9\nz8tHgbuADSOY08hjWeNIjAAuBz4Wwb9T5zFLqfDhnBHxNnAkcD3wMHBJRDwi6TBJh+W77Qk8KOk+\n4CfAPrUcs0bfJ3ucoot+C4vgTrJx/WNTZzFrRaWZwCWxMdkPqPUimNuo41gxJD4G3AZ8PIIXF7e/\nWbvyoxf7dhpwiot+e4jgMbLLPcemzmLWakrR8Ut0AueTLc3wZiOOYcWTWAN4EPhkxPuGFZuVhjv+\nHuSjQM4ATnDRby8RPEM2tn984ihmLaXtCz/webKlGS5OHcQa4kzgcxLDUwcxaxVtXfglBpBN8z/e\nSzO0p3wph/8Cvpc6i1mraOvCD+wLvARcmzqINdTZwDYSm6QOYtYK2vbmrsSSwCPAoRFMqdf7WnOS\nOBLYOYJdUmcxK5Jv7r7fQcATLvqlMQnYUGKb1EHMml1bdvwSywDTgT3yWZ5WAhIHAwcA20dUtICg\nWctzx7/A4cA9Lvql80tgdbIHA5lZL9qu45dYDpgJfCaCB2pPZq1EYh/gG8BW7vqtDNzxZ44AbnbR\nL61LgUHAzqmDmDWrtur4JQaTdfvbR/BQfZJZq5HYk2wNny3c9Vu7c8cPY4A/ueiX3uXAksDnUgcx\na0Zt0/FLLA/MALaN4NH6JbNWJPF5sjV8NvWsbWtnZe/4vw5MdtG33JXAO2RrNZlZN23R8UusQNbt\nj4xgRn2TWauS2AU4HdjYXb+1qzJ3/N8ArnbRt4VMBl4HvpA6iFkzafmO392+9UXis8CpZA9rcddv\nbaesHf9Y4BoXfevFNcCb+Fq/2XtauuPvNpJn6wimNyaZtTqJXcnW6/cIH2s7Zez4xwDXuujbYlwN\nvAvsmjqIWTNo2Y5f4oNks3S3ieCxxiWzdiCxGwvG9TfHF71ZHZSt4z8SuN5F3yp0Vf67Z/Na6bVk\nxy8xiKzb3y6CaY1NZu1CYg+yNXxGuOu3dlGmjv9wYIqLvvXT74GBwKjUQcxSarmOX2JZFqy3/2Dj\nk1k7ydfrH0t2b6g5vvjNalCWjv8rwO0u+lalS4GVgP9IHcQslZbq+PNn6c4EPhfBPcUks3YjsT9w\nSASdqbOY1aoMHf/BwL0u+laj3wJrSfyf1EHMUmiZjl9iKWA6sHcEdxSXzNqRxCHAPhF8OnUWs1q0\ne8e/P/CYi77Vya+Aj0tsmTqIWdFaouOXWAJ4BDg0gr8Um8zalcQRwE4RXsrBWlc7d/x7Af8Ebk4d\nxNrK+cDmEhunDmJWpKYv/BIfAMYBJ3vctdVTBPOAHwLHp85iVqSmL/xk66jPA/6YOoi1pfOA7SXW\nTx3ErChNXfglhLt9a6AIXgUm4K7fSmSJ1AEWYxSwFAtWVjRrhLOBGRLrRPB46jBmjda0HX+3bv9U\nPzXJGimCl8gu+RydOotZEZp2OGc+q/J8YP0I3k6XzMpAYhXgUaAjgmdS5zGrVLsN5xwHnO6ib0WI\n4Dngl8BRqbOYNVpTdvwSWwCXA+tG8GbaZFYWEmsBDwAfi+D51HnMKtFOHf/xwA9c9K1IETwFXAZ8\nPXUWs0Zquo5fYkPgz8A6Efw7dS4rF4lhwB1k/9t8OXUes8Vpl47/OOAsF31LIYIZwPVkj/c0a0tN\n1fFDrAvcibstS0iiA/gT/l+ntYB26PiPBv7HRd9SimAq2eWeg1NnMWuEmgu/pFGSpkmaLumYXvaZ\nkL9+v6RN+ni7vYCzas1kVgenAt/OHwBk1lZqKvySBpBNdx8FbADsK2n4Qvt0AcMiYj2yB6Wf29v7\nbcNGr2yItqglk1k9RHDn8vzqhe346P/uJU3ZWbquI/taNmt5ta7VMwKYERFPAki6GNiN7KEp8+0K\nXAgQEX+TtIKk1SJizsJvditT1x4NZ3VITI2YXGM2s6p1SF27s+xqF/DvNeZvGw3r+mvT2kGtl3rW\nBGZ3+/ipfNvi9lmrtzecBMOGwpgac5nVZAiM7V70wV+b1j5q7fgrHRK08B3nHv/e+Pz36TBcUmdE\nTKkyl1lNBsMyPW0fBAOLzmLWnaROoLOW96i18D8NDOn28RCyjr6vfdbKty1ifP77nfDIdBd9S2gu\nvN7T9lezhwKZJZM3xFPmfyzppP6+R62Xeu4G1pP0EUlLAXuz6Nr5VwEH5AFHAi/1dH1/vkNh5iyY\nWGMus5rMhgmjYUb3bf7atHZRU8cfEW9LOpJspuMA4OcR8Yikw/LXz4uIyZK6JM0AXgMO6u39uuC6\nWTDRN88stakRkzskumDMcjBwFttt8TRr//SpuNBfm9bymmrmbn9nn5kVReIQYK8Idkqdxay7dpi5\na9asfgVsILF56iBmtXLhN6tAvkT4D/BD2S0hiVUk+lr9oLL38aUes8pILAs8AWwfwcOp81j5SJwO\nDI7giAXb+l87XfjN+kHieGB4BPunzmLlIrEi2UizTSP4+4LtLvxmDSWxPDATGBHB46nzWHlInEi2\nVPhB79/uwm/WcBInA6tEcFjqLFYOEoOAx4FPRfDo+19z4TdrOImVgceAjSJ6noVuVk8SRwFbRrDX\noq+58JsVQuJHABF8M3UWa28Sy5B1+10R3Lfo6y78ZoWQWBN4EPh4BM+lzmPtS+KrwGcj+GzPr7vw\nmxVG4jzguQhOSJ3F2pPEkmSXFb8Uwe097+PCb1YYiXWAu4B1I3gpdR5rPxIHAgdEsEPv+3jJBrPC\n5MM5/wAcmTqLtR+JAWQzxU+u93u78JvV5jRgbD7czqye9gReoNva+/Xiwm9WgwimATeBx/Rb/Uh8\nABgHnBxR8ZMOK+bCb1a7U4CjJD+W0erms8DbwLWNeHMXfrMaRfAA2dPoDkmdxVqfhIATgFMa0e2D\nC79ZvZwMHC2xVOog1vI+AywH/L5RB3DhN6uDCO4EHgEOTJ3FWlfe7Z9Idm3/3UYdx4XfrH6+BxyX\nT7oxq0YnsArwu0YexIXfrE4iuA14EtgvcRRrXd8BTo3gnUYexIXfrL6+DxyfT74xq5jEtsBQ4DeN\nPpYLv1l9TQGeBfZOnMNaz4nAaRG81egDea0eszqT2An4MdDRyBt01j4ktiS7rr9eBG/27+96rR6z\nZvBH4BWyKfdmlfgOcHp/i3613PGbNYDEzsAPgE+467e+SIwALiPr9t/o/993x2/WLK4D/g3skTqI\nNb353X6/i3613PGbNYhEF3AGsLG7fuuJxBbA5cCwagu/O36z5nIt8Dqwe+og1rS+A5xRZLcP7vjN\nGkpiF7I1+z/prt+6k9gMuJKs23+9+vdxx2/WbCYDb+Cu3xZ1Elm3X3XRr5Y7frMGy7v+0/G1fstJ\nbA5cQY3dfvZe7vjNmtFkshE+Htdv832XbE2ewrt9cMdvVohus3k3avQCXNbcJEayYJZuzTd13fGb\nNa8/Ai8Be6UOYsl9l2y9/UJH8nTnjt+sIBI7AucAG0bwduo8Vrx8Bc5fAR+v1/IM7vjNmtufgTnA\nvqmDWDLfBb5f1Jo8vXHHb1YgiU7g58D6RSy/a80j/9z/DBhez8+9O36zJhfBFOBx4KDEUaxA+bN0\nTwFOaoYf+C78ZsUbB5wosUzqIFaYLmB54OLUQcCF36xwEdwJ3AMcnjqLNZ7EB4CTgRObZSivC79Z\nGicCx0gMSh3EGm4P4G2ymbpNwYXfLIEIHgBuBL6eOos1jsQSwPeBEyJojpE0eFSPWTIS6wF/JRvT\n/WLqPFZ/Ev8JHAxs16jCX03tdOE3S0jiPOClCI5JncXqK795/yjwpQhua9xxXPjNWorEGsCDZCt3\nPpU6j9WPxDfJOv3dGnscF36zliNxGrBKBIemzmL1IbE8MB3YPoKHGnssF36zliOxAvAYWXf4SOo8\nVjuJU4A1Iho/Ua/Qwi9pJeASYG3gSWCviHiph/2eBF4B3gHeiogRvbyfC7+VlsS3ga0j/KSuViex\nOjAV2CSCWY0/XrFLNhwL3BARHyNbfOrYXvYLoDMiNumt6JsZZwObS2yVOojV7DvABUUU/WrV0vFP\nA7aLiDmSPgxMiYj1e9jvCWDziHhhMe/njt9KLR/6NxrYtpnGfFvlJIYDN5MtwtdnzavfMYvt+FeL\niDn5n+cAq/WyXwB/knS3pNE1HM+s3f0KWA74QuogVrUzgNOLKvrVWqKvFyXdAHy4h5fGdf8gIkJS\nbx3KNhHxD0mrADdImhYRt/RyvPHdPpwSEVP6ymfWTiJ4R+JbwP9IXJ16zXbrH4ntgQ7gi409jjqB\nzpreo8ZLPZ0R8U9JqwM39XSpZ6G/cxLwakT8sIfXfKnHDJCYDFwfwVmps1hl8oXY7gLOjOCSYo9d\n7KWeq4AD8z8fSA8LEElaVtLg/M/LAZ8hm6xiZr07GhgnsWLqIFaxLwFvkT1EvenVOpzzd8BQug3n\nlLQGMCkidpG0DnB5/leWAH4dEaf18n7u+M1yEj8FXongW6mzWN8kBgLTgP0iuLX443sCl1lbkPgw\n2VjwrSKYnjqP9U7iBOCTEeyZ5vgu/GZtQ+IYskldDV3rxaonsRZwP7B5BE+kyeBn7pq1k58AG0p8\nOnUQ69XpwLmpin61+hzOaWbpRPCGxLc+yO8m7ax9Hh1MLD0XXp8NE6ZGTE6dr6w6pK4hMHYpVlx1\nCzYe/gqPfhmeSR2rX1z4zZrYcAa8+UlWX/U3xNrzt42GdTskXPyL1yF1bQVnTYJh8C9gCqPh9A5p\nXit9Pnypx6yJrc27Y3/D0wO7b5sEw4bCmFSZymwIjM2K/gKt+Plw4TdrYoNhmZ62D4KBPW23xmqX\nz4cLv1kTmwuv97T9VZhXdBZrn8+HC79ZE5sNE0bDjO7bDoWZs2Biqkxl9hBfmLwPH3mn+7ZW/Hx4\nHL9Zk+uQuobCmOVg4FNstdFMPn3xs/HdI1LnKpt8PZ6/rsq42zfj1PUHwcBXYd4smJjyxq4ncJm1\nOYkNgZuAjgieTZ2nTCRGAweRPS/h3dR55nPhNysBiR8Aq0fw5dRZykJiVbIFJneK4L7Uebpz4Tcr\nAYnlyNbx+WoE16fOUwYSvwGejuDbqbMsrJra6QlcZi0mgtckvkr2wJaOCF5LnamdSewMjCR7yEpb\ncMdv1qIkLgLmRHBU6iztSmIQ2f+uRkdwQ+o8PfGlHrMSkViF7LrzZyO4O3WediTxY2CliPceOtV0\nXPjNSkZif+AoYISf0VtfElsCV5KNoHo+dZ7eeFlms/K5CJgNnJA6SDvJn6p1ITCmmYt+tdzxm7U4\nidWB+4BdfMmnPiR+BKwRwT6psyyOR/WYlVAE/5D4v8CFEptF9LyejFVG4lPAPsBGqbM0ii/1mLWH\n3wKPAt9NHaSV5aN4fgEcHsELieM0jC/1mLWJfHbpA8CeEdyaOk8rkjgXWC6CA1JnqZQv9ZiVWATP\n5uvJXCSxSQT/Sp2plUjsDuwEbJI6S6O54zdrMxITgVWBfSJojm/wJiexFvC/wG4R3JE6T394OKeZ\nAXwb2IBsJUlbDIkBZMNiJ7Ra0a+WO36zNpQv3zyFbAnhRxPHaWoS44AdgR0jeGdx+zcbd/xmBkAE\nD5FN6ro0X83TeiCxPXAksH8rFv1queM3a1MSIhuaOICssDXHN3uTkFgTuAs4III/pc5TLXf8Zvae\nvNAfTrac8NcSx2kqEksBlwJnt3LRr5Y7frM2J7Eu8Ffg8xHcnjpPM8hHPg0Fdm+mxyhWwx2/mS0i\ngpnAwWTX+9dKnSc1iYOAUcCBrV70q+XCb1YCEVwDTACuzpclKCWJTuB04HMRvJQ4TjK+1GNWEvnN\n3klkk7t2L9MoFgCJjwG3AF+K4M+p89SLL/WYWa/ym71fAwYBZyaOUyiJDwHXAOPaqehXy4XfrETy\np3TtAeySL+Xc9vJ5DFcBv4/gZ6nzNAMv0mZWMhH8S+IzwC0SL0dwfupMjSKxNHA58BhwbOI4TcOF\n36yEIpgl8WlgisQrEVyWOlO9SSwB/BqYC4wu6wienrjwm5VUBI9JdAF/lHgtgmtTZ6qXfOG1ScBg\nYNcI3k4cqan4Gr9ZiUVwH7Ab2WMbd0+dpx4klgR+CXwE+EIEb6RN1Hzc8ZuVXAS3S4wCrpFYLoKL\nUmeqVn5N/xJgSaArgnmJIzUlF34zI4J7JHYgu+yzXATnpc7UX/nond8DLwN75SOYrAe+1GNmAETw\nMLAd8G2JM/Pr5C1BYghwKzAb2NdFv28u/Gb2nnxdny2BLYArJAYnjrRYEiOBO8ieonWob+Qunpds\nMLNF5DdIJwLbAHtsiIYNgbGDYZm58PpsmDA1YnLRuTqkru457uG4R57l1P2Ag/L1iEqnmtrpa/xm\ntogI3pI4HDhsMFfcuRnLv3EhL686//XRsG6HRJHFv0Pq2grOmgTD5m/bl19v/1fu+8bfY3Ipi361\n3PGbWZ8+rRVuvYGXt1l4exdcNzli56Jy7Cxddy3slDpHs/EibWZWdyvyco/XzAfBwCJzLMvSH2yG\nHO2g6sIv6YuSHpL0jqRN+9hvlKRpkqZLOqba45lZGnPh9Z62v8CqhRRciWUkjp7NNpv39PqreKx+\nf9XS8T8I7A7c3NsOkgYAZ5M97WYDYF9Jw2s4ZilI6kydoVn4XCyQ6lzMhgmjYUb3bQcyeM5d/PdQ\nidskPi/V/+qBxEoS44AngW2eI742P8eUfJ9DYeas7Ca09UPVN3cjYhqA1OelpRHAjIh4Mt/3YrLp\n4Y9Ue9yS6GTB13bZdeJzMV8nCc7F1IjJHdmiPmMGwcBXYd4s5k6cyx5/BL4AjAPOkLgAuCzi/T8k\n+iNfWG07YC/gi8CVwI4RTIUb6ZCe6YIx02H4evDILJiYYnRRq2v0qJ41ySZUzPcU2RhhM2sheXHt\nqcD+TuJSYFtgX+BWiX/m+94N3AP8PX8IzCLyJRY6gE2BrYDPkXX4lwEbRfB0TzkkjZ8eMb4O/7RS\n6rPwS7oB+HAPLx0fEVdX8P7NMWTIzBomL+q3kK3vP4Zs7P8OwEFkl3qXk3gW+BfwErAUsGL+a2Vg\nJtkPiLuB70XwZNH/hrKpeTinpJuAoyLinh5eGwmMj4hR+cfHAe9GxBk97OsfEmZmVUg1gau3g94N\nrCfpI8AzwN5k/x1chMfwm5kVo5bhnLtLmg2MBK6RdG2+fQ1J1wBExNvAkcD1wMPAJRHhG7tmZgk1\nzcxdMzMrRqEzdyuZzCVpQv76/ZI2KTJfkRZ3LiTtl5+DByTdJukTKXIWodJJfpK2kPS2pC8Uma9I\nFX6PdEq6V9JUSVMKjliYCr5HVpZ0naT78nPxnwliNpyk8yXNkfRgH/v0r25GRCG/gAFkky8+QvZ0\nnPuA4Qvt0wVMzv+8JXBHUfmK/FXhudgKWD7/86gyn4tu+90I/AHYI3XuhF8XKwAPAWvlH6+cOnfC\nczEeOG3+eQBeAJZInb0B5+JTwCbAg7283u+6WWTH/95kroh4C5g/mau7XYELASLib8AKklYrMGNR\nFnsuIuL2iHg5//BvwFoFZyxKJV8XAGPIxnY/V2S4glVyLr4E/L+IeAogIp4vOGNRKjkX/wDmr9/z\nQeCFyO4rtpWIuIVsKGxv+l03iyz8PU3mWrOCfdqx4FVyLro7hJ4nz7SDxZ4LSWuSfdOfm29q1xtT\nlXxdrAesJOkmSXdL2r+wdMWq5FxMAjaU9AxwP/D1grI1m37XzSLX46/0m3XhYZ3t+E1e8b9J0vbA\nwWSTYtpRJefiJ8CxERHK1ghp16G/lZyLJclmue4ALAvcLumOiJje0GTFq+RcHA/cFxGdktYFbpC0\ncUTMbXAYiJn+AAABeklEQVS2ZtSvullk4X8aGNLt4yFkP5n62metfFu7qeRckN/QnQSMioi+/qvX\nyio5F5sBF+frQq0M7CzprYi4qpiIhankXMwGno+IecA8STcDGwPtVvgrORdbA6cARMRMSU8AHyeb\nP1Qm/a6bRV7qeW8yl6SlyCZzLfyNexVwALw36/eliJhTYMaiLPZcSBoKXA58OSKqXvSqBSz2XETE\nOhHx0Yj4KNl1/sPbsOhDZd8jVwLbShogaVmym3kPF5yzCJWci2nAjgD5Ne2PA48XmrI59LtuFtbx\nR8TbkuZP5hoA/DwiHpF0WP76eRExWVKXpBnAa2RrfbSdSs4F8B2ytUzOzTvdtyJiRKrMjVLhuSiF\nCr9Hpkm6DngAeBeYFBFtV/gr/Lo4FbhA0v1kTezREfFistANIum3ZCuWrpxPmj2J7JJf1XXTE7jM\nzErGj140MysZF34zs5Jx4TczKxkXfjOzknHhNzMrGRd+M7OSceE3MysZF34zs5L5/xOrsy6AA/ff\nAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1119b7160>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Number of observations\n",
"N = 10\n",
"\n",
"# Degree of polynomial\n",
"M = 4\n",
"\n",
"xground = arange(0, 1, 0.01)\n",
"yground = sin(2*pi*xground)\n",
"\n",
"# Get some points at regular interval\n",
"x = arange(0, 1, 1./N)\n",
"tground = sin(2*pi*x)\n",
" \n",
"plot(xground,yground)\n",
"plot(x, tground, 'ro')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we add a small level of random noise having a Gaussian distribution to each such point:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"[<matplotlib.lines.Line2D at 0x111634e80>]"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu85XO9x/HXO6OMM1NynCMxkzKINs7ouJNxKDO7E5VL\nEV00rjHulBx0UkLCjEtjXKIcSu6ZS8g4iCjXwaghzVDhUMwwZPicP34/7Blr7732Xmv9vr+1fu/n\n4zGP9lr7t3+/t1+zP/Nd39/3oojAzMyq4x2pA5iZWbFc+M3MKsaF38ysYlz4zcwqxoXfzKxiXPjN\nzCqm4cIv6XxJT0l6oJfvj5H0vKR78j9HN3pNMzMbvCFNOMcFwCTgoj6OuTkitmvCtczMrEENt/gj\n4hbgb/0cpkavY2ZmzVFEH38Am0q6T9JUSWsXcE0zM+tFM7p6+nM3MCIiXpI0DrgKWKOA65qZWQ0t\nL/wRMb/H19MknSVp+Yh4rudxkrxokJnZIETEgLrTW174Ja0IPB0RIWlDQEsW/TcMNHynknRcRByX\nOkcZ+F68xffiLb4XbxlMo7nhwi/pEmBLYAVJ84BjgaUBImIysCOwr6RFwEvA5xu9ppmZDV7DhT8i\ndunn+2cCZzZ6HTMzaw7P3C2nmakDlMjM1AFKZGbqACUyM3WAdqaybMQiKdzHb2Y2MIOpnW7xm5lV\njAu/mVnFuPCbmVWMC7+ZWcW48JuZVYwLv5lZxbjwm5lVjAu/mVnFuPCbmVWMC7+ZWcW48JuZVYwL\nv5lZxbjwm5lVjAu/mVnFuPCbmVWMC7+ZWcW48JuZVYwLv5lZxbjwm5lVjAu/mVnFuPCbmVWMC7+Z\nWcW48JuZVYwLv5lZxbjwm5lVjAu/mVnFuPCbmVWMC7+ZWcW48JuZVYwLv5lZxbjwm5lVzJDUAaxc\nuqTuETBhOCwzH16eBxNnRUxNncvMmseF397UJXVvAqdPgVFvvLcnrNYl4eJv1jnc1WNvGgETehZ9\ngCkwaiQckCqTmTVfw4Vf0vmSnpL0QB/HTJT0B0n3SRrd6DWtNYbDMrXeHwZDi85iZq3TjBb/BcDY\n3r4pqRsYFRGrA3sBZzfhmtYC8+HlWu8vgIVFZzGz1mm48EfELcDf+jhkO+DC/NjfAMtJWrHR61rz\nzYOJe8Kcnu+Nh0fnwqRUmcys+Yp4uLsyMK/H6yeAVYCnCrh2JUi8E/hXYDjwbrKumReBF/I/T0fw\nWn/nmRUxtUuiGw4YBkMXwMK5MMkPds06S1GjerTE66h5kHRcj5czI2JmqwK1K4llgE2BrYB1gLWA\nDwDPkhX5+WRdNsuS/SPwHmC4xBzgYeAe4FfA72r9Y5AXeRd6s5KSNAYY09A5ImrW4IEGWRW4NiLW\nqfG9H5IV8Uvz17OBLSPiqSWOi4hY8h8IAySWA3YCdgY2Bh4EbgTuBmYDcyJ4pY+fHwasAXwY2BDY\nmuxT103ApcA1EbX7982s3AZTO4so/N3A/hHRLWlj4LSI2LjGcS78PUiIrEDvBWwLXA/8D3BjBM83\n4fzvI3sovxuwPvBz4KwI7m303GZWnCSFX9IlwJbACmT99scCSwNExOT8mDPIisyLwFci4u5mhO9E\nEksBnwW+TtZXPxH4aUSfD9AbveYqwO7A/sADwInAzIjaXXJmVh7JWvzNUPXCn7fwPwWcRDZK6nvA\ntRG8XmCGd5F9AjgCeAY4JII7i7q+mQ2cC3+bklgXOBVYCTgUmJ6ytZ1/6vgi8B2yB8HfiFhsZJaZ\nlcRgaqeXbEhIYqjEycANwBXAehFMS93FEsFrEVwArAk8DtwrcaDkvy9mncAt/kQkNiWb9XwvsH8E\nzySO1CuJNYDzgdeBPSIWn+RlZum4xd8GJIZIfBe4HPhmBJ8rc9EHiOD3ZA/wrwDukNgrfyZhZm3I\nLf4CSbyfbNz8QmC3shf8WiTWBC4D7gP2jWBB4khmleYWf4lJ/AfwW7Lx+OPasegDRPAI2SSyV4E7\nJdZOHMnMBsiFvwAS+wEXA7tH8O0ih2i2QgQvRbAH8H1gptT76qxmVj7u6mmhfFjkKWQzb/8zgkcT\nR2o6ic3IZv1+O4KzUucxqxqP4y8RiX8i689fFtixlTNvU5NYDfgFMAM4tJ6VQM2sOdzHXxL5omrX\nk62YOa6Tiz5A/klmU+DfgIukbMkOMysnF/4mk/hXYCZwJ9mY93+kTVSM/B+3cWTLQF+eLx9tZiXk\nwt9EEiOAW4CrgYPb/SHuQEWwkGyBuZeAqRLDE0cysxpc+JskX+FyJnBOBMemXnYhlfwTzheAx4Dr\n8mcdZlYiLvxNILES2cYoP4zglNR5Ussf7u4FPApcIzE0cSQz68GFv0F5n/6NwEURnJw6T1nk3Vzj\ngb8CV7rP36w8PJyzAfnonZuBqyM4JnWeMpIYQrZz2LuAHSJYlDiSWUfxOP4C5S3YGWSrax5U1T79\neuTDO68FngTG+16ZNY8Lf0HyGbk/I1uvZteqjd4ZjHzD918Bv4zg6NR5zDrFYGrnkFaF6VT5csRn\nkI1X/6SLfn0iWCDxSeA2ib9GcEbqTGZV5cI/cEcCGwFjIngldZh2EsEzEtsCt0rMi+Dq1JnMqsiF\nfwAkPgt8Ddg4ghdS52lHEfxR4tNkE7zmRnBP6kxmVePhnHWS+HdgMrB9BE+mztPOIrgL2Be4Ot+c\nxswK5MJfh3xW7lXAnhHcnTpPJ4jg58APySZ4LZs6j1mVeFRPP/JZp7cAl0VwYuo8nSR/UH4h8E5g\nFw/zNBs4D+dssrwwXUA2+WhXF6bmy+dD3ApcGsH3U+cxazceztl8+wHrA5u46LdGBC/nD81/I3Fv\nBDekzmTW6dzi74XEFmRbCm7aiVsmlo3EVsAlZCOmHk8cx6xteAeuJslX27wU+JKLfjEiuAk4CW/i\nYtZybvG/LQdDyLZNnBnBt1LnqZL8mcplwNMR7Jc6j1k7cIu/OY4FFgHHpw5SNflzlK8Cn5D4fOo8\nZp3KLf7FMrAtcB7w0QieSpmlyiRGA78ENo/gkdR5zMrMLf4GSKwM/Aj4got+WvkyDkcDl3n3LrPm\nc4ufN5dZvgG4McJdPGWQ9/dfAjwbwddS5zErK7f4B+/I/H9PSJrC3pT39+8DdEtsnzqPWSepfItf\nYmPgarJ+/SeKvr71TWJT4Aqy/3+8OJ7ZEpK0+CWNlTRb0h8kHVnj+2MkPS/pnvxPaXZfkngP2X6w\n+7jol1MEvwbOBC7Ku+TMrEENtfglLQU8AmxDtp/qXcAuEfFwj2PGAIdExHb9nKvwFr/Ej4EXI9in\nyOvawOQF/1fAdRGclDqPWZmkWKtnQ2BORDyeB7gU2B54eInjkk/MWpLEzmT5R6fOYn2L4DWJLwJ3\nScyI4L7UmczaWaNdPSsD83q8fiJ/r6cANpV0n6SpktZu8JoNyzf/mATsHsFLqfNY/yL4E3AY8BMv\n6WDWmEZb/PX0E90NjIiIlySNI9vQZI0Grzto+TDB84GzI7gzVQ7rW5fUPQImDIdl5sPL82AixI+B\n7chmVR+WOKJZ22q08D8JjOjxegQs/pA0Iub3+HqapLMkLR8Rzy15MknH9Xg5MyJmNpivln2B5YHv\ntODc1gRdUvcmcPoUGPXGe3vCaiAeJPYB7pP4RQQz06U0SyN/bjqmoXM0+HB3CNnD3a2BPwN38vaH\nuysCT0dESNoQ+FlErFrjXC1/uCsxCrgD2MxLAZTXOGn6NNh2yfe7YfrUiHES3cBZwLre9N6qrvCH\nuxGxSNL+wAxgKeC8iHhY0t759ycDOwL7SloEvARpFt/KR4b8CDjeRb/chlO7D38Y2fINEUyVuAE4\nGdi7yGxmnaDhHbgiYhowbYn3Jvf4+kyycdipTQBeByamDmJ9mw8v13p/ASzs8fJQ4H6JT0Twy2KS\nmXWGSizZILEG8E1gjwheT53H+jYPJu4Jc3q+Nx4enZuNxAIggueBPYFz84l4Zlanjl+yIe/iuQW4\nJOKtwmHl1iV1j4QDhsHQBbBwLkyaFTF1yeMkJgNLRTA+QUyz5AZTO6tQ+A8hm1S2VW+t/VpDB2sV\nGSsfiXcD9wN7RzAjdR6zoqWYuVtqEqsBR5Ft4N1r0a81dLBLwsW//CJ4QWJvYLLEOhHM7/eHzCqu\nY/v484laU4ATIhbvL+5pBEzoWfTJfmjUSDig1RmtOfKW/kzgu4mjmLWFji38wHhgGHB6Xwf1N3TQ\n2sYhwA4Sm6cOYlZ2HVn4JVYha/3tEcGivo6tc+iglVwEz5F9SjvXa/mY9a3jCn/exXMWcGYEs/o7\nvp6hg0XpkrrHSdN3lmaOk6Z3Sd1FZ2hnEVwOPAgckzqLWZl14sPdHYDVyGYM92tWxNSubA2AfocO\ntpIfMjfN/mQTuy6N4P7UYczKqKOGc0q8F5gF7BzBbc1JVoz+1qdJkaldSewFfBXYNILXUucxayVv\ntg7fA65pt6IPfsjcZOcCrwD7pQ5iVkYd09Uj8THgk8BHUmcZDD9kbp4IXs9b/bdKXBWx2GZBZpXX\nES1+iXcBk4EJ+RoubadMD5k7QQSzyRbkOyt/4G9muY7o45c4BvhoBNs3OVah6l2fxuoj8U7gXuDo\nCK5IncesFSq5Vo/E6sDtwPoRzG1+MmtnElsCPwHW9nIO1okqV/jzj/DXA1Mj+EFrklm7kzgfeCGC\ng1JnMWu2Ko7q2RX4Z7y5ivXtCODzEuunDmJWBm3b4s/H7D8EbB/Bna1LZp1A4svA18hWavXYfusY\nVWvxnwBc6aJvdboQeBHYN3UQs9TassUvsRFwFbBWBH9vbTLrFBJrAzcD60Tw19R5zJqhEi1+iSHA\n2cDhLvo2EBE8BJwHnJI6i1lKbVf4yabhPw9cnDqItaVvA5tJbJ06iFkqbdXVI7ES8ACwRQQPF5PM\nOo3E9sCJwHoRvJI6j1kjqtDV8wPgHBd9a0QEVwO/Bw5LncUshbZp8ecfzc8DPhLBi8Uls04ksSrw\nO7KlPh5Pm8Zs8Dq2xZ+vuXIGcJCLvjVDXuxPBU5LHMWscG1R+IGDgceAq1MHsY5yMrC2xCdTBzEr\nUum7eiRGAPcAG0XwaPHJrJNJfIJseHBXhPc+sPbTqV09pwKTXPStFSL4JXA3cGTqLGZFKXWL360x\nK4I/VVo766gWf76r1hlku2q56FvL5FszngycnjqLWRFKW/iBQ4GHI7gudRCrhFOBURKfSh3ErNVK\n2dUjMZKs33WDCP6YNplVhcQ2wDlkc0X8KdPaQid19ZwKTHTRtyJFcAPwW/yg1zpc6Vr8EtsCZ5G1\nul5OncuqxQ96rd20fYs/f6A7CTjQRd9SyB/0fh/P6LUO1nDhlzRW0mxJf5BU8yOypIn59++TNLqP\n0x0MPBLBLxrNZdaAHwBrSPxn6iBmrdBQV4+kpYBHgG2AJ4G7gF0i4uEex3QD+0dEt6SNgNMjYuMa\n5wqIZ4ENI3hs0KHMmqDHHBJ3OVqppejq2RCYExGPR8SrwKXA9kscsx3ZfqdExG+A5SSt2Mv5znTR\ntzLIZ/TeCxyROotZszVa+FcG5vV4/UT+Xn/HrNLL+b7XYB6zZjoYmJAv4WzWMRot/PX2Ey35MaTm\nz3nstJVJBHPJhhafmjqLWTMNafDnnwRG9Hg9gqxF39cxq+TvvY2k43q8nBkRMxvMZ9ao7wOzJMZG\nMD11GDNJY4AxDZ2jwYe7Q8ge7m4N/Bm4k74f7m4MnNbbw92BPqAwK4JEN9nwznW8R6+VTeEPdyNi\nEbA/MAN4CPhpRDwsaW9Je+fHTAUekzQHmAzs18g1zYoWwVTgYbL1o8zaXqlm7o6FGfNg4qzsHwuz\n0pD4INlw5fXzvn+zUmj7mbvTYNtN4PSurHvIrDTydaMmkU3uMktCYiWJrRs9T6kKP8AUGDUSDkid\nw6yGk4DR+eQusxROJpsw25BGR/W0xDAYmjqD2ZIiWChxIDBJYl0/6LUiSWwJfAxYq9Fzla7FD7AA\nj+e3csrXkfo92eQus0JILE22I+EhEbzY6PlKV/jHw6Nzs75Us7I6EDgsX8LZrAj7A38BLm/GyUo1\nqmccTJ8Lkzyqx8qgS+oeAROGwzLz4eWeI84kjiNbwG2ntCmt00msBNwPbB7BI2///sBH9ZSq8HsC\nl5VFl9S9CZw+BUa98d6eMOd2OHBWxFSJocCDwN4RXJ8uqXU6iZ8A8yL4Ru3vt/lwTrOyGAETehZ9\nWHzEWb6u1IHAGfkGQmZNJzEG2AI4vpnndeE3q2E4LFPr/Z4jziK4lmzJEs/otabLH+ieCRzcjAe6\nPbnwm9Uwn9qbr9QYcXYgcKiXbrYWOAiYC1zZ7BO78JvVMA8m7glzer5Xa8RZPqP3NLx0szWRxCrA\nkcCEiLqXv6///H64a1Zbl9Q9Eg4YBkMXwMLeRpxJLAM8ABwUwXWtyNHb6CLrTBI/A2ZHcEz/x3pU\nj1kSEtsCZwFdzdxQqL/RRc26jpVHviTID8mGC/f7d8mjeswSiWAG8DvgqGaet7/RRdZZ8k+PZwL7\nt3JHQhd+s+Y5GNhXYs1mnbCe0UXWUY4EHsj3gGgZF36zJongSeA7wJnS2/aZHpQBjC6yNicxiuyT\n3EGtvpYLv1lzTQL+BfhcM05W7+gia295Q+EM4MQiNvrxw12zJpPYBPg5sHYEzzd6vnpHF1n7ktgR\nOA4YHcGrA/tZj+oxKwWJc4B/RLB/6ixWbhLvIVv36fMR3Drwn3fhNysFieXJfpm3j+DO1HmsvCQm\nAUMjGD+4n3fhNysNid3I1vHZIIJFqfNY+UhsAFxLNmb/2cGdw+P4zcrkYuBZPObeapAYAkwGDh9s\n0R/0td3iN2sdidWBXwPrRzAvdR4rD4mDgE8B2zSyHo+7esxKSOK/gA3I+vvL8QtnSUmMBO4GNqu1\nq9bAzuWuHrMyOhFYDdghdRBLLx+zfxZwWqNFf7Bc+M1aLIJ/AHsBp0sslzqPJbcTsCpwUqoA7uox\nK4jE2cA7Itg7dRZLQ+K9ZMN8d4jg9uac0338ZqXVY6LOLhHckjqPFS+f2PdqBF9r3jkHXjuHNOvi\nZta3CJ6XmACcK7FeRO0F2Kwz5RunjwM+kjiK+/jNihTBFWS7dfW7s5J1DollgXOB/SJ4IXked/WY\nFUvifcD9wNgI7k6dx1pP4vvA+yPYtfnndh+/WVuQ+BLZuusbDnQ1RmsvEhsC1wDrRPBM88/vcfxm\n7eIi4Cng8NRBrHUk3gmcBxzciqI/WG7xmyUi8QGyfXo/FsFDqfNY80l8CxhNC2dtu6vHrM1I7APs\nAWzqFTw7i8RoYAbwbxH8uXXXcVePWbuZDDwPHJY6iDVP3sVzIXBoK4v+YA26xS9peeCnwAeAx4Gd\nI+LvNY57HHgBeA14NSI27OV8bvFbJeVdPr8FtopgVuo81jiJbwPrAp9u9cJ8Rbf4vw5cHxFrADfm\nr2sJYExEjO6t6JtVWQR/Ao4CfiSxdOo81hiJj5KtzbRPWVdjbaTwb0f2UYb8fz/dx7FuyZv17Vyy\nTVt6a0BZG5BYhqweHhLBX1Ln6U0jXT1/i4j35l8LeO6N10sc9xhZH+ZrwOSImNLL+dzVY5UmsTJw\nDzAugt+lzmMDJ3EKMBLYuajWftPX6pF0PfC+Gt/6Zs8XERGSevuP3Cwi/iLpX4DrJc2OiJoLVEk6\nrsfLmRExs698Zp0kgifztXx+IrF+BAtTZ7L6SWwFfA5Yr5VFX9IYYExD52igxT+brO/+r5JWAm6K\niA/38zPHAgsi4pQa33OL3wyQuAR4OoIDU2ex+uQrr95P1q8/rdhrF/tw9xrgS/nXXwKuqhFoWUnD\n86//CfgE2QJVZta7rwGfldgmdRCr20RgatFFf7AaHc75M7L+rMfJh3NKej8wJSI+KelDwBX5jwwB\nLo6IE3o5n1v8ZjmJjwPnk03+eTZ1HuudxM7A8cDoCF4s/vqeuWvWMfIVHUcBnynrsMCqk1gVuJOE\nD+Q9c9ess3yT7BP1PqmD2Fu6pO5x0vSd9I6bN2bTe1bgu9e22ygst/jNSkxiTeBWYEwED6bOU3Vd\nUvcmcPqU7JMYAHvCnNvhwFkRU1NkclePWQeS2INs7f6Nqj7Es0vqHgEThsMy8+HleTCxyII7Tpo+\nDbZd8v1umD41YlxROXrynrtmnekCYBvgdLKlACqpl9b2al0SRRX/4bBMrfeHwdAirt8s7uM3K7n8\nwe7ewJYSu6XOk8oImNCz6ANMgVEj4YCiMsyHV2q9v4D2+iTmwm/WBiKYD+wEnCqxVuo8KZShtX0X\nJz61KysvVuTHw6NzYVJRGZrBXT1mbSKC+yW+DlwmsVGKMeMpzYeXa71fVGs7W5LhiI/fyaV7d/Pk\nrsNg6AJYOBcmpXqwO1h+uGvWRiQE/AhYCti9SuP7a/Xxj4dH74AJrS68+QJ6dwJfjuD6Vl5roDyq\nx6wCJJYFbgN+FMHpqfMUqUvqHgkHFNnalngXcDNwdQQ1Vx5IyYXfrCLyGaN3AJ+PYGbaNJ1N4hxg\neWCnMn7C8sxds4qI4HFgd+ASiRGJ43Qsib2AzYCvlLHoD5Zb/GZtTOJwsjXgPxbBS6nzdBKJzYAr\ngc0j+H3qPL1xV49ZxeQPey8kG9L4uQheTxypI0h8iOw5ypcjmJE6T1/c1WNWMXn3w57ASsB/J47T\nEfJNVX4BHF/2oj9YLvxmbS6CV4DPALtI7J46TzuTGEK2z8iNEZyZOk+reAKXWQeI4BmJTwEzJeZ5\npM/A5d1mZwABHJw4Tku5xW/WISJ4iOxB788k1kudpw0dA2xANmxzUeowreTCb9ZBIriJbM/eqRIf\nTJ2nXUjsQzY8tjtfF6mjuavHrMNEcJnEisAMic0ieCZ1pjKT+AxZa3+LCJ5KnacIbvGbdaAIziB7\nSDlD4r2p85SVxLbAZOBTETyaOk9RXPjNOtd/ATPJiv97EmcpHYn/AH5Ctpl9W+2Z2ygXfrMOlY/x\nP5RsVcmpEsMTRyoNiS2AS4EdI7gtdZ6iufCbdbC8+E8AHgKuk3h34kjJSWwOXA7sGsHNqfOk4MJv\n1uHyZRz2Bh4EbpT458SRksn79K8AvhDBDanzpOLCb1YBefHfD/gVcLPESokjFU5iB+DHwKfLtplK\n0Vz4zSoigojgSOBi4BZp8Y3LO5nEV8lm5W4bwa9T50nN4/jNKiaCEySeBW6V2DGCW/v7mS6pewRM\nGA7LzIeX58HEdthnVuIdwHfINqofE8EjiSOVggu/WQVFcI7E48AVEgdHcHFvx9ba63ZPWK1LoszF\nX2IocBHZyqUbR/B/iSOVhrt6zCoqgl8CWwHHS3w3X5nybUbAhJ5FH2AKjBoJBxSRczAkRpLtk/sK\nsLWL/uJc+M0qLIIHgY3IFie7XuJ9Sx4zHJap9bPDss1fSkdiHHAX8HNg93zZauvBhd+s4iJ4GhgL\n/C/wO4kxPb8/H16u9XMLYGHr09VPYojEd4ApZBOzTuqkfXKbyYXfzIjgtQiOBb4C/I/EDySWBZgH\nE/eEOT2PHw+PzoVJKbLWIrE28GuyTy7rR3BL4kil5j13zWwxEiuQFfWPAl+J4LYuqXskHDAMhi6A\nhXNhUhke7ObPJQ4jW5riaOCcqrXyvdm6mTVNvlzxmcBU4OgI/po40mIktgJOBZ4Bxkfwp8SRkvBm\n62bWNBFcCawN/A2YJXFUPkQyKYnVJa4Czicbo/+Jqhb9wRp04Ze0k6QHJb0maf0+jhsrabakP0g6\ncrDXM7PiRfD3CA4HNgb+HZgjcXiKxd4kuiQuAu7I/6wVwWVV69pphkZa/A8AnyEbCVCTpKXIpkmP\nJWs57CJprQauWQmSxqTOUBa+F29JeS8imBPBZ4FuYDTwR4kTJdZs5XXzkTpjJa4FbgAeBlYD3RFR\ne7SR9W/QhT8iZkfE7/s5bENgTkQ8HhGvkq1/vf1gr1khY1IHKJExqQOUyJjUASK4L4JdyUbPLE22\n4NsdEvtJrNKMa0gsJbGhxCnAPOBbwC+AD0ZwQgR/pwT3op21esmGlcn+j3vDE2STRcysjUXwGHCI\nxBHAx4HdgP/O1wC6EbiNrHX+SAQv9nYeCQHvAz4MrEM2k3hL4C/AlXh9nZbos/BLuh7ePpMPOCoi\nrq3j/O57M+tgESwCpgHT8gXR1gW2JvtkfwSwusTfgOeA+fmfIcC7geFk9eVVYDbZZjGXAfuWbQRR\np2l4OKekm4BDI+LuGt/bGDguIsbmr78BvB4RJ9Y41v9ImJkNwkCHczarq6e3i/4WWF3SqsCfgc8B\nu9Q60GP4zcyK0chwzs9Imkc2zOs6SdPy998v6TqAiFgE7A/MIPsY99OIeLjx2GZmNlilmblrZmbF\nKHTmbj2TuSRNzL9/n6TRReYrUn/3QtIX8ntwv6TbJK2bImcR6p3kJ2kDSYskfbbIfEWq83dkjKR7\nJM2SNLPgiIWp43dkBUnTJd2b34svJ4jZcpLOl/SUpAf6OGZgdTPyjThb/QdYimyFv1XJxv/eC6y1\nxDHdwNT8642AO4rKV+SfOu/FJsB78q/HVvle9DjuV2TjuXdInTvh34vlgAeBVfLXK6TOnfBeHAec\n8MZ9AJ4FhqTO3oJ7sQXZpLkHevn+gOtmkS3+eiZzbQdcCBARvwGWk7RigRmL0u+9iIjbI+L5/OVv\noDmTY0qo3kl+B5BtrPFMkeEKVs+92BW4PCKeAIiITt1Zqp578Rd4c+mIdwPPRvZcsaNExC1k6yX1\nZsB1s8jCX2sy18p1HNOJBa+ee9HTV8lWSOxE/d4LSSuT/dKfnb/VqQ+m6vl7sTqwvKSbJP1W0u6F\npStWPfdiCvARSX8G7gMOLChb2Qy4bha52Xq9v6xLDuvsxF/yuv+bJG0F7AFs1ro4SdVzL04Dvh4R\nIUn0Pny43dVzL5YG1iebJLUscLukOyLiDy1NVrx67sVRwL0RMUbSasD1ktaLiPktzlZGA6qbRRb+\nJ4ERPV6PIPuXqa9jVsnf6zT13AvyB7pTgLER0ddHvXZWz734KHBpVvNZARgn6dWIuKaYiIWp517M\nA/4vIhYCCyX9L7Ae0GmFv557sSnZssxExKOS/gisSTZ/qEoGXDeL7Op5czKXpHeSTeZa8hf3GuCL\n8Oas37/XGEC7AAAA+0lEQVRHxFMFZixKv/dC0kjgCmC3iJhT4xydot97EREfiogPRsQHyfr59+3A\nog/1/Y5cDWwuaSlJy5I9zHuo4JxFqOdezAa2Acj7tNcEHis0ZTkMuG4W1uKPiEWS3pjMtRRwXkQ8\nLGnv/PuTI2KqpG5Jc4AXyfb/7Dj13AvgGOC9wNl5S/fViNgwVeZWqfNeVEKdvyOzJU0H7gdeB6ZE\nRMcV/jr/XnwXuEDSfWSN2CMi4rlkoVtE0iVkC9etkE+aPZasy2/QddMTuMzMKsZbL5qZVYwLv5lZ\nxbjwm5lVjAu/mVnFuPCbmVWMC7+ZWcW48JuZVYwLv5lZxfw/0HJA1Kocg8AAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x111634e48>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Create our gaussian noise\n",
"std_deviation = 0.3\n",
"noise = std_deviation * randn(N)\n",
"t = tground + noise\n",
"\n",
"plot(xground,yground)\n",
"plot(x, t, 'ro')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"The values of the coefficents will be determinted by fitting the polynomia to the training data. To do this, we define an _error function_ that measures the misfit between the function $y(x, w)$ and the training set points.\n",
"\n",
"One simple choice is the **sum of squares** of the errors between the predictions $(x_n, w)$ and the target value $t_n$ for each point $x_n$, so that we minimize\n",
"\n",
"$$E(w) = \\frac{1}{2}[(y(x_1,w) - t_1)^2 + ... + (y(x_N, w) - t_N)^2]$$\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Proposition 1.1** There exists unique $w^* = (w_0, ..., w_M)$ that minimize the error function $E(w)$ above. More precisely, $w^*$ are given by the solution to the following set of linear equations:\n",
"$$A_{i0}w_0 + ... + A_{iM}w_M = T_i, i = (0, ..., M)$$\n",
"where \n",
"$$A_{ij} = x_1^{i+j} + ... + x_N^{i+j}$$\n",
"\n",
"$$T_i = x_1^i t_1 + ... + x_N^i t_N$$\n",
"_proof_\n",
"\n",
"$$E(w) = \\frac{1}{2} \\sum_{n=1}^N(w_0+w_1x_n+...+w_Mx_n^M - t_n)^2$$\n",
"\n",
"Differenting with respect to $w_i$, we get \n",
"\n",
"$$\\sum_{n=1}^N(\\sum_{j=0}^Mw_jx_n^j-t_n)x_n^i = 0$$\n",
"\n",
"$$\\sum_{j=0}^M \\sum_{n=1}^Nx_n^{(i+j)}w_j - \\sum_{n=1}^Nt_nx_n^i = 0$$.\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ 15.41883605 -11.98130554 -8.63334807 5.77124322 -0.0995302 ]\n"
]
},
{
"data": {
"text/plain": [
"[<matplotlib.lines.Line2D at 0x1119a9898>]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecFeX1x/HPV7CgYAuJDYhR1EgwBhN7wxgVVmOXqIld\nrAj2FnWYny2WWMCOPbHHgiigoMHYsMSKQQ020FiIRgMG+/n98Qxwd7m7e/eWmVvO+/Xal7t35s4c\nL7vnzn3mPOeRmeGcc65xLJR1AM4559Llid855xqMJ37nnGswnvidc67BeOJ3zrkG44nfOecaTMmJ\nX9K1kj6U9HIr2/tL+kzS88nXKaWe0znnXPE6l+EY1wEjgRvb2OcRM9u+DOdyzjlXopKv+M3sUeA/\n7eymUs/jnHOuPNIY4zdgI0kvShorqU8K53TOOdeKcgz1tOc5oKeZ/U/SQOAeYPUUzuuccy6Piid+\nM5uV8/04SZdJWtbMPsndT5I3DXLOuSKYWYeG0yue+CUtB3xkZiZpPUAtk/5cHQ2+XkkabmbDs46j\nGvhrMZ+/FvP5azFfMRfNJSd+SbcAmwPdJc0AImBhADO7EtgVOFTSN8D/gN1LPadzzrnilZz4zWyP\ndrZfClxa6nmcc86Vh8/crU6Tsg6gikzKOoAqMinrAKrIpKwDqGWqloVYJJmP8TvnXMcUkzv9it85\n5xqMJ37nnGswnvidc67BeOJ3zrkG44nfOecajCd+55xrMJ74nXOuwXjid865BuOJ3znnGownfuec\nazCe+J1zrsF44nfOuQbjid855xqMJ37nnGswnvidc67BeOJ3zrkG44nfOecajCd+55xrMJ74nXOu\nwXjid865BuOJ3znnGownfuecazCe+J1zrsF44nfOuQbjid855xqMJ37nnGswnvidc67BeOJ3zrkG\n44nfOecajCd+55xrMJ74nXOuwXTOOgBXXfpKTT1haDdYbBZ8MQNGTDEbm3Vczrny8cTv5ukrNW0I\nF4+C3nMfGwyr9pXw5O9c/fChHjdPTxiam/QBRkHvXnBEVjE558qv5MQv6VpJH0p6uY19Rkj6p6QX\nJfUr9ZyuMrrBYvke7wpd0o7FOVc55bjivw4Y0NpGSU1AbzNbDTgIuLwM53QVMAu+yPf4bJiTdizO\nucopOfGb2aPAf9rYZXvghmTfp4ClJS1X6nld+c2AEYNhWu5jB8Ib02FkVjE558ovjZu7KwEzcn5+\nF+gBfJjCuRuCxCLAD4BuwJKEoZnPgf8mXx+Z8W17x5liNravRBMc0RW6zIY502Gk39h1rr6kVdWj\nFj9b3p2k4Tk/TjKzSZUKqFZJLAZsBGwBrAWsCfwQ+JiQ5GcRhmwWJ7wJLAV0k5gGTAWeBx4G/p7v\nzSBJ8p7onatSkvoD/Us6hlneHNzRQFYGxpjZWnm2XUFI4rcmP78KbG5mH7bYz8ys5RuEAySWBnYD\nBgEbAK8ADwHPAa8C08z4so3ndwVWB34MrAdsSfjU9VfgVuBes/zj+8656lZM7kwj8TcBQ8ysSdIG\nwEVmtkGe/Tzx55AQIUEfBGwDTABuBh4y47MyHH95wk353wHrAH8BLjPjhVKP7ZxLTyaJX9ItwOZA\nd8K4fQQsDGBmVyb7XEJIMp8D+5nZc+UIvh5JdAJ2Bk4kjNWPAG4za/MGeqnn7AHsBQwBXgbOASaZ\n5R+Sc85Vj8yu+Muh0RN/coX/a+BcQpXUH4AxZnyXYgyLEj4BHA/MBI424+m0zu+c6zhP/DVK4qfA\nhcAKwDHA+CyvtpNPHXsDZxJuBJ9k1qwyyzlXJYrJnd6yIUMSXSTOAyYCdwFrmzEu6yEWM7414zpg\nDeBt4AWJYZL/vjhXD/yKPyMSGxFmPb8ADDFjZsYhtUpideBa4Dtgf7Pmk7ycc9nxK/4aINFZ4izg\nTuD3ZvymmpM+gBmvE27g3wVMljgouSfhnKtBfsWfIokVCXXzc4DfVXvCz0diDeAO4EXgUDNmZxyS\ncw3Nr/irmMQvgWcJ9fgDazHpA5jxGmES2dfA0xJ9Mg7JOddBnvhTIHEYcBOwlxmnp1miWQlm/M+M\n/YHzgUlS691ZnXPVx4d6Kigpi/wjYebtdma8kXFIZSexMWHW7+lmXJZ1PM41Gq/jryISSxDG8xcH\ndq3kzNusSawK3Ac8ABxTSCdQ51x5+Bh/lUiaqk0gdMwcWM9JHyD5JLMR8DPgRim07HDOVSdP/GUm\n8QNgEvA0oeb9q2wjSkfy5jaQ0Ab6zqR9tHOuCnniLyOJnsCjwGjgqFq/idtRZswhNJj7HzBWolvG\nITnn8vDEXyZJh8tJwFVmRFm3XchK8gnnt8CbwP3JvQ7nXBXxxF8GEisQFka5wow/Zh1P1pKbuwcB\nbwD3SnTJOCTnXA5P/CVKxvQfAm4047ys46kWyTDXgcAHwN0+5u9c9fByzhIk1TuPAKPNOC3reKqR\nRGfCymGLAruY8U3GITlXV7yOP0XJFewDhO6aRzbqmH4hkvLOMcB7wIH+WjlXPp74U5LMyL2d0K9m\nz0ar3ilGsuD7w8CDZpySdTzO1YticmfnSgVTr5J2xJcQ6tW39aRfGDNmS2wLPC7xgRmXZB2Tc43K\nE3/HnQCsD/Q348usg6klZsyU2AZ4TGKGGaOzjsm5RuSJvwMkdgYOBzYw479Zx1OLzHhLYkfCBK/p\nZjyfdUzONRov5yyQxC+AK4EdzHgv63hqmRnPAIcCo5PFaZxzKfLEX4BkVu49wGAznss6nnpgxl+A\nKwgTvBbPOh7nGolX9bQjmXX6KHCHGedkHU8+irUM0APonnwtAywMdEq+vgRmAbOBT4B3gfcssi8y\nCTiR3Ci/AVgE2MPLPJ3rOC/nLLMkMV1HmHy0Z9aJSbG6EFofrwv8AlgD6E1I8jOAmcC/gf8QSk2/\nAb4jxN8V6EZ4Y1gJWBH4FPgHMCX5ehZ40SJLbZJVMh/iMeBWM85P67zO1QtP/GUmcThwMLChGZ+n\nfv5YnYH1gK2Sr37Aq8AzwN+BV4BpwEyLOvYPqVgLAcsDfYC+wE8J1Uo9k+M/AowFnrPIKlqyKtEL\neIqwNOXESp7LuXrjib+MJDYlLCm4UZpLJibJvj+wG7AT8D7wIGFhl8cssv9V+PzLAhsCvwSaCMNG\nYwkT1h6yyL6uyHnFFsAthIqptytxDufqkSf+Mkm6bT4LHGDG+FTOGas3oanZvoRhm9uBv1hkb6Vx\n/jbiWgXYHtgdWIXwZngd8GxHP2W0ey5xNKGl88ZmZHr/wbla4Ym/LHHQmXB1PcmMuKLnCsMtA4Gj\nCEMtNwJXW2SvVvK8xUreBPYADiDcH7gCuNkim12W44d7KncAH5lxWDmO6Vy988Rfljg4HdgAGFCp\nRcMVaxFCAj2OcBP2fMLVfU3MBE7esLYCDgE2BUYBIyyy90s+tliKcP/iFDNuLfV4ztU7T/wlx8A2\nwDXAz834sOzHj9UJ2BOIgbeAc4AJ5R4ySVPyKeAowhDNncCZFtnbJR1T9CPc19jEjNdKDtK5OuaJ\nv6TzsxJhXH93Mx4p67FjCdgOOItQT3+SRVbWc2RNsboDwwgzcm8nvAEUPcNZ4mBCe4z1k7V8nXN5\neOIv+tx0AiYCD5lxRlmPHWsN4GKgF3A8cH8tX+G3J3kDOI5wo/oawhvAZx0+ThjvvwX42IzDyxul\nc/XDE3/R5+Zkwpj1r8o1rq9YSwCnAfsTrvQvqVQpZDVSrBWA0wmfdIYTblp3aGJYssLZ84SFbryT\np3N5eOIv6rxsAIwmjOu/W5ZjxvolcDXwBHCsRfZBOY5bixSrH3AB8D3gEIvsiQ49X2wE3EX49/Hm\neM61kEnilzQAuIjQE+ZqMzunxfb+hMT6ZvLQnWa2wHBKFok/qSB5HjjGjLtLPl6spYBzCSWah1hk\nY0s9Zj1I7nEMIrwB3Ee4x/FJwc8XpxImtW1dqUor52pVMbmzpO6ckjoRVqMaQJj6v4ekNfPs+oiZ\n9Uu+yjqGXqJLCEsBliPpb0RYf1fAWp7057PIzCK7DfgJoXz1FcXarQOHOIuwdsQxlYjPuUZT6kIs\n6wHTzEL5nqRbgR2AqS32y7w+vyWJQYT4+5V0nNBi4WTgMOAgi+zeMoRXlyyyT4EhivVn4HrF2hUY\nYpHNbPN5xrcSewPPSDxgxotpxOtcvSq1H/9KhPYCc72bPJbLgI0kvShprKQ+JZ6zZMniHyMJTcGK\n7n2jWCsSFhDfDFjHk35hLLLJhDfc6cBLirVju88x3gGOBf6cdPR0zhWp1MRfyA2C54CeZrY2Idne\nU+I5S5KUCV4LXG7G00UfJ9amhC6WE4CtLbJ/lSnEhmCRzbHIjgN2Af6oWFcmlVAA9JWaBkrjB0mT\nBkrj+0pNwJ+A16C8JbfONZpSh3reI7TxnasnNK+MMbNZOd+Pk3SZpGXNFry5J2l4zo+TzGxSifHl\ncyiwLHBmMU9OblQOA04C9rHIUmniVq8ssieSyp9LgL8r1p4/Gc7yG8LFo8JaAwAMhlVBvIIdArwo\ncZ8Zk7KK27msJAUz/Us6RilVPZI6E67AtgT+BTwN7GFmU3P2WQ74yMxM0nrA7Wa2cp5jVbyqR6I3\nMJnQ/bHDrQCSHjtXAmsDu2TdObPeKNaewEW9H2bm63+jT8tfhiYYP9ZsoEQTcBnwU1/03jW61Kt6\nzOwbYAjwAGElp9vMbKqkgyUdnOy2K/CypBcIZZ+7l3LOYiWzc68Hzigy6XcnDOssCWzqSb/8LLKb\ngY0/WJsf/nYXmL1I8+1doQuAGWMJM63PSz1I5+pAw0zgkjiKsLBJfzM6tKJU0nbhfkIv+pMrvSJV\no9tqUT3YawBbPdET7roN1vx3eHzuFT/Mm4PxEjDYjAezi9a5bKV+xV8rJFYHfg/sX0TS34CwDOFZ\nFtmJnvQr7/2vuGihe5l27BOw+X4wZnU4EN6YHooDADDjM2AwcHXyJuCcK1DdX/EnQzyPAreYzU8c\nBT03VhNheGgfi2xcuWNzresrNfWCI2avwnKTd6FP54+5fU4v9m35xitxJdDJjAMzCtW5THmvnrzH\n5WjCpLItWrva7ys19YSh3WCxWfDFDBjxynC6E9ov7JjUnbuMJA3f7iRUke2Tu+6wxJKEIZ+DzXgg\noxCdy4wn/gWOyarAU4QFvKfl26ev1NSydHD9DZn5zC/BFmZzi6zlLGSXAcValND4bg1gh9zVvpIF\ndK4E1jJjViuHcK4u+Rh/jmSi1ijg7NaSPkBPGJqb9C/cAD5aj+9vfBX/8KRfPZJlKfcGxgCTFWvt\nedvClf4kQk8f51w76jbxExYC6UpYBKVV3Zg//f/sTeCydeGR62GFNrvHuCwkzd5OJyxoM0Gxts7Z\nfDSwi8Qm2UTnXO2oy8Qv0YNw9be/GW0u/jELvgA4c1O4ce2Q9Ht9BrPx5f6qVdLpc2fgT4q1D4AZ\nnwBHEKp8vJePc22ouzH+ZIhnNPCcGcPb27+v1LTo5twwey26P3I9LD87lA5OhqFTLN3WyvluMqcd\nQy1RrDWBsYSx/7MsMpO4E3jNjJOzjc65dPjNXUBiVyAG+pnxVbv7xxqmrzhh0yt4dblPWGg2zJkO\nI7NI+nn600x7EoZ58m9dUvEzjlCyO4zhthyhymdLM17KNDjnUtDwiV9iGWAKMMiMx9vdP9bBhGZr\nm1tk75Ry7lINlMaPg21aPp47W9Xll6x8NobQIHAfhtt+wAHARr5il6snirU00NUim9cM06t64A/A\nvQUm/UGExdC3zDrpQ/ObzLnm9qdxrbPIPiO8aXYFRnN4n5uALwmL4zhXT/YAzi71IHWT+CU2A7YF\nTmx331hbEab/D7TI3qh0bIWYe5O5Jb/JXBiLbA6ht/+/+f7U8awy4Sggkpq1DXeu1m1L6BtWkrpI\n/BKLEibwDE16uLS+b6z1gJuAXS2yqhkDngEjBtN8vkHL/jSubRbZ18C+wMvsvfXlLPPmKOCy5Ia/\nczVNsboQVvsreYZ6XYzxS5wG/NyMHdrcL9bqhIZrB1lkY4o5VyXN7U/TFbpkdZO5HiSL5ZyHaWv+\n+O6izF7xJDPuyjou50qhWAOBkyyyzZo93og3dyVWA54E1jFjeqv7xfoB8ARwtkV2TfGRulqQJP/h\nfLXEvox8tTOzevzY2zm4WqZYlwDvWmR/aPZ4o93cTT7CXw6c1U7SXxy4F7jVk35jSGb5Rizy+VUM\nXn8Jvv/KBVnH5FyxkguZsozvQ40nfmBP4HvAiNZ2UKxOwJ+BfwKnphSXqxIW2Zl0+moke2y/n/pd\nv0C5rHM1Yk2gE6FcvWQ1m/iTmv3zCe1422rLcA6wDHCARVUyruVSZefOPJWZfe6ifzRaxy3vVT6u\nFm0L3F+uHFaziZ9Qy3q3GU+3toNi7Ufoxb+LRdbuLF5Xx24Z8xum7PEeC331bDLb17laUrZhHqjR\nxC+xPiGht9qPRbE2IVztb2+RfZJWbK46mWFM/MOveXpoV77tPEmxlss6JucKkczWXQd4uFzHrLnE\nL9GZcEP3ODM+zbtPrJWBO4C9vae+m8uMfzBp+Ehe2OdLYKJidc86JucKsBXwWO7Kc6WqucRPmIb/\nGWES1gIUawlCd85zLLLxaQbmasLpjLlqSd7v9xKhp/8yWQfkXDu2JXShLZuaSvwSKxD66xxmxgI3\nOZKSp1HAC7SzAItrTGZ8DgsN48pnf863Cz8GjFWsblnH5Vw+itUZ2I5Qjl42NZX4gQuAq8xobfjm\nKMKarId4BY9rjRmjYaHXOWPOB4TyuHuT6fDOVZtNgHcsslbnKRWjZhK/xJbAhsCZebfH2oKwJN/O\nScMu59oyFOt0NOd9cDbwL+AuxVok66Cca2En4O5yH7QmEr/EIsAlwJHho3qL7bF6ADcDv6uGFsuu\n+pnxNnAhny93AaGx25eEpRw7ZRmXc3MlQ9c7AveU+9g1kfgJQzhvEm7aNqNYCwO3ASMssolpB+Zq\n2nlAH4bb1sDuwPeBy5M/OOey9jPgK+CVch+46hN/0k/9OELL5Xzj9n8APiXU7DtXMDO+BIYAIxhu\nIswN+RllWOjCuTLYCbinEvcrqz7xAxcCI81YYMEUxdqJsPjG3hbZd6lH5mqeGQ8CzwEnWGSzgIHA\nrxXr+Gwjc44dqcD4PlR54pfYGugHnLvAtlirEhZfGWSRfZx2bK6uHA0MkVg1+V3aBjgsafnhXOqS\n/PYDYHIljl+1iT9ZVesSwhBPsyqdpPriVuBMi6zVXj3OFcKMGYTx/osBkoWstwHOUqxfZxmba1g7\nAqMrNZJRtYkfOAaYapa3MdGZwPu00Y7ZuQ66EOgt8WsAi+w1YHvgGsXaNNPIXCPaiQpU88xVlStw\nSfQijLuua8ZbzfaLNYAwO7efRfbv9CN19UriV8BVwE/mfspUrK0I6zlsaZGVpRe6c21RrOWBqcDy\nFtmX7e5fRytwXQiMyJP0VwCuI9Tre9J3ZWXGROBZ4IR5j0U2gVBOPFaxvJe/S8OuwJhCkn6xqi7x\nS2xDKKlrdkNXsRYCbgCussgeySI21xCOIbnRO/cBi+xm4CLgAcVaNrPIXKP4DWFuUsVUVeJPbuiO\nBIaZ8UWLzUOBJYDTUw/MNYzkRu/5hEQ///HILiB0SPS+Pq5iki4EfYAJlTxPyYlf0gBJr0r6p6QT\nWtlnRLL9RUn92jjcUcBrZtzX7Pmx1gJ+D+xlkbW1zKJz5XABsLrEdi0ePx6YDtzkrR1chexGqOap\n6IqBJSV+SZ0IJZcDCO9Se0has8U+TUBvM1sNOIiwiEprjgWGNXt+rMUIfXiOs8jeLCVe5wphxlfA\nEcDFEovNezyU1u0HLA1c5K0dXAUMosLDPFD6Ff96wDQze9vMvibU1u/QYp/tCWPzmNlTwNJSq8ve\nXWpGy+R+NvDa3GM4l4ZkRu8LhKv8+Y+HG247AZsTWok4VxaK9UOgN2VcYrE1pSb+lYAZOT+/mzzW\n3j49WjneH3J/UKxfEj76HOz99V0GjgKGSqyc+6BF9hmhtcPhivXbLAJzdWkQcLdF9nWlT1Rq4i80\nGbf8SJz3ebkzdBVrKULp5mBvyeCyYMZ0QmnxhQtsi+w9wpJ4FyZrQThXqopX88zVucTnvwfk1jb3\nJFzRt7VPj+SxBUgaPu+HnVmXnzLeIhtXYozOleJ8YIrEADOareFskU1RrN2B2xRrC4us7O1zXWNQ\nrN6EPNluqbqk/kD/ks5XysxdSZ0J4+9bElYxehrYw8ym5uzTBAwxsyZJGwAXmdkGeY41f+Zu6I9y\nMbB20jHRucxINBHKO9dKWjk33x5rL+AMYEOL7F9px+dqn2KdAqxgkR3e4eemPXPXzL4h9DN/APgH\ncJuZTZV0sKSDk33GAm9KmkbopnlYW8dUrO8l++3nSd9VAzPGEqbQH5N3e2R/IrR6uN8XbncdlVSH\n7Q3cmNo5q6lXzwB44KGhLPb1sjxvkR2VdUzOzSXxI+AZYJ1k7L/59vDHexWwIrCDzzdxhVKsDQhV\niz8upoil5nv1HLIG2ywqNlrtQv6WdSzO5Ur6Ro0kTO5acHv4gz0M6ARc4jX+rgP2Am4sJOlLrCCx\nZaknrKrEf9i2cP89LNz7Mw7KOhbn8jgX6JcsELSApAxvELAhLer/nctHsRYlVPP8ucCnnAf8qtTz\nVlXi33kqbPYOdAXvheKqTlJuPAwYmfSVWnCfyP5LKPMcoli/STM+V5OagJctsnfa21Fic2AzQiFB\nSaoq8Z/9UPjvbJqvuOVctUj6SL1OmNyVf5+wgtd2hCGfjdOKzdWkgm7qSixMaI9ztBmfl3rSqkr8\nXb+CA+GN6WEs1blqNQw4VqLV/vwW2YuEsds7FWu11CJzNUOxugNbAHcWsPsQwqqDhezb/rmrqapn\nIIyfDiOnhBJQ5zLVV2rqCUO7wWKz4IsZMGLu76bEcMJKXbu1dQzFOojQ02dDXzzI5VKsw4GNLLI2\n235IrAC8BGxixmsLbu94VU9VJf6OBu9cpfSVmjaEi0eFplkADIZpT8KwKWZjJboArwAHm7XdO12x\nzgE2Bn5lkbVcZ8I1KMV6BjjFInugzf3En4EZZpyUf3uNl3M6Vy16wtDcpA8wCnr3Cu2ac2/0XtLa\njd4cJxFmtl+XrCTnGpxirQN8H5jY5n6iP7ApZbihm8t/CZ3Loxvz+/Dnyq04M2MMoWVJ3hm98/YL\nffz3AX5Imf+AXc0aDFxtkX3b2g7JDd1LgaPKcUM3lyd+5/KYxQJLfwJ5K86GAce0bN3ckkU2h7BW\nxSDFOrAsQbqapFhdCbX717Wz65GEFd/uLncMnvidy2MGjBgM03Ify1dxlszovYg8rZtbsshmEmr8\nz1SsrcoZr6spuwN/S1p75yXRAzgBGGpWcPv7gvnNXeda0Vdq6gVHdIUus2FOaxVnyfKMLwNHmnF/\ne8dVrM2AvwBbWmQvFxJHa9VFrvYo1tPAcIta/zeUuB141YzT2j2eV/U4lw2JbYDLgL65Cwq1un+s\nPQnLirbZyrm96qJyxO7So1j9gNHAj1ob309aglxBKBdu/3fJq3qcy4YZDwB/B04uaP/Ibia0Hx+T\njPnm1V51kas5bd7UTT49XgoMKSTpF8sTv3PlcxRwqMQaBe5/NvA8cKti5V0Nr5DqIlcbkjf43YFr\n29jtBODlZA2IivHE71yZmPEecCZwqbTAOtML7h/a8B4KLAKMyNfKuQPVRa767Q1MSno5LUCiN+GT\n3JGVDsQTv3PlNZIwMaegzpxJK+ddgU3IMx+g0OoiV92SiXvDCBVgC24PFwqXAOfkW+in3EpdbN05\nl8OMbyQOAf4iMc6Mz9p9TmT/VaxtgScU6x2L7I6526aYje0bFv1tt7rIVbVtgP8Bj7ayfRegB628\nMZSbV/U4VwESVwFfmTGk4OfE+hnwILCTRfZ4xYJzqVOs8cCtFtn1C2wTSxH6Pu1uxmMdPrZX9ThX\nNU4EdpFYr9AnWGQvML+V8+oVi8ylSrHWBH4G3NrKLmcA44tJ+sXyxO9cBZjxCaEd85VS4UOqSafG\n3wPjFOsHlYrPpWoocGW+zqwS6wK7Eap5UuOJ37nKuQn4mA7W3Ftk1yTPHaNYS1QiMJcOxVqWUMJ5\n+QLbwgXBlcBxZnycZlye+J2rkKTHyqHAyW2t1tWKCJhKGzX+riYcBIyxyD7Is20I8B8KX2i9bPzm\nrnMVJnEqsC6wQ0cabinWwsB9wNvAIUndv6sRitUFeIuwAM+UZttEL+A5YON8q2p16Dx+c9e5qnQO\nsCqhZK9gOTX+6xHG/V1tOQCYnCfpi9DX6aJSk36xPPE7V2FmfEX4yH+xxNIdem5ks4Am4ADF2r8S\n8bnyU6xFCDf3z86zeTdgZeDcNGPK5YnfuRSY8ThwL+Hqv2PPjex9YABwVjLRy1W/PYF/WmRP5T4o\nsQxhktbg5IIgE574nUvPicC2Ept29IkW2WvAjsD1irV+2SNzZaNYnQj/1mfl2XwOcLcZT6YbVXOe\n+J1LSdK+YShwddJ+t2PPj2wysC8wWrEK7QDq0rcT8Cnw19wHk4XTBwInZRBTM574nUuRGXcRVutq\nd2WlvM+P7H7C1eQDirVSOWNzpUuasZ0CnJVbhSWxOHA1cJgZ/80qvrk88TuXviHAgRLrFPPkpN/L\nZYTkv2w5A3MlGwR8BYxp8fj/AU+bLfB4JjzxO5cyMz4gVHxcI7FwkYc5DxgP3Oeze6tDMu/idODk\nFlf76wG/I7Rlrgqe+J3Lxo3Ah4Q3gA5LEsvxwOvAHUn5oMvWvsB0i2zi3AckFgGuAY4yY2ZWgbXk\nid+5DCQzeA8GjpboU9QxIvuOsIbr18CNSTWJy0AyS/c0Flxz+feE2butdebMhCd+5zJixjuEG4HX\nd6SDZ7NjhNm9vwF+AFyeb/lGl4rDgGdz6/Yl+hF6NR3SkVYdafDE71y2rgQ+A44t9gBJu98dgLWB\ncz35p0uxliIMu50y77EwxHMDcIwZ/8oqttYUnfglLStpgqTXJT0oKe9UdElvS3pJ0vOSni4+VOfq\nT3IleCBwjETfoo8TWjsMJMzwPbVM4bnCnEbowPlKzmOnEoZ4Uu+8WYhSrvhPBCaY2erAQ8nP+RjQ\n38z6mVnwbnc8AAAPAElEQVTBqxE51yiSIZ+TCUM+xVb5YJF9AmwF/Faxirpp7DpGsX4M7E3O2L7E\nzwm9mapuiGeuUhL/9oSPMiT/3bGNff2jp3Ntu5qwaEtrF1AFSfq+bwkcqliHlyMwl18ypHYRYbLW\nRwDJjOwbgKPNeD/L+NpSSuJfzsw+TL7/EFiulf0MmCjpWUmDSzifc3UruTLcHzgiuWIs/liRvUtI\n/ico1oHliM/ltR3wQ+CSnMfOJCygc3MmERWozUoCSROA5fNsatYb3MxMUmsfaTY2s/clfR+YIOlV\nM3u0lfMNz/lxkplNais+5+qJGe9JDAX+LLGOGXOKPlZkbynWlsDDivWdRXZt+SJ1irUocCFwWFJZ\nhcQWhAqrtSs5xCOpP9C/pGMUuwKXpFcJY/cfSFoB+KuZ/bid50TAbDP7Y55tvgKXc4DELcBHZqXP\n9FSs1YGHgVMtsutKDs4BoFinAOtaZDsASCwFvEQY1x+Xaiwpr8B1L7BP8v0+wD15AlpcUrfk+yWA\nrQkNqpxzrTsc2FniV6UeyCJ7nTDsc7pi7Vvq8RwoVh/gSOCInIdHAGPTTvrFKuWKf1ngdqAXYU3Q\nQWb2qaQVgVFmtq2kVYC7kqd0Bm4ys3wr0vgVv3M5JLYCrgV+ZsbHJR8vtHGeCJxukV1V6vEaVTI7\n+jHgTxbZZQASg4AzgH5mfJ56TEXkTl9s3bkqJXE+0BvYqRxjxorVm1B6fb5FNrLU4zUixTqS0G9/\nC4vsO4mVgaeBgWb8PZOYPPE7Vz8kFgWeBEaZcXlZjhlrZULyv8IiO68cx2wUirUK8PRKd3PiWi+y\na1fU5V02+Ok0trtrpp18QGZxeeJ3rr5IrEEYWuhvxivt7V/QMcMCLg8BdwCn5bYQdvklQzwTl3iT\nN/a4kc1HhU9iAAyGaU/CsClmYzOJLeWbu865CjPjNeAE4BaJLmU5ZmTvAZsBTcAlyapRNaGv1DRQ\nGj9ImjRQGt9Xakrp1CcCC238Z3rkJn2AUdC7V/MbvVWvZv7BnWtg1wFTgIvLdcBkpukWwE+Am2qh\nn39fqWlDuHgcbHM7bD4OttkQLq508lesTQiJfc+lvsu/VnJXyvOmnBZP/M5VuZze/ZtL/K5sx43s\nv4TGbosDYxRryXIduxJ6wtC0r7YV63uEWbgHWGTvzYIv8+03m+In22XBE79zNcCMWcBuwIUSa5bt\nuJHNAXYhdJL8m2KtWK5jl1s30r3aTnrxXAv8JVnknmc458M9WalZkj8Q3pgONVUl5YnfuRphxkuE\nseY7JMq2zq5F9g1hwZBbgScV6yflOnY5zYIv8j1ewavtk4EVSBrnSWzxMcdv9TQ/OLgJxg+CR5pg\n/GQYmtWN3WJ5VY9zNURCwPVAJ2CvcveEUaw9CR0n97HIqmoW6twx/tzhngPhjUokXsXaFbgAWN8i\ne19iJUK9/r5mTCjnuUrl5ZzONQCJxYHHgevNynfDd97xY21MKPU8H7iwmso9+0pNveCIrtBlNsyZ\nDiMrkPR/AYwDtrbInk/mUzwCjDYjb+eBLHnid65BJDNGJwO7mzGp7MeP9UNgNPA8cIhFlvemZr1J\n5jg8BRxhkd0NIHEVsCywWzUurOJ1/M41CDPeBvYi1Pf3LPvxI3sH2AToBjyWzPita4r1feBBYERO\n0j8I2BjYrxqTfrE88TtXo5Kx5guAu5Phn/IeP7LZhEqiW4DJijWw3OeoFknZ5kTgLovsXACJjQnN\n13ZKqqrqhg/1OFfDkpu9NxBKGn9jxncVOU+sTQlvADcC0dzFR+qBYi1NSPoPAydYZCaxCuE+yr5m\nPJBpgO3woR7nGkwy/DCYUHb4fxU7T2SPAj8H1gYeTxZ4qXmK1R14gJDk5yb9pYD7gDOqPekXyxO/\nczXOjC8JrYL3kNirYueJ7EPCOrM3EJL/Qckkp5qUtKl+gtCw7sgk6XcmrDPykBmXZhpgBflQj3N1\nQqIPMAkYVIlKn2bnirUm8CfgM+Agi+yNSp6v3BRrQ8IiUdHchWmSYbPLgZWB7cz4JrsIC+flnM41\nuGTB79uArcx4saLnitUZGAacBJxLqPmv6rH/5BPKvsA5wL4WzZ8DIBEB2xNaYNfMzVxP/M45JHYj\nzL7dxIy3Kn6+sEDJFUAP4GiLbHylz1kMxVqKEGdfYHeLbN76BhKHAMcCG5vxYUYhFsUTv3MOAIkh\nwFBCIptZ8fOFK+lfE2b7TgOOy02sWUtaK98IjAeOSZrThW1iJ+BSYFMzamrICjzxO+dySJxBWGxl\nSzP+k8o5Q1//wwmNzSYBZ1hkL6dx7lbiWYEwDLUFObNx520X2xDuVWS2Zm6pPPE75+ZJblb+kTAD\ndyszPkvt3LG6Ejp+HkMolRwJPJJW3x/F6kZ4AzoWGAWcmUxIm7+P+CXhfsiOZjyeRlyV4InfOddM\nkvxHAv2AAWnftFSsxYH9CW8CIoyx32yR/btC5+tBGOI6AJgAnGqR/XOB/cSmwJ2E/juPVCKWtHji\nd84tQGIh4EpgDUKZ4n9TjyHcA9gMOIQw/PR3QuK9L+kLVMqxuwM7EBaU2YAwz+Bii+ztvPuLTQil\nnHuaMbGUc1cDT/zOubyS5H8p8AvClf/HmcUSqwuwDbAzMICwnOHjhG6jrxFWA3vbIvuixfM6Ad8j\nzFL+GWEm8S8I6wZPILyR3J8sKZn/3PPH9H9bbX31i+WJ3znXqmTY5w/AtoQx//czDmnuJ4FVCR0w\n10++XwXoCXxLeFP4irDwzDLAp8CHwEvAs4RPDk9bZP9r91xiF8IErR3NeKLs/zMZ8cTvnGuXxEmE\nMfABZkzLOp58kqv7xYBFk6/vgI+TZSI7fjxxAKHTZpMZz5ct0Crgid85V5Ckz/z/Abua8Vh7+/eV\nmnrC0G6w2Cz4YgaMqIV1ZpMhrjMJ7aW3NeO1jEMqu2JyZ+dKBeOcq15mXCXxNnCXxFFm3NTavvnW\nuh0Mq/aVqObkL9GFMGlrBWADMypSSVSLvDuncw3KjAcJE5vOkDgr6Uy5gJ4wNDfpA4yC3r3giDTi\nLIZEL8I6uV8SJrB50s/hid+5BmbGK4SbqusCEySWb7lPtzDWvoCuYfGXqiMxEHgG+AuwV9K22uXw\nxO9cgzPjI0JZ5d+Av0v0z90+C77I97zZMCff41mR6CxxJmGm7q5mnFtP6+SWkyd+5xxmfGtGBOwH\n3Cxxwdx1fGfAiME0r/45EN6YHmYEV4VkLYInCJ9c1jHj0YxDqmpe1eOca0aiOyGp/xzYz4zH+0pN\nveCIrtBlNsyZDiOr4cZucl/iWEJPoFOAqxrtKt/LOZ1zZZPTrngscIoZH2QcUjPJojMXAjOBA80o\nqfVDrfLF1p1zZWPG3UAf4D/AFImTkxLJTEmsJnEPcC2hRn/rRk36xSo68UvaTdIrkr6VtE4b+w2Q\n9Kqkf0o6odjzOefSZ8anZhxHaH72C2CaxHESS6Ydi0RfiRsJPX0mA2uacUejDe2UQylX/C8DOxEq\nAfKS1Am4hFAx0AfYQ9KaJZyzIUjqn3UM1cJfi/myfC3MmGbGzoTOmv2AtyTOkVijkudNKnUGSIwB\nJgJTgVVBk83yVxu59hWd+M3sVTN7vZ3d1gOmmdnbZvY1cCuhfaprW/+sA6gi/bMOoIr0zzoAM140\nY09C9czCwCMSkyUOk+hRjnNIdJJYT+KPwAwgBu4DfmTG2WZ8ShW8FrWs0i0bViL8w831LmGyiHOu\nhpnxJnC0xPHAVsDvgP+T+Bh4iNBmeSrwmhmft3acpGPo8sCPgbUIM4k3B94H7gb612N/nay1mfgl\nTYAFZ/IBJ5vZmAKO72NvztUxM74BxgHjkoZoPwW2JHyyPx5YTeI/wCfArOSrM7Ak0I2QX74GXgX+\nAdwBHFptFUT1puRyTkl/BY4xs+fybNsAGG5mA5KfTwK+M7Nz8uzrbxLOOVeErLpztnbSZ4HVJK0M\n/Av4DbBHvh29ht8559JRSjnnTpJmEMq87pc0Lnl8RUn3A5jZN8AQ4AHCx7jbzGxq6WE755wrVtXM\n3HXOOZeOVGfuFjKZS9KIZPuLkvqlGV+a2nstJP02eQ1ekvS4pJ9mEWcaCp3kJ2ldSd9I2jnN+NJU\n4N9If0nPS5oiaVLKIaamgL+R7pLGS3oheS32zSDMipN0raQPJb3cxj4dy5tmlsoXYbHkacDKhPrf\nF4A1W+zTBIxNvl8fmJxWfGl+FfhabAgslXw/oJFfi5z9HibUc++SddwZ/l4sDbwC9Eh+7p513Bm+\nFsOBs+e+DsDHQOesY6/Aa7EpYdLcy61s73DeTPOKv5DJXNsDNwCY2VPA0pKWSzHGtLT7WpjZk2b2\nWfLjU1CeyTFVqNBJfkcQFtaYmWZwKSvktdgTuNPM3gUws3pdWaqQ1+J9mNc6YkngY7PiFmOvZmb2\nKKFfUms6nDfTTPz5JnOtVMA+9ZjwCnktch1A6JBYj9p9LSStRPijvzx5qF5vTBXye7EasKykv0p6\nVtJeqUWXrkJei1HATyT9C3gRGJZSbNWmw3kzzcXWC/1jbVnWWY9/5AX/P0naAtgf2Lhy4WSqkNfi\nIuBEMzNJovXy4VpXyGuxMLAOYZLU4sCTkiab2T8rGln6CnktTgZeMLP+klYFJkha28xmVTi2atSh\nvJlm4n8P6Jnzc0/CO1Nb+/RIHqs3hbwWJDd0RwEDzKytj3q1rJDX4ufArSHn0x0YKOlrM7s3nRBT\nU8hrMQP4t5nNAeZI+huwNlBvib+Q12IjQltmzOwNSW8BaxDmDzWSDufNNId65k3mkrQIYTJXyz/c\ne4G9Yd6s30/N7MMUY0xLu6+FpF7AXcDvzGxanmPUi3ZfCzNbxcx+ZGY/IozzH1qHSR8K+xsZDWwi\nqZOkxQk38/6RcpxpKOS1eBX4FUAypr0G8GaqUVaHDufN1K74zewbSXMnc3UCrjGzqZIOTrZfaWZj\nJTVJmgZ8Tlj/s+4U8loApwHLAJcnV7pfm9l6WcVcKQW+Fg2hwL+RVyWNB14CvgNGmVndJf4Cfy/O\nAq6T9CLhIvZ4M/sks6ArRNIthMZ13ZNJsxFhyK/ovOkTuJxzrsH40ovOOddgPPE751yD8cTvnHMN\nxhO/c841GE/8zjnXYDzxO+dcg/HE75xzDcYTv3PONZj/B/NjX7HizehtAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1117d4390>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"\n",
"# Define our error function\n",
"def err(w, x, t):\n",
" return 0.5 * sum((polyval(w,x)-t)**2)\n",
"\n",
"# We'll use the built-in function polyfit to minimize the error function E(w)\n",
"w = polyfit(x, t, M)\n",
"print(w)\n",
"\n",
"# Let's see what our polynomial looks like (green)\n",
"plot(xground, yground)\n",
"plot(x, t, 'ro')\n",
"plot(xground, polyval(w, xground), 'g')"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train Error: 0.200314182519\n",
"Test Error: 0.227500550622\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYHHWZ9vHvbThkIBHEOCgQZCCImOwqrnIQlKCLJKNy\nUETQFVAJgpJkeQ1GEanUsuAieXclYUXMAoqr4gEFkRAMYgCVg7wCQgCXxGDCYRlPYAITIPK8f1Ql\nTELPTM/0obqn7s91zXVNd9dUPalM313zq99BEYGZmZXHS4ouwMzMmsvBb2ZWMg5+M7OScfCbmZWM\ng9/MrGQc/GZmJVNz8Eu6RNLjku7p5/XJkp6UdGf+dUatxzQzs+HbrA77uBSYD1w2wDY3RsShdTiW\nmZnVqOYr/oi4GfjLIJup1uOYmVl9NKONP4C3SLpb0kJJr2vCMc3MrB/1aOoZzK+B8RHxtKSpwJXA\na5pwXDMzq6DhwR8Rq/t8f62kL0vaLiL+3Hc7SZ40yMxsGCJiSM3pDQ9+SdsDPRERkvYGtGnorzfU\n4kcqSXMiYk7RdbQCn4sX+Fy8wOfiBcO5aK45+CV9GzgQGCdpFZAAmwNExEXAkcDJktYBTwNH13pM\nMzMbvpqDPyKOGeT1/wT+s9bjmJlZfXjkbmtaUnQBLWRJ0QW0kCVFF9BClhRdQDtTqyzEIincxm9m\nNjTDyU5f8ZuZlYyD38ysZBz8ZmYl4+A3MysZB7+ZWck4+M3MSsbBb2ZWMg5+M7OScfCbmZWMg9/M\nrGQc/GZmJePgNzMrGQe/mVnJOPjNzErGwW9mVjIOfjOzknHwm5mVjIPfzKxkHPxmZiXj4DczKxkH\nv5lZyTj4zcxKxsFvZlYyDn4zs5Jx8JuZlYyD38ysZBz8ZmYl4+A3MysZB7+ZWck4+M3MSsbBb2ZW\nMpsVXYC1lklS93iYMRZGr4a1q2DevRELi67LzOrHwW8bTJK694PzF8CE9c9Ng90mSdQa/urq6qaz\ncwYdHaPp7V1LT8+8WLHCHyhmBXDw2wbjYUbf0AdYABO6YTow7JBWV1c3Eyeez6xZL+x77tzd1NWF\nw9+s+Wpu45d0iaTHJd0zwDbzJD0o6W5Je9V6TGuMsTC60vNjoKOmHXd2ztgo9AFmzZpAZ+f0mvZr\nZsNSj5u7lwJT+ntRUjcwISJ2B04ELqzDMa0BVsPaSs+vgd6adtzRUfEDhY6O2j5QzGxYag7+iLgZ\n+MsAmxwKfD3f9jZgW0nb13pcq79VMG8aLOv73AmwfCXMr2nHvb0VP1Do7a3tA8XMhqUZbfw7Aqv6\nPH4Y2Al4vAnHtiG4N2LhJIlumD4GOtZA70qYX3Ovnp6eecydu9tGzT3nnbecnp7aPlDMbFiadXNX\nmzyOihtJc/o8XBIRSxpVkFWWh3xdb7jGihUL1dUFs2dPp6Ojg97eXnp65vvGrtnQSZoMTK5pHxEV\nM3iohewCXB0Rf1fhta+Qhfjl+eMHgAMj4vFNtouI2PQDwszMBjCc7GzGyN0fAccCSNoXeGLT0Dcz\ns+apualH0reBA4FxklYBCbA5QERcFBELJXVLWgY8BXyk1mOamdnw1aWppx7c1GNmNnSt2tRjZmYt\nxMFvZlYyDn4zs5Jx8JuZlYyD38ysZBz8ZmYl4+A3MysZB7+ZWck4+M3MSsbBb2ZWMg5+M7OScfCb\nmZWMg9/MrGQc/GZmJePgNzMrGQe/mVnJOPjNzErGwW9mVjIOfjOzknHwm5mVjIPfzKxkNiu6AHuB\nurq66eycQUfHaHp719LTMy9WrFhYdF1mNrI4+FuEurq6mTjxfGbNmrDhyblzd1NXFw5/M6snRUTR\nNQAgKSJCRddRFO2zzyLOPfeQF70we/aiuO22qQP+bKqXATsB4/KvlwGbA6Pyr2eA1cAa4M/Aw8Aj\nkcTaev4bzKz5hpOdvuJvFR0do/t5vmP9t0rVAbwBeDPwJmAPYAJZyK8C/gD8EfgL8BywDnge2BIY\nA4wl+2DYEdhBqZ4A7gPuzb/uAO6OJNbV/d9nZi3Dwd8qensrX33/7a9bKlUCHAzsBTwA/Aq4GfgK\nsAz4QyRD+9NNqV4CvBJ4HTAJ2AeYAYxXql8BNwILgV9HEs8P559kZq3JTT0tomIb/7z0b7z09pV0\nPX0FsBj4eSTxdEPrSLUdsB/wdqCbrNloIfBd4KeRxHONPL6ZDc1wstPB3yKUagJLx57Dmh0PY9RW\nz/DUukd5qufcuO2xSwuua1fgUOBoYFfg+8ClwB1D/SvDzOrPwd9m8uaWqcCpwN8DlwH/FUk8UGhh\n/cg/BI4BPgY8QdbU9K1IYk2hhZmVmIO/TSjVFmQBehrZTdi5wPcjiWcKLaxK+QfWwcBJwFuBBcC8\nSOKxQgszKyEHf4tTqlHAB4EUWAGcCyxu5yaT/K+AU4EPAVcAZ0cSDxValFmJOPhblFIJeDdwDll/\n+s9GEjcWW1V9KdU4YCZwMtmN4LMjiUeKrcps5HPw10k9p05Qqj2A84GdgU8D17TzFf5g8g+A04AT\ngIvJPgCeLLYqs5HLwV+POipPnbCMpUtnDiX8lWpr4Ezgo2RX+heUqSukUr0KOIvsL505ZDetPTDM\nrM4c/PWoo4apEzbsI9Xbgf8CfgnMiiT+t75Vtg+l2gv4d+DlwEmRxC8LLslsRBlOdtY8LbOkKZIe\nkPSgpNkVXp8s6UlJd+ZfZ9R6zIaqYuqE/ijVNkp1EfA14JRI4p/KHPoAkcSdZIPBzga+p1QX5YPE\nzKwgNQW/pFHABcAUsqH/x0jas8KmN0bEXvnXv9ZyzIbrb+qE3t7egX5Mqd4C3AUI+LtIwjNq5iKJ\niCS+A0wk6766VKneX3BZZqVV6xX/3sCyiHgoIp4DLgcOq7Bd4U04Vevpmcfcucs2eu6885bT0zO/\n0uZKtZlSnQn8AJgZSZzom5mVRRJPRBKnAEcAZynVd5TqFUXXZVY2tU7StiPZrJDrPUw22VdfAbxF\n0t3AI8CsiLivxuM2TKxYsVBdXTB79nQ6Ojro7e2lp2d+pRu7SrUD2Yfds8AbI4lHm15wG4okbs3b\n/v+F33c8oAP3+j3a5q9efMasOWoN/mruDP8aGB8RT0uaClwJvKbG4zZUHjwDho9SvZUs9L9C1mXR\nM1gOQSTRq66unzHptUeTzt5rwwv54jMTH3qI8TBjLIxeDWtXwbx7w81nZvVQa/A/Aozv83g82VX/\nBhGxus/310r6sqTtIuLPm+5M0pw+D5dExJIa66u7fDDWTOCzwHGRxKKCS2pfnZ0z+NTsnTZ6btas\nCZudeuq/7AfbLMjWGgBgGuw2ScLhb2UnaTIwuZZ91Br8dwC7S9oFeBT4ANkcNBtI2h7oiYiQtDdZ\nF9IXhT5ARMypsZ6GyufYuQh4PbBvJLGi4JLaWz89qDo2154LYKu+zy2ACd0wnUH+EjMb6fIL4iXr\nH0tKhrqPmoI/ItZJOgW4jmyJv4sj4n5JH89fvwg4EjhZ0jrgabLpfdtOPiL1CrIVrt4aSTxVcEnt\nr58eVJs9+/SWlZ4fA4N2qTWzwXkAVxXyaReuIZuL/nS359dHxVHS5523fNKNN/71nqee2mvT7bth\n0cKIqgbRmZWFR+42gFLtS3ZD+vRI4pKi6xlp8nmRNupBNfGhh9gPzu/bxn8CLL8VZriN32xjXmy9\nzpSqm2wU7nGRxLUFlzMi9deDapJEN0zfYjTbP/pK9vzdBH75pwPwjXSzOvAVPzBJ6t606+DSOYwD\nvggcHkncWkRdlsknfLuCrBfZcY1ed9isnbipZxgmSd2bNisctyV/+OFhsPp1HBhJ3N/smuzFlGpL\nsonv9gAO82pfZhkH/zBMlRZdCy+ajfOdW3DTT56JA5tdj/UvH0NxBtlc/4dGEncXXJJZ4QqZnbPd\njYWKfcm3fbaqUcnWRPlkb2eRLWizWKneWXRNZu2o9MG/Gir2JV8DA87GacXJZ/p8L/ANpTqu6HrM\n2k3pg38VzDt2NH/q+9wJsHwlVJyNs5EmSd1TpUVHSUumSosmSd3NrqFdRBI/Jxu2PkepPpc3A5lZ\nFUrfxq9UM8fex+x9f8hvt30OrYHelTC/2f3FK91kngbLboGZ7rvev7zHz7XAzWTTYntwnZWKb+4O\n9ZipPk422dqBkcTvm3nsTfV3k9mjVQenVNsAV5NNEHhcmdY2tnJRqm2BMZHEhskwfXN3CJTqKLLF\n0N9RdOhD/zeZPT/N4PKFbw4BxgBXKdVWg/yIWbs6BvhCrTspZfAr1cFkbfhTI4nlRdcDvslcq0ii\nF3gf2SR61ynVSwsuyawR3kU2b1hNShf8SrU38E3gyEjiN0XXs94qmDcNNlrysaibzO0qb+I5HrgH\nuN6LuttIolQdwNvIZkOubV9lauNXqtcANwInRhJXN/JYwzFJ6t4Zpo+BjqJuMo8EeQ+f84CDgYMj\niZ6CSzKrmVJNBT4bSbxto+d9c3eA/afqBH4JfCGSuLhRx7HWkIf/HOAo4O2e4sHanVJdADwcSfzb\nRs/75m5l+c2+HwGXO/TLIR/lmwD/DfxMqXYouiaz4covZOrSvg8lCH6lGkX25n8Q+HzB5ViTRRJn\nA18HlijVjkXXYzZMe5KtcnhvPXY24oMfOBd4GfCxSFqkXcuaKpL4AnAxDn9rX+8CrqlXho3o4Feq\njwCHAe+LJJ4tuh4rTiRxLrAAuCEf7WvWTurWzAMjOPiV6gCyq/1DI4k/F12PFS+S+CJwGfBTpdq+\n6HrMqpGP1n0jcEO99jkig1+pdgG+BxzrhVSsr7zN/7tk/fzHFV2PWRUOBn5ez5XnRlzwK9XWwFXA\nuZGE12i1SlKyuX0WK9XLii7GbBDvosK61LUYUcGfd3laANwFnF9wOdai8htknyMbzLdQqcYWXJJZ\nRUq1GfBusu7odTOigh84lWxN1pPcg8cGkv9+nErWPe5H+XB4s1ZzAPD7SGJlPXc6YoJfqQ4iW5Lv\nvfmEXWYDysP/JOBR4AdKtUXBJZlt6gjgh/Xe6YgIfqXaCfgW8E+tMMWytY9I4m9kE7s9Q7aU46hi\nKzLL5E3XhwNX1nvfbR/8SrU58B1gXiRxfdH1WPvJZ/U8GngFcKGXcbQW8QbgWWBpvXfc9sEP/Bvw\nBFmffbNhiSTWkg32ewN1WOjCrA6OAK5sxP3Ktg5+pTqCbPGNY73WqtUqklgNTAXeo1SfLroeK73D\naUD7PrRx8CvVbsBFwFGRxJ+KrsdGhvx36RDgE/mUH2ZNl+dbJ3BrI/bflsGf9764HDg7kri96Hps\nZMkXsj4EOEep3lN0PVZKhwNXNaoloy2DHzgbeAyYV3QhNjJFEr8FDgUuVqq3Fl2Plc4RNKA3z3pt\nF/xKNYWsB8ZHPUjLGimS+BXwIeD7SjWp6HqsHJTqlcBE6jgp26baKvjz6XQvJeuv/8ei67GRL5JY\nTDbCd6FSjS+6HiuFI4GrI4lnGnWAtgl+pXoJ2UpKX40kbiy6HiuPSOJbwJeA65Rqu6LrsRHvA2Rj\nkxqmbYIfmAFsDZxVdCFWPpHEv5PNkOh5faxh8lkIXgcsbuRxag5+SVMkPSDpQUmz+9lmXv763ZL2\nGvIxUv0d2WyKH44k1tVas9kwfRpYCXzTUztYg7yfrDdPQ1cMrCn4JY0CLgCmkH1KHSNpz0226QYm\nRMTuwInAhUM6RqrRZPPwnBZJ/K6Wes1qkXet+wiwLfAlT+1gDXAUDW7mgdqv+PcGlkXEQxHxHFnf\n+sM22eZQsrZ5IuI2YFtpSMvefQH47fp9mBUpv+F2BHAgcFrB5dgIolSvBibQwN4869Ua/DsCq/o8\nfjh/brBtdqpm50r1drI/fT7urpvWKiKJJ8mmdvikUn2o6HpsxDgK+GE+aWBD1Rr81Ybxpn8SD/pz\nSrUNWdfNaZ6SwVpNJPEI2ZJ4/5GvBWFWq4b35llvsxp//hGgb9/m8WRX9ANts1P+3ItImrPhwXt5\nM3/Pokji2hprNGuISOJepToa+I5SHRRJ1H36XCsHpZpAlpODdlWXNBmYXNPxooYWFEmbkbW/v4Ns\nFaPbgWMi4v4+23QDp0REt6R9gS9FxL4V9hURIYB8fpTzgdfnMyaatSyl+jDwr8B+kcSjRddj7Uep\nzgBeFUl8csg/2yc7q1VTU09ErANOAa4D7gO+ExH3S/q4pI/n2ywEfidpGdlsmp8YaJ9K9fJ8u484\n9K0dRBLfAL4KXOOF222o8t5hxwKXNe2YtVzx15OkmALX3foetnziH7grkji16JrMqpW/eb8K7AAc\n5vEmVi2l2pes1+Jrh9OJpelX/PV2LRwy5Sb23+OL3FR0LWZDkb9hPwGMAi5wH38bgg8DlzWz52JL\nBT/At59k812f5sSi6zAbqrwb3lHAfmSjfM0GpFRbkvXm+e9mHrflgh9gDHguFGtLkcRfybp5nqJU\nHyi6Hmt53cA9kcTvm3nQlgz+NdBbdA1mw5Wv4PVusiaf/Yuux1paU2/qrtdywX8CLF8J84uuw6wW\nkcTdZG23VyjV7kXXY61HqcYBBwFXNPvYLRX83bDoVphxb9YF1KxQk6TuqdKio6QlU6VFk7IxKVWL\nJBYBZ5It4jKuMVVaG/sAcE3ePNhULdWdc6hdkswaZZLUvR+cvyCbNAuAabDsFpg51AsTpToX2B/4\nx0hibb1rtfakVL8CzogkrqtpP+3endOsVYyHGX1DH2ABTNgZpg9jd58lG9l+ab6SnJWcUr0ReAVw\nfRHH9y+hWQVjYXSl54fT4yyfx/844NVkUzuYTQP+K5L4WxEHd/CbVbAaKjbJDLfHWSTRS7ZWxVFK\ndUIttVl7U6oxZO37lxZVg4PfrIJVMG8aLOv7XK09ziKJP5D18T9bqQ6utUZrW0cDN+VTexfCN3fN\n+jFJ6t4Zpo+Bjge23nrr+3ff/SXrttlmNb29a+npmRcrVgyr95lSvQ34PvCOSOKeauoYDzPGwujV\nsHYVzHPPt/alVLcDcyKpz//hcLLTwW82CHV1dTNx4vnMmvXCzd65c5exdOnMGsL/g2TLig44lXM9\nexdZ8ZRqL+AqoKte7fvu1WPWCJ2dMzYKfYBZsybQ2TmcHj4ARBLfIpt+/Oq8zbeiOvcusuIVelN3\nPQe/2WA6Oir28KGjo9Y5pb4A3AlcrlQVV8OrZ+8iK1b+AX80cEnRtTj4zQbT21t50FVvb01zSuXT\n8J4MbAHMqzSVc717F1mhjgWW5HM5FcrBbzaYnp55zJ27UQ8fzjtvOT09Nc8plU/lfCRwAPCpTV9v\nRO8ia7584N5M4EtF1wK+uWtWFXV1ddPZOZ2Ojg56e3vp6Zk/3Bu7FfefajzwS+D/RBLf6/ta395F\na6B3Jcz3jd3myv//Z9DRMXo4vbqUaipwDvDGei+44l49Zm1Mqd4A/AQ4IpL4RdH1WKYevbqUahFw\neSTxtbrX5149Zu0rkriLF6Zyfk3R9Viuxl5dSrUn8Abg8gZUNywOfrMWks/U+DngWqXqLLoeox69\numYAF7XSzKwOfrMWE0lcDHyTrI//1kXXU3o19OpSqu3IunBeWOeqauLgN2tNCXA/A/TxtyaprVfX\nicDVkcT/NqS2YfLNXbMWpVSbAz8GHgJOqndvEKvecHp1KVUHsIJsAZ57G1abe/WYjSxKNRa4Cbgi\nkvBc/m1EqU4hC/3DG3oc9+oxG1kiidVAN/Axpfpo0fVYdZRqC+A0smk5Wo6D36zFRRKPAVOAc5Tq\nXUXXY1X5IPBgJHFb0YVU4uA3awORxG+Bw4GvKdU+Rddj/VOqUcBnyEbqtiQHv1mbiCRuBY4HrlKq\nPQoux/p3BPAE8LOiC+mPg9+sjUQS15BdTV6nVDsWXY9tLJ+M7QzgnFbuheXgN2sz+XwvXyYL/+0K\nLsc2dhTwLHB10YUMxMFv1p7OAxYBP/bo3taQj7s4Czi9la/2wcFv1pbyYPk08D/A9/Lug1as44GV\nkcT1RRcyGAe/WZuKJJ4nW8P1OeCyvDeJFSAfpXsmcHrRtVTDwW/WxvIVvD4AdAIXVlq+0ZriE8Ad\nrdpvf1MOfrM2l0/3exjweuCLDv/mUqptyJrdzii6lmoNO/glbSdpsaT/kfQTSdv2s91Dkn4j6U5J\ntw+/VDPrTz61w1SyEb6fL7icsjmTbAbOpUUXUq1arvg/AyyOiNcAP80fVxLA5IjYKyL2ruF4ZjaA\nSOLPwMHAh5TqtKLrKQOlei1wLG3Str9eLcF/KPD1/Puvkw0n74//9DRrgnze93cAJyvVJ4uuZyTL\nm9S+RDZYq6foeoailuDfPiIez79/HNi+n+0CuF7SHZKm1XA8M6tCJPEwWfjPVqoTiq5nBHs38Grg\ngqILGaoBV/aRtBh4ZYWXPtf3QUSEpP4GLOwfEY9JegWwWNIDEXFzP8eb0+fhkohYMlB9ZlZZJLFC\nqd4B3KBUz0cSlxRd00iiVFsC/wF8Iu9Z1bxjS5OByTXtY7gLsUh6gKzt/n8lvQr4WUS8dpCfSYA1\nEfF/K7zmhVjM6kypXgPcAHw+kri06HpGCqU6A3hzJHFY4bU0eSGWHwHH5d8fB1xZoaCtJI3Nv98a\neCdwTw3HNLMhiCT+h6zZ5yylOr7gckYEpXod8M/A9KJrGa5arvi3A74L7Ey2JuhREfGEpB2ABRHx\nLkm7Aj/If2Qz4JsRUXFFGl/xmzVOPo3z9cBZkcRXi66nXeWjo38OfCOS+HLR9YDX3DWzASjVBLKu\n13MjiflF19OOlOqfyebbPyifMqNwDn4zG5BS7UIW/l+JJM4rtpr2olS7Arfv8jU+89qHOHIsjF4N\na1fBvHsjFhZWl4PfzAaTL+DyU+B7wJmtPoVwK8ibeK4fdxPLD7+BAxfAhPWvTYNlt8DMosK/2Td3\nzawNRRKPAG8DuoEL8lWj2sIkqXuqtOgoaclUadEkqbtJh/4M8JI33cD4vqEPsAAm7NxmN3rb5j/c\nzOonH2l6EDAR+GY7zOc/SereD86/Fg75Lhx4LRyyH5zf6PBXqgPIgv2DY2HLStuMgY5G1lBvDn6z\nkook/ko2sdtWwNVK9dKCSxrQeJjR7KttpXo58C3gY5HEI6thbaXt1kBvo2poBAe/WYlFEr3A+4AV\nwE1KtUPBJfVrLIyu9HyjrrbzuXguAb6fL3LPKpg3DZb13e4EWL4S2qqX1IBTNpjZyBdJrFOqk4HZ\nwC1K1d2KUwwXcLV9OvAq4P3rn7g3YuEkiW6YPgY61kDvSphfZK+e4XCvHjPbQKk+SDbj5HGRxLVF\n19PX+jb+vs09J8DyW2FGvYNXqY4E/h3YJ5J4rJ77rjd35zSzminV/mRdPecC/9FK3T0nSd07N/hq\nW6neBFwLvDOSuLOe+24EB7+Z1YVSvRq4CrgTOCmSeKbgkpoiH+NwGzA9kvhh0fVUw/34zawuIonf\nAwcAY4Gf5yN+RzSlegXwE2Beu4T+cDn4zayiSGIN2Y3NbwO3KtXUgktqmLzb5vXADyKJLxZdT6O5\nqcfMBqVUbyX7ALgMSJq9+EgjKdW2ZKF/AzC7le5pVMNNPWbWEJHEzcA/AK8HfpEv8NL2lGoccB3w\nC9ow9IfLwW9mVYkkHidbZ/brZOF/Yj7IqS3l01T/kmzCun8uS+iDm3rMbBiUak/gG8CTwImRxPKC\nSxoSpdqPbJGopN0XpnF3TjNrGqXaDJgJfBb4Ilmf/5Zu+8//QjkeOBc4PpL2GnFbiYPfzJouX6Dk\nK8BOwP+JJBYVXFJFSrUNWZ2TgKNbcVqK4XDwm1kh8ivp95CN9l0GnNZKwZpPrXwZsAj4VD453Yjg\n4DezQuXz+n+SbOGSJcC/RhL3FFjPq8iaoQ6ijUbjDoWD38yaRl1d3XR2zqCjYzS9vWvp6ZkXK1Ys\nBFCqMcDJwKfIukrOB25sVs8ZpRpL9gE0C1gAnJ0PSBtxHPxm1hTq6upm4sTzmTXrhYVR5s5dxtKl\nM9eHP4BSbQV8lOxDQGRt7N+KJP7YkLpS7QTMAD4GLAY+H0k82IhjtQoHv5k1hfbZZxHnnnvIi16Y\nPXtR3Hbbi6Z2yO8BvA04iWyt3/8HXAH8OJ8XaPi1ZIOwDiNbUGZfsnEG50cSD9Wy33YxnOz0Qixm\nNnQdHRVXw6Kjo+JqWHkTz43AjUrVARwCvBdIlOoZsuagW4Hfkq0G9lAksdHCK0o1Cng52eIobyAb\nSfwmsnWDF5ONKzg6X1LSBuDgN7Oh6+2tuBoWvb2D9pbJe9RcCVyZ/yWwG7A/sA/ZGsC7AuOV6m/A\nM8CzwCjgZcATwOPAb4A7yAZh3R5JPF3jv6hU3NRjZkNWsY3/vPOWc999M/q28Q97/9nV/Whgy/zr\neeBPkcS6Wvc90riN38waYpLUPR5mjIXRq2HtKpi3dJddoLNzOh0dHfT29tLTM78eoW9D4+A3s7qr\ntNbtNFh2C8xst0XGRyJPy2xmdTceZvQNfYAFMGFnmF5UTVYbB7+ZDWhs1tb+ImOgYg8ea30OfjMb\n0Gqo2INnDYyY+W7KxsFvZgNaBfOmZROvbXACLF+ZTcNgbcg3d81sUJOk7p1h+hjoWAO9K2G+b+y2\nBvfqMTMrGffqMTOzQQ07+CW9X9JSSX+T9MYBtpsi6QFJD0qaPdzjmZlZfdRyxX8PcARwU38bSBoF\nXABMAV4HHCNpzxqOWQqSJhddQ6vwuXjBSDkX6urq1j77LNLkyUu0zz6L1NXVPeR9jJBzUZRhT9IW\nEQ8ASAM2Le0NLIvIpkeVdDnZ9Kn3D/e4JTGZbPUi87noazJtfi76mcd/N3V1McTpHibT5ueiSI1u\n498RWNXn8cP5c2ZWRp2dMzYKfYBZsybQ2elRwE004BW/pMXAKyu8dHpEXF3F/lujy5CZtYYhzuNv\njVFzd05JPwM+FRG/rvDavsCciJiSP/4s8HxEnFthW39ImJkNQ1ErcPV30DuA3SXtAjwKfAA4ptKG\n7sNvZtYctXTnPELSKrI1Lq+RdG3+/A6SrgGIiHXAKcB1wH3AdyLCN3bNzArUMiN3zcysOZo6crea\nwVyS5uWv3y1pr2bW10yDnQtJH8rPwW8k/ULS3xdRZzNUO8hP0pslrZP03mbW10xVvkcmS7pT0r2S\nljS5xKYcWg/QAAADA0lEQVSp4j0yTtIiSXfl5+L4AspsOEmXSHpc0j0DbDO03IyIpnyRLZa8DNgF\n2By4C9hzk226gYX59/sAtzarvmZ+VXku9gO2yb+fUuZz0We7G4AfA+8ruu4Cfy+2BZYCO+WPxxVd\nd4HnYg7whfXnAfgTsFnRtTfgXLwV2Au4p5/Xh5ybzbzi3zCYKyKeA9YP5urrUODrABFxG7CtpO2b\nWGOzDHouIuKWiHgyf3gbsFOTa2yWan4vIFvt6fvAH5pZXJNVcy4+CFwREQ8DRMQfm1xjs1RzLh4D\nXpp//1LgTxEjbzH2iLgZ+MsAmww5N5sZ/NUM5qq0zUgMvKEObPsYMFKnwB30XEjakexNf2H+1Ei9\nMVXN78XuwHaSfibpDkkfblp1zVXNuVgATJT0KHA3MLNJtbWaIedmvbpzVqPaN+um3TpH4pu86n+T\npIOAjwL7N66cQlVzLr4EfCYiQtkcISO1628152Jz4I3AO4CtgFsk3RoRDza0suar5lycDtwVEZMl\n7QYslvT6iFjd4Npa0ZBys5nB/wgwvs/j8WSfTANts1P+3EhTzbkgv6G7AJgSEQP9qdfOqjkX/wBc\nns8LNQ6YKum5iPhRc0psmmrOxSrgjxHRC/RKugl4PTDSgr+ac/EW4GyAiFguaQWwB9n4oTIZcm42\ns6lnw2AuSVuQDeba9I37I+BY2DDq94mIeLyJNTbLoOdC0s7AD4B/iohlFfYxUgx6LiJi14joiogu\nsnb+k0dg6EN175GrgAMkjZK0FdnNvPuaXGczVHMuHgD+ESBv094D+F1Tq2wNQ87Npl3xR8Q6SesH\nc40CLo6I+yV9PH/9oohYKKlb0jLgKeAjzaqvmao5F8CZwMuAC/Mr3eciYu+iam6UKs9FKVT5HnlA\n0iLgN8DzwIKIGHHBX+XvxTnApZLuJruI/XRE/LmwohtE0reBA4Fx+aDZhKzJb9i56QFcZmYl46UX\nzcxKxsFvZlYyDn4zs5Jx8JuZlYyD38ysZBz8ZmYl4+A3MysZB7+ZWcn8f4g4ubDBFJQXAAAAAElF\nTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x111980080>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Now we will define a testing set to see\n",
"# how well this model works, generated similarly\n",
"# to our training data\n",
"Ntest = 8\n",
"xtest = random_sample(Ntest)\n",
"ytest = sin(2*pi*xtest) + randn(Ntest) * std_deviation\n",
"\n",
"plot(xground, polyval(w, xground), 'g')\n",
"plot(x, t, 'ro')\n",
"plot(xtest, ytest, 'co')\n",
"\n",
"print(\"Train Error:\", err(w, x, t))\n",
"print(\"Test Error:\", err(w, xtest, ytest))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# What if we use a higher order polynomial?\n",
"w_higher = polyfit(x, t, 9)\n",
"\n",
"\n",
"plot(x, t, 'ro')\n",
"plot(xtest, ytest, 'co')\n",
"plot(xground, polyval(w_higher, xground), 'g', scaley=False)\n",
"\n",
"print(\"Train Error:\", err(w_higher, x, t))\n",
"print(\"Test Error:\", err(w_higher, xtest, ytest))\n",
"\n",
"# It passes directly through all our training points, but generalises\n",
"# poorly for predicting the test points - overfitting!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We use the root-mean-square error defined by\n",
"$$ E_RMS = \\sqrt{\\frac{2E(w^*)}{N}} $$\n",
"\n",
"This function is defined on both training data and test data set. \n",
"- When evaluating on training data set, it measures how the model fit with the training data.\n",
"- When evaluating on test data set, it measures how well we are doing in predicting the value of $t$ on the new data observations of $x$.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Root Mean Square error - in same units as data\n",
"def rms(w, x, t):\n",
" return sqrt(2 * err(w, x, t)/len(x))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Let's compare the train and test error for different\n",
"# orders of polynomial\n",
"\n",
"test_err = []\n",
"train_err = []\n",
"\n",
"maxorder = 10\n",
"\n",
"for m in range(0,maxorder):\n",
" weights = polyfit(x, t, m)\n",
" train_err.append(rms(weights, x, t))\n",
" test_err.append(rms(weights, xtest, ytest))\n",
"\n",
"# Let's see the rms error on training data set (blue)\n",
"plot(range(0,maxorder), train_err, 'bo-')\n",
"\n",
"# Let's see the rms error on test data set (red)\n",
"plot(range(0,maxorder), test_err, 'ro-')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### References \n",
"- http://nbviewer.ipython.org/github/jamt9000/prml/blob/master/1.1-polycurve.ipynb"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.2"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment