LinearRegression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression\n",
    "\n",
    "Linear regression is a mathematical approach to modeling the unknown parameters of linear predictor functions from data estimates. Let's suppose we have a discrete [uniformly distributed](https://en.wikipedia.org/wiki/Uniform_distribution_(discrete) set of data (we'll apply an ongoing bias and a spreading factor to better model an event sampling)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFGdJREFUeJzt3X2MXFd9xvHncRxHG8DuGtZ21YAXWhWnlJSEJq0EUiaJ\n3aagJqhICW6LCF0qoZQGFYnGUFVZ/qCN+aMIiUaixaAEkRBIlReqVnEse1SBVLzNm9MQGyS029DG\nu0sCpmlWxSW//jF3zaw9uztvd+69534/0sqz13N3fj67fubsueec64gQAKD6NhRdAABgOAh0AEgE\ngQ4AiSDQASARBDoAJIJAB4BEdBXotrfY/prtZ2w/bfs3bI/bPmj7hO2HbW/Ju1gAwOq67aF/RtI/\nRcTFkn5N0nFJ+yQdiog3Sjos6WP5lAgA6IbXW1hke7OkxyPiF886flzSlRExb3uHpGZE7MqvVADA\nWrrpob9e0g9sf9H2Y7b/zvaFkrZHxLwkRcRJSdvyLBQAsLZuAn2jpMsk/W1EXCbpf9Qabjm7a88e\nAgBQoI1dPOf7kp6NiH/LPv8HtQJ93vb2tiGXhU4n2yboAaAPEeFenr9uDz0bVnnW9i9nh66R9LSk\nhyTdlB17n6QH1/gapfq47bbbCq+hCjWVtS5qoqY61NWPbnroknSLpC/bPl/S9yS9X9J5kr5q+48k\nzUm6oa8KAABD0VWgR8STki7v8Fe7h1sOAKBftVwp2mg0ii7hHGWsSSpnXdTUHWrqXlnr6tW689AH\nfgE78n4NAEiNbcWwL4oCAKqBQAeARBDoAJAIAh0AEkGgA0AiCHQASASBDgCJINABIBEEOgAkgkAH\ngEQQ6ACQCAIdABJBoANAIgh0AEgEgQ4AiSDQASARBDoAJIJAB4BEEOgAkAgCHQASQaADQCIIdABI\nBIEOAIkg0AEgEQQ6gKQtLi5qZmZGi4uLRZeSOwIdQLLuuede7dy5S3v2fFA7d+7SPffcW3RJuXJE\nrP8ke1bSKUkvSzodEVfYHpd0r6SdkmYl3RARpzqcG928BgAM0+Lionbu3KWlpSOSLpF0TGNjV2lu\n7rgmJiaKLm9dthUR7uWcbnvoL0tqRMSlEXFFdmyfpEMR8UZJhyV9rJcXBoA8zc7OatOmSbXCXJIu\n0fnn79Ts7GxxReWs20B3h+deL+nO7PGdkt41rKIAYFCTk5P6yU9mJR3LjhzT6dNzmpycLK6onHUb\n6CHpEdsztj+QHdseEfOSFBEnJW3Lo0AA6MfExIQOHLhDY2NXafPmyzQ2dpUOHLijEsMt/ep2DP3n\nI+I52xOSDkq6RdKDEbG17TnPR8SrO5zLGDqAwiwuLmp2dlaTk5OVCvN+xtA3dvOkiHgu+3PR9gOS\nrpA0b3t7RMzb3iFpYbXzp6enzzxuNBpqNBq91AgAfZuYmKhEkDebTTWbzYG+xro9dNsXStoQES/a\nfoVaPfRPSLpG0gsRsd/2rZLGI2Jfh/PpoQNAj/rpoXcT6K+XdL9a4+gbJX05Im63vVXSVyW9VtKc\nWtMWf9ThfAIdAHqUS6APikAHIFV3LLsoec5DB4C+1W3FZlHooQPIVdVXbBaFHjqA0qnjis2iEOgA\nclXHFZtFIdAB5KqOKzaLwhg6gJFglktvmLYIAIngoigA1BiBDgCJINABIBEEOgAkgkAHgEQQ6ACQ\nCAIdQOktLi5qZmZGi4uLRZdSagQ6gFJjp8busbAIQGnVeadGFhYBSAo7NfaGQAdQWuzU2BsCHUBp\nsVNjbxhDB1B6ddypkd0WASARXBQFgBoj0AEgEQQ6ACSCQAeARBDoAEqB/VoGR6ADKBz7tQwH0xYB\nFKrO+7WshWmLACqH/VqGp+tAt73B9mO2H8o+H7d90PYJ2w/b3pJfmQBSxX4tw9NLD/3Dkr7d9vk+\nSYci4o2SDkv62DALA1AP7NcyPF2Nodu+SNIXJX1S0kci4jrbxyVdGRHztndIakbErg7nMoYOYF11\n3K9lLf2MoW/s8nmflvRRSe3DKtsjYl6SIuKk7W29vDAAtJuYmCDIB7RuoNt+p6T5iHjCdmONp67a\nDZ+enj7zuNFoqNFY68sAQP00m001m82Bvsa6Qy62/0rSH0r6P0ljkl4l6X5Jvy6p0TbkciQiLu5w\nPkMuANCjXKYtRsTHI+J1EfEGSe+RdDgi3ivp65Juyp72PkkP9lgvAGCIBpmHfrukPbZPSLom+xwA\nUBBWigJACbFSFABqjEAHgEQQ6ACQCAIdABJBoANAIgh0AEgEgQ6g8rh9XQuBDqDSuH3dz7CwCEBl\npXz7OhYWAagVbl+3EoEOoLK4fd1KBDqAyuL2dSsxhg6g8lK8fV0/Y+gEOgCUEBdFAaDGCHQASASB\nDgCJINABrMAy+uoi0AGcwTL6amOWC1CgMk23S3kZfRUxywWokLL1hllGX3300IEClLE3XMaa6owe\nOlARefeG+7mwyTL66qOHDhQgz97wPffcq6mpm7VpU2vjqgMH7tDevTf2VFtZxvXrjKX/QIUsB+/5\n5+/U6dNzPQdvJwybpKOfQN+YVzEA1rZ3743avfvqofaGl4dylpbOHcoh0NNHoAMFmpiYGGrQrtwf\nvNVDr/P+4HXDRVGgojpd+OTCZr2tO4Zu+wJJ/yJpk1o9+vsi4hO2xyXdK2mnpFlJN0TEqQ7nM4aO\nkajTxbz1LnzWqS1SldtFUdsXRsRLts+T9E1Jt0h6t6TnI+JTtm+VNB4R+zqcS6Ajd4PO7BjEqMOz\nzBc+eSMZntzmoUfES9nDC9TqpYek6yXdmR2/U9K7enlhYFgWFxc1NXWzlpaO6NSpR7W0dERTUzeP\nZHOpIlZ7lnVFZ9lWvtZSRKz7oVbwPy7px5L+Ojv2w7Oe88Iq5waQp6NHj8aWLZeFFGc+Nm++NI4e\nPZrr6y4sLMTY2NaQnsxe98kYG9saCwsLSb5u1Wqquiw7u8ro5Y+uZrlExMuSLrW9WdL9tt+kVi99\nxdNWO396evrM40ajoUaj0dWbDdCNomZ2FDVFcPnC59TUVSvmsBc5xMF0ycE1m001m82BvkbPC4ts\n/6WklyR9QFIjIuZt75B0JCIu7vD86PU1gF7lsUhnPUWPZZdpvLrotkhRLhdFbb9G0umIOGV7TNLD\nkm6XdKVawyz7uSiKMigi4Ip4IynSWm1ct7bIW16B/ma1LnpuyD7ujYhP2t4q6auSXitpTq1piz/q\ncD6BjqSVqafcrX5q7mYmURXboqzYywXAuvqZ4smQyuixfS6ANfU7xbOsUyWxEoEO1Ei/wbxyJpHE\nHjHlRKADNdJvMLNHTDUwhg7UzCCzUbjoOTpcFEUlEArF43tQfgQ6Sq/ITbSAKiHQUWpMfQO6x7RF\nlFodp751ugkFkBcCHSNTt6lvg24ny5sBekWgY2TqNPVt0D3a2Vsc/WAMHSNXhxkWMzMz2rPngzp1\n6tEzxzZvvkyHDn1Ol19++Zrncq0BUn9j6F3thw4M07DvdF9Gg+zRzt7i6BdDLkAOBhleqtu1BgwP\nQy7oSx2GTYah33Zib3EwDx0jweKg0eBNs94I9Iqq0n9cLtgBo8HCogqq2vS0Oi4OAqqCHnqBqtjb\nrWLNQBXRQ6+YKvZ267Q4CKgaeugFqnJvt0rj/kAVsbCoYpZ7u1NTV62YnlaFgKzD4iCgauihlwC9\nXQBnY9piDfFmAKSJi6I1U7UpjwDyRQ+9Ajr1wqt8QbWK+E0Io0YPPUGr9cKrOOWxqvhNCFVBD73E\n1uqFS6KHPgL8JoSi0ENPzFq9cBb49K6fW7rxmxCqZN1At32R7cO2n7b9lO1bsuPjtg/aPmH7Ydtb\n8i+3XtbbF3vv3hs1N3dchw59TnNzx9nxcA39DpuwNzkqJSLW/JC0Q9JbssevlHRC0i5J+yX9eXb8\nVkm3r3J+oH933/2VGBvbGps3XxpjY1vj7ru/UnRJlbOwsBBjY1tDejKkCOnJGBvbGgsLC12dz/cA\nRciyc92Mbv/oeQzd9gOSPpt9XBkR87Z3SGpGxK4Oz49eXwMrMcNiMIPc33MZ3wOMWu4Li2xPSmpK\n+lVJz0bEeNvfvRARWzucQ6CjUFzYRBXlupeL7VdKuk/ShyPiRdtnp/SqqT09PX3mcaPRUKPR6KVG\nYCDd7JlDDxxFazabajabA32NrnrotjdK+kdJ/xwRn8mOPSOp0TbkciQiLu5wbi166ARC+a32PeKW\neiij3IZcbN8l6QcR8ZG2Y/slvRAR+23fKmk8IvZ1ODf5QE81EOrwJsVwDMoql3nott8m6Q8kXW37\ncduP2b5WrVkue2yfkHSNpNv7KbrqFhcXNTV1s5aWjujUqUe1tHREU1M39zTXuYzqsjqSeeZIybpj\n6BHxTUnnrfLXu4dbTvUsB8LSUufFP1XU/ibV+ncd09TUVdq9++rK/ptWs3KeeevfyjxzVBUrRc/S\n62rCFBee1KnXyopbpIRAb9PPMEOKgZDim9RaWHGLVLA5V2bQi2OpXUBcvtDbPs2vDEGXWjsDq+GO\nRQMYxmrC1JQtPFOdTQR0QqAPgOlr5cb3B3XD9rkDSHEsPCV1ulAL9Ise+lnKNsyAFnroqJtc93Kp\ni4mJCQKihLrZjwWoO3roiUvtN47U/j3AargoihWYFQJUF4GOMxhzBqqNWS44o8hZIf3cjBnA4Aj0\nRBW1fH/QXRp5MwD6R6Anqoh59YNuJVyXLXuBvDCGnrhRzgoZZPsExvyBlZiHjnOMcl79IHuLp7iv\nPDBqDLlgaAYZ5qnblr1AHhhywdD1O8xT1i17gSIwDx2Vx0pQoIVAB4BEsLCoxJhfDSBvBPoIML8a\nwCgw5JIz5lcD6AdDLl0Y9dAHd9oBMCq1CvQihj6YXw1gVGoz5FLk0AfzqwH0iqX/ayhyafnevTdq\n9+6rmV8NIFe1CfRB9hkZBu5VCiBv646h2z5ge972sbZj47YP2j5h+2HbW/Itc3BFbCcLAKO07hi6\n7bdLelHSXRFxSXZsv6TnI+JTtm+VNB4R+1Y5vxRj6MtYWg6gCnJb+m97p6SvtwX6cUlXRsS87R2S\nmhGxa5VzSxXoAFAFo5yHvi0i5iUpIk5K2tbn1wEADMmw5qHTBQeAgvU7y2Xe9va2IZeFtZ48PT19\n5nGj0VCj0ejzZTFMXE8AyqPZbKrZbA70NbodQ59Uawz9zdnn+yW9EBH7q3ZRFC3Li502bWpN52Sx\nE1AuuVwUtX23pIakV0ual3SbpAckfU3SayXNSbohIn60yvkEesmwYRhQfrmsFI2I31/lr3b38kIo\nD27IDKSpVptzoYUNw4A0Eeg1xKpZIE212W0R52KWC1Be3CQaABLBHYsAoMYIdABIBIEOAIkg0Hsw\n6htMA0AvCPQuFXGDaQDoBbNcusBSeQCjxiyXnCwvlW+FudS+VB4AyoJA7wJL5QFUAYHeBZbKA6gC\nxtB7wFJ5AKPC0n8ASAQXRQGgxgh0AEgEgQ4AiSDQASARBDoAJIJAB4BEEOgAkAgCHQASQaADQCII\ndABIBIEOAIkg0AEgEQQ6ACRioEC3fa3t47a/Y/vWYRUFAOhd34Fue4Okz0r6bUlvkrTX9q5hFZan\nZrNZdAnnKGNNUjnroqbuUFP3ylpXrwbpoV8h6bsRMRcRpyV9RdL1wykrX2X85pWxJqmcdVFTd6ip\ne2Wtq1eDBPovSHq27fPvZ8cAAAXgoigAJKLvW9DZ/k1J0xFxbfb5PkkREfvPeh73nwOAPozsnqK2\nz5N0QtI1kp6TdFTS3oh4pq8vCAAYyMZ+T4yIn9r+kKSDag3dHCDMAaA4fffQAQDlMpKLorZvs/19\n249lH9eO4nVXqaV0i6Fsz9p+0vbjto8WVMMB2/O2j7UdG7d90PYJ2w/b3lKSugr7ebJ9ke3Dtp+2\n/ZTtW7LjhbZVh7r+NDteZFtdYPtb2c/1U7Zvy44X1lZr1FR4RtnekL32Q9nnPbfTSHroWaP9d0T8\nTe4vtnYdGyR9R61x//+SNCPpPRFxvOC6vifprRHxwwJreLukFyXdFRGXZMf2S3o+Ij6VvfmNR8S+\nEtRV2M+T7R2SdkTEE7ZfKelRtdZfvF8FttUadd2oAv/v2b4wIl7Krrl9U9Itkt6tYtuqU02/o4Iz\nyvafSXqrpM0RcV0///9GOW2xp6u1OSnrYiir4CmkEfENSWe/oVwv6c7s8Z2S3jXSorRqXVJBP08R\ncTIinsgevyjpGUkXqeC2WqWu5XUhhf3fi4iXsocXqHXNLlR8W3WqSSqwnWxfJOkdkj7fdrjndhpl\niHzI9hO2P1/Er+6Zsi6GCkmP2J6x/cdFF9NmW0TMS63AkLSt4HraFf7zZHtS0lsk/auk7WVpq7a6\nvpUdKqytsmGExyWdlPRIRMyo4LZapSap2J+pT0v6qH725iL10U5DC3Tbj9g+1vbxVPbn70q6Q9Ib\nIuItajVioUMvJfS2iLhMrXfoP8mGGcqoLFfQC/95yoY17pP04axHfHbbFNJWHeoqtK0i4uWIuFSt\n32KusP0mFdxWHWr6FRXYTrbfKWk++w1rrd8S1m2nvqctnvNKEXu6fOrfS/r6sF63R/8p6XVtn1+U\nHStURDyX/blo+361hoa+UWxVkqR529sjYj4bo10ouiCp1U5tn47858n2RrVC80sR8WB2uPC26lRX\n0W21LCJ+bLsp6VqVoK3OrumssfNRt9PbJF1n+x2SxiS9yvaXJJ3stZ1GNctlR9unvyfp30fxuh3M\nSPol2zttb5L0HkkPFVSLpNYFmqxXJduvkPRbKq59rJU9hIck3ZQ9fp+kB88+YURW1FWCn6cvSPp2\nRHym7VgZ2uqcuopsK9uvWR66sD0maY9aY/uFtdUqNR0vsp0i4uMR8bqIeINamXQ4It6r1pvKTdnT\numuniMj9Q9Jdko5JekLSA2qNDY3ktTvUcq1aK1y/K2lfUXW01fP6rF0el/RUUTVJulutmT//K+k/\n1Jq1MS7pUNZeByX9XEnqKuznSa3e1E/bvmePZT9TW4tsqzXqKrKt3pzV8URWw19kxwtrqzVqKkVG\nSbpS0kP9thMLiwAgEey2CACJINABIBEEOgAkgkAHgEQQ6ACQCAIdABJBoANAIgh0AEjE/wP2MQ6S\nxynPXgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x404d550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# Generate N samples in range [0;1[ with a uniform distribution\n",
    "def generateRandomData(numberOfSamples):\n",
    "    x = np.zeros(shape=numberOfSamples)\n",
    "    y = np.zeros(shape=numberOfSamples)\n",
    "\n",
    "    for i in range(0, numberOfSamples):\n",
    "        x[i] = i\n",
    "        # Generates values along a line with a fixed degree of variance\n",
    "        y[i] = i + np.random.uniform(0, 1) * 20\n",
    "\n",
    "    return x, y\n",
    "\n",
    "def plotScatterData(x, y):\n",
    "    plt.scatter(x, y)\n",
    "    plt.show()\n",
    "\n",
    "x, y = generateRandomData(35)\n",
    "plotScatterData(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the case for a linear regression (linear polynomial) we're interested in determining the parameters $\\theta_1$ and $\\theta_2$ for the *hypothesis* $h_\\theta(x)$ defined as\n",
    "\n",
    "$$\n",
    "h_\\theta(x) = \\theta_0 + \\theta_1x\n",
    "$$\n",
    "\n",
    "If we define a generic $h_\\theta(x)$ by assigning, for instance, $\\theta_1 = 1$ and $\\theta_2 = 1$, we will have an error that can be estimated by using the **sum of the squared error** method\n",
    "\n",
    "$$\n",
    "J(\\theta_1, \\theta_2)=\\sum^{m}_{i=1}{(h_\\theta(x^{(i)}) - y^{(i)})^2}\n",
    "$$\n",
    "\n",
    "and therefore the optimization problem we need to solve is finding $min \\ J(\\theta_1, \\theta_2)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient Descent\n",
    "A straightforward approach to solving the optimization problem explained in the previous paragraph is the gradient descent method. The idea is to compute $\\nabla J(\\theta_1, \\theta_2)$ and use it as a negative gradient to find the decreasing direction of the multi-variable function $J$.\n",
    "\n",
    "Therefore (we're adding a $\\frac{1}{2m}$ factor to simplify computation)\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "& \\frac{\\partial}{\\partial \\theta_1}J(\\theta_1, \\theta_2) = \\frac{1}{m} \\sum^{m}_{i=1}{(h_\\theta(x^{(i)}) - y^{(i)})} \\\\\n",
    "& \\frac{\\partial}{\\partial \\theta_2}J(\\theta_1, \\theta_2) = \\frac{1}{m} \\sum^{m}_{i=1}{(h_\\theta(x^{(i)}) - y^{(i)}) x^{(i)}}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "If we choose a suitable $\\alpha$ factor (the *learning rate*) for the descent in the $-\\nabla$ direction (a compromise between convergence speed and actually converging to a function minimum), the algorithm becomes\n",
    "\n",
    "    while(!converged) {\n",
    "      theta1 = theta1 - alpha * nabla[0];\n",
    "      theta2 = theta2 - alpha * nabla[1];\n",
    "      // Verify convergence\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHepJREFUeJzt3XmUVeWV9/HvRoaUUbDAEkzUKo1G1Dgm4pvWxEIlTnGM\ngaB2mohGgwhJZxmIybvE1avTkqkXOMSJGNGAoAKaaEchcmNrWikZxCCgrULQV4oKCIoiQ9V+/3gu\nUpQ13Pmce+7vs1Ytbh3uuWdzqtj3ufs8zz7m7oiISPnrFnUAIiJSGEroIiIJoYQuIpIQSugiIgmh\nhC4ikhBK6CIiCZFRQjezPmb2kJktN7NlZnaSmVWb2VNmttLMnjSzPsUOVkREOpbpCH0S8IS7HwEc\nC6wAxgPz3P1w4Gngx8UJUUREMmFdLSwys97AYnf/XJvtK4BT3b3RzAYAKXcfWLxQRUSkM5mM0A8G\n/mFm95rZIjO7y8z2BPq7eyOAu68F9itmoCIi0rlMEnp34ATgNnc/AfiAUG5pO7RXDwERkQh1z+A5\nbwFr3P3F9PePEBJ6o5n1b1VyWdfezmamRC8ikgN3t2ye3+UIPV1WWWNmn09vOh1YBjwGjEhv+xfg\n0U5eI1ZfN954Y+QxlENMcY1LMSmmSogrF5mM0AHGAL83sx7AG8B3gD2AmWZ2BbAaGJpTBCIiUhAZ\nJXR3fwk4sZ2/OqOw4YiISK4qcqVofX191CF8QhxjgnjGpZgyo5gyF9e4stXlPPS8D2DmxT6GiEjS\nmBle6IuiIiJSHpTQRUQSQgldRCQhlNBFRBJCCV1EJCGU0EVEEkIJXUQkIZTQRUQSQgldRCQhlNBF\nRBJCCV1EJCGU0EVEEkIJXUQkIZTQRUQSQgldRCQhlNBFRBJCCV1EJCGU0EVEEkIJXUQkIZTQRUQS\nQgldRCQhlNBFRBJCCV1EJCGU0EVEEkIJXUQSrampiYaGBpqamqIOpeiU0EUksaZPn0Ft7UCGDLmG\n2tqBTJ8+I+qQisrcvesnma0CNgEtwHZ3H2Rm1cAMoBZYBQx1903t7OuZHENEpJCampqorR3Ili3z\ngWOApVRVDWb16hXU1NREHV6XzAx3t2z2yXSE3gLUu/vx7j4ovW08MM/dDweeBn6czYFFRIpp1apV\n9OxZR0jmAMfQo0ctq1atii6oIss0oVs7z70AuC/9+D7gwkIFJSKSr7q6OrZtWwUsTW9Zyvbtq6mr\nq4suqCLLNKE7MNfMGszsyvS2/u7eCODua4H9ihGgiEguampqmDLldqqqBtO79wlUVQ1mypTby6Lc\nkqtMa+j7u/s7ZlYDPAWMAR51976tnrPe3fu1s69q6CISmaamJlatWkVdXV1ZJfNcaujdM3mSu7+T\n/rPJzOYAg4BGM+vv7o1mNgBY19H+EyZM+PhxfX099fX12cQoIpKzmpqaskjkqVSKVCqV12t0OUI3\nsz2Bbu6+2cw+TRih3wScDmxw94lmNg6odvfx7eyvEbqISJZyGaFnktAPBmYT6ujdgd+7+81m1heY\nCRwIrCZMW9zYzv5K6CIiWSpKQs+XErqIQPnWsqNSzHnoIiI5q7QVm1HRCF1EiqrcV2xGRSN0EYmd\nSlyxGRUldBEpqkpcsRkVJXQRKapKXLEZFdXQRaQkNMslO5q2KCKSELooKiJSwZTQRUQSQgldRCQh\nlNBFRBJCCV1EJCGU0EVEEkIJXURir6mpiYaGBpqamqIOJdaU0EUk1tSpMXNaWCQisVXJnRq1sEhE\nEkWdGrOjhC4isaVOjdlRQheR2FKnxuyohi4isVeJnRrVbVFEJCF0UVREpIIpoYuIJIQSuohIQiih\ni4gkhBK6iMSC+rXkTwldRCKnfi2FoWmLIhKpSu7X0hlNWxSRsqN+LYWTcUI3s25mtsjMHkt/X21m\nT5nZSjN70sz6FC9MEUkq9WspnGxG6GOBV1p9Px6Y5+6HA08DPy5kYCJSGdSvpXAyqqGb2QHAvcC/\nA//q7ueb2QrgVHdvNLMBQMrdB7azr2roItKlSuzX0plcaujdM3zefwLXA63LKv3dvRHA3dea2X7Z\nHFhEpLWampqCJ/J16+Cuu6CqCn74w4K+dCx1mdDN7Fyg0d2XmFl9J0/tcBg+YcKEjx/X19dTX9/Z\ny4iI5GfJEpg0CebMgUsugbFjo46oa6lUilQqlddrdFlyMbOfAZcDO4AqYG9gNvAloL5VyWW+ux/R\nzv4quYhI0e3YAY8+CpMnwxtvwLXXwlVXQb9+UUeWm6K3zzWzU4EfpmvoPwfWu/tEMxsHVLv7+Hb2\nUUIXkaJ591245x647TY44IAwGr/wQujRI+rI8lPMGnp7bgZmmtkVwGpgaB6vJSKSlVdeCaPxGTPg\nvPPgkUfgi1+MOqpoaaWoiJSNlhZ44olQH//b3+Caa+Dqq2HAgKgjK7xSj9BFRErivffg3nvhllug\nujqUVb75TejVK+rI4kUJXURi67XXQhJ/4AEYMgSmToUvfxksq3Fr5VBCF5FYcYe5c0N9fMECuPJK\neOklOPDAqCOLPyV0EYmFDz6A++8Pibx791BWeeihsChIMqOELiKRWrUqTDm89174ylfC4/p6lVVy\nofa5IlJy7vCXv8DFF4ephi0t0NAAs2fD4MFK5rnSCF1ESuajj2DatFBW+egjGDMmXOjca6/8XleN\nvQKN0EWk6N5+G37yEzjoIHj4YZg4MSwMGjUq/2Su29ftooVFIlI0zz8fFgE9+SRcdhmMHg2HH164\n10/y7et0CzoRidy2baGsctJJcOmlMGgQvPlmmE9eyGQOun1dW6qhi0hBrFsHd94Jv/kNHHFEKLGc\ney7ssUfxjrn77evCCL2Sb1+nEbqI5GXxYhgxIoy+16wJ5ZU//xnOP7+4yRx0+7q2VEMXkazt2BFu\nHjFpUphHHnXv8STOcil6P/RcKKGLJMeGDbt6jx94YFjNedFFYWWnFJa6LYpIUSxbFuaOz5wZeo/P\nmqXe43GkhC4i7Wpu3tV7/JVXQu/xFSugf/+oI5OOKKGLyG42bdrVe7xv31BWGToUevaMOjLpihK6\niADw6qs7e4+3cOKJ73LLLd04++xq9VUpI5q2KFLB3OGpp8J88VNOgTVrXuGjj45hwYKvccklh/Lg\ng5W7jL4caZaLSISimm73wQehKdbkyaGUMnYsnHFGEwMHJnMZfTnS0n+RMhJFU6lVq+D666G2NozM\nf/MbWLIErrgCGhu1jL7caYQuEoFSNpXa2Xt88mR45pmwqvPaa+Hgg6OLSbqmeegiZWJnU6ktWz45\nGi5E8mxqamLlytUsXHg49967N1u3dt17fOcy+pEjB9OjRy3bt6+u6GX05UgjdJEIFHM0fNttj/H9\n7/8vzc0jMHuR66/vxc9+dirdMiywJnEZfTnS0n+RMjJ9+gxGjhy122h4+PBhOb2We+g9/otffMTs\n2VuAZmBfVDYpXyq5iJSR4cOHccYZp+U1Gt62DR56KKzmXL8eLrhgHb17/zPvvfeX9DMKW8qReNMI\nXaQMNTaG3uN33AFHHhnq4+eeCxs26MJmUmjaokjCLVoUZqkMHAivvbaFX/3qb0yf3vRx73H1B69s\nXY7QzawX8AzQk1CiedjdbzKzamAGUAusAoa6+6Z29tcIXUoiqRfzduyA2bPDtMPVq8OUw+rq2Xz/\n+1fSs2e4Y0/b+ntSz0UlKdpFUTPb090/NLM9gOeAMcA3gPXu/nMzGwdUu/v4dvZVQpei23mBsaME\nV0zFSp7r1+/qPV5bG1ZzXnghvPtufMsqeiMpnFwSOu6e8RewJ/AicCKwAuif3j4AWNHBPi5STOvW\nrfOqqr4OL3mY7/GSV1X19XXr1hX92NOmPehVVX29T58TvKqqr0+b9mDer/nyy+5XXeW+zz7u3/62\n+8KFu//9ggULvE+fE9L/1vDVu/fxvmDBgryPnY9inItKls6d2eXojJ4Uau2LgfeA/0hve7fNczZ0\nsG8J/ulSyaJKcIV8I9mxw/3RR91PO819//3db7rJfe3a4h+3UOIYU7nLJaFnNG3R3VuA482sNzDb\nzI4C2tZROqyrTJgw4ePH9fX11NfXZ3JYkYxEdef3Qqz2bN17vF+/UFb55jc77z0exxWdxV75WglS\nqRSpVCq/F8n2HQD4v8APgeXsXnJZ3sHzi/9WJhVv58f93r2PL9nH/XxGpStXuo8e7V5d7T5smPtf\n/+re0pL98RcsWBCLUbBG6IVHMUouhOVmfdKPqwgzXs4BJgLj0tvHATd3sH9p/vVS8aJIcNm8kbS0\nuP/pT+5nn+1eU+N+ww3ub71VslALorNzHMWbapLlktAzmbZ4NHAfoY7eDZjh7v9uZn2BmcCBwGrC\ntMWN7ezvXR1DpJx1NbNj8+bQFOuWW3b1Hh8+HKqqIgg2LZfZKJnMJNIsl8JRLxeRGHnzzTDl8He/\ng69+NSTyr36VyG/plssUT7XWLT2tFBWJmDukUnDRRfClL4VtL74Is2bBqadGn8ybmpoYOXIUW7bM\nZ9OmhWzZMp+RI0fR1NTU6X47L3rq5hfxpuZcIgWwZQtMmxZWc27bFnqr3H9/x73Ho5LrbJSoZhJJ\ndjRCF8nDW2/BT34SVnLOmgW/+AUsWwbf+178kjm0TcyQaWJWj5jyoBq6SJY83Xt80qRwX87LLoPr\nroPPfz7qyDKTTx92XfQsHV0UlbJQrklh2zaYOTMk8g0bQhL/znegT5+oI8teuf4MKokSusRelE20\nctXYGPqO33EHfOELoT5+zjmhXa1IsSihS6yV29S3hQvDRc7HHoOhQ0MiP+qoqKOSSqFpixJr5TD1\nbceOcEu3U06Biy8OCfz118PdgXJJ5k1NTTQ0NHQ5LVCkEJTQpWRynWFRCuvXw803wyGHhBWdP/hB\nSOQ/+hH07Zvba06fPoPa2oEMGXINtbUDmT59Rlb7681AspZtr4Bsv1AvF2klbv0+WvceHzHCfdGi\nwrxuvs2q1FtcKEYvl3yphi5tRT3DorkZHn88zFZZvjzMGb/6athvv8Ido6GhgSFDrmHTpoUfb+vd\n+wTmzbuTE088sdN9y+1agxRHLjV0rRSVkqupqYkkMW3cGHqP33or7Ltv6K1yySWd9x7PVT4rK9Vb\nXHKlGrok3sqVMHp0qI83NMDvfw8vvACXXlqcZA75rayM87UGiTeVXCQnUZdNutLSElZxTpoEixbB\nd78L11wDn/1saePI9Tzls5pTkkHz0KUk4rw4aPNmuO++MFPlU5/a1Xv8U5+KOrLsxf1NU4pLCb1M\nldN/3LhesHvjjV29xwcPDouAvvKV6NvViuRKC4vKUL5zlUstTouD3GH+fLjwQhg0KCzFX7QIHn44\nHjeSECk1jdAjFNfRbmfiEPOWLeHC5uTJYWXn2LFw+eXw6U+X5PAiJaERepmJ02g3U1H2xX7rLbjh\nhtB7fM4c+NWvQu/xq69WMhcBjdAjFYfRbq5KVfd3h//5nzBbZe7cMBK/7jo47LCiHVIkFrSwqMzs\nHO2OHDl4t+lpcU/mUPzFQVu37uo9vnFjSOJ33w29exftkCJlTyP0GCinWS7FtnZt6Dt+551w9NG7\neo93U3FQKoymLVagpLwZvPhiGI3/8Y8wbFhI5EceGXVUItHRRdEKU25THtvavj2UVU4+Gb7xDTjm\nmNCy9o47lMxFcqERehlobxRezhdU16+Hu+6C22+Hgw8O0w4vuAC6x/iKTlI+CUn50Ag9gToahZfj\nlMeXX4arroJDD4VXXw23dnvmmTA6j3MyL/dPQlI5NEKPsc5G4UBZjNCbm+EPfwiLgFauDL3Hv/vd\nwvYeL6Zy/iQk5U0j9ITpbBQe5QKfTGzcCL/+dZgvfvPNcOWV8Oab8NOfRpfMc7mlWzl+EpIK1tUt\njYADgKeBZcDLwJj09mrgKWAl8CTQp4P987wRU+XK5DZm69at8wULFmR8a7NiW77cfdQo9+pq90sv\ndX/++agjCnK9pVu+t5ITyRU53IIuk4Q+ADgu/XivdAIfCEwEfpTePg64uYP9S/OvT6i43YOzPc3N\n7k884X7mme777ef+05+6v/121FHtUqj7e8b5ZyDJk0tCz7qGbmZzgFvTX6e6e6OZDQBS7j6wned7\ntseQ3cV1hkXr3uNVVWG2yre+Fb/e4/nc33OnuP4MJLmKvrDIzOqAFPAFYI27V7f6uw3u3redfZTQ\nE6bceo/rwqaUo6L2cjGzvYCHgbHuvtnM2mbpDrP2hAkTPn5cX19PfX19NjFKDOzsPT55Mjz7LFxx\nReg9XlsbdWRdy6RnjkbgErVUKkUqlcrrNTIaoZtZd+CPwH+5+6T0tuVAfauSy3x3P6KdfStihJ7U\nhPDhh7t6j7e0hNF4ufYe7+hnFOdb6knlKlrJxcymAv9w939ttW0isMHdJ5rZOKDa3ce3s2/iE3oS\nE8KaNfDLX37I1Kk9GDSoheuv78Xpp8e3rJIrlWMkrooyD93MTgYuA04zs8VmtsjMziLMchliZiuB\n04Gbcwm63DU1NTFy5Ci2bJnPpk0L2bJlPiNHjspqrnNcuMNzz8HQoXDEEVu57bb72bHjEv77vz9D\nU9OMxCVz0DxzSZYuE7q7P+fue7j7ce5+vLuf4O5/cvcN7n6Gux/u7l9z942lCDhukpAQtm6FqVPh\nxBNhxAg47rjNNDcfTnPzl9m8+dGyfpPqSl1d+FQFS9NblrJ9+2rq6uqiC0okR1op2ka2qwnLOSGs\nXQsTJkBdHTzwANx0U1ieP2TIcnr16kc5v0llKu4rbkWyku3E9Wy/KKOFRbmuJiy3hScNDe6XX+6+\nzz7uV1/tvmzZ7n9fiasj47biVoRSLCzKVrlcFM334ljcZ7ls3w6zZoWbSLz9NoweHfqrVFe3//yd\nF3pbT/OLw4XeuJ9nkULRHYvyUIjVhHH0j3/s6j1+6KFh2uH552fWrjZuyTOJs4lEOqKEnoekTV9b\nujSMxmfNgosuCon8uOOijip3Sfv5iHRF7XPzkISLY83NMGdOWI5/9tlwyCHhRhK//W15J3NIxmwi\nkWLTCL2NuJUZMrFxI0yZArfeCv37hyZZl1wCPXpEHVnhaIQulaaovVwqRU1NTdkkiBUrQqfD6dPD\niPzBB+Gkk6KOqjgy6cciUuk0Qi8zLS3w5JOhPr54cbid2/e+B5/5TPvPL8dPHJ1J2r9HpCO6KJpg\n77+/q/f4nntm1ntcs0JEypcSegK9/nqojU+dGi52jh0Lp5zSdZMs1ZxFyptmuSSEO/z5z2G++Ekn\nQc+eobzy8MOZ30giylkhudyMWUTyp4QeIx9+CHffDcccE+aNf/3rsHo1TJwIBx2U3WtF1WNm+vQZ\n1NYOZMiQa6itHcj06TOy2l9vBiJ5yLZXQLZflFEvl6j8/e/u48a577uv+3nnuc+d697Skv/rlrrH\nTKFuxpxtLx2RJEK9XMrHzt7jkyfDvHnw7W+H/iqHHlrY45RyVkg+7RNU8xfZneahl4GtW2HGjDDt\n8L33Qmnlnnugd+/iHK+U8+p3L/OEpJxpmWdnzX/Llk/W/JXQRTKjhF4i77wDd9wBd94Jxx4L//Zv\ncNZZ0C1BVzHyWfyTz5uBiAQquRRZQ0MYjT/+eJg3PmYMHPGJW2knS65lnri27BWJguahx8T27fDI\nI6E+/s47oTZ+xRUd9x6XXbQSVCRQQo9Y697jhx22q/f4HntEHZmIlBstLIrI0qXh7j+HHRZWdj7+\nOMyfH/qQ70zmml8tIsWmhJ6j5maYPTssxz/nnNB7/LXXQhvbY4/d/bn5LrYREcmESi5ZevfdcMOI\nW2+F/fcPvVUuvrjj3uOaXy0iuVDJJQO5lj6WL4dRo8JIfPFimDkT/vpXGDas8xtJ6E47IlIqFZXQ\nsy19tLSEeviZZ4bSSk0NvPIKPPAAZHrf6Kh6qohI5amYkks2pY/334ff/S70Ht9771BWGTYMevXK\n7diaXy0i2dLS/05ksrT89ddDEr//fjj99FArP/nkzNrVdmb48GGcccZpml8tIkVVMQm9o6XltbV1\nzJsXVnM+/zyMHBlq5Nm2q+1KOd2rVETKU5clFzObAnwdaHT3Y9LbqoEZQC2wChjq7ps62D8WJRfY\nvfSxbds6LrtsNs8/H4rhY8fCZZeF27uJiEStKCtFzewUYDMwtVVCnwisd/efm9k4oNrdx3ewf2wS\nOsDSpev55S+38cQT/fmnf+rG2LFw2mn5l1VERAqpKDV0d3/WzGrbbL4AODX9+D4gBbSb0ONm06Z+\n9OsHL7wAn/tc1NGIiBRORrNc0gn9D61G6BvcvW+rv9/t+zb7xmqELiJSDqJcWKSMLSISsVxnuTSa\nWX93bzSzAcC6zp48YcKEjx/X19dTX1+f42GlkNSqViQ+UqkUqVQqr9fItORSRyi5HJ3+fiKwwd0n\nlttFUQl2zvjp2TNM59RiJ5F4KdYsl2lAPdAPaARuBOYADwEHAqsJ0xY3drC/EnrMqGGYSPwVa5bL\npR381RnZHEjiQzdkFkmmimrOJYEahokkkxJ6BaqpqWHKlNupqhpM794nUFU1mClTbtfoXKTMVUy3\nRfkkzXIRiS/dJFpEJCF0xyIRkQqmhC4ikhBK6CIiCaGEnoVcbzAtIlIKSugZyvYG0yIipaZZLhnQ\nUnkRKTXNcimSnUvlQzKH1kvlRUTiQgk9A1oqLyLlQAk9A1oqLyLlQDX0LGipvIiUipb+i4gkhC6K\niohUMCV0EZGEUEIXEUkIJXQRkYRQQhcRSQgldBGRhFBCFxFJCCV0EZGEUEIXEUkIJXQRkYRQQhcR\nSQgldBGRhFBCFxFJiLwSupmdZWYrzOxVMxtXqKBERCR7OSd0M+sG3AqcCRwFDDezgYUKrJhSqVTU\nIXxCHGOCeMalmDKjmDIX17iylc8IfRDwmruvdvftwIPABYUJq7ji+MOLY0wQz7gUU2YUU+biGle2\n8knonwXWtPr+rfQ2ERGJgC6KiogkRM63oDOz/wNMcPez0t+PB9zdJ7Z5nu4/JyKSg5LdU9TM9gBW\nAqcD7wALgOHuvjynFxQRkbx0z3VHd282s9HAU4TSzRQlcxGR6OQ8QhcRkXgpyUVRM7vRzN4ys0Xp\nr7NKcdwOYondYigzW2VmL5nZYjNbEFEMU8ys0cyWttpWbWZPmdlKM3vSzPrEJK7Ifp/M7AAze9rM\nlpnZy2Y2Jr090nPVTlzXpbdHea56mdkL6d/rl83sxvT2yM5VJzFFnqPMrFv62I+lv8/6PJVkhJ4+\nae+7+6+LfrDO4+gGvEqo+/8/oAH4lruviDiuN4Avuvu7EcZwCrAZmOrux6S3TQTWu/vP029+1e4+\nPgZxRfb7ZGYDgAHuvsTM9gIWEtZffIcIz1UncQ0jwv97Zranu3+Yvub2HDAG+AbRnqv2YjqbiHOU\nmf0A+CLQ293Pz+X/XymnLWZ1tbZI4roYyoh4Cqm7Pwu0fUO5ALgv/fg+4MKSBkWHcUFEv0/uvtbd\nl6QfbwaWAwcQ8bnqIK6d60Ii+7/n7h+mH/YiXLNzoj9X7cUEEZ4nMzsAOAe4p9XmrM9TKZPIaDNb\nYmb3RPHRPS2ui6EcmGtmDWZ2VdTBtLKfuzdCSBjAfhHH01rkv09mVgccBzwP9I/LuWoV1wvpTZGd\nq3QZYTGwFpjr7g1EfK46iAmi/Z36T+B6dr25QA7nqWAJ3czmmtnSVl8vp/88D7gdOMTdjyOcxEhL\nLzF0srufQHiHvjZdZoijuFxBj/z3KV3WeBgYmx4Rtz03kZyrduKK9Fy5e4u7H0/4FDPIzI4i4nPV\nTkxHEuF5MrNzgcb0J6zOPiV0eZ5ynrb4iSO5D8nwqXcDfyjUcbP0NnBQq+8PSG+LlLu/k/6zycxm\nE0pDz0YbFQCNZtbf3RvTNdp1UQcE4Ty1+rbkv09m1p2QNO9390fTmyM/V+3FFfW52snd3zOzFHAW\nMThXbWNqUzsv9Xk6GTjfzM4BqoC9zex+YG2256lUs1wGtPr2YuBvpThuOxqAQ82s1sx6At8CHoso\nFiBcoEmPqjCzTwNfI7rzY+w+QngMGJF+/C/Ao213KJHd4orB79NvgVfcfVKrbXE4V5+IK8pzZWb7\n7ixdmFkVMIRQ24/sXHUQ04ooz5O73+DuB7n7IYSc9LS7/zPhTWVE+mmZnSd3L/oXMBVYCiwB5hBq\nQyU5djuxnEVY4foaMD6qOFrFc3D6vCwGXo4qJmAaYebPVuDvhFkb1cC89Pl6CtgnJnFF9vtEGE01\nt/qZLUr/TvWN8lx1EleU5+rodBxL0jH8JL09snPVSUyxyFHAqcBjuZ4nLSwSEUkIdVsUEUkIJXQR\nkYRQQhcRSQgldBGRhFBCFxFJCCV0EZGEUEIXEUkIJXQRkYT4/+ue6NETmxy8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x70622b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def gradientDescent(x, y, alphaFactor, numIterations):\n",
    "    theta = np.ones(2)\n",
    "    x_transposed = x.transpose()\n",
    "    n_samples, = np.shape(x)\n",
    "\n",
    "    np.seterr(all='raise') # Watch out for overflows\n",
    "\n",
    "    for i in range(0, numIterations):\n",
    "\n",
    "        hypothesis = np.zeros(n_samples)\n",
    "        for sample in range(0, n_samples):\n",
    "            hypothesis[sample] =  theta[0] + theta[1] * x[sample]\n",
    "\n",
    "        error_vector = hypothesis - y\n",
    "\n",
    "        gradient = np.zeros(2)\n",
    "        gradient[0] = np.sum(error_vector) / n_samples\n",
    "        gradient[1] = np.dot(x_transposed, error_vector) / n_samples\n",
    "\n",
    "        theta = theta - alphaFactor * gradient\n",
    "    \n",
    "    return theta\n",
    "\n",
    "def plotLinearPredictor(x, y, theta_coefficients):\n",
    "    plt.scatter(x, y)\n",
    "    \n",
    "    y_regression = np.zeros(len(x))\n",
    "    for sample in range(0, len(x)):\n",
    "        y_regression[sample] = theta_coefficients[0] + theta_coefficients[1] * x[sample]\n",
    "\n",
    "    plt.plot(x, y_regression)\n",
    "    plt.show()\n",
    "    \n",
    "theta_coefficients = gradientDescent(x, y, 0.001, 100000)\n",
    "\n",
    "plotLinearPredictor(x, y, theta_coefficients)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multivariate gradient descent\n",
    "\n",
    "The generalized version for multiple features is\n",
    "\n",
    "$$\n",
    "\\forall \\ j \\quad \\theta_j = \\theta_j - \\alpha \\frac{\\partial}{\\partial \\theta_j}J(\\theta_1, \\theta_2, \\cdots, \\theta_n) = \\frac{1}{m} \\sum^{m}_{i=1}{(h_\\theta(x^{(i)} - y^{(i)})^2} x^{(i)}_j\n",
    "$$\n",
    "\n",
    "where $x^{(i)}_j$ is the $j$-th feature $i$-th sample."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normal equation method\n",
    "\n",
    "An analytical solution exists and as it is the common case in calculus it involves \n",
    "\n",
    "$$\n",
    "\\theta \\in \\mathbb R^{n+1} \\quad \\forall \\ j \\quad \\frac{\\partial}{\\partial \\theta_j}J(\\theta) = 0\n",
    "$$\n",
    "\n",
    "although this is straightforward, there's a simpler method to calculate the system solution vector. Let us define a *design matrix* and a $y$ vector for the training data\n",
    "\n",
    "$$\n",
    " X = \n",
    " \\begin{bmatrix}\n",
    " X^{(1)}_0 & X^{(1)}_1 & \\cdots & X^{(1)}_n \\\\\n",
    " X^{(2)}_0 & X^{(2)}_1 & \\cdots & X^{(2)}_n \\\\\n",
    " \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    " X^{(m)}_0 & X^{(m)}_1 & \\cdots & X^{(m)}_n \\\\\n",
    " \\end{bmatrix}\n",
    " \\qquad\n",
    " y =\n",
    " \\begin{bmatrix}\n",
    " y^{(1)} \\\\\n",
    " y^{(2)} \\\\\n",
    " \\vdots \\\\\n",
    " y^{(m)} \\\\\n",
    " \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "It turns out that\n",
    "\n",
    "$$\n",
    "\\theta = (X^TX)^{-1} X^Ty\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHeZJREFUeJzt3XmUVeWV9/HvRoaUUbDAEkw0VRqNqHFMxE5rYqESpzjG\nQFA7TUSjQYSkswxEfZe4enVeydQLHNqJGHFAiAKa6BuByI2taaVkEIOAtgoRI0UFBCUiQ9V+/3gu\nUpQ13Pmce+7vs1Ytbh3uuWdzqtj3ufs8zz7m7oiISPnrFnUAIiJSGEroIiIJoYQuIpIQSugiIgmh\nhC4ikhBK6CIiCZFRQjezPmb2WzNbbmbLzOxEM6s2szlmttLMnjazPsUOVkREOpbpCH0S8JS7Hw4c\nA6wAxgPz3P0w4BngJ8UJUUREMmFdLSwys97AYnf/fJvtK4BT3L3RzAYAKXcfWLxQRUSkM5mM0A8C\n/m5m95nZIjO728z2BPq7eyOAu68F9itmoCIi0rlMEnp34Hjgdnc/HvgHodzSdmivHgIiIhHqnsFz\n1gBvu/tL6e8fIyT0RjPr36rksq69nc1MiV5EJAfubtk8v8sRerqs8raZfSG96TRgGfAEMCK97V+B\nxzt5jVh93XTTTZHHUA4xxTUuxaSYKiGuXGQyQgcYAzxkZj2AN4HvAnsAM8zscmA1MDSnCEREpCAy\nSuju/jJwQjt/dXphwxERkVxV5ErR+vr6qEP4hDjGBPGMSzFlRjFlLq5xZavLeeh5H8DMi30MEZGk\nMTO80BdFRUSkPCihi4gkhBK6iEhCKKGLiCSEErqISEIooYuIJIQSuohIQiihi4gkhBK6iEhCKKGL\niCSEErqISEIooYuIJIQSuohIQiihi4gkhBK6iEhCKKGLiCSEErqISEIooYuIJIQSuohIQiihi4gk\nhBK6iEhCKKGLiCSEErqISEIooYuIJIQSuogkWlNTEw0NDTQ1NUUdStEpoYtIYk2bNp3a2oEMGXI1\ntbUDmTZtetQhFZW5e9dPMlsFbAJagO3uPsjMqoHpQC2wChjq7pva2dczOYaISCE1NTVRWzuQLVvm\nA0cDS6mqGszq1SuoqamJOrwumRnubtnsk+kIvQWod/fj3H1Qett4YJ67HwY8A/wkmwOLiBTTqlWr\n6NmzjpDMAY6mR49aVq1aFV1QRZZpQrd2nns+cH/68f3ABYUKSkQkX3V1dWzbtgpYmt6ylO3bV1NX\nVxddUEWWaUJ3YK6ZNZjZFelt/d29EcDd1wL7FSNAEZFc1NTUMGXKHVRVDaZ37+OpqhrMlCl3lEW5\nJVeZ1tD3d/d3zawGmAOMAR53976tnrPe3fu1s69q6CISmaamJlatWkVdXV1ZJfNcaujdM3mSu7+b\n/rPJzGYDg4BGM+vv7o1mNgBY19H+EyZM+PhxfX099fX12cQoIpKzmpqaskjkqVSKVCqV12t0OUI3\nsz2Bbu6+2cw+TRih3wycBmxw94lmNg6odvfx7eyvEbqISJZyGaFnktAPAmYR6ujdgYfc/RYz6wvM\nAA4EVhOmLW5sZ38ldBGRLBUloedLCV1EoHxr2VEp5jx0EZGcVdqKzahohC4iRVXuKzajohG6iMRO\nJa7YjIoSuogUVSWu2IyKErqIFFUlrtiMimroIlISmuWSHU1bFBFJCF0UFRGpYEroIiIJoYQuIpIQ\nSugiIgmhhC4ikhBK6CIiCaGELiKx19TURENDA01NTVGHEmtK6CISa+rUmDktLBKR2KrkTo1aWCQi\niaJOjdlRQheR2FKnxuwooYtIbKlTY3ZUQxeR2KvETo3qtigikhC6KCoiUsGU0EVEEkIJXUQkIZTQ\nRUQSQgldRGJB/Vryp4QuIpFTv5bC0LRFEYlUJfdr6YymLYpI2VG/lsLJOKGbWTczW2RmT6S/rzaz\nOWa20syeNrM+xQtTRJJK/VoKJ5sR+ljg1VbfjwfmufthwDPATwoZmIhUBvVrKZyMauhmdgBwH/Af\nwL+5+3lmtgI4xd0bzWwAkHL3ge3sqxq6iHSpEvu1dCaXGnr3DJ/3n8B1QOuySn93bwRw97Vmtl82\nBxYRaa2mpqbgiXzdOrj7bqiqgh/9qKAvHUtdJnQzOwdodPclZlbfyVM7HIZPmDDh48f19fXU13f2\nMiIi+VmyBCZNgtmz4eKLYezYqCPqWiqVIpVK5fUaXZZczOynwGXADqAK2BuYBXwZqG9Vcpnv7oe3\ns79KLiJSdDt2wOOPw+TJ8OabcM01cOWV0K9f1JHlpujtc83sFOBH6Rr6z4D17j7RzMYB1e4+vp19\nlNBFpGjeew/uvRduvx0OOCCMxi+4AHr0iDqy/BSzht6eW4AZZnY5sBoYmsdriYhk5dVXw2h8+nQ4\n91x47DH40peijipaWikqImWjpQWeeirUx//yF7j6arjqKhgwIOrICq/UI3QRkZJ4/3247z649Vao\nrg5llW99C3r1ijqyeFFCF5HYev31kMQffBCGDIGpU+ErXwHLatxaOZTQRSRW3GHu3FAfX7AArrgC\nXn4ZDjww6sjiTwldRGLhH/8II/Bbb4Xu3UNZ5be/DYuCJDNK6CISqVWrwpTD++6Dr341PK6vV1kl\nF2qfKyIl5w5/+hNcdFGYaugODQ0waxYMHqxkniuN0EWkZD76CKZNC9MOP/oIxowJZZa99srvddXY\nK9AIXUSK7m9/gxtvhNraUBefODEsDBo1Kv9krtvX7aKFRSJSNC+8EEbjTz8Nl14Ko0fDYYcV7vWT\nfPs63YJORCK3bRs89BCceCJccgkMGgRvvRVmrxQymYNuX9eWaugiUhDr1sGdd4avww+HG26Ac86B\nPfYo3jF3v31dGKFX8u3rNEIXkbwsXgwjRoTR95o1obzyxz/CeecVN5mDbl/XlmroIpK1HTvCzSMm\nTQrzyKPuPZ7EWS5F74eeCyV0keTYsGFX7/EDDwyrOS+8MKzslMJSt0URKYply0JvlRkzQu/xmTPV\nezyOlNBFpF3Nzbt6j7/6aug9vmIF9O8fdWTSESV0EdnNpk27eo/37RvKKkOHQs+eUUcmXVFCFxEA\nXnttZ+/xFk444T1uvbUbZ51Vrb4qZUTTFkUqmDvMmRPmi598Mrz99qt89NHRLFjwdS6++BAeeaRy\nl9GXI81yEYlQVNPtdvYenzw5lFLGjoXTT29i4MBkLqMvR1r6L1JGomgqtWoVXHddaJI1Zw7813/B\nkiVw+eXQ2Khl9OVOI3SRCJSyqdTO3uOTJ8Ozz4ZVnddcAwcdFF1M0jXNQxcpEzubSm3Z8snRcCGS\nZ1NTEytXrmbhwsO477692bq1697jO5fRjxw5mB49atm+fXVFL6MvRxqhi0SgmKPh229/gh/84H9p\nbh6B2Utcd10vfvrTU+iWYYE1icvoy5GW/ouUkWnTpjNy5KjdRsPDhw/L6bXcQ+/xn//8I2bN2gI0\nA/uiskn5UslFpIwMHz6M008/Na/R8LZt4Q5AkybB+vVw/vnr6N37X3j//T+ln1HYUo7Em0boImWo\nsRHuuiv0Hj/iiFAfP+cc2LBBFzaTQtMWRRJu0aIwS2XgQHj99S388pd/Ydq0po97j6s/eGXrcoRu\nZr2AZ4GehBLNo+5+s5lVA9OBWmAVMNTdN7Wzv0boUhJJvZi3YwfMmhWmHa5eHaYcVlfP4gc/uIKe\nPcMde9rW35N6LipJ0S6Kmtme7v6hme0BPA+MAb4JrHf3n5nZOKDa3ce3s68SuhTdzguMHSW4YipW\n8ly/flfv8drasJrzggvgvffiW1bRG0nh5JLQcfeMv4A9gZeAE4AVQP/09gHAig72cZFiWrdunVdV\n9XV42cN8j5e9qqqvr1u3rujHfvjhR7yqqq/36XO8V1X19YcffiTv13zlFfcrr3TfZx/373zHfeHC\n3f9+wYIF3qfP8el/a/jq3fs4X7BgQd7HzkcxzkUlS+fO7HJ0Rk8KtfbFwPvA/01ve6/NczZ0sG8J\n/ulSyaJKcIV8I9mxw/3xx91PPdV9//3db77Zfe3a4h+3UOIYU7nLJaFnNG3R3VuA48ysNzDLzI4E\n2tZROqyrTJgw4ePH9fX11NfXZ3JYkYxEdef3Qqz2bN17vF+/UFb51rc67z0exxWdxV75WglSqRSp\nVCq/F8n2HQD4P8CPgOXsXnJZ3sHzi/9WJhVv58f93r2PK9nH/XxGpStXuo8e7V5d7T5smPuf/+ze\n0pL98RcsWBCLUbBG6IVHMUouhOVmfdKPqwgzXs4GJgLj0tvHAbd0sH9p/vVS8aJIcNm8kbS0uP/h\nD+5nneVeU+N+/fXua9aULNSC6OwcR/GmmmS5JPRMpi0eBdxPqKN3A6a7+3+YWV9gBnAgsJowbXFj\nO/t7V8cQKWddzezYvDk0xbr11l29x4cPh6qqCIJNy2U2SiYziTTLpXDUy0UkRt56K0w5/M1v4Gtf\nC4n8a18j8lu65TLFU611S08rRUUi5g6pFFx4IXz5y2HbSy/BzJlwyinRJ/OmpiZGjhzFli3z2bRp\nIVu2zGfkyFE0NTV1ut/Oi566+UW8qTmXSAFs2QIPPxxWc27bFnqrPPBAx73Ho5LrbJSoZhJJdjRC\nF8nDmjVwww1hJefMmfDzn8OyZfD978cvmUPbxAyZJmb1iCkPqqGLZMnTvccnTQr35bz0Urj2WvjC\nF6KOLDP59GHXRc/S0UVRKQvlmhS2bYMZM0Ii37AhJPHvfhf69Ik6suyV68+gkiihS+xF2UQrV42N\noe/4nXfCF78Y6uNnnx3a1YoUixK6xFq5TX1buDBc5HziCRg6NCTyI4+MOiqpFJq2KLFWDlPfduwI\nt3Q7+WS46KKQwN94I9wdKJdk3tTURENDQ5fTAkUKQQldSibXGRalsH493HILHHxwWNH5wx+GRP7j\nH0Pfvrm95rRp06mtHciQIVdTWzuQadOmZ7W/3gwka9n2Csj2C/VykVbi1u+jde/xESPcFy0qzOvm\n26xKvcWFYvRyyZdq6NJW1DMsmpvhySfDbJXly8Oc8auugv32K9wxGhoaGDLkajZtWvjxtt69j2fe\nvLs44YQTOt233K41SHHkUkPXSlEpuZqamkgS08aNoff4bbfBvvuG3ioXX9x57/Fc5bOyUr3FJVeq\noUvirVwJo0eH+nhDAzz0ELz4IlxySXGSOeS3sjLO1xok3lRykZxEXTbpSktLWMU5aRIsWgTf+x5c\nfTV89rOljSPX85TPak5JBs1Dl5KI8+KgzZvh/vvDTJVPfWpX7/FPfSrqyLIX9zdNKS4l9DJVTv9x\n43rB7s03d/UeHzw4LAL66lejb1crkistLCpD+c5VLrU4LQ5yh/nz4YILYNCgsBR/0SJ49NF43EhC\npNQ0Qo9QXEe7nYlDzFu2hAubkyeHlZ1jx8Jll8GnP12Sw4uUhEboZSZOo91MRdkXe80auP760Ht8\n9mz45S9D7/GrrlIyFwGN0CMVh9FurkpV93eH//mfMFtl7twwEr/2Wjj00KIdUiQWtLCozOwc7Y4c\nOXi36WlxT+ZQ/MVBW7fu6j2+cWNI4vfcA717F+2QImVPI/QYKKdZLsW2dm3oO37XXXDUUbt6j3dT\ncVAqjKYtVqCkvBm89FIYjf/+9zBsWEjkRxwRdVQi0dFF0QpTblMe29q+PZRVTjoJvvlNOPro0LL2\nzjuVzEVyoRF6GWhvFF7OF1TXr4e774Y77oCDDgrTDs8/H7rH+IpOUj4JSfnQCD2BOhqFl+OUx1de\ngSuvhEMOgddeC7d2e/bZMDqPczIv909CUjk0Qo+xzkbhQFmM0Jub4Xe/C4uAVq4Mvce/973C9h4v\npnL+JCTlTSP0hOlsFB7lAp9MbNwIv/pVmC9+yy1wxRXw1ltw443RJfNcbulWjp+EpIJ1dUsj4ADg\nGWAZ8AowJr29GpgDrASeBvp0sH+eN2KqXJncxmzdunW+YMGCjG9tVmzLl7uPGuVeXe1+ySXuL7wQ\ndURBrrd0y/dWciK5Iodb0GWS0AcAx6Yf75VO4AOBicCP09vHAbd0sH9p/vUJFbd7cLanudn9qafc\nzzjDfb/93G+80f2dd6KOapdC3d8zzj8DSZ5cEnrWNXQzmw3clv46xd0bzWwAkHL3ge0837M9huwu\nrjMsWvcer6oKs1W+/e349R7P5/6eO8X1ZyDJVfSFRWZWB6SALwJvu3t1q7/b4O5929lHCT1hyq33\nuC5sSjkqai8XM9sLeBQY6+6bzaxtlu4wa0+YMOHjx/X19dTX12cTo8TAzt7jkyfDc8/B5ZeH3uO1\ntVFH1rVMeuZoBC5RS6VSpFKpvF4joxG6mXUHfg/8P3eflN62HKhvVXKZ7+6Ht7NvRYzQk5oQPvxw\nV+/xlpYwGi/X3uMd/YzifEs9qVxFK7mY2VTg7+7+b622TQQ2uPtEMxsHVLv7+Hb2TXxCT2JCePtt\n+MUvPmTq1B4MGtTCddf14rTT4ltWyZXKMRJXRZmHbmYnAZcCp5rZYjNbZGZnEma5DDGzlcBpwC25\nBF3umpqaGDlyFFu2zGfTpoVs2TKfkSNHZTXXOS7c4fnnYehQOPzwrdx++wPs2HEx//3fn6GpaXri\nkjlonrkkS5cJ3d2fd/c93P1Ydz/O3Y939z+4+wZ3P93dD3P3r7v7xlIEHDdJSAhbt8LUqXDCCTBi\nBBx77Gaamw+jufkrbN78eFm/SXWlri58qoKl6S1L2b59NXV1ddEFJZIjrRRtI9vVhOWcENauhQkT\noK4OHnwQbr45LM8fMmQ5vXr1o5zfpDIV9xW3IlnJduJ6tl+U0cKiXFcTltvCk4YG98suc99nH/er\nrnJftmz3v6/E1ZFxW3ErQikWFmWrXC6K5ntxLO6zXLZvh5kzw00k3nkHRo8O/VWqq9t//s4Lva2n\n+cXhQm/cz7NIoeiORXkoxGrCOPr733f1Hj/kkDDt8LzzMmtXG7fkmcTZRCIdUULPQ9Kmry1dGkbj\nM2fChReGRH7ssVFHlbuk/XxEuqL2uXlIwsWx5maYPTssxz/rLDj44HAjiV//uryTOSRjNpFIsWmE\n3kbcygyZ2LgRpkyB226D/v1Dk6yLL4YePaKOrHA0QpdKU9ReLpWipqambBLEihWh0+G0aWFE/sgj\ncOKJUUdVHJn0YxGpdBqhl5mWFnj66VAfX7w43M7t+9+Hz3ym/eeX4yeOziTt3yPSEV0UTbAPPtjV\ne3zPPTPrPa5ZISLlSwk9gd54I9TGp04NFzvHjoWTT+66SZZqziLlTbNcEsId/vjHMF/8xBOhZ89Q\nXnn00cxvJBHlrJBcbsYsIvlTQo+RDz+Ee+6Bo48O88a/8Q1YvRomToTPfS6714qqx8y0adOprR3I\nkCFXU1s7kGnTpme1v94MRPKQba+AbL8oo14uUfnrX93HjXPfd1/3c891nzvXvaUl/9ctdY+ZQt2M\nOdteOiJJhHq5lI+dvccnT4Z58+A73wn9VQ45pLDHKeWskHzaJ6jmL7I7zUMvA1u3wvTpYdrh+++H\n0sq990Lv3sU5Xinn1e9e5glJOdMyz86a/5Ytn6z5K6GLZEYJvUTefRfuvBPuuguOOQb+/d/hzDOh\nW4KuYuSz+CefNwMRCVRyKbKGhjAaf/LJMG98zBg4/BO30k6WXMs8cW3ZKxIFzUOPie3b4bHHQn38\n3XdDbfzyyzvuPS67aCWoSKCEHrHWvccPPXRX7/E99og6MhEpN1pYFJGlS8Pdfw49NKzsfPJJmD8/\n9CHfmcw1v1pEik0JPUfNzTBrVliOf/bZoff466+HNrbHHLP7c/NdbCMikgmVXLL03nvhhhG33Qb7\n7x96q1x0Uce9xzW/WkRyoZJLBnItfSxfDqNGhZH44sUwYwb8+c8wbFjnN5LQnXZEpFQqKqFnW/po\naQn18DPOCKWVmhp49VV48EHI9L7RUfVUEZHKUzEll2xKHx98AL/5Teg9vvfeoawybBj06pXbsTW/\nWkSypaX/nchkafkbb4Qk/sADcNppoVZ+0kmZtavtzPDhwzj99FM1v1pEiqpiEnpHS8tra+uYNy+s\n5nzhBRg5MtTIs21X25VyulepiJSnLksuZjYF+AbQ6O5Hp7dVA9OBWmAVMNTdN3WwfyxKLrB76WPb\ntnVceuksXnghFMPHjoVLLw23dxMRiVpRVoqa2cnAZmBqq4Q+EVjv7j8zs3FAtbuP72D/2CR0gKVL\n1/OLX2zjqaf688//3I2xY+HUU/Mvq4iIFFJRauju/pyZ1bbZfD5wSvrx/UAKaDehx82mTf3o1w9e\nfBE+//mooxERKZyMZrmkE/rvWo3QN7h731Z/v9v3bfaN1QhdRKQcRLmwSBlbRCRiuc5yaTSz/u7e\naGYDgHWdPXnChAkfP66vr6e+vj7Hw0ohqVWtSHykUilSqVRer5FpyaWOUHI5Kv39RGCDu08st4ui\nEuyc8dOzZ5jOqcVOIvFSrFkuDwP1QD+gEbgJmA38FjgQWE2Ytrixg/2V0GNGDcNE4q9Ys1wu6eCv\nTs/mQBIfuiGzSDJVVHMuCdQwTCSZlNArUE1NDVOm3EFV1WB69z6eqqrBTJlyh0bnImWuYrotyidp\nlotIfOkm0SIiCaE7FomIVDAldBGRhFBCFxFJCCX0LOR6g2kRkVJQQs9QtjeYFhEpNc1yyYCWyotI\nqWmWS5HsXCofkjm0XiovIhIXSugZ0FJ5ESkHSugZ0FJ5ESkHqqFnQUvlRaRUtPRfRCQhdFFURKSC\nKaGLiCSEErqISEIooYuIJIQSuohIQiihi4gkhBK6iEhCKKGLiCSEErqISEIooYuIJIQSuohIQiih\ni4gkhBK6iEhC5JXQzexMM1thZq+Z2bhCBSUiItnLOaGbWTfgNuAM4EhguJkNLFRgxZRKpaIO4RPi\nGBPEMy7FlBnFlLm4xpWtfEbog4DX3X21u28HHgHOL0xYxRXHH14cY4J4xqWYMqOYMhfXuLKVT0L/\nLPB2q+/XpLeJiEgEdFFURCQhcr4FnZn9EzDB3c9Mfz8ecHef2OZ5uv+ciEgOSnZPUTPbA1gJnAa8\nCywAhrv78pxeUERE8tI91x3dvdnMRgNzCKWbKUrmIiLRyXmELiIi8VKSi6JmdpOZrTGzRemvM0tx\n3A5iid1iKDNbZWYvm9liM1sQUQxTzKzRzJa22lZtZnPMbKWZPW1mfWISV2S/T2Z2gJk9Y2bLzOwV\nMxuT3h7puWonrmvT26M8V73M7MX07/UrZnZTentk56qTmCLPUWbWLX3sJ9LfZ32eSjJCT5+0D9z9\nV0U/WOdxdANeI9T9/wY0AN929xURx/Um8CV3fy/CGE4GNgNT3f3o9LaJwHp3/1n6za/a3cfHIK7I\nfp/MbAAwwN2XmNlewELC+ovvEuG56iSuYUT4f8/M9nT3D9PX3J4HxgDfJNpz1V5MZxFxjjKzHwJf\nAnq7+3m5/P8r5bTFrK7WFklcF0MZEU8hdffngLZvKOcD96cf3w9cUNKg6DAuiOj3yd3XuvuS9OPN\nwHLgACI+Vx3EtXNdSGT/99z9w/TDXoRrdk7056q9mCDC82RmBwBnA/e22pz1eSplEhltZkvM7N4o\nPrqnxXUxlANzzazBzK6MOphW9nP3RggJA9gv4nhai/z3yczqgGOBF4D+cTlXreJ6Mb0psnOVLiMs\nBtYCc929gYjPVQcxQbS/U/8JXMeuNxfI4TwVLKGb2VwzW9rq65X0n+cCdwAHu/uxhJMYaeklhk5y\n9+MJ79DXpMsMcRSXK+iR/z6lyxqPAmPTI+K25yaSc9VOXJGeK3dvcffjCJ9iBpnZkUR8rtqJ6Qgi\nPE9mdg7QmP6E1dmnhC7PU87TFj9xJPchGT71HuB3hTpult4BPtfq+wPS2yLl7u+m/2wys1mE0tBz\n0UYFQKOZ9Xf3xnSNdl3UAUE4T62+Lfnvk5l1JyTNB9z98fTmyM9Ve3FFfa52cvf3zSwFnEkMzlXb\nmNrUzkt9nk4CzjOzs4EqYG8zewBYm+15KtUslwGtvr0I+EspjtuOBuAQM6s1s57At4EnIooFCBdo\n0qMqzOzTwNeJ7vwYu48QngBGpB//K/B42x1KZLe4YvD79GvgVXef1GpbHM7VJ+KK8lyZ2b47Sxdm\nVgUMIdT2IztXHcS0Isrz5O7Xu/vn3P1gQk56xt3/hfCmMiL9tMzOk7sX/QuYCiwFlgCzCbWhkhy7\nnVjOJKxwfR0YH1UcreI5KH1eFgOvRBUT8DBh5s9W4K+EWRvVwLz0+ZoD7BOTuCL7fSKMpppb/cwW\npX+n+kZ5rjqJK8pzdVQ6jiXpGG5Ib4/sXHUSUyxyFHAK8ESu50kLi0REEkLdFkVEEkIJXUQkIZTQ\nRUQSQgldRCQhlNBFRBJCCV1EJCGU0EVEEkIJXUQkIf4/XCvo11jAMkYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7085f60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def normEquation(x, y):\n",
    "    ll = []\n",
    "    for sample in x:\n",
    "        ll += [[1] + [sample]]\n",
    "    X = np.matrix(ll)\n",
    "    ll = [[sample] for sample in y]\n",
    "    y = np.matrix(ll)\n",
    "    x_trans = X.T\n",
    "    A = (x_trans * X).I\n",
    "    theta = A * x_trans * y\n",
    "    return theta\n",
    "\n",
    "theta_coefficients = normEquation(x, y)\n",
    "plotLinearPredictor(x, y, theta_coefficients)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Considerations\n",
    "\n",
    "The normal equation method doesn't require an iterative approach and doesn't need features adjustments like *feature scaling*, *mean normalization* or *learning rate adjustment* either. Downside is that the normal equation method doesn't scale well with a huge number of features (big matrix inversion ~ doesn't scale up well)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}