Feature Scaling And Learning Rate In GradientDescent

FeatureScalingAndLearningRateInGradientDescent.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Scaling and Learning Rate in Gradient Descent\n",
    "- [Gradient Descent in Practice I - Feature Scaling (Coursera)](https://www.coursera.org/learn/machine-learning/lecture/xx3Da/gradient-descent-in-practice-i-feature-scaling)\n",
    "- [Gradient Descent in Practice II - Learning Rate (Coursera)](https://www.coursera.org/learn/machine-learning/lecture/3iawu/gradient-descent-in-practice-ii-learning-rate)\n",
    "- [Feature Scalingはなぜ必要？](https://qiita.com/ttskng/items/2a33c1ca925e4501e609)\n",
    "\n",
    "[ここ](https://gist.github.com/shotahorii/63bf6f73a511626e882791f342abdc23)で勾配降下法を用いた線形回帰について書いた。ただし、上の記事のコードではLearning Rateを決め打ちで書いただけで、Learning Rateの選び方について触れていなかった。ここでは、Learning Rateの選び方について書く。また、複数のパラメータについて勾配降下法を実行する際に必須の前処理(**Feature Scaling**)についても触れる。\n",
    "\n",
    "ちなみに、様々な勾配降下法について様々な最適化手法が存在する([勾配降下法の最適化アルゴリズムを概観する](https://postd.cc/optimizing-gradient-descent/))が、この記事ではそのような手法には触れず、あくまでナイーブな実装での話に止まる。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Feature Scaling**  \n",
    "複数の変数について勾配降下法でパラメータを最適化する場合、各変数のスケールが揃っていない場合には正規化あるいは標準化等によってスケールを揃えることで最適化が効率化される。理由を以下に示す。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/latex": [
       "学習させるモデルとして２つの特徴量$$x_1,x_2$$をもつ線形モデル$$f(x_1,x_2)$$を考える。\n",
       "それぞれの特徴量についてのパラメータを$$w_1,w_2$$とする。\n",
       "コスト関数$$E({\\bf w})$$を平均二乗誤差（Mean Squared Error）とすると、勾配降下法におけるパラメータ更新幅は以下となる。\n",
       "<br><br>\n",
       "$$\\Delta {\\bf w} = (\\Delta w_1, \\Delta w_2) = -\\alpha \n",
       "(\\frac{\\partial E({\\bf w})}{\\partial w_1}, \\frac{\\partial E({\\bf w})}{\\partial w_2})$$<br>\n",
       "$$=-\\alpha(\\frac{1}{m}\\sum_{i=1}^m (f(x_{1i},x_{2i}) - y_i)x_{1i},\n",
       "           \\frac{1}{m}\\sum_{i=1}^m (f(x_{1i},x_{2i}) - y_i)x_{2i})$$\n",
       "<br><br>\n",
       "となり、$$\\Delta w_1$$は$$x_1$$の大きさに、$$\\Delta w_2$$は$$x_2$$の大きさにそれぞれ依存していることがわかる。\n",
       "<br>\n",
       "例えば$$x_1$$のスケールが大きく$$x_2$$のスケールが小さい場合、$$x_2$$の更新幅は小さく、学習が$$x_1$$\n",
       "に比べて遅くなってしまう。結果としてパラメータ間で更新幅に偏りが生じる。<br>\n",
       "特徴量にFeature scalingを行うことによって、最適な解を見つけるまでのステップが少なくなる。"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%latex\n",
    "学習させるモデルとして２つの特徴量$$x_1,x_2$$をもつ線形モデル$$f(x_1,x_2)$$を考える。\n",
    "それぞれの特徴量についてのパラメータを$$w_1,w_2$$とする。\n",
    "コスト関数$$E({\\bf w})$$を平均二乗誤差（Mean Squared Error）とすると、勾配降下法におけるパラメータ更新幅は以下となる。\n",
    "<br><br>\n",
    "$$\\Delta {\\bf w} = (\\Delta w_1, \\Delta w_2) = -\\alpha \n",
    "(\\frac{\\partial E({\\bf w})}{\\partial w_1}, \\frac{\\partial E({\\bf w})}{\\partial w_2})$$<br>\n",
    "$$=-\\alpha(\\frac{1}{m}\\sum_{i=1}^m (f(x_{1i},x_{2i}) - y_i)x_{1i},\n",
    "           \\frac{1}{m}\\sum_{i=1}^m (f(x_{1i},x_{2i}) - y_i)x_{2i})$$\n",
    "<br><br>\n",
    "となり、$$\\Delta w_1$$は$$x_1$$の大きさに、$$\\Delta w_2$$は$$x_2$$の大きさにそれぞれ依存していることがわかる。\n",
    "<br>\n",
    "例えば$$x_1$$のスケールが大きく$$x_2$$のスケールが小さい場合、$$x_2$$の更新幅は小さく、学習が$$x_1$$\n",
    "に比べて遅くなってしまう。結果としてパラメータ間で更新幅に偏りが生じる。<br>\n",
    "特徴量にFeature scalingを行うことによって、最適な解を見つけるまでのステップが少なくなる。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**実際に試してみる**  \n",
    "今回の記事では、3変数を用いた線形回帰のパラメータ推定を例に勾配降下法を試す。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from functools import reduce\n",
    "import numpy as np\n",
    "from scipy import stats\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def h(t0,t1,t2,t3,x):\n",
    "    return t0+t1*x[1]+t2*x[2]+t3*x[3]\n",
    "\n",
    "def cost(t0,t1,t2,t3,X,y):\n",
    "    return reduce(lambda a,b:a+b ,[(h(t0,t1,t2,t3,X[i]) - y[i])**2 for i in range(len(X))])/(2*len(X))\n",
    "\n",
    "def gradientdescent(X,y,a,t0=0,t1=0,t2=0,t3=0,max_itr=2000):\n",
    "    vis = [] # for visualization\n",
    "    itr = 0\n",
    "    while itr<max_itr:\n",
    "        vis.append((itr,cost(t0,t1,t2,t3,X,y)))\n",
    "        itr += 1\n",
    "        grad_0 = reduce(lambda v,w: v+w, [h(t0,t1,t2,t3,X[i])-y[i] for i in range(len(x))])/len(x)\n",
    "        grad_1 = reduce(lambda v,w: v+w, [(h(t0,t1,t2,t3,X[i])-y[i])*X[i][1] for i in range(len(X))])/len(X)\n",
    "        grad_2 = reduce(lambda v,w: v+w, [(h(t0,t1,t2,t3,X[i])-y[i])*X[i][2] for i in range(len(X))])/len(X)\n",
    "        grad_3 = reduce(lambda v,w: v+w, [(h(t0,t1,t2,t3,X[i])-y[i])*X[i][3] for i in range(len(X))])/len(X)\n",
    "        t0 = t0 - a*grad_0\n",
    "        t1 = t1 - a*grad_1\n",
    "        t2 = t2 - a*grad_2\n",
    "        t3 = t3 - a*grad_3\n",
    "    return t0,t1,t2,t3,vis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 標準化なしの場合"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "d = load_boston()\n",
    "y = d.target\n",
    "X = d.data[:,[0,1,5]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t0: -26.9715839377\n",
      "t1: -0.246330944912\n",
      "t2: 0.0493986650856\n",
      "t3: 7.92936571896\n"
     ]
    }
   ],
   "source": [
    "# 正規方程式を解いて解析的に求めた解\n",
    "X = np.hstack([np.ones((len(x), 1), float),X])\n",
    "t0,t1,t2,t3 = np.dot(np.dot(np.linalg.inv(np.dot(X.T,X)),X.T),y)\n",
    "print('t0:',t0)\n",
    "print('t1:',t1)\n",
    "print('t2:',t2)\n",
    "print('t3:',t3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "t0,t1,t2,t3,vis = gradientdescent(X,y,a=0.002)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t0: -0.584679927366\n",
      "t1: -0.319522037899\n",
      "t2: 0.0772205709412\n",
      "t3: 3.76725625386\n"
     ]
    }
   ],
   "source": [
    "print('t0:',t0)\n",
    "print('t1:',t1)\n",
    "print('t2:',t2)\n",
    "print('t3:',t3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmAAAAGoCAYAAADhOFmUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGCtJREFUeJzt3W+snmd9H/DvL3XSkGQJKSQOitNmqEADihSgmFE0capO\nIawSQZuUFWjZxKioAoOWNyS8ifeqBAm0SVMmFRgzKChNqShFQiWJ0lOVCWgCZGQ4C25Vh8SNnbDQ\nQAoUm/z24nlcDsbGx/Y51/3Yz+cjPfJ9rnP/uY51+Tlf/67rue/q7gAAMM4ZU3cAAGDZCGAAAIMJ\nYAAAgwlgAACDCWAAAIMJYAAAgx0zgFXVz1bVF6vqK1V1f1XdNG+/sKruqKoHq+qzVXXBmmNurKrd\nVfVAVV29mT8AAMCpptZzH7CqOqe7v1tVP5PkfyV5R5J/m+T/dff7qurdSS7s7huq6oVJbk3ysiTb\nktyV5HnthmMAAEnWOQXZ3d+db/5ski1JOsm1SXbO23cmed18+7VJbuvug929J8nuJNs3qsMAAKe6\ndQWwqjqjqr6SZF+SO7v7niRbu3t/knT3viQXz3e/NMnDaw7fO28DACDrr4A93d0vzmxKcXtVvSiz\nKtiP7bbRnQMAOB1tOZ6du/vbVbWa5Jok+6tqa3fvr6pLkjw2321vksvWHLZt3vZjqkpgAwBOGd1d\nG3WuYy7Cr6pnJznQ3U9W1TOSfDbJe5O8KskT3X3zURbhvzyzqcc7c4RF+FVlXT7rtmPHjuzYsWPq\nbnAKMFY4HsYL61VVGxrA1lMBe06SnVV1RmZTln/Y3Z+pqi8kub2q3pzkoSTXJUl376qq25PsSnIg\nyfWSFgDAjxwzgHX3/UlecoT2J5L8q6Mc8/tJfv+kewcAcBpyJ3xOCSsrK1N3gVOEscLxMF6Yyrpu\nxLopF7YGDAA4RWz0GjAVMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAA\ngMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDB\nBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQw\nAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACA\nwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMGO\nGcCqaltV3V1VX6uq+6vqP83bb6qqR6rqy/PXNWuOubGqdlfVA1V19Wb+AAAAp5rq7p++Q9UlSS7p\n7vuq6rwkX0pybZJ/l+Q73f2Bw/a/IsnHk7wsybYkdyV5Xh92oao6vAkAYCFVVbq7Nup8x6yAdfe+\n7r5vvv1UkgeSXHqoP0c45Nokt3X3we7ek2R3ku0b010AgFPfca0Bq6rLk1yV5IvzprdX1X1V9aGq\numDedmmSh9cctjc/CmwAAEtv3QFsPv34iSTvnFfCbkny3O6+Ksm+JO/fnC4CAJxetqxnp6rakln4\n+lh3fypJuvvxNbt8MMmn59t7k1y25nvb5m0/YceOHf+0vbKykpWVlXV2GwBg86yurmZ1dXXTzn/M\nRfhJUlUfTfLN7n7XmrZLunvffPv3krysu99QVS9McmuSl2c29XhnLMIHAE5hG70I/5gVsKp6ZZI3\nJrm/qr6SpJO8J8kbquqqJE8n2ZPkrUnS3buq6vYku5IcSHK9pAUA8CPrqoBtyoVVwACAU8Tw21AA\nALCxBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACA\nwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBBDAAgMEEMACAwQQwAIDBJg1g3VNe\nHQBgGipgAACDqYABAAwmgAEADGYKEgBgMBUwAIDBVMAAAAZTAQMAGEwFDABgMBUwAIDBVMAAAAZT\nAQMAGEwFDABgMBUwAIDBVMAAAAZTAQMAGEwFDABgMBUwAIDBVMAAAAZTAQMAGEwAAwAYzBQkAMBg\nKmAAAIOpgAEADKYCBgAwmAoYAMBgKmAAAIOpgAEADKYCBgAwmAoYAMBgKmAAAIOpgAEADKYCBgAw\nmAoYAMBgxwxgVbWtqu6uqq9V1f1V9Y55+4VVdUdVPVhVn62qC9Ycc2NV7a6qB6rq6qOdWwUMAFhG\n66mAHUzyru5+UZJXJHlbVf1SkhuS3NXdL0hyd5Ibk6SqXpjkuiRXJHlNkluqqo50YgEMAFhGxwxg\n3b2vu++bbz+V5IEk25Jcm2TnfLedSV43335tktu6+2B370myO8n2De43AMAp67jWgFXV5UmuSvKF\nJFu7e38yC2lJLp7vdmmSh9cctnfe9hNUwACAZbTuAFZV5yX5RJJ3zithh8cncQoAYB22rGenqtqS\nWfj6WHd/at68v6q2dvf+qrokyWPz9r1JLltz+LZ520943/t25NxzZ9srKytZWVk57h8AAGCjra6u\nZnV1ddPOX72OecCq+miSb3b3u9a03Zzkie6+uareneTC7r5hvgj/1iQvz2zq8c4kz+vDLlRVvX9/\n5+KLAwCw0Koq3X3EDxWeiGNWwKrqlUnemOT+qvpKZlON70lyc5Lbq+rNSR7K7JOP6e5dVXV7kl1J\nDiS5/vDwdYg1YADAMlpXBWxTLlzV+/Z1tm6d5PIAAOu20RUwjyICABjMo4gAAAZTAQMAGEwFDABg\nMBUwAIDBVMAAAAabNIA9/fSUVwcAmIYpSACAwVTAAAAGE8AAAAYzBQkAMJgKGADAYAIYAMBgAhgA\nwGACGADAYBbhAwAMpgIGADCYAAYAMJgABgAwmDVgAACDqYABAAwmgAEADCaAAQAMZg0YAMBgKmAA\nAIMJYAAAgwlgAACDCWAAAINZhA8AMJgKGADAYAIYAMBgAhgAwGDWgAEADKYCBgAwmAAGADCYAAYA\nMJgABgAwmEX4AACDqYABAAwmgAEADCaAAQAMZg0YAMBgKmAAAIMJYAAAgwlgAACDWQMGADDYpAHs\nhz+c8uoAANMQwAAABps0gB08OOXVAQCmoQIGADCYChgAwGAqYAAAg6mAAQAMpgIGADCYAAYAMJgp\nSACAwY4ZwKrqw1W1v6q+uqbtpqp6pKq+PH9ds+Z7N1bV7qp6oKqu/mnnVgEDAJbReipgH0ny6iO0\nf6C7XzJ//VmSVNUVSa5LckWS1yS5parqaCdWAQMAltExA1h3fy7Jt47wrSMFq2uT3NbdB7t7T5Ld\nSbYf7dwqYADAMjqZNWBvr6r7qupDVXXBvO3SJA+v2WfvvO2IBDAAYBmdaAC7Jclzu/uqJPuSvP9E\nTmIKEgBYRltO5KDufnzNlx9M8un59t4kl6353rZ52xF94Qs7smPHbHtlZSUrKysn0h0AgA21urqa\n1dXVTTt/dfexd6q6PMmnu/vK+deXdPe++fbvJXlZd7+hql6Y5NYkL89s6vHOJM/rI1ykqvq3f7vz\nB3+wUT8KAMDmqKp091E/WHi8jlkBq6qPJ1lJ8qyq+kaSm5L8alVdleTpJHuSvDVJuntXVd2eZFeS\nA0muP1L4OsQaMABgGa2rArYpF67qN72ps3PnJJcHAFi3ja6AeRQRAMBgAhgAwGCeBQkAMJgKGADA\nYCpgAACDqYABAAymAgYAMJgKGADAYAIYAMBgpiABAAZTAQMAGEwFDABgMBUwAIDBBDAAgMFMQQIA\nDKYCBgAwmAoYAMBgKmAAAIOpgAEADKYCBgAwmAAGADCYKUgAgMEEMACAwQQwAIDBJg1gBw5MeXUA\ngGlMGsB+8IOke8oeAACMN2kAO+MMn4QEAJbPpAHsrLNMQwIAy2fSAHbmmbNpSACAZTJ5AFMBAwCW\njSlIAIDBJq+AmYIEAJbN5AFMBQwAWDaTT0GqgAEAy0YFDABgsMkrYAIYALBsJq+AmYIEAJbN5AFM\nBQwAWDamIAEABpu8AmYKEgBYNpMHMBUwAGDZTD4FqQIGACwbFTAAgMEmr4AJYADAspm8AmYKEgBY\nNpMHMBUwAGDZmIIEABhs8gqYKUgAYNlMHsBUwACAZTP5FKQKGACwbFTAAAAGm7wCJoABAMtm8gqY\nKUgAYNlMHsBUwACAZWMKEgBgsMkrYKYgAYBlM3kAUwEDAJbNMQNYVX24qvZX1VfXtF1YVXdU1YNV\n9dmqumDN926sqt1V9UBVXf3Tzu0+YADAMlpPBewjSV59WNsNSe7q7hckuTvJjUlSVS9Mcl2SK5K8\nJsktVVVHO7EKGACwjI4ZwLr7c0m+dVjztUl2zrd3JnndfPu1SW7r7oPdvSfJ7iTbj3Zui/ABgGV0\nomvALu7u/UnS3fuSXDxvvzTJw2v22ztvOyKL8AGAZbRRi/D7RA4yBQkALKMtJ3jc/qra2t37q+qS\nJI/N2/cmuWzNftvmbUf0sY/tyN/8TbJjR7KyspKVlZUT7A4AwMZZXV3N6urqpp2/uo9dvKqqy5N8\nuruvnH99c5Inuvvmqnp3kgu7+4b5Ivxbk7w8s6nHO5M8r49wkarqe+7p/M7vJPfeu2E/DwDAhquq\ndPdRP1h4vI5ZAauqjydZSfKsqvpGkpuSvDfJH1XVm5M8lNknH9Pdu6rq9iS7khxIcv2RwtchZ5+d\nfP/7J/0zAACcUtZVAduUC1f17t2da65J/vqvJ+kCAMC6bHQFbNI74auAAQDLSAADABhMAAMAGEwA\nAwAYbNIAtmVLUpUcPDhlLwAAxpo0gCWzKtj3vjd1LwAAxlmIAGYaEgBYJgIYAMBgAhgAwGACGADA\nYAIYAMBgAhgAwGACGADAYAIYAMBgAhgAwGACGADAYAIYAMBgAhgAwGACGADAYAIYAMBgAhgAwGAC\nGADAYAIYAMBgCxHAvve9qXsBADDO5AHs3HOT73536l4AAIyzEAHsqaem7gUAwDiTB7Dzzkv+4R+m\n7gUAwDiTB7BzzxXAAIDlIoABAAy2EAHMGjAAYJksRABTAQMAlsnkAcwifABg2UwewM4+O/nBD5If\n/nDqngAAjDF5AKtKzjlHFQwAWB6TB7DEOjAAYLksRACzDgwAWCYLEcDcigIAWCYLE8BUwACAZSGA\nAQAMthAB7LzzTEECAMtjIQKYChgAsEwEMACAwRYigJ13XvKd70zdCwCAMRYigJ1/fvLtb0/dCwCA\nMRYigD3zmcmTT07dCwCAMRYigF1wQfL3fz91LwAAxliYAKYCBgAsi4UIYKYgAYBlshABzBQkALBM\nFiKAqYABAMtkIQKYChgAsEyqu6e5cFUfuvaBA8nZZycHDyZVk3QHAOCoqirdvWEpZSEqYGeemTzj\nGR7IDQAsh4UIYIlpSABgeSxMALMQHwBYFgsTwFTAAIBlseVkDq6qPUmeTPJ0kgPdvb2qLkzyh0l+\nIcmeJNd19zFrW898pgAGACyHk62APZ1kpbtf3N3b5203JLmru1+Q5O4kN67nRM9+dvL44yfZGwCA\nU8DJBrA6wjmuTbJzvr0zyevWc6KLLkq++c2T7A0AwCngZANYJ7mzqu6pqrfM27Z29/4k6e59SS5e\nz4kuukgFDABYDie1BizJK7v70aq6KMkdVfVgZqFsrXXd6fWii5IHHzzJ3gAAnAJOKoB196PzPx+v\nqj9Jsj3J/qra2t37q+qSJI8d7fgdO3b80/aZZ67k8cdXTqY7AAAbYnV1Naurq5t2/hN+FFFVnZPk\njO5+qqrOTXJHkv+c5NeSPNHdN1fVu5Nc2N03HOH4Xnvtz38++d3fTb74xRPqDgDAptnoRxGdTAVs\na5JPVlXPz3Nrd99RVfcmub2q3pzkoSTXredk1oABAMtiIR7Gnczugr9tW/Kd70zSHQCAozotH8ad\nJOefn/zgB8n3vz91TwAANtfCBLAqN2MFAJbDwgSwJLn00uTv/m7qXgAAbK6FCmDbtiUPPzx1LwAA\nNtfCBbBHHpm6FwAAm2uhAthllwlgAMDpb6ECmAoYALAMBDAAgMEEMACAwRbmTvjJ7Eas558/uxv+\nmWdO0i0AgJ9w2t4JP0nOOmtWBfvbv526JwAAm2ehAliSPP/5yde/PnUvAAA2jwAGADDYQgawBx+c\nuhcAAJtHAAMAGGzhAtiVVyZf/Woy0YczAQA23cIFsK1bk3PO8UlIAOD0tXABLEle+tLkS1+auhcA\nAJtjIQPYL/9ycu+9U/cCAGBzLGQAe8Urkr/8y6l7AQCwORbqUUSHfP/7yUUXJQ8/nDzzmYM7BgBw\nmNP6UUSHnH128iu/kvz5n0/dEwCAjbeQASxJfv3Xk09+cupeAABsvIWcgkySxx6b3ZT1kUeS884b\n2DEAgMMsxRRkklx8cfKqVyW33jp1TwAANtbCVsCS5POfT17/+tnDuc86a1DHAAAOszQVsGR2O4or\nr0ze+96pewIAsHEWugKWzNaAvfSlyUc/mrz61QM6BgBwmKWqgCXJtm3JH/9x8lu/lXzoQx7SDQCc\n+ha+AnbIAw8kv/mbSVXylrfMqmGXXz77GgBgM210BeyUCWBJ8vTTyWc+k3z848lf/EXyve8lP//z\nyXOekzzrWbMbuB56nXXWLJwd7+vH+3ikfmvz97RxbYvSj0VvW5R+LHrbovRj0dsWpR+nYtui9ONk\n206k/ed+bokD2OEeeyzZuzd59NHkiSdmjzA69PrHf5xNVx7Pa60jdU2bvyd/d+PbFqUfi962KP1Y\n9LZF6cep2LYo/TjZthNtf/JJAQwAYKilW4QPAHC6EcAAAAYTwAAABhPAAAAGE8AAAAYTwAAABhPA\nAAAGE8AAAAYTwAAABhPAAAAGE8AAAAYTwAAABhPAAAAGE8AAAAYTwAAABhPAAAAGE8AAAAYTwAAA\nBhPAAAAGE8AAAAYTwAAABhPAAAAGE8AAAAbbtABWVddU1f+tqq9X1bs36zoAAKeaTQlgVXVGkv+W\n5NVJXpTk9VX1S5txLZbD6urq1F3gFGGscDyMF6ayWRWw7Ul2d/dD3X0gyW1Jrt2ka7EEvEmyXsYK\nx8N4YSqbFcAuTfLwmq8fmbcBACw9i/ABAAar7t74k1b9iyQ7uvua+dc3JOnuvnnNPht/YQCATdLd\ntVHn2qwA9jNJHkzya0keTfJXSV7f3Q9s+MUAAE4xWzbjpN39w6p6e5I7Mpvm/LDwBQAwsykVMAAA\njm6SRfhu0srhqmpPVf3vqvpKVf3VvO3Cqrqjqh6sqs9W1QVr9r+xqnZX1QNVdfV0PWeEqvpwVe2v\nqq+uaTvu8VFVL6mqr87fe/7L6J+DzXeUsXJTVT1SVV+ev65Z8z1jZUlV1baquruqvlZV91fVO+bt\nQ95bhgcwN2nlKJ5OstLdL+7u7fO2G5Lc1d0vSHJ3khuTpKpemOS6JFckeU2SW6pqwxZGspA+ktl7\nxlonMj7+e5L/2N3PT/L8qjr8nJz6jjRWkuQD3f2S+evPkqSqroixsswOJnlXd78oySuSvG2eR4a8\nt0xRAXOTVo6k8pPj8dokO+fbO5O8br792iS3dffB7t6TZHdm44rTVHd/Lsm3Dms+rvFRVZck+Wfd\nfc98v4+uOYbTxFHGSjJ7jznctTFWllZ37+vu++bbTyV5IMm2DHpvmSKAuUkrR9JJ7qyqe6rqLfO2\nrd29P5n9Q0ly8bz98DG0N8bQMrr4OMfHpZm93xzivWe5vL2q7quqD62ZUjJWSJJU1eVJrkryhRz/\n754TGi9uxMqieGV3vyTJv86sDPwvMwtla/nECD+N8cHR3JLkud19VZJ9Sd4/cX9YIFV1XpJPJHnn\nvBI25HfPFAFsb5KfX/P1tnkbS6y7H53/+XiSP8lsSnF/VW1NknmJ97H57nuTXLbmcGNoOR3v+DBu\nllR3P94/+sj/B/OjJQvGypKrqi2Zha+Pdfen5s1D3lumCGD3JPnFqvqFqjoryW8k+dMJ+sGCqKpz\n5v8DSVWdm+TqJPdnNi7+w3y3f5/k0D+OP03yG1V1VlX98yS/mNnNfjm9VX58Hc9xjY/5VMKTVbV9\nvnD2TWuO4fTyY2Nl/kv0kH+T5P/Mt40V/keSXd39X9e0DXlv2ZQbsf40btLKEWxN8smaPZ5qS5Jb\nu/uOqro3ye1V9eYkD2X26ZN0966quj3JriQHkly/5n+3nIaq6uNJVpI8q6q+keSmJO9N8kfHOT7e\nluR/Jjk7yWcOfRqO08dRxsqvVtVVmX3aek+StybGyrKrqlcmeWOS+6vqK5lNNb4nyc05/t89xz1e\n3IgVAGAwi/ABAAYTwAAABhPAAAAGE8AAAAYTwAAABhPAAAAGE8AAAAYTwAAABvv/W0qViBgwugQA\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112c2f048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1,1,figsize=(10,7))\n",
    "chart = ax.plot([v[0] for v in vis],[v[1] for v in vis])\n",
    "# 横軸がIteration回数、縦軸がコスト関数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "解析的に求めた解とかなり遠いし、学習が遅々として進んでいない。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 標準化した場合"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "d = load_boston()\n",
    "y = d.target\n",
    "X = d.data[:,[0,1,5]]\n",
    "X = stats.zscore(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t0: 22.5328063241\n",
      "t1: -2.11673199573\n",
      "t2: 1.15095904462\n",
      "t3: 5.56580032253\n"
     ]
    }
   ],
   "source": [
    "# 正規方程式を解いて解析的に求めた解\n",
    "X = np.hstack([np.ones((len(x), 1), float),X])\n",
    "t0,t1,t2,t3 = np.dot(np.dot(np.linalg.inv(np.dot(X.T,X)),X.T),y)\n",
    "print('t0:',t0)\n",
    "print('t1:',t1)\n",
    "print('t2:',t2)\n",
    "print('t3:',t3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "t0,t1,t2,t3,vis = gradientdescent(X,y,a=0.002)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t0: 22.1217532897\n",
      "t1: -2.13738566899\n",
      "t2: 1.27245685301\n",
      "t3: 5.40764852412\n"
     ]
    }
   ],
   "source": [
    "print('t0:',t0)\n",
    "print('t1:',t1)\n",
    "print('t2:',t2)\n",
    "print('t3:',t3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmAAAAGoCAYAAADhOFmUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu01XWd//Hnm6sKgiAIihhopqAokuBdzpgpWallmres\n0dZqRh2dnGnSZpX81jRroqx+tRrzlzqKpZHZ5CXNe8c0x9AJA0URL4giHFG8hHgDP78/vps44IFz\nYe/92ZfnY63v2t/zPd999hvXl8PLz+f9/XwjpYQkSZKqp1fuAiRJkpqNAUySJKnKDGCSJElVZgCT\nJEmqMgOYJElSlRnAJEmSqqzTABYR/SPijxExJyLmRcSFpeNDIuL2iFgQEbdFxOB277kgIhZGxGMR\ncUQl/wCSJEn1JrqyDlhEbJVSWhURvYE/AOcAxwEvp5S+HRFfBYaklM6PiPHA1cBkYEfgTmDX5IJj\nkiRJQBenIFNKq0q7/YE+QAKOAWaWjs8Eji3tHw3MSimtTiktAhYCU8pVsCRJUr3rUgCLiF4RMQdY\nBtyRUnoQGJFSagNIKS0DtiudPgp4rt3bl5SOSZIkia6PgL2XUtqHYkpxSkTsQTEKtt5p5S5OkiSp\nEfXpzskppdcjohWYBrRFxIiUUltEjAReLJ22BBjd7m07lo6tJyIMbJIkqW6klKJcP6vTJvyIGAa8\nm1J6LSK2BG4DvgVMBVaklGZspAl/P4qpxzvooAk/IuzLV5dNnz6d6dOn5y5DdcBrRd3h9aKuioiy\nBrCujIBtD8yMiF4UU5a/SCndEhEPANdGxOnAs8AJACml+RFxLTAfeBc406QlSZK0TqcBLKU0D5jU\nwfEVwOEbec9/AP+x2dVJkiQ1IFfCV11oaWnJXYLqhNeKusPrRbl0aSHWinywPWCSJKlOlLsHLOsI\n2Cuv5Px0SZKkPLIGsDlzcn66JElSHgYwSZKkKjOASZIkVZkBTJIkqcqyBrBnnoFVq3JWIEmSVH1Z\nA9juu8PcuTkrkCRJqr6sAWzSJKchJUlS88kawPbZB/70p5wVSJIkVV/2AOYImCRJajZZH0W0cmVi\n+HB47TXo2zdLGZIkSZ1qqEcRDRgAH/gAzJ+fswpJkqTqyhrAwGlISZLUfLIHMO+ElCRJzSZ7APNO\nSEmS1GyyNuGnlFixAsaMgVdfhV7Z46AkSdL7NVQTPsDQocX21FO5K5EkSaqO7AEMbMSXJEnNpWYC\nmH1gkiSpWdREAJs0yQAmSZKaR/YmfIClS2HPPeGllyDK1t4mSZJUHg3XhA+w/faw5ZawaFHuSiRJ\nkiqvJgIYwL77woMP5q5CkiSp8moqgD30UO4qJEmSKq9mAtjkyQYwSZLUHGqiCR+KBvxddoFXXnFF\nfEmSVFsasgkfYNiwYkX8hQtzVyJJklRZNRPAwGlISZLUHGoqgNmIL0mSmoEBTJIkqcpqpgkf4NVX\nYfToohG/T58sZUmSJL1PwzbhA2yzDeywAzz+eO5KJEmSKqemAhg4DSlJkhqfAUySJKnKai6ATZ7s\nMyElSVJjq6kmfICVK2HEiKIRv1+/DIVJkiRtoKGb8AEGDoSxY+HRR3NXIkmSVBk1F8DAPjBJktTY\najKA2QcmSZIaWU0GsClT4I9/zF2FJElSZdRcEz7AO+/AkCHQ1lb0hEmSJOXU8E34UNz9uPfe9oFJ\nkqTGVJMBDGC//ZyGlCRJjalmA9j++8MDD+SuQpIkqfxqNoDtt18RwDK1qEmSJFVMzQawD3wA3nsP\nnnsudyWSJEnlVbMBLKKYhrQPTJIkNZqaDWCwbhpSkiSpkdR0AHMETJIkNaKaXIh1rddfhx12gFde\ngb59q1SYJEnSBppiIda1Bg2CsWNh7tzclUiSJJVPTQcwcEFWSZLUeOoigNmIL0mSGknNBzAb8SVJ\nUqPpNIBFxI4RcXdEPBoR8yLiH0rHL4yI5yPiT6VtWrv3XBARCyPisYg4YnMKHD8eli6FFSs256dI\nkiTVjj5dOGc1cF5K6eGIGAj8b0TcUfre91JK32t/ckSMA04AxgE7AndGxK6d3vK4Eb17w777wuzZ\nMG1a5+dLkiTVuk5HwFJKy1JKD5f2VwKPAaNK3+7odsxjgFkppdUppUXAQmDK5hRpH5gkSWok3eoB\ni4gxwERgbVfW2RHxcERcFhGDS8dGAe2f4LiEdYGtRw48EO6/f3N+giRJUu3ocgArTT9eB5xbGgm7\nGNg5pTQRWAZ8tzIlwgEHFI34a9ZU6hMkSZKqpys9YEREH4rw9dOU0g0AKaXl7U65FLiptL8EGN3u\nezuWjr3P9OnT/7rf0tJCS0tLh58/bBhsvz3MmwcTJ3alYkmSpJ5rbW2ltbW1Yj+/S48iioirgJdS\nSue1OzYypbSstP9lYHJK6eSIGA9cDexHMfV4B/C+JvyuPIqovTPOgEmT4KyzuvwWSZKksqj6o4gi\n4iDgFOCwiJjTbsmJb0fE3Ih4GJgKfBkgpTQfuBaYD9wCnNnTOyDbO+gg+8AkSVJjqOmHcbf3+OPF\nMhSLFlWuJkmSpI401cO429ttN1i5EpZ02E0mSZJUP+omgEUUy1H84Q+5K5EkSdo8dRPAoOgDM4BJ\nkqR6V1cBzBEwSZLUCOqmCR/gzTeLNcFefBEGDKhQYZIkSRto2iZ8gC23hL32Kh7MLUmSVK/qKoCB\nfWCSJKn+GcAkSZKqrK56wADa2oo1wVasgF51Fx8lSVI9auoeMIARI4pG/Pnzc1ciSZLUM3UXwKCY\nhrzvvtxVSJIk9UxdBrBDDoF7781dhSRJUs/UZQA79FC45x7I1L4mSZK0WeoygO26K6xZA4sW5a5E\nkiSp++oygEWsGwWTJEmqN3UZwACmToXf/z53FZIkSd1XtwHs0EMNYJIkqT7VbQAbPx5efRWWLMld\niSRJUvfUbQDr1atYjsJRMEmSVG/qNoCB05CSJKk+GcAkSZKqrO4ext3emjWw7bawcCEMH16mwiRJ\nkjbQ9A/jbq93bzjwQB9LJEmS6ktdBzBwGlKSJNWfug9gU6e6Ir4kSaovdd0DBvDOO0Uf2HPPwTbb\nlKEwSZKkDdgDtoF+/WC//eC++3JXIkmS1DV1H8AAWlrgd7/LXYUkSVLXNEQAO+wwuPvu3FVIkiR1\nTd33gAG8+27RB/bMM8WrJElSOdkD1oG+feHgg6G1NXclkiRJnWuIAAZOQ0qSpPphAJMkSaqyhglg\nEyfCiy/CCy/krkSSJGnTGiaA9erlchSSJKk+NEwAA6chJUlSfWi4AHbXXZBpZQ1JkqQuaagAtvvu\n8PbbxXpgkiRJtaqhAliE05CSJKn2NVQAAwOYJEmqfQ3xKKL2nnkGDjgAli4tRsQkSZI2l48i6sTY\nsbDlljB/fu5KJEmSOtZwAQzg8MPhzjtzVyFJktSxhgxgRxwBt9+euwpJkqSONVwPGMCKFTBmDCxf\nDv37V+QjJElSE7EHrAuGDoVx4+D++3NXIkmS9H4NGcDAaUhJklS7DGCSJElV1pA9YADvvgvDhsGT\nT8Lw4RX7GEmS1ATsAeuivn2hpcXlKCRJUu1p2AAGcOSRTkNKkqTa07BTkFBMP06dCs8/72OJJElS\nzzkF2Q277FKsA+ZjiSRJUi1p6AAW4d2QkiSp9jR0AAMDmCRJqj0N3QMG8OqrsNNO8OKLsMUWFf84\nSZLUgKreAxYRO0bE3RHxaETMi4hzSseHRMTtEbEgIm6LiMHt3nNBRCyMiMci4ohyFdsT22wDEybA\n73+fswpJkqR1ujIFuRo4L6W0B3AAcFZE7A6cD9yZUtoNuBu4ACAixgMnAOOAjwEXR+S9B/Goo+C3\nv81ZgSRJ0jqdBrCU0rKU0sOl/ZXAY8COwDHAzNJpM4FjS/tHA7NSSqtTSouAhcCUMtfdLUcdBbfc\nkrMCSZKkdbrVhB8RY4CJwAPAiJRSGxQhDdiudNoo4Ll2b1tSOpbNxInwl78U64JJkiTl1uUAFhED\ngeuAc0sjYRt20Ofp5u+CCEfBJElS7ejTlZMiog9F+PppSumG0uG2iBiRUmqLiJHAi6XjS4DR7d6+\nY+nY+0yfPv2v+y0tLbS0tHSr+O446ij4yU/gnHMq9hGSJKlBtLa20traWrGf36VlKCLiKuCllNJ5\n7Y7NAFaklGZExFeBISml80tN+FcD+1FMPd4B7LrhmhPVWoZirddfh1GjYNkyGDCgah8rSZIaQI5l\nKA4CTgEOi4g5EfGniJgGzAA+GhELgI8A3wJIKc0HrgXmA7cAZ1Y1aW3EoEEweTLcfXfuSiRJUrNr\n+IVY27voInjqKfjxj6v6sZIkqc6VewSsSz1gjeLjH4dp0yClojFfkiQph4Z/FmR7u+8OvXrB/Pm5\nK5EkSc2sqQLY2uUobr45dyWSJKmZNVUAA9cDkyRJ+TVVEz7AqlUwciQsXlw8qFuSJKkzVV+GotFs\ntRVMnerDuSVJUj5NF8AAjj4abrih8/MkSZIqoemmIKFYDX/cOGhrg379spQgSZLqiFOQZTByZLEk\nxT335K5EkiQ1o6YMYADHHOM0pCRJyqMppyChWIx12jR49llXxZckSZvmFGSZjBsH/fvDnDm5K5Ek\nSc2maQNYRDENeeONuSuRJEnNpmkDGNgHJkmS8mjqAHbAAfD880UfmCRJUrU0dQDr0wc+/nGnISVJ\nUnU1dQAD+8AkSVL1Ne0yFGu98QbssAMsWgRDhuSuRpIk1SKXoSizAQPgsMMcBZMkSdXT9AEM4Ljj\n4Fe/yl2FJElqFk0/BQnw2mswenRxR+SgQbmrkSRJtcYpyAoYPBgOOQRuvjl3JZIkqRkYwEqchpQk\nSdXiFGTJyy/DzjvDCy8UjfmSJElrOQVZIdtuC1OmwK235q5EkiQ1OgNYO8cdB9ddl7sKSZLU6JyC\nbKetDXbbDZYtgy22yF2NJEmqFU5BVtCIEbD33nDHHbkrkSRJjcwAtoHPfMZpSEmSVFlOQW5gyRKY\nMAGWLoX+/XNXI0mSaoFTkBU2ahTsuSfcdlvuSiRJUqMygHXgpJNg1qzcVUiSpEblFGQHli+HXXct\npiNdlFWSJDkFWQXDh8P++8NvfpO7EkmS1IgMYBtx4onw85/nrkKSJDUipyA34rXXYKed4NlnYZtt\nclcjSZJycgqySgYPhsMOg+uvz12JJElqNAawTXAaUpIkVYJTkJvwxhvFumBPPAHbbZe7GkmSlItT\nkFU0YAAcdZSPJpIkSeVlAOvESSc5DSlJksrLKchOvPNOMQ05ezaMHZu7GkmSlINTkFXWr1/RjP+z\nn+WuRJIkNQoDWBd87nNw1VVQBwN2kiSpDhjAumDyZOjdGx54IHclkiSpERjAuiACTjsNfvrT3JVI\nkqRGYBN+Fy1eDJMmwZIl0L9/7mokSVI12YSfyU47wYQJcPPNuSuRJEn1zgDWDaedVjTjS5IkbQ6n\nILvh9deLkbAnn4Rhw3JXI0mSqsUpyIwGDSoeTfSLX+SuRJIk1TMDWDd9/vNw5ZW5q5AkSfXMANZN\nhx8ObW0wd27uSiRJUr0ygHVT797whS/A5ZfnrkSSJNUrm/B7YNEi2HdfeP552GKL3NVIkqRKswm/\nBowZA/vsA9dfn7sSSZJUjzoNYBFxeUS0RcTcdscujIjnI+JPpW1au+9dEBELI+KxiDiiUoXndsYZ\nTkNKkqSe6XQKMiIOBlYCV6WU9ioduxD4S0rpexucOw64BpgM7AjcCeza0VxjPU9BArz1FoweDbNn\nw9ixuauRJEmVVPUpyJTSfcArHdXSwbFjgFkppdUppUXAQmDKZlVYo7bYAk4+Ga64InclkiSp3mxO\nD9jZEfFwRFwWEYNLx0YBz7U7Z0npWEM644xiTbA1a3JXIkmS6klPA9jFwM4ppYnAMuC75Supfuy1\nF4wcCXfckbsSSZJUT/r05E0ppeXtvrwUuKm0vwQY3e57O5aOdWj69Ol/3W9paaGlpaUn5WT1xS/C\nT34C06Z1fq4kSaoPra2ttLa2Vuznd2kdsIgYA9yUUppQ+npkSmlZaf/LwOSU0skRMR64GtiPYurx\nDhq0CX+tlSuLB3TPmwejGnayVZKk5lb1JvyIuAa4H/hQRCyOiL8Fvh0RcyPiYWAq8GWAlNJ84Fpg\nPnALcGZDpKxNGDgQTjoJLr00dyWSJKleuBJ+GTzyCBx5ZLFCft++uauRJEnl5kr4NWjPPWGXXeDG\nG3NXIkmS6oEBrEzOPBMuvjh3FZIkqR44BVkm77xTNOO3tsLuu+euRpIklZNTkDWqX79iYdZLLsld\niSRJqnWOgJXRs8/CpEmweDEMGJC7GkmSVC6OgNWwD3wADjwQZs3KXYkkSaplBrAyO/NM+M//hAYb\n3JMkSWVkACuzI48sVse/777clUiSpFplACuzXr3g3HPh+9/PXYkkSapVNuFXwMqVMGYMzJ4NO++c\nuxpJkrS5bMKvAwMHwumnw49+lLsSSZJUixwBq5DFi2HixOL5kIMG5a5GkiRtDkfA6sROO8Hhh8OV\nV+auRJIk1RpHwCro/vvhtNNgwQLo3Tt3NZIkqaccAasjBxwAQ4fCb36TuxJJklRLDGAVFAH/+I8u\nSSFJktZnAKuwz3wGnnoKHnoodyWSJKlWGMAqrF8/OO88mDEjdyWSJKlW2IRfBStXwtixRVP+rrvm\nrkaSJHWXTfh1aOBA+Pu/h4suyl2JJEmqBY6AVcny5fChD8H8+bD99rmrkSRJ3eEIWJ0aPhxOOQV+\n8IPclUiSpNwcAauiZ56BffeFp5+GwYNzVyNJkrrKEbA6NnYsTJsGl1ySuxJJkpSTI2BVNnduEcKe\nfhq22CJ3NZIkqSscAatze+0FkybBf/1X7kokSVIujoBlMHt2sUL+woXQv3/uaiRJUmccAWsAU6bA\nHnvAFVfkrkSSJOXgCFgmDzwAn/1sMQrWr1/uaiRJ0qY4AtYg9t8fdt8drrwydyWSJKnaHAHL6P77\n4eST4YknHAWTJKmWOQLWQA48sHg491VX5a5EkiRVkyNgmd13H5x2GixYAH375q5GkiR1xBGwBnPw\nwbDzzjBzZu5KJElStTgCVgMeeACOP764I9LV8SVJqj2OgDWg/fcvVse/+OLclUiSpGpwBKxGPPII\nfOQjxSjYoEG5q5EkSe05Atag9twTjjwSvve93JVIkqRKcwSshjzzDOy7Lzz+OAwfnrsaSZK0VrlH\nwAxgNebss4vlKL7//dyVSJKktQxgDW7ZsuJB3XPmwE475a5GkiSBPWANb+RI+Lu/g298I3clkiSp\nUhwBq0Gvvw4f+hD89rewzz65q5EkSY6ANYFBg2D6dPinfwIzqiRJjccAVqO++EVoa4ObbspdiSRJ\nKjenIGvYrbfCuecWi7T6oG5JkvJxCrKJTJsGY8bAJZfkrkSSJJWTI2A1bt48OPzwYnHWIUNyVyNJ\nUnNyHbAm9KUvwcCB8N3v5q5EkqTmZABrQm1txbMi77kHxo/PXY0kSc3HHrAmNGIEfP3rcM45Lksh\nSVIjMIDViTPPhOXL4brrclciSZI2l1OQdeTee+GUU2D+/KInTJIkVYdTkE3skEPg0EPh3/89dyWS\nJGlzOAJWZ5YuhQkT4A9/gN12y12NJEnNwRGwJrf99vC1r9mQL0lSPTOA1aF/+IdiJGzWrNyVSJKk\nnug0gEXE5RHRFhFz2x0bEhG3R8SCiLgtIga3+94FEbEwIh6LiCMqVXgz69sXLr0UzjsPXn45dzWS\nJKm7ujICdgVw5AbHzgfuTCntBtwNXAAQEeOBE4BxwMeAiyOibPOlWme//eCEE+ArX8ldiSRJ6q5O\nA1hK6T7glQ0OHwPMLO3PBI4t7R8NzEoprU4pLQIWAlPKU6o29M1vwl13wd13565EkiR1R097wLZL\nKbUBpJSWAduVjo8Cnmt33pLSMVXA1lvDj35UPCvyzTdzVyNJkrqqXE343o+XySc/CfvsA//2b7kr\nkSRJXdWnh+9ri4gRKaW2iBgJvFg6vgQY3e68HUvHOjR9+vS/7re0tNDS0tLDcprbD38Ie+0Fxx9f\nhDFJkrR5WltbaW1trdjP79JCrBExBrgppTSh9PUMYEVKaUZEfBUYklI6v9SEfzWwH8XU4x3Arh2t\nuOpCrOX105/Cd74DDz4I/fvnrkaSpMZS7oVYOw1gEXEN0AJsC7QBFwLXA7+kGO16FjghpfRq6fwL\ngDOAd4FzU0q3b+TnGsDKKCX41Kdgjz18VJEkSeVW9QBWKQaw8mtrg733hhtvhCneeypJUtn4KCJt\n1IgRRT/Y5z/vXZGSJNUyR8Aa0Gc/C6NHw0UX5a5EkqTG4BSkOvXSS8VdkbNmwaGH5q5GkqT65xSk\nOjVsGFx2GXzuc/DKhs8wkCRJ2TkC1sDOPRdeeAGuvRZ8IqckST3nCJi6bMYMWLAArrgidyWSJKk9\nR8Aa3Pz5MHUq3Hcf7LZb7mokSapPjoCpW8aPL54TefLJ8M47uauRJEngCFhTWLtK/gc/6NIUkiT1\nhCNg6rYIuPxyuO46uOGG3NVIkiRHwJrI7NnwiU/A//wP7LJL7mokSaofjoCpx6ZMgQsvhOOO81FF\nkiTl5AhYk0mpaMgfMKBYrFWSJHXOETBtlgi49FK4/37XB5MkKRdHwJrU2vXBbrsNJk3KXY0kSbXN\nETCVxfjxcMklxfIUbW25q5EkqbkYwJrYccfB3/4tfPrT8PbbuauRJKl5OAXZ5N57D44/HgYPLtYK\n86HdkiS9n1OQKqtevWDmTPjf/4Uf/jB3NZIkNYc+uQtQfgMHFivkH3AAjBsHRxyRuyJJkhqbI2AC\nYMwYuPZaOPVUmDcvdzWSJDU2A5j+6pBD4Ac/gI9/HJYsyV2NJEmNyylIreekk2DxYjjqKLj3Xhg0\nKHdFkiQ1Hu+C1PukBGeeCU89BTffDH375q5IkqS8yn0XpAFMHVq9ulikdfhwl6eQJMllKFQVffrA\nrFnwyCPwta/lrkaSpMZiANNGDRgAt9wCN94IM2bkrkaSpMZhE742adgwuP324g7JbbaBL30pd0WS\nJNU/A5g6NWoU3HEHHHpocVfkSSflrkiSpPpmAFOX7LIL3HorHH44bL01fOITuSuSJKl+2QOmLpsw\nAW66CU4/vegNkyRJPWMAU7dMmVI05X/hC4YwSZJ6ygCmbtt/f0OYJEmbwwCmHjGESZLUcwYw9Vj7\nEHbjjbmrkSSpfngXpDbL/vsXz4v85Cfh9dfh1FNzVyRJUu0zgGmzTZ4Md90FRx4Jr74KZ5+duyJJ\nkmqbAUxlsccecO+98NGPFiHsX//VB3hLkrQxkVLK88ERKddnq3KWLi1Gwj76UfjOd6CXXYaSpAYQ\nEaSUyja0YABT2a1YUfSE7bQTXHkl9O+fuyJJkjZPuQOY4xMqu6FD4c47YfXqYiRsxYrcFUmSVFsM\nYKqILbeEX/yiuEvywAPh6adzVyRJUu0wgKlievWCb38bzjkHDj4Y/vjH3BVJklQbDGCquDPPhJ/8\npOgL+9nPclcjSVJ+NuGrah55BI49tti+9S3o4yIokqQ64V2QqmsrVsCJJxb7s2YVDfuSJNU674JU\nXRs6tHh494QJMGUKPPpo7ookSao+A5iqrk8f+O534cILoaXFvjBJUvNxClJZzZ0Lxx8PU6fCD35Q\nLF8hSVKtcQpSDWWvveChh+AvfynWDHviidwVSZJUeQYwZbf11nDNNcVyFQcdBD//ee6KJEmqLKcg\nVVPmzCnukpw8GX70I9hmm9wVSZLkFKQa3D77FCFs8GDYe29obc1dkSRJ5ecImGrWb38LX/winHwy\nfPOb0L9/7ookSc3KETA1jY99DP78Z3jqqWJKcs6c3BVJklQeBjDVtGHD4Fe/gn/+Z5g2DS64AN58\nM3dVkiRtns0KYBGxKCL+HBFzImJ26diQiLg9IhZExG0RMbg8papZRcBppxVrhj31VNEb9vvf565K\nkqSe26wesIh4GvhwSumVdsdmAC+nlL4dEV8FhqSUzu/gvfaAqUeuvx7OPhs++cniod6DjfiSpAqr\ntR6w6OBnHAPMLO3PBI7dzM+Q1nPssfDII7BmDYwfXzzKyCwvSaon5RgBexVYA/y/lNJlEfFKSmlI\nu3NWpJSGdvBeR8C02R54AM46C7baqlg3bO+9c1ckSWpEtTYCdlBKaRJwFHBWRBwCbJiqTFmqmP33\nh9mz4dRT4Ygj4Jxz4NVXc1clSdKm9dmcN6eUlpZel0fE9cAUoC0iRqSU2iJiJPDixt4/ffr0v+63\ntLTQ0tKyOeWoSfXuDV/6EnzmM/C1r8G4cfCNbxRriPXtm7s6SVI9am1tpbWCq4H3eAoyIrYCeqWU\nVkbEAOB24P8AHwFWpJRm2ISvHObMga98BZ5/HmbMgKOPLu6klCSpp8o9Bbk5AWws8GuKKcY+wNUp\npW9FxFDgWmA08CxwQkrpfZNCBjBVUkpw663wL/8CQ4fCRRcVi7lKktQTNRPANvuDDWCqgjVr4Mor\niynJgw6C6dOLOyclSeqOWmvCl2pa795wxhnwxBOw777wN38Dp5wCCxbkrkyS1MwMYGoKAwYU05FP\nPgl77gkHH1ysrv/kk7krkyQ1IwOYmsrWWxfPk3zySdh112IZi9NOKxZ2lSSpWgxgakqDB8PXv14E\nsXHj4KMfhU98Au6911X1JUmVZxO+BLz1FsycWdwtOXx4MV159NHQy/9FkSThXZBSRa1ZA7/+dbF+\n2OuvF485+sIXYNCg3JVJknLyLkipgnr3LlbUnz0bLr8c/vAHGDMGzj4bHn88d3WSpEZhAJM6EFHc\nKfmLX8C8eTBkCLS0FM+bvOEGWL06d4WSpHrmFKTURW+/Db/8Jfz4x/DMM/D5z8Pppxd3U0qSGptT\nkFIm/fvDqacW05J33VWMgh18MEydClddBatW5a5QklQvHAGTNsM778BvfgOXXQYPPACf+hScfHIx\nXdm7d+7qJEnl4l2QUo1asgRmzYKrr4Zly+DEE4sw9uEPFz1lkqT6ZQCT6sBjj8HPfw7XXFOMhJ10\nEhx3XPEYJMOYJNUfA5hUR1KCBx8swtivfw19+8KnP11MVU6Z4kKvklQvDGBSnUoJ5syB//7vYnvt\ntSKIfeqyQyP3AAAIhUlEQVRTcMgh0K9f7golSRtjAJMaxOOPF6Ni119f7B92GBx1FHzsY7Djjrmr\nkyS1ZwCTGtDy5XDbbXDLLcXrqFFFGDvqKDjggGLqUpKUjwFManBr1hSPQrrllmJbuLBYb+yww4pt\n771d4kKSqs0AJjWZl1+Ge+6Bu+8utmXLisVf1way8eO9s1KSKs0AJjW5pUvhd79bF8heew0OPLDY\nDjoIJk+GLbfMXaUkNRYDmKT1vPAC3H9/8Yik+++HRx6BCRPWBbIDDoAddshdpSTVNwOYpE1atapY\ne2xtKPvjH4sm/g9/GPbdd902YkTuSiWpfhjAJHVLSrB4MTz00PrbwIFFEPvwh2HiRNhrLxg92n4y\nSeqIAUzSZksJnn56XRibO7fY3nyzmL7ca69imzCheHzSoEG5K5akvAxgkipm+XKYN6/Y5s4tXh99\ntJiuHD8edt99/W3YsNwVS1J1GMAkVdWaNfDUU8UDxh9/vNgWLCi+7tVr/UC2226w664wdixssUXu\nyiWpfAxgkmpCSsWI2Yah7Mkni56zYcNg551hl12K17XbLrvA8OH2mkmqLwYwSTVvzRpYsqQYOXv6\n6XXb2q/feqsIYzvtVDT+r93Wfj1qFPTvn/tPIUnrGMAk1b3XXoNnnilGyp57bt3r2u2FF2Do0PUD\n2qhRsP32xTZyZPE6ZIgjaZKqwwAmqeGtWVM8cmltIFu8uAhlS5eu25YtK+7aXBvGNgxn229fTHUO\nG1ZsgwYZ1iT1nAFMkkpWrSqC2LJl6wezta/Ll8NLLxWvb79dBLG1oax9OGt/bNtti5G1bbaBrbc2\ntEkqGMAkqQfeeqt4sHn7UPbSS+vvL19enPPKK/Dqq8V7Bg8uwtiQIeuCWfvXDY8NGlQEt623Lha7\n7dUr959cUjkYwCSpSt59t+hXe+WVdaGs/WtHx15/Hf7yl2JbtQq22mpdIGu/tQ9qG24DBhTbVlut\nvw0YUDxovU+f3P9lpOZjAJOkOvHee/DGG+uHsrXbhsc2DG5rtzfeWP/rVauKALZhMOsorG21VbEe\n2xZbFHeVbmq/K8cczVMzM4BJUhNLqehn2zCUdRTW3nijmEZ9++3itf3+hq9d+V6fPuvCWP/+xUPe\n+/UrXje239n3u/K+Pn2KrXfv9V87OtaTc3r1stdPnSt3AHMgW5LqSMS6UamhQ6v3uSkVU7Ltw9m7\n7xbbO++s/9qTY6tWbfy81auLO2M7et3U97p6znvvbTqkrd169dr01tk5lf4ZEZveunJOud9bqc9c\n+3dh7WtHxypxTjkZwCRJnYooRqX69Sv61BrJe+8VQayzkLaprbNzKv0z1qwpQnJn23vvdbzf3fdW\n430bey+s/9rRsXKfU4kJO6cgJUmSOlHuKUhbKiVJkqrMACZJklRlBjBJkqQqM4BJkiRVmQFMkiSp\nygxgkiRJVWYAkyRJqjIDmCRJUpUZwCRJkqrMACZJklRlBjBJkqQqM4BJkiRVmQFMkiSpygxgkiRJ\nVWYAkyRJqjIDmCRJUpUZwCRJkqrMACZJklRlFQtgETEtIh6PiCci4quV+hxJkqR6U5EAFhG9gB8B\nRwJ7ACdFxO6V+Cw1h9bW1twlqE54rag7vF6US6VGwKYAC1NKz6aU3gVmAcdU6LPUBPwlqa7yWlF3\neL0ol0oFsFHAc+2+fr50TJIkqenZhC9JklRlkVIq/w+N2B+YnlKaVvr6fCCllGa0O6f8HyxJklQh\nKaUo18+qVADrDSwAPgIsBWYDJ6WUHiv7h0mSJNWZPpX4oSmlNRFxNnA7xTTn5YYvSZKkQkVGwCRJ\nkrRxWZrwXaRVG4qIRRHx54iYExGzS8eGRMTtEbEgIm6LiMHtzr8gIhZGxGMRcUS+ylUNEXF5RLRF\nxNx2x7p9fUTEpIiYW/rd83+r/edQ5W3kWrkwIp6PiD+Vtmntvue10qQiYseIuDsiHo2IeRFxTul4\nVX63VD2AuUirNuI9oCWltE9KaUrp2PnAnSml3YC7gQsAImI8cAIwDvgYcHFElK0xUjXpCorfGe31\n5Pr4MXBGSulDwIciYsOfqfrX0bUC8L2U0qTSditARIzDa6WZrQbOSyntARwAnFXKI1X53ZJjBMxF\nWtWR4P3X4zHAzNL+TODY0v7RwKyU0uqU0iJgIcV1pQaVUroPeGWDw926PiJiJLB1SunB0nlXtXuP\nGsRGrhUofsds6Bi8VppWSmlZSunh0v5K4DFgR6r0uyVHAHORVnUkAXdExIMR8cXSsREppTYo/qIA\n25WOb3gNLcFrqBlt183rYxTF75u1/N3TXM6OiIcj4rJ2U0peKwIgIsYAE4EH6P6/PT26XlyIVbXi\noJTSJOAoimHgQyhCWXveMaJN8frQxlwM7JxSmggsA76buR7VkIgYCFwHnFsaCavKvz05AtgSYKd2\nX+9YOqYmllJaWnpdDlxPMaXYFhEjAEpDvC+WTl8CjG73dq+h5tTd68PrpkmllJandbf8X8q6lgWv\nlSYXEX0owtdPU0o3lA5X5XdLjgD2IPDBiPhARPQDTgRuzFCHakREbFX6PxAiYgBwBDCP4rr4Qum0\nzwNr/3LcCJwYEf0iYizwQYrFftXYgvX7eLp1fZSmEl6LiCmlxtnT2r1HjWW9a6X0j+hanwYeKe17\nrei/gPkppR+0O1aV3y0VWYh1U1ykVR0YAfw6isdT9QGuTindHhEPAddGxOnAsxR3n5BSmh8R1wLz\ngXeBM9v9360aUERcA7QA20bEYuBC4FvAL7t5fZwFXAlsAdyy9m44NY6NXCt/ExETKe62XgR8CbxW\nml1EHAScAsyLiDkUU41fA2bQ/X97un29uBCrJElSldmEL0mSVGUGMEmSpCozgEmSJFWZAUySJKnK\nDGCSJElVZgCTJEmqMgOYJElSlRnAJEmSquz/A95xohraIOyEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11404e2e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1,1,figsize=(10,7))\n",
    "chart = ax.plot([v[0] for v in vis],[v[1] for v in vis])\n",
    "# 横軸がIteration回数、縦軸がコスト関数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Learning Rateの選び方**  \n",
    "基本的に, 1, 0.1, 0.01, ... と異なるオーダーのLearning Rateを試し、Iteration毎のコスト関数の値の推移を視覚化して収束を見る。  \n",
    "例えば上のデータをそのまま使うと..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "t0,t1,t2,t3,vis = gradientdescent(X,y,a=2,max_itr=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t0: 5.5798038348e+14\n",
      "t1: 1.42448409567e+30\n",
      "t2: -1.65017024306e+30\n",
      "t3: -1.68346699379e+30\n"
     ]
    }
   ],
   "source": [
    "print('t0:',t0)\n",
    "print('t1:',t1)\n",
    "print('t2:',t2)\n",
    "print('t3:',t3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAGtCAYAAAAlE2HVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGxxJREFUeJzt3X2QpWdZJ+DfnRkGFYQVEHYJJisGiEYDpjSyGKWR1UR3\nJZYoFXBR8StlGaXcXQ1SxaaptUqswlq1wI+42Sy4y+ISCgiWmpQrLZUSNHwGSEKCwXyhCZEkQMrI\nZHLvH+ckts1M95mZ85w+3X1dVV1zznueft8nPJnUze99zv1WdwcAgPk6YbsnAACwGymyAAAGUGQB\nAAygyAIAGECRBQAwgCILAGCAhRdZVXVJVd1RVdfMOP5FVfWxqvpIVf2vdcd/pKpuqKqPV9UPj5sx\nAMDRq0X3yaqqs5J8Pskbu/v0LcaekuQPkjyvuz9bVU/o7ruq6iuSvC/JGUkqyfuTnNHd9w6ePgDA\nTBaeZHX3VUnuXn+sqp5aVX9cVVdX1Z9X1dOnH/1kktd392env3vX9PjZSa7s7nu7+54kVyY5Z0H/\nCAAAW1qWPVkXJ7mgu785yS8k+e3p8acneUZVXVVVf1FVZ0+Pn5jk1nW/f/v0GADAUti/3ROoqkcl\neU6St1RVTQ8/Yvrn/iSnJPn2JCcleXdVff3iZwkAcHS2vcjKJE27u7vPOMxntyV5b3c/mORvquqG\nJE/LJLlaWTfuKUneNXqiAACz2vJ24SzfBqyqlar6YFV9tKpmKXZq+pPu/lyST1bVD6w730Mb4t+e\n5HnTY0/IpMC6KckVSb6zqh473QT/ndNjAABLYZY9WZdmstH8sKrqsUlen+Tfd/fXJ/nBzU5WVW9K\n8hdJnl5Vt1TVy5L8UJIfr6oPVdVHk7wgSbr7iiR/X1UfS/L/kvzn7r67u+9O8l8z+YbhXyZ59XQD\nPADAUpiphUNVnZzknYdruVBVP53kX3X3fxkwPwCAHWke3y58epLHVdW7pi0YXjqHcwIA7Gjz2Pi+\nP5OmoN+R5FFJ3lNV7+nuT2wcWFWL7XwKAHAcuru2HnV48yiybktyV3ffn+T+qnp3kmcm+aIiK0kW\n3WGe+VldXc3q6up2T4NjYO12Nuu3s1m/neufOksdm1lvFz78bcDDeEeSs6pqX1V9WZJvSXLdcc0K\nAGCH2zLJmn4bcCXJ46vqliQXJTmQpLv74u6+vqquSHJNkkNJLu7uawfOGQBg6W1ZZHX3S2YY89ok\nr53LjFhaKysr2z0FjpG129ms385m/faumVo4zO1iVW1PFgCwE1TVcW18X5YHRAMA7CqKLACAARRZ\nAAADKLIAAAZQZAEADKDIAgAYQJEFADCAIgsAYABFFgDAAIosAIABFFkAAAMosgAABlBkAQAMoMgC\nABhAkQUAMIAiCwBgAEUWAMAAiiwAgAEUWQAAAyiyAAAGUGQBAAygyAIAGECRBQAwgCILAGAARRYA\nwACKLACAARRZAAADKLIAAAZQZAEADKDIAgAYQJEFADCAIgsAYABFFgCwpz344JjzKrIAgD3t5JOT\nW26Z/3kVWQDAnnbwYPKIR8z/vIosAGBPO3gwOXBg/udVZAEAe9oXviDJAgCYO0kWAMAA25ZkVdUl\nVXVHVV2zxbhvrqqDVfX985seAMA4hw4lVcm+ffM/9yxJ1qVJzt5sQFWdkOQ1Sa6Yx6QAABZhVIqV\nzFBkdfdVSe7eYtjPJrksyZ3zmBQAwCKM2o+VzGFPVlU9Ocn3dfdvJ6njnxIAwGKMTLL2z+Ecv57k\nwnXvNy20VldXH369srKSlZWVOUwBAODorW9Eura2lrW1tbmdu7p760FVJyd5Z3effpjPbnroZZIn\nJLkvyU919+WHGduzXA8AYBFuvjn5tm87/GN1qirdfcx36WZNsipHSKi6+6nrJnNpJsXYFxVYAADL\nZtQjdZIZiqyqelOSlSSPr6pbklyU5ECS7u6LNwwXUwEAO8bIje9bFlnd/ZJZT9bdP3Z80wEAWJxt\nbeEAALBbLXULBwCAnUqSBQAwgCQLAGAASRYAwACSLACAASRZAAADjGxGqsgCAPasL3zB7UIAgLmT\nZAEADGDjOwDAADa+AwAMIMkCABhAkgUAMIAkCwBgAEkWAMAAWjgAAAygGSkAwACSLACAASRZAAAD\nSLIAAAbQwgEAYAAtHAAABpBkAQAMIMkCABhAkgUAMIAkCwBgAC0cAAAG0IwUAGAASRYAwACSLACA\nASRZAAADaOEAADCAFg4AAANIsgAABpBkAQAMIMkCABhAkgUAMGfdWjgAAMzdoUNJVbJv35jzK7IA\ngD1pZIqVzFBkVdUlVXVHVV1zhM9fUlUfnv5cVVXfMP9pAgDM18hH6iSzJVmXJjl7k89vSvLt3f3M\nJL+c5PfmMTEAgJFGJ1n7txrQ3VdV1cmbfP7edW/fm+TEeUwMAGCkke0bkvnvyfqJJH8853MCAMzd\nyPYNyQxJ1qyq6nlJXpbkrM3Gra6uPvx6ZWUlKysr85oCAMDMNiZZa2trWVtbm9v5q7u3HjS5XfjO\n7j79CJ+fnuStSc7p7r/e5Dw9y/UAAEa79trkhS9Mrrvu8J9XVbq7jvX8s94urOnP4SZwUiYF1ks3\nK7AAAJbJ6D1ZW94urKo3JVlJ8viquiXJRUkOJOnuvjjJq5I8LslvVVUlOdjdZ46bMgDA8dv2PVnd\n/ZItPv/JJD85txkBACzAtjcjBQDYjZahGSkAwK4jyQIAGGCnNSMFANgRRm98V2QBAHuSJAsAYABJ\nFgDAAJIsAIABJFkAAANo4QAAMIBmpAAAA0iyAAAGkGQBAAwgyQIAGEALBwCAAbRwAAAYQJIFADCA\nJAsAYABJFgDAAJIsAIABtHAAABhAM1IAgAEkWQAAA0iyAAAGkGQBAAyghQMAwABaOAAADCDJAgAY\nQJIFADCAJAsAYABJFgDAAFo4AAAMoBkpAMAAkiwAgAEkWQAAA0iyAADmrFuRBQAwd4cOJVXJvn3j\nrqHIAgD2nNGNSBNFFgCwB41uRJoosgCAPWgpkqyquqSq7qiqazYZ85tVdWNVfaiqnjXfKQIAzNey\nJFmXJjn7SB9W1Xcn+ZruflqS85P8zpzmBgAwxOhvFiYzFFndfVWSuzcZcm6SN07H/mWSx1bVk+Yz\nPQCA+RvdiDSZz56sE5Pcuu797dNjAABLaRFJ1v6xp/9iq6urD79eWVnJysrKoqcAAOxxh0uy1tbW\nsra2NrdrVHdvPajq5CTv7O7TD/PZ7yR5V3f/wfT99Ume2913HGZsz3I9AICRrr46+emfTt73viOP\nqap0dx3rNWa9XVjTn8O5PMkPTyfz7CT3HK7AAgBYFoto4bDl7cKqelOSlSSPr6pbklyU5ECS7u6L\nu/uPqup7quoTSe5L8rKREwYAOF6LaOGwZZHV3S+ZYcwF85kOAMB4S9GMFABgt1mWZqQAALvKUjQj\nBQDYbXZKM1IAgB1FkgUAMIAkCwBgAEkWAMAAWjgAAAyghQMAwACSLACAASRZAAADSLIAAAaQZAEA\nDKCFAwDAAJqRAgAMIMkCABhAkgUAMIAkCwBgAC0cAAAG0MIBAGAASRYAwACSLACAASRZAAADSLIA\nAAbQwgEAYADNSAEABpBkAQAMIMkCABhAkgUAMIAWDgAAA2jhAAAwgCQLAGAASRYAwACSLACAOeuW\nZAEAzN2hQ8kJJ0x+RlJkAQB7yiIakSaKLABgj1lEI9JEkQUA7DGSLACAAZYqyaqqc6rq+qq6oaou\nPMznj6mqy6vqQ1X1kar60bnPFABgDhbRviGZociqqhOSvC7J2UlOS/Liqjp1w7CfSfKx7n5Wkucl\n+bWq2j/vyQIAHK9FtG9IZkuyzkxyY3ff3N0Hk7w5ybkbxnSSL5++/vIkf9/dD8xvmgAA87E0SVaS\nE5Pcuu79bdNj670uyddV1aeSfDjJy+czPQCA+VqmJGsWZyf5YHc/Ock3Jnl9VT16TucGAJibRW18\nn2Xf1O1JTlr3/inTY+u9LMmvJEl3/3VVfTLJqUnet/Fkq6urD79eWVnJysrKUU0YAOB4HKmFw9ra\nWtbW1uZ2neruzQdU7Uvy8STPT/K3Sf4qyYu7+7p1Y16f5M7ufnVVPSmT4uqZ3f2ZDefqra4HADDS\n2lpy0UXJn//55uOqKt1dx3qdLZOs7j5UVRckuTKT24uXdPd1VXX+5OO+OMkvJ/mfVXXN9Nd+cWOB\nBQCwDBbVjHSmNgvd/SdJnrHh2O+ue/23mezLAgBYakvVjBQAYLfwWB0AgAEkWQAAAyxTM1IAgF1j\npzUjBQDYESRZAAADSLIAAAaw8R0AYAAtHAAABpBkAQAMIMkCABhAkgUAMIAkCwBgAEkWAMAAmpEC\nAAygGSkAwACSLACAASRZAAAD2PgOADCAFg4AAANIsgAABpBkAQAMIMkCABhACwcAgAG0cAAAGECS\nBQAwgCQLAGAASRYAwACSLACAAbRwAAAYQDNSAIABJFkAAANIsgAA5qxbkgUAMHeHDiX79iUnLKAC\nUmQBAHvGoto3JIosAGAPWVQj0kSRBQDsIZIsAIABJFkAAANIsgAABlhU+4ZkxiKrqs6pquur6oaq\nuvAIY1aq6oNV9dGqetd8pwkAcPwW1Yg0SfZvNaCqTkjyuiTPT/KpJFdX1Tu6+/p1Yx6b5PVJvqu7\nb6+qJ4yaMADAsVq2JOvMJDd2983dfTDJm5Ocu2HMS5K8tbtvT5Luvmu+0wQAOH6LTLJmKbJOTHLr\nuve3TY+t9/Qkj6uqd1XV1VX10nlNEABgXhaZZG15u/AoznNGku9I8qgk76mq93T3JzYOXF1dffj1\nyspKVlZW5jQFAIDNbdbCYW1tLWtra3O71ixF1u1JTlr3/inTY+vdluSu7r4/yf1V9e4kz0yyaZEF\nALBIm7Vw2Bj+vPrVrz6ua81yu/DqJKdU1clVdSDJeUku3zDmHUnOqqp9VfVlSb4lyXXHNTMAgDlb\nZDPSLZOs7j5UVRckuTKTouyS7r6uqs6ffNwXd/f1VXVFkmuSHEpycXdfO3TmAABHaZHNSGfak9Xd\nf5LkGRuO/e6G969N8tr5TQ0AYL6WrYUDAMCusGwtHAAAdgVJFgDAAJIsAIABJFkAAANIsgAABpBk\nAQAMsMhmpIosAGDPWGQzUkUWALBnSLIAAAaQZAEADGDjOwDAAFo4AAAMIMkCABhAkgUAMIAkCwBg\nAEkWAMAAkiwAgAE0IwUAGEAzUgCAASRZAAADSLIAAAaw8R0AYAAtHAAABpBkAQAMIMkCABhAkgUA\nMIAkCwBgAEkWAMAAmpECAAygGSkAwJx1u10IADB3DzyQ7NuXnLCg6keRBQDsCYtMsRJFFgCwRyyy\nfUOiyAIA9ghJFgDAAJIsAIABJFkAAAMsshFposgCAPaIRTYiTRRZAMAesZRJVlWdU1XXV9UNVXXh\nJuO+uaoOVtX3z2+KAADHb+mSrKo6Icnrkpyd5LQkL66qU48w7jVJrpj3JAEAjtcybnw/M8mN3X1z\ndx9M8uYk5x5m3M8muSzJnXOcHwDAXCxjC4cTk9y67v1t02MPq6onJ/m+7v7tJDW/6QEAzMeik6z9\nczrPrydZv1friIXW6urqw69XVlaysrIypykAABzZVknW2tpa1tbW5na96u7NB1Q9O8lqd58zff+K\nJN3dv7puzE0PvUzyhCT3Jfmp7r58w7l6q+sBAIzwtrclb3hD8va3zza+qtLdx3yHbpYk6+okp1TV\nyUn+Nsl5SV68fkB3P3XdhC5N8s6NBRYAwHZa9J6sLYus7j5UVRckuTKTPVyXdPd1VXX+5OO+eOOv\nDJgnAMBxWco9Wd39J0meseHY7x5h7I/NYV4AAHO1lM1IAQB2uqVrRgoAsBtIsgAABpBkAQAMcP/9\nySMfubjrKbIAgD3hrruSr/zKxV1PkQUA7Al33qnIAgCYu09/WpEFADB3d96ZPPGJi7ueIgsA2BMW\nnWRt+YDouV7MA6IBgG3QnXzJlyT33JN86ZfO9jvH+4BoSRYAsOt99rOTRqSzFljzoMgCAHa9T396\nsfuxEkUWALAHLLp9Q6LIAgD2AEkWAMAAkiwAgAEW3b4hUWQBAHvAohuRJoosAGAPkGQBAAxg4zsA\nwAA2vgMADLAdSZZnFwIAu1p38shHJp/73OTPWXl2IQDAJu69d/Jw6KMpsOZBkQUA7Grb0b4hUWQB\nALvcdrRvSBRZAMAutx2b3hNFFgCwy21H+4ZEkQUA7HKSLACAASRZAAADSLIAAAaQZAEADKCFAwDA\nANvVjNSzCwGAXas7OXAg+fznj/6xOp5dCABwBPfckzzqUYt/bmGiyAIAdrHt2vSeKLIAgF1su9o3\nJIosAGAXk2QBAAwgyQIAGGDpk6yqOqeqrq+qG6rqwsN8/pKq+vD056qq+ob5TxUA4OhsVyPSZIYi\nq6pOSPK6JGcnOS3Ji6vq1A3Dbkry7d39zCS/nOT35j1RAICjtV2NSJPZkqwzk9zY3Td398Ekb05y\n7voB3f3e7r53+va9SU6c7zQBAI7eUidZmRRMt657f1s2L6J+IskfH8+kAADmYTs3vu+f58mq6nlJ\nXpbkrCONWV1dffj1yspKVlZW5jkFAICHHc3G97W1taytrc3t2ls+u7Cqnp1ktbvPmb5/RZLu7l/d\nMO70JG9Nck53//URzuXZhQDAQjz44ORxOvfdN3l+4dFaxLMLr05ySlWdXFUHkpyX5PINkzgpkwLr\npUcqsAAAFunuu5NHP/rYCqx52PJ2YXcfqqoLklyZSVF2SXdfV1XnTz7ui5O8KsnjkvxWVVWSg919\n5siJAwBsZjs3vScz3C6c68XcLgQAFuTd705e+crkqquO7fcXcbsQAGDH2e4kS5EFAOxK29m+IVFk\nAQC71HY+tzBRZAEAu5QkCwBgAEkWAMAAkiwAgAEkWQAAA2x3CwfNSAGAXefBByeP0/mHf0ge8Yhj\nO4dmpAAAG3zmM8ljHnPsBdY8KLIAgF1nuze9J4osAGAX2u5N74kiCwDYhSRZAAADSLIAAAaQZAEA\nDCDJAgAYYLsbkSaKLABgF3K7EABgALcLAQAGWIYky7MLAYBd5dCh5JGPTO6/P9m//9jP49mFAADr\nvO99ydOednwF1jwosgCAXeWyy5If+IHtnkWyzTUeAMD8dE+KrLe/fbtnIskCAHaRD3xgcpvw9NO3\neyaKLABgF3nLW5If/MGkjnm7+vwosgCAXeGhW4XLsB8rUWQBALvEhz+cPPhg8o3fuN0zmVBkAQC7\nwmWXLc+twkSRBQDsAt2T/VjLcqswUWQBALvARz866fD+Td+03TP5J4osAGDHe2jD+7LcKkwUWQDA\nLvDQfqxlosgCAHa0a69NPvvZ5Mwzt3sm/5wiCwDY0S67LHnhC5MTlqyqWbLpAAAcnWVqQLqeIgsA\n2LE+/vHkrruS5zxnu2fyxRRZAMCO9cY3LuetwiTZv90TAAA4WgcPJhdemLztbckVV2z3bA5PkQUA\n7Cif+lTyohclj3lM8v73J4973HbP6PBmCteq6pyqur6qbqiqC48w5jer6saq+lBVPWu+02QZrK2t\nbfcUOEbWbmezfjub9ZuvtbVJV/ezz07+8A+Xt8BKZiiyquqEJK9LcnaS05K8uKpO3TDmu5N8TXc/\nLcn5SX5nwFzZZv5DsXNZu53N+u1s1m8+7rsvec1rkvPOS97whuRVr1rOfVjrzXK78MwkN3b3zUlS\nVW9Ocm6S69eNOTfJG5Oku/+yqh5bVU/q7jvmPWEAYPd64IHk3nsnDUbf//7kAx+Y/PnJTyZnnZVc\nfXXyVV+13bOczSxF1olJbl33/rZMCq/Nxtw+PfZFRdb3fu9RzpCl8fGPT/5FZ+exdjub9dvZdsP6\ndc9+/KFj3V/8+tChyc8DD/zTn//4j8nnP5987nOTnwceSB796OTUU5Mzzkie+9zk538+Oe205MCB\nMf98o1Qf6X+5hwZUvTDJ2d39U9P3/yHJmd39c+vGvDPJr3T3X0zf/2mSX+zuD2w41+YXAwBYIt19\nzI+cniXJuj3JSeveP2V6bOOYr9pizHFNFABgJ5lly9jVSU6pqpOr6kCS85JcvmHM5Ul+OEmq6tlJ\n7rEfCwDYy7ZMsrr7UFVdkOTKTIqyS7r7uqo6f/JxX9zdf1RV31NVn0hyX5KXjZ02AMBy23JPFgAA\nR29hHSZmaWjKcqiqp1TVn1XVx6rqI1X1c9PjX1FVV1bVx6vqiqp67HbPlcOrqhOq6gNVdfn0vbXb\nIaYtcN5SVddN/w5+i/XbOarq56vqo1V1TVX976o6YP2WV1VdUlV3VNU1644dcb2q6pemjdevq6rv\n2ur8CymyZmloylJ5IMl/7O7TkvybJD8zXa9XJPnT7n5Gkj9L8kvbOEc29/Ik1657b+12jt9I8kfd\n/bVJnplJT0LrtwNU1ZOT/GySM7r79Ey25Lw41m+ZXZpJbbLeYderqr4uyYuSfG2S707yW1W16Rf6\nFpVkPdzQtLsPJnmooSlLqLv/rrs/NH39+STXZfKN0XOTvGE67A1Jvm97ZshmquopSb4nyX9fd9ja\n7QBV9Zgk39bdlyZJdz/Q3ffG+u0k+5I8qqr2J/nSTL5pb/2WVHdfleTuDYePtF4vSPLm6d/Lv0ly\nY764b+g/s6gi63ANTU9c0LU5DlX1r5M8K8l7kzzcxb+7/y7JE7dvZmzivyX5hSTrN1xau53hq5Pc\nVVWXTm/3XlxVXxbrtyN096eS/FqSWzIpru7t7j+N9dtpnniE9TpS4/UjWvKn/rCdqurRSS5L8vJp\norXxWxK+NbFkqurfJbljmkRuFmNbu+W0P8kZSV7f3Wdk8m3tV8TfvR2hqv5FJinIyUmenEmi9UOx\nfjvdMa/XooqsWRqaskSmUfdlSX6/u98xPXxHVT1p+vm/THLnds2PI/rWJC+oqpuS/J8k31FVv5/k\n76zdjnBbklu7+33T92/NpOjyd29n+LdJburuz3T3oSRvS/KcWL+d5kjrNVPj9fUWVWTN0tCU5fI/\nklzb3b+x7tjlSX50+vpHkrxj4y+xvbr7ld19Unc/NZO/Z3/W3S9N8s5Yu6U3vUVxa1U9fXro+Uk+\nFn/3dopbkjy7qr5kuiH6+Zl8AcX6LbfKP0/+j7Relyc5b/qN0a9OckqSv9r0xIvqk1VV52TyrZmH\nGpq+ZiEX5qhV1bcmeXeSj2QSk3aSV2byL9P/zaSSvznJi7r7nu2aJ5urqucm+U/d/YKqelys3Y5Q\nVc/M5EsLj0hyUybNnffF+u0IVXVRJv8H52CSDyb5iSRfHuu3lKrqTUlWkjw+yR1JLkry9iRvyWHW\nq6p+KcmPZ7K+L+/uKzc9v2akAADzZ+M7AMAAiiwAgAEUWQAAAyiyAAAGUGQBAAygyAIAGECRBQAw\nwP8Hrt2wId8hQpoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114aa2470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1,1,figsize=(10,7))\n",
    "chart = ax.plot([v[0] for v in vis],[v[1] for v in vis])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "a=2ではどうやら発散している。aを小さくする。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "t0,t1,t2,t3,vis = gradientdescent(X,y,a=1,max_itr=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t0: 22.5328063241\n",
      "t1: -2.11673199573\n",
      "t2: 1.15095904462\n",
      "t3: 5.56580032253\n"
     ]
    }
   ],
   "source": [
    "print('t0:',t0)\n",
    "print('t1:',t1)\n",
    "print('t2:',t2)\n",
    "print('t3:',t3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl0AAAGoCAYAAAB8NVAcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFwRJREFUeJzt3W+MZXd93/HP196Fhn+uodgmXgeDoI6NQhwE61BUdVqI\nMUWyiZpY/FGV1ESKBAQEVYWXJ94+SlwFqkitHzT80QaZOjZVwJaiYlvWpKEJDil2cNmNu9Jqjb2w\nA9SU4kaRZr3fPriXMF7PegbPzO/emXm9pKu598y95/xWPwa9+Z1zD9XdAQBga50z6wEAAOwGogsA\nYADRBQAwgOgCABhAdAEADCC6AAAGWDO6quq5VXV/VT1QVQ9V1U3T7edX1d1V9XBVfamqzlvxmQNV\ndbSqjlTV1Vv5DwAA2A5qPffpqqrndfffVNW5Sf57kg8m+RdJ/nd3/7uq+miS87v7xqq6IsmtSd6Q\nZF+Se5O8ut0QDADYxdZ1erG7/2b69LlJ9iTpJNclOTTdfijJO6bPr01yW3ef6u7jSY4m2b9ZAwYA\n2I7WFV1VdU5VPZDkZJJ7uvurSS7s7qUk6e6TSS6Yvv3iJI+u+PiJ6TYAgF1rvStdp7v7FzI5Xbi/\nql6TyWrXU9622YMDANgp9vwkb+7u/1tVi0muSbJUVRd291JVXZTkO9O3nUhyyYqP7Ztue4qqEmkA\nwLbR3bWRz6/n24v/4EffTKyqn0ryS0mOJLkzya9P3/ZrSb44fX5nkndW1XOq6hVJXpXkL84yeI9t\n+rjppptmPgYP87cbH+Zuez/M3/Z9bIb1rHS9LMmhqjonk0j7w+7+46r6SpLbq+qGJI8kuX4aUoer\n6vYkh5MsJ3lfb9ZoAQC2qTWjq7sfSvK6VbY/nuQtZ/nMbyf57Q2PDgBgh3BHep6VhYWFWQ+BDTB/\n25e5297M3+62rpujbsmBq5x1BAC2hapKb/WF9AAAbJzoAgAYQHQBAAwgugAABhBdAAADiC4AgAFE\nFwDAAKILAGAA0QUAMIDoAgAYYKbR5f8FCADYLWYaXU8+OcujAwCMM9PoOnVqlkcHABhHdAEADCC6\nAAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAA\nogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwwEyja3l5lkcH\nABjHShcAwACiCwBgANEFADCA6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCA\n6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwABrRldV7auq+6rqG1X1UFX91nT7TVX1WFV9bfq4\nZsVnDlTV0ao6UlVXn23fogsA2C32rOM9p5J8pLsfrKoXJPkfVXXP9Hef6O5PrHxzVV2e5PoklyfZ\nl+Teqnp1d/fTdiy6AIBdYs2Vru4+2d0PTp8/keRIkounv65VPnJdktu6+1R3H09yNMn+1fYtugCA\n3eInuqarqi5NcmWS+6ebPlBVD1bVJ6vqvOm2i5M8uuJjJ/LjSHsK0QUA7Bbrjq7pqcXPJ/nQdMXr\nliSv7O4rk5xM8vGf9OCiCwDYLdZzTVeqak8mwfXZ7v5iknT3d1e85feT3DV9fiLJJSt+t2+67WmO\nHTuYgwcnzxcWFrKwsLD+kQMAbJHFxcUsLi5u6j5rlevbn/6mqj9I8r3u/siKbRd198np8w8neUN3\nv7uqrkhya5KrMjmteE+Sp11IX1W9f3/n/vsDADDXqirdvdq17Ou25kpXVb0pyXuSPFRVDyTpJB9L\n8u6qujLJ6STHk/xmknT34aq6PcnhJMtJ3rfaNxcTpxcBgN1jXStdW3Lgqn7tazt/9VczOTwAwLpt\nxkqXO9IDAAwgugAABhBdAAADiC4AgAFmGl3Ly7M8OgDAOFa6AAAGEF0AAAOILgCAAUQXAMAAogsA\nYADRBQAwwEyjK0lOn571CAAAtt5Mo2vPHqtdAMDuILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEF\nADCA6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMILoAAAYQ\nXQAAA4guAIABZhpde/eKLgBgd7DSBQAwgOgCABhg5tG1vDzLEQAAjDHz6LLSBQDsBqILAGAA0QUA\nMIDoAgAYQHQBAAwgugAABhBdAAADiC4AgAFEFwDAAKILAGAA0QUAMIDoAgAYQHQBAAwgugAABhBd\nAAADiC4AgAFEFwDAAKILAGAA0QUAMIDoAgAYYM3oqqp9VXVfVX2jqh6qqg9Ot59fVXdX1cNV9aWq\nOm/FZw5U1dGqOlJVV59t36ILANgt1rPSdSrJR7r7NUnemOT9VfWzSW5Mcm93X5bkviQHkqSqrkhy\nfZLLk7wtyS1VVavtWHQBALvFmtHV3Se7+8Hp8yeSHEmyL8l1SQ5N33YoyTumz69Nclt3n+ru40mO\nJtm/2r5FFwCwW/xE13RV1aVJrkzylSQXdvdSMgmzJBdM33ZxkkdXfOzEdNvTiC4AYLdYd3RV1QuS\nfD7Jh6YrXn3GW858vSbRBQDsFnvW86aq2pNJcH22u7843bxUVRd291JVXZTkO9PtJ5JcsuLj+6bb\nnubTnz6YRx5JDh5MFhYWsrCw8Gz+DQAAm2pxcTGLi4ubus/qXnuBqqr+IMn3uvsjK7bdnOTx7r65\nqj6a5PzuvnF6If2tSa7K5LTiPUle3WccqKr6a1/r3HBD8sADm/gvAgDYZFWV7l71i4HrteZKV1W9\nKcl7kjxUVQ9kchrxY0luTnJ7Vd2Q5JFMvrGY7j5cVbcnOZxkOcn7zgyuH9m71+lFAGB3WNdK15Yc\nuKqPHOn88i8nR47MZAgAAOuyGStd7kgPADDAzKNreXmWIwAAGGPm0WWlCwDYDUQXAMAAogsAYADR\nBQAwgOgCABhAdAEADCC6AAAGmGl0nXtu8uSTyYxuig8AMMxMo6vqx+EFALCTzTS6EqcYAYDdQXQB\nAAwgugAABhBdAAADiC4AgAFEFwDAAKILAGAA0QUAMIDoAgAYQHQBAAwgugAABhBdAAADiC4AgAFE\nFwDAAKILAGAA0QUAMIDoAgAYQHQBAAww8+jau1d0AQA738yjy0oXALAbzEV0LS/PehQAAFtrLqLL\nShcAsNOJLgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsA\nYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6\nAAAGWDO6qupTVbVUVV9fse2mqnqsqr42fVyz4ncHqupoVR2pqqvX2r/oAgB2g/WsdH0myVtX2f6J\n7n7d9PFfk6SqLk9yfZLLk7wtyS1VVc+0c9EFAOwGa0ZXd385yfdX+dVqMXVdktu6+1R3H09yNMn+\nZ9q/6AIAdoONXNP1gap6sKo+WVXnTbddnOTRFe85Md12VqILANgNnm103ZLkld19ZZKTST7+bAew\nd6/oAgB2vj3P5kPd/d0VL38/yV3T5yeSXLLid/um21Z18ODBLC0ljz2WLC4uZGFh4dkMBwBgUy0u\nLmZxcXFT91ndvfabqi5Ncld3/9z09UXdfXL6/MNJ3tDd766qK5LcmuSqTE4r3pPk1b3KQaqquzuH\nDye/8ivJ4cOb9U8CANhcVZXufsYvB65lzZWuqvpckoUkL6mqbya5Kck/raork5xOcjzJbyZJdx+u\nqtuTHE6ynOR9qwXXUwbgmi4AYBdY10rXlhx4utJ17Fjylrckx47NZBgAAGvajJUud6QHABhAdAEA\nDCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwwMyj65zpCE6fnu04AAC20syjK7HaBQDsfKIL\nAGAA0QUAMIDoAgAYQHQBAAwgugAABhBdAAADiC4AgAFEFwDAAKILAGAA0QUAMIDoAgAYQHQBAAwg\nugAABhBdAAADiC4AgAFEFwDAAHMRXXv3ii4AYGebi+iy0gUA7HRzE13Ly7MeBQDA1pmb6LLSBQDs\nZKILAGAA0QUAMIDoAgAYQHQBAAwgugAABhBdAAADiC4AgAFEFwDAAKILAGAA0QUAMIDoAgAYQHQB\nAAwgugAABhBdAAADiC4AgAFEFwDAAKILAGAA0QUAMIDoAgAYQHQBAAwgugAABhBdAAADiC4AgAFE\nFwDAAKILAGAA0QUAMIDoAgAYYM3oqqpPVdVSVX19xbbzq+ruqnq4qr5UVeet+N2BqjpaVUeq6ur1\nDGLvXtEFAOxs61np+kySt56x7cYk93b3ZUnuS3IgSarqiiTXJ7k8yduS3FJVtdYBrHQBADvdmtHV\n3V9O8v0zNl+X5ND0+aEk75g+vzbJbd19qruPJzmaZP9ax9izJ1leXu+QAQC2n2d7TdcF3b2UJN19\nMskF0+0XJ3l0xftOTLc9IytdAMBOt1kX0vdGPiy6AICdbs+z/NxSVV3Y3UtVdVGS70y3n0hyyYr3\n7ZtuW9XBgweTJI8/nvzwhwtJFp7lcAAANs/i4mIWFxc3dZ/VvfYiVVVdmuSu7v656eubkzze3TdX\n1UeTnN/dN04vpL81yVWZnFa8J8mre5WDVNXfbT5+PFlYmPwEAJg3VZXuXvPLgc9kzZWuqvpcJktQ\nL6mqbya5KcnvJLmjqm5I8kgm31hMdx+uqtuTHE6ynOR9qwXX0wbh9CIAsMOta6VrSw68YqXr5Mnk\nyisnPwEA5s1mrHS5Iz0AwACiCwBgANEFADCA6AIAGEB0AQAMMBfRde65yZNPJjP6IiUAwJabi+iq\n+nF4AQDsRHMRXYlTjADAzia6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6\nAAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwwNxE1969ogsA2LnmJrqsdAEAO9lcRdfy8qxHAQCw\nNeYquqx0AQA7legCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEA\nDCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQX\nAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGmJvo2rtXdAEAO9fcRJeVLgBgJ5ur6FpenvUoAAC2\nxp6NfLiqjif5QZLTSZa7e39VnZ/kD5O8PMnxJNd39w/WHIiVLgBgB9voStfpJAvd/QvdvX+67cYk\n93b3ZUnuS3JgPTsSXQDATrbR6KpV9nFdkkPT54eSvGM9OxJdAMBOttHo6iT3VNVXq+o3ptsu7O6l\nJOnuk0kuWM+ORBcAsJNt6JquJG/q7m9X1UuT3F1VD2cSYiud+Xr1gYguAGAH21B0dfe3pz+/W1Vf\nSLI/yVJVXdjdS1V1UZLvnO3zBw8e/Lvnl1++kFOnFjYyHACATbG4uJjFxcVN3Wd1r2sh6ukfrHpe\nknO6+4mqen6Su5P82yRvTvJ4d99cVR9Ncn5337jK53vlsb/1reT1r5/8BACYJ1WV7q6N7GMjK10X\nJvmjqurpfm7t7rur6i+T3F5VNyR5JMn16xqI04sAwA72rFe6NnzgM1a6Hn88edWrJj8BAObJZqx0\nzdUd6a10AQA7legCABhAdAEADCC6AAAGmJvoOmc6ktOnZzsOAICtMDfRlVjtAgB2LtEFADCA6AIA\nGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMILoAAAYQXQAAA8xV\ndO3dK7oAgJ1prqLLShcAsFPNXXQtL896FAAAm2/uostKFwCwE4kuAIABRBcAwACiCwBgANEFADCA\n6AIAGEB0AQAMMFfR9bznJQ8+mHTPeiQAAJurekaFU1V95rG//e3k6qsnj9/93aRqJkMDAHiKqkp3\nb6hM5mql62UvS/7kT5I///Pkve91qhEA2DnmKrqS5MUvTu65JzlxIvnVX03+9m9nPSIAgI2bu+hK\nkuc/P7nrruQ5z0ne/vbkhz+c9YgAADZmrq7pOtOTTybvf3/yZ3+WvPvdyf79yetfn7zoRYMGCQCQ\nzbmma66jK5l8k/ELX0j+9E+T+++ffLvx0kuTq65KLrsseeELn/7Yuzc599ynPs45Z3Jh/mqP1ce3\nuf9eAGB7eOlLk+c+96nbdkV0nWl5OXnooUmAHTs2OfX4o8cTT0x+Li9PVslWPk6fngTcmY/VuGUF\nAOxed9yRvPGNT922K6MLAGC0HXfLCACAnUp0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEF\nADCA6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMsGXRVVXX\nVNVfV9X/qqqPbtVxAAC2gy2Jrqo6J8l/SPLWJK9J8q6q+tmtOBazsbi4OOshsAHmb/syd9ub+dvd\ntmqla3+So939SHcvJ7ktyXVbdCxmwH9xbG/mb/syd9ub+dvdtiq6Lk7y6IrXj023AQDsSi6kBwAY\noLp783da9YtJDnb3NdPXNybp7r55xXs2/8AAAFuku2sjn9+q6Do3ycNJ3pzk20n+Ism7uvvIph8M\nAGAb2LMVO+3uJ6vqA0nuzuQU5qcEFwCwm23JShcAAE81kwvp3Th1+6iqfVV1X1V9o6oeqqoPTref\nX1V3V9XDVfWlqjpv1mPl7KrqnKr6WlXdOX1t/raJqjqvqu6oqiPTv8OrzN/2UFUfrqr/WVVfr6pb\nq+o55m5+VdWnqmqpqr6+YttZ56uqDlTV0enf5tXrOcbw6HLj1G3nVJKPdPdrkrwxyfun83Vjknu7\n+7Ik9yU5MMMxsrYPJTm84rX52z5+L8kfd/flSX4+yV/H/M29qvrpJL+V5HXd/dpMLud5V8zdPPtM\nJm2y0qrzVVVXJLk+yeVJ3pbklqpa8yL7Wax0uXHqNtLdJ7v7wenzJ5IcSbIvkzk7NH3boSTvmM0I\nWUtV7Uvyz5N8csVm87cNVNWLkvzj7v5MknT3qe7+QczfdnFukudX1Z4kP5XkRMzd3OruLyf5/hmb\nzzZf1ya5bfo3eTzJ0Uz65hnNIrrcOHWbqqpLk1yZ5CtJLuzupWQSZkkumN3IWMO/T/Jvkqy8gNP8\nbQ+vSPK9qvrM9PTwf6qq58X8zb3u/laSjyf5Ziax9YPuvjfmbru54CzzdWbLnMg6WsbNUVmXqnpB\nks8n+dB0xevMb2D4RsYcqqq3J1marlY+09K3+ZtPe5K8Lsl/7O7XJfl/mZzu8Pc356rq72eySvLy\nJD+dyYrXe2LutrsNzdcsoutEkp9Z8XrfdBtzaro0/vkkn+3uL043L1XVhdPfX5TkO7MaH8/oTUmu\nrapjSf5zkn9WVZ9NctL8bQuPJXm0u/9y+vq/ZBJh/v7m31uSHOvux7v7ySR/lOQfxdxtN2ebrxNJ\nLlnxvnW1zCyi66tJXlVVL6+q5yR5Z5I7ZzAO1u/TSQ539++t2HZnkl+fPv+1JF8880PMXnd/rLt/\nprtfmcnf2n3d/S+T3BXzN/empzUerap/ON305iTfiL+/7eCbSX6xqv7e9ALrN2fyZRZzN98qTz0r\ncLb5ujPJO6ffSH1FkldlciP4Z975LO7TVVXXZPKNnB/dOPV3hg+CdamqNyX5b0keymRZtZN8LJP/\ncN2eSek/kuT67v4/sxona6uqf5LkX3f3tVX14pi/baGqfj6TL0HsTXIsyb/K5AJt8zfnquqmTP7H\nznKSB5L8RpIXxtzNpar6XJKFJC9JspTkpiRfSHJHVpmvqjqQ5L2ZzO+HuvvuNY/h5qgAAFvPhfQA\nAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAG+P+D43DM3N5akgAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114edc6d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1,1,figsize=(10,7))\n",
    "chart = ax.plot([v[0] for v in vis],[v[1] for v in vis])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "正しい値を算出できていて、収束してるっぽい。  \n",
    "これで良さそうだが、学習率を小さくしてどうなるか見てみる。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "t0,t1,t2,t3,vis = gradientdescent(X,y,a=0.1,max_itr=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t0: 22.5322078213\n",
      "t1: -2.11698783642\n",
      "t2: 1.15264613748\n",
      "t3: 5.5639292939\n"
     ]
    }
   ],
   "source": [
    "print('t0:',t0)\n",
    "print('t1:',t1)\n",
    "print('t2:',t2)\n",
    "print('t3:',t3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl0AAAGoCAYAAAB8NVAcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmQXeV55/HvI7U2QBubhJCQxGJAxAYECDCLe4CAFCpg\nl2O8xFTCxI5jvFDGmYD8D3JV4oRU8ITUmGRsGAowHizjYMDAIFS449hjwB6QwZYAARYIgVrIgFgl\ntLzzx7mNWqKlvlLfe7b7/VSduueeuz3UofHP7/ue50RKCUmSJLXXsKILkCRJ6gSGLkmSpBwYuiRJ\nknJg6JIkScqBoUuSJCkHhi5JkqQcDBq6ImJURDwYEY9ExGMRcUXj+MSIWBQRT0TEvRExvt9n5kfE\n8ohYFhFnt/MfQJIkqQqimT5dEbFHSumtiBgO/Bz4MvBR4PcppX+MiMuAiSmlyyNiFnAzcAIwFVgM\nHJZsCCZJkjpYU9OLKaW3GrujgC4gAecDNzSO3wB8uLF/HnBLSmlTSmkFsByY06qCJUmSqqip0BUR\nwyLiEWA1cF9K6ZfApJRSL0BKaTWwf+PtBwIr+318VeOYJElSx2p2pGtLSulYsunCORFxFNlo1zZv\na3VxkiRJddG1K29OKb0WET3AXKA3IiallHojYjKwpvG2VcC0fh+b2ji2jYgwpEmSpMpIKcVQPt/M\n1Yv79l2ZGBFjgD8ElgF3AH/eeNufAbc39u8APhERIyNiJnAo8NAOiner6HbFFVcUXoOb568TN89d\ntTfPX3W3VmhmpOsA4IaIGEYW0r6fUro7Ih4AFkbEfwWeBS5oBKmlEbEQWApsBC5OrapWkiSpogYN\nXSmlx4DZAxx/GThrB5/5e+Dvh1ydJElSTdiRXrulu7u76BI0BJ6/6vLcVZvnr7M11Ry1LT8c4ayj\nJEmqhIggtXshvSRJkoau0NC1eXORvy5JkpSfQkPXK68U+euSJEn5KTR0rV1b5K9LkiTlx9AlSZKU\ng0JD10svFfnrkiRJ+XGkS5IkKQeGLkmSpBwYuiRJknLgmi5JkqQcONIlSZKUA0OXJElSDgxdkiRJ\nOTB0SZIk5aDQ0LV+PWzYUGQFkiRJ+Sg0dO27r6NdkiSpMxi6JEmScmDokiRJykHhocsGqZIkqRMU\nGrr228+RLkmS1BkKH+kydEmSpE5g6JIkScpB4aHLNV2SJKkTuKZLkiQpB4WPdBm6JElSJzB0SZIk\n5aDQ0LXPPlnoSqnIKiRJktqv0NA1ZgyMGAGvv15kFZIkSe1XaOgCF9NLkqTOUHjocl2XJEnqBIYu\nSZKkHJQidNkgVZIk1V3hocs1XZIkqRMUHrqcXpQkSZ3A0CVJkpQDQ5ckSVIOCg9d++3nQnpJklR/\nhYcuR7okSVInMHRJkiTlIFJBd5uOiJRSYtMmGD0aNmyA4cMLKUWSJGmnIoKUUgzlOwof6erqgvHj\n4eWXi65EkiSpfQoPXWCDVEmSVH+lCF2u65IkSXVn6JIkScpBaUKXvbokSVKdlSJ0uaZLkiTVXSlC\nl9OLkiSp7gxdkiRJOTB0SZIk5aAUocubXkuSpLorRehypEuSJNWdoUuSJCkHpQhdY8dmN7xev77o\nSiRJktqjFKErwtEuSZJUb4OGroiYGhH3R8RvI+KxiPhS4/gVEfF8RDzc2Ob2+8z8iFgeEcsi4uxm\nCrFBqiRJqrOuJt6zCbg0pbQkIvYC/l9E3Nd47ZsppW/2f3NEHAlcABwJTAUWR8RhKaW0sx9xpEuS\nJNXZoCNdKaXVKaUljf03gGXAgY2XY4CPnA/cklLalFJaASwH5gz2O4YuSZJUZ7u0pisiZgDHAA82\nDn0xIpZExLURMb5x7EBgZb+PrWJrSNshQ5ckSaqzpkNXY2rxVuCSxojXNcDBKaVjgNXAVUMpxAap\nkiSpzppZ00VEdJEFrptSSrcDpJT6R6TvAHc29lcB0/q9NrVx7D0WLFjw7v4rr3SzaVN3k2VLkiS1\nT09PDz09PS39zhhkfXv2pogbgbUppUv7HZucUlrd2P8KcEJK6VMRMQu4GTiRbFrxPuA9C+kjYptD\nt9wCt90G3/9+C/6pJEmSWigiSCkNtJa9aYOOdEXEKcCfAo9FxCNAAr4GfCoijgG2ACuAzwGklJZG\nxEJgKbARuHiwKxfBNV2SJKnemhrpassPbzfS9etfw4UXwqOPFlKOJEnSDrVipKsUHenBkS5JklRv\npRnpWr8exo3L7sEYQ8qRkiRJrVWrka7Ro2HUKHj99aIrkSRJar3ShC7Iphjt1SVJkuqoVKHLm15L\nkqS6KlXocjG9JEmqq9KFLqcXJUlSHZUqdE2aBL29RVchSZLUeqUKXQccAC++WHQVkiRJrWfokiRJ\nykGpQteUKfDCC0VXIUmS1HqlCl2OdEmSpLoqZegq6M5EkiRJbVOq0DV2bHbfRW8FJEmS6qZUoQtc\n1yVJkuqpdKHLdV2SJKmODF2SJEk5MHRJkiTloHSha8oUQ5ckSaqf0oWuAw5wIb0kSaqfUoYuR7ok\nSVLdGLokSZJyYOiSJEnKQelC14QJ8M478OabRVciSZLUOqULXRGOdkmSpPopXegCQ5ckSaofQ5ck\nSVIOShm6vOm1JEmqm1KGLke6JElS3Ri6JEmScmDokiRJykEpQ5druiRJUt2UMnQ50iVJkuqmlKFr\nn32yjvTr1xddiSRJUmuUMnRFwKRJsHp10ZVIkiS1RilDF2TrupxilCRJdVHa0HXAAS6mlyRJ9VHq\n0OVIlyRJqgtDlyRJUg5KG7pc0yVJkuqktKHLNV2SJKlOSh26HOmSJEl1YeiSJEnKQaSUivnhiLSz\n3968GcaMyTrTjxiRY2GSJEnbiQhSSjGU7yjtSNfw4bDffnallyRJ9VDa0AVOMUqSpPowdEmSJOWg\n1KHLXl2SJKkuSh267NUlSZLqovShy5EuSZJUB4YuSZKkHJQ6dLmmS5Ik1UWpQ5cjXZIkqS5K25Ee\nYNOmrCv9+vVZs1RJkqQi1LojPUBXF+y9N6xZU3QlkiRJQ1Pq0AVOMUqSpHoYNHRFxNSIuD8ifhsR\nj0XElxvHJ0bEooh4IiLujYjx/T4zPyKWR8SyiDh7KAW6mF6SJNVBMyNdm4BLU0pHAScDX4iII4DL\ngcUppcOB+4H5ABExC7gAOBKYB1wTEbs9B2qDVEmSVAeDhq6U0uqU0pLG/hvAMmAqcD5wQ+NtNwAf\nbuyfB9ySUtqUUloBLAfm7G6BTi9KkqQ62KU1XRExAzgGeACYlFLqhSyYAfs33nYgsLLfx1Y1ju0W\nQ5ckSaqDpkNXROwF3Apc0hjx2r7fQ1t6T7imS5Ik1UFXM2+KiC6ywHVTSun2xuHeiJiUUuqNiMlA\nX2OHVcC0fh+f2jj2HgsWLHh3v7u7m+7u7ve8xzVdkiQpbz09PfT09LT0O5tqjhoRNwJrU0qX9jt2\nJfBySunKiLgMmJhSuryxkP5m4ESyacX7gMO274TaTHNUgGefhVNPhZUrB32rJElSW7SiOeqgoSsi\nTgF+CjxGNoWYgK8BDwELyUa1ngUuSCm92vjMfOAvgI1k05GLBvjepkLXhg0wbhy89ZZd6SVJUjFy\nCV3t0mzogmxd10MPwdSpbS5KkiRpALW/DVCf6dOzaUZJkqSqqkzoWrGi6CokSZJ2XyVC14wZjnRJ\nkqRqq0TocnpRkiRVXSVC14wZTi9KkqRqq0TocqRLkiRVXSVaRrz5Juy7b9arK4Z0saYkSdKu65iW\nEXvuCXvtBWvWDP5eSZKkMqpE6ALXdUmSpGqrTOhyXZckSaqyyoQuR7okSVKVVSZ0OdIlSZKqzNAl\nSZKUg8qELqcXJUlSlVUmdPWNdBXUVkySJGlIKhO6xo+Hri54+eWiK5EkSdp1lQld4LouSZJUXZUK\nXa7rkiRJVVWp0OVIlyRJqqpKhS5HuiRJUlVVKnQ50iVJkqrK0CVJkpSDSoUupxclSVJVVSp07b03\nbNwI69YVXYkkSdKuqVToishGu5xilCRJVVOp0AWu65IkSdVUudDlui5JklRFlQtdjnRJkqQqqlzo\ncqRLkiRVUeVClyNdkiSpiioXurx6UZIkVVHlQtf++8Prr8ObbxZdiSRJUvMqF7oi4KCD4Lnniq5E\nkiSpeZULXeBiekmSVD2VDF0uppckSVVTydDlSJckSaqaSoYuR7okSVLVVDJ0OdIlSZKqppKhy5Eu\nSZJUNZFSKuaHI9Lu/vbmzbDnnrBuHYwa1eLCJEmSthMRpJRiKN9RyZGu4cPhwAPt1SVJkqqjkqEL\nnGKUJEnVUtnQ5WJ6SZJUJZUNXdOnG7okSVJ1VDZ0HXIIPPVU0VVIkiQ1p7Kh64gj4Ikniq5CkiSp\nOZVsGQHw2mswZUr2OKyy0VGSJFVBx7aMABg3LttWrSq6EkmSpMFVNnQBHH64U4ySJKkaDF2SJEk5\nqHzoevzxoquQJEkaXOVDlyNdkiSpCiodumwbIUmSqqKyLSMANm+GvfaC3/8e9tijRYVJkiRtp6Nb\nRgAMH551pl++vOhKJEmSdq7SoQtc1yVJkqph0NAVEddFRG9EPNrv2BUR8XxEPNzY5vZ7bX5ELI+I\nZRFxdrsK72PokiRJVdDMSNf1wDkDHP9mSml2Y/s/ABFxJHABcCQwD7gmIoY0/zkY20ZIkqQqGDR0\npZR+BrwywEsDhanzgVtSSptSSiuA5cCcIVU4CEe6JElSFQxlTdcXI2JJRFwbEeMbxw4EVvZ7z6rG\nsbY5/HB48kko6CJMSZKkpuxu6LoGODildAywGriqdSXtmokTYcwYePHFoiqQJEkaXNfufCil9FK/\np98B7mzsrwKm9XttauPYgBYsWPDufnd3N93d3btTzrtTjFOm7NbHJUmSttHT00NPT09Lv7Op5qgR\nMQO4M6X0/sbzySml1Y39rwAnpJQ+FRGzgJuBE8mmFe8DDhuoC2ormqP2+exn4bjj4K/+qiVfJ0mS\ntI1WNEcddKQrIr4HdAP7RMRzwBXAf4mIY4AtwArgcwAppaURsRBYCmwELm5ZstoJr2CUJEllV+nb\nAPW580645hq4556WfJ0kSdI2Ov42QH1sGyFJksquFiNdGzfC2LHw6qswenRLvlKSJOldjnQ1jBgB\nM2fCU08VXYkkSdLAahG6wClGSZJUboYuSZKkHNQqdNk2QpIklVWtQpcjXZIkqaxqF7q88bUkSSqj\n2oSuffeF4cNhzZqiK5EkSXqv2oQugCOOcIpRkiSVU61Cl+u6JElSWRm6JEmSclC70GXbCEmSVEa1\nC12OdEmSpDKqxQ2v+7zzDowbB6+9BiNHtvSrJUlSB/OG19sZORKmT4fly4uuRJIkaVu1Cl0ARx8N\nS5YUXYUkSdK2ahe6jj0WHnmk6CokSZK2VbvQdcwxhi5JklQ+tVpID7B6NRx1FKxdCzGk5W6SJEkZ\nF9IPYPLkbEH9ypVFVyJJkrRV7UIXOMUoSZLKp5ah69hjvYJRkiSVS21DlyNdkiSpTGoZupxelCRJ\nZVPL0HXIIfDKK/Dyy0VXIkmSlKll6Bo2zM70kiSpXGoZusApRkmSVC61DV1ewShJksqk1qHLkS5J\nklQWtbsNUJ8NG2DChGwx/ZgxbfsZSZLUAbwN0E6MGgWHHw6PPVZ0JZIkSTUOXeC6LkmSVB61Dl1e\nwShJksqi1qHLxfSSJKksaruQHmDdOpgyBV57DYYPb+tPSZKkGnMh/SDGj4fJk+HJJ4uuRJIkdbpa\nhy5wilGSJJVDR4Qur2CUJElFq33o8gpGSZJUBrUPXX3TiwVdLyBJkgR0QOg64IDsysVVq4quRJIk\ndbLah64IpxglSVLxah+6wCsYJUlS8TomdD38cNFVSJKkTtYRoeukk+AXv3AxvSRJKk5HhK6DDoKR\nI+Gpp4quRJIkdaqOCF0RcNpp8J//WXQlkiSpU3VE6AI49VT42c+KrkKSJHWqjgldjnRJkqQidUzo\nOuooWLsWVq8uuhJJktSJOiZ0DRsGp5ziFKMkSSpGx4QucF2XJEkqTkeFLtd1SZKkokQqqGNoRKS8\nf3vDBthnH3jxRRg7NtefliRJFRYRpJRiKN/RUSNdo0bB7NlZd3pJkqQ8dVToAtd1SZKkYgwauiLi\nuojojYhH+x2bGBGLIuKJiLg3Isb3e21+RCyPiGURcXa7Ct9dp51m6JIkSflrZqTreuCc7Y5dDixO\nKR0O3A/MB4iIWcAFwJHAPOCaiBjS/GernXwy/PKX8M47RVciSZI6yaChK6X0M+CV7Q6fD9zQ2L8B\n+HBj/zzglpTSppTSCmA5MKc1pbbGhAlwyCHwyCNFVyJJkjrJ7q7p2j+l1AuQUloN7N84fiCwst/7\nVjWOlcqpp9o6QpIk5atVC+mL6Tuxm1xML0mS8ta1m5/rjYhJKaXeiJgMrGkcXwVM6/e+qY1jA1qw\nYMG7+93d3XR3d+9mObvm1FPhi1+ELVuy2wNJkiT119PTQ09PT0u/s6nmqBExA7gzpfT+xvMrgZdT\nSldGxGXAxJTS5Y2F9DcDJ5JNK94HHDZQF9QimqP2N3Mm3H03HHlkYSVIkqSKyKU5akR8D/i/wPsi\n4rmIuAj4B+API+IJ4MzGc1JKS4GFwFLgbuDiQpPVTnhLIEmSlKeOug1Qf9/+drau68YbCytBkiRV\nhLcBGgJHuiRJUp46NnQdcQS8/jo8/3zRlUiSpE7QsaErIhvtavGFCZIkSQPq2NAFMG9edgWjJElS\nu3XsQnrIphaPPhp6e6FrdzuWSZKk2nMh/RBNnQoHHQQPPFB0JZIkqe46OnQBnHsu/PjHRVchSZLq\nztB1Ltx1V9FVSJKkuuv40DVnDqxeDc89V3QlkiSpzjo+dA0fDnPnOtolSZLaq+NDFzjFKEmS2q+j\nW0b0eeUVmD49ax0xZkzR1UiSpLKxZUSLTJwIxxwDP/lJ0ZVIkqS6MnQ1OMUoSZLaydDV0Be6SjLj\nKUmSasbQ1XDUUdnj0qXF1iFJkurJ0NUQ4RSjJElqH0NXP4YuSZLULraM6Oftt2HSJHj22eyKRkmS\nJLBlRMuNGQOnnw6LFhVdiSRJqhtD13acYpQkSe3g9OJ2Vq6EY4+FF16AkSOLrkaSJJWB04ttMG0a\nzJoF99xTdCWSJKlODF0DuPBCuOmmoquQJEl14vTiAF59NbsB9ooVXsUoSZKcXmybCRPg7LNh4cKi\nK5EkSXVh6NoBpxglSVIrOb24A++8A1OnwgMPwMEHF12NJEkqktOLbTRyJHz84/Dd7xZdiSRJqgND\n1070TTGWeEBOkiRVhKFrJ044AYYNy6YYJUmShsLQtRMRLqiXJEmt4UL6QaxYAccf722BJEnqZC6k\nz8GMGdltge6+u+hKJElSlRm6muAUoyRJGiqnF5vQd1ug3/0O9t676GokSVLenF7MyYQJcM45cMst\nRVciSZKqytDVpM9/Hq6+GrZsKboSSZJURYauJnV3w157wV13FV2JJEmqIkNXkyLgq1+Fq64quhJJ\nklRFhq5d8LGPwTPPwK9+VXQlkiSpagxdu2DECLjkEke7JEnSrrNlxC567TWYORMefjhrIyFJkurP\nlhEFGDcOLroou5JRkiSpWY507YaVK+Hoo7P1XRMmFF2NJElqN0e6CjJtGsybB9/5TtGVSJKkqnCk\nazc9/DCcfz48/TSMHFl0NZIkqZ0c6SrQ7Nlw2GGwcGHRlUiSpCowdA3BX/911j6iwgN2kiQpJ4au\nIZg7Nwtc//7vRVciSZLKzjVdQ7R4MXzuc7B0KYwaVXQ1kiSpHVzTVQJnnQVHHAHf+lbRlUiSpDJz\npKsFli6FD30IHn8c9tmn6GokSVKrtWKky9DVIhdfnLWO+Od/LroSSZLUaoauElmzBmbNgl/8Imsl\nIUmS6sM1XSWy//5ZC4nLLiu6EkmSVEZDGumKiBXAOmALsDGlNCciJgLfB6YDK4ALUkrrBvhsrUa6\nANavzxbV33gjnH560dVIkqRWKcNI1xagO6V0bEppTuPY5cDilNLhwP3A/CH+RmWMHg3f+AZceils\n2VJ0NZIkqUyGGrpigO84H7ihsX8D8OEh/kalfOITMGwY3Hxz0ZVIkqQyGer04jPAq8Bm4H+mlK6N\niFdSShP7vefllNLeA3y2dtOLfR54AD7yEViyBCZNKroaSZI0VGWYXjwlpTQb+CPgCxFxGrB9kqpn\nstqJk06Ciy6Cz37W+zJKkqRM11A+nFJ6sfH4UkT8CJgD9EbEpJRSb0RMBtbs6PMLFix4d7+7u5vu\n7u6hlFMqCxbAiSfCddfBZz5TdDWSJGlX9PT00NPT09Lv3O3pxYjYAxiWUnojIvYEFgFfB84EXk4p\nXRkRlwETU0qXD/D52k4v9vnNb6C7Gx56CA4+uOhqJEnS7iq0OWpEzARuI5s+7AJuTin9Q0TsDSwE\npgHPkrWMeHWAz9c+dAFcdRXcdhv8x3/A8OFFVyNJknaHHekrYMsWOOMMmDfPxqmSJFWVoasinn0W\njj8eFi+Go48uuhpJkrSrynD1opowfTr80z/Bpz+dda2XJEmdx5GunKQEn/xktq7ru9+FGFJWliRJ\neXKkq0Ii4Prr4emns3YSkiSpswypT5d2zZgxcPvtWfPUQw+FCy8suiJJkpQXQ1fOJk2Cu+7K+ndN\nnw6nn150RZIkKQ9OLxZg1iz43vfgggvgySeLrkaSJOXB0FWQs86Cv/1bOPdcWLu26GokSVK7GboK\n9JnPwEc/CuedB+vWFV2NJElqJ0NXwb7xDTjuuKxr/UsvFV2NJElqF0NXwYYNg3/5F5g7Fz70IVi1\nquiKJElSO3j1YglEwN/9HYwfD6edlt0u6OCDi65KkiS1kqGrRP7mb2Ds2GzE6957s6scJUlSPRi6\nSubzn8+C1xlnwG23wcknF12RJElqBdd0ldCnPw3XXQfnnw9XX53dt1GSJFWbN7wusWeegY99DGbO\nzELY+PFFVyRJUmfyhtc1d/DB8POfw377wfHHw69/XXRFkiRpdxm6Sm70aPjXf4UFC7Iu9tde63Sj\nJElV5PRihSxdCp/6VDbydc01cNhhRVckSVJncHqxw8yaBb/6Fcybl13V+PWvw/r1RVclSZKaYeiq\nmK4uuPRSeOSRbI3XBz6QNVOVJEnl5vRixd15J3zpS9n9G6+4IgthkiSptZxeFH/8x9lar1NOgXPO\ngT/5E3j00aKrkiRJ2zN01cAee2RTjk8/DR/8oOFLkqQyMnTVyEDh68wzYeFCeOedoquTJKmzuaar\nxjZsgB/9CP7t32DZMrjoIvjLv8w63EuSpOa5pks7NWoUfPzj8JOfQE9PFsJOOCFrsvrtb8OaNUVX\nKElS53Ckq8O8/TbcdRf88Idwzz0we3a2/usjH4EDDii6OkmSyqkVI12Grg729tuwaBHceiv8+Mdw\nyCHZKNhZZ2VXQ44ZU3SFkiSVg6FLLfPOO/DQQ1mj1cWLs8arc+bAGWdk3e+PPx7GjSu6SkmSimHo\nUtu89hr89KfZerAHH4QlS2DGDDjxxGw77jg48sjsiklJkurO0KXcbNyY9f168MGtIezJJ2HaNPiD\nP4D3vz97fN/7smnKvfYqumJJklrH0KVCbdwIy5fDY4/Bb36TbcuXZ33CJkyAww6DQw/NWlQcdFAW\n0Pq20aOLrl6SpOYZulRKW7bACy9kAeypp+B3v4OVK+G557LHVatg/PjsasnJk7fd9t8f9t0X9tkn\n2/bdNxs1iyH9ay5J0tAYulRJW7ZAby+sXr31sW/r7YXf/z7b1q7NHjduhIkTs6A2YcLWbfx4GDs2\nC2Vjx27d9twz2/bYY+vjHntkV2OOHp1tw+xQJ0naBYYudYT16+HVV7Nt3bqt+6++Cm+8Aa+/nm19\n+2++mW1vvZVtfc/Xr9+6jRiRha9Ro3a8jRgBI0du+zhiBHR1bX3s2x8+fOvzrq7s+fbbsGHv3e//\nONAW8d79gR4H2vpegx2/p/9rA+038zjYse33+xvoeLveuyOtGEUt80hsmWuTymq//bL/HejP0CXt\nhpSyFhnr12e9yjZseO+2cWP2nu0fN23K9vseN26EzZuz532PfdvmzdtuW7YM/Lh5c1bTli3bbn3H\nt3+t/7H+j9tvW7Zs/ecdaOv/2kD7zTwOdmz7/e3PQzPHWvHeHWnFf4LK/J+xMtcmldkPfpC1S+rP\n0CVJkpQD770oSZJUEYYuSZKkHBi6JEmScmDokiRJyoGhS5IkKQeGLkmSpBwYuiRJknJg6JIkScqB\noUuSJCkHhi5JkqQcGLokSZJyYOiSJEnKgaFLkiQpB4YuSZKkHBi6JEmScmDokiRJyoGhS5IkKQeG\nLkmSpBy0LXRFxNyIeDwinoyIy9r1O5IkSVXQltAVEcOA/wGcAxwFfDIijmjHb6kYPT09RZegIfD8\nVZfnrto8f52tXSNdc4DlKaVnU0obgVuA89v0WyqA/+GoNs9fdXnuqs3z19naFboOBFb2e/5845gk\nSVJHciG9JElSDiKl1PovjTgJWJBSmtt4fjmQUkpX9ntP639YkiSpTVJKMZTPtyt0DQeeAM4EXgQe\nAj6ZUlrW8h+TJEmqgK52fGlKaXNEfBFYRDaFeZ2BS5IkdbK2jHRJkiRpW4UspLdxanVExNSIuD8i\nfhsRj0XElxvHJ0bEooh4IiLujYjxRdeqHYuIYRHxcETc0Xju+auIiBgfET+IiGWNv8MTPX/VEBFf\niYjfRMSjEXFzRIz03JVXRFwXEb0R8Wi/Yzs8XxExPyKWN/42z27mN3IPXTZOrZxNwKUppaOAk4Ev\nNM7X5cDilNLhwP3A/AJr1OAuAZb2e+75q46rgbtTSkcCRwOP4/krvYiYAnwJmJ1S+gDZcp5P4rkr\ns+vJskl/A56viJgFXAAcCcwDromIQRfZFzHSZePUCkkprU4pLWnsvwEsA6aSnbMbGm+7AfhwMRVq\nMBExFfgj4Np+hz1/FRAR44DTUkrXA6SUNqWU1uH5q4rhwJ4R0QWMAVbhuSutlNLPgFe2O7yj83Ue\ncEvjb3IFsJws3+xUEaHLxqkVFREzgGOAB4BJKaVeyIIZsH9xlWkQ/x34b0D/BZyev2qYCayNiOsb\n08PfjoiIDL7jAAACCElEQVQ98PyVXkrpBeAq4DmysLUupbQYz13V7L+D87V9lllFE1nG5qhqSkTs\nBdwKXNIY8dr+CgyvyCihiDgX6G2MVu5s6NvzV05dwGzgWyml2cCbZNMd/v2VXERMIBslmQ5MIRvx\n+lM8d1U3pPNVROhaBRzU7/nUxjGVVGNo/FbgppTS7Y3DvRExqfH6ZGBNUfVpp04BzouIZ4D/DZwR\nETcBqz1/lfA8sDKl9KvG8x+ShTD//srvLOCZlNLLKaXNwG3AB/HcVc2OztcqYFq/9zWVZYoIXb8E\nDo2I6RExEvgEcEcBdah5/wtYmlK6ut+xO4A/b+z/GXD79h9S8VJKX0spHZRSOpjsb+3+lNKFwJ14\n/kqvMa2xMiLe1zh0JvBb/PurgueAkyJidGOB9ZlkF7N47sot2HZWYEfn6w7gE40rUmcCh5I1gt/5\nlxfRpysi5pJdkdPXOPUfci9CTYmIU4CfAo+RDasm4Gtk/3ItJEv6zwIXpJReLapODS4iPgR8NaV0\nXkTsjeevEiLiaLKLIEYAzwAXkS3Q9vyVXERcQfZ/djYCjwCfAcbiuSuliPge0A3sA/QCVwA/An7A\nAOcrIuYDf0F2fi9JKS0a9DdsjipJktR+LqSXJEnKgaFLkiQpB4YuSZKkHBi6JEmScmDokiRJyoGh\nS5IkKQeGLkmSpBwYuiRJknLw/wFkT/NlIMIwiQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112e00d68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1,1,figsize=(10,7))\n",
    "chart = ax.plot([v[0] for v in vis],[v[1] for v in vis])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "t0,t1,t2,t3,vis = gradientdescent(X,y,a=0.03,max_itr=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t0: 21.4613148728\n",
      "t1: -2.14587965744\n",
      "t2: 1.3696290117\n",
      "t3: 5.23953286317\n"
     ]
    }
   ],
   "source": [
    "print('t0:',t0)\n",
    "print('t1:',t1)\n",
    "print('t2:',t2)\n",
    "print('t3:',t3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl0AAAGoCAYAAAB8NVAcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUVdWZ9/Hvw6wiiCiTOIsCzhNGcSiNmhi7BRNf2rFj\n0iadqG0S2yRqrzfS3SsdTbfaQ+LbnWhsk3YijpiWdq5O1CQag4KCiiCiIOWM4giy3z/OLasEpIqq\nW3ff4ftZa69z7q576z54xPVz7332iZQSkiRJ6lm9chcgSZLUCAxdkiRJFWDokiRJqgBDlyRJUgUY\nuiRJkirA0CVJklQBHYauiOgfEb+PiJkRMTsiLij1D4mIOyPiqYi4IyIGt/vMeRExLyLmRsSRPfkH\nkCRJqgXRmX26ImLDlNI7EdEbeAA4C/gC8GpK6YcR8V1gSErp3IgYD1wN7AuMBu4GxiQ3BJMkSQ2s\nU9OLKaV3Sqf9gT5AAiYBV5X6rwIml86PAa5LKa1MKS0E5gETylWwJElSLepU6IqIXhExE1gK3JVS\nehgYnlJqAUgpLQWGld6+BfB8u48vLvVJkiQ1rM6OdK1KKe1JMV04ISJ2phjt+tjbyl2cJElSveiz\nPm9OKb0ZEc3AZ4GWiBieUmqJiBHAS6W3LQa2bPex0aW+j4kIQ5okSaoZKaXozuc7c/fiZq13JkbE\nBsARwFxgOnBq6W1fBG4tnU8Hjo+IfhGxLbAD8NAnFG+r0XbBBRdkr8Hm9WvE5rWr7eb1q91WDp0Z\n6RoJXBURvShC2vUppdsj4nfAtIj4MvAcMKUUpOZExDRgDrACOD2Vq1pJkqQa1WHoSinNBvZaS/9r\nwOGf8JkfAD/odnWSJEl1wh3p1SVNTU25S1A3eP1ql9eutnn9GlunNkftkS+OSKtWJaJbS9IkSZJ6\nXkSQenohfU965pmc3y5JklQ5WUPXfffl/HZJkqTKMXRJkiRVQNY1XSNGJJYswXVdkiSpqtX8mq5+\n/eCpp3JWIEmSVBlZQ9ehhzrFKEmSGoOhS5IkqQKyrul67rnEPvtAS4vruiRJUvWq+TVdW20FG28M\nTzyRswpJkqSel/0xQE4xSpKkRmDokiRJqoCsa7pSKvbp2nVXePll6JU9AkqSJK2p5td0AYwaBZtt\nBrNm5a5EkiSp52QPXeAUoyRJqn9VEbqamgxdkiSpvmVf0wWwdCmMGwevvAK9e2cpR5Ik6RPVxZou\ngBEjYORImDkzdyWSJEk9oypCF7iuS5Ik1beqCl3NzbmrkCRJ6hlVsaYLivVc228Pr74KffpkKUmS\nJGmt6mZNFxR7dW29NTzySO5KJEmSyq9qQhfAYYfBPffkrkKSJKn8qip0HXEE3HVX7iokSZLKr2rW\ndAEsX15sHbF0KWy0UZayJEmS1lBXa7oABg6EvfaCX/86dyWSJEnlVVWhC5xilCRJ9cnQJUmSVAFV\ntaYL4MMPYfPN4fHHYdSoDIVJkiStpu7WdEHxwOvDDoO7785diSRJUvlUXegCpxglSVL9qbrpRYAF\nC2DiRFiyBKJbA3mSJEndV5fTiwDbbQcbblis65IkSaoHVRm6AI480ilGSZJUP6o2dB1xBNx5Z+4q\nJEmSyqMq13QBvPEGbLklvPwyDBhQwcIkSZJWU7drugA22QR23hkefDB3JZIkSd1XtaELXNclSZLq\nR1WHLvfrkiRJ9aJq13QBrFgBm20G8+cXR0mSpBzqek0XQN++cPDBcM89uSuRJEnqnqoOXeAUoyRJ\nqg9VH7paF9NnmgWVJEkqi6oPXTvtVByffDJvHZIkSd1R9aErAj73Obj99tyVSJIkdV3Vhy6Ao44y\ndEmSpNpW1VtGtFq+HEaOhCVLYOONe7gwSZKk1dT9lhGtBg6E/fd36whJklS7aiJ0geu6JElSbauJ\n6UWAp5+Gww6D558vFtdLkiRVSsNMLwKMGQP9+8Pjj+euRJIkaf3VTOhy6whJklTLaiZ0gaFLkiTV\nrppZ0wXw7rswfDgsWgSbbNJDhUmSJK2modZ0AWywARx4oA/AliRJtafD0BURoyPi3oh4IiJmR8Rf\nlfoviIgXIuKPpfbZdp85LyLmRcTciDiynAV/7nMwY0Y5f6MkSVLP63B6MSJGACNSSo9GxEDgEWAS\n8GfAWymlS1Z7/zjgGmBfYDRwNzBm9bnErkwvAixYABMnwuLF0KumxukkSVKtqsj0YkppaUrp0dL5\ncmAusEVrDWv5yCTgupTSypTSQmAeMKE7Rba33XYweDA8+mi5fqMkSVLPW6+xoojYBtgD+H2p68yI\neDQiLo+IwaW+LYDn231sMW0hrSx8ALYkSao1nQ5dpanFG4BvlEa8LgO2SyntASwFLu6ZEtfkui5J\nklRr+nTmTRHRhyJw/SKldCtASunldm/5KXBb6XwxsGW7n40u9a1h6tSpH503NTXR1NTUqaIPPrjY\nmf7VV2Ho0E59RJIkqdOam5tpbm4u6+/s1D5dEfFz4JWU0tnt+kaklJaWzr8F7JtSOjEixgNXA/tR\nTCveRRkX0reaNAmOPx5OOKHLv0KSJKlTyrGQvsORroiYCJwEzI6ImUACzgdOjIg9gFXAQuAvAVJK\ncyJiGjAHWAGc3q109Qk+9zn41a8MXZIkqTbU1I707S1ZArvsAi0t0LdvGQuTJElaTcPtSN/eqFGw\nww7wm9/krkSSJKljNRu6AI45Bm69NXcVkiRJHavp0DVpEkyfDplmSCVJkjqtpkPXLrtABMyenbsS\nSZKkdavp0BXhFKMkSaoNNR26oG2KUZIkqZrV7JYRrVasgBEjYNYs2KKsT3iUJEkqNPSWEa369i0e\ngH3bbR2/V5IkKZeaD11QTDG6rkuSJFWzmp9eBHjzTRg9GhYvho03LsuvlCRJ+ojTiyWDBsEBB8Ad\nd+SuRJIkae3qInSBU4ySJKm61cX0IsALL8Duu8PSpT4AW5IklZfTi+2MHg3bbgsPPJC7EkmSpDXV\nTegCpxglSVL1qqvQ1fpIIB+ALUmSqk1dha7ddisClw/AliRJ1aauQlcEfP7zcOONuSuRJEn6uLoK\nXQDHHQc33JC7CkmSpI+ru9C1336wbBnMnZu7EkmSpDZ1F7p69XKKUZIkVZ+6C13gFKMkSao+dRm6\nJk4sdqafNy93JZIkSYW6DF29ezvFKEmSqktdhi4ophgNXZIkqVrUzQOvV7dyJYwcCQ8/DNts02Nf\nI0mSGoAPvF6HPn1g8mRHuyRJUnWo29AF3sUoSZKqR91OLwKsWAEjRsBjj8Ho0T36VZIkqY45vdiB\nvn3hmGPgpptyVyJJkhpdXYcugC98wSlGSZKUX11PLwK8/34xxThnTnE3oyRJ0vpyerET+veHo4+G\nm2/OXYkkSWpkdR+6oLiLcdq03FVIkqRGVvfTiwDvvQejRsHs2bDFFhX5SkmSVEecXuykAQNg0iRH\nuyRJUj4NEboATjgBrrkmdxWSJKlRNUzoOuwwWLQI5s3LXYkkSWpEDRO6+vSBKVPguutyVyJJkhpR\nw4QuaJtizHTvgCRJamANFbr237+4k/Gxx3JXIkmSGk1Dha4IOP54uPba3JVIkqRG0xD7dLU3axb8\nyZ/AwoXQq6EipyRJ6ir36eqCXXeFQYPgwQdzVyJJkhpJw4WuiGJBvVOMkiSpkhpuehFg/vxiUf3i\nxdC3b5YSJElSDXF6sYu23x622w7uuSd3JZIkqVE0ZOgCpxglSVJlNeT0IsCLL8L48bBkCWywQbYy\nJElSDXB6sRtGjoS994Zf/Sp3JZIkqRE0bOgC+PM/h6uuyl2FJElqBA07vQiwfDlsuSU8+SQMH561\nFEmSVMWcXuymgQNh0iS4+urclUiSpHrX0KEL4ItfdIpRkiT1vIYPXYccAsuWwaOP5q5EkiTVs4YP\nXb16wSmnONolSZJ6VoehKyJGR8S9EfFERMyOiLNK/UMi4s6IeCoi7oiIwe0+c15EzIuIuRFxZE/+\nAcrhz/8crrkGVqzIXYkkSapXnRnpWgmcnVLaGdgfOCMixgLnAnenlHYC7gXOA4iI8cAUYBxwFHBZ\nRHRrtX9PGzMGdtgBZszIXYkkSapXHYaulNLSlNKjpfPlwFxgNDAJaJ2UuwqYXDo/BrgupbQypbQQ\nmAdMKHPdZXfqqU4xSpKknrNea7oiYhtgD+B3wPCUUgsUwQwYVnrbFsDz7T62uNRX1aZMKR6A/eqr\nuSuRJEn1qNOhKyIGAjcA3yiNeK2+s2nenU67afBgOOoouO663JVIkqR61Kczb4qIPhSB6xcppVtL\n3S0RMTyl1BIRI4CXSv2LgS3bfXx0qW8NU6dO/ei8qamJpqam9Sq+3L74Rfje9+CMM7KWIUmSMmtu\nbqa5ubmsv7NTjwGKiJ8Dr6SUzm7XdxHwWkrpooj4LjAkpXRuaSH91cB+FNOKdwFjVn/mTzU8Bmh1\nH35YPBbonntg3Ljc1UiSpGpRjscAdRi6ImIi8GtgNsUUYgLOBx4CplGMaj0HTEkpvVH6zHnAXwAr\nKKYj71zL76260AXwne8Ue3ddeGHuSiRJUrWoSOjqKdUaup54Ao48Ep57Dvp0avJVkiTVOx943QN2\n3hm22so9uyRJUnkZutbiq1+Fn/wkdxWSJKmeOL24Fm+/XSyonzULRo/OXY0kScrN6cUestFGcPzx\n8LOf5a5EkiTVC0e6PsHMmTB5MixYAL17565GkiTl5EhXD9pzT9h8c7jrrtyVSJKkemDoWgcX1EuS\npHJxenEd3nqr2D5izhwYOTJ3NZIkKRenF3vYxhvDccfBf/5n7kokSVKtc6SrAw89BCecAPPmFY8H\nkiRJjceRrgrYd99ixOvee3NXIkmSapmhqwMR8JWvwE9/mrsSSZJUy5xe7IQ33oBttimmGDffPHc1\nkiSp0pxerJBNNik2SnVBvSRJ6ipHujrpoYeKRwPNm+cO9ZIkNRpHuipowgTYbDOYMSN3JZIkqRYZ\nutbDmWfCj36UuwpJklSLnF5cD++9V+xQf//9sOOOuauRJEmV4vRihQ0YAKedBpddlrsSSZJUaxzp\nWk+LFsGee8Jzz8HAgbmrkSRJleBIVwZbbQWHHAL/9V+5K5EkSbXE0NUFZ5wBP/4x1OBAnSRJysTQ\n1QWHHQYrV8Kvf527EkmSVCsMXV0Q4fYRkiRp/biQvoveegu23hpmzYLRo3NXI0mSepIL6TPaeGM4\n6ST4j//IXYkkSaoFjnR1w5NPQlNTsX1E//65q5EkST3Fka7Mxo6FPfaAq6/OXYkkSap2hq5u+uu/\nhksucfsISZK0boaubjr8cOjdG+64I3clkiSpmhm6uikCzj4bLr44dyWSJKmauZC+DD74ALbdFm6/\nHXbfPXc1kiSp3FxIXyX69Ss2S73kktyVSJKkauVIV5m89hrssAM8/jiMGpW7GkmSVE6OdFWRTTct\nNkv9t3/LXYkkSapGjnSV0fz5sN9+sHAhDByYuxpJklQujnRVme23L3aov/LK3JVIkqRq40hXmf32\nt3DyyfD008X+XZIkqfY50lWF9t8fhg+HW27JXYkkSaomhq4ecM458MMf+mggSZLUxtDVAyZPhjff\nhHvvzV2JJEmqFoauHtCrF5x3Hnz/+7krkSRJ1cLQ1UNOOAGefbZYWC9JkmTo6iF9+8J3vgP/8A+5\nK5EkSdXALSN60HvvwXbbwYwZPghbkqRa5pYRVW7AADj7bPjBD3JXIkmScnOkq4ctX16Mdt1/P+y4\nY+5qJElSVzjSVQMGDoQzz4SLLspdiSRJysmRrgp47TUYMwZmzoSttspdjSRJWl+OdNWITTeF006D\nf/qn3JVIkqRcHOmqkKVLYfx4mDu3eDajJEmqHY501ZARI+CUU1zbJUlSo3Kkq4JefBF22QVmz4ZR\no3JXI0mSOqscI12Grgo75xx4/334t3/LXYkkSeosQ1cNeuklGDcOHn0UttwydzWSJKkzXNNVg4YN\ng69+Fb7//dyVSJKkSuowdEXEFRHREhGz2vVdEBEvRMQfS+2z7X52XkTMi4i5EXFkTxVey845B264\nARYuzF2JJEmqlM6MdF0JfGYt/ZeklPYqtf8BiIhxwBRgHHAUcFlEdGsorh4NHQqnnw5///e5K5Ek\nSZXSYehKKd0PvL6WH60tTE0CrksprUwpLQTmARO6VWGd+ta3YPp0eOaZ3JVIkqRK6M6arjMj4tGI\nuDwiBpf6tgCeb/eexaU+rWbIEDjrLPi7v8tdiSRJqoSuhq7LgO1SSnsAS4GLy1dS4/jGN+B//gee\nfDJ3JZIkqaf16cqHUkovt3v5U+C20vlioP1GCKNLfWs1derUj86bmppoamrqSjk1a9AgOPtsmDoV\nrrsudzWSJKlVc3Mzzc3NZf2dndqnKyK2AW5LKe1aej0ipbS0dP4tYN+U0okRMR64GtiPYlrxLmDM\n2jbkatR9ulb39tswZgzcdhvsvXfuaiRJ0tqUY5+uDke6IuIaoAkYGhGLgAuAQyNiD2AVsBD4S4CU\n0pyImAbMAVYAp5us1m2jjeB734Pvfhfuugu811OSpPrkjvRVYMWK4pmM//qv8Jm1bc4hSZKyckf6\nOtG3L/zgB8Vo16pVuauRJEk9wdBVJY49FjbcEK6+OnclkiSpJzi9WEXuvx9OPrnYQmLAgNzVSJKk\nVk4v1pkDD4Q99oAf/zh3JZIkqdwc6aoyc+fCIYfAU08Vu9ZLkqT8HOmqQ+PGweTJcOGFuSuRJEnl\n5EhXFVqyBHbdFWbOhK22yl2NJElypKtOjRoFZ5wB55+fuxJJklQujnRVqbffhrFj4frr4YADclcj\nSVJjc6Srjm20UbGu66yz3DBVkqR6YOiqYieeCP36wVVX5a5EkiR1l9OLVe7hh2HSpGLD1EGDclcj\nSVJjKsf0oqGrBnzpSzBsGFx0Ue5KJElqTIauBvHii8UWEr/9LYwZk7saSZIajwvpG8TIkfDtb8M5\n5+SuRJIkdZWhq0Z885vwxBNw5525K5EkSV1h6KoR/fvDxRcX4euDD3JXI0mS1pehq4YccwxsvTVc\nemnuSiRJ0vpyIX2NmT8f9tsP/vAH2Gab3NVIktQYXEjfgLbfvphiPOus3JVIkqT1YeiqQd/+Njz9\nNNx6a+5KJElSZzm9WKPuuw9OPbW4o3HgwNzVSJJU39wctcGdckqxh9cPf5i7EkmS6puhq8G1tBQ7\n1d9zT3GUJEk9w4X0DW74cPj7v4evfx1WrcpdjSRJWhdDV437yldg5Ur42c9yVyJJktbF6cU6MGsW\nHH44zJwJW2yRuxpJkuqP04sCYLfd4PTT4WtfA3OsJEnVydBVJ84/HxYuhGuvzV2JJElaG6cX68gf\n/gBHH11MNw4fnrsaSZLqh1tGaA3nngsLFsC0abkrkSSpfrimS2u44IJipOumm3JXIkmS2nOkqw49\n8AD8n/8Djz8Om26auxpJkmqf04v6RN/8Jrz+Olx1Ve5KJEmqfYYufaK33y62krjkEpg0KXc1kiTV\nNkOX1umBB+C44+DRR72bUZKk7jB0qUN/8zfw2GNw220Q3fpXRZKkxuXdi+rQBRfAiy/CT3+auxJJ\nkhqbI10NYO5cOPhgePBBGDMmdzWSJNUeR7rUKePGFSNep5wCK1fmrkaSpMZk6GoQp58OgwfDP/xD\n7kokSWpMTi82kCVLYM89i0X1EybkrkaSpNrh9KLWy6hRcNllcNJJ8OabuauRJKmxONLVgL7+dXjj\nDbjmGreRkCSpMxzpUpdccgnMmQOXX567EkmSGocjXQ3qySfhoIPg3nth111zVyNJUnVzpEtdNnZs\nMeI1ZQosX567GkmS6p8jXQ3uS1+ClOA//zN3JZIkVS9HutRtP/oRPPQQXHVV7kokSapvjnSJxx+H\nQw+F//1fGD8+dzWSJFUfR7pUFrvsAv/4j3DssbBsWe5qJEmqT4506SOnn17sWn/TTdDLOC5J0kcc\n6VJZ/fM/w8sv+3xGSZJ6giNd+pglS2DffYuNU486Knc1kiRVB0e6VHajRsH118Opp8L8+bmrkSSp\nfhi6tIYDD4Tvfa9YWP/227mrkSSpPji9qLVKqRjt+uADH4wtSVJFphcj4oqIaImIWe36hkTEnRHx\nVETcERGD2/3svIiYFxFzI+LI7hSnfCLg3/8dFiyA738/dzWSJNW+zkwvXgl8ZrW+c4G7U0o7AfcC\n5wFExHhgCjAOOAq4LMIxklq1wQZwyy3w05/CtGm5q5EkqbZ1GLpSSvcDr6/WPQlofXDMVcDk0vkx\nwHUppZUppYXAPGBCeUpVDiNHwq23whlnFI8LkiRJXdPVhfTDUkotACmlpcCwUv8WwPPt3re41Kca\ntscecMUVxcL655/v+P2SJGlNfcr0e1wRX+eOOQbmzYM//VO4/34YODB3RZIk1Zauhq6WiBieUmqJ\niBHAS6X+xcCW7d43utS3VlOnTv3ovKmpiaampi6Wo0o4+2yYOxdOOql4VFDv3rkrkiSpZzQ3N9Pc\n3FzW39mpLSMiYhvgtpTSrqXXFwGvpZQuiojvAkNSSueWFtJfDexHMa14FzBmbXtDuGVEbfrgA/jM\nZ2D33eHSS91KQpLUGCq1ZcQ1wIPAjhGxKCK+BFwIHBERTwGfLr0mpTQHmAbMAW4HTjdZ1Zd+/eDm\nm+Gee+Cf/il3NZIk1Q43R1WXvPACTJxY7OF18sm5q5EkqWeVY6SrXAvp1WBGj4YZM+DQQ2HYMDjS\nbXAlSVonn72oLhs/Hm68sRjpeuSR3NVIklTdDF3qlgMPhJ/8pNhKYv783NVIklS9nF5Ut02eDEuX\nFnc1/uY3xS72kiTp4wxdKouvfQ1eeQWOOAL+939h6NDcFUmSVF28e1FlkxKcdx7cfXexpcTgwbkr\nkiSpPMpx96KhS2WVEpx1Fvzxj3DnnbDRRrkrkiSp+wxdqkqrVsFpp8GiRfCrX8GAAbkrkiSpewxd\nqlofflg8o3H58uI5jf365a5IkqSuq8hjgKSu6N0bfvGL4njCCcUzGyVJamSGLvWYvn1h2jRYsQKm\nTDF4SZIam6FLPap/f7jhBoiAL3wB3n8/d0WSJOVh6FKP69evGPHq3x+OPRbeey93RZIkVZ6hSxXR\nty9cey0MGgSTJsG77+auSJKkyjJ0qWL69oX/+i/YbLPiWY1vv527IkmSKsfQpYrq0wd+/nPYcsvi\nkUGvvZa7IkmSKsPQpYrr3RuuuAI+9Sk45BBYsiR3RZIk9TxDl7Lo1QsuvrjYw+ugg2D+/NwVSZLU\ns/rkLkCNKwLOPx823RQOPhhmzIDddstdlSRJPcPQpey+9jUYMgQOPxxuvhkmTsxdkSRJ5ef0oqrC\nn/1Z8digyZPhxhtzVyNJUvn5wGtVlZkzi+0kvvUtOPvsYgpSkqTcyvHAa0OXqs6iRXD00cWdjf/8\nz8U2E5Ik5WToUt1atgyOOw4GDCh2sh84MHdFkqRGVo7Q5ZouVaXBg+H222HzzYsRrxdfzF2RJEnd\nY+hS1erbt9hE9fOfhwkT4OGHc1ckSVLXOb2omnDrrfCVr8All8DJJ+euRpLUaFzTpYby+OMwaVIx\n8nXhhcXjhCRJqgRDlxrOq6/ClCnQr1+xwH6TTXJXJElqBC6kV8MZOhTuuAN23LFY5zV7du6KJEnq\nHEOXak6fPvAv/wL/9//CYYfBz3+euyJJkjrm9KJq2uzZxX5eTU1FEBswIHdFkqR65PSiGt6uuxZb\nSbz+OhxwACxYkLsiSZLWztClmjdoEFx/PZx6KnzqU3DLLbkrkiRpTU4vqq787ndw4onw2c/CxRfD\nBhvkrkiSVA+cXpRW86lPwcyZ8MYbsM8+3t0oSaoehi7VncGD4eqr4bvfLe5u/NGPwEFVSVJuTi+q\nrj3zDJxwAowYAZdfDsOH565IklSLnF6UOrDDDvDAA7DbbrD77vDLX+auSJLUqBzpUsP4/e/hi1+E\nPfcsphyHDs1dkSSpVjjSJa2H/fYrFtmPHFmMfN12W+6KJEmNxJEuNaRf/xq+9CU48EC45BJHvSRJ\n6+ZIl9RFBx8Mjz0GQ4bALrvANdd4h6MkqWc50qWG99BDcNppsMUW8P/+H2yzTe6KJEnVxpEuqQwm\nTIBHHilGv/bZBy69FFauzF2VJKneONIltTNvHvzlX8Jrr8GPfwwTJ+auSJJUDcox0mXoklaTUvEA\n7XPOgU9/Gn74QzdVlaRG5/Si1AMi4PjjYe5cGDasWGj/r//qlKMkqXsc6ZI6MGcO/NVfwcsvF9tL\nHH547ookSZXm9KJUISnBTTcVD9EeOxb+8R9h3LjcVUmSKsXpRalCIuALX4AnnoDDDivudDzzzGL0\nS5KkzjB0Seuhf384++xivVevXjB+PFx4IbzzTu7KJEnVztAldcFmmxWL6++/v9jja8yYYmPVFSty\nVyZJqlaGLqkbdtoJfvlLmD4dbrmlWOd1zTWwalXuyiRJ1caF9FIZ3XcfnHcevPsuXHABTJ5cTENK\nkmqbdy9KVSgluO02+Nu/Lfb2+t734NhjDV+SVMuyh66IWAgsA1YBK1JKEyJiCHA9sDWwEJiSUlq2\nls8aulTXUoL//u8ifL3/fhG+Pv95w5ck1aJqCF0LgL1TSq+367sIeDWl9MOI+C4wJKV07lo+a+hS\nQ0gJZswowtdbb8F3vgMnngj9+uWuTJLUWdUQup4F9kkpvdqu70ngkJRSS0SMAJpTSmPX8llDlxpK\nSnDvvcUWE089VWw9cdppMHBg7sokSR2phs1RE3BXRDwcEaeV+oanlFoAUkpLgWHd/A6pLkQUD9C+\n6y64+WZ48EHYdtti2rGlJXd1kqSe1t3QNTGltBfwOeCMiDiIIoi153CWtJq994Zp0+CBB4rANXYs\nfPnLMGtW7sokST2lT3c+nFJ6sXR8OSJuASYALRExvN304kuf9PmpU6d+dN7U1ERTU1N3ypFqzo47\nwn/8B3z/+/CTn8BRRxUB7JvfhKOPdtG9JOXS3NxMc3NzWX9nl9d0RcSGQK+U0vKI2Ai4E/hb4NPA\naymli1xIL62fDz4oNlu99FJ44w34+tfh1FNh6NDclUlSY8u6kD4itgVuppg+7ANcnVK6MCI2BaYB\nWwLPUWz28XHPAAALJklEQVQZ8cZaPm/okj5BSvDb3xaPFrrttmKT1dNPh333LdaGSZIqK/vdi936\nYkOX1CkvvwxXXgn//u+w6abw1a/C8cfDoEG5K5OkxmHokhrIhx/CHXfAFVfAPfcUu9z/xV/AxImO\nfklSTzN0SQ2qpQV+8Qu4/PLi9Ze/DCefDKNG5a1LkuqVoUtqcCkV+3397GfF3l/77AOnnFKMgrnp\nqiSVj6FL0kfefRemT4ef/7zY/+uYY+Ckk4oNWft0a3MYSZKhS9JatbTAtdcW7dln4bjjisX3Bx7o\n3l+S1BWGLkkdWrAArr8errsOXnkFpkwpQtj++xvAJKmzDF2S1svcuUUAu/HGIoAdeyx8/vNwyCHQ\nt2/u6iSpehm6JHXZ00/DTTcVAezZZ+FP/qRYB3bkkS7Cl6TVGboklcVzzxWL8KdPh9/9Dg46CP70\nT4s2enTu6iQpP0OXpLJbtqzYhHX6dJgxowhdRx1VtAMOcBpSUmMydEnqUStXwu9/X4SvGTNg/vxi\nC4qjjoIjjoCtt85doSRVhqFLUkUtXVqMgt1xB9x9N2yyCRx+eBHADj20eC1J9cjQJSmbVatg1qwi\nfN11V7Ez/rhxRfg69NDimZAbb5y7SkkqD0OXpKrx3nvFVOR99xXtkUdg112hqalYmD9xIgwenLtK\nSeoaQ5ekqvXuu/Db30JzM9x/Pzz8MGy/fRHAWkPYFlvkrlKSOsfQJalmfPAB/PGPRQD7zW+K50Nu\nuGGxM35r23NP6Ncvd6WStCZDl6SalRI880wxGvbgg8XxmWdgt91gwoSi7bsv7LCDjyuSlJ+hS1Jd\neeutYjTsoYfa2ptvwj77wN57w157FcfttoPo1n/6JGn9GLok1b2WFvjDH4ow9sgjxfGtt4qpyD33\nhN13hz32gLFjnZqU1HMMXZIa0ksvFeHrscfg0UeL48KFsNNORQjbdVfYZZfiOHKko2KSus/QJUkl\n77wDjz9eBLDZs4vz2bOL/cR22aVo48cXbeedYfPNDWOSOs/QJUnrkFIxPTl7NsyZ09aeeKJYnD9u\nXDEtOXZsMUo2dixssw306ZO7cknVxtAlSV2QUjFFOWcOPPUUPPlk23HpUth2Wxgzpmg77th2PmqU\nd1JKjcrQJUll9u67xdYV8+YV7emn286XLStGwrbf/uNt222L/g02yF29pJ5i6JKkClq+HBYsgPnz\n29qCBfDss7BoEQwZUgSw1hC29dZtbautis1gJdUmQ5ckVYlVq2DJkiKAPfssPPfcx9vzzxcPAN9y\nyzXb6NFFGzUK+vfP/SeRtDaGLkmqEatWFevInn++aIsWtZ2/8AIsXgwvvgibbFI8k7K1jRpVbHsx\nalTb+bBh0Lt37j+R1FgMXZJUR1qDWWsIW7KkCGJLlny8vf46DB0KI0YUIWzECBg+vK0NG9Z2PnSo\nAU0qB0OXJDWglSuLcLZ0adFefLHYGqOlpehvPW9pKRb/b7JJEcQ23/zjbbPNPt6GDi3ahhu6h5m0\nOkOXJGmdVq6E114rwtjLLxfHV14pzl95pa29/DK8+mrRUmoLYEOHwqabrtmGDCnC3JAhbW3wYLfU\nUP0ydEmSyu6dd9oC2GuvtbXXXy+Or75anL/xRnFsbW+9VdwsMHhwEcha2+DBa2+DBq3ZNt7YzWlV\nnQxdkqSq8eGHRfB6442Pt2XL1t7eegvefLM4bz0uX17cwbnxxm1t0CAYOLA4HzhwzfPWttFGbcfV\nW9++uf/pqNYZuiRJdSWlYqStNZC99VbRli9vO7Y/f/vttr7W8/bH1tarV7FWbaONimP78w02aOtr\nPW9/XFsbMGDNY2vr399p1npk6JIkqQMpwQcfFGHunXeKENZ6fPfdtv61nb/7btv5e++1vW5//v77\nxev2rV+/tgDWPoy1Hlc/b239+q153v64+nn71rdvx8e+fYu7Wb1RYv0ZuiRJqjIpFUGsfRh7//22\ngNbZ9sEHRWs9b+1fsaLtZ+3fs2LFmj9r7WvtX7GiqK9PnzXDWN++bf2r97X2d3S+rta795qvW/va\nH1fv76hv9dar1ye/Xtd5r17rDqOGLkmStF4+/PDjYay1rVz5yeetr9sf13b+4Ydtr1fvb/1Z6/ev\n3tf6/tb+9j/rqK99W7Wqa+cpFaGrVy+47z446KCP/3MzdEmSJJVBSkX78MO20a/2yhG6vDFXkiQ1\nvIi2ka6e4v0VkiRJFWDokiRJqgBDlyRJUgUYuiRJkirA0CVJklQBhi5JkqQKMHRJkiRVgKFLkiSp\nAgxdkiRJFWDokiRJqgBDlyRJUgUYuiRJkirA0CVJklQBhi5JkqQKMHRJkiRVgKFLkiSpAgxdkiRJ\nFWDokiRJqoAeC10R8dmIeDIino6I7/bU90iSJNWCHgldEdEL+BHwGWBn4ISIGNsT36U8mpubc5eg\nbvD61S6vXW3z+jW2nhrpmgDMSyk9l1JaAVwHTOqh71IG/oejtnn9apfXrrZ5/RpbT4WuLYDn271+\nodQnSZLUkFxIL0mSVAGRUir/L434FDA1pfTZ0utzgZRSuqjde8r/xZIkST0kpRTd+XxPha7ewFPA\np4EXgYeAE1JKc8v+ZZIkSTWgT0/80pTShxFxJnAnxRTmFQYuSZLUyHpkpEuSJEkfl2UhvRun1o6I\nGB0R90bEExExOyLOKvUPiYg7I+KpiLgjIgbnrlWfLCJ6RcQfI2J66bXXr0ZExOCI+GVEzC39PdzP\n61cbIuJbEfF4RMyKiKsjop/XrnpFxBUR0RIRs9r1feL1iojzImJe6e/mkZ35joqHLjdOrTkrgbNT\nSjsD+wNnlK7XucDdKaWdgHuB8zLWqI59A5jT7rXXr3b8C3B7SmkcsDvwJF6/qhcRo4C/AvZKKe1G\nsZznBLx21exKimzS3lqvV0SMB6YA44CjgMsiosNF9jlGutw4tYaklJamlB4tnS8H5gKjKa7ZVaW3\nXQVMzlOhOhIRo4HPAZe36/b61YCIGAQclFK6EiCltDKltAyvX63oDWwUEX2ADYDFeO2qVkrpfuD1\n1bo/6XodA1xX+ju5EJhHkW/WKUfocuPUGhUR2wB7AL8DhqeUWqAIZsCwfJWpA5cC3wbaL+D0+tWG\nbYFXIuLK0vTwTyJiQ7x+VS+ltAS4GFhEEbaWpZTuxmtXa4Z9wvVaPcssphNZxs1R1SkRMRC4AfhG\nacRr9TswvCOjCkXE0UBLabRyXUPfXr/q1AfYC/hxSmkv4G2K6Q7//lW5iNiEYpRka2AUxYjXSXjt\nal23rleO0LUY2Krd69GlPlWp0tD4DcAvUkq3lrpbImJ46ecjgJdy1ad1mggcExELgGuBwyLiF8BS\nr19NeAF4PqX0h9LrGylCmH//qt/hwIKU0msppQ+Bm4ED8NrVmk+6XouBLdu9r1NZJkfoehjYISK2\njoh+wPHA9Ax1qPN+BsxJKf1Lu77pwKml8y8Ct67+IeWXUjo/pbRVSmk7ir9r96aUTgFuw+tX9UrT\nGs9HxI6lrk8DT+Dfv1qwCPhURAwoLbD+NMXNLF676hZ8fFbgk67XdOD40h2p2wI7UGwEv+5fnmOf\nroj4LMUdOa0bp15Y8SLUKRExEfg1MJtiWDUB51P8yzWNIuk/B0xJKb2Rq051LCIOAf46pXRMRGyK\n168mRMTuFDdB9AUWAF+iWKDt9atyEXEBxf/srABmAqcBG+O1q0oRcQ3QBAwFWoALgFuAX7KW6xUR\n5wF/QXF9v5FSurPD73BzVEmSpJ7nQnpJkqQKMHRJkiRVgKFLkiSpAgxdkiRJFWDokiRJqgBDlyRJ\nUgUYuiRJkirA0CVJklQB/x+Jd/tT12w0ywAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1142469b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1,1,figsize=(10,7))\n",
    "chart = ax.plot([v[0] for v in vis],[v[1] for v in vis])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "どんどん収束スピードが遅くなっていることがわかる。つまり学習率が小さすぎる状態。\n",
    "\n",
    "このように、グラフを見ながら学習率を調節する。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}