Math behind the Linear Regression with Gradient Descent from scratch

LinearRegrassion.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c7b727ce",
   "metadata": {},
   "source": [
    "\n",
    "# Linear Regression With Gradient Descent\n",
    "\n",
    "\n",
    "![reg](./images/regression.svg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e929f281",
   "metadata": {},
   "source": [
    "X_data = Matrix of Independent features of size mxn , or X ∈ ℝᵐˣⁿ\n",
    "\n",
    "xⁱ = ith observation of X_data or ith row of matrix X_data , or xⁱ ∈ ℝⁿ\n",
    "\n",
    "Y = Target vector of size mx1, or Y ∈ ℝᵐ\n",
    "\n",
    "yⁱ = ith target value or ith data point of vector Y , or yⁱ ∈ ℝ\n",
    "\n",
    "Ŷ = Predicted target vector of size mx1, or Ŷ ∈ ℝᵐ\n",
    "\n",
    "ŷⁱ = ith predicted target value or ith data point of vector Ŷ, or ŷⁱ ∈ ℝ\n",
    "\n",
    "W = Weights vector of size (n+1)x1, or W ∈ ℝⁿ⁺¹\n",
    "\n",
    "$$\\large Y =\n",
    "\\begin{bmatrix}\n",
    "y^{1}\\\\\n",
    "y^{2}\\\\\n",
    "y^{3}\\\\\n",
    "\\vdots\\\\\n",
    "y^{m}\\\\\n",
    "\\end{bmatrix}, \\large X_{data}  =  \\begin{bmatrix} \n",
    "x^{1}_{1} & x^{1}_{2} & x^{1}_{3} &  \\dots & x^{1}_{n} \\\\\n",
    "x^{2}_{1} & x^{2}_{2} & x^{2}_{3} &  \\dots & x^{2}_{n} \\\\\n",
    "x^{3}_{1} & x^{3}_{2} & x^{3}_{3} &  \\dots & x^{3}_{n} \\\\\n",
    "\\vdots &  & \\vdots\\\\\n",
    "x^{m}_{1} & x^{m}_{2} & x^{m}_{3}  & \\dots & x^{m}_{n} \\\\\n",
    "\\end{bmatrix}, \\large \\hat{Y} = \\begin{bmatrix}\n",
    "\\hat{y}^{1}\\\\\n",
    "\\hat{y}^{2}\\\\\n",
    "\\hat{y}^{3}\\\\\n",
    "\\vdots\\\\\n",
    "\\hat{y}^{m}\\\\\n",
    "\\end{bmatrix}, \\large W =\n",
    "\\begin{bmatrix}\n",
    "w_{1}\\\\\n",
    "w_{2}\\\\\n",
    "w_{3}\\\\\n",
    "\\vdots\\\\\n",
    "w_{n}\\\\\n",
    "w_{n+1}\\\\\n",
    "\\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e34ae574",
   "metadata": {},
   "source": [
    "$$\\large \\hat{y}^{i} = \\sum \\limits _{k=1} ^{n} w_{k}\\times x_{k}^{i} + b = \\sum \\limits _{k=1} ^{n+1} w_{k}\\times x_{k}^{i} \\quad\\quad\\quad\\quad w_{n+1} = b \\quad  \\text{and} \\quad x_{n+1} = 1\n",
    "$$\n",
    "\n",
    "$$\\large \\hat{y}^{i} = \\begin{bmatrix}x^{i}_{1} & x^{i}_{2} & x^{i}_{3} & \\dots & x^{1}_{n} & 1\\end{bmatrix} \\begin{bmatrix} w_{1}\\\\\n",
    "w_{2}\\\\\n",
    "w_{3}\\\\\n",
    "\\vdots\\\\\n",
    "w_{n}\\\\\n",
    "w_{n+1}\\\\\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "\n",
    "$$\\large \\hat{Y}  = \\begin{bmatrix}\n",
    "\\hat{y}^{1}\\\\\n",
    "\\hat{y}^{2}\\\\\n",
    "\\hat{y}^{3}\\\\\n",
    "\\vdots\\\\\n",
    "\\hat{y}^{m}\\\\\n",
    "\\end{bmatrix} = \\small \\begin{bmatrix} \n",
    "x^{1}_{1} & x^{1}_{2} & x^{1}_{3} & \\dots & x^{1}_{n} & 1\\\\\n",
    "x^{2}_{1} & x^{2}_{2} & x^{2}_{3} & \\dots & x^{2}_{n} & 1\\\\\n",
    "x^{3}_{1} & x^{3}_{2} & x^{3}_{3} & \\dots & x^{3}_{n} & 1\\\\\n",
    "\\vdots  & \\vdots\\\\\n",
    "x^{i}_{1} & x^{i}_{2} & x^{i}_{3} & \\dots & x^{i}_{n} & 1\\\\\n",
    "\\vdots & & \\dots & & \\vdots\\\\\n",
    "x^{m}_{1} & x^{m}_{2} & x^{m}_{3} & \\dots & x^{m}_{n} & 1\\\\\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "w_{1}\\\\\n",
    "w_{2}\\\\\n",
    "w_{3}\\\\\n",
    "\\vdots\\\\\n",
    "w_{n}\\\\\n",
    "w_{n+1}\\\\\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "\n",
    "$$\\hat{Y} = X.W $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d43c4a73",
   "metadata": {},
   "source": [
    "Lest Call that Big fat Matrix X , \n",
    "\n",
    "So Our final expression for $\\hat{Y} = XW$ , which is a vector of size mx1\n",
    "\n",
    "Residuals $e = (Y − \\hat{Y})$, which is a vector of size mx1\n",
    "\n",
    "Square Sum of residuals $e^{2} = (Y − \\hat{Y})^{2}$ , which is a vector of size mx1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6aab87b",
   "metadata": {},
   "source": [
    "$$\\large Y =\n",
    "\\begin{bmatrix}\n",
    "y^{1}\\\\\n",
    "y^{2}\\\\\n",
    "y^{3}\\\\\n",
    "\\vdots\\\\\n",
    "y^{m}\\\\\n",
    "\\end{bmatrix}, \\large \\hat{Y} = \\begin{bmatrix}\n",
    "\\hat{y}^{1}\\\\\n",
    "\\hat{y}^{2}\\\\\n",
    "\\hat{y}^{3}\\\\\n",
    "\\vdots\\\\\n",
    "\\hat{y}^{m}\\\\\n",
    "\\end{bmatrix}, \\large e =\n",
    "\\begin{bmatrix}\n",
    "y^{1}-\\hat{y}^{1}\\\\\n",
    "y^{2}-\\hat{y}^{2}\\\\\n",
    "y^{3}-\\hat{y}^{3}\\\\\n",
    "\\vdots\\\\\n",
    "y^{m}-\\hat{y}^{m}\\\\\n",
    "\\end{bmatrix} = \\begin{bmatrix}\n",
    "e^{1}\\\\\n",
    "e^{2}\\\\\n",
    "e^{3}\\\\\n",
    "\\vdots\\\\\n",
    "e^{m}\\\\\n",
    "\\end{bmatrix}$$ "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76b09cd3",
   "metadata": {},
   "source": [
    "### Cost Function\n",
    "\n",
    "$$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m} \\sum_{k=1}^{m} \\left (y^{k}-\\hat{y}^{k} \\right )^{2} $$\n",
    "\n",
    "$$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m} \\sum_{k=1}^{m} ( e^{k} )^{2}  = \\frac{1}{m} \\left [ (e^{1})^{2} + (e^{2})^{2}+ (e^{2})^{3}+ \\dots + (e^{2})^{m}  \\right ]$$\n",
    "\n",
    "$$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m} \\begin{bmatrix}e^{1} & e^{2} & e^{3} & \\dots & e^{m} &\\end{bmatrix} \\begin{bmatrix} e^{1} \\\\\n",
    "e^{2} \\\\\n",
    "e^{3} \\\\\n",
    "\\vdots\\\\\n",
    "e^{m} \\\\\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "$$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m} e^{T}e$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9b33b8f",
   "metadata": {},
   "source": [
    "#### Cost Function simplification\n",
    "\n",
    "$$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m} e^{T}e = \\frac{1}{m}(Y-\\hat{Y})^{T}(Y-\\hat{Y}) = \\frac{1}{m}(Y^{T}-\\hat{Y}^{T})(Y-\\hat{Y})$$\n",
    "\n",
    "$$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m}(Y^{T}Y-Y^{T}\\hat{Y} - \\hat{Y}^{T}Y +\\hat{Y}^{T}\\hat{Y})$$\n",
    "\n",
    "from equation 4 $\\hat{Y} = X.W  $\n",
    "\n",
    "$$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m}(Y^{T}Y-Y^{T}XW - (XW)^{T}Y +(XW)^{T}(XW))$$\n",
    "\n",
    "$$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m}(Y^{T}Y-Y^{T}XW - W^{T}X^{T}Y + W^{T}X^{T}XW ) \\label{eq:eq8.1} \\tag{8.1}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4fcfcb3",
   "metadata": {},
   "source": [
    "Our learning problem reduces to that of finding a good set of weights/parameters for our model which minimizes the cost function.\n",
    " \n",
    "$$\\hat{W} \\in \\mathbb{R}^{n+1}$$ \n",
    "\n",
    "$$\\large \\hat{W} = \\text{argmin}_{W}L_{c}(Y,\\hat{Y}) \\quad\\quad \\text{where,}\\quad W \\in \\mathbb{R}^{n+1}      $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c913dd",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "#### Gradient Descent  \n",
    "\n",
    "Inorder to find the minimum value of Cost function we will be using Gradient Descent method. \n",
    "Given a function defined by a set of parameters, gradient descent starts with an initial set of parameter W₀ and iteratively moves toward a set of parameter values that minimize the function. This iterative minimization is achieved using calculus, taking steps η in the negative direction of the function gradient.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8d39784",
   "metadata": {},
   "source": [
    "\n",
    "$\\large W_{t+1} = W_{t} - \\eta \\times \\nabla_{w} L_{C}(Y,\\hat{Y})   \\label{eq:eq10} \\tag{10}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adaf8a68",
   "metadata": {},
   "source": [
    "\n",
    "#### Claculations of Gredient \n",
    "\n",
    "$$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) = \\frac{dL_{C}(\\hat{Y},Y)}{dW}= \\begin{bmatrix}\n",
    "\\frac{\\partial L_{C}(Y,\\hat{Y}) }{\\partial w_{1}} \\\\\n",
    "\\frac{\\partial L_{C}(Y,\\hat{Y}) }{\\partial w_{2}} \\\\\n",
    "\\frac{\\partial L_{C}(Y,\\hat{Y}) }{\\partial w_{2}} \\\\\n",
    "\\vdots\\\\\n",
    "\\frac{\\partial L_{C}(Y,\\hat{Y}) }{\\partial w_{n}} \\\\\n",
    "\\frac{\\partial L_{C}(Y,\\hat{Y}) }{\\partial w_{n+1}} \\\\\n",
    "\\end{bmatrix}  \n",
    "\\label{eq:eq11} \\tag{11}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bfe91cd",
   "metadata": {},
   "source": [
    "#### Partial Diff of Cost Function with respect to Weights\n",
    "\n",
    "$\\large \\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{i}} = \\frac{\\partial  }{\\partial w_{i}} \\left[\\frac{1}{m} \\sum_{k=1}^{m}  \\left ( y^{k} -\\hat{y}^{k}) \\right )^{2}  \\right ]$\n",
    "\n",
    "$\\large = \\frac{1}{m} \\sum_{k=1}^{m}  \\frac{\\partial}{\\partial w_{i}} \\left ( y^{k} -\\hat{y}^{k}\\right)^{2}$\n",
    "\n",
    "$\\large = \\frac{1}{m} \\sum_{k=1}^{m}  \\frac{\\partial}{\\partial \\hat{y}^{k}} \\left ( y^{k} -\\hat{y}^{k}\\right)^{2} \\frac{\\partial \\hat{y}^{k}}{\\partial w_{i}}$\n",
    "\n",
    "$\\large =  -\\frac{2}{m} \\sum_{k=1}^{m} \\left ( y^{k} -\\hat{y}^{k}\\right) \\frac{\\partial \\hat{y}^{k}}{\\partial w_{i}}$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2c3b592",
   "metadata": {},
   "source": [
    "Partial Differentiation of $\\hat{y}^{k}$ with respect to ith weights $w_{i}$\n",
    "\n",
    "$\\large \\frac{\\partial \\hat{y}^{k} }{\\partial w_{i}} =  \\frac{\\partial }{\\partial w_{i}} \\sum_{i=1}^{n+1} (w_{i}x^{k}_{i})  = x^{k}_{i}$\n",
    "\n",
    "Replaceing equation above\n",
    "\n",
    "$\\large \\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{i}} = -\\frac{2}{m} \\sum_{k=1}^{m} \\left ( y^{k} -\\hat{y}^{k}\\right) \\frac{\\partial \\hat{y}^{k}}{\\partial w_{i}} = -\\frac{2}{m} \\sum_{k=1}^{m} (y^{k} -\\hat{y}^{k})x^{k}_{i}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8519b59",
   "metadata": {},
   "source": [
    "#### Expressing gredient interms of matix\n",
    "\n",
    "$\\large \\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{i}} =  -\\frac{2}{m} \\sum_{k=1}^{m}  \\left(y^{k}-\\hat{y}^{k}\\right)x^{k}_{i} =  -\\frac{2}{m} \\sum_{k=1}^{m}  e^{k} x^{k}_{i}$\n",
    "\n",
    "$\\large \\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{i}}  = -\\frac{2}{m} \\left[e^{1}x^{1}_{i} + e^{2}x^{2}_{i} + e^{2}x^{3}_{i} + \\dots + e^{m}x^{m}_{i} \\right ] $\n",
    "\n",
    "$\\large \\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{i}} = -\\frac{2}{m} \\begin{bmatrix} e^{1}& e^{2} & e^{3} &\\dots &e^{m}\\end{bmatrix} \\begin{bmatrix}\n",
    "x^{1}_{i}\\\\\n",
    "x^{2}_{i}\\\\\n",
    "x^{3}_{i}\\\\\n",
    "\\vdots\\\\\n",
    "x^{m}_{i}\\\\\n",
    "\\end{bmatrix}= -\\frac{2}{m} e^{T}X_{i^{th} Column} $\n",
    "\n",
    "$\\nabla_{\\theta} L_{C}(\\hat{Y},Y)  = \\begin{bmatrix}\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{1}} \\\\\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{2}} \\\\\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{2}} \\\\\n",
    "\\vdots\\\\\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{n}} \\\\\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{n+1}} \\\\\n",
    "\\end{bmatrix} \n",
    "= -\\frac{2}{m} e^{T}X $"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a7cf134",
   "metadata": {},
   "source": [
    "\n",
    "#### Claculations of Gredient Using Matrix \n",
    "\n",
    "from eyqution 8.1 we know  \n",
    "\n",
    "$\\large L_{C}(Y,\\hat{Y}) = \\frac{1}{m}(Y^{T}Y-Y^{T}XW - W^{T}X^{T}Y + W^{T}X^{T}XW )$\n",
    "\n",
    "$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) = \\frac{dL_{C}(\\hat{Y},Y)}{dW}$\n",
    "\n",
    "$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) =  \\frac{1}{m} \\frac{d(Y^{T}Y-Y^{T}XW - W^{T}X^{T}Y + W^{T}X^{T}XW)}{dW}$\n",
    "\n",
    "$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) =  \\frac{1}{m} \\left[ -\\frac{d(Y^{T}XW)}{dW} - \\frac{d(W^{T}X^{T}Y)}{dW} + \\frac{d(W^{T}X^{T}XW)}{dW} \\right]$\n",
    "\n",
    "$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) =  \\frac{1}{m} \\left[ -Y^{T}X - (X^{T}Y)^{T} + 2W^{T}X^{T}X \\right]$\n",
    "\n",
    "from equation 4  $\\hat{Y} = X.W  $\n",
    "\n",
    "$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) =  \\frac{1}{m} \\left[ -Y^{T}X - Y^{T}X + 2(XW)^{T}X \\right]$\n",
    "\n",
    "$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) =  \\frac{1}{m} \\left[ -2Y^{T}X + 2\\hat{Y}^{T}X\\right]$\n",
    "\n",
    "$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) =  -\\frac{2}{m} \\left[ (Y^{T}- \\hat{Y}^{T})X  \\right]$\n",
    "\n",
    "$\\large \\nabla_{w} L_{C}(\\hat{Y},Y) =  -\\frac{2}{m} e^{T}X $\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a27909c1",
   "metadata": {},
   "source": [
    "#### Finally Gredient in Matrix format\n",
    "\n",
    "$$ \\nabla_{\\theta} L_{C}(\\hat{Y},Y)  = \\begin{bmatrix}\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{1}} \\\\\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{2}} \\\\\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{2}} \\\\\n",
    "\\vdots\\\\\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{n}} \\\\\n",
    "\\frac{\\partial L_{C}(\\hat{Y},Y) }{\\partial w_{n+1}} \\\\\n",
    "\\end{bmatrix} \n",
    "= -\\frac{2}{m} e^{T}X \\label{eq:eq12} \\tag{12}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5de80ead",
   "metadata": {},
   "source": [
    "##### Steps for numpy Implementation\n",
    "1. Capture No of datapoints(m) and number of indepemdent features(n) from X_data matrix size.\n",
    "2. Create the Matrix X by appending a column of 1 on X_data\n",
    "3. Create a Weights vector (weights )with 0 values of size (n+1)x1\n",
    "4. set iteration_counter = 0 and loss_history = []\n",
    "4. If iteration_counter less then max_iter then go to step 5 else step 11\n",
    "5. Compute Y_predict by multiplying X and weights\n",
    "6. Compute Error vector (e) subtracting Y_predict from Y\n",
    "7. Compute loss using equation 8, and save it in loss_history list\n",
    "8. Update weights using equation 10\n",
    "9. Update iteration_counter and repeat step 4 to 9\n",
    "10. Return Y_predict, loss_history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4d721e56",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import make_regression\n",
    "import  matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a1e857d",
   "metadata": {},
   "source": [
    "### Generating data of shape 1000X5 \n",
    "\n",
    "\n",
    "$\\large m = 1000 \\quad  \\text {and } \\quad n = 5 $\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ca58bc2d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_data.shape (1000, 5),y.shape (1000,)\n",
      "coeffecient [65.30099317 62.37184401 91.55702227 34.77747286 55.15562951], bias 5\n"
     ]
    }
   ],
   "source": [
    "bias_ = 5 \n",
    "x_data,y,coef = make_regression(n_samples=1000,n_features=5,n_informative=5,bias=bias_,coef=True)\n",
    "print(f'x_data.shape {x_data.shape},y.shape {y.shape}')\n",
    "print(f'coeffecient {coef}, bias {bias_}' )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abeb251f",
   "metadata": {},
   "source": [
    "#### Creating Model Class default max_iteration_count is 100 and learnng rate is 0.001\n",
    "\n",
    "##### max_iteration= 1000 \n",
    "##### eta = 0.001\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f130e289",
   "metadata": {},
   "outputs": [],
   "source": [
    "class LinearRegression:\n",
    "    \n",
    "    def __init__(self, iteration = 1000,lr = 0.001):\n",
    "        self.weights = None\n",
    "        self.max_iteration = iteration\n",
    "        self.lr = lr\n",
    "    \n",
    "    def fit(self,x,y):\n",
    "        no_of_obs,no_of_fea = x.shape               #step - 1\n",
    "        x = np.column_stack((x,np.ones(no_of_obs))) #step - 2\n",
    "        self.weights = np.zeros(no_of_fea+1)        #step - 3\n",
    "        i , history = 0, []                         #step - 4\n",
    "        while i < self.max_iteration:               #step - 5\n",
    "            y_pred = self.forward_pass(x)           #step - 6\n",
    "            error_vec = y - y_pred                  #step - 7\n",
    "            loss = self.compute_loss(error_vec)     #step - 8\n",
    "            history.append(loss)\n",
    "            grediant = self.compute_grediant(x,error_vec)  #step - 9\n",
    "            self.weights = self.weights + self.lr*grediant #step - 9\n",
    "            i += 1                                         #step - 10\n",
    "        print(f'Loss : {loss}')    \n",
    "        return self.forward_pass(x),history    \n",
    "        \n",
    "    def forward_pass(self,x ):\n",
    "        return x.dot(self.weights)\n",
    "    \n",
    "    def compute_grediant(self,x,error_vec):\n",
    "        no_of_obs,_ = x.shape\n",
    "        return (2/no_of_obs)*error_vec.T.dot(x)\n",
    "    \n",
    "    def compute_loss(self,error_vec):\n",
    "        ssd = error_vec*error_vec\n",
    "        return np.mean(np.sqrt(ssd))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1072e84",
   "metadata": {},
   "source": [
    "#### Running the model for 3000 iteration "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6cf14141",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss : 0.5078941397041853\n",
      "Final iteration loss 0.5078941397041853\n",
      "Model Parameter [64.96124197 62.00281208 91.16393802 34.77883773 54.91523277  4.92118004]\n",
      "Expected Parameter [65.30099317 62.37184401 91.55702227 34.77747286 55.15562951], bias 5\n"
     ]
    }
   ],
   "source": [
    "regessor = LinearRegression(iteration=3000)\n",
    "y_pred,loss_history = regessor.fit(x_data,y)\n",
    "\n",
    "print(f'Final iteration loss {loss_history[-1]}')\n",
    "print(f'Model Parameter {regessor.weights}')\n",
    "print(f'Expected Parameter {coef}, bias {bias_}')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d0a1c5ce",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABCsAAAE9CAYAAAAifOhwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABRnklEQVR4nO3deXxU1f3/8dcnK/smEHZBwRU3RNSqLa5VsFJbtdrNqq1t7WK3X4u1/WoXrd33TVtbba1LW61WqLvRuou4IKIVAQUEUVFkkSXJ+f2RCwaSQJAkd5K8no/HPObeM3cm7xzHyeUz55wbKSUkSZIkSZIKRVHeASRJkiRJkuqyWCFJkiRJkgqKxQpJkiRJklRQLFZIkiRJkqSCYrFCkiRJkiQVFIsVkiRJkiSpoJTkHaAl9e3bNw0fPjzvGC1u5cqVdO3aNe8YHYJ93Trs59ZjX7eeQuvrRx555JWUUr+8c3QELXE+Umjvp0Jgn9RnnzTMfqnPPqnPPqmvJfpkc+cj7bpYMXz4cKZNm5Z3jBZXWVnJ+PHj847RIdjXrcN+bj32desptL6OiOfzztBRtMT5SKG9nwqBfVKffdIw+6U++6Q++6S+luiTzZ2POA1EkiRJkiQVFIsVkiRJkiSpoFiskCRJkiRJBcVihSRJkiRJKigWKyRJkiRJUkGxWCFJkiRJkgqKxQpJkiRJklRQLFZIkiRJkqSCYrFCkiRJkiQVFIsVkiQVsMfnv85j81/PO4YKyINzXuWhxVV5x5AkqUWV5B1AkiTV98bqdYz97m2sraoBYN5FE3NOpELxh3vm8sictfy/lIiIvONIktQiHFkhSVKB+ca/ZrDn+bdsKFT8/VMH5pxIheTduw9g6erEkwvfyDuKJEktxpEVkiQViHuefYUP//HBDfuffNcOnHPMrjkmUiE6fJf+BHDLU4vZY0jPvONIktQiLFZIkpSz11auZZ/v3Lphf7uuZdz91UPpWu6fadXXu2sZO/cp4uaZi/nyUTvnHUeSpBbhWZAkSTlJKfGVvz/BP6cv2NB2/WcOYq+hvfILpTZh3/4lXPH0Cua+spIRfbvmHUeSpGaX+5oVEVEcEY9GxI3Z/oiIeDAiZkfE1RFRlrWXZ/uzs8eH5xpckqRtcPuslxhxztQNhYovHrET8y6aaKFCTbJPRTEAtz61OOckkiS1jNyLFcDZwKw6+98HfppSGgm8BpyRtZ8BvJa1/zQ7TpKkNuXl5WsYPnkKZ1w2DYChfTrz9HeO5uwjRuWcTG1J385F7D6oB7fMfCnvKJIktYhcixURMQSYCPwh2w/gMOAf2SGXAe/Ntidl+2SPHx5er0uS1EZU1SSGT57CfhfctqHtP2cfwn+/ehidSotzTKa26qjdBvDIC6/x8vI1eUeRJKnZ5T2y4mfAV4GabH874PWUUlW2vwAYnG0PBuYDZI8vy46XJKmgDZ88hY/fsmrD/heOGMW8iyay68AeOaZSW/fu0RWkBLfNcnSFJKn9yW2BzYg4FliSUnokIsY34+ueCZwJUFFRQWVlZXO9dMFasWJFh/g9C4F93Trs59ZjX7esq55ey03z1m3UdvGRXSgrfpHKyhdzSqX2YueK7gzr04VbZi7mlHHD8o4jSVKzyvNqIAcBx0XEBKAT0AP4OdArIkqy0RNDgIXZ8QuBocCCiCgBegKvbvqiKaWLgYsBxo4dm8aPH9/Sv0fuKisr6Qi/ZyGwr1uH/dx67OuWsWzVOvb69i0btb1nh1J+eeZROSVSexQRHLVbBZff/zwr1lTRzUvdSpLakdymgaSUzkkpDUkpDQdOBu5IKX0IuBM4ITvsVOD6bPuGbJ/s8TtSSqkVI0uStEXDJ0+pV6iYd9FE3r9TWU6J1J4dtfsA1lbXUPnMkryjSJLUrAqxBP814KqI+C7wKPDHrP2PwF8iYjawlNoChyRJBeGDlzzAfc9tPOBv1rePpnOZi2eq5ey7fW+261rGLTNf4tg9B+UdR5KkZlMQxYqUUiVQmW3PAcY1cMxq4MRWDSZJ0hYsXraaA753+0Ztp4wbxvfet0dOidSRFBcFR+xawdQZi1hTVU15icUxSVL7UBDFCkmS2qLhk6fUa5t30cQckqgjO3qPAVw9bT73PPsKh+9akXccSZKahcUKSZK20ohzprDpqklPf+doOpX6rbZa30E79qVHpxKmzFhksUKS1G7ktsCmJEltzbMvLWf45I0LFSfuO4R5F020UNGGRcSlEbEkIp6s09YnIm6NiGez+95Ze0TELyJidkQ8ERFj8kteq6ykiKN2H8CtT73EmqrqvONIktQsLFZIkrQFKSWGT57CkT+9e6P2eRdN5Icn7pVTKjWjPwNHb9I2Gbg9pTQKuD3bBzgGGJXdzgR+20oZN2viHgNZvrqKe2e/kncUSZKahdNAJEnajIbWpfjfd4+hrMR6f3uRUro7IoZv0jwJGJ9tX0btQuBfy9ovzy6f/kBE9IqIgSmlRa0Ut0EHjexL904lTJ2xmMN2cSqIJKnt80xLkqQGTH/htXqFis8dNpJ5F020UNExVNQpQCwG1lcABgPz6xy3IGvLVVlJEUfuVsEtMxeztqom7ziSJG0zR1ZIklRHSokR50yt1+5VPjqulFKKiLTlI98SEWdSO02EiooKKisrmzXTihUr6r3mMKp4Y3UVv7vuDvbs1/FO8Rrqk47OPmmY/VKffVKffVJfa/dJx/tLJklSIxqa8jHnwgkUFUUOaZSzl9ZP74iIgcCSrH0hMLTOcUOyto2klC4GLgYYO3ZsGj9+fLOGq6ysZNPXPLCqmj/OvI359OPz4zveWioN9UlHZ580zH6pzz6pzz6pr7X7xHGskqQO746nX6pXqPjOpN2Zd9FECxUd1w3Aqdn2qcD1ddo/ml0V5ABgWd7rVaxXXlJcOxXkqZdYV+1UEElS2+bICklSh+WUDwFExJXULqbZNyIWAOcBFwHXRMQZwPPASdnhU4EJwGxgFXBaqwfejAl7DOTaRxdy7+xXGL9z/7zjSJL0tlmskCR1SA1N+Zj7vQlEOJKio0kpndLIQ4c3cGwCPtOyid6+Q3bqS7fyEv4zY7HFCklSm+Y0EElSh3Lh1Fn1ChW/+dAY5l000UKF2rzykmKO2LU/Nz+12KkgkqQ2zZEVkqQOobomsePXnfKh9m/CHgP512MvOhVEktSmWayQJLV7TvlQR/KunfvRvVMJNzz+osUKSVKbZbFCktRuHf2zu3l68fKN2r54xE6cfcSonBJJLa+8pJgJowdy4xMvsvr4ajqVFucdSZKkreaaFZKkdufNtdUMnzylXqFi3kUTLVSoQ5i09yBWrq3m9llL8o4iSdLb4sgKSVK74pQPCfbfYTv6dy/nhscXMnHPgXnHkSRpq1mskCS1Cw0VKX5wwp6cNHZoDmmkfBUXBcfuOYi/PvA8y95cR8/OpXlHkiRpqzgNRJLUpi1dubbBQsW8iyZaqFCHNmnvQaytruHmJxfnHUWSpK3myApJUpvVWJFCEuw5pCfDt+vC9Y8v5KT9LNxJktoWixWSpDanoSLFpR8by2G7VOSQRipMEcFxew/ml3c8y5I3VtO/R6e8I0mS1GROA5EktRlzX1nZ6GgKCxVSfcftNYiU4N9PLMo7iiRJW8WRFZKkNsEpH9LWG9m/G7sP6sENjy3kjINH5B1HkqQms1ghSSpoDRUprjvrHewzrHcOaaS2Z9Leg7hw6tPMfWUlI/p2zTuOJElN4jQQSVJBuvt/Lzc6msJChdR079lrEBFw/WML844iSVKTObJCklRwnPIhNZ+BPTtzwIjtuO7RhZx9+CgiIu9IkiRtkcUKSVLBaKhIcedXxjt0XdpG7993CF/5++NMe/419hveJ+84kiRtkdNAJEm5+8XtzzY6msJChbTtjhk9gC5lxfxj2oK8o0iS1CSOrJAk5copH1LL61pewjGjBzJlxiLOP253OpcV5x1JkqTNslghScpFQ0WK6d88kj5dy3JII7V/7993MP+cvoBbnlrMpL0H5x1HkqTNchqIJKlVnXrpQ42OprBQIbWcA0Zsx+BenfnHI04FkSQVPkdWSJJaRUqJEedMrdfulA+pdRQVBe8bM5hf3zmbxctWM6Bnp7wjSZLUKIsVkqQW19BIiqe/czSdSp03L7Wm940Zwi/vmM11jy7k0+N3zDuOJEmNslghSWoxDRUpwNEUUl5G9O3Kvtv35p/TF/Cpd+1AROQdSZKkBrlmhSSp2dXUpEbXpbBQIeXr/WOGMHvJCp5YsCzvKJIkNcqRFZKkZtVQkWLOhRMoKvIbXKkQTNxzIN/690z+OX0Bew3tlXccSZIa5MgKSVKzGD55SqOjKSxUSIWjZ+dSjtp9ANc/9iKr11XnHUeSpAZZrJAkbZO1VTVO+ZDamJP3G8qyN9dx88zFeUeRJKlBTgORJL1tDRUp5n5vgov2SQXuwB22Y1ifLlz50AtM2ntw3nEkSarHYoUkaat5lQ+pbSsqCj6w31B+ePMzzH1lJSP6ds07kiRJG3EaiCSpyVasqXLKh9ROnLjvEIqLgqsfnp93FEmS6nFkhSSpSbzKh9S+9O/RicN26c8/HlnAl4/aidJiv8OSJBUO/ypJkjaroat89OhU4lU+pHbglHFDeWXFGm6ftSTvKJIkbcSRFZKkBi1a9iYHfu+Oeu1O95Daj3eO6seAHp246uEXOHr0gLzjSJK0gcUKSVI9XuVD6hhKios4aewQfnnnbBa+/iaDe3XOO5IkSYDTQCRJdTQ05ePQnfsx76KJFiqkdurEsUMB+Ps0F9qUJBWO3IoVETE0Iu6MiKciYmZEnJ2194mIWyPi2ey+d9YeEfGLiJgdEU9ExJi8sktSe/P4/NcbvcrHn04bl0MiSa1laJ8uHDyyL9c8PJ/qmpR3HEmSgHxHVlQBX04p7QYcAHwmInYDJgO3p5RGAbdn+wDHAKOy25nAb1s/siS1P8MnT2HSr+/dqG3u9ya4NoXUgXxw3DBeXLaaymdcaFOSVBhyK1aklBallKZn28uBWcBgYBJwWXbYZcB7s+1JwOWp1gNAr4gY2LqpJan9GD55Ch+7aeVGbf/v3Ts75UPKRMQXs9GfT0bElRHRKSJGRMSD2UjPqyOiLO+czeGI3Sqo6FHOXx54Pu8okiQBBbLAZkQMB/YBHgQqUkqLsocWAxXZ9mCg7mTKBVnbojptRMSZ1I68oKKigsrKyhbLXShWrFjRIX7PQmBftw77uWXds3Adf5ixtl77n4/uCiygsnJB64fqAHxfty0RMRj4PLBbSunNiLgGOBmYAPw0pXRVRPwOOIN2MNqztLiIU8YN4+e3P8vzr65k++265h1JktTB5V6siIhuwD+BL6SU3qj7bV5KKUXEVk2eTCldDFwMMHbs2DR+/PhmTFuYKisr6Qi/ZyGwr1uH/dxyGluXQi3P93WbVAJ0joh1QBdqvyA5DPhg9vhlwPm0g2IFwCnjhvGrO2ZzxYMv8PUJu+YdR5LUweVarIiIUmoLFVeklK7Nml+KiIEppUXZNI/1kycXAkPrPH1I1iZJ2oKGihR/+OhYSpbMyiGNVPhSSgsj4kfAC8CbwC3AI8DrKaWq7LD1ozzbhYoenXj37gO4Ztp8vnTkTnQqLc47kiSpA8utWBG1Qyj+CMxKKf2kzkM3AKcCF2X319dp/2xEXAXsDyyrM11EktSAr/7jca6ZVn9ax/rRFJUWK6QGZVcjmwSMAF4H/g4cvRXPb9FpqS01rWiPztVMWbWOH159B4cMKW32129JTrWqzz5pmP1Sn31Sn31SX2v3SZ4jKw4CPgLMiIjHsravU1ukuCYizgCeB07KHptK7TzR2cAq4LRWTStJbUhKiRHnTK3X7pQPqcmOAOamlF4GiIhrqT136RURJdnoikZHebb0tNSWmlb0rpS49vm7efj1Yr754YOb/fVbklOt6rNPGma/1Gef1Gef1NfafZJbsSKldA/Q2HLzhzdwfAI+06KhJKkdaGjKx78/ezB7DOmZQxqpzXoBOCAiulA7DeRwYBpwJ3ACcBUbjwBtFyKCjxywPd+8fiaPzX+dvYf2yjuSJKmDyu3SpZKk5rXn+Tc3uoCmhQpp66SUHgT+AUwHZlB7znQx8DXgSxExG9iO2imt7cp79xlM17Ji/nK/lzGVJOUn96uBSJK2jVM+pJaRUjoPOG+T5jnAuBzitJrunUp535ghXD1tPudO3JU+XcvyjiRJ6oAsVkhSG9bQSIp7vnYoQ3p3ySGNpPbiIwduz18eeJ6rHn6Bs8aPzDuOJKkDslghSW1QQ0UKcDSFpOaxU0V3Dhq5HZff9zyfOGQHSoudOSxJal3+5ZGkNqSmJjW6LoWFCknN6YyDR7D4jdVMneGV4iVJrc+RFZLURjRUpJj5rXfTtdyPcknNb/xO/dmhb1cuvWcux+01iIjGLuImSVLzc2SFJBW44ZOnNDqawkKFpJZSVBScdtBwHl+wjEeefy3vOJKkDsZihSQVqHXVNU75kJSr9+87hJ6dS/njPXPzjiJJ6mD8Sk6SClBDRYpnLzjGRe4ktaouZSWcMm4YF9/9HPOXrmJoH680JElqHZ71SlIB2dyUDwsVkvJw6ju2pyiCy+6bl3cUSVIHssUz34joGhFF2fZOEXFcRJS2fDRJ6jiWr17nlA9JBWlgz85M2GMgVz88nxVrqvKOI0nqIJoyDeRu4JCI6A3cAjwMfAD4UEsGk6SOoqEixZwLJ1BU5Mr7kgrD6QeP4IbHX+Sah+dz+sEj8o4jSeoAmjKmOFJKq4D3Ab9JKZ0I7N6ysSSp/dvclA8LFZIKyd5DezF2+9788Z65rKuuyTuOJKkDaFKxIiIOpHYkxfqz6uKWiyRJ7du8V1Y65UNSm/PJd+3IwtffZMoTi/KOIknqAJoyDeQLwDnAdSmlmRGxA3Bni6aSpHbKKR+S2qrDd+nPqP7d+N1dzzFp70FE+LklSWo5WyxWpJTuAu6KiC7Z/hzg8y0dTJLak4aKFIAjKSS1GUVFwSfftSNf+fvjVD7zMofu0j/vSJKkdqwpVwM5MCKeAp7O9veKiN+0eDJJagdue+olp3xIajeO22sQg3p24rd3PZd3FElSO9eUaSA/A94N3ACQUno8It7ZkqEkqT1oqEgx93sTHDotqc0qKynijEN24Ds3PsUjz7/Gvtv3zjuSJKmdasoCm6SU5m/SVN0CWSSpXWjoKh9jhvVi3kUTLVRIavNO3m8ovbqU8jtHV0iSWlBTRlbMj4h3ACkiSoGzgVktG0uS2p7v/WcWv79rTr12p3tIak+6lpfw0QOH84vbn2X2kuWM7N8970iSpHaoKSMrPgV8BhgMLAT2zvYlSUBKieGTp9QrVLguhaT26mPvGE6n0iJ+10CBVpKk5tCUq4G8AnyoFbJIUpvT0LoUpx80gv97z245pJGk1tGnaxkn7zeMvz7wPF84YhRDenfJO5IkqZ3ZYrEiIv4EpE3bU0qnt0giSWoDjv/NvTz6wuv12h1JIamj+OS7duBvD77Abyuf44Lj98g7jiSpnWnKmhU31tnuBBwPvNgycSSpsKWUGHHO1HrtFikkdTQDe3bmxLFDuGbafD5z6EgG9eqcdyRJUjvSlGkg/6y7HxFXAve0WCJJKlANTfn4wQl7ctLYoTmkkaT8nXXoSK6ZNp/f3fUc3540Ou84kqR2pCkjKzY1Cujf3EEkqVA1VKQAR1NI0uBenTlh3yFc9dB8zho/kgE9O+UdSZLUTmzxaiARsTwi3lh/D/wb+FrLR5OkfFXXpAYLFV7lQ5Lectb4kdSkxO/uei7vKJKkdqQp00C8eLakDqehIsV1Z72DfYb1ziGNJBWuoX268L4xg/nbQy9w1vgd6d/D0RWSpG3XaLEiIsZs7okppenNH0eS8uWUD0naep89dBT/nL6Q3901x0s3S5KaxeZGVvx4M48l4LBmziJJuVlTVc3O37ipXrtFCknasmHbdeH4fQZzxYPP86nxO9C/u6MrJEnbptFiRUrp0NYMIkl5aWg0xf3nHMbAnl6GT5Ka6rOHjuS6Rxfymzuf4/zjds87jiSpjWvS1UAiYjSwG7ChTJ5SurylQklSa3DKhyQ1n+F9u3LivkP424Mv8Il37sDgXhZ8JUlvX1OuBnIe8MvsdijwA+C4Fs4lSS1m5Zoqr/IhSS3g84ePAuAXtz2bcxJJUlvXlJEVJwB7AY+mlE6LiArgry0bS5JaRkNFipnfejddy5s00EyStBmDenXmwwdsz2X3z+PMd+3Ajv265R1JktRGbXFkBfBmSqkGqIqIHsASYGjLxpKk5jV88pRGR1NYqJCk5nPWoTtSXlLET2/9X95RJEltWFOKFdMiohdwCfAIMB24vyVDSVJzeXn5Gqd8SFIr6tutnDMOHsGNTyxi5ovL8o4jSWqjtvh1YkrprGzzdxFxE9AjpfREy8aSpG3XUJFi9gXHUFLclDqtJOnt+vghO3D5/c/z41v+x6Uf2y/vOJKkNmiLxYqIuAG4Crg+pTSvxRNJ0jbyKh+SlK+enUv55Lt24Ac3PcO0eUsZO7xP3pEkSW1MU75e/DFwMPBURPwjIk6IiE5bepIktbbZS5Y75UOSCsTH3jGcvt3K+f5NT5NSyjuOJKmNaco0kLuAuyKiGDgM+ARwKdCjhbNJUpM1VKSY+70JREQOaSS1B9maXX8ARgMJOB14BrgaGA7MA05KKb2WT8LC1qWshC8eOYpzr3uSW556iXfvPiDvSJKkNqRJE7cjojPwfuBTwH7AZS0ZSpKaanNX+bBQIWkb/Ry4KaW0C7WXcZ8FTAZuTymNAm7P9tWID4wdysj+3bjoP0+zrrom7ziSpDZki8WKiLiG2j/OhwG/AnZMKX2upYNJ0uZMnbHIKR+SWkxE9ATeCfwRIKW0NqX0OjCJt760uQx4bx752oqS4iK+PmEX5r6ykr89+ELecSRJbcgWp4FQ+0f6lJRSdUuHkaSmcMqHpFYwAngZ+FNE7EXt5dvPBipSSouyYxYDFQ09OSLOBM4EqKiooLKyslnDrVixotlfs6VESuzap4gf/mcm/VbOpUtpy3xWt6U+aS32ScPsl/rsk/rsk/pau0+asmbFza0RRJK2pKEiRUlRMPvCCTmkkdTOlQBjgM+llB6MiJ+zyZSPlFKKiAZXjkwpXQxcDDB27Ng0fvz4Zg1XWVlJc79mS+q30zLe86t7eLxqAOccuWuL/Iy21ietwT5pmP1Sn31Sn31SX2v3SZPWrJCkPP3gpqcbnfJhoUJSC1kALEgpPZjt/4Pa4sVLETEQILtfklO+NmX04J4cv89g/nTvPOYvXZV3HElSG9DmihURcXREPBMRsyPCRa2kdiylxPDJU/hN5XMbtbsuhaSWllJaDMyPiJ2zpsOBp4AbgFOztlOB63OI1yZ95aidCeBHtzyTdxRJUhvQ6DSQiBizuSemlKY3f5zNyy6f+mvgSGq/8Xg4Im5IKT3V2lkktayGRlK8d+9B/OzkfXJII6mD+hxwRUSUAXOA06j9oueaiDgDeB44Kcd8bcqgXp35xCE78Ks7Z3PqO4YzZljvvCNJkgrY5tas+HF23wkYCzwOBLAnMA04sGWjNWgcMDulNAcgIq6idlVuixVSOzHh5//lqUUr67U7kkJSa0spPUbtOdCmDm/lKO3Gp8bvyDXT5nP+DTP511kHUVTkwsiSpIY1Og0kpXRoSulQYBEwJqU0NqW0L7APsLC1Am5iMDC/zv6CrE1SG7d+ysdTi97YqN0pH5KaKiLOjogeUeuPETE9Io7KO5fe0q28hK9P2JUnFizj74/M3/ITJEkdVlMuXbpzSmnG+p2U0pMR0TLLODeDlr5UWCHysjqtx75uGR+7qf5IitNGl/GuIaX2dwvzPd167OtWcXpK6ecR8W6gN/AR4C/ALfnGUl2T9h7EFQ8+zw9ueoajRw+kZ+fSvCNJkgpQU4oVT0TEH4C/ZvsfAp5ouUibtRAYWmd/CJuM8mjpS4UVIi+r03rs6+bV0LoUAH8+uqv93Ep8T7ce+7pVrJ9TMAH4S0ppZkQ4z6DARATnH7c77/nlPfz01v9x/nG75x1JklSAmnI1kNOAmcDZ2e2prC0PDwOjImJEttjVydSuyi2pDVk/5WNTTvmQtI0eiYhbqC1W3BwR3YGanDOpAbsP6skp44bxlwee55nFy/OOI0kqQFscWZFSWh0RvwOmppRyvdZUSqkqIj4L3AwUA5emlGbmmUnS1mmoSHHDZw9izyG9Wj+MpPbmDGBvYE5KaVVE9CG/L1i0BV85amemzFjE+TfM5G+f2B8HwUiS6triyIqIOA54DLgp2987InIbzZBSmppS2imltGNK6YK8ckjaOsMnT2l0NIWFCknN5EDgmZTS6xHxYeAbwLKcM6kRvbuW8eWjdub+Oa8yZcaivONIkgpMU6aBnEftJUNfhw2X8RrRcpEktSc1NU75kNRqfgusioi9gC8DzwGX5xtJm/PBccPYfVAPvv3vp3hj9bq840iSCkhTihXrUkqbfiuRWiKMpPZl+OQp7PD1qRu1PfT1wy1SSGopVSmlBEwCfpVS+jXQPedM2oziouDC4/fg5RVr+PHNuc42liQVmKZcDWRmRHwQKI6IUcDngftaNpaktqyxq3xYpJDUwpZHxDnUXrL0kIgoArwuZoHba2gvTj1wOJfdP4/3jRnCXkN75R1JklQAmjKy4nPA7sAa4G/Uzv08uyVDSWqb1lRVO+VDUp4+QO35yukppcXUXuL8h/lGUlN8+aid6N+9nHOunUFVtRdwkSQ1bWTFxJTSucC56xsi4kTg7y2WSlKb01CRYua33k3X8qZ8zEjStkspLY6IK4D9IuJY4KGUkmtWtAHdO5Vy/nt259NXTOfP983j44fskHckSVLOmjKy4pwmtknqgDZ3lQ8LFZJaU0ScBDwEnAicBDwYESfkm0pNdfToARy+S39+fMv/WPDaqrzjSJJy1ui/JCLiGGACMDgiflHnoR5AVUsHk1TYlq1ax17fvqVeu9M9JOXoXGC/lNISgIjoB9wG/CPXVGqSiOBbk3bnyJ/czXnXz+QPp44lIvKOJUnKyea+9nwRmAYcBzxSp3058MWWDCWpsDU0kmL2BcdQUtyUwVqS1GKK1hcqMq/StFGkKhBDenfhS0fuxAVTZ3HD4y8yae/BeUeSJOWk0WJFSulx4PGIuA5YmVKqBoiIYqC8lfJJKiBe5UNSgbspIm4Grsz2PwBM3czxKkCnHzyCKTMWcd4NM3nHjn3p193TTknqiJrybcMtQOc6+52pHVIpqYOYv3SVV/mQVPBSSv8PuBjYM7tdnFL6Wr6ptLWKi4Ifnbgnq9ZW881/PUlKKe9IkqQcNGX1u04ppRXrd1JKKyKiSwtmklRAGipSzLlwAkVFziOWVHhSSv8E/pl3Dm2bkf2788UjduL7Nz3NlBmLOHbPQXlHkiS1sqYUK1ZGxJiU0nSAiNgXeLNlY0nKm1M+JLUVEbEcaOjr9wBSSqlHK0dSM/jEISO46clF/N/1Mzlwh+3YrpvTQSSpI2lKseILwN8j4kVq/+gPoHYOqKR26OF5Sznxd/fXa7dIIalQpZS6551Bza+kuIgfnrgXx/7iHv7vhpn8+oNj8o4kSWpFWyxWpJQejohdgJ2zpmdSSutaNpakPDQ0mmLu9yZ46ThJUi52qujO2UeM4oc3P8PEPRYxYY+BeUeSJLWSRosVEXFYSumOiHjfJg/tFBGklK5t4WySWklDRYpeXUp57P+OyiGNJElv+eQ7d+DmmYv5+nUz2Hf73lT06JR3JElSK9jc1UDeld2/p4HbsS2cS1IruHb6gkav8mGhQpJUCEqKi/jpB/Zm9bpqvvL3x6mp8eogktQRNDqyIqV0XnZ/WuvFkdRanPIhSWorduzXjW9M3I1v/OtJ/nzfPE4/eETekSRJLWxz00C+tLknppR+0vxxJLW0hooUJ+w7hB+duFcOaSRJapoP7T+MO59ewkU3Pc1BI/uy8wDXVZWk9mxz00C6Z7exwKeBwdntU4DLMUttzDnXzmh0yoeFCklSoYsIvn/CnvToVMLZVz3KmqrqvCNJklrQ5qaBfAsgIu4GxqSUlmf75wP1/8UjqWA1VqSQJKkt6dutnO+/f0/OuGwaP7r5Gc6duFvekSRJLWSLly4FKoC1dfbXZm2SClxDRYofnbgXJ+w7JIc0kiRtu8N3reBD+w/jkv/O5eBR/XjXTv3yjiRJagFNKVZcDjwUEddl++8FLmuxRJK22djv3sYrK9bUa3c0hSSpPfjGxN2YNu81vnT1Y0w9+xAvZypJ7dDm1qwAIKV0AXAa8Fp2Oy2ldGFLB5O09VJKDJ88pV6hYt5FEy1USJLajc5lxfz6Q/uwam01n7/yUaqqa/KOJElqZk0ZWQHQBXgjpfSniOgXESNSSnNbMpikrdPQlI/rznoH+wzrnUMaSZJa1sj+3fnue0fz5b8/zs9vf5Z9y/JOJElqTlssVkTEedReEWRn4E9AKfBX4KCWjSapKRoqUoBTPiRJ7d/79x3CA3Ne5Vd3zubL+3ZifN6BJEnNpikjK44H9gGmA6SUXowIL2wt5SylxIhzptZrt0ghSepIvj1pNI8veJ3fP7GCk45aTX/Xr5CkdmGLa1YAa1NKCUgAEdG1ZSNJ2pLhk6fUK1TcN/kwCxWSpA6nc1kxv/7gGNZUw2evfJR1rl8hSe1CU4oV10TE74FeEfEJ4DbgkpaNJakhwydPaXDax7yLJjKoV+ccEkmSlL9RFd05bfdyHpq7lAunzso7jiSpGWx2GkhEBHA1sAvwBrXrVvxfSunWVsgmKbOuuoZR5/6nXrsjKSRJqnXgoBLWdR/EpffOZc8hPTl+nyF5R5IkbYPNFitSSikipqaU9gAsUEg5aGgkxRPnH0WPTqU5pJEkqXCdM2EXnlq0jMn/nMGo/t0ZPbhn3pEkSW9TU6aBTI+I/Vo8iaSNbG7Kh4UKSWodEVEcEY9GxI3Z/oiIeDAiZkfE1RHhBTMLSGlxEb/64Bj6dC3jk395hKUr1+YdSZL0NjWlWLE/8EBEPBcRT0TEjIh4oqWDSR3V8tXrGi1SOO1Dklrd2UDdRRC+D/w0pTQSeA04I5dUalTfbuX87sP78vKKNXzuyulUueCmJLVJTbl06btbPIUkoOEpH89892jKS4pzSCNJHVtEDAEmAhcAX8rW8joM+GB2yGXA+cBvcwmoRu01tBcXvHc0/+8fT3DB1Fmc957d844kSdpKjRYrIqI/8HVgJDAD+F5K6Y3WCiZ1JA0VKcAFNCUpZz8Dvgp0z/a3A15PKVVl+wuAwTnkUhOcOHYosxYt59J757JDv2585IDt844kSdoKmxtZcTnwCPBL4FjgF8DHWiGT1GEseG0VB3//znrtFikkKV8RcSywJKX0SESMfxvPPxM4E6CiooLKyspmzbdixYpmf822rqE+Obhb4pF+xZx3/ZMsW/Aso/s2ZVBx++H7pGH2S332SX32SX2t3Seb+8QemFI6N9u+OSKmt0YgqaNoaDTF7AuOoaS4KUvJSJJa2EHAcRExAegE9AB+DvSKiJJsdMUQYGFDT04pXQxcDDB27Ng0fvz4Zg1XWVlJc79mW9dYn4x7RxUn/PY+fj/jTa49a39GVXSv/+R2yvdJw+yX+uyT+uyT+lq7Tzb7r6KI6B0RfSKiD1C8yb6kt2FzV/mwUCFJhSGldE5KaUhKaThwMnBHSulDwJ3ACdlhpwLX5xRRTdStvIQ/fmw/ykuLOf2yh3l1xZq8I0mSmmBz/zLqSe00kPW3HsD0bHtay0eT2pcH57zqVT4kqe37GrWLbc6mdg2LP+acR00wuFdn/nDqWJa8sYYz//IIq9dV5x1JkrQFjU4Dyb5JkNQMGipSzLlwAkVFkUMaSdLWSClVApXZ9hxgXJ559PbsPbQXPzlpbz7zt+mcfdWj/OZD+1Ls32FJKlgda5UhqZU1VKTo2bmUx887Koc0kiR1bBP3HMhLb+zGt298im9e/yQXvHc0tVeklSQVGosVUguYOmMRZ11Rf01ap3tIkpSv0w8ewZLla/jdXc/Rv3s5Xzhip7wjSZIa0GixIiKmAmellOa1XhypbUspMeKcqfXa535vgt/cSJJUIL529M68vHwNP7vtWfp1L+dD+2+fdyRJ0iY2N7LiT8AtEXEZ8IOU0rpWyiS1SQ1N+XjPXoP45Sn75JBGkiQ1JiK46P17sHTlGr75ryfp262cd+8+IO9YkqQ6Gr0aSErp78AYaq8CMi0ivhIRX1p/a7WEUoG7cOqsRq/yYaFCkqTCVFpcxK8/NIY9h/Tic1c+yj3PvpJ3JElSHVtas2ItsBIoB7oDNS2eSGojGpvy4boUkiS1DV3KSvjTx/bjlEse4BOXT+PyM8ax3/A+eceSJLGZkRURcTTwGNAFGJNSOi+l9K31t235oRHxw4h4OiKeiIjrIqJXncfOiYjZEfFMRLy7bp6sbXZETN6Wny9tq+GTp9QrVHz3vaMtVEiS1Mb07lrGX87Yn4G9OnHanx7m8fmv5x1JksRmihXAucCJKaXJKaVVzfxzbwVGp5T2BP4HnAMQEbsBJwO7A0cDv4mI4ogoBn4NHAPsBpySHSu1qm/cs6rRKR8fPsDFuSRJaov6dS/nbx8/gD5dy/jopQ/x1Itv5B1Jkjq8za1ZcUhKaWZL/NCU0i0ppaps9wFgSLY9CbgqpbQmpTQXmA2My26zU0pzUkprgauyY6VWUVOTGD55CgtWpI3a51000dEUkiS1AwN6duKKj+9Pl7JiPvLHB5m9ZHnekSSpQ9vcyIrWcjrwn2x7MDC/zmMLsrbG2qUWN3zyFHb4+sZTPq78xAEWKSRJameG9unC3z5xAEVFwSmXPMizL1mwkKS8bGmBzbctIm4DGroG1LkppeuzY84FqoArmvHnngmcCVBRUUFlZWVzvXTBWrFiRYf4PVvbx25a2WD7n4/uypr5M6ic3+DDaga+p1uPfd167GupbRjRtytXfmJ/TrnkQU6++AH++vH92XVgj7xjSVKH02LFipTSEZt7PCI+BhwLHJ5SWj+2fiEwtM5hQ7I2NtO+6c+9GLgYYOzYsWn8+PFbG73NqayspCP8nq1lc1f5sK9bh/3ceuzr1mNfS23HyP7dufrMA/jgJQ9yyiUP8Ncz9mf04J55x5KkDiWXaSDZlUa+Chy3yeKdNwAnR0R5RIwARgEPAQ8DoyJiRESUUbsI5w2tnVvtX0NX+bj/nMOc8iFJUgezQ79uXPPJA+laVsIHL3mAx7xKiCS1qrzWrPgV0B24NSIei4jfAWQLel4DPAXcBHwmpVSdLcb5WeBmYBZwTUst/qmOafjkKY1e5WNgz845JJIkSXkbtl0Xrv7kAfTsUsqH//Agjzy/NO9IktRhtNg0kM1JKY3czGMXABc00D4VqD82X9oGVdU1jDz3P/XaHUkhSZIAhvTuwjWfPJAPXvIgH/7DQ/z2w2MYv3P/vGNJUruXS7FCKgQNjaSYcf5RdO9UmkMaSZJUqAb27Mw1nzyQUy99iI9fNo0fn7QXk/b2wnSS1JIsVqjDaahIAY6mkCRJjevXvZyrPnkAZ14+jbOveoxXV6zl9INH5B1LktotixXqMNZUVbPzN26q126RQpIkNUWPTqX8+bRxnH3Vo3z7xqd4deUavnLUzkRE3tEkqd2xWKEOoaHRFM9ecAylxXmtMStJktqiTqXF/OZD+/KNf83g13c+x8vL13DB8Xt4TiFJzcxihdo1p3xIkqTmVlwUXHj8HvTrVs4v7pjNwtff5Dcf2peenV33SpKai8UKtUtvrF7HnuffUq/dIoUkSWoOEcGXjtqZYdt15Zxrn+D9v72PP31sP4b26ZJ3NElqFyxWqN1paDTF7AuOocThmZIkqZmdsO8QBvfqzKf++gjv/fW9XPzRsey7fe+8Y0lSm+e/3tRuDJ88pV6hYp9hvZh30UQLFZIkqcUcuON2XHfWO+jeqYRTLnmAfz/+Yt6RJKnN819wavOWrlzb4GiKeRdN5LqzDsohkSRJ6mh26NeNa886iL2H9OJzVz7K9296muqalHcsSWqznAaiNq2hIsWcCydQVOQlxCRJUuvq07WMv358f77175n8tvI5nly4jF+esg+9upTlHU2S2hxHVqhNamjKxyffuQPzLppooUKSJOWmrKSIC47fg4vetwcPzlnKe351D7MWvZF3LElqcyxWqE2Z+8rKRqd8nDNh1xwSSZIk1XfyuGFc9ckDWFtVw/t+cx83uI6FJG0Vp4GozXDKhyRJakvGDOvNvz93MGf9dTqfv/JRHpm3lHMm7Eqn0uK8o0lSwbNYoYLXUJHiJyftxfvGDMkhjSRJUtP1796Jv33iAL5/09P88Z65THv+NX79wTEM79s172iSVNCcBqKC9eTCZY1O+bBQIUmS2oqykiK+eexuXPLRsSx47U2O/eU9TguRpC1wZIUKUmNFCkmSpLbqyN0qmHr2IXzub7XTQu5/7lXOe89uTguRpAZYrFBBaahIcduX3snI/t1zSCNJktS8BvfqzNWfPJAf3fIMv79rDg/NfZWffWAf9hjSM+9oklRQnAaigvDY/NcbHU1hoUKSJLUnpcVFnHPMrvzljHGsXFPN8b+5l1/e/ixV1TV5R5OkguHICuXOKR+SJKkjOmRUP27+wjv55vVP8uNb/8cdzyzhJyftzQgX35QkixXKT0NFimnfOIK+3cpzSCNJktT6enYp5Ren7MMRu1XwjetmMOHn/+XrE3flQ+OGeXl2SR2a00DU6u7638uNjqawUCFJkjqi4/YaxM1ffCdjh/fmm/96kpMveYA5L6/IO5Yk5caRFWpVTvmQJElq2MCenbn89HH8fdoCvjvlKY7++X/5whGj+MQhO1Ba7HeMkjoWixVqFQ0VKZ7+ztFeqkuSVJAiYihwOVABJODilNLPI6IPcDUwHJgHnJRSei2vnGp/IoKT9hvK+J37cd4NM/nBTc9w4+OL+MEJezJ6sFcMkdRxWKJVi7r/uVfrFSp2GdCdeRdNtFAhSSpkVcCXU0q7AQcAn4mI3YDJwO0ppVHA7dm+1Oz69+jEbz+8L7/78BheXrGGSb++l+/c+BTLV6/LO5oktQpHVqhFpJQYcc7Ueu1O+ZAktQUppUXAomx7eUTMAgYDk4Dx2WGXAZXA13KIqA7i6NEDOXCHvlx009Nceu9cbnj8Rc6dsCuT9h5EhAtwSmq/LFao2TU05WPOhRNc0VqS1CZFxHBgH+BBoCIrZAAspnaaiNSienYp5Xvv24OT9xvKN69/ki9c/Rh/e+gFvjNpNDsP6J53PElqERYr1GzuePolTv/ztI3afnziXrx/3yE5JZIkadtERDfgn8AXUkpv1P0mO6WUIiI18rwzgTMBKioqqKysbNZcK1asaPbXbOs6Sp98YffE3T3L+Pv/lnLMz+/miGElTBpZRtfS+l8KdZQ+2Vr2S332SX32SX2t3ScWK7TNnPIhSWqPIqKU2kLFFSmla7PmlyJiYEppUUQMBJY09NyU0sXAxQBjx45N48ePb9ZslZWVNPdrtnUdqU8OA85euZYf3PwMVz38Ag8uCT5/+Cg+csD2lJW8tSRdR+qTrWG/1Gef1Gef1NfafWKxQtukoSkfc783wTmUkqQ2LWr/kP0RmJVS+kmdh24ATgUuyu6vzyGeRO+uZXzvfXvw0QO358Kps/jOjU9x+f3z+NrRu3DM6AGei0lq8yxW6G35z4xFfPqK6Ru1/fPT72Df7XvnlEiSpGZ1EPARYEZEPJa1fZ3aIsU1EXEG8DxwUj7xpFq7DuzBX87Yn7v+9zIXTpnFWVdMZ8ywXpw7cbe8o0nSNrFYoa1SXZPY8esbT/kY2qcz//3qYTklkiSp+aWU7gEa+2r68NbMIjXFu3bqx8Ej+/L3afP58a3/4/2/vY+9+hXTd9QyRg/umXc8SdpqFivUZHucdzPL11Rt1Oa6FJIkSYWhuCg4edww3rPXIP583zx+ffszHPvLezhm9AC+eORO7FThlUMktR0WK7RFt896iTMu2/gqH9O+cQR9u5XnlEiSJEmN6VpewmcOHcnwqvk8w2AuvWcuN81czHv2HMQXjhjFDv265R1RkrbIYoUata66hlHn/mejts8fNpIvHbVzTokkSZLUVF1Lgy+N34nT3jGc3989h8vum8eNT7zIxD0H8el37chug3rkHVGSGmWxQg1q6CofTvmQJElqe3p3LWPyMbtwxsEj+MM9c/jr/c/z78df5NCd+3HWoSPZb3ifvCNKUj0WK7SR/z77Mh/540MbtT1x/lH06FSaUyJJkiQ1h37dyznnmF05610jufz+efzpvnmc+Lv7GTe8D58+dEfG79TPS55KKhgWKwTA6nXV7PLNmzZq+857R/ORA7bPKZEkSZJaQs8upXzu8FGcccgIrnpoPpf8dw6n/elhRvXvxmkHjeD4fQbTuaw475iSOjiLFeLIn9zFs0tWbNg/eGRf/vrx/XNMJEmSpJbWpayE0w8ewYcP2J5/P/4il947l69fN4Mf3Pw0p4wbxkcP3J6BPTvnHVNSB2WxogO76clFfOqv0zdqm33BMZQUF+WUSJIkSa2trKSI9+87hPeNGczD817j0nvm8vu7nuPiu+dwzOgBnHbQcMYM6+0UEUmtymJFB7RyTRW7n3fzRm3XnfUO9hnWO6dEkiRJyltEMG5EH8aN6MP8pau4/P55XPXwfG58YhE7V3TnlHFDOX6fIfTs4lpmklqexYoOZux3b+WVFWs37E/ccyC//uCYHBNJkiSp0Azt04VzJ+7GF47YiX8//iJXPvQC5//7Kb73n6eZuOdAPjhuGPtu72gLSS3HYkUH8egLr3H8b+7bqO25CydQXOQfGEmSJDWsa3kJJ48bxsnjhvHkwmVc+dALXP/Yi1w7fSE7VXTjpLFDOW7vQfTv3invqJLaGYsV7dy66hqO+9W9zFr0xoa2m75wCLsM6JFjKkmSJLU1owf35ILj9+DrE3blxide5G8Pzee7U2Zx4dRZHDKqH+8bM5ijdhvglUQkNQuLFe3YH/47h+9OmbVh/5en7MN79hqUYyJJkiS1dV3LS/jAfsP4wH7DmL1kBdc9uoDrpi/k7Kseo1t5CUePHsD7xgzmgBHbUeQoXklvk8WKduiZxct598/u3rB/1G4V/P4j+zqnUJIkSc1qZP9u/L9378KXj9yZB+cu5drpC/jPk4v5xyMLqOhRzjGjBzJxz4HsO6y3hQtJWyXXYkVEfBn4EdAvpfRK1P5r+ufABGAV8LGU0vTs2FOBb2RP/W5K6bI8Mhey1euqOfpndzPv1VUb2h4693DnEEqSJKlFFRUFB+64HQfuuB3fnjSaW55azJQnFvG3h17gz/fN21C4mLDHQMZub+FC0pblVqyIiKHAUcALdZqPAUZlt/2B3wL7R0Qf4DxgLJCARyLihpTSa62bunD96o5n+dEt/9uwf/FH9uWo3QfkmEiSJEkdUeeyYibtPZhJew9mxZoqbp/1ElNnvFW46N+9nKNHD+CIXSvYf4c+lJe4xoWk+vIcWfFT4KvA9XXaJgGXp5QS8EBE9IqIgcB44NaU0lKAiLgVOBq4snUjF54ZC5bxsZtWArWFiuP3GcxPTtrLKR+SJEnKXbfykgYLF9dMm8/l9z9P17Ji3rlTP47YtYJDd+lPn65leUeWVCByKVZExCRgYUrp8U3+UT0YmF9nf0HW1lh7h/Xm2mre9cM7WbJ8zYa26d880g94SZIkFaS6hYvV66q5d/Yr3DZrCXc8/RL/eXIxRQFjhvXm8F0rOHSXfuxc0d0v4KQOrMWKFRFxG9DQPIRzga9TOwWkJX7umcCZABUVFVRWVrbEj8nVP/63lhvnrNuwf9buiXFDu/HEw/flmKpjWLFiRbt8TxUa+7n12Netx76WpLd0Ki3m8F0rOHzXCmpqRjPzxTe4ddZL3D7rJb5/09N8/6an6de9nENG9uWQnfpy0Mi+rsMmdTAtVqxIKR3RUHtE7AGMANaPqhgCTI+IccBCYGidw4dkbQupnQpSt72ykZ97MXAxwNixY9P48eMbOqxNeuT5pbz/t/dv2P/g/sO44L2jueuuu2hPv2chq6ystK9bgf3ceuzr1mNfS1LDioqCPYb0ZI8hPfnSkTuxaNmb/PfZV/jvs69w5zNLuPbRhQDsMqA779ypH4eM6svY7fvQucy1LqT2rNWngaSUZgD91+9HxDxgbHY1kBuAz0bEVdQusLkspbQoIm4GLoyI3tnTjgLOaeXouVm+eh0Hfu8OVqypAqC0OJj2jSPp2bk052SSJElS8xrYszMnjR3KSWOHUlOTeGrRG9z97Mv893+v8Od753Hx3XMoLQ72GtKLcSP6sP8O27Hv9r3pVp7rhQ4lNbNC+z96KrWXLZ1N7aVLTwNIKS2NiO8AD2fHfXv9Ypvt3bf+PZM/3Ttvw/5VZx7AATtsl18gSZIkqZUUFQWjB/dk9OCenDV+JKvWVvHg3KU8MOdVHpyzlN/fPYffVD5HcVEwelCP2uLFiO3Yb3gfenbxiz2pLcu9WJFSGl5nOwGfaeS4S4FLWylW7u577hU+eMmDG/bPOHgE3zx2txwTSZIkSfnqUlbCoTv359Cdawdqr1xTxfQXXuOhuUt5cM5SLrvveS7571wAduzXlX2G9WafYb3YZ2hvdqroRklxUZ7xJW2F3IsV2tiyVevY+zu3kFLtfs/Opdw7+TCHtUmSJEmb6FpewiGj+nHIqH4ArF5XzWPzX2favKU8Nv917nh6Cf94ZAEAXcqK2XNIT7ZLa1nbbzF7D+vlop1SAfNfwAUipcTkf87g6mlvXaH12rPewZhhvTfzLEmSJEnrdSot5oAdttswbTqlxAtLV/HoC6/z6Auv8ej813l44TqmzH0EgP7dy9l9UA9GD+7J7oN6svugHgzp3dlLpkoFwGJFAbjzmSWc9qeHN+x//rCRfOmonXNMJEmSJLV9EcH223Vl++268t59BgNwy+130mfHvXhiwTKefHEZMxe+wd3PvkJ1Te3Q5p6dSxk9uAejB/Vkt0E92GVAD0b07UpZiVNIpNZksSJHr65Yw77fvW3D/sCenbjjy+O9DJMkSZLUQsqKg7HD+zB2eJ8NbavXVfP04uU8uXAZM198g5kvLuNP981jbVUNACVFwQ79urJTRXd2rujOqIru7DygO8P6dKG4yFEYUkuwWJGDlBKfu/JRbnxi0Ya2Gz93MKMH98wxlSRJktQxdSotZu+hvdh7aK8Nbeuqa3ju5RU8s3g5/3tpOc8sXsETC5ZtdA5fXlLEqIpu7NS/Ozv278aIvl033DqV+gWktC0sVrSym55czKf++siG/a8evTNnjR+ZYyJJkiRJmyotLmKXAbXTQOpauaaK2UtW8MxLy3n2peU889IK7nvuVa59dOFGxw3u1ZkRfbuyQ7+3Chg79uvGoF6dHY0hNYHFilby0hur2f/C2zfs79ivK1PPPoTyEiuukiRJUlvRtbyEvYb2Yq86ozCgtogx79WVzHl5JXNfqb3NeXkF101fyPI1VRuOKy0OBvfqzNA+XWpvvbswtE9nhmXbvbqUusCnhMWKFldTk/jE5dO4/eklG9pu+eI72amie46pJEmSJDWnruUl2RVFNp7anVLi1ZVrsyLGCua+sor5r61iwdJV3PTkYpauXLvR8d3KSxjSOyte9OnCoF6dGdizU3brTL/u5Y7MUIdgsaIFXf/YQs6+6rEN++e9ZzdOO2hEfoEkSZIktaqIoG+3cvp2K2fciD71Hl+xpor5S1fV3l57c8P2vFdX8t9nX+HNddUbHV9cFFR0L2dgr84M6NmJQVkRY2DPTgzs1ZmKHrU/q7TYq5eobbNY0QIWvLaKg79/54b9PYf05J+ffocfGJIkSZI20q28hF0H9mDXgT3qPZZS4vVV61i0bDWLlr258f3rq3nqxTe47amXWJNdtaSuPl3L6NetnH7d69y6ldO/R/lG7T07l7bGryltNYsVzai6JvGhPzzAA3OWbmi78yvjGdG3a46pJEmSJLVFEUHvrmX07lrGboPqFzPgrYLGi8veZNHrq1myfA0vL1/DyytWs+SNNby8Yg3z5q3k5eVrGixqlBUX0a00MfCJ/9Knaxl9upbRu0t237WMPl3K6N21tPaxLrVtfgmr1mCxoplc/fALfO2fMzbsX/S+PTh53LAcE0mSJElq7+oWNDZdL6OulBLL11TVFjLq3lasYcazz1PevRNLV61l/tJVvLpyLctXVzX6Wt07lWwoavTqUkrPzrW3Hp2y+84l2X3dtlK6l5dQ5HobaiKLFdto3isrGf+jyg37+4/ow98+cYCL3kiSJEkqGBFBj061xYMd+3Xb6LHKTosZP36/jdrWVdfw2qq1vLZyHUtXruW1VWt5deVaXlu5dsP+0pVreXXFWua+spJlb67jjTfXUZM2lwG6l5fQs8tbxY0enUrpWl5Ct/JiunUqybZrbw1udyqhS2mxRY8OwGLF27Suuob3//Y+nliwbEPbPV87lCG9u+SYSpIkSZK2XWlxEf27d6J/905Nfk5NTWLl2qqscJHdr163oZDxxpvZ9uqqDW1zXlnByjXVrFhTxYo1VVRvrtpRR9eyjYsbXcqK6VJWQufSYjqXFdO5tJguZcV0yu47193e6Jg6z8na/OK5MFiseBsuu28e590wc8P+zz6wN+/dZ3COiSRJUmuKiKOBnwPFwB9SShflHEmScldUFHTvVEr3TqXQe+ufn1JiTVUNK9ZUsXJNFctX196vXLt+u7q2PXu87vaqNdW8vHwNb66r5s211by5rppVa6tYva7+Oh1bUlZSRGnU0PXe2ygvLaKsuIjykmLKS4soL8m2S4ooL629LyvZtL3O9obH6z6/iNLit25lxUWUFMeG7dKS2u2SoiCi4xZOLFZshSXLVzPugts37B+2S3/+8NGxDkGSJKkDiYhi4NfAkcAC4OGIuCGl9FS+ySSpbYsIOpXWjoDo2628WV6zpiaxuqq2gLFqbTWr160vZFS/VdhYW82qddWsXvvWY7PnPk+/ARWsqapmTVUNa9bVbNh+/c11rFlXzdqqmtrHqt56bG0Di5hui9KsiFF72/x2WUltgaO0uIjSkqwIUhSUFAclRUUUFwUlRUFxcVDa2H5xUFxUf7+kqIg5r1Yzvll/u82zWLEVbpn50obtB845nAE9mz4kSpIktRvjgNkppTkAEXEVMAmwWCFJBaaoKOhSVkKXshK224rnVVYuZvz4Pbb659XUJNZW1ylgrKtfzFi9rpqq6kRVTQ1rqxPrqmpYV13Dupo629U1rKtOjW6vra6hqk772qoaVq2t3rC9rrqGquz1qmoS1TW1x1XXpA37VU2ccrPegK7BWe/f6i552yxWbIUP7DeUiXsMpHfXsryjSJKk/AwG5tfZXwDsn1MWSVIBKSoKOhXVjg6B0rzjbFZKbxUtqmoS1VkBpbomsa7OflVNoqo68egj01o1n8WKrVBaXGShQpIkbVFEnAmcCVBRUUFlZWWzvv6KFSua/TXbOvukPvukYfZLffZJffZJfT1jVav2icUKSZKkrbMQGFpnf0jWtkFK6WLgYoCxY8em8ePHN2uAyspKmvs12zr7pD77pGH2S332SX32SX2t3SdFrfaTJEmS2oeHgVERMSIiyoCTgRtyziRJUrviyApJkqStkFKqiojPAjdTe+nSS1NKM7fwNEmStBUsVkiSJG2llNJUYGreOSRJaq+cBiJJkiRJkgqKxQpJkiRJklRQLFZIkiRJkqSCYrFCkiRJkiQVFIsVkiRJkiSpoFiskCRJkiRJBSVSSnlnaDER8TLwfN45WkFf4JW8Q3QQ9nXrsJ9bj33degqtr7dPKfXLO0RH0ELnI4X2fioE9kl99knD7Jf67JP67JP6WqJPGj0fadfFio4iIqallMbmnaMjsK9bh/3ceuzr1mNfqzn5fqrPPqnPPmmY/VKffVKffVJfa/eJ00AkSZIkSVJBsVghSZIkSZIKisWK9uHivAN0IPZ167CfW4993XrsazUn30/12Sf12ScNs1/qs0/qs0/qa9U+cc0KSZIkSZJUUBxZIUmSJEmSCorFijYqIr4cESki+mb7ERG/iIjZEfFERIypc+ypEfFsdjs1v9RtR0T8MCKezvryuojoVeexc7J+fiYi3l2n/eisbXZETM4leDtgPzafiBgaEXdGxFMRMTMizs7a+0TErdlnwq0R0Ttrb/RzRE0TEcUR8WhE3Jjtj4iIB7M+vToiyrL28mx/dvb48FyDq83oyJ+RETEvImZExGMRMS1r63CfZxFxaUQsiYgn67RtdT+0p/PDRvrk/IhYmL1fHouICXUea/fncs15DtBe3iub6ZMO+16JiE4R8VBEPJ71ybey9q0+f2msr7ZJSslbG7sBQ4Gbqb1me9+sbQLwHyCAA4AHs/Y+wJzsvne23Tvv36HQb8BRQEm2/X3g+9n2bsDjQDkwAngOKM5uzwE7AGXZMbvl/Xu0tZv92Oz9ORAYk213B/6XvYd/AEzO2ifXeX83+Dnibav6/EvA34Abs/1rgJOz7d8Bn862zwJ+l22fDFydd3ZvhX/r6J+RwLz15z112jrc5xnwTmAM8OTb7Yf2dn7YSJ+cD3ylgWM7xLlcc50DtKf3ymb6pMO+V7L/3t2y7VLgwey//1advzTWV9uaz5EVbdNPga8CdRccmQRcnmo9APSKiIHAu4FbU0pLU0qvAbcCR7d64jYmpXRLSqkq230AGJJtTwKuSimtSSnNBWYD47Lb7JTSnJTSWuCq7FhtHfuxGaWUFqWUpmfby4FZwGBq+/Sy7LDLgPdm2419jqgJImIIMBH4Q7YfwGHAP7JDNu3r9f8N/gEcnh0vbY6fkfV1uM+zlNLdwNJNmre2H9rV+WEjfdKYDnEu14znAO3mvbKZPmlMu3+vZP+9V2S7pdktsfXnL4311TaxWNHGRMQkYGFK6fFNHhoMzK+zvyBra6xdTXc6tZVmsJ9bmv3YQrJhevtQWzGvSCktyh5aDFRk2/b/tvkZtYXkmmx/O+D1OoXPuv25oa+zx5dlx0ub09H/H03ALRHxSEScmbX5eVZra/uho/TPZ7MpDZeun+5AB+yTbTwHaJf9skmfQAd+r0TtFNbHgCXUFqOeY+vPX1qkTyxWFKCIuC0inmzgNgn4OvB/eWdsD7bQz+uPOReoAq7IL6m0bSKiG/BP4AsppTfqPpZqx+55WahtFBHHAktSSo/knUVqxw5OKY0BjgE+ExHvrPugn2e17IcNfgvsCOwNLAJ+nGuanHgOUF8DfdKh3ysppeqU0t7UjiQfB+ySb6K3lOQdQPWllI5oqD0i9qB2DtDj2WjhIcD0iBgHLKR2LYv1hmRtC4Hxm7RXNnvoNqixfl4vIj4GHAscnn2YQ+P9zGba1XSb61+9DRFRSu0f5CtSStdmzS9FxMCU0qJsiOeSrN3+f/sOAo7LFuXqBPQAfk7tMNqS7NuHuv25vq8XREQJ0BN4tfVjq43p0P+PppQWZvdLIuI6ak+q/TyrtbX90O7PD1NKL63fjohLgBuz3Q5zLtdM5wDt6r3SUJ/4XqmVUno9Iu4EDmTrz19a5DPXkRVtSEppRkqpf0ppeEppOLXDa8aklBYDNwAfzVbyPQBYlg3xuhk4KiJ6Z0OajsratBkRcTS1w7mPSymtqvPQDcDJ2Uq4I4BRwEPAw8CobOXcMmoXnLmhtXO3A/ZjM8rmEP4RmJVS+kmdh24A1q/mfSpwfZ32hj5HtAUppXNSSkOyz+aTgTtSSh8C7gROyA7btK/X/zc4ITu+w327pa3WYT8jI6JrRHRfv03t+cyT+Hm23tb2Q7s/P9xkjZLjqX2/QAc5l2vGc4B2815prE868nslIvpFdtXDiOgMHEntWh5be/7SWF9tE0dWtB9TqV3FdzawCjgNIKW0NCK+Q+3/VADfTik1dQGijuxX1K5me2s2iuWBlNKnUkozI+Ia4Clqp4d8JqVUDRARn6X2w7sYuDSlNDOf6G1XSqnKfmxWBwEfAWZkcxGhdirZRcA1EXEGtVcVOil7rMHPEW2TrwFXRcR3gUepPUkiu/9LRMymdlG4k3PKpzakg39GVgDXZX+TS4C/pZRuioiH6WCfZxFxJbXfdPeNiAXAeWzl53p7Oz9spE/GR8Te1E5zmAd8EqADncs1yzlAO3uvNNYnp3Tg98pA4LKIKKZ2IMM1KaUbI+IptuL8ZXN9tS3CL3IkSZIkSVIhcRqIJEmSJEkqKBYrJEmSJElSQbFYIUmSJEmSCorFCkmSJEmSVFAsVkiSJEmSpIJisULqILLrZt8TEcfUaTsxIm6qs/9gRDwWES9ExMvZ9mMRMTyX0LWZ/hwRJ2z5SEmS1F5ExH3Z/fCI+GAzv/bXG/pZkgqLly6VOpCIGA38HdiH2uvUPwocnVJ6bpPjPgaMTSl9dpP2kpRSVSvFXf8z/wzcmFL6R2v+XEmSlL+IGA98JaV07FY8Z7PnKxGxIqXUrRniSWpBjqyQOpCU0pPAv4GvAf8HXL5poWJTEXF+RPwlIu4F/hIRH4uIX9V5/MbsRIKIOCoi7o+I6RHx94jotslr7RIRD9XZHx4RM7Lt/4uIhyPiyYi4OCKigSzzIqJvtj02Iiqz7a4RcWlEPBQRj0bEpKx996ztsYh4IiJGbX2vSZKk1hYRK7LNi4BDsr/lX4yI4oj4YXbO8EREfDI7fnxE/DcibgCeytr+FRGPRMTMiDgza7sI6Jy93hV1f1Y2CvWH2bnIjIj4QJ3XroyIf0TE0xFxxfrzlIi4KCKeyrL8qDX7SGrvSvIOIKnVfQuYDqwFxjbxObsBB6eU3sxGXdSTFRG+ARyRUloZEV8DvgR8e/0xKaWnI6IsIkaklOYCHwCuzh7+VUrp29lr/QU4ltrCSlOcC9yRUjo9InoBD0XEbcCngJ+nlK6IiDKguImvJ0mSCsNk6oysyIoOy1JK+0VEOXBvRNySHTsGGJ2dYwCcnlJaGhGdgYcj4p8ppckR8dmU0t4N/Kz3AXsDewF9s+fcnT22D7A78CJwL3BQRMwCjgd2SSml7BxEUjOxWCF1MFkh4WpgRUppTROfdkNK6c0tHHMAtUWNe7MvG8qA+xs47hpqixQXZfcfyNoPjYivAl2APsBMml6sOAo4LiK+ku13AoZlP//ciBgCXJtSeraJrydJkgrTUcCeddaz6gmMovZLmIfqFCoAPh8Rx2fbQ7PjXt3Max8MXJlSqgZeioi7gP2AN7LXXgAQEY8Bw4EHgNXAHyPiRuDGbf/1JK1nsULqmGqyW1OtrLNdxcZTyDpl9wHcmlI6ZQuvdTXw94i4FkgppWcjohPwG2rXyZgfEefXed266v7suo8H8P6U0jObHD8rIh4EJgJTI+KTKaU7tpBPkiQVrgA+l1K6eaPG2impKzfZPwI4MKW0Kps62tC5RVPV/YKnGihJKVVFxDjgcOAE4LPAYdvwMyTV4ZoVkrbWPGDviCiKiKHAuKz9AWqHRI6EDetI7LTpk7M1MqqBb/LWFJD1Jw+vZOtcNHb1j3nAvtn2++u03wx8rs780X2y+x2AOSmlXwDXA3tu3a8qSZJythzoXmf/ZuDTEVEKEBE7RUTXBp7XE3gtK1TsQu0I0PXWrX/+Jv4LfCBbF6Mf8E7goQaOI/vZ3YCeKaWpwBepnT4iqZk4skLS1roXmEvt4lWzqF3/gpTSy9l6Fldmc0ihdg2L/zXwGlcDPwRGZM99PSIuAZ4EFgMPN/Kzv0XtUMvvAJV12r8D/Ax4IiKKsnzHAicBH4mIddnrXrj1v64kScrRE0B1RDwO/Bn4ObVTMKZnX1K8DLy3gefdBHwqW1fiGWq/VFnvYmrPGaanlD5Up/064EDgcSABX00pLc6KHQ3pDlyfjRANatfqktRMvHSpJEmSJEkqKE4DkSRJkiRJBcVihSRJkiRJKigWKyRJkiRJUkGxWCFJkiRJkgqKxQpJkiRJklRQLFZIkiRJkqSCYrFCkiRJkiQVFIsVkiRJkiSpoPx/oe2dlw800/8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1296x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, (ax1,ax2) = plt.subplots(nrows=1, ncols=2,figsize=(18,5))\n",
    "\n",
    "\n",
    "iterations = list(range(1,len(loss_history)+1))\n",
    "\n",
    "_=ax1.plot(y,y_pred)\n",
    "ax1.set_xlabel('Y True values')\n",
    "ax1.set_ylabel('Y Predicted values')\n",
    "ax1.grid()\n",
    "_=ax2.plot(iterations,loss_history)\n",
    "ax2.set_xlabel('iterations')\n",
    "ax2.set_ylabel('loss')\n",
    "ax2.grid()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f7e7eac",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "164a171c-64da-4cda-80ae-9e6a4c679f2b",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c50fdbae-18ab-4f80-8411-e27d31c79526",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}