Hello World in Pytorch | Creating Neural Network from Scratch in Pytorch.ipynb

hello-world-in-pytorch-creating-neural-network-from-scratch-in-pytorch.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Hello World in Pytorch | Creating Neural Network from Scratch in Pytorch.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyOEIBveNOZJZ4wPGGggIpp2",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/harshityadav95/df6097e07c454a2ec39b2f4f73204f29/hello-world-in-pytorch-creating-neural-network-from-scratch-in-pytorch.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-xeUf9afOpfk",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "82c9b4d6-68e2-4df2-96f5-cfab2526e4ee"
      },
      "source": [
        "#Already installed in the colab no need to install\n",
        "#!pip3 install torch\n",
        "\n"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (1.5.0+cu101)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from torch) (0.16.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torch) (1.18.4)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fgt1kkDSboL0",
        "colab_type": "text"
      },
      "source": [
        "[Reference Article Theory](https://colab.research.google.com/drive/109gHWFUlUzuwhgXROpzIuVoSPZA_qeoy)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_K1XY54LPQ0g",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch\n",
        "import torch.nn as nn\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w62iqWA6Y1hz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# x is [numbers of hours studied , numbers of hrs slept]\n",
        "X = torch.tensor(([2, 9], [1, 5], [3, 6]), dtype=torch.float) # 3 X 2 tensor\n",
        "\n",
        "# y is the grade obtained by the student\n",
        "y = torch.tensor(([92], [100], [89]), dtype=torch.float) # 3 X 1 tensor\n",
        "\n",
        "#test value case\n",
        "xPredicted = torch.tensor(([4, 8]), dtype=torch.float) # 1 X 2 tensor"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i8_yBx5kY5JZ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "ce5eb8c5-fff7-4782-a9cc-c15d441eb26d"
      },
      "source": [
        "print(X.size())\n",
        "print(y.size())"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch.Size([3, 2])\n",
            "torch.Size([3, 1])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zOjhDeT-ZRJu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#scaling the value based on the range of (max tensor value,0)\n",
        " # scale units\n",
        "X_max, _ = torch.max(X, 0)\n",
        "xPredicted_max, _ = torch.max(xPredicted, 0)\n",
        "\n",
        "#then dividing all the values to the max value for normalizing the data\n",
        "X = torch.div(X, X_max)\n",
        "xPredicted = torch.div(xPredicted, xPredicted_max)\n",
        "y = y / 100  # max test score is 100\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fmFxSLpsavsd",
        "colab_type": "text"
      },
      "source": [
        "## Model (Computation Graph)\n",
        "Once the data has been processed and it is in the proper format, all you need to do now is to define your model. Here is where things begin to change a little as compared to how you would build your neural networks using, say, something like Keras or Tensorflow. However, you will realize quickly as you go along that PyTorch doesn't differ much from other deep learning tools. At the end of the day we are constructing a computation graph, which is used to dictate how data should flow and what type of operations are performed on this information. \n",
        "\n",
        "For illustration purposes, we are building the following neural network or computation graph:\n",
        "\n",
        "\n",
        "![alt text](https://drive.google.com/uc?export=view&id=1l-sKpcCJCEUJV1BlAqcVAvLXLpYCInV6)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NhSIszUJa6qa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Neural_Network(nn.Module):\n",
        "    def __init__(self, ):\n",
        "        super(Neural_Network, self).__init__()\n",
        "        # parameters\n",
        "        # TODO: parameters can be parameterized instead of declaring them here\n",
        "        self.inputSize = 2\n",
        "        self.outputSize = 1\n",
        "        self.hiddenSize = 3\n",
        "        \n",
        "        # weights\n",
        "        self.W1 = torch.randn(self.inputSize, self.hiddenSize) # 3 X 2 tensor\n",
        "        self.W2 = torch.randn(self.hiddenSize, self.outputSize) # 3 X 1 tensor\n",
        "        \n",
        "    def forward(self, X):\n",
        "        self.z = torch.matmul(X, self.W1) # 3 X 3 \".dot\" does not broadcast in PyTorch\n",
        "        self.z2 = self.sigmoid(self.z) # activation function\n",
        "        self.z3 = torch.matmul(self.z2, self.W2)\n",
        "        o = self.sigmoid(self.z3) # final activation function\n",
        "        return o\n",
        "        \n",
        "    def sigmoid(self, s):\n",
        "        return 1 / (1 + torch.exp(-s))\n",
        "    \n",
        "    def sigmoidPrime(self, s):\n",
        "        # derivative of sigmoid\n",
        "        return s * (1 - s)\n",
        "    \n",
        "    def backward(self, X, y, o):\n",
        "        self.o_error = y - o # error in output\n",
        "        self.o_delta = self.o_error * self.sigmoidPrime(o) # derivative of sig to error\n",
        "        self.z2_error = torch.matmul(self.o_delta, torch.t(self.W2))\n",
        "        self.z2_delta = self.z2_error * self.sigmoidPrime(self.z2)\n",
        "        self.W1 += torch.matmul(torch.t(X), self.z2_delta)\n",
        "        self.W2 += torch.matmul(torch.t(self.z2), self.o_delta)\n",
        "        \n",
        "    def train(self, X, y):\n",
        "        # forward + backward pass for training\n",
        "        o = self.forward(X)\n",
        "        self.backward(X, y, o)\n",
        "        \n",
        "    def saveWeights(self, model):\n",
        "        # we will use the PyTorch internal storage functions\n",
        "        torch.save(model, \"NN\")\n",
        "        # you can reload model with all the weights and so forth with:\n",
        "        # torch.load(\"NN\")\n",
        "        \n",
        "    def predict(self):\n",
        "        print (\"Predicted data based on trained weights: \")\n",
        "        print (\"Input (scaled): \\n\" + str(xPredicted))\n",
        "        print (\"Output: \\n\" + str(self.forward(xPredicted)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LZn3ohohbMZ_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 343
        },
        "outputId": "ca6def37-e51b-4a42-8e30-24bc27ea57d1"
      },
      "source": [
        "\n",
        "NN = Neural_Network()\n",
        "for i in range(1000):  # trains the NN 1,000 times\n",
        "    if (i % 100) == 0:\n",
        "        print (\"#\" + str(i) + \" Loss: \" + str(torch.mean((y - NN(X))**2).detach().item()))  # mean sum squared loss\n",
        "    NN.train(X, y)\n",
        "NN.saveWeights(NN)\n",
        "NN.predict()\n",
        "\n",
        "print(\"Finished training!\")"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "#0 Loss: 8721.85546875\n",
            "#100 Loss: 8608.6669921875\n",
            "#200 Loss: 8608.6669921875\n",
            "#300 Loss: 8608.6669921875\n",
            "#400 Loss: 8608.6669921875\n",
            "#500 Loss: 8608.6669921875\n",
            "#600 Loss: 8608.6669921875\n",
            "#700 Loss: 8608.6669921875\n",
            "#800 Loss: 8608.6669921875\n",
            "#900 Loss: 8608.6669921875\n",
            "Predicted data based on trained weights: \n",
            "Input (scaled): \n",
            "tensor([4., 8.])\n",
            "Output: \n",
            "tensor([1.])\n",
            "Finished training!\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/torch/serialization.py:402: UserWarning: Couldn't retrieve source code for container of type Neural_Network. It won't be checked for correctness upon loading.\n",
            "  \"type \" + obj.__name__ + \". It won't be checked \"\n"
          ],
          "name": "stderr"
        }
      ]
    }
  ]
}