UFLDL - parameter initialization

gistfile1.txt

{
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%reload_ext octavemagic"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Inspect `initializeParameters(hiddenSize, visibleSize)` function:\n",
      "\n",
      "* `hiddenSize`: size of the _hidden_ layer\n",
      "* `visibleSize`: size of the _input_ and _output_ layer\n",
      "\n",
      "Let $s_1$ = `inputSize`, $s_2$ = `hiddenSize`, $s_3$ = `outputSize`, $|W|$ denotes the number of parameters in $W$,\n",
      "$$|W_1| = s_2\\times s_1$$\n",
      "$$|W_2| = s_3 \\times s_2$$\n",
      "\n",
      "Since we also have parameters from bias terms $b_1$ and $b_2$, the output $theta$ has a size of:\n",
      "$$|\\theta| = s_1 s_2 + s_2 s_3 + s_2 + s_3$$\n",
      "\n",
      "Also note we don't intitialize values for bias terms $b_1$ and $b_2$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "$W_{ij}$ is draw uniformly from the interval $\\left[ -\\sqrt\\frac{6}{n_{in}+n_{out}+1} \\sqrt\\frac{6}{n_{in}+n_{out}+1} \\right]$"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Try running in Octave"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "octave theta = initializeParameters(hiddenSize=2, visibleState=3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "theta =\n",
        "\n",
        "  0.95099\n",
        "  -0.49266\n",
        "  -0.99896\n",
        "  0.43443\n",
        "  0.35011\n",
        "  -0.58586\n",
        "  0.67555\n",
        "  0.43513\n",
        "  0.15096\n",
        "  0.10365\n",
        "  0.41750\n",
        "  0.97747\n",
        "  0.00000\n",
        "  0.00000\n",
        "  0.00000\n",
        "  0.00000\n",
        "  0.00000\n"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Try running in Matlab"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Make symbolic links to matlab binary\n",
      "\n",
      "`ln -s /Applications/MATLAB_R2013a.app/bin/matlab /usr/local/bin/matlab`\n",
      "\n",
      "[Reference: matlabbridge](https://github.com/arokem/python-matlab-bridge)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import pymatbridge as pymat\n",
      "ip = get_ipython()\n",
      "pymat.load_ipython_extension(ip)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Starting MATLAB on http://localhost:54257\n",
        " visit http://localhost:54257/exit.m to shut down same\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "MATLAB started and connected!\n"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%matlab \n",
      "cd '/Users/greeness/dataset/ufldl'\n",
      "theta = initializeParameters(2, 3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "\n",
        "theta =\n",
        "\n",
        "    0.9143\n",
        "   -0.0292\n",
        "    0.6006\n",
        "   -0.7162\n",
        "   -0.1565\n",
        "    0.8315\n",
        "    0.5844\n",
        "    0.9190\n",
        "    0.3115\n",
        "   -0.9286\n",
        "    0.6983\n",
        "    0.8680\n",
        "         0\n",
        "         0\n",
        "         0\n",
        "         0\n",
        "         0\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 18
    }
   ],
   "metadata": {}
  }
 ]
}