Tech Fest Northwest Workshop -- BYOB: Build Your Own Brain

BYOB-Learn-to-be-Logical.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:8d73b0dd47f7ea6d93b13eed84a0a75d2b4cdb73c24b94336388ea8d32b8aa93"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Data\n",
      "First we need example inputs and outputs.\n",
      "So lets use a simple real world system ... an *AND* logic gate..."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from itertools import product\n",
      "training_set = [\n",
      "    [i1, i2, i1 and i2]\n",
      "     for i1, i2 in product([0, 1], [0, 1])\n",
      "    ] * 2\n",
      "print(training_set)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[0, 0, 0], [0, 1, 0], [1, 0, 0], [1, 1, 1], [0, 0, 0], [0, 1, 0], [1, 0, 0], [1, 1, 1]]\n"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Training\n",
      "Just push the weights halfway toward the right answer"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def learn(training_set=(), weights=(0., 0.), learning_rate=0.5):\n",
      "    for i1, i2, o1 in training_set:\n",
      "        prediction = sum([w * i for i, w in zip([i1, i2], weights)])\n",
      "        error = o1 - prediction\n",
      "        print('{} - {} = {}'.format(o1, prediction, error))\n",
      "        weights = [w + error * learning_rate for w in weights]\n",
      "        print('new weights = {}'.format(weights))\n",
      "    return weights\n",
      "\n",
      "print(learn(training_set))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "0 - 0.0 = 0.0\n",
        "new weights = [0.0, 0.0]\n",
        "0 - 0.0 = 0.0\n",
        "new weights = [0.0, 0.0]\n",
        "0 - 0.0 = 0.0\n",
        "new weights = [0.0, 0.0]\n",
        "1 - 0.0 = 1.0\n",
        "new weights = [0.5, 0.5]\n",
        "0 - 0.0 = 0.0\n",
        "new weights = [0.5, 0.5]\n",
        "0 - 0.5 = -0.5\n",
        "new weights = [0.25, 0.25]\n",
        "0 - 0.25 = -0.25\n",
        "new weights = [0.125, 0.125]\n",
        "1 - 0.25 = 0.75\n",
        "new weights = [0.5, 0.5]\n",
        "[0.5, 0.5]\n"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Smarter Learning\n",
      "How would you improve this learning approach?\n",
      "Are we pushing too hard (learning rate too high)?"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Your Turn\n",
      "See if you can train a neuron to behave like a *NOR* logic gate"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Recursive Network\n",
      "Let's train a NN to beat you at Rock, Paper, Scissors\n",
      "All we have to do is play against it\n",
      "And punish or reward it with each good/bad move"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Tic-Tac-Toe?\n",
      "How long would it take to train a neural net\n",
      "To win at tic-tac-toe?\n",
      "With this simple training and back-propagation approach?"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}