array-list.ipynb

balanced-bsts-demo.ipynb

binary-search-trees.ipynb

bst-ds-demo.ipynb

hashtable-demo.ipynb

iterators-demo.ipynb

lang-intro-part1.ipynb

lang-intro-part2.ipynb

lang-intro-part3.ipynb

linked-list-demo.ipynb

linked-structures-demo.ipynb

on-recursion-demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# On Recursion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Agenda\n",
    "\n",
    "1. Recursion\n",
    "    - stopping recursion: simplification & base cases\n",
    "2. Recursive \"shapes\":\n",
    "    - Linear (single) recursion:\n",
    "        - Factorial\n",
    "        - Addition\n",
    "        - Binary search\n",
    "    - Tree (multiple) recursion: *divide and conquer*\n",
    "        - Fibonacci numbers\n",
    "        - Tower of Hanoi\n",
    "        - Merge sort\n",
    "        - Making change\n",
    "3. The Call Stack and Stack Frames\n",
    "    - simulating recursion\n",
    "    - debugging with `pdb` and `%debug`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Recursion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recursive functions, directly or indirectly, call themselves. \n",
    "\n",
    "Recursive solutions are applicable when a problem can be broken down into more easily solved sub-problems that resemble the original, and whose solutions can then be combined.\n",
    "\n",
    "E.g., computing the combined price of a bunch of nested shopping bags of items:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Bag:\n",
    "    def __init__(self, price, *contents):\n",
    "        self.price = price\n",
    "        self.contents = contents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bag1 = Bag(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bag2 = Bag(5, Bag(3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bag3 = Bag(5, Bag(4, Bag(3)), Bag(2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bag4 = Bag(0, Bag(5), Bag(10), Bag(3, Bag(2), Bag(100)), Bag(9, Bag(2, Bag(25))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def price(bag):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "price(bag1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stopping recursion: simplification & base case(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.setrecursionlimit(200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def silly_rec(n):\n",
    "    print(n)\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "silly_rec(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Recursive \"shapes\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linear recursion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Factorial\n",
    "\n",
    "$n! = \\begin{cases}\n",
    "        1 & \\text{if}\\ n=0 \\\\\n",
    "        n \\cdot (n-1)! & \\text{if}\\ n>0\n",
    "      \\end{cases}$\n",
    "\n",
    "\n",
    "i.e., $n! = n \\cdot (n-1) \\cdot (n-2) \\cdots 3 \\cdot 2 \\cdot 1$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rec_factorial(n):\n",
    "    print('n = ', n)\n",
    "    pass\n",
    "\n",
    "rec_factorial(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Addition of two positive numbers $m$, $n$\n",
    "\n",
    "$m + n = \\begin{cases}\n",
    "           m & \\text{if}\\ n=0 \\\\\n",
    "           (m + 1) + (n - 1) & \\text{if}\\ n > 0\n",
    "      \\end{cases}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add(m, n):\n",
    "    print('m, n = ', (m, n))\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add(5, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add(5, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add(5, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Binary search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bin_search(x, lst):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bin_search(20, list(range(100)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bin_search(-1, list(range(100)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bin_search(50.5, list(range(100)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tree recursion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Fibonacci numbers\n",
    "\n",
    "$fib(n) = \\begin{cases}\n",
    "            0 & \\text{if}\\ n=0 \\\\\n",
    "            1 & \\text{if}\\ n=1 \\\\\n",
    "            fib(n-1) + fib(n-2) & \\text{otherwise}\n",
    "          \\end{cases}$\n",
    "          \n",
    "i.e., 0, 1, 1, 2, 3, 5, 8, 13, 21, ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rec_fib(n):\n",
    "    print('n = ', n)\n",
    "    pass\n",
    "\n",
    "rec_fib(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Tower of Hanoi\n",
    "\n",
    "Setup: three rods, with one or more discs of different sizes all stacked on one rod, smallest (top) to largest (bottom). E.g.,\n",
    "\n",
    "         ||          ||          ||     \n",
    "         ==          ||          ||     \n",
    "        ====         ||          ||     \n",
    "       ======        ||          ||     \n",
    "    ------------------------------------\n",
    "    \n",
    "Goal: move all the discs, one by one, to another rod, with the rules being that (1) only smaller discs can be stacked on larger ones and (2) only the top disc in a stack can be moved to another rod.\n",
    "\n",
    "For three discs, as shown above, we would carry out the following sequence to move the stack to the rightmost rod. The rods are abbreviated L (left), M (middle), R (right):\n",
    "1. Move the small disc (0) from L to R\n",
    "2. Move the medium disc (1) from L to M\n",
    "3. Move 0 from R to M (R is empty)\n",
    "4. Move the large disc (2) from L to R\n",
    "5. Move 0 from M to L\n",
    "6. Move 1 from M to R\n",
    "7. Move 0 from L to R (done)\n",
    "\n",
    "Can you come up with the sequence needed to move a stack of 4 discs from one rod to another? 5 discs? An arbitrary number of discs?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "height = 3\n",
    "towers = [[] for _ in range(3)]\n",
    "towers[0] = list(range(height, 0, -1))\n",
    "\n",
    "def move(frm, to):\n",
    "    towers[to].append(towers[frm].pop(-1))\n",
    "    display()\n",
    "\n",
    "def hanoi(frm, to, using, levels):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "towers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import sleep\n",
    "from IPython.display import clear_output\n",
    "\n",
    "def display():\n",
    "    clear_output(True)\n",
    "    print('{:^12}'.format('||') * 3)\n",
    "    for level in range(height, 0, -1):\n",
    "        for t in towers:\n",
    "            try:\n",
    "                print('{:^12}'.format('==' * t[level-1]), end='')\n",
    "            except IndexError:\n",
    "                print('{:^12}'.format('||'), end='')\n",
    "        print()\n",
    "    print('-' * 36)\n",
    "    sleep(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hanoi(0, 2, 1, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Mergesort"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def merge(l1, l2): # O(N), where N is the number of elements in the two lists\n",
    "    merged = []\n",
    "    i1 = i2 = 0\n",
    "    while i1 < len(l1) or i2 < len(l2):\n",
    "        if i2 == len(l2) or (i1 < len(l1) \n",
    "                             and l1[i1] < l2[i2]):\n",
    "            merged.append(l1[i1])\n",
    "            i1 += 1\n",
    "        else:\n",
    "            merged.append(l2[i2])\n",
    "            i2 += 1\n",
    "    return merged"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l1 = [1, 5, 9]\n",
    "l2 = [2, 6, 8, 11]\n",
    "merge(l1, l2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mergesort(lst):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "lst = list(range(10))\n",
    "random.shuffle(lst)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mergesort(lst)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def insertion_sort(lst):\n",
    "    for i in range(1, len(lst)):\n",
    "        for j in range(i, 0, -1):\n",
    "            if lst[j-1] > lst[j]:\n",
    "                lst[j-1], lst[j] = lst[j], lst[j-1] # swap\n",
    "            else:\n",
    "                break   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Heap:\n",
    "    def __init__(self):\n",
    "        self.data = []\n",
    "\n",
    "    @staticmethod\n",
    "    def _parent(idx):\n",
    "        return (idx-1)//2\n",
    "        \n",
    "    @staticmethod\n",
    "    def _left(idx):\n",
    "        return idx*2+1\n",
    "\n",
    "    @staticmethod\n",
    "    def _right(idx):\n",
    "        return idx*2+2\n",
    "    \n",
    "    def _heapify(self, idx=0):\n",
    "        while True:\n",
    "            l = Heap._left(idx)\n",
    "            r = Heap._right(idx)\n",
    "            maxidx = idx\n",
    "            if l < len(self) and self.data[l] > self.data[idx]:\n",
    "                maxidx = l\n",
    "            if r < len(self) and self.data[r] > self.data[maxidx]:\n",
    "                maxidx = r\n",
    "            if maxidx != idx:\n",
    "                self.data[idx], self.data[maxidx] = self.data[maxidx], self.data[idx]\n",
    "                idx = maxidx\n",
    "            else:\n",
    "                break\n",
    "            \n",
    "    def add(self, x):\n",
    "        self.data.append(x)\n",
    "        i = len(self.data) - 1\n",
    "        p = Heap._parent(i)\n",
    "        while i > 0 and self.data[p] < self.data[i]:\n",
    "            self.data[p], self.data[i] = self.data[i], self.data[p]\n",
    "            i = p\n",
    "            p = Heap._parent(i)\n",
    "        \n",
    "    def max(self):\n",
    "        return self.data[0]\n",
    "\n",
    "    def pop_max(self):\n",
    "        ret = self.data[0]\n",
    "        self.data[0] = self.data[len(self.data)-1]\n",
    "        del self.data[len(self.data)-1]\n",
    "        self._heapify()\n",
    "        return ret\n",
    "    \n",
    "    def __bool__(self):\n",
    "        return len(self.data) > 0\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.data)\n",
    "\n",
    "\n",
    "def heapsort(iterable):\n",
    "    heap = Heap()\n",
    "    for x in iterable:\n",
    "        heap.add(x)\n",
    "    sorted_lst = []\n",
    "    while heap:\n",
    "        sorted_lst.append(heap.pop_max())\n",
    "    sorted_lst.reverse()\n",
    "    return sorted_lst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import timeit\n",
    "import random\n",
    "insertionsort_times = []\n",
    "heapsort_times = []\n",
    "mergesort_times = []\n",
    "for size in range(100, 3000, 100):\n",
    "    insertionsort_times.append(timeit.timeit(stmt='insertion_sort(lst)',\n",
    "                               setup='import random ; from __main__ import insertion_sort ; '\n",
    "                                         'lst = [random.random() for _ in range({})]'.format(size),\n",
    "                               number=1))\n",
    "    heapsort_times.append(timeit.timeit(stmt='heapsort(lst)',\n",
    "                               setup='import random ; from __main__ import heapsort ; '\n",
    "                                         'lst = [random.random() for _ in range({})]'.format(size),\n",
    "                               number=1))\n",
    "    mergesort_times.append(timeit.timeit(stmt='mergesort(lst)'.format(size),\n",
    "                               setup='import random ; from __main__ import mergesort ; '\n",
    "                                         'lst = [random.random() for _ in range({})]'.format(size),\n",
    "                               number=1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.plot(insertionsort_times, 'ro')\n",
    "plt.plot(heapsort_times, 'b^')\n",
    "plt.plot(mergesort_times, 'gs')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Making Change\n",
    "\n",
    "Question: how many different ways are there of making up a specified amount of money, given a list of available denominations?\n",
    "\n",
    "E.g., how many ways of making 10 cents, given 1c, 5c, 10c, 25c coins?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def change(amount, denoms):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "change(5, (1, 5, 10, 25))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "change(10, (1, 5, 10, 25))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "change(1000, (1, 5, 10, 25))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. The Call Stack"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simulating recursive `factorial`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Stack(list):\n",
    "    push = list.append\n",
    "    pop  = lambda self: list.pop(self, -1)\n",
    "    peek = lambda self: self[-1]\n",
    "    empty = lambda self: len(self) == 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "call_stack = Stack()\n",
    "\n",
    "def call(arg):\n",
    "    call_stack.push('<frame begin>')\n",
    "    call_stack.push(('arg', arg))\n",
    "\n",
    "def get_arg():\n",
    "    return call_stack.peek()[-1]\n",
    "\n",
    "def save_local(name, val):\n",
    "    call_stack.push(('local', name, val))\n",
    "    \n",
    "def restore_local():\n",
    "    return call_stack.pop()[2]\n",
    "    \n",
    "def return_with(val):\n",
    "    while call_stack.pop() != '<frame begin>':\n",
    "        pass\n",
    "    call_stack.push(('ret', val))\n",
    "    \n",
    "def last_return_val():\n",
    "    return call_stack.pop()[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "call(10) # initial call\n",
    "while True: # recursive calls\n",
    "    n = get_arg()\n",
    "    if n == 1:\n",
    "        return_with(1)\n",
    "        break\n",
    "    else:\n",
    "        save_local('n', n)\n",
    "        call(n-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "call_stack"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ret = last_return_val()\n",
    "n = restore_local()\n",
    "return_with(n * ret)\n",
    "call_stack"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Debugging with `pdb` and `%debug`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.setrecursionlimit(100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rec_factorial(n):\n",
    "    if n <= 1:   # detect base case\n",
    "        raise Exception('base case!')\n",
    "    else:\n",
    "        return n * rec_factorial(n-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rec_factorial(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%debug\n",
    "# commands to try:\n",
    "# help, where, args, p n, up, u 10, down, d 10, l, up 100, u, d (& enter to repeat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bin_search(x, lst):\n",
    "    if len(lst) == 0:\n",
    "        return False\n",
    "    else:\n",
    "        print('lo, hi = ', (lst[0], lst[-1]))\n",
    "        mid = len(lst) // 2\n",
    "        if x == lst[mid]:\n",
    "            import pdb ; pdb.set_trace()\n",
    "            return True\n",
    "        elif x < lst[mid]:\n",
    "            return bin_search(x, lst[:mid])\n",
    "        else:\n",
    "            return bin_search(x, lst[mid+1:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bin_search(20, list(range(100)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}

priority-queues-demo.ipynb

searching-sorting-timing.ipynb

stacks-and-queues-demo.ipynb