Decision Trees with NumPy!

Decision_Trees.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Implementing C4.5 and ID3 Decision Tree Algorithms with NumPy\n",
    "\n",
    "## We will apply these trees to the the UCI car evaluation dataset $^1$\n",
    "$^1$ https://archive.ics.uci.edu/ml/datasets/car+evaluation\n",
    "\n",
    "Disclaimer: I have neither verified nor validated this code, so use it at your own risk! Also, it was not designed to accommodate continuous attributes (data that must be split by creating buckets such as \"x<5.236\")."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start by importing NumPy, and then explore the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "#you only need matplotlib if you want to create some plots of the data\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data_path = \"/home/brb/repos/examples/decision trees/UCI_cars\"\n",
    "data = np.genfromtxt(data_path, delimiter=\",\", dtype=str)\n",
    "labels = [\"buying\", \"maint\", \"doors\", \"persons\", \"lug_boot\", \"safety\", \"class\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "records: 1728\n",
      "example record: ['vhigh' 'vhigh' '2' '2' 'small' 'low' 'unacc']\n",
      "\n",
      "columns:\n",
      "\n",
      "\tbuying: ['high' 'low' 'med' 'vhigh']\n",
      "\tmaint: ['high' 'low' 'med' 'vhigh']\n",
      "\tdoors: ['2' '3' '4' '5more']\n",
      "\tpersons: ['2' '4' 'more']\n",
      "\tlug_boot: ['big' 'med' 'small']\n",
      "\tsafety: ['high' 'low' 'med']\n",
      "\tclass: ['acc' 'good' 'unacc' 'vgood']\n"
     ]
    }
   ],
   "source": [
    "print(\"records: {}\".format(len(data)))\n",
    "print(\"example record: {}\".format(data[0]))\n",
    "print(\"\\ncolumns:\\n\")\n",
    "columns = []\n",
    "for col in range(len(data[0])):\n",
    "    print(\"\\t\" + labels[col] + \": {}\".format(np.unique(data[:,col])))\n",
    "    columns.append(np.unique(data[:,col]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The backbone of the decision tree algorithms is a criterion (e.g. entropy, Gini, error) with which we can choose the best (in a greedy sense) attribute to add to the tree. ID3 and C4.5 use information gain (entropy) and normalized information gain, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def weighted_entropy(data, col_num):\n",
    "    entropies = []\n",
    "    n_s = []\n",
    "    entropy_of_attribute = entropy(data[:,col_num])\n",
    "    for value in columns[col_num]:\n",
    "        candidate_child = data[data[:,col_num] == value]\n",
    "        n_s.append(len(candidate_child))\n",
    "        entropies.append(entropy(candidate_child[:,6]))\n",
    "    n_s = np.array(n_s)\n",
    "    n_s = n_s / np.sum(n_s)\n",
    "    weighted_entropy = n_s.dot(entropies)\n",
    "    return weighted_entropy, entropy_of_attribute\n",
    "    \n",
    "def entropy(data):\n",
    "    classes = np.unique(data)\n",
    "    n = len(data)\n",
    "    n_s = []\n",
    "    for class_ in classes:\n",
    "        n_s.append(len(data[data==class_]))\n",
    "    n_s = np.array(n_s)\n",
    "    n_s = n_s/n\n",
    "    n_s = n_s * np.log2(n_s)\n",
    "    return max(0,-np.sum(n_s))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## To store our tree, we wll use dictionaries. Each node of the tree is a Python dict."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['acc' 'good' 'unacc' 'vgood']\n"
     ]
    }
   ],
   "source": [
    "def build_node(data, entropy, label, depth, class_=\"TBD\", parent=None):\n",
    "    new_node = dict()\n",
    "    new_node['data'] = data\n",
    "    new_node['entropy'] = entropy\n",
    "    new_node['label'] = label\n",
    "    new_node['depth'] = depth\n",
    "    new_node['class'] = class_\n",
    "    new_node['parent'] = parent\n",
    "    new_node['children'] = []\n",
    "    return new_node\n",
    "\n",
    "root = build_node(data, entropy(data[:,6]), \"all data\", 0)\n",
    "classes = np.unique(root['data'][:,6])\n",
    "print(classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions that helps us build our tree and classify its leaves. find_best_split acts on a node, and returns the attribute that leads to the best (possibly normalized) information gain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('safety', 'unacc')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def find_best_split(node, c45 = False):\n",
    "    data = node['data']\n",
    "    entropy = node['entropy']\n",
    "    gains = []\n",
    "    for col_num in range(len(columns) - 1):\n",
    "        new_entropy, entropy_of_attribute = weighted_entropy(data, col_num)\n",
    "        if c45:\n",
    "            if entropy_of_attribute==0:\n",
    "                gains.append(0)\n",
    "            else:\n",
    "                gains.append((entropy - new_entropy) / (entropy_of_attribute))\n",
    "        else:\n",
    "            gains.append(entropy - new_entropy)\n",
    "    if np.max(gains) > 10**-3 :\n",
    "        best_attribute = np.argmax(gains)\n",
    "        return best_attribute\n",
    "    else:\n",
    "        return -1\n",
    "    \n",
    "def classify(node_data):\n",
    "    data = node_data[:, 6]\n",
    "    n_s = []\n",
    "    for class_ in classes:\n",
    "        n_s.append(len(data[data==class_]))\n",
    "    return columns[-1][np.argmax(n_s)]\n",
    "    \n",
    "labels[find_best_split(root)], classify(root['data'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## This function is recursive and will construct a decision tree out of a root node that contains your training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def build_tree(node, c45 = False, max_depth = 999, noisy=False):\n",
    "    next_split_attribute = find_best_split(node, c45)\n",
    "    if next_split_attribute == -1 or node['depth'] == max_depth:\n",
    "        node['class'] = classify(node['data'])\n",
    "        #this if statement just handles some printing of the tree (rudimentary visualization)\n",
    "        if noisy:\n",
    "            label = []\n",
    "            label.append(node['label'])\n",
    "            temp_parent = node\n",
    "            while temp_parent['parent']:\n",
    "                temp_parent = temp_parent['parent']\n",
    "                label.append(temp_parent['label'])\n",
    "            depth = node['depth']\n",
    "            for i, layer_label in enumerate(reversed(label)):\n",
    "                for _ in range(i):\n",
    "                    print(\"\\t\", end=\"\")\n",
    "                if i==depth:\n",
    "                    print(\"{} -> class {}\".format(layer_label, node['class']))\n",
    "                else:\n",
    "                    print(\"{}\".format(layer_label))\n",
    "                    \n",
    "    else:\n",
    "        for value in columns[next_split_attribute]:\n",
    "            data = node['data'][ node['data'][:, next_split_attribute] == value ]\n",
    "            entropy_ = entropy(data[:, 6])\n",
    "            new_node = build_node(data, entropy_, \"{} == {}\".format(\n",
    "                                            labels[next_split_attribute],value),\n",
    "                                            node['depth'] + 1, parent=node)\n",
    "            build_tree(new_node, c45, max_depth, noisy)\n",
    "            node['children'].append(new_node)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lastly, before building the tree, we need a function to check the tree's accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7002314814814815"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def correct(decision_tree):\n",
    "    if not decision_tree['children']:\n",
    "        return np.sum(classify(decision_tree['data'])==decision_tree['data'][:,6])\n",
    "    else:\n",
    "        n_correct = 0\n",
    "        for child in decision_tree['children']:\n",
    "            n_correct += correct(child)\n",
    "    return n_correct\n",
    "\n",
    "correct(root)/1728"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's make a tree!\n",
    "\n",
    "### But first, a quick look at the class distribution after splitting on safety, an important attribute according to our algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAEtxJREFUeJzt3X+wZGV95/H3R1EQh+WHDL+GqVxWBwxuxdHMEjYmEaNZ+bEbMFED5eJIzE62AmVMkWRJqnbDJnGDlaiJFX/suKJDYgRcYSFAiEiCym4AB0QcBHRWJjLMZLgKjBiUCHz3j35u2Q537q++d+6dJ+9XVVef85znnPPtPtOfPvN097mpKiRJ/XrWYhcgSVpYBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMei1pSbYkec1ulv1kkvtmuJ2Tkmyd3+om3c9xSb6Q5LEkb1vo/UkzYdBrr1VVn6uq4xa7jl38JnBTVR1QVe+dquOeevORDHppfv0QcPdiFyENM+i1N1id5K4kO5NclmQ/eOYZcZKXDw2bfKL1/f3hDSU5P8lDSbYnOWd3O0zyliRfa9u6P8mbWvsLk/xNkm8m+UaSjyU5qC37G+BVwJ8m+XaSY5Psm+SPknw9yY4kH0zyvCTPB/4KOKr1/XaSo5I8nuQFQ3X8aJLxJM+ZzydU/7wY9NobvBE4GTgG+BHgLbt2SPJc4Ergo8AhwMeB1+3S7QjgQGAF8FbgfUkOnmRbzwfeC5xSVQcAPw7cObEY+APgKOCHgZXAhQBV9dPA54DzqmpZVX0FeCdwLLAaeFHb93+tqn8ETgG2tb7LqmobcFN7vBP+A3BpVX1v+qdJmpxBr73Be6tqW1U9DPwlg9Dc1YnAPq3v96rqCuC2Xfp8D/jdtvw64NvA7sb4nwb+VZLnVdX2qroboKo2V9UNVfVEVY0D7wZeOdkGkgT4j8CvVdXDVfUY8N+BM6d4rBsYhDtJng2cBfzZFP2laRn02hv8w9D048CySfocBTxYP3iVvgd26fPNqnpyum21s+1fAP4TsD3JtUleDJDksCSXJnkwybeAPwcO3U3dy4H9gduTPJrkUeD61r47VwHHJ/mXwM8AO6tq1zcsaVYMevViO7CinUVPWDnXjVXVX1fVzwBHAvcCH2qL/gAo4Eeq6l8wOPvO5FvhG8B3gJdU1UHtdmBVTby5POPSsVX1XeBy4E3A2Xg2r3lg0KsXfwc8BZyXZJ8kpwMnzGVDSQ5P8rNtrP4JBkM8T7XFB7T5R5OsAH5jd9upqqcZvEG8J8lhbdsrkry2ddkBvCDJgbusegmDzyF+lsH/GKSRGPTqQlX9E/BzDD5kfZTBmfY1DIJ6tp4FnA9sAx5mMAb/K23ZfwNeDuwErgWumGZb/xnYDNzShno+TftcoKruZfCh8dfa0M5Rrf3/MPiM4I6q2jKH+qUfEP/wiHqV5Fbgg1X1kcWuZbbaVzX/oqr+52LXor2fZ/TqRpJXJjmiDd2sZfBVzOsXu67ZSvKvGfyv4bLFrkV92GexC5Dm0XEMPshcBvw/4PVVtX1xS5qdJBuAM4BfbV/HlEbm0I0kdc6hG0nq3JIYujn00ENrbGxsscuQpL3K7bff/o2qmuoHeMASCfqxsTE2bty42GVI0l4lyd/PpJ9DN5LUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1Lkl8cvYUYxdcO2i7XvLRact2r4laaY8o5ekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdW7aoE+yMsnfJrknyd1JfrW1X5jkwSR3ttupQ+v8VpLNSe5L8tqFfACSpKnN5C9MPQmcX1V3JDkAuD3JDW3Ze6rqj4Y7JzkeOBN4CXAU8Okkx1bVU/NZuCRpZqY9o6+q7VV1R5t+DLgHWDHFKqcDl1bVE1V1P7AZOGE+ipUkzd6sxuiTjAEvA25tTecluSvJxUkObm0rgAeGVtvKJG8MSdYl2Zhk4/j4+KwLlyTNzIyDPsky4JPA26vqW8AHgBcCq4HtwLsmuk6yej2joWp9Va2pqjXLly+fdeGSpJmZUdAneQ6DkP9YVV0BUFU7quqpqnoa+BDfH57ZCqwcWv1oYNv8lSxJmo2ZfOsmwIeBe6rq3UPtRw51ex2wqU1fDZyZZN8kxwCrgNvmr2RJ0mzM5Fs3rwDOBr6U5M7W9tvAWUlWMxiW2QL8MkBV3Z3kcuDLDL6xc67fuJGkxTNt0FfVzUw+7n7dFOu8A3jHCHVJkuaJv4yVpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5/aZrkOSlcAlwBHA08D6qvqTJIcAlwFjwBbgjVX1SJIAfwKcCjwOvKWq7liY8rWnjV1w7aLsd8tFpy3KfqUezOSM/kng/Kr6YeBE4NwkxwMXADdW1SrgxjYPcAqwqt3WAR+Y96olSTM2bdBX1faJM/Kqegy4B1gBnA5saN02AGe06dOBS2rgFuCgJEfOe+WSpBmZ1Rh9kjHgZcCtwOFVtR0GbwbAYa3bCuCBodW2trZdt7UuycYkG8fHx2dfuSRpRmYc9EmWAZ8E3l5V35qq6yRt9YyGqvVVtaaq1ixfvnymZUiSZmlGQZ/kOQxC/mNVdUVr3jExJNPuH2rtW4GVQ6sfDWybn3IlSbM1bdC3b9F8GLinqt49tOhqYG2bXgtcNdT+5gycCOycGOKRJO150369EngFcDbwpSR3trbfBi4CLk/yVuDrwBvasusYfLVyM4OvV54zrxVLkmZl2qCvqpuZfNwd4NWT9C/g3BHrkiTNE38ZK0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM5NG/RJLk7yUJJNQ20XJnkwyZ3tdurQst9KsjnJfUleu1CFS5JmZiZn9B8FTp6k/T1VtbrdrgNIcjxwJvCSts77kzx7voqVJM3etEFfVZ8FHp7h9k4HLq2qJ6rqfmAzcMII9UmSRjTKGP15Se5qQzsHt7YVwANDfba2NknSIplr0H8AeCGwGtgOvKu1Z5K+NdkGkqxLsjHJxvHx8TmWIUmazpyCvqp2VNVTVfU08CG+PzyzFVg51PVoYNtutrG+qtZU1Zrly5fPpQxJ0gzMKeiTHDk0+zpg4hs5VwNnJtk3yTHAKuC20UqUJI1in+k6JPk4cBJwaJKtwO8AJyVZzWBYZgvwywBVdXeSy4EvA08C51bVUwtTuiRpJqYN+qo6a5LmD0/R/x3AO0YpSpI0f/xlrCR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS56YN+iQXJ3koyaahtkOS3JDkq+3+4NaeJO9NsjnJXUlevpDFS5KmN5Mz+o8CJ+/SdgFwY1WtAm5s8wCnAKvabR3wgfkpU5I0V9MGfVV9Fnh4l+bTgQ1tegNwxlD7JTVwC3BQkiPnq1hJ0uzNdYz+8KraDtDuD2vtK4AHhvptbW3PkGRdko1JNo6Pj8+xDEnSdOb7w9hM0laTdayq9VW1pqrWLF++fJ7LkCRNmGvQ75gYkmn3D7X2rcDKoX5HA9vmXp4kaVRzDfqrgbVtei1w1VD7m9u3b04Edk4M8UiSFsc+03VI8nHgJODQJFuB3wEuAi5P8lbg68AbWvfrgFOBzcDjwDkLULMkaRamDfqqOms3i149Sd8Czh21KEnS/PGXsZLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6ty0fxxc0uIYu+DaRdnvlotOW5T9auF4Ri9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUudG+h59ki3AY8BTwJNVtSbJIcBlwBiwBXhjVT0yWpmSpLmajzP6V1XV6qpa0+YvAG6sqlXAjW1ekrRIFmLo5nRgQ5veAJyxAPuQJM3QqEFfwKeS3J5kXWs7vKq2A7T7wyZbMcm6JBuTbBwfHx+xDEnS7ox6rZtXVNW2JIcBNyS5d6YrVtV6YD3AmjVrasQ6JEm7MdIZfVVta/cPAVcCJwA7khwJ0O4fGrVISdLczTnokzw/yQET08C/BTYBVwNrW7e1wFWjFilJmrtRhm4OB65MMrGdv6iq65N8Hrg8yVuBrwNvGL1MSdJczTnoq+prwEsnaf8m8OpRipIkzR9/GStJnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0b9eqVkrTXG7vg2kXb95aLTlvwfXhGL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHVuwYI+yclJ7kuyOckFC7UfSdLUFiTokzwbeB9wCnA8cFaS4xdiX5KkqS3UGf0JwOaq+lpV/RNwKXD6Au1LkjSFVNX8bzR5PXByVf1Smz8b+LGqOm+ozzpgXZs9Drhvjrs7FPjGCOVqYXhclh6PydI0ynH5oapaPl2nfea48elkkrYfeEepqvXA+pF3lGysqjWjbkfzy+Oy9HhMlqY9cVwWauhmK7ByaP5oYNsC7UuSNIWFCvrPA6uSHJPkucCZwNULtC9J0hQWZOimqp5Mch7w18CzgYur6u6F2BfzMPyjBeFxWXo8JkvTgh+XBfkwVpK0dPjLWEnqnEEvSZ0z6CVNKslJSa5Z7DoEScaSbJrr+ga9JHVuyQT9ru9YSX49yYVJbkryziS3JflKkp8c6v+5JHe0248PrfubSb6U5ItJLmptL0ry6dZ2R5IX7vlH2Zck/zvJ7Unubr90nriY3R3teb6xtS1L8pF2TO5K8vOLW/neq70WfmVo/sIk5yd5fzsO1yS5rv06nSSvTvKF9txfnGTfadpPTnJvkpuBn1uUB9mBJP+lPY83JPl4y7PVSW5pr4Erkxzc+u6u/Ufb6+jvgHNHKqiqlsQNGAM2Dc3/OnAhcBPwrtZ2KvDpNr0/sF+bXgVsbNOnAP8X2L/NH9LubwVe16b3m1jubaRjNvHcPg/YBBwOPAAcs8vydwJ/PLTewYtd+956A14GfGZo/svAm4HrGJy4HQE8Ary+/Tt/ADi29b0EePsM2lcx+HX75cA1i/2Y97YbsAa4s70uDgC+2vLsLuCVrc/vTrwmZtj+h8P5ONvbkjmjn8YV7f52Bm8IAM8BPpTkS8AnGFwlE+A1wEeq6nGAqno4yQHAiqq6srV9d2K5RvK2JF8EbmHwS+h1wGer6n4YPPet32sYXM2U1v7Ini60F1X1BeCwJEcleSmDUH858Imqerqq/gH429b9OOD+qvpKm98A/NQU7S9u7V+tQbr8+Z55VN35CeCqqvpOVT0G/CXwfOCgqvpM67MB+KkkB86w/c9GKWihrnUzF0/yg0NJ+w1NP9Hun+L7Nf8asAN4aVvvu6097HJdHSa/9o5GkOQkBgH+b6rq8SQ3AV9kECLP6M4zj4nm7n8xOGM/gsGVYV+0m367+3c/1evB4zS6+cibeX3NLKUz+h0MzlRe0MYL/900/Q8EtlfV08DZDH6BC/Ap4BeT7A+Q5JCq+hawNckZrW3fieWaswOBR1rIvxg4EdgXeGWSY2Dw3Le+nwKGr1x68J4utjOXMrisyOsZhP7NwM8neVaSw4GTWr97gbEkE28EZwOfmab9mKHPr85a6AfSqZuBf59kvyTLgNOAfwQemfiMkfacV9XO3bQ/CuxM8hOt/U2jFLRkgr6qvsdgfOpW4BoG/+im8n5gbZJbgGMZPJFU1fUMrquzMcmdDMbGYPAEvi3JXQzG8I+Y9wfxz8v1wD7t+fw9BsM34wyGb65oQzqXtb6/DxycZFNrf9ViFNyLGlxO5ADgwaraDnySwYUENwH/g8FraGdVfRc4B/hEG+J8GvjgNO3rgGvbh7F/v4cfWheq6vMMMuiLDIadNwI7gbXAH7bXzGoGeccU7ecA72sfxn5nlJq8BILUgSTLqurbSV4A3Aa8oo3XaxEMHY/9gc8C66rqjsWqZymN0Uuau2uSHAQ8F/g9Q37Rrc/gz6fuB2xYzJAHz+glqXtLZoxekrQwDHpJ6pxBL0mdM+glqXMGvSR17v8DciGdBVX6bssAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAERNJREFUeJzt3W2QnWV9x/Hvr0TwmfCwUJqkxIfoaF+AzA7Q2qoV2hG0DVOl4ihkMG36Aq2tdTRax6exCm9Enba0GdEGa1FEKan1iaKArYIuyKOgRAZhDZJVHhQpKvjvi3NtPSSb7NlkNyEX38/Mzrmv//0/930dZvjtvdc590mqCklSv35td09AkrSwDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9NpjJLk1ybG7ex5bSvK4JP+R5N4kn9zd85G2tGh3T0DqwMuAg4EDqurB2ZqTFLCiqjYu+MwkvKKX5sOhwHdGCXlpdzDotUdKsk+S9yfZ1H7en2Sftu/SJC9t27+bpJIc38bHJrl6G8c8MslEkh8nuTPJ+4b2fTLJD9ryzGVJfqvV3wm8DXh5kvuSrG71Vye5McndSb6Q5NBWv6wd8prW//Ik1yf5o6FzPSbJD5McPu//4fSoZNBrT/W3wNHA4cBhwJHAW9u+S4EXtO3nAbcAzx8aX7qNY34A+EBVPRl4GnDe0L7PASuAg4CrgI8BVNXbgfcAn6iqJ1bV2UlOAN4C/AkwBnwFOLf1P68d77DW/wngHOBVQ+c6Hrijqmb8hSTNlUGvPdUrgXdV1eaqmgLeCZzc9l3Kw4P9vUPj57PtoP8F8PQkB1bVfVV1+fSOqvpwVf2kqn4GvAM4LMm+2zjOXwDvraob23LOe4DDp6/qZ/CvwPFJntzGJwMf3eYrl+bIoNee6jeA7w2Nv9dqAF8DnpHkYAZX/OcAy5IcyODK/zJmthp4BnBTkm8keQlAkr2SnJ7ku0l+DNza+g/cxnEOBT6Q5J4k9wB3AQGWzNRcVZuA/wFemmQxcBztLwZpPvipG+2pNjEI1Bva+Ddbjaq6P8mVwOuA66vq50m+Crwe+G5V/XCmA1bVzcArkvwag2WX85Mc0LZXAscyCPl9gbsZhPdMbgf+rqrmEtbrgT9j8P/k16rq+3N4rrRdXtFrT3Uu8NYkY+1K/W0MlkCmXQq8hl8t01yyxXgrSV6VZKyqfgnc08oPAU8Cfgb8CHg8g6WY7fkn4M1Db9jum+TEof13Ak/d4jn/DhzB4JfTObMcX5oTg157qncDE8C1wHUM3iB999D+SxkE9GXbGM/kRcANSe5j8MbsSVX1AIPg/R7wfeBbwOXbPgRU1QXAGcDH21LP9QyWY6a9A1jflnb+tD3nf4FPAU8BPr2940tzFf/hEemRIcnbgGdU1atmbZbmwDV66REgyf4M3gw+ebZeaa5cupF2syR/zuAN3M9V1faWlqQd4tKNJHVupCv6JIuTnJ/kpnZb928n2T/JRUlubo/7td4k+WCSjUmuTXLEwr4ESdL2jHRFn2Q98JWq+lCSvRl8xOwtwF1VdXqStcB+VfWm9p0ir2VwG/dRDG4pP2p7xz/wwANr+fLlO/lSJOnR5corr/xhVY3N1jdr0Lfbsq8BnlpDzUm+Dbygqu5IcghwSVU9M8k/t+1zt+zb1jnGx8drYmJipBcmSRpIcmVVjc/WN8rSzVOBKeAjSb6Z5ENJngAcPB3e7fGg1r+EwRtL0yaZ4dbvJGvaNwVOTE1NjTANSdKOGCXoFzG4Y++sqnoO8FNg7Xb6Z7otfKs/G6pqXVWNV9X42Nisf3lIknbQKEE/CUxW1RVtfD6D4L+zLdnQHjcP9S8bev5S2neQSJJ2vVmDvqp+ANye5JmtdAyD28A3AKtabRVwYdveAJzSPn1zNHDv9tbnJUkLa9Q7Y18LfKx94uYW4FQGvyTOa/+izm3A9Jc2fZbBJ242Ave3XknSbjJS0Ld/6Wamd3aPmaG3gNN2cl6SpHniVyBIUucMeknqnEEvSZ3za4ql7Vi+9j9327lvPf3Fu+3c6otX9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1bqSgT3JrkuuSXJ1kotX2T3JRkpvb436tniQfTLIxybVJjljIFyBJ2r65XNH/flUdXlXjbbwWuLiqVgAXtzHAccCK9rMGOGu+JitJmrudWbpZCaxv2+uBE4bq59TA5cDiJIfsxHkkSTth1KAv4ItJrkyyptUOrqo7ANrjQa2+BLh96LmTrfYwSdYkmUgyMTU1tWOzlyTNatGIfc+tqk1JDgIuSnLTdnozQ622KlStA9YBjI+Pb7VfkjQ/Rrqir6pN7XEzcAFwJHDn9JJMe9zc2ieBZUNPXwpsmq8JS5LmZtagT/KEJE+a3gb+ELge2ACsam2rgAvb9gbglPbpm6OBe6eXeCRJu94oSzcHAxckme7/t6r6fJJvAOclWQ3cBpzY+j8LHA9sBO4HTp33WUuSRjZr0FfVLcBhM9R/BBwzQ72A0+ZldpKkneadsZLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS50YO+iR7Jflmks+08VOSXJHk5iSfSLJ3q+/Txhvb/uULM3VJ0ijmckX/OuDGofEZwJlVtQK4G1jd6quBu6vq6cCZrU+StJuMFPRJlgIvBj7UxgFeCJzfWtYDJ7TtlW1M239M65ck7QajXtG/H3gj8Ms2PgC4p6oebONJYEnbXgLcDtD239v6HybJmiQTSSampqZ2cPqSpNnMGvRJXgJsrqorh8sztNYI+35VqFpXVeNVNT42NjbSZCVJc7dohJ7nAn+c5HjgscCTGVzhL06yqF21LwU2tf5JYBkwmWQRsC9w17zPXJI0klmv6KvqzVW1tKqWAycBX6qqVwJfBl7W2lYBF7btDW1M2/+lqtrqil6StGvszOfo3wS8PslGBmvwZ7f62cABrf56YO3OTVGStDNGWbr5f1V1CXBJ274FOHKGngeAE+dhbpKkeeCdsZLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS52YN+iSPTfL1JNckuSHJO1v9KUmuSHJzkk8k2bvV92njjW3/8oV9CZKk7Rnliv5nwAur6jDgcOBFSY4GzgDOrKoVwN3A6ta/Gri7qp4OnNn6JEm7yaxBXwP3teFj2k8BLwTOb/X1wAlte2Ub0/YfkyTzNmNJ0pyMtEafZK8kVwObgYuA7wL3VNWDrWUSWNK2lwC3A7T99wIHzHDMNUkmkkxMTU3t3KuQJG3TSEFfVQ9V1eHAUuBI4FkztbXHma7ea6tC1bqqGq+q8bGxsVHnK0maozl96qaq7gEuAY4GFidZ1HYtBTa17UlgGUDbvy9w13xMVpI0d6N86mYsyeK2/TjgWOBG4MvAy1rbKuDCtr2hjWn7v1RVW13RS5J2jUWzt3AIsD7JXgx+MZxXVZ9J8i3g40neDXwTOLv1nw18NMlGBlfyJy3AvCVJI5o16KvqWuA5M9RvYbBev2X9AeDEeZmdJGmneWesJHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktS5WYM+ybIkX05yY5Ibkryu1fdPclGSm9vjfq2eJB9MsjHJtUmOWOgXIUnatlGu6B8E/qaqngUcDZyW5NnAWuDiqloBXNzGAMcBK9rPGuCseZ+1JGlkswZ9Vd1RVVe17Z8ANwJLgJXA+ta2Hjihba8EzqmBy4HFSQ6Z95lLkkYypzX6JMuB5wBXAAdX1R0w+GUAHNTalgC3Dz1tstW2PNaaJBNJJqampuY+c0nSSEYO+iRPBD4F/FVV/Xh7rTPUaqtC1bqqGq+q8bGxsVGnIUmao5GCPsljGIT8x6rq06185/SSTHvc3OqTwLKhpy8FNs3PdCVJczXKp24CnA3cWFXvG9q1AVjVtlcBFw7VT2mfvjkauHd6iUeStOstGqHnucDJwHVJrm61twCnA+clWQ3cBpzY9n0WOB7YCNwPnDqvM5YkzcmsQV9V/83M6+4Ax8zQX8BpOzkvSdI88c5YSeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzswZ9kg8n2Zzk+qHa/kkuSnJze9yv1ZPkg0k2Jrk2yRELOXlJ0uxGuaL/F+BFW9TWAhdX1Qrg4jYGOA5Y0X7WAGfNzzQlSTtq1qCvqsuAu7YorwTWt+31wAlD9XNq4HJgcZJD5muykqS529E1+oOr6g6A9nhQqy8Bbh/qm2y1rSRZk2QiycTU1NQOTkOSNJv5fjM2M9RqpsaqWldV41U1PjY2Ns/TkCRN29Ggv3N6SaY9bm71SWDZUN9SYNOOT0+StLN2NOg3AKva9irgwqH6Ke3TN0cD904v8UiSdo9FszUkORd4AXBgkkng7cDpwHlJVgO3ASe29s8CxwMbgfuBUxdgzpKkOZg16KvqFdvYdcwMvQWctrOTkiTNH++MlaTOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6tyBBn+RFSb6dZGOStQtxDknSaOY96JPsBfwDcBzwbOAVSZ493+eRJI1mIa7ojwQ2VtUtVfVz4OPAygU4jyRpBIsW4JhLgNuHxpPAUVs2JVkDrGnD+5J8ewHmIu2sA4Ef7o4T54zdcVbtYQ4dpWkhgj4z1GqrQtU6YN0CnF+aN0kmqmp8d89D2hkLsXQzCSwbGi8FNi3AeSRJI1iIoP8GsCLJU5LsDZwEbFiA80iSRjDvSzdV9WCS1wBfAPYCPlxVN8z3eaRdxOVF7fFStdXyuSSpI94ZK0mdM+glqXMGvSR1zqCXpM4Z9HpUSLI8yfVD4zckeUeSS5KckeTrSb6T5PeG+r+S5Kr28ztDz31jkuuSXJPk9FZ7epL/arWrkjxt179KaWYLcWestKdZVFVHJjkeeDtwLLAZ+IOqeiDJCuBcYDzJccAJwFFVdX+S/dsxPgacXlUXJHksXkTpEcSgl+DT7fFKYHnbfgzw90kOBx4CntHqxwIfqar7AarqriRPApZU1QWt9sCumrg0CoNejxYP8vCr7McObf+sPT7Er/6f+GvgTuCw9rzp8A5bf3fTTN/vJD1i+OelHi3uBA5KckCSfYCXzNK/L3BHVf0SOJnBXd4AXwReneTxAEn2r6ofA5NJTmi1fab3S48EBr0eFarqF8C7gCuAzwA3zfKUfwRWJbmcwbLNT9txPs/gu5smklwNvKH1nwz8ZZJrga8Cvz7vL0LaQX4FgiR1zit6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI693+4bkeHVWhQDwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAE8NJREFUeJzt3X+wZ3V93/HnS0BQcQTChazLhiW6acR0XOyG0KoJERoB4wDT0GJbZSjppjM40Y6xRWcyYitTGIukVqWzFnXVVCVRAiqxLiiijfy4kGVdQMtGiaxLl6v8kB+GyvLuH9/PLV+Xu/d+7y/u3Q/Px8yZc87nfM457+9y5vU9fO75fr+pKiRJ/XrOUhcgSVpcBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMej1rJTk/yacW4bjvTfKjJP9noY8tzYVBLy2gJKuAtwNHV9UvjtD/uiS/v/iV6dnMoJcW1pHAj6vqvqUuRJpk0GtZS3J3knck2ZLk0SSXJTk8yV8meTjJNUkOHup/XJK/SvJgktuSHD+07agkX2/7bQIOnea8hyb5YjvO/Um+keQ5bdt5Sf6mHeeOJKe39hOBTcCLkzyS5OPT1ZTkAuA1wAdb/w8m+VCSi3er5QtJ3rYw/6J6VqoqJ6dlOwF3AzcAhwMrgfuAW4FjgP2BrwLvbn1XAj8GTmFwE/OP2/pY2/4t4P1tv98EHgY+tYfz/ifgvwH7tek1QNq2M4AXt3P8M+BRYEXbdjywfeg4M9V0HfD7Q/2PBXYAz2nrhwKPAYcv9X8Lp7138o5ee4P/WlU7q+qHwDeAG6vqr6vqceAKBqEP8C+Bq6vq6qp6sqo2AePAKUl+Cfh14I+r6vGquh74wjTn/BmwAjiyqn5WVd+oqgKoqj+rqh3tHJ8F7mIQ0FPZY01Tda6qm4CHgBNa05nAdVW1c+Z/JmlqBr32BsMh99Mp1g9sy0cCZ7QhkgeTPAi8mkFgvxh4oKoeHdr3b6c55/uAbcBXknwvyXmTG5K8OcnmoXP8GnseBpqupj3ZyOANgjb/5DR9pRntu9QFSAvoHuCTVfWvd9+Q5Ejg4CQvGAr7XwKm/PrWqnqYwdMzb0/ycuBrSW5mEP4fYXDH/a2q2pVkM5DZ1jR5qinaPgVsTfIK4GXAX+xhX2kk3tGrJ58C3pDkdUn2SXJAkuOTHFFVf8tgyOQ9SZ6b5NXAG/Z0oCS/m+SlSQL8BNjVphcwCOeJ1u9sBnf0s66pbd8J/PLwDlW1HbiZwZ3856rqp7P/p5CeYtCrG1V1D3Aq8C4GQXwP8A6eus7/OfAbwP3Au4FPTHO4NcA1wCMM/oj74aq6rqruAC5ubTuBvw/8r3nU9F+A30vyQJIPDO26sR3bYRvN2+RTBJKWkSS/yeD/BlZX1ZNLXY/2bt7RS8tMkv2AtwL/3ZDXQjDopWUkycuABxk8lfMnS1yOOuHQjSR1zjt6SercsniO/tBDD63Vq1cvdRmStFe55ZZbflRVYzP1WxZBv3r1asbHx5e6DEnaqySZ7tPd/59DN5LUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1Lll8cnY+Vh93peW7Nx3X/j6JTu3JI1qxjv69tNnNyW5LcntSd7T2j+e5PvtR5I3J1nb2pPkA0m2JdmS5JWL/SIkSXs2yh3948Brq+qR9oMI30zyl23bO6rqz3frfzKDn2Fbw+Bn2y5tc0nSEpjxjr4GHmmr+7Vpui+xPxX4RNvvBuCgJCvmX6okaS5G+mNs+/X6zcB9wKaqurFtuqANz1ySZP/WtpLBDyBP2t7adj/m+iTjScYnJibm8RIkSdMZKeiraldVrQWOAI5N8mvAO4FfBX4dOAT49617pjrEFMfcUFXrqmrd2NiMX6csSZqjWT1eWVUPAtcBJ1XVvW145nHgY8Cxrdt2YNXQbkcAOxagVknSHIzy1M1YkoPa8vOAE4HvTI67JwlwGrC17XIV8Ob29M1xwENVde+iVC9JmtEoT92sADYm2YfBG8PlVfXFJF9NMsZgqGYz8G9a/6uBU4BtwGPA2QtftiRpVDMGfVVtAY6Zov21e+hfwLnzL02StBD8CgRJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHVuxqBPckCSm5LcluT2JO9p7UcluTHJXUk+m+S5rX3/tr6tbV+9uC9BkjSdUe7oHwdeW1WvANYCJyU5DrgIuKSq1gAPAOe0/ucAD1TVS4FLWj9J0hKZMehr4JG2ul+bCngt8OetfSNwWls+ta3Ttp+QJAtWsSRpVkYao0+yT5LNwH3AJuBvgAer6onWZTuwsi2vBO4BaNsfAn5himOuTzKeZHxiYmJ+r0KStEcjBX1V7aqqtcARwLHAy6bq1uZT3b3X0xqqNlTVuqpaNzY2Nmq9kqRZmtVTN1X1IHAdcBxwUJJ926YjgB1teTuwCqBtfxFw/0IUK0mavVGeuhlLclBbfh5wInAn8DXg91q3s4Ar2/JVbZ22/atV9bQ7eknSM2PfmbuwAtiYZB8GbwyXV9UXk9wBfCbJe4G/Bi5r/S8DPplkG4M7+TMXoW5J0ohmDPqq2gIcM0X79xiM1+/e/nfAGQtSnSRp3vxkrCR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOjdj0CdZleRrSe5McnuSt7b285P8MMnmNp0ytM87k2xL8t0kr1vMFyBJmt6+I/R5Anh7Vd2a5IXALUk2tW2XVNV/Hu6c5GjgTODlwIuBa5L8SlXtWsjCJUmjmfGOvqrurapb2/LDwJ3Ayml2ORX4TFU9XlXfB7YBxy5EsZKk2ZvVGH2S1cAxwI2t6S1JtiT5aJKDW9tK4J6h3bYzxRtDkvVJxpOMT0xMzLpwSdJoRg76JAcCnwPeVlU/AS4FXgKsBe4FLp7sOsXu9bSGqg1Vta6q1o2Njc26cEnSaEYK+iT7MQj5P62qzwNU1c6q2lVVTwIf4anhme3AqqHdjwB2LFzJkqTZGOWpmwCXAXdW1fuH2lcMdTsd2NqWrwLOTLJ/kqOANcBNC1eyJGk2Rnnq5lXAm4BvJ9nc2t4FvDHJWgbDMncDfwBQVbcnuRy4g8ETO+f6xI0kLZ0Zg76qvsnU4+5XT7PPBcAF86hLkrRA/GSsJHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6N2PQJ1mV5GtJ7kxye5K3tvZDkmxKclebH9zak+QDSbYl2ZLklYv9IiRJezbKHf0TwNur6mXAccC5SY4GzgOurao1wLVtHeBkYE2b1gOXLnjVkqSRzRj0VXVvVd3alh8G7gRWAqcCG1u3jcBpbflU4BM1cANwUJIVC165JGkksxqjT7IaOAa4ETi8qu6FwZsBcFjrthK4Z2i37a1t92OtTzKeZHxiYmL2lUuSRjJy0Cc5EPgc8Laq+sl0Xadoq6c1VG2oqnVVtW5sbGzUMiRJszRS0CfZj0HI/2lVfb4175wckmnz+1r7dmDV0O5HADsWplxJ0myN8tRNgMuAO6vq/UObrgLOastnAVcOtb+5PX1zHPDQ5BCPJOmZt+8IfV4FvAn4dpLNre1dwIXA5UnOAX4AnNG2XQ2cAmwDHgPOXtCKJUmzMmPQV9U3mXrcHeCEKfoXcO4865IkLRA/GStJnTPoJalzBr0kdc6gl6TOGfSS1LlRHq+UntVWn/elJTnv3Re+fknOq/54Ry9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOjdj0Cf5aJL7kmwdajs/yQ+TbG7TKUPb3plkW5LvJnndYhUuSRrNKHf0HwdOmqL9kqpa26arAZIcDZwJvLzt8+Ek+yxUsZKk2Zsx6KvqeuD+EY93KvCZqnq8qr4PbAOOnUd9kqR5ms8Y/VuSbGlDOwe3tpXAPUN9tre2p0myPsl4kvGJiYl5lCFJms5cg/5S4CXAWuBe4OLWnin61lQHqKoNVbWuqtaNjY3NsQxJ0kzmFPRVtbOqdlXVk8BHeGp4ZjuwaqjrEcCO+ZUoSZqPOQV9khVDq6cDk0/kXAWcmWT/JEcBa4Cb5leiJGk+9p2pQ5JPA8cDhybZDrwbOD7JWgbDMncDfwBQVbcnuRy4A3gCOLeqdi1O6ZKkUcwY9FX1ximaL5um/wXABfMpSpK0cPxkrCR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOjdj0Cf5aJL7kmwdajskyaYkd7X5wa09ST6QZFuSLUleuZjFS5JmNsod/ceBk3ZrOw+4tqrWANe2dYCTgTVtWg9cujBlSpLmasagr6rrgft3az4V2NiWNwKnDbV/ogZuAA5KsmKhipUkzd5cx+gPr6p7Adr8sNa+ErhnqN/21vY0SdYnGU8yPjExMccyJEkzWeg/xmaKtpqqY1VtqKp1VbVubGxsgcuQJE2aa9DvnBySafP7Wvt2YNVQvyOAHXMvT5I0X3MN+quAs9ryWcCVQ+1vbk/fHAc8NDnEI0laGvvO1CHJp4HjgUOTbAfeDVwIXJ7kHOAHwBmt+9XAKcA24DHg7EWoWZI0CzMGfVW9cQ+bTpiibwHnzrcoSdLC8ZOxktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknq3Iw/Dj6dJHcDDwO7gCeqal2SQ4DPAquBu4F/WlUPzK9MSdJcLcQd/W9X1dqqWtfWzwOurao1wLVtXZK0RBZj6OZUYGNb3gictgjnkCSNaL5BX8BXktySZH1rO7yq7gVo88Om2jHJ+iTjScYnJibmWYYkaU/mNUYPvKqqdiQ5DNiU5Duj7lhVG4ANAOvWrat51iFJ2oN53dFX1Y42vw+4AjgW2JlkBUCb3zffIiVJczfnoE/ygiQvnFwGfgfYClwFnNW6nQVcOd8iJUlzN5+hm8OBK5JMHud/VNWXk9wMXJ7kHOAHwBnzL1OSNFdzDvqq+h7wiinafwycMJ+iJEkLZ75/jJWkvd7q8760ZOe++8LXL/o5/AoESeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXOLFvRJTkry3STbkpy3WOeRJE1vUYI+yT7Ah4CTgaOBNyY5ejHOJUma3mLd0R8LbKuq71XV/wU+A5y6SOeSJE1j30U67krgnqH17cBvDHdIsh5Y31YfSfLdOZ7rUOBHc9x3XnLRUpxVS2BJrjGvr2eHXDSv6+vIUTotVtBnirb6uZWqDcCGeZ8oGa+qdfM9jrQnXmNaTM/E9bVYQzfbgVVD60cAOxbpXJKkaSxW0N8MrElyVJLnAmcCVy3SuSRJ01iUoZuqeiLJW4D/CewDfLSqbl+Mc7EAwz/SDLzGtJgW/fpKVc3cS5K01/KTsZLUOYNekjpn0EvSMpdkdZKtc93foJekzi2boN/9HSvJHyU5P8l1SS5KclOS/53kNUP9v5Hk1jb9o6F9/12Sbye5LcmFre2lSa5pbbcmeckz/yq1XCX5iyS3JLm9fWp78ov5bm3XzLWt7cAkH2vX15Yk/2RpK9dylOSPk3wnyaYkn255tjbJDe26uSLJwa3vntr/Qbv2vgWcO6+CqmpZTMBqYOvQ+h8B5wPXARe3tlOAa9ry84ED2vIaYLwtnwz8FfD8tn5Im98InN6WD5jc7uRU9XPXyfOArcDhDL7G46jdtl8E/MnQfgcvde1Oy2sC1gGb27X0QuCulmdbgN9qff7D5HU0Yvv7hvNxttNifQXCQvt8m9/C4A0BYD/gg0nWAruAX2ntJwIfq6rHAKrq/iQvBFZW1RWt7e+eqcK11/jDJKe35VUMvofp+qr6Pgyuo7btRAYfAKS1P/CMVqm9wauBK6vqpwBJvgC8ADioqr7e+mwE/izJi0Zs/ySDm9g5WU5B/wQ/P5R0wNDy422+i6dq/rfATuAVbb/J8A67fa8OU3/3jgRAkuMZBPg/rKrHklwH3Ab8vam68/TrSxq2EHmzoNfZshmjZxDahyX5hST7A787Q/8XAfdW1ZPAmxh8AhfgK8C/SvJ8gCSHVNVPgO1JTmtt+09ulxhcSw+0kP9V4Dhgf+C3khwFg+uo9f0K8JbJHSfHU6Uh3wTekOSAJAcCrwceBR6Y/Bsjg8z6elU9tIf2B4GHkry6tf+L+RS0bIK+qn7GYHzqRuCLwHdm2OXDwFlJbmAwbPNoO86XGXyvzniSzQzGxmDwD/iHSbYwGMP/xQV/EdpbfRnYt10b/xG4AZhgMHzz+SS3AZ9tfd8LHJxka2v/7aUoWMtXVd3MIINuYzDsPA48BJwFvK9dZ2sZ5B3TtJ8NfKj9Mfan86nJr0CQpAWW5MCqeqSNHFwPrK+qW5eqnuU0Ri9JvdjQfj71AGDjUoY8eEcvSd1bNmP0kqTFYdBLUucMeknqnEEvSZ0z6CWpc/8PkuX65pWz1YUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for safety in columns[5]:\n",
    "    plt.hist(data[data[:,5]==safety, 6])\n",
    "    plt.title(safety + \" safety\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "all data\n",
      "\tsafety == high -> class unacc\n",
      "all data\n",
      "\tsafety == low -> class unacc\n",
      "all data\n",
      "\tsafety == med -> class unacc\n",
      "\n",
      "Tree Accuracy: 0.7002314814814815\n"
     ]
    }
   ],
   "source": [
    "root = build_node(data, entropy(data[:,6]), \"all data\", 0)\n",
    "build_tree(root, max_depth=1,  noisy=True)\n",
    "print(\"\\nTree Accuracy: {}\".format(correct(root)/1728))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "all data\n",
      "\tsafety == high\n",
      "\t\tpersons == 2 -> class unacc\n",
      "all data\n",
      "\tsafety == high\n",
      "\t\tpersons == 4 -> class acc\n",
      "all data\n",
      "\tsafety == high\n",
      "\t\tpersons == more -> class acc\n",
      "all data\n",
      "\tsafety == low -> class unacc\n",
      "all data\n",
      "\tsafety == med\n",
      "\t\tpersons == 2 -> class unacc\n",
      "all data\n",
      "\tsafety == med\n",
      "\t\tpersons == 4 -> class acc\n",
      "all data\n",
      "\tsafety == med\n",
      "\t\tpersons == more -> class acc\n",
      "\n",
      "Tree Accuracy: 0.7777777777777778\n"
     ]
    }
   ],
   "source": [
    "root = build_node(data, entropy(data[:,6]), \"all data\", 0)\n",
    "build_tree(root, max_depth=2,  noisy=True)\n",
    "print(\"\\nTree Accuracy: {}\".format(correct(root)/1728))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFZFJREFUeJzt3Xu8JGV95/HPV1C8gYAckKujCGRx14x6gq5GxVUTIKxoEqNEEaKbkURWTXSVmN3VuLohRkNijOi4EtEoQoKsrBIVL2g0Ah4uIgaMgFwGhpkDqIAoCvzyRz1n7Tn0uczpPnOk+Lxfr36dqqeeqvp198y3q5+u6k5VIUnqr/utdAGSpOVl0EtSzxn0ktRzBr0k9ZxBL0k9Z9BLUs8Z9PchSa5K8uw5lj0tybcXuZ0Dk6wbb3VD97NfkguT3JrkVcu9v3FJsipJJdl6pWuRwKBXU1X/VFX7rXQds7weOLuqtq2qd83XcUu9+IwqyTZJPpDk6vYCdmGSg1e6LvWbQa+fZ48EvrXSRYzZ1sC1wDOAhwH/Azg1yarl3rHvMO67DPr7ntVJLk7ygySnJHkg3POIOMkTBoZN/r71fevghpK8NsnGJOuT/M5cO0xyVJIr27a+m+TFrX3vJF9IclOSG5N8JMn2bdkXgGcC705yW5J929HwO5Jck2RDkvcmeVCShwD/COzW+t6WZLcktyd5+EAdT0wyneT+Q2o8IMnXkny/3Z93J3nAwPJKcnSS7yT5XpK/SZK2bKtW141JrgR+ba7Hoqp+WFVvrqqrquruqvok8F3gifM8dl9N8tftObssybMGlj+svUNYn+S6JG9NstWsdY9PcjPw5iSPSfKltq0bk5wysK2nJPl6W/b1JE8ZWHZ2kv/Vtndrks8m2akte2CSv2vP4/fburvM9RhoBVSVt/vIDbgKOA/YDdgRuBQ4ui07EFjXph8AXA28Grg/8OvAT4C3DvS9E3hLW34IcDuww5B9PgS4Bdivze8KPLZNPwZ4DrANMAF8GfjLgXXPBv7LwPxfAme02rcF/h/wp7PrH+h/JvB7A/PHA389x2PzRODJdEfcq9pj85qB5QV8Etge2AuYBg5qy44GLgP2bLV9sfXfehHPyS7Aj4FfmGP5Ue2x/oP2WL8Q+AGwY1v+f4H3tcd55/b8vmLWuv+13a8HAScDf0x3kPdA4Jdb3x2B7wFHtL6Ht/mHDzwXVwD7tu2cDRzXlr2iPRcPBrZqj+V2K/3v3dvAv6OVLsDbFnyyu6B/ycD824H3tun/H5TA04HrgAz0/QqbBv2PBoMM2Ag8ecg+HwJ8H/gN4EEL1Pc84MKB+bNpQQ8E+CGw98Dy/wh8d3b9A8tfCHy1TW8F3AAcsMjH6jXA6QPzNROKbf5U4Ng2/QXaC2ab/5XFBH0L7s8B75unz1HA9bOei/NaIO8C3DH4uLaA/uLAutfM2t6HgLXAHrPajwDOm9X2NeCogefivw8s+33g0236ZcA/A49b6X/j3obfHLq577lhYPp24KFD+uwGXFftf3Fz7aw+N1XVnQttq6p+SBe4RwPrk3wqyS8AJNk5ycfakMMtwN8BO81R9wTdEeP5bXjg+8CnW/tcPgHsn+TRdO8cflBV5w3r2IaGPpnkhlbL/x5Sy1yP3W5s+vhcPU9NM/u7H/BhundKxyzQffZzcXXb5yPpXizWDzwm76M7sp8x+3l7Pd2L5nlJvpXkZQP3YXbdVwO7D8zPdf8/DHwG+FiS65O8fdjwmFaOQa9h1gO7z4xBN3sudWNV9Zmqeg7dsM1lwPvboj+lO/J9XFVtB7yELoSGuZHuXcRjq2r7dntYVc2EzT2+hrWqfkx35P1iuiPWD89T5gmttn1aLW+cp5bZ1rPp47PXfJ3b4/oBuiPy36iqny6w/dnPxV50R/nX0h3R7zTwmGxXVY8d6LvJ41JVN1TV71bVbnRDLu9J8pi2vUfO2u9edO/s5lVVP62qP6mq/YGnAIcCL11oPW05Br2G+RpwF3BMkq2THAYcsJQNJdklyXPbB6Z3ALe1bUM3zn4b8P0kuwP/ba7tVNXddC8QxyfZuW179yS/2rpsAB6e5GGzVv0Q3RDGc+neMcxlW7rPEm5r7zh+b/H3klOBVyXZI8kOwLEL9D8B+HfAf66qHy1i+zu37d8/yQvaumdW1Xrgs8A7k2yX5H7tA+5nzLWhJC9Iskeb/R7dC8FddJ9n7Jvkt9tz/kJgf7rPJeaV5JlJ/kP7EPgW4Kf87DnWzwGDXvdQVT+h+wD25XTj6y+h+w9/xxI2dz/gtXRHjDfTnVb4+23ZnwBPoPtw8VPAxxfY1huAy4Fz2vDK54D9Ws2X0X3QeGUbxtittX8VuBu4oKqummfbrwN+G7iV7gXllHn6zvZ+uqGLbwAXzHc/kjyS7kh6NXDDwFlCL55n++cC+9C9q3kb8JtVdVNb9lK6D8//hS64/4HundNcfgk4N8ltdB9sv7qqvtu2dyjdc3UT3RDPoVV147z3vPOItt9b6D7E/hLzv6hqC8umQ3/ScEnOpfvg9m9XupbNle5UzY9W1f9Z6Vo2V5Kj6D6Q/uWVrkX3Xh7Ra6gkz0jyiPY2/kjgcXQfft6rJPkluncNm3OELvWKV8ppLvvRjT0/lO786d9sY8L3GklOojtl89VVdetK1yOtFIduJKnnFhy6SbJnki8mubSdd/vq1r5jkrPaJeFntbMNSOddSS5Pd6n9E5b7TkiS5rbgEX2SXYFdq+qCJNsC59O9HT4KuLmqjktyLN3l729IcgjdJdeHAE8C/qqqnjTfPnbaaadatWrVyHdGku5Lzj///Burar6LBoFFjNG3cdn1bfrWJJfSXS13GN1l5wAn0V0i/YbW/qF2Jd85SbZPsut847urVq1iampqoVIkSQOSLHgVNmzmWTfpvkr18XTn9e4yE97t78xl17uz6WXX69j0MuqZba1JMpVkanp6enPKkCRthkUHfZKHAqfRfaPfLfN1HdI27PL0tVU1WVWTExMLvvOQJC3RooK+fUHRacBHqmrmqr8Nbfx+Zhx/Y2tfx6bf+7EH3VWRkqQVsJizbma+gOnSqvqLgUVnAEe26SPpvilwpv2l7eybJ9N9Y+C96vxrSeqTxVww9VS6b/77ZpKLWtsbgePofgLt5cA1wAvasjPpzri5nO6rTOf85SFJ0vJbzFk3X2Hur2t91uyGdrbNK0esS5I0Jn7XjST1nEEvST1n0EtSz/ntldICVh37qRXZ71XH/dqK7Ff94xG9JPWcQS9JPWfQS1LPGfSS1HMGvST1nEEvST1n0EtSzxn0ktRzBr0k9ZxBL0k9Z9BLUs8Z9JLUcwa9JPWcQS9JPbeYHwc/McnGJJcMtJ2S5KJ2u2rmt2STrEryo4Fl713O4iVJC1vM99F/EHg38KGZhqp64cx0kncCPxjof0VVrR5XgZKk0Szmx8G/nGTVsGVJAvwW8J/GW5YkaVxGHaN/GrChqr4z0PaoJBcm+VKSp821YpI1SaaSTE1PT49YhiRpLqMG/eHAyQPz64G9qurxwB8CH02y3bAVq2ptVU1W1eTExMSIZUiS5rLkoE+yNfDrwCkzbVV1R1Xd1KbPB64A9h21SEnS0o1yRP9s4LKqWjfTkGQiyVZt+tHAPsCVo5UoSRrFYk6vPBn4GrBfknVJXt4WvYhNh20Ang5cnOQbwD8AR1fVzeMsWJK0eRZz1s3hc7QfNaTtNOC00cuSJI2LV8ZKUs8Z9JLUcwa9JPWcQS9JPWfQS1LPGfSS1HMGvST1nEEvST1n0EtSzxn0ktRzBr0k9ZxBL0k9Z9BLUs8Z9JLUcwa9JPWcQS9JPWfQS1LPGfSS1HOL+c3YE5NsTHLJQNubk1yX5KJ2O2Rg2R8luTzJt5P86nIVLklanMUc0X8QOGhI+/FVtbrdzgRIsj/dj4Y/tq3zniRbjatYSdLmWzDoq+rLwM2L3N5hwMeq6o6q+i5wOXDACPVJkkY0yhj9MUkubkM7O7S23YFrB/qsa233kGRNkqkkU9PT0yOUIUmaz1KD/gRgb2A1sB54Z2vPkL41bANVtbaqJqtqcmJiYollSJIWsqSgr6oNVXVXVd0NvJ+fDc+sA/Yc6LoHcP1oJUqSRrGkoE+y68Ds84GZM3LOAF6UZJskjwL2Ac4brURJ0ii2XqhDkpOBA4GdkqwD3gQcmGQ13bDMVcArAKrqW0lOBf4FuBN4ZVXdtTylS5IWY8Ggr6rDhzR/YJ7+bwPeNkpRkqTx8cpYSeo5g16Ses6gl6SeM+glqecMeknqOYNeknrOoJeknjPoJannDHpJ6jmDXpJ6zqCXpJ4z6CWp5wx6Seo5g16Ses6gl6SeM+glqecMeknqOYNeknpuwaBPcmKSjUkuGWj78ySXJbk4yelJtm/tq5L8KMlF7fbe5SxekrSwxRzRfxA4aFbbWcC/r6rHAf8K/NHAsiuqanW7HT2eMiVJS7Vg0FfVl4GbZ7V9tqrubLPnAHssQ22SpDEYxxj9y4B/HJh/VJILk3wpydPmWinJmiRTSaamp6fHUIYkaZiRgj7JHwN3Ah9pTeuBvarq8cAfAh9Nst2wdatqbVVNVtXkxMTEKGVIkuax5KBPciRwKPDiqiqAqrqjqm5q0+cDVwD7jqNQSdLSLCnokxwEvAF4blXdPtA+kWSrNv1oYB/gynEUKklamq0X6pDkZOBAYKck64A30Z1lsw1wVhKAc9oZNk8H3pLkTuAu4OiqunnohiVJW8SCQV9Vhw9p/sAcfU8DThu1KEnS+HhlrCT1nEEvST1n0EtSzxn0ktRzBr0k9ZxBL0k9Z9BLUs8Z9JLUcwa9JPWcQS9JPWfQS1LPGfSS1HMGvST1nEEvST1n0EtSzxn0ktRzBr0k9ZxBL0k9t6igT3Jiko1JLhlo2zHJWUm+0/7u0NqT5F1JLk9ycZInLFfxkqSFLfaI/oPAQbPajgU+X1X7AJ9v8wAHA/u02xrghNHLlCQt1aKCvqq+DNw8q/kw4KQ2fRLwvIH2D1XnHGD7JLuOo1hJ0uYbZYx+l6paD9D+7tzadweuHei3rrVtIsmaJFNJpqanp0coQ5I0n+X4MDZD2uoeDVVrq2qyqiYnJiaWoQxJEowW9BtmhmTa342tfR2w50C/PYDrR9iPJGkEowT9GcCRbfpI4BMD7S9tZ988GfjBzBCPJGnL23oxnZKcDBwI7JRkHfAm4Djg1CQvB64BXtC6nwkcAlwO3A78zphrliRthkUFfVUdPseiZw3pW8ArRylKkjQ+XhkrST1n0EtSzxn0ktRzBr0k9ZxBL0k9Z9BLUs8Z9JLUcwa9JPWcQS9JPWfQS1LPGfSS1HMGvST1nEEvST1n0EtSzxn0ktRzBr0k9ZxBL0k9Z9BLUs8t6qcEh0myH3DKQNOjgf8JbA/8LjDd2t9YVWcuuUJJ0kiWHPRV9W1gNUCSrYDrgNPpfgz8+Kp6x1gqlCSNZFxDN88Crqiqq8e0PUnSmIwr6F8EnDwwf0ySi5OcmGSHYSskWZNkKsnU9PT0sC6SpDEYOeiTPAB4LvD3rekEYG+6YZ31wDuHrVdVa6tqsqomJyYmRi1DkjSHcRzRHwxcUFUbAKpqQ1XdVVV3A+8HDhjDPiRJSzSOoD+cgWGbJLsOLHs+cMkY9iFJWqIln3UDkOTBwHOAVww0vz3JaqCAq2YtkyRtYSMFfVXdDjx8VtsRI1UkSRorr4yVpJ4z6CWp5wx6Seo5g16Ses6gl6SeM+glqecMeknqOYNeknrOoJeknjPoJannDHpJ6jmDXpJ6zqCXpJ4z6CWp5wx6Seo5g16Ses6gl6SeM+glqedG+ilBgCRXAbcCdwF3VtVkkh2BU4BVdL8b+1tV9b1R9yVJ2nzjOqJ/ZlWtrqrJNn8s8Pmq2gf4fJuXJK2A5Rq6OQw4qU2fBDxvmfYjSVrAOIK+gM8mOT/Jmta2S1WtB2h/d569UpI1SaaSTE1PT4+hDEnSMCOP0QNPrarrk+wMnJXkssWsVFVrgbUAk5OTNYY6JElDjHxEX1XXt78bgdOBA4ANSXYFaH83jrofSdLSjBT0SR6SZNuZaeBXgEuAM4AjW7cjgU+Msh9J0tKNOnSzC3B6kpltfbSqPp3k68CpSV4OXAO8YMT9SJKWaKSgr6orgV8c0n4T8KxRti1JGg+vjJWknjPoJannDHpJ6jmDXpJ6zqCXpJ4z6CWp5wx6Seo5g16Ses6gl6SeM+glqecMeknqOYNeknrOoJeknjPoJannDHpJ6jmDXpJ6zqCXpJ4z6CWp55Yc9En2TPLFJJcm+VaSV7f2Nye5LslF7XbI+MqVJG2uUX4z9k7gtVV1QZJtgfOTnNWWHV9V7xi9PEnSqJYc9FW1Hljfpm9Ncimw+7gKkySNx1jG6JOsAh4PnNuajklycZITk+wwxzprkkwlmZqenh5HGZKkIUYO+iQPBU4DXlNVtwAnAHsDq+mO+N85bL2qWltVk1U1OTExMWoZkqQ5jBT0Se5PF/IfqaqPA1TVhqq6q6ruBt4PHDB6mZKkpRrlrJsAHwAuraq/GGjfdaDb84FLll6eJGlUo5x181TgCOCbSS5qbW8EDk+yGijgKuAVI1UoSRrJKGfdfAXIkEVnLr0cSdK4eWWsJPWcQS9JPWfQS1LPGfSS1HMGvST1nEEvST1n0EtSzxn0ktRzBr0k9ZxBL0k9Z9BLUs8Z9JLUcwa9JPWcQS9JPWfQS1LPGfSS1HMGvST1nEEvST23bEGf5KAk305yeZJjl2s/kqT5LUvQJ9kK+BvgYGB/uh8M33859iVJmt9yHdEfAFxeVVdW1U+AjwGHLdO+JEnz2HqZtrs7cO3A/DrgSYMdkqwB1rTZ25J8e5lqkUa1E3Djlt5p/mxL71H3Qo9cTKflCvoMaatNZqrWAmuXaf/S2CSZqqrJla5DWqrlGrpZB+w5ML8HcP0y7UuSNI/lCvqvA/skeVSSBwAvAs5Ypn1JkuaxLEM3VXVnkmOAzwBbASdW1beWY1/SFuAQo+7VUlUL95Ik3Wt5Zawk9ZxBL0k9Z9BLUs8Z9JLUcwa97jOSrEpyycD865K8OcnZSf4syXlJ/jXJ0wb6/1OSC9rtKQPrvj7JN5N8I8lxre0xST7X2i5IsveWv5fSPS3XlbHSvc3WVXVAkkOANwHPBjYCz6mqHyfZBzgZmExyMPA84ElVdXuSHds2PgIcV1WnJ3kgHkjp54RBL3U+3v6eD6xq0/cH3p1kNXAXsG9rfzbwt1V1O0BV3ZxkW2D3qjq9tf14SxUuLcSg133JnWx6lP3Agek72t+7+Nn/iz8ANgC/2NabCe8w67ubGP79TtLPBd9a6r5kA7Bzkocn2QY4dIH+DwPWV9XdwBF0V3kDfBZ4WZIHAyTZsapuAdYleV5r22ZmubTSDHrdZ1TVT4G3AOcCnwQuW2CV9wBHJjmHbtjmh207n6b77qapJBcBr2v9jwBeleRi4J+BR4z9TkhL4FcgSFLPeUQvST1n0EtSzxn0ktRzBr0k9ZxBL0k9Z9BLUs8Z9JLUc/8GBaM0SfxbsckAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFVlJREFUeJzt3Xu0JWV95vHvI81FLnJtkIuxURqM44qIHYZoVBLMEkGFUZyBUdI6ZHqZ6OB1lHGtGRl1RVgaNSZegqLTBkdFJMGAoyKKl4mg3XgDQWEAoaWBQ6C5iMjtN3/U22bbnNOn++xz+jQv389ae+1db71V9VbV3s9++62zq1NVSJL69aj5boAkaW4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoH0GSXJvkuVPMe1aSn27geg5Nsmp2Wzfpdg5I8v0kdyY5ca63N1uSLEpSSRbMd1skMOjVVNW3quqA+W7HOt4MXFhVO1TVB9ZXcVN9+cymJIuT3JPkjPlui/pm0Gtz9njgsvluxBz6IPC9TbUx/4XxyGXQP/IcmORHSW5P8tkk28BDe8RJDhoZNvlcq/vO0RUleWOSm5OsTvLKqTaY5BVJrm7ruibJy1r5E5N8Lcm/JLklyaeS7NTmfQ34I+Bvk9yVZP8kWyd5T5LrktyU5CNJHp1kO+D/AHu1uncl2SvJ3Ul2HWnH05NMJNlykjYenOQ7Sda0/fnbJFuNzK8kr0pyZZLbknwwSdq8LVq7bklyNXDkdCchybHAGuCCaeqdnOSsdvzvTHJJkqeOzN8ryefbfl0zOsQ1suwZSe4AXtH2c0WSO9oxfO9I/RcluawdgwuT/O7IvGuTvGmK985uSc5ty92a5FtJzJbNSVX5eIQ8gGuB7wJ7AbsAlwOvavMOBVa111sBPwdeC2wJvBi4F3jnSN37gbe3+UcAdwM7T7LN7YA7gAPa9J7Av2mv9wP+BNgaWAh8E3j/yLIXAn82Mv1+4Aut7TsA/wS8a932j9T/IvDnI9PvA/5mimPzdOAQYAGwqB2b143ML+BcYCfgd4AJ4PA271XAFcDjWtu+3uovmGJbjwF+1uqfDJyxnnN2MnAfcEw71m8CrmmvHwWsBP5HO2dPAK4GnrfOske3uo8GvgMc3+ZvDxzSXu8P/LKdjy0Zhs2uArbagPfOu4CPtOW2BJ4FZL7f7z7+9eG37iPPB6rqhqq6lSEoD5ykztrA+0BV3VdVZzN8yEfdB7y9zf8icBcw1Rj/g8BTkjy6qlZX1WUAVXVVVZ1fVb+uqgngvcBzJltB6z3/Z+D1VXVrVd0J/CVw7Hr2dTnw8rb8FsBxwN9PVrGqVlbVRVV1f1VdC/zdJG05parWVNV1DGG+9tj9e4YvqOvbcX3XetoE8A7g9Kq6fpp6a62sqrOq6j6GY7QNwzn6fWBhVb29qu6tqquBj/Lbx+Q7VfWPVfVgVf2K4bztl2S3qrqrqi5q9f4DcF47H/cB72H4YnjGyLqmeu/cx/AF/vj2fvhWVXkTrc2IQf/Ic+PI67sZenXr2gv4xTof1nVD6V+q6v7p1lVVv2QIkVcBq5Ocl+RJAEl2T/KZJL9oQwtnALtN0e6FwLbAyjZEsAb4UiufyjnAk5M8gaGnentVrfuFRWvL/m344cbWlr+cpC1THbu9+O3j8/OpGpTkQOC5DP+62FC/WXdVPQisatt8PMNw1ZqRY/JWYI/Jlm1OYOi9X5Hke0leMLIPv2l32871wN4jy061/+9m6P1/pQ3RnbQR+6ZNwKDXZFYDe68dg24eN9OVVdWXq+pPGHp9VzD0OmHo+Rbwe1X1GIbedyZfC7cAv2IY9tmpPXasqrVh85AeZFXdA5wJvAw4nil6882HW9sWt7a8dT1tWddqfvv4/M566h7KMDR0XZIbGYZiXpLkkvUs85t1t7HvfYAbGIL4mpHjsVMNf6F0xMiyv3VcqurKqjoO2B04FTirXeO4geGLY+120rb7i/W0a+0676yqN1bVE4AXAm9Icth0y2nTMeg1me8ADwCvSbIgyVHAwTNZUZI92kW+7YBfMwzxPNBm79Cm1yTZG/ivU62n9TA/Crwvye5t3XsneV6rchOwa5Id11n0k8ArgBcx/IthKjswXEu4q/2L4883fC85EzgxyT5JdgbW16M9DXgiw7DHgQxj2+cBz1vPMk9P8uIMfzXzOobjeBHDcNodSd7SLkpvkeQpSX5/qhUleXmShe14rmnFD7R9ODLJYe1i9Rvbdv55up1P8oIk+7Uvhzva+h6YZjFtQga9HqKq7mW4AHsCQxi8nOFC5K9nsLpHMYTGDcCtDOPef9Hm/U/gIOB2hrA7e5p1vYVhiOCiNrzyVdp1gaq6Avg0cHUbxtirlf9fhmsEl7Sx96m8CfiPwJ0MXyif3Yh9/CjwZeCHwCXr24+quruqblz7YPiiu6ddo5jKOQzDX7cx/MvkxW0s/AGGHvSBDBdobwE+Bqz7ZTfqcOCyJHcBfw0cW1X3VNVPGc7z37T1vBB4YXsvTGcxw7m4i6GT8KGqunADltMmEq+ZaEMkuRj4SFV9Yr7bsrEy/Knm/66qj813WzZWkpOB/arq5fPdFj182aPXpJI8J8lj29DNUuD3GC5+Pqy0YYyD2LgeutQVfymnqRzAMG67PfD/gGOqavX8NmnjJFnO8Dfkr21/jik9Ijl0I0mdc+hGkjq3WQzd7LbbbrVo0aL5boYkPaysXLnylqpa348Ggc0k6BctWsSKFSvmuxmS9LCSZMpfYY9y6EaSOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjq3WfwyVg8fi046b162e+0pR87LdqUe2KOXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6N23QJ/l4kpuTXDpStkuS85Nc2Z53buVJ8oEkVyX5UZKD5rLxkqTpbUiP/n8Bh69TdhJwQVUtBi5o0wDPBxa3xzLgw7PTTEnSTE0b9FX1TeDWdYqPApa318uBo0fKP1mDi4Cdkuw5W42VJG28mY7R71FVqwHa8+6tfG/g+pF6q1rZQyRZlmRFkhUTExMzbIYkaTqzfTE2k5TVZBWr6rSqWlJVSxYuXDjLzZAkrTXToL9p7ZBMe765la8CHjdSbx/ghpk3T5I0rpkG/ReApe31UuCckfI/bX99cwhw+9ohHknS/Jj2vxJM8mngUGC3JKuAtwGnAGcmOQG4Dnhpq/5F4AjgKuBu4JVz0GZJ0kaYNuir6rgpZh02Sd0CXj1uoyRJs8dfxkpS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdW6soE/y+iSXJbk0yaeTbJNk3yQXJ7kyyWeTbDVbjZUkbbwZB32SvYETgSVV9RRgC+BY4FTgfVW1GLgNOGE2GipJmplxh24WAI9OsgDYFlgN/DFwVpu/HDh6zG1IksYw46Cvql8A7wGuYwj424GVwJqqur9VWwXsPdnySZYlWZFkxcTExEybIUmaxjhDNzsDRwH7AnsB2wHPn6RqTbZ8VZ1WVUuqasnChQtn2gxJ0jTGGbp5LnBNVU1U1X3A2cAzgJ3aUA7APsANY7ZRkjSGcYL+OuCQJNsmCXAY8BPg68Axrc5S4JzxmihJGsc4Y/QXM1x0vQT4cVvXacBbgDckuQrYFTh9FtopSZqhBdNXmVpVvQ142zrFVwMHj7NeSdLs8ZexktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXNjBX2SnZKcleSKJJcn+YMkuyQ5P8mV7Xnn2WqsJGnjjduj/2vgS1X1JOCpwOXAScAFVbUYuKBNS5LmyYyDPsljgGcDpwNU1b1VtQY4Cljeqi0Hjh63kZKkmRunR/8EYAL4RJLvJ/lYku2APapqNUB73n2yhZMsS7IiyYqJiYkxmiFJWp9xgn4BcBDw4ap6GvBLNmKYpqpOq6olVbVk4cKFYzRDkrQ+4wT9KmBVVV3cps9iCP6bkuwJ0J5vHq+JkqRxzDjoq+pG4PokB7Siw4CfAF8AlraypcA5Y7VQkjSWBWMu/1+ATyXZCrgaeCXDl8eZSU4ArgNeOuY2JEljGCvoq+oHwJJJZh02znolSbPHX8ZKUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHVu7KBPskWS7yc5t03vm+TiJFcm+WySrcZvpiRppmajR/9a4PKR6VOB91XVYuA24IRZ2IYkaYbGCvok+wBHAh9r0wH+GDirVVkOHD3ONiRJ4xm3R/9+4M3Ag216V2BNVd3fplcBe0+2YJJlSVYkWTExMTFmMyRJU5lx0Cd5AXBzVa0cLZ6kak22fFWdVlVLqmrJwoULZ9oMSdI0Foyx7DOBFyU5AtgGeAxDD3+nJAtar34f4IbxmylJmqkZ9+ir6r9V1T5VtQg4FvhaVb0M+DpwTKu2FDhn7FZKkmZsLv6O/i3AG5JcxTBmf/ocbEOStIHGGbr5jaq6ELiwvb4aOHg21itJGp+/jJWkzhn0ktS5WRm6mU+LTjpv3rZ97SlHztu2JWlD2aOXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1LmH/S0QpF7N1+09vLVHf+zRS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1LkZB32SxyX5epLLk1yW5LWtfJck5ye5sj3vPHvNlSRtrHF69PcDb6yq3wUOAV6d5MnAScAFVbUYuKBNS5LmyYyDvqpWV9Ul7fWdwOXA3sBRwPJWbTlw9LiNlCTN3KyM0SdZBDwNuBjYo6pWw/BlAOw+xTLLkqxIsmJiYmI2miFJmsTYQZ9ke+DzwOuq6o4NXa6qTquqJVW1ZOHCheM2Q5I0hbGCPsmWDCH/qao6uxXflGTPNn9P4ObxmihJGsc4f3UT4HTg8qp678isLwBL2+ulwDkzb54kaVwLxlj2mcDxwI+T/KCVvRU4BTgzyQnAdcBLx2uiJM2tRSedN2/bvvaUI+d8GzMO+qr6NpApZh820/VKkmaXv4yVpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnZuToE9yeJKfJrkqyUlzsQ1J0oaZ9aBPsgXwQeD5wJOB45I8eba3I0naMHPRoz8YuKqqrq6qe4HPAEfNwXYkSRsgVTW7K0yOAQ6vqj9r08cD/7aqXrNOvWXAsjZ5APDTGW5yN+CWGS6rueN52fx4TjZP45yXx1fVwukqLZjhytcnk5Q95Nukqk4DTht7Y8mKqloy7no0uzwvmx/PyeZpU5yXuRi6WQU8bmR6H+CGOdiOJGkDzEXQfw9YnGTfJFsBxwJfmIPtSJI2wKwP3VTV/UleA3wZ2AL4eFVdNtvbGTH28I/mhOdl8+M52TzN+XmZ9YuxkqTNi7+MlaTOGfSS1DmDXtKkkhya5Nz5bocgyaIkl850eYNekjq32QT9ut9YSd6U5OQkFyY5Ncl3k/wsybNG6n8rySXt8YyRZd+c5MdJfpjklFa2X5KvtrJLkjxx0+9lX5L8Y5KVSS5rv3Ree0O7S9pxvqCVbZ/kE+2c/CjJS+a35Q9f7bPwFyPTJyd5Y5IPtfNwbpIvtl+ok+SwJN9vx/7jSbaepvzwJFck+Tbw4nnZyQ4k+e/tOJ6f5NMtzw5MclH7DPxDkp1b3anKn94+R98BXj1Wg6pqs3gAi4BLR6bfBJwMXAj8VSs7Avhqe70tsE17vRhY0V4/H/hnYNs2vUt7vhj4d+31Nmvn+xjrnK09to8GLgX2AK4H9l1n/qnA+0eW23m+2/5wfQBPA74xMv0T4E+BLzJ03B4L3AYc097n1wP7t7qfBF63AeWLGX7hfiZw7nzv88PtASwBftA+FzsAV7Y8+xHwnFbn7Ws/ExtY/u7RfNzYx2bTo5/G2e15JcMXAsCWwEeT/Bj4HMOdMgGeC3yiqu4GqKpbk+wA7F1V/9DK7lk7X2M5MckPgYsYfg29DPhmVV0Dw7Fv9Z7LcEdTWvltm7qhvaiq7wO7J9kryVMZQv0g4HNV9WBV3Qh8vVU/ALimqn7WppcDz15P+ZNa+ZU1pMsZm2avuvOHwDlV9auquhP4J2A7YKeq+karsxx4dpIdN7D878dp0Fzc62am7ue3h5K2GXn96/b8AP/a5tcDNwFPbcvd08rDQ++tM9n9dzSGJIcyBPgfVNXdSS4EfsgQIg+pziT3O9KMncXQY38sw91h95ui3lTv+/V9HjxP45uNvJnVz8zm1KO/iaGnsmsbL3zBNPV3BFZX1YPA8Qy/wgX4CvCfkmwLkGSXqroDWJXk6Fa29dr5mrEdgdtayD8JOATYGnhOkn1hOPat7leA39y9dO0YpGbsMwy3FjmGIfS/DbwkyaOS7AEc2updASxKsvaL4HjgG9OU7zty/eq4ud6RTn0beGGSbZJsDxwJ/BK4be01Rtoxr6rbpyhfA9ye5A9b+cvGadBmE/RVdR/D+NTFwLkMb7r1+RCwNMlFwP4MB5Kq+hLDvXVWJPkBw9gYDAfwxCQ/YhjDf+ys78Qjy5eABe14voNh+GaCYfjm7Dak89lW953AzkkubeV/NB8N7kUNtxTZAfhFVa0GPs9wM8FLgb9j+AzdXlX3AK8EPteGOB8EPjJN+TLgvHYx9uebeNe6UFXfY8igHzIMO68AbgeWAu9un5kDGfKO9ZS/Evhguxj7q3Ha5C0QpA4k2b6q7kqyK/Bd4JltvF7zYOR8bAt8E1hWVZfMV3s2pzF6STN3bpKdgK2Adxjy8+60DP+F6jbA8vkMebBHL0nd22zG6CVJc8Ogl6TOGfSS1DmDXpI6Z9BLUuf+P/OJkE6+wguCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFdpJREFUeJzt3Xu4JVV95vHvK80d5NogNIZGBYzxiYotYryhmIlCFCbiRAdJB0kYjUZNNEKcyehEnwmazKgxXgKiNIk3vELAaBRFJZHGBhRBMDCAgDTQKKDgFfnNH7XabI7n1mefw2lWfz/Ps5+za9WqWmtX9X537VW7qlNVSJL69YDF7oAkaWEZ9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPo78eSXJvkGVPMe3KSb81yPQcnuWF+ezdpO/snuTjJD5K8fKHbmy9JliepJEsWuy/SXBj0naqqL1fV/ovdjwleA5xbVdtX1d9OV/G++vCRNgUGve5LewOXLXYn7i8W+xtEks0Ws33NH4P+/u/RSS5JckeSDyfZCn75iDjJASPDJh9pdd84uqIkr0pyS5K1SY6ZqsEkv5/k6raua5Ic1cofmuTzSb6b5NYk70+yY5v3eeBpwN8luTPJfkm2TPI3Sa5LcnOSdyfZOsm2wD8De7a6dybZM8kPk+wy0o/HJlmXZPNJ+nhgkq8kub29nr9LssXI/Ery4iRXJrktyTuSpM3brPXr1iRXA4dNtwPaENqftf1wV5JTkuye5J/bNvpckp1G6j8nyWWtb+cm+dUJ6zo+ySXAXUmWtNf+sfZar5lu2CvJqW07fra1/cUke4/Mf3ib970k30ryXyYs+64kn0pyF/C0JIcm+WZb13eSvHqk/h8muaqt68wke85y+z6s9euOto0/PN321TyoKh/30wdwLXABsCewM3A58OI272DghvZ8C+DbwCuAzYHfAX4KvHGk7t3AX7b5hwI/BHaapM1tge8D+7fpPYBfa88fBvwmsCWwFPgS8NaRZc8F/mBk+q3Ama3v2wP/BPzVxP6P1P8U8JKR6bcAb59i2zwWOAhYAixv2+aVI/MLOAvYEfgVYB3wzDbvxcAVwINb377Q6i+ZZj+cD+wOLANuAS4CHtO2xeeB17W6+wF3te20OcNw1lXAFiPr+lpre2uGg7ELgf/Z9uNDgKuB35qiL6cCPwCe0tp+G3DeyL67HjimbZcDgFtH9t+pwB3AE1u7WwFrgSe3+TsBB7TnT2/LHtDaeTvwpVlu3w8C/32kjSct9nup98eid8DHGDtvCIUXjky/GXh3e/6LoGxv+u8AGal7HvcO+h+NBlkLq4MmaXNb4HbgucDWM/TvCODikelzaUEPpAXeQ0fmPwG4ZmL/R+b/LvCv7flmwE3AgbPcVq8EPjEyXaMBA5wOnNCef572gdmm/xMzB/1RI9MfA941Mv3HwCfb878ATh+Z94C2bw4eWdeLRuY/HrhuQnt/Drxvir6cCnxoZHo74OcMHxy/C3x5Qv2/5z8+hE4FTpsw/zrgvwEPnFB+CvDmCe38DFg+i+17GnASsNdiv4c2lYdDN/d/N408/yHDG26iPYHvVHuXNddPqPPdqrp7pnVV1V0MgfFiYG2Ss5M8HCDJbkk+1L7ifx/4R2DXKfq9FNgGuLANYdwOfLqVT+UM4BFJHsJwRHxHVV0wWcU2NHRWkptaX/73JH2Zatvtyb23z7en6dN6N488/9Ek06Pr/sX6quqe1taykfqjbe/NMIR1+8h2ei3Dt4ep/GL5qroT+F5rd2/g8RPWdRTwoCnahuED/VDg22245QlTvI47ge9OeB1Tbd/XMHzQX9CGsF40zWvRPDDoNw1rgWXrx0ibB891ZVX1mar6TYZhmyuAk9usv2I4kvv1qnog8EKGN/RkbmUIwF+rqh3bY4eqWh8Gv3Rb1ar6McOR4VHA0cA/TNPNd7W+7dv68tpp+jLRWu69fX5llsvNxo0MgQtA2ycPZjiqX2/iB/I1I9toxxp+tXToNG38ou9JtmMYfrqxreuLE9a1XVW9ZIq2qaqvVtXhwG7AJxm2/2SvY1tglwmvY1JVdVNV/WFV7cnwbeGdSR4203KaO4N+0/AVhq/vL2sn9w4HDpzLitpJxue0N/ZPgDvbumEYZ78TuD3JMuDPplpPO5I9GXhLkt3aupcl+a1W5WZglyQ7TFj0NOD3gecwfGOYyvYM5xLubN84XjJN3YlOB16eZK92EvWEDVh2Nus+LMkh7STyqxi2479NUf8C4PvtBO3W7UTxI5M8bpo2Dk3ypAwnn98ArK6q6xnGzPdLcnSSzdvjcaMng0cl2SLJUUl2qKqfMWzP9fv6A8AxSR6dZEuGb0yrq+ramTZAkucl2atN3sbw4fLzaRbRmAz6TUBV/ZThBOyxDOPrL2R40/9kDqt7AEM43cgwJPBU4I/avP/FcHLuDuBs4OMzrOt4hhOR57fhlc8B+7c+X8Fw0u7qNsywZyv/V+Ae4KIZQuXVwH9lODF5MrAhv+w4GfgM8HWGk6ozvY5Zq6pvMWz/tzN8q3k28Oy2jyar//NW59HANW2Z9wATPwBHfQB4HcP+eSzDNyCq6gcM5xuez7D/bgLexHAydSpHA9e2/fPi1neq6hyG8w0fY/gG9NC23tl4HLA6yZ0MJ+NfUVXXzHJZzUHuPWyrTUWS1Qwnbt+32H3ZUBl+qvmBqnrPYvdlY5PkVIaT2P9jsfuijYdH9JuIJE9N8qA2dLMS+HWGk5/3K23I4gA27Ahd2qR5745Nx/4M48PbAf8POLKq1i5ulzZMklUMP9l8RRuGkDQLDt1IUudmHLpJ8t4Ml8VfOlK2c7uM+sr2d6dWniR/2y6LviTJAQvZeUnSzGY8ok/yFIafzJ1WVY9sZW8GvldVJyY5geFS+eOTHMpwFeChDFf0va2qHj9TJ3bddddavnz5eK9EkjYxF1544a1VNd1FhsAsxuir6ktJlk8oPpzhEnWAVQyXth/fyk9rV2Cen2THJHvMNBa8fPly1qxZM1NXJEkjkszmqu05/+pm9/Xh3f7u1sqXce9LqG/g3pdEj3bwuCRrkqxZt27dHLshSZrJfP+8crJLzCcdG6qqk6pqRVWtWLp0xm8ekqQ5mmvQ35xkD4D295ZWfgP3vkfIXgxX4EmSFslcg/5MYGV7vpLhroLry3+v/frmIIa7C96vfqstSb2Z8WRskg8ynHjdNcP/WPQ64ETg9CTHMtyv+nmt+qcYfnFzFcNtSaf8X4okSfeN2fzq5gVTzDpkkroFvHTcTkmS5o/3upGkzhn0ktQ5g16SOufdK7VBlp9w9qK0e+2Jhy1Ku1IPPKKXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnRsr6JP8SZLLklya5INJtkqyT5LVSa5M8uEkW8xXZyVJG27OQZ9kGfByYEVVPRLYDHg+8CbgLVW1L3AbcOx8dFSSNDfjDt0sAbZOsgTYBlgLPB34aJu/CjhizDYkSWOYc9BX1XeAvwGuYwj4O4ALgdur6u5W7QZg2WTLJzkuyZoka9atWzfXbkiSZjDO0M1OwOHAPsCewLbAsyapWpMtX1UnVdWKqlqxdOnSuXZDkjSDcYZungFcU1XrqupnwMeB3wB2bEM5AHsBN47ZR0nSGMYJ+uuAg5JskyTAIcA3gS8AR7Y6K4EzxuuiJGkc44zRr2Y46XoR8I22rpOA44E/TXIVsAtwyjz0U5I0R0tmrjK1qnod8LoJxVcDB46zXknS/PHKWEnqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdGyvok+yY5KNJrkhyeZInJNk5yWeTXNn+7jRfnZUkbbhxj+jfBny6qh4OPAq4HDgBOKeq9gXOadOSpEUy56BP8kDgKcApAFX106q6HTgcWNWqrQKOGLeTkqS5G+eI/iHAOuB9SS5O8p4k2wK7V9VagPZ3t8kWTnJckjVJ1qxbt26MbkiSpjNO0C8BDgDeVVWPAe5iA4ZpquqkqlpRVSuWLl06RjckSdMZJ+hvAG6oqtVt+qMMwX9zkj0A2t9bxuuiJGkccw76qroJuD7J/q3oEOCbwJnAyla2EjhjrB5KksayZMzl/xh4f5ItgKuBYxg+PE5PcixwHfC8MduQJI1hrKCvqq8BKyaZdcg465UkzR+vjJWkzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1Llx/8/YRbf8hLMXre1rTzxs0dqWpNnyiF6SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOjR30STZLcnGSs9r0PklWJ7kyyYeTbDF+NyVJczUfR/SvAC4fmX4T8Jaq2he4DTh2HtqQJM3RWEGfZC/gMOA9bTrA04GPtiqrgCPGaUOSNJ5xj+jfCrwGuKdN7wLcXlV3t+kbgGVjtiFJGsOcgz7JbwO3VNWFo8WTVK0plj8uyZoka9atWzfXbkiSZjDOEf0TgeckuRb4EMOQzVuBHZOs/0/H9wJunGzhqjqpqlZU1YqlS5eO0Q1J0nTmHPRV9edVtVdVLQeeD3y+qo4CvgAc2aqtBM4Yu5eSpDlbiN/RHw/8aZKrGMbsT1mANiRJs7Rk5iozq6pzgXPb86uBA+djvZKk8XllrCR1zqCXpM4Z9JLUOYNekjpn0EtS5+blVzeS5t/yE85elHavPfGwRWlXC8cjeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHVuzkGf5MFJvpDk8iSXJXlFK985yWeTXNn+7jR/3ZUkbahxjujvBl5VVb8KHAS8NMkjgBOAc6pqX+CcNi1JWiRzDvqqWltVF7XnPwAuB5YBhwOrWrVVwBHjdlKSNHfzMkafZDnwGGA1sHtVrYXhwwDYbT7akCTNzdhBn2Q74GPAK6vq+xuw3HFJ1iRZs27dunG7IUmawlhBn2RzhpB/f1V9vBXfnGSPNn8P4JbJlq2qk6pqRVWtWLp06TjdkCRNY5xf3QQ4Bbi8qv7vyKwzgZXt+UrgjLl3T5I0riVjLPtE4GjgG0m+1speC5wInJ7kWOA64HnjdVGSNI45B31VnQdkitmHzHW9kqT55ZWxktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SercOHevlKQuLD/h7EVr+9oTD1vwNjyil6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnVuQoE/yzCTfSnJVkhMWog1J0uzMe9An2Qx4B/As4BHAC5I8Yr7bkSTNzkIc0R8IXFVVV1fVT4EPAYcvQDuSpFlIVc3vCpMjgWdW1R+06aOBx1fVyybUOw44rk3uD3xrjk3uCtw6x2W1cNwvGx/3ycZpnP2yd1UtnanSkjmufDqZpOyXPk2q6iTgpLEbS9ZU1Ypx16P55X7Z+LhPNk73xX5ZiKGbG4AHj0zvBdy4AO1IkmZhIYL+q8C+SfZJsgXwfODMBWhHkjQL8z50U1V3J3kZ8BlgM+C9VXXZfLczYuzhHy0I98vGx32ycVrw/TLvJ2MlSRsXr4yVpM4Z9JLUOYNe0qSSHJzkrMXuhyDJ8iSXznV5g16SOrfRBP3ET6wkr07y+iTnJnlTkguS/HuSJ4/U/3KSi9rjN0aWfU2SbyT5epITW9nDknyulV2U5KH3/avsS5JPJrkwyWXtSuf1N7S7qG3nc1rZdkne1/bJJUmeu7g9v/9q74U/Gpl+fZJXJXln2w9nJflUu0KdJIckubht+/cm2XKG8mcmuSLJecDvLMqL7ECSv2jb8bNJPtjy7NFJzm/vgU8k2anVnar8se199BXgpWN1qKo2igewHLh0ZPrVwOuBc4H/08oOBT7Xnm8DbNWe7wusac+fBfwbsE2b3rn9XQ385/Z8q/XzfYy1z9Zv262BS4HdgeuBfSbMfxPw1pHldlrsvt9fH8BjgC+OTH8T+D3gUwwHbg8CbgOObP/Orwf2a3VPA145i/J9Ga5wPx04a7Ff8/3tAawAvtbeF9sDV7Y8uwR4aqvzl+vfE7Ms/+vRfNzQx0ZzRD+Dj7e/FzJ8IABsDpyc5BvARxjulAnwDOB9VfVDgKr6XpLtgWVV9YlW9uP18zWWlyf5OnA+w9XQxwFfqqprYNj2rd4zGO5oSiu/7b7uaC+q6mJgtyR7JnkUQ6gfAHykqu6pqpuAL7Tq+wPXVNW/t+lVwFOmKX94K7+yhnT5x/vmVXXnScAZVfWjqvoB8E/AtsCOVfXFVmcV8JQkO8yy/B/G6dBC3Otmru7m3kNJW408/0n7+3P+o89/AtwMPKot9+NWHn753jqT3X9HY0hyMEOAP6GqfpjkXODrDCHyS9WZ5H5HmrOPMhyxP4jh7rAPm6LeVP/up3s/uJ/GNx95M6/vmY3piP5mhiOVXdp44W/PUH8HYG1V3QMczXAVLsC/AC9Ksg1Akp2r6vvADUmOaGVbrp+vOdsBuK2F/MOBg4Atgacm2QeGbd/q/gvwi7uXrh+D1Jx9iOHWIkcyhP55wHOTPCDJ7sDBrd4VwPIk6z8Ijga+OEP5PiPnr16w0C+kU+cBz06yVZLtgMOAu4Db1p9jpG3zqrpjivLbgTuSPKmVHzVOhzaaoK+qnzGMT60GzmL4RzeddwIrk5wP7MewIamqTzPcW2dNkq8xjI3BsAFfnuQShjH8B837i9i0fBpY0rbnGxiGb9YxDN98vA3pfLjVfSOwU5JLW/nTFqPDvajhliLbA9+pqrXAxxhuJngp8PcM76E7qurHwDHAR9oQ5z3Au2coPw44u52M/fZ9/NK6UFVfZcigrzMMO68B7gBWAn/d3jOPZsg7pik/BnhHOxn7o3H65C0QpA4k2a6q7kyyC3AB8MQ2Xq9FMLI/tgG+BBxXVRctVn82pjF6SXN3VpIdgS2ANxjyi+6kDP+F6lbAqsUMefCIXpK6t9GM0UuSFoZBL0mdM+glqXMGvSR1zqCXpM79fw87Yc7hHrnYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for persons in columns[3]:\n",
    "    indices1 = data[:,5]==\"high\"\n",
    "    indices2 = data[:,3]==persons\n",
    "    indices = np.alltrue([indices1,indices2], axis=0)\n",
    "    plt.hist(data[indices, 6])\n",
    "    plt.title(\"high safety and {} persons\".format(persons))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## On this dataset, C4.5 and ID3 get similar accuracies..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Accuracy Comparison\n",
      "---------\n",
      " ID3    C4.5\n",
      "0.700   0.700\n",
      "0.700   0.700\n",
      "0.778   0.778\n",
      "0.819   0.824\n",
      "0.899   0.894\n",
      "0.964   0.964\n",
      "1.000   1.000\n"
     ]
    }
   ],
   "source": [
    "print(\"Training Accuracy Comparison\")\n",
    "print(\"---------\")\n",
    "print(\" ID3    C4.5\")\n",
    "for depth in range(7):\n",
    "    root = build_node(data, entropy(data[:,6]), \"all data\", 0)\n",
    "    build_tree(root, max_depth=depth, c45=False)\n",
    "    id3=correct(root)/1728\n",
    "    root = build_node(data, entropy(data[:,6]), \"all data\", 0)\n",
    "    build_tree(root, max_depth=depth, c45=True)\n",
    "    c45=correct(root)/1728\n",
    "    print('{:.3f} '.format(round(id3,3)), ' {:.3f}'.format(round(c45,3)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}

How to print tree in visulization