Michael0x2a · October 16, 2014 06:29
diff --git a/math-308-midterm-1-review.ipynb b/math-308-midterm-1-review.ipynb
 {
 "metadata": {
  "name": "",
  "signature": "sha256:335d80491f3857e5a5b5e9056726be1c349edffb5b5b8b9b2ab2e7034e78d13a"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Midterm 1 Review\n",
      "\n",
      "$$\n",
      "% units\n",
      "\\newcommand{u}[1]{\\;\\mathrm{#1}}\n",
      "%\n",
      "% derivative\n",
      "\\newcommand{dd}[2]{\\frac{\\mathrm{d}#1}{\\mathrm{d}#2}}\n",
      "%\n",
      "% double-derivative\n",
      "\\newcommand{ddd}[2]{\\frac{\\mathrm{d^2}#1}{\\mathrm{d}#2^2}}\n",
      "%\n",
      "% single-derivative\n",
      "\\newcommand{df}[1]{\\frac{\\mathrm{d}}{\\mathrm{d}#1}}\n",
      "%\n",
      "% derivative component\n",
      "\\newcommand{d}[1]{\\mathrm{d}#1}\n",
      "%\n",
      "% real numbers\n",
      "\\newcommand{R}{\\mathbb{R}}\n",
      "%\n",
      "% annotate\n",
      "\\newenvironment{annotate}[0]{\\begin{array}{cr}}{\\end{array}}\n",
      "%\n",
      "% rowop\n",
      "\\newcommand{rowop}[1]{\\quad\\overset{\\sim}{\\scriptstyle{#1}}\\quad}\n",
      "%\n",
      "% sin/cos/tan/ln with parens\n",
      "\\newcommand{cosp}[1]{\\cos{\\left(#1\\right)}}\n",
      "\\newcommand{sinp}[1]{\\sin{\\left(#1\\right)}}\n",
      "\\newcommand{tanp}[1]{\\tan{\\left(#1\\right)}}\n",
      "\\newcommand{ln}[1]{\\lnp{\\left(#1\\right)}}\n",
      "%\n",
      "% shorter frac\n",
      "\\newcommand{f}[2]{\\frac{#1}{#2}}\n",
      "%\n",
      "% rref\n",
      "\\newcommand{rref}[1]{\\mathrm{rref}\\!\\left(#1\\right)}\n",
      "%\n",
      "% parens\n",
      "\\newcommand{p}[1]{\\left(#1\\right)}\n",
      "\\newcommand{b}[1]{\\left\\{#1\\right\\}}\n",
      "%\n",
      "% column vectors\n",
      "\\newcommand{colvec}[1]{\\begin{pmatrix}#1\\end{pmatrix}}\n",
      "\\newcommand{bcolvec}[1]{\\begin{bmatrix}#1\\end{bmatrix}}\n",
      "%\n",
      "% vector symbols\n",
      "\\newcommand{v}[1]{\\vec{#1}}\n",
      "\\newcommand{vn}[2]{\\vec{#1}_{#2}}\n",
      "%\n",
      "% spans\n",
      "\\newcommand{span}[1]{\\mathrm{span}\\left\\{#1\\right\\}}\n",
      "$$\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Section 1.1: Systems of Linear Equations\n",
      "\n",
      "**Theorem 1.2**\n",
      "\n",
      "A system of linear equations has no solutions, exactly one solution, or infinitely many solutions.\n",
      "\n",
      "Proof:\n",
      "\n",
      "Take any linear system, form the augmented matrix, use Gaussian elimination to reduce to echelon form, and extra the transformed system. There are 3 possible outcomes:\n",
      "\n",
      "1. The system has an equation of the form $0 = c$ for $c \\neq 0$. In this case, the system has no solutions.\n",
      "2. The transformed system has no free variables and hence exactly one solution.\n",
      "4. The transformed system has one or more free variables and hence infinitely many solutions.\n",
      "\n",
      "---\n",
      "\n",
      "**Def: Linear Equation**\n",
      "\n",
      "A **linear equation** is an equation with the form:\n",
      "\n",
      "$$\n",
      "a_1 x_1 + a_2 x_2 + a_3 x_3 + \\cdots + a_n x_n = b\n",
      "$$\n",
      "\n",
      "...where $a_1, a_2, \\cdots, a_n$ and $b$ are constants and $x_1, x_2, \\cdots x_n$ are variables or unknowns.\n",
      "\n",
      "**Def: Solution of Linear Equation**\n",
      "\n",
      "A **solution** $(s_1, s_2, \\cdots s_n)$ is an ordered set of $n$ numbers such that we we set $x_1 = s_1, x_2 = s_2, \\cdots x_n = s_n$ then the given linear equation is satisfied.\n",
      "\n",
      "**Def: Solution set**\n",
      "\n",
      "The **solution set** for a linear equation consists of the set of all solutions to the given equation.\n",
      "\n",
      "**Def: Consistent Linear System, Inconsistent Linear System**\n",
      "\n",
      "If a linear system has at least one solution, then we say it is **consistent**. If not, then it is **inconsistent**.\n",
      "\n",
      "**Def: Leading Variable**\n",
      "\n",
      "A variable that appears as the first term in at least one equation is called a **leading variable**.\n",
      "\n",
      "**Def: Free Variable**\n",
      "\n",
      "Every variable of an echelon system which is not a leading variable.\n",
      "\n",
      "**Def: Echelon System**\n",
      "\n",
      "An **Echelon System** is a system of equations in the following format:\n",
      "\n",
      "$$\n",
      "\\left\\{\n",
      "\\begin{array}{rl}\n",
      "    a_{11} x_1 + a_{12} x_2 + \\cdots + a_{1n}x_n &= b_1 \\\\\n",
      "                 a_{22} x_2 + \\cdots + a_{2n}x_n &= b_2 \\\\\n",
      "                                         \\vdots & \\\\\n",
      "                                       a_{nn}x_n &= b_n \n",
      "\\end{array}\\right.\n",
      "$$\n",
      "\n",
      "More specifically:\n",
      "\n",
      "1. Every variable is the leading variable of _at most one equation_\n",
      "2. The system is organized in a descending \"stair step\" pattern so that the index of the leading variables increases from top to bottom.\n",
      "3. Every equation has a leading variable."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Section 1.2: Linear Systems and Matrices\n",
      "\n",
      "**Theorem 1.6**\n",
      "\n",
      "A given matrix is equivalent to a unique matrix that is in reduced echelon form.\n",
      "\n",
      "---\n",
      "\n",
      "\n",
      "**Def: Equivalent linear systems**\n",
      "\n",
      "Two linear systems are said to be **equivalent** if they have the same set of solutions.\n",
      "\n",
      "**Def: Elementary Operations**\n",
      "\n",
      "Any operation which can be applied to a linear system that produces a new system equivalent to the old one, such that the solution set is unchanged. There are only 3 valid row operations:\n",
      "\n",
      "1. Interchange the position of the two equations\n",
      "2. Multiply an equation by a nonzero constant\n",
      "3. Add a multiple of one equation to another.\n",
      "\n",
      "**Def: Matrix**\n",
      "\n",
      "A rectangular table of numbers.\n",
      "\n",
      "**Def: Augmented Matrix**\n",
      "\n",
      "A matrix containing all the coefficients of a linear equation, including the constant terms on the right side of the equation.\n",
      "\n",
      "**Def: Equivalent Matrices**\n",
      "\n",
      "Two matrices are **equivalent** if one can be obtained from the other through a sequence of elementary row operations.\n",
      "\n",
      "**Def: Zero Row, Nonzero Row**\n",
      "\n",
      "A **zero row** is a row consisting entirely of zeros. A **nonzero row** contains at least one nonzero entry.\n",
      "\n",
      "**Def: Gaussian Elimination**\n",
      "\n",
      "A procedure which transforms an augmented matrix into echelon form.\n",
      "\n",
      "**Def: Echelon Form**\n",
      "\n",
      "A matrix is in **echelon form** if:\n",
      "\n",
      "1. Every leading term is in a column to the left of the leading term of the row below it.\n",
      "2. Any zero rows are at the bottom of the matrix.\n",
      "\n",
      "**Def: Gauss-Jordan Elimination**\n",
      "\n",
      "An operation performed on a system in **echelon form** that:\n",
      "\n",
      "1. Multiplies each nonzero row by the reciprocol of the pivot so that we end up with a 1 as the leading term in each nonzero row.\n",
      "2. Use row operations to introduce zeros in the entries _above_ each pivot position.\n",
      "\n",
      "**Def: Reduced Echelon Form**\n",
      "\n",
      "A matrix is in **reduced echelon form** if:\n",
      "\n",
      "1. It is in echeclon form.\n",
      "2. All pivot positions contain a 1.\n",
      "3. The only nonzero term in a pivot column is the pivot position.\n",
      "\n",
      "**Def: Homogeneous linear equation, Homogeneous linear system**\n",
      "\n",
      "A linear equation is **homogeneous** if it has the form:\n",
      "\n",
      "$$\n",
      "a_1 x_1 + a_2 x_2 + \\cdots + a_n x_n = 0\n",
      "$$\n",
      "\n",
      "A **homogeneous linear system** is a system of equations made entirely up of homgeneous linear equations.\n",
      "\n",
      "Note: all homogeneous systems are consistent since they all satisfy the trivial solution. Therefore, all homogeneous solutions either has a unique solution or infinitely many solutions.\n",
      "\n",
      "**Def: Trivial Solution**\n",
      "\n",
      "The trivial solution to an equation is:\n",
      "\n",
      "$$\n",
      "x_1 = 0, x_2 = 0, \\ldots, x_n = 0\n",
      "$$\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## 2.1: Vectors\n",
      "\n",
      "$$\n",
      "\\newcommand{\\aa}{\\mathbf{a}}\n",
      "\\newcommand{\\bb}{\\mathbf{b}}\n",
      "\\newcommand{\\uu}{\\mathbf{u}}\n",
      "\\newcommand{\\vv}{\\mathbf{v}}\n",
      "\\newcommand{\\ww}{\\mathbf{w}}\n",
      "$$\n",
      "\n",
      "**Theorem 2.2**\n",
      "\n",
      "Let $a$ and $b$ be scalars, and $\\uu, \\vv, \\ww$ be vectors in $\\R^n$. Then:\n",
      "\n",
      "1. $\\uu + \\vv = \\vv + \\uu$\n",
      "2. $a(\\uu + \\vv) = a\\uu + a\\vv$\n",
      "3. $(a + b)\\uu = a\\uu + b\\uu$\n",
      "4. $(\\uu + \\vv) + \\ww = \\uu + (\\vv + \\ww)$\n",
      "5. $a(b\\uu) = (ab)\\uu$\n",
      "6. $\\uu + (-\\uu) = 0$\n",
      "7. $\\uu + \\vec{0} = \\vec{0} + \\uu = \\uu$\n",
      "8. $1\\uu = \\uu$\n",
      "\n",
      "---\n",
      "\n",
      "**Def: Vector**\n",
      "\n",
      "A **vector** is an ordered list of real numbers $u_1, u_2, \\ldots, u_n$ expressed as:\n",
      "\n",
      "$$\n",
      "\\uu = \\bcolvec{u_1 \\\\ u_2 \\\\ \\vdots \\\\ u_n}\n",
      "$$\n",
      "\n",
      "The set of all vectors with $n$ entries is denoted by $\\R^n$.\n",
      "\n",
      "**Def: Linear Combination**\n",
      "\n",
      "If $\\uu_1, \\uu_2, \\ldots, \\uu_m$ are vectors and $c_1, c_2, \\ldots, c_m$ are scalars, then:\n",
      "\n",
      "$$\n",
      "c_1 \\uu_1 + c_2 \\uu_2 + \\cdots + c_m \\uu_m\n",
      "$$\n",
      "\n",
      "...is a **linear combination** of the vectors. Note that the scalars may be negative or zero."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Section 2.2: Span\n",
      "\n",
      "\n",
      "\n",
      "$$\n",
      "\\newcommand{set}[1]{\\left\\{#1\\right\\}}\n",
      "\\newcommand{defu}[1]{\\uu_1, \\uu_2, \\ldots, \\uu_{#1}}\n",
      "\\newcommand{defum}{\\defu{m}}\n",
      "\\newcommand{defun}{\\defu{n}}\n",
      "\\newcommand{defset}[1]{\\set{\\uu_1, \\uu_2, \\ldots, \\uu_{#1}}}\n",
      "\\newcommand{defsetm}{\\defset{m}}\n",
      "\\newcommand{defsetn}{\\defset{n}}\n",
      "\\newcommand{sdefset}[1]{\\mathrm{span}\\!\\defset{#1}}\n",
      "\\newcommand{sdefsetm}{\\sdefset{m}}\n",
      "\\newcommand{sdefsetn}{\\sdefset{n}}\n",
      "\\newcommand{chain}[2]{#1_1, #1_2, \\ldots #1_{#2}}\n",
      "\\newcommand{chaineq}[3]{#1_1 #2_1, #1_2 #2_2, \\ldots #1_{#3} #2_{#3}}\n",
      "$$\n",
      "\n",
      "**Theorem 2.6**\n",
      "\n",
      "Let $\\uu_1, \\uu_2, \\ldots, \\uu_m$ and $\\vv$ be vectors in $\\R^n$. Then, $\\vv$ is an element of $\\sdefsetm$ if and only if the linear system represented by the augmented matrix:\n",
      "\n",
      "$$\n",
      "    \\lbrack \\uu_1 \\quad \\uu_2 \\quad \\ldots \\quad \\uu_m \\quad \\vv \\rbrack\n",
      "$$\n",
      "\n",
      "...has a solution.\n",
      "\n",
      "Proof:\n",
      "\n",
      "The vector $\\vv$ is in $\\sdefsetm$ if and only if there exist scalars $x_1, x_2, \\ldots, x_m$ that satisfy:\n",
      "\n",
      "$$\n",
      "x_1 \\uu_1 + x_2 \\uu_2 + \\cdots + x_m \\uu_m = \\vv\n",
      "$$\n",
      "\n",
      "This is true if and only if the corresponding linear system has a solution. As noted above, this linear system is equal to the augmented matrix, so the proof is complete.\n",
      "\n",
      "---\n",
      "\n",
      "**Theorem 2.7**\n",
      "\n",
      "Let $\\defum$ and $\\uu$ be vectors in $\\R^n$. If $\\uu$ is in $\\sdefsetm$ then $\\mathrm{span}\\!\\{\\uu, \\chain{\\uu}{m}\\} = \\sdefsetm$.\n",
      "\n",
      "Proof:\n",
      "\n",
      "Let $S_0 = \\mathrm{span}\\!\\{\\uu, \\uu_1, \\uu_2, \\ldots, \\uu_m\\}$ and $S_1 = \\sdefsetm$. We need to show that $S_0 = S_1$, which we can do by showing each set is a subset of the other.\n",
      "\n",
      "1. First, suppose that a vector $\\vv$ is in $S_1$. Then, there exist scalars $\\chain{a}{m}$ such that:\n",
      "\n",
      "    $$\n",
      "    \\vv = \\chaineq{a}{\\uu}{m} = 0\\uu + \\chaineq{a}{\\uu}{m}\n",
      "    $$\n",
      "    \n",
      "    If we can add in $0\\uu$ (right-hand-side), then $\\vv$ must also be in $S_0$ so $S_1$ is a subset of $S_0$.\n",
      "    \n",
      "2. Second, suppose that a vector $\\vv$ is in S_0. Then there exist scalars $\\chain{b}{m}$ such that $\\vv = b_0 \\uu + \\chaineq{b}{\\uu}{m}$. Since $\\uu$ is in $S_1$, there also exist scalars $\\chain{c}{m}$ such that $\\uu = \\chaineq{c}{\\uu}{m}$. Then we substitute:\n",
      "\n",
      "    $$\n",
      "    \\vv = b_0(\\chaineq{c}{\\uu}{m}) + \\chaineq{b}{\\uu}{m} \\\\\n",
      "    \\vv = (b_0 c_1 + b_1)\\uu_1 + (b_0 c_2 + b_2) \\uu_2 + \\cdots + (b_0 c_m + b_m) \\uu_m\n",
      "    $$\n",
      "    \n",
      "    Hence $\\vv$ is in S_1, so $S_0$ is a subset of $S_1$. Since the two sets are subsets of each other, it follows that $S_0 = S_1$.\n",
      "\n",
      "---\n",
      "\n",
      "**Theorem 2.8**\n",
      "\n",
      "Let $\\defsetm$ be a set of vectors in $\\R^n$. If $m < n$, then this set does not span $\\R^n$. If $m \\geq n$, then the set may or may not span $\\R^n$ -- we cannot tell without additional information.\n",
      "\n",
      "---\n",
      "\n",
      "**Theorem 2.10**\n",
      "\n",
      "Let $\\chain{\\aa}{m}$ and $\\bb$ be vectors in $\\R^n$. Then, the following statements are equivalent. That is, if one is true, then so are the others, and if one is false, then so are the others.\n",
      "\n",
      "1. $\\bb$ is in $\\span{\\chain{\\aa}{m}}$\n",
      "2. The vector equation $\\chaineq{x}{\\aa}{m} = \\bb$ has at least one solution.\n",
      "3. The linear system corresponding to $[\\aa_1 \\quad \\aa_2 \\quad \\cdots \\quad \\aa_m \\quad b]$ has at least one solution.\n",
      "4. The equation $A\\mathbf{x} = \\bb$ has at least one solution.\n",
      "\n",
      "\n",
      "**Def: Span**\n",
      "\n",
      "Let $\\defsetm$ be a set of vectors in $\\R^n$. The **span** of this set is denoted $\\sdefsetm$ and is defined to be the set of all linear combinations:\n",
      "\n",
      "$$\n",
      "x_1 \\uu_1 + x_2 \\uu_2 + \\cdots + x_m \\uu_m\n",
      "$$\n",
      "\n",
      "If $\\sdefsetm = \\R^n$, then we say that the set $\\defsetm$ spans $\\R^n$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Section 2.3: Linear Independence\n",
      "\n",
      "$$\n",
      "\\newcommand{\\zv}{\\mathbf{0}}\n",
      "$$\n",
      "\n",
      "**Theorem 2.12**\n",
      "\n",
      "Suppose that $\\{\\zv, \\defum\\}$ is a set of vectors in $\\R^n$. Then, the set is linearly dependent.\n",
      "\n",
      "Proof:\n",
      "\n",
      "Given the following expression:\n",
      "\n",
      "$$\n",
      "x_0 \\zv + \\chaineq{x}{\\uu}{m} = \\zv\n",
      "$$\n",
      "\n",
      "...$x_0$ may be any value even if the values of $x_1 = x_2 = \\cdots = x_m = 0$. Hence, we have found a nontrivial solution and so the set is linearly dependent.\n",
      "\n",
      "---\n",
      "\n",
      "**Theorem 2.13**\n",
      "\n",
      "Suppose that $\\defsetm$ is a set of vectors in $\\R^n$. if $n < m$, then the set is linearly dependent. In other words, if the number of vectors $m$ exceeds the number of components $n$, then the set is linearly dependent.\n",
      "\n",
      "Proof:\n",
      "\n",
      "If we solve, we'll end up with unknowns, and so we have more than one solution.\n",
      "\n",
      "---\n",
      "\n",
      "**Theorem 2.14**\n",
      "\n",
      "Let $\\defsetm$ be a set of vectors in $\\R^n$. Then, the set is linearly dependent if and only if one of the vectors in the set is in the span of the other vectors.\n",
      "\n",
      "\n",
      "---\n",
      "\n",
      "**Theorem 2.15**\n",
      "\n",
      "Let $A = [\\aa_1 \\quad \\aa_2 \\quad \\cdots \\quad \\aa_m]$ and $\\mathbf{x} = \\{\\chain{x}{m}\\}$. The set $A$ is linearly independent if and only if the homogeneous linear system:\n",
      "\n",
      "$$\n",
      "A\\mathbf{x} = \\zv\n",
      "$$\n",
      "\n",
      "...has only the trivial solution.\n",
      "\n",
      "---\n",
      "\n",
      "**Theorem 2.17**\n",
      "\n",
      "Let $\\mathbf{x}_p$ be a particular solution to $A\\mathbf{x} = \\bb$.\n",
      "\n",
      "Then, all solutions to this expression will have the form $\\mathbf{x}_g = \\mathbf{x}_p + \\mathbf{x}_h$ where $\\mathbf{x}_h$ is a solution to the associated homogeneous system $A\\mathbf{x} = \\zv$.\n",
      "\n",
      "---\n",
      "\n",
      "**Theorem 2.19: The Big Theorem (Version 1)**\n",
      "\n",
      "Let $\\mathcal{A} = \\defset{n}$ be a set of $n$ vectors in $\\R^n$, and let $A = [a_1 \\cdots a_n]$. Then the following are equivalent:\n",
      "\n",
      "1. $\\mathcal{A}$ spans $\\R^n$\n",
      "2. $\\mathcal{A}$ is linearly independent.\n",
      "3. $A\\mathbf{x} = \\bb$ has a unique solution for all $b$ in $\\R^n$.\n",
      "\n",
      "**Def: Linear Independence**\n",
      "\n",
      "Let $\\defsetm$ be a set of vectors in $\\R^n$. If the only solution to the vector equation is the trivial solution, then the set is **linearly independent**. If there are nontrivial solutions, then the set is **linearly dependent**."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Useful tidbits\n",
      "\n",
      "Is it true that a span of $n$ vectors will alwasy span $\\R^n$?\n",
      "\n",
      "FALSE, counterexample:\n",
      "\n",
      "$$\n",
      "\\vv_1 = \\bcolvec{1\\\\1}, \\quad \\bcolvec{2\\\\2}\n",
      "$$\n",
      "\n",
      "---\n",
      "\n",
      "Suppose that $\\chain{\\uu}{m}$ are vectors in $\\R^n$. If $m < n$, can $\\defsetm$ span $\\R^n$??\n",
      "\n",
      "FALSE\n",
      "\n",
      "---\n",
      "\n",
      "Every set of $n$ distinct non-zero vectors in $\\R^n$ spans $\\R^n$.\n",
      "\n",
      "FALSE\n",
      "\n",
      "---\n",
      "\n",
      "Let $\\defsetn \\subseteq \\R^n$, and let $A$ be the matrix whose $j$th column os $\\uu_j$. If $\\text{ref}(A)$ has $n$ leading 1's, then $\\sdefset{n} = \\R^n$.\n",
      "\n",
      "TRUE\n",
      "\n",
      "---\n",
      "\n",
      "If $S \\subseteq T$ and $S$ is linearly dependent, then $T$ is linearly dependent.\n",
      "\n",
      "TRUE\n",
      "\n",
      "---\n",
      "\n",
      "If $S \\subseteq T$, and $T$ is linearly indepenent, then $S$ is linearly independent.\n",
      "\n",
      "TRUE\n",
      "\n",
      "---\n",
      "\n",
      "If $\\uu_4$ is a linearly combination of $\\{\\uu_1, \\uu_2, \\uu_3\\}$, then $\\span{\\uu_1, \\uu_2\\ \\uu_4} = \\span{\\uu_1, \\uu_2, \\uu_3}$.\n",
      "\n",
      "FALSE"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
 }
	{
	"metadata": {
	"name": "",
	"signature": "sha256:335d80491f3857e5a5b5e9056726be1c349edffb5b5b8b9b2ab2e7034e78d13a"
	},
	"nbformat": 3,
	"nbformat_minor": 0,
	"worksheets": [
	{
	"cells": [
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"# Midterm 1 Review\n",
	"\n",
	"$$\n",
	"% units\n",
	"\\newcommand{u}[1]{\\;\\mathrm{#1}}\n",
	"%\n",
	"% derivative\n",
	"\\newcommand{dd}[2]{\\frac{\\mathrm{d}#1}{\\mathrm{d}#2}}\n",
	"%\n",
	"% double-derivative\n",
	"\\newcommand{ddd}[2]{\\frac{\\mathrm{d^2}#1}{\\mathrm{d}#2^2}}\n",
	"%\n",
	"% single-derivative\n",
	"\\newcommand{df}[1]{\\frac{\\mathrm{d}}{\\mathrm{d}#1}}\n",
	"%\n",
	"% derivative component\n",
	"\\newcommand{d}[1]{\\mathrm{d}#1}\n",
	"%\n",
	"% real numbers\n",
	"\\newcommand{R}{\\mathbb{R}}\n",
	"%\n",
	"% annotate\n",
	"\\newenvironment{annotate}[0]{\\begin{array}{cr}}{\\end{array}}\n",
	"%\n",
	"% rowop\n",
	"\\newcommand{rowop}[1]{\\quad\\overset{\\sim}{\\scriptstyle{#1}}\\quad}\n",
	"%\n",
	"% sin/cos/tan/ln with parens\n",
	"\\newcommand{cosp}[1]{\\cos{\\left(#1\\right)}}\n",
	"\\newcommand{sinp}[1]{\\sin{\\left(#1\\right)}}\n",
	"\\newcommand{tanp}[1]{\\tan{\\left(#1\\right)}}\n",
	"\\newcommand{ln}[1]{\\lnp{\\left(#1\\right)}}\n",
	"%\n",
	"% shorter frac\n",
	"\\newcommand{f}[2]{\\frac{#1}{#2}}\n",
	"%\n",
	"% rref\n",
	"\\newcommand{rref}[1]{\\mathrm{rref}\\!\\left(#1\\right)}\n",
	"%\n",
	"% parens\n",
	"\\newcommand{p}[1]{\\left(#1\\right)}\n",
	"\\newcommand{b}[1]{\\left\\{#1\\right\\}}\n",
	"%\n",
	"% column vectors\n",
	"\\newcommand{colvec}[1]{\\begin{pmatrix}#1\\end{pmatrix}}\n",
	"\\newcommand{bcolvec}[1]{\\begin{bmatrix}#1\\end{bmatrix}}\n",
	"%\n",
	"% vector symbols\n",
	"\\newcommand{v}[1]{\\vec{#1}}\n",
	"\\newcommand{vn}[2]{\\vec{#1}_{#2}}\n",
	"%\n",
	"% spans\n",
	"\\newcommand{span}[1]{\\mathrm{span}\\left\\{#1\\right\\}}\n",
	"$$\n"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"## Section 1.1: Systems of Linear Equations\n",
	"\n",
	"Theorem 1.2\n",
	"\n",
	"A system of linear equations has no solutions, exactly one solution, or infinitely many solutions.\n",
	"\n",
	"Proof:\n",
	"\n",
	"Take any linear system, form the augmented matrix, use Gaussian elimination to reduce to echelon form, and extra the transformed system. There are 3 possible outcomes:\n",
	"\n",
	"1. The system has an equation of the form $0 = c$ for $c \\neq 0$. In this case, the system has no solutions.\n",
	"2. The transformed system has no free variables and hence exactly one solution.\n",
	"4. The transformed system has one or more free variables and hence infinitely many solutions.\n",
	"\n",
	"---\n",
	"\n",
	"Def: Linear Equation\n",
	"\n",
	"A linear equation is an equation with the form:\n",
	"\n",
	"$$\n",
	"a_1 x_1 + a_2 x_2 + a_3 x_3 + \\cdots + a_n x_n = b\n",
	"$$\n",
	"\n",
	"...where $a_1, a_2, \\cdots, a_n$ and $b$ are constants and $x_1, x_2, \\cdots x_n$ are variables or unknowns.\n",
	"\n",
	"Def: Solution of Linear Equation\n",
	"\n",
	"A solution $(s_1, s_2, \\cdots s_n)$ is an ordered set of $n$ numbers such that we we set $x_1 = s_1, x_2 = s_2, \\cdots x_n = s_n$ then the given linear equation is satisfied.\n",
	"\n",
	"Def: Solution set\n",
	"\n",
	"The solution set for a linear equation consists of the set of all solutions to the given equation.\n",
	"\n",
	"Def: Consistent Linear System, Inconsistent Linear System\n",
	"\n",
	"If a linear system has at least one solution, then we say it is consistent. If not, then it is inconsistent.\n",
	"\n",
	"Def: Leading Variable\n",
	"\n",
	"A variable that appears as the first term in at least one equation is called a leading variable.\n",
	"\n",
	"Def: Free Variable\n",
	"\n",
	"Every variable of an echelon system which is not a leading variable.\n",
	"\n",
	"Def: Echelon System\n",
	"\n",
	"An Echelon System is a system of equations in the following format:\n",
	"\n",
	"$$\n",
	"\\left\\{\n",
	"\\begin{array}{rl}\n",
	" a_{11} x_1 + a_{12} x_2 + \\cdots + a_{1n}x_n &= b_1 \\\\\n",
	" a_{22} x_2 + \\cdots + a_{2n}x_n &= b_2 \\\\\n",
	" \\vdots & \\\\\n",
	" a_{nn}x_n &= b_n \n",
	"\\end{array}\\right.\n",
	"$$\n",
	"\n",
	"More specifically:\n",
	"\n",
	"1. Every variable is the leading variable of _at most one equation_\n",
	"2. The system is organized in a descending \"stair step\" pattern so that the index of the leading variables increases from top to bottom.\n",
	"3. Every equation has a leading variable."
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"## Section 1.2: Linear Systems and Matrices\n",
	"\n",
	"Theorem 1.6\n",
	"\n",
	"A given matrix is equivalent to a unique matrix that is in reduced echelon form.\n",
	"\n",
	"---\n",
	"\n",
	"\n",
	"Def: Equivalent linear systems\n",
	"\n",
	"Two linear systems are said to be equivalent if they have the same set of solutions.\n",
	"\n",
	"Def: Elementary Operations\n",
	"\n",
	"Any operation which can be applied to a linear system that produces a new system equivalent to the old one, such that the solution set is unchanged. There are only 3 valid row operations:\n",
	"\n",
	"1. Interchange the position of the two equations\n",
	"2. Multiply an equation by a nonzero constant\n",
	"3. Add a multiple of one equation to another.\n",
	"\n",
	"Def: Matrix\n",
	"\n",
	"A rectangular table of numbers.\n",
	"\n",
	"Def: Augmented Matrix\n",
	"\n",
	"A matrix containing all the coefficients of a linear equation, including the constant terms on the right side of the equation.\n",
	"\n",
	"Def: Equivalent Matrices\n",
	"\n",
	"Two matrices are equivalent if one can be obtained from the other through a sequence of elementary row operations.\n",
	"\n",
	"Def: Zero Row, Nonzero Row\n",
	"\n",
	"A zero row is a row consisting entirely of zeros. A nonzero row contains at least one nonzero entry.\n",
	"\n",
	"Def: Gaussian Elimination\n",
	"\n",
	"A procedure which transforms an augmented matrix into echelon form.\n",
	"\n",
	"Def: Echelon Form\n",
	"\n",
	"A matrix is in echelon form if:\n",
	"\n",
	"1. Every leading term is in a column to the left of the leading term of the row below it.\n",
	"2. Any zero rows are at the bottom of the matrix.\n",
	"\n",
	"Def: Gauss-Jordan Elimination\n",
	"\n",
	"An operation performed on a system in echelon form that:\n",
	"\n",
	"1. Multiplies each nonzero row by the reciprocol of the pivot so that we end up with a 1 as the leading term in each nonzero row.\n",
	"2. Use row operations to introduce zeros in the entries _above_ each pivot position.\n",
	"\n",
	"Def: Reduced Echelon Form\n",
	"\n",
	"A matrix is in reduced echelon form if:\n",
	"\n",
	"1. It is in echeclon form.\n",
	"2. All pivot positions contain a 1.\n",
	"3. The only nonzero term in a pivot column is the pivot position.\n",
	"\n",
	"Def: Homogeneous linear equation, Homogeneous linear system\n",
	"\n",
	"A linear equation is homogeneous if it has the form:\n",
	"\n",
	"$$\n",
	"a_1 x_1 + a_2 x_2 + \\cdots + a_n x_n = 0\n",
	"$$\n",
	"\n",
	"A homogeneous linear system is a system of equations made entirely up of homgeneous linear equations.\n",
	"\n",
	"Note: all homogeneous systems are consistent since they all satisfy the trivial solution. Therefore, all homogeneous solutions either has a unique solution or infinitely many solutions.\n",
	"\n",
	"Def: Trivial Solution\n",
	"\n",
	"The trivial solution to an equation is:\n",
	"\n",
	"$$\n",
	"x_1 = 0, x_2 = 0, \\ldots, x_n = 0\n",
	"$$\n",
	"\n"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"## 2.1: Vectors\n",
	"\n",
	"$$\n",
	"\\newcommand{\\aa}{\\mathbf{a}}\n",
	"\\newcommand{\\bb}{\\mathbf{b}}\n",
	"\\newcommand{\\uu}{\\mathbf{u}}\n",
	"\\newcommand{\\vv}{\\mathbf{v}}\n",
	"\\newcommand{\\ww}{\\mathbf{w}}\n",
	"$$\n",
	"\n",
	"Theorem 2.2\n",
	"\n",
	"Let $a$ and $b$ be scalars, and $\\uu, \\vv, \\ww$ be vectors in $\\R^n$. Then:\n",
	"\n",
	"1. $\\uu + \\vv = \\vv + \\uu$\n",
	"2. $a(\\uu + \\vv) = a\\uu + a\\vv$\n",
	"3. $(a + b)\\uu = a\\uu + b\\uu$\n",
	"4. $(\\uu + \\vv) + \\ww = \\uu + (\\vv + \\ww)$\n",
	"5. $a(b\\uu) = (ab)\\uu$\n",
	"6. $\\uu + (-\\uu) = 0$\n",
	"7. $\\uu + \\vec{0} = \\vec{0} + \\uu = \\uu$\n",
	"8. $1\\uu = \\uu$\n",
	"\n",
	"---\n",
	"\n",
	"Def: Vector\n",
	"\n",
	"A vector is an ordered list of real numbers $u_1, u_2, \\ldots, u_n$ expressed as:\n",
	"\n",
	"$$\n",
	"\\uu = \\bcolvec{u_1 \\\\ u_2 \\\\ \\vdots \\\\ u_n}\n",
	"$$\n",
	"\n",
	"The set of all vectors with $n$ entries is denoted by $\\R^n$.\n",
	"\n",
	"Def: Linear Combination\n",
	"\n",
	"If $\\uu_1, \\uu_2, \\ldots, \\uu_m$ are vectors and $c_1, c_2, \\ldots, c_m$ are scalars, then:\n",
	"\n",
	"$$\n",
	"c_1 \\uu_1 + c_2 \\uu_2 + \\cdots + c_m \\uu_m\n",
	"$$\n",
	"\n",
	"...is a linear combination of the vectors. Note that the scalars may be negative or zero."
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"## Section 2.2: Span\n",
	"\n",
	"\n",
	"\n",
	"$$\n",
	"\\newcommand{set}[1]{\\left\\{#1\\right\\}}\n",
	"\\newcommand{defu}[1]{\\uu_1, \\uu_2, \\ldots, \\uu_{#1}}\n",
	"\\newcommand{defum}{\\defu{m}}\n",
	"\\newcommand{defun}{\\defu{n}}\n",
	"\\newcommand{defset}[1]{\\set{\\uu_1, \\uu_2, \\ldots, \\uu_{#1}}}\n",
	"\\newcommand{defsetm}{\\defset{m}}\n",
	"\\newcommand{defsetn}{\\defset{n}}\n",
	"\\newcommand{sdefset}[1]{\\mathrm{span}\\!\\defset{#1}}\n",
	"\\newcommand{sdefsetm}{\\sdefset{m}}\n",
	"\\newcommand{sdefsetn}{\\sdefset{n}}\n",
	"\\newcommand{chain}[2]{#1_1, #1_2, \\ldots #1_{#2}}\n",
	"\\newcommand{chaineq}[3]{#1_1 #2_1, #1_2 #2_2, \\ldots #1_{#3} #2_{#3}}\n",
	"$$\n",
	"\n",
	"Theorem 2.6\n",
	"\n",
	"Let $\\uu_1, \\uu_2, \\ldots, \\uu_m$ and $\\vv$ be vectors in $\\R^n$. Then, $\\vv$ is an element of $\\sdefsetm$ if and only if the linear system represented by the augmented matrix:\n",
	"\n",
	"$$\n",
	" \\lbrack \\uu_1 \\quad \\uu_2 \\quad \\ldots \\quad \\uu_m \\quad \\vv \\rbrack\n",
	"$$\n",
	"\n",
	"...has a solution.\n",
	"\n",
	"Proof:\n",
	"\n",
	"The vector $\\vv$ is in $\\sdefsetm$ if and only if there exist scalars $x_1, x_2, \\ldots, x_m$ that satisfy:\n",
	"\n",
	"$$\n",
	"x_1 \\uu_1 + x_2 \\uu_2 + \\cdots + x_m \\uu_m = \\vv\n",
	"$$\n",
	"\n",
	"This is true if and only if the corresponding linear system has a solution. As noted above, this linear system is equal to the augmented matrix, so the proof is complete.\n",
	"\n",
	"---\n",
	"\n",
	"Theorem 2.7\n",
	"\n",
	"Let $\\defum$ and $\\uu$ be vectors in $\\R^n$. If $\\uu$ is in $\\sdefsetm$ then $\\mathrm{span}\\!\\{\\uu, \\chain{\\uu}{m}\\} = \\sdefsetm$.\n",
	"\n",
	"Proof:\n",
	"\n",
	"Let $S_0 = \\mathrm{span}\\!\\{\\uu, \\uu_1, \\uu_2, \\ldots, \\uu_m\\}$ and $S_1 = \\sdefsetm$. We need to show that $S_0 = S_1$, which we can do by showing each set is a subset of the other.\n",
	"\n",
	"1. First, suppose that a vector $\\vv$ is in $S_1$. Then, there exist scalars $\\chain{a}{m}$ such that:\n",
	"\n",
	" $$\n",
	" \\vv = \\chaineq{a}{\\uu}{m} = 0\\uu + \\chaineq{a}{\\uu}{m}\n",
	" $$\n",
	" \n",
	" If we can add in $0\\uu$ (right-hand-side), then $\\vv$ must also be in $S_0$ so $S_1$ is a subset of $S_0$.\n",
	" \n",
	"2. Second, suppose that a vector $\\vv$ is in S_0. Then there exist scalars $\\chain{b}{m}$ such that $\\vv = b_0 \\uu + \\chaineq{b}{\\uu}{m}$. Since $\\uu$ is in $S_1$, there also exist scalars $\\chain{c}{m}$ such that $\\uu = \\chaineq{c}{\\uu}{m}$. Then we substitute:\n",
	"\n",
	" $$\n",
	" \\vv = b_0(\\chaineq{c}{\\uu}{m}) + \\chaineq{b}{\\uu}{m} \\\\\n",
	" \\vv = (b_0 c_1 + b_1)\\uu_1 + (b_0 c_2 + b_2) \\uu_2 + \\cdots + (b_0 c_m + b_m) \\uu_m\n",
	" $$\n",
	" \n",
	" Hence $\\vv$ is in S_1, so $S_0$ is a subset of $S_1$. Since the two sets are subsets of each other, it follows that $S_0 = S_1$.\n",
	"\n",
	"---\n",
	"\n",
	"Theorem 2.8\n",
	"\n",
	"Let $\\defsetm$ be a set of vectors in $\\R^n$. If $m < n$, then this set does not span $\\R^n$. If $m \\geq n$, then the set may or may not span $\\R^n$ -- we cannot tell without additional information.\n",
	"\n",
	"---\n",
	"\n",
	"Theorem 2.10\n",
	"\n",
	"Let $\\chain{\\aa}{m}$ and $\\bb$ be vectors in $\\R^n$. Then, the following statements are equivalent. That is, if one is true, then so are the others, and if one is false, then so are the others.\n",
	"\n",
	"1. $\\bb$ is in $\\span{\\chain{\\aa}{m}}$\n",
	"2. The vector equation $\\chaineq{x}{\\aa}{m} = \\bb$ has at least one solution.\n",
	"3. The linear system corresponding to $[\\aa_1 \\quad \\aa_2 \\quad \\cdots \\quad \\aa_m \\quad b]$ has at least one solution.\n",
	"4. The equation $A\\mathbf{x} = \\bb$ has at least one solution.\n",
	"\n",
	"\n",
	"Def: Span\n",
	"\n",
	"Let $\\defsetm$ be a set of vectors in $\\R^n$. The span of this set is denoted $\\sdefsetm$ and is defined to be the set of all linear combinations:\n",
	"\n",
	"$$\n",
	"x_1 \\uu_1 + x_2 \\uu_2 + \\cdots + x_m \\uu_m\n",
	"$$\n",
	"\n",
	"If $\\sdefsetm = \\R^n$, then we say that the set $\\defsetm$ spans $\\R^n$."
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"## Section 2.3: Linear Independence\n",
	"\n",
	"$$\n",
	"\\newcommand{\\zv}{\\mathbf{0}}\n",
	"$$\n",
	"\n",
	"Theorem 2.12\n",
	"\n",
	"Suppose that $\\{\\zv, \\defum\\}$ is a set of vectors in $\\R^n$. Then, the set is linearly dependent.\n",
	"\n",
	"Proof:\n",
	"\n",
	"Given the following expression:\n",
	"\n",
	"$$\n",
	"x_0 \\zv + \\chaineq{x}{\\uu}{m} = \\zv\n",
	"$$\n",
	"\n",
	"...$x_0$ may be any value even if the values of $x_1 = x_2 = \\cdots = x_m = 0$. Hence, we have found a nontrivial solution and so the set is linearly dependent.\n",
	"\n",
	"---\n",
	"\n",
	"Theorem 2.13\n",
	"\n",
	"Suppose that $\\defsetm$ is a set of vectors in $\\R^n$. if $n < m$, then the set is linearly dependent. In other words, if the number of vectors $m$ exceeds the number of components $n$, then the set is linearly dependent.\n",
	"\n",
	"Proof:\n",
	"\n",
	"If we solve, we'll end up with unknowns, and so we have more than one solution.\n",
	"\n",
	"---\n",
	"\n",
	"Theorem 2.14\n",
	"\n",
	"Let $\\defsetm$ be a set of vectors in $\\R^n$. Then, the set is linearly dependent if and only if one of the vectors in the set is in the span of the other vectors.\n",
	"\n",
	"\n",
	"---\n",
	"\n",
	"Theorem 2.15\n",
	"\n",
	"Let $A = [\\aa_1 \\quad \\aa_2 \\quad \\cdots \\quad \\aa_m]$ and $\\mathbf{x} = \\{\\chain{x}{m}\\}$. The set $A$ is linearly independent if and only if the homogeneous linear system:\n",
	"\n",
	"$$\n",
	"A\\mathbf{x} = \\zv\n",
	"$$\n",
	"\n",
	"...has only the trivial solution.\n",
	"\n",
	"---\n",
	"\n",
	"Theorem 2.17\n",
	"\n",
	"Let $\\mathbf{x}_p$ be a particular solution to $A\\mathbf{x} = \\bb$.\n",
	"\n",
	"Then, all solutions to this expression will have the form $\\mathbf{x}_g = \\mathbf{x}_p + \\mathbf{x}_h$ where $\\mathbf{x}_h$ is a solution to the associated homogeneous system $A\\mathbf{x} = \\zv$.\n",
	"\n",
	"---\n",
	"\n",
	"Theorem 2.19: The Big Theorem (Version 1)\n",
	"\n",
	"Let $\\mathcal{A} = \\defset{n}$ be a set of $n$ vectors in $\\R^n$, and let $A = [a_1 \\cdots a_n]$. Then the following are equivalent:\n",
	"\n",
	"1. $\\mathcal{A}$ spans $\\R^n$\n",
	"2. $\\mathcal{A}$ is linearly independent.\n",
	"3. $A\\mathbf{x} = \\bb$ has a unique solution for all $b$ in $\\R^n$.\n",
	"\n",
	"Def: Linear Independence\n",
	"\n",
	"Let $\\defsetm$ be a set of vectors in $\\R^n$. If the only solution to the vector equation is the trivial solution, then the set is linearly independent. If there are nontrivial solutions, then the set is linearly dependent."
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"## Useful tidbits\n",
	"\n",
	"Is it true that a span of $n$ vectors will alwasy span $\\R^n$?\n",
	"\n",
	"FALSE, counterexample:\n",
	"\n",
	"$$\n",
	"\\vv_1 = \\bcolvec{1\\\\1}, \\quad \\bcolvec{2\\\\2}\n",
	"$$\n",
	"\n",
	"---\n",
	"\n",
	"Suppose that $\\chain{\\uu}{m}$ are vectors in $\\R^n$. If $m < n$, can $\\defsetm$ span $\\R^n$??\n",
	"\n",
	"FALSE\n",
	"\n",
	"---\n",
	"\n",
	"Every set of $n$ distinct non-zero vectors in $\\R^n$ spans $\\R^n$.\n",
	"\n",
	"FALSE\n",
	"\n",
	"---\n",
	"\n",
	"Let $\\defsetn \\subseteq \\R^n$, and let $A$ be the matrix whose $j$th column os $\\uu_j$. If $\\text{ref}(A)$ has $n$ leading 1's, then $\\sdefset{n} = \\R^n$.\n",
	"\n",
	"TRUE\n",
	"\n",
	"---\n",
	"\n",
	"If $S \\subseteq T$ and $S$ is linearly dependent, then $T$ is linearly dependent.\n",
	"\n",
	"TRUE\n",
	"\n",
	"---\n",
	"\n",
	"If $S \\subseteq T$, and $T$ is linearly indepenent, then $S$ is linearly independent.\n",
	"\n",
	"TRUE\n",
	"\n",
	"---\n",
	"\n",
	"If $\\uu_4$ is a linearly combination of $\\{\\uu_1, \\uu_2, \\uu_3\\}$, then $\\span{\\uu_1, \\uu_2\\ \\uu_4} = \\span{\\uu_1, \\uu_2, \\uu_3}$.\n",
	"\n",
	"FALSE"
	]
	},
	{
	"cell_type": "code",
	"collapsed": false,
	"input": [],
	"language": "python",
	"metadata": {},
	"outputs": []
	}
	],
	"metadata": {}
	}
	]
	}