gistfile1.htm

The Limits of TDD
#date 2009-11-09 21:47
#tags python,tdd

<p>
    My <a href="/2009/11/how_i_started_tdd.html">last post</a> about TDD
    generated some great responses, some of which were skeptical. A few common
    complaints about TDD were brought up, and posed with civility, so I'd like
    to address them.
</p>

<h4>Complaint: You weren't stupid enough</h4>

<p>
    When TDDing Fibonacci, we could get to a point where we have this
    function (and I did write exactly this code in my last post):
</p>

<pre><code>def fib(n):
        if n == 0:
            return 0
        else:
            return 1</code></pre>

<p>
    But why should we write that? Why not this instead?
</p>

<pre><code>def fib(n):
    return [0, 1, 1][n]</code></pre>

<p>
    This comes down to how we define "simple". In TDD, we make tests pass by
    making the simplest possible change.  So, which of the above two is
    simpler?
</p>
<p>
    Defining that word is our job; TDD as a process says nothing about it.
    The definition is a <em>huge</em> variable and, in my experience, it's the
    primary axis along which our skill as TDDers grows once we reach minimal
    competence.  Note that we still have to define "simple" even if we're not
    doing TDD, but we won't have the test-driving pressure forcing the
    definition to be refined.
</p>

<p>
    Regardless of how "simple" is defined, we must eventually accept that an
    arbitrarily long list is not the simplest thing. At that point, we
    refactor.  Depending on the definition of simple, it may take seven tests
    to get to the final refactor instead of five. So what? Two TDDers need not
    generate the same tests, and this isn't a problem at all.
</p>

<h4>Complaint: TDDed tests are prescriptive</h4>

<p>
    This is a complaint that TDDed code does exactly what the tests say it
    should do, so there might be bugs. If I write the wrong test, the
    reasoning goes, then it will drive me to write the wrong code.
</p>

<p>
    When would we write the wrong test? Only when we misunderstand the
    problem. If we misunderstand the problem, and we go straight to the code,
    then we're be encoding our incorrect understanding <em>directly in the
        code</em>. That's bad. By writing the tests first, we have some extra
    protection against misunderstanding: every assumption about what the
    system <em>should</em> do is encoded as a test, and each test has a good
    name.
</p>
<p>
    Often, this will point out our confusion during the TDD process &ndash;
    we'll find that we want to write a test whose name contradicts another
    test's name. Even if we translate our misunderstanding into a bug,
    however, good test names make it easy to revisit our assumptions later. A
    subtle, five-character change to the code may have been driven by a
    sixty-character test name, which will be easier to understand.
</p>

<h4>Complaint: Choosing tests is hard</h4>

<p>
    When TDDing Fibonacci, I tested fib(0) first. Why did I test fib(1) next
    instead of fib(37) or fib(51)?
</p>

<p>
    Because it was obvious! The problem domain of a unit test is necessarily
    small, so it's usually clear what the next step is.  If the next step
    isn't clear, it probably means that the unit under test is too large
    (making it hard to think about extending it for another case), or that we
    don't understand the problem well enough (making it hard to think about
    what the code should do at all). In either case, TDD has just helped us:
    it's either pointed out a bad design, which we should fix, or it's pointed
    a gap in our knowledge about the problem, in which case we should put the
    keyboard away and fill that gap.
</p>

<h4>Complaint: The code you TDDed was bad</h4>

<p>
    The particular code I came up with in my last blog post was a slow,
    recursive Fibonacci solution. Two people mentioned this in the comments.
</p>

<p>
    TDD doesn't solve problems like "my run time is superlinear" or "my
    database loads aren't eager enough."  It's not <em>supposed</em> to solve
    those problems!  TDD frees us to solve those hard problems well by (1)
    pushing us toward a good, decoupled design and (2) providing us with
    large, fast test suites.
</p>

<h4>Complaint: TDD requires too much typing</h4>

<p>
    This one has the easiest answer of all: <a
        href="http://anarchycreek.com/2009/05/26/how-tdd-and-pairing-increase-production/">typing
        is not the bottleneck</a>.  Just think about it for a minute. Go back
    and look at how many lines of code you actually generated yesterday. How
    long would it take you to type it all in one long burst? A few minutes?
    Seriously, typing is not the bottleneck.
</p>

<h4>TDD is not magic</h4>

<p>
Let's recap:
</p>

<ul>
    <li>Complaint: You weren't stupid enough</li>
    <li>Response: There's more than one legitimate definition of "stupid".</li>
</ul>
<ul>
    <li>Complaint: TDDed tests are prescriptive</li>
    <li>Response: This is a feature. Stating our assumptions up front exposes
    misunderstandings.</li>
</ul>
<ul>
    <li>Complaint: Choosing tests is hard</li>
    <li>Response: This is also a feature. It tells us that our design is bad
    or that we don't understand the problem.</li>
</ul>
<ul>
    <li>Complaint: The code you TDDed was bad!</li>
    <li>Response: TDD does not free us from thinking. TDD is not magic.</li>
</ul>
<ul>
    <li>Complaint: It's too much typing.</li>
    <li>Response: Typing is not the bottleneck.</li>
</ul>

<p>
    Many complaints about TDD are complaints that it <em>doesn't</em> solve
    some problem.  These are not problems with TDD &ndash; it's not supposed
    to solve every problem!
</p>
<p>
    Dynamic languages don't make coffee, continuous integration doesn't shine
    shoes, and TDD doesn't make code scale. It's simply the basis of a solid,
    disciplined process for building software &ndash; a beginning, not an end.
</p>