Combinatorial explosion and testing

gistfile1.txt

I found http://groups.google.com/group/growing-object-oriented-software/browse_thread/thread/47695af2c6b5adda fascinating, so I decided to make it a little more concrete. It's also in ruby using rspec, sorry about that. Here's the code under test:

class EligibleForDiscountPolicy
  def initialize transaction, user
    @transaction = transaction
    @user = user
  end

  def decide
    return true if @user.is_gold_member?
    return false if @transaction.coupon_was_used?
    return false if @transaction.contains_new_release?
    return true
  end
end

The idea here is that you're writing software for a video rental store. This particular class is responsible for determining whether a particular user is eligible for a discount on a particular transaction. If they have a gold membership, they always are. If not, two other conditions apply. Here are two approaches for testing this:

describe EligibleForDiscountPolicy do
  [
   [true,true,true,     true],
   [true,true,false,    true],
   [true,false,true,    true],
   [true,false,false,   true],
   [false,true,true,    false],
   [false,true,false,   false],
   [false,false,true,   false],
   [false,false,false,  true]
  ].each do |gold, coupon, new_release, result|
    scenario =  "When the user #{gold ? 'IS' : 'IS NOT'} a gold member"
    scenario += ", a coupon #{coupon ? 'IS' : 'IS NOT'} used"
    scenario += ", and the transaction #{new_release ? 'DOES' : 'DOES NOT'} contain a new release"
    scenario += ", the user #{result ? 'IS' : 'IS NOT'} eligible for a discount"
    
    specify scenario do
      user = stub :is_gold_member? => gold
      transaction = stub :coupon_was_used? => coupon, :contains_new_release? => new_release
      EligibleForDiscountPolicy.new(transaction,user).decide.should == result
    end
  end
end

This generates a test (spec) for each of the 8 scenarios in the truth table you could build from the above code. The output looks something like this:

/Users/andrewwagner/ruby> rspec -f nested combinatorial2_spec.rb

EligibleForDiscountPolicy
  When the user IS a gold member, a coupon IS used, and the transaction DOES contain a new release, the user IS eligible for a discount
  When the user IS a gold member, a coupon IS used, and the transaction DOES NOT contain a new release, the user IS eligible for a discount
  When the user IS a gold member, a coupon IS NOT used, and the transaction DOES contain a new release, the user IS eligible for a discount
  When the user IS a gold member, a coupon IS NOT used, and the transaction DOES NOT contain a new release, the user IS eligible for a discount
  When the user IS NOT a gold member, a coupon IS used, and the transaction DOES contain a new release, the user IS NOT eligible for a discount
  When the user IS NOT a gold member, a coupon IS used, and the transaction DOES NOT contain a new release, the user IS NOT eligible for a discount
  When the user IS NOT a gold member, a coupon IS NOT used, and the transaction DOES contain a new release, the user IS NOT eligible for a discount
  When the user IS NOT a gold member, a coupon IS NOT used, and the transaction DOES NOT contain a new release, the user IS eligible for a discount

Finished in 0.20627 seconds
8 examples, 0 failures

This is a very data-driven approach, which has its advantages and disadvantages. Here is another approach:

require './combinatorial'

describe EligibleForDiscountPolicy do
  let (:user) { Object.new }
  let (:transaction) { Object.new }

  context "when the user is a gold member" do
    before { user.stub :is_gold_member? => true }

    it "doesn't matter whether a coupon was used" do
      transaction.should_not_receive :coupon_was_used?
      EligibleForDiscountPolicy.new(transaction,user).decide()
    end

    it "doesn't matter whether there was a new release" do
      transaction.should_not_receive :contains_new_release?
      EligibleForDiscountPolicy.new(transaction,user).decide()
    end

    it "the user is always eligible for a discount" do
      EligibleForDiscountPolicy.new(transaction,user).decide().should == true
    end
  end

  context "when the user is not a gold member" do
    before { user.stub :is_gold_member? => false }
    context "when a coupon was used" do
      before { transaction.stub :coupon_was_used? => true }

      it "doesn't matter whether there was a new release" do
        transaction.should_not_receive :contains_new_release? 
        EligibleForDiscountPolicy.new(transaction,user).decide()
      end

      it "the user is not eligible for a discount" do
        EligibleForDiscountPolicy.new(transaction,user).decide().should == false
      end
    end

    context "when a coupon was not used" do
      before { transaction.stub :coupon_was_used? => false }
      
      context "when renting a new release" do
        before { transaction.stub :contains_new_release? => true }

        it "the user is not eligible for a discount" do
          EligibleForDiscountPolicy.new(transaction,user).decide().should == false
        end
      end

      context "when not renting a new release" do
        before { transaction.stub :contains_new_release? => true }
        
        it "the user is eligible for a discount" do
          EligibleForDiscountPolicy.new(transaction,user).decide().should == false
        end
      end
    end
  end
end

And the resulting output:
/Users/andrewwagner/ruby> rspec -f nested combinatorial_spec.rb 

EligibleForDiscountPolicy
  when the user is a gold member
    doesn't matter whether a coupon was used
    doesn't matter whether there was a new release
    the user is always eligible for a discount
  when the user is not a gold member
    when a coupon was used
      doesn't matter whether there was a new release
      the user is not eligible for a discount
    when a coupon was not used
      when renting a new release
        the user is not eligible for a discount
      when not renting a new release
        the user is eligible for a discount

Finished in 0.20714 seconds
7 examples, 0 failures

I do feel like the second approach is more expressive, somehow, both in the actual spec code and the output. In both cases, though, I feel like if there were more conditions to check, the number of tests would increase exponentially.

Thoughts?

I like the code in the second one, but I still want to see a summary of the behavior as a table.

I prefer second approach. Also we can collect the stubs, contexts and tests as we run the spec and create a summary table, outputting something like this:

| is_gold_member? | coupon_was_used? | contains_new_release? | decide |
| true | nil | nil | true |
| false | true | nil | false |
| false | false | true | false |
| false | false | false | true |

Also I think the last test is wrong in the second approach, the stub should be false and the should should be true :)