Cabal file with test-suite block and explanation in a USAGE.md file.

README.md

Integrating Test Suites with Cabal Haskell Projects

Basics

Add a Test-Suite block to your project Cabal file like below:

Test-Suite projectname-testsuite
  Type:                     exitcode-stdio-1.0
  Main-is:                  TestSuite.hs
  Build-depends:            base, ....your test framework deps here...

OR:

-- This appears to be contested on ML and in comments below - I have NOT tested this method!
Test-Suite awesomesauce-testsuite
  Type:                 detailed-1.0
  Test-Module:          Awesomesauce.Tests
  Build-depends:        base, QuickCheck, HUnit 

Note: you might need to run: cabal configure --enable-tests and cabal build again.

Now you should be able to just run: cabal test and it should integrate.

Other options

There are two built in test suite Cabal interfaces:

• exitcode-stdio-1.0
• detailed-1.0

The first test suite interface built into Cabal, exitcode-stdio-1.0, depends on the block specifying a Main-is: attribute which is the path of a "main" Haskell file (i.e. a runnable file with a main defined).

The second test suite interface built into Cabal, detailed-1.0, depends on the test-module attribute inside the Test-Suite Cabal block being set to the name of the module to run. This module should export a function named tests with type signature: tests :: [Test]. The Test type can be found in module Distribution.TestSuite. It has two wrapper functions in this module: pure and impure.

Written an internal test library or framework?

Any makers of test frameworks will need to (at a minimum) define instances of the TestOptions and ImpureTestable classes (also exported from Distribution.TestSuite). If the test framework also has a pure runner, you could also define PureTestable class instances for your framework.

QuickCheck & HUnit Support

You will find cabal packages with name prefixes of cabal-test-XXX if your favorite open source Haskell test library or framework (e.g. QuickCheck, HUnit) is supported already.

Test & Have Fun!

awesomesauce.cabal

-- Say you have a Cabal-ized Haskell project called 'awesomesauce'
Name:                awesomesauce
Version:             0.1
Synopsis:            The awesomesauce
Description:         Awesomesauceness for Haskell (as if it needs more?)
License:             BSD3
License-file:        LICENSE
Author:              Susan Potter
Maintainer:          [email protected]
Copyright:           2011
Category:            Development
Build-type:          Simple
Extra-source-files:  README.md
Cabal-version:       >=1.2

Test-Suite awesomesauce-testsuite
  Type:                 exitcode-stdio-1.0
  Main-is:              ExitCodeTestsExample.hs
  Build-depends:        base, QuickCheck, HUnit

-- OR 
-- This appears to be contested - I have NOT tested this method!
Test-Suite awesomesauce-testsuite
  Type:                 detailed-1.0
  Test-Module:          QCTestModule
  Build-depends:        base, QuickCheck, HUnit, test-framework


-- Then your Executable or Library Cabal sections/blocks under here.

ExitCodeTestsExample.hs

module Main where

import Test.QuickCheck (quickCheck)
import Your.Module (encrypt, decrypt)

prop_reverseReverse :: [Char] -> Bool
prop_reverseReverse s = (reverse . reverse) s == s

prop_encryptDecrypt :: [Char] -> Bool
prop_encryptDecrypt s = (encrypt . decrypt) s == s

main = do
  quickCheck prop_encryptDecrypt
  quickCheck prop_reverseReverse

QCTestModule.hs

module QCTestModule where

import Test.Framework
import Test.Framework.Providers.HUnit
import Test.Framework.Providers.QuickCheck
import Your.Module (encrypt, decrypt)

prop_reverseReverse :: [Char] -> Bool
prop_reverseReverse s = (reverse . reverse) s == s

prop_encryptDecrypt :: [Char] -> Bool
prop_encryptDecrypt s = (encrypt . decrypt) s == s

tests =
  [ test "prop_reverseReverse" prop_reverseReverse
  , test "prop_encryptDecrypt" prop_encryptDecrypt
  , ...
  ]

-- or have a testsuite function that is exported
testSuite = testGroup "Group name"
  [ testProperty "desc1" prop_reverseReverse
  , testProperty "desc2" prop_encryptDecrypt
  ]

@jberryman Which version of QuickCheck are you using? With type: exitcode-stdio-1.0 our test suites run correctly (the exit code when a failure exists is 1 and when all properties pass the process exit code is 0). I admit I have never used type: detailed-1.0. It was added to the README.md since the Cabal docs had this as an option for the type in the test-suite block.

The versions (where the exitcode-stdio-1.0 method above works) are:

• GHC 7.0.4
• Cabal library 1.10.2.0
• cabal-install 0.10.2
• QuickCheck 2.4.1.1

The only difference I have is I am using cabal v1.10.1.0, but it doesn't seem to be related to cabal.

I have a test module that looks like:

prop_simple_creation :: [Char] -> Property
prop_simple_creation a = length a < 5

main = do
     quickCheck prop_simple_creation

When I run this directly (from the cabal-built binary in the dist directory) it prints this:

*** Failed! Falsifiable (after 8 tests and 7 shrinks):    
"aaaaa"

but returns status 0. Very weird.

quickCheck and those similar functions are run in the IO monad only for printing the result to stdout,
however they will not abort the computation with some exception (e.g. exitFailure).
This is why Cabal reports Pass no matter if the properties passed or not.

You could add something like:

import System.Exit

isPass :: Result -> Bool
isPass (Success n l o) = True
isPass _ = False

check :: Testable prop => prop -> IO ()
check p = do
    r <- quickCheckResult p
    if not $ isPass r then exitFailure else return ()

then your main could look like:

main = do
  check prop_encryptDecrypt
  check prop_reverseReverse