Run iOS unit tests (in a Xcode workspace) when a file changes and *only* those tests related to the changed file. Also trims otest output and colors test results.

Instructions.md

Instructions

Enable ‘Run’ in the ‘Build’ section of your test bundle’s scheme. (See http://www.raingrove.com/2012/03/28/running-ocunit-and-specta-tests-from-command-line.html)

Install the Kicker tool and the dependencies

$ gem install kicker open4 colored xcodebuild-rb

Install the .kick example file and the otest wrapper

$ curl https://raw.github.com/gist/3011214/8f660de3ef091770290660680e7c195052b3b0e0/kick.rb > .kick
$ curl https://raw.github.com/gist/3011214/b1dd39b5509a87e16c691177926eced004fa2d14/run-tests.rb > run-tests
$ chmod +x run-tests

Update the files for your application’s requirements

$ edit .kick
$ edit run-tests

Finally, start Kicker

$ kicker -c

And an example of the output:

21:04:55.90 | Executing: xcodebuild -sdk iphonesimulator -workspace AppName.xcworkspace -scheme AppUnitTests build

Building target: AppUnitTests (in AppName.xcproject)
====================================================

Configuration: Debug
................................

Finished in 2.041412 seconds.
Build succeeded.

21:04:59.47 | Success
21:04:59.47 | Executing: ./run-tests AppPostsSyncSpec

2012-07-05 21:05:02.204 otest[8715:7803] + 'AppPostsSyncSpec synchronizes remote Posts with the local DB' [PASSED]
Executed 1 test, with 0 failures (0 unexpected) in 2.188 (2.213) seconds

21:05:02.21 | Success

kick.rb

require 'rubygems'
require 'xcodebuild'

def xcodebuild
  formatter     = XcodeBuild::Formatters::ProgressFormatter.new
  reporter      = XcodeBuild::Reporter.new(formatter)
  output_buffer = XcodeBuild::OutputTranslator.new(reporter)
  arguments     = "-sdk iphonesimulator -workspace AppName.xcworkspace -scheme AppUnitTests build"

  perform_work("xcodebuild #{arguments}") do |status|
    status.exit_code = XcodeBuild.run(arguments, output_buffer)
  end
end

def run_tests(tests)
  if xcodebuild.success?
    execute "./run-tests #{tests.join(' ')}"
  end
end

recipe :ignore
ignore /DerivedData/
ignore /\.(xcodeproj|xcworkspace)/
ignore /\.sw\w$/ # vim swap files

# Adapt this `process` block for your application's specific requirements.
process do |files|
  tests = files.take_and_map do |file|
    case file
    # a changed test file should always run
    when %r{Test/}
      file
    # runs *all* sync tests when any of the base classes change
    when %r{Classes/Sync/Base Classes}
      Dir.glob("Test/Sync/*.m")
    # runs only the sync test that maps to the changed file
    when %r{Classes/Sync/App(\w+)Sync\.(h|m)$}
      Dir.glob("Test/Sync/App#{$1}SyncSpec.m")
    end
  end
  run_tests(tests.map { |m| File.basename(m, '.m') }) unless tests.empty?
end

run-tests.rb

#!/usr/bin/env ruby

require 'rubygems'
require 'open4'
require 'colored'
require 'pathname'

class OTestOutput < Array
  def initialize(io, verbose, source_root)
    @io, @verbose, @source_root = io, verbose, Pathname.new(source_root)
  end

  def <<(line)
    return if !@verbose && line =~ /^Test (Case|Suite)/
    super
    @io << case line
    when /\[PASSED\]/
      line.green
    when /\[PENDING\]/
      line.yellow
    when /^(.+?\.m)(:\d+:\s.+?\[FAILED\].+)/m
      # shorten the path to the test file to be relative to the source root
      if $1 == 'Unknown.m'
        line.red
      else
        (Pathname.new($1).relative_path_from(@source_root).to_s + $2).red
      end
    else
      line
    end
    self
  end
end

product_name     = 'AppName'
test_bundle_name = 'AppUnitTests'

source_root        = File.expand_path('../..', __FILE__)
derived_data_root  = File.join(source_root, 'DerivedData')
built_products_dir = File.join(derived_data_root, product_name, 'Build/Products/Debug-iphonesimulator')
dev_root           = '/Applications/Xcode.app/Contents/Developer'
sdk_root           = File.join(dev_root, 'Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.1.sdk')

ENV['DYLD_FRAMEWORK_PATH']           = "#{built_products_dir}:#{File.join(sdk_root, 'Applications/Xcode.app/Contents/Developer/Library/Frameworks')}"
ENV['DYLD_LIBRARY_PATH']             = built_products_dir
ENV['DYLD_NEW_LOCAL_SHARED_REGIONS'] = 'YES'
ENV['DYLD_NO_FIX_PREBINDING']        = 'YES'
ENV['DYLD_ROOT_PATH']                = sdk_root
ENV['IPHONE_SIMULATOR_ROOT']         = sdk_root
ENV['CFFIXED_USER_HOME']             = File.expand_path('~/Library/Application Support/iPhone Simulator/')

verbose = !!ARGV.delete('--verbose')
test_suites = ARGV.empty? ? 'All' : ARGV.uniq.join(',')

command = "#{File.join(sdk_root, 'Developer/usr/bin/otest')} -SenTest #{test_suites} #{File.join(built_products_dir, "#{test_bundle_name}.octest")}"
# otest only seems to use stderr, but let's be safe
stdout, stderr = OTestOutput.new(STDOUT, verbose, source_root), OTestOutput.new(STDERR, verbose, source_root)
status = Open4.spawn(command, :stdout => stdout, :stderr => stderr, :status => true)
exit status.exitstatus

This looks more promising than my approach. Does it run UI-dependent unit tests from the command line? (ie. something that creates a UIFont or something)

Last time I looked, I thought otest only ran 'logic' unit tests and you needed to actually boot the app in the simulator with the right dynamic load paths set for it to run the tests, but maybe I was misinformed? (my approach was to try and reproduce this functionality, by writing code that would automatically load tests with dlopen and run the tests when the host app was run using waxsim or ios-sim).

My attempts to reproduce the functionality of Apple's RunPlatformUnitTests shell script haven't worked with the recent Xcode 4.5 developer previews. They do work under 4.4 though, as long as you do the standard hack of editing a copy of the shell script.

I'd love to see this kind of functionality built into the xcodebuild-rb or xcoder gems. xcoder has a nice output formatter for Kiwi/OCUnit tests that might be useful too.

This looks more promising than my approach. Does it run UI-dependent unit tests from the command line? (ie. something that creates a UIFont or something)

Last time I looked, I thought otest only ran 'logic' unit tests and you needed to actually boot the app in the simulator with the right dynamic load paths set for it to run the tests, but maybe I was misinformed?

I haven’t tried that yet, as the app I was working on did need that.

(my approach was to try and reproduce this functionality, by writing code that would automatically load tests with dlopen and run the tests when the host app was run using waxsim or ios-sim).

My attempts to reproduce the functionality of Apple's RunPlatformUnitTests shell script haven't worked with the recent Xcode 4.5 developer previews. They do work under 4.4 though, as long as you do the standard hack of editing a copy of the shell script.

When I played with hacking those scripts, nothing really worked to my satisfaction, so I extracted what I needed. I assume we’ll have to do the same for application tests. If you have such tests laying around, then by all means please test it and/or extract from the shell script what we need :)