In Xcode 9.3 code coverage gained improved reporting and performance. The time to load code coverage has been greatly reduced. Accuracy has also been improved. Especially header files were not analysed properly in Xcode 9. This has been fixed in 9.3.
You can now enable or disable code coverage on a per-target basis. This is nice because throw party code will then no longer pollute your coverage results. You can exclude targets in your build scheme editor for your test target.
- You want to have different test for different builds.
- You can tell a scheme to not run certain tests.
- In Xcode 10, there is the option to use opt-in behaviour for tests. So you can configure a scheme to either auto-include or auto-exclude tests.
By default, tests in Xcode run on alphabetical order. This is nice in a way, but allows tests to depend on each other. This is not good practice. Every test should properly set up and tear down their own state. To help make sure you have no implicit dependencies between tests. Xcode 10 introduces a randomisation mode that can be enabled in the scheme editor.
Since Xcode 9, you can test your app on multiple destinations at the same time. This helps you to quickly test an app on several devices. This option is only available using xcodebuild on the command line.
In Xcode 10, you can use parallel distributed testing. This means that you can run multiple tests in parallel on a single device. This feature is available in the Xcode UI, but also in xcodebuild.
When running tests in parallel, xcode will launch several test runners that will get assigned tests from the testing bundle. This distribution is done by class meaning a class is guaranteed to be tested by a single runner.
Xcode created and deletes clones of a simulator device. Each simulator gets its own test runner. The original simulator instance is not used during testing. Instead, it is used as a blueprint for the test runners. Each simulator has its own data container. This means that you can’t rely on changes to the files system on one simulator instance to exist on another simulator instance.
Using xcodebuild you can override the maximum number of runners for parallel testing. By default Xcode will try to figure out a good default for this value so you’re not required to set it.
If you have a very long running test class, it might be worth it to split the class into multiple classes. This helps Xcode to divide tests more evenly over the test runners. This is mostly worth it when a single class slows down your test suite.
If you test performance, don’t use parallel tests. Parallel tests put the system under a heavy load so performance won’t be measured very well. You might want to pull performance tests out into their own, non-parallel tested schema.