ScalaCheck The Definitive Guide.md

ScalaCheck The Definitive Guide

Chapter 1

• A specification is a definition of a program’s behavior in the general case. Preferably, a specification should not deal with concrete examples of program functionality. Instead it should provide an abstract description of the behavior, that you can use both when implementing the program and when verifying it.

• Specifications, on the other hand, give us a more complete picture of the program but they are often informal and difficult to verify formally. Instead, tests are used to verify specific cases of a specification.

• Tests as specification is a concept that is gaining popularity in many test-driven software development processes.

• (...) instead start considering the tests themselves a specification for your program. This has the obvious advantage of making the specification easy to verify: you just run all the tests and make sure they pass. (...) The downside is that by the very nature of tests, such a specification will not be complete. The specification completeness will solely depend on the number of tests that you are willing to define and maintain.

• Property-based testing, as implemented in ScalaCheck, is similar to the concept of tests as specification, in the sense that it results in a specification that you can verify cheaply. However, while the idea of tests as specification is to make your specification more test-centered, property-based testing goes in the opposite direction by making your tests more specification-like. It does so by generalizing tests to properties.

• (...) the tests don’t really describe the method’s behavior; they merely give us a number of usage examples. A property, on the other hand, will give us a more general description of the behavior.

• It will run each test in an attempt to falsify the property, and only if each test for a given property passes will ScalaCheck regard the property as true.

Benefits from property-based testing

• Test coverage: Of course, if you write manual JUnit tests carefully you can get good coverage too, but ScalaCheck makes it harder to miss out on many cases. (...) ScalaCheck’s testing is not completely randomized: by default, when coming up with integers or lists or other basic inputs, it makes sure to always include common edge cases. It includes zero, one, maximum and minimum integers, and empty lists. Such cases can trigger unexpected behavior in code but are easy to miss if you write your test cases manually.

• Specification completeness: This can be useful not only because it enables better testing, but also because it forces you to reflect about your code’s exact behavior. (...) Sometimes it’s not worthwhile to specify a method completely, because it would mean the property would be just as complex as the implementation.

• Maintenance: One of the benefits of unit testing in general is that it gives you confidence to refactor your code often. However, unit tests can be an obstruction when you need to make changes that affect many test cases. Cutting down on testing code means less code to maintain and refactor. Additionally, testing code is sometimes low quality compared to other parts of a project.

• Test readability: is a matter of personal taste, but I often find it easier to read and understand one concise property rather than go through several similar test cases to find out the intended behavior of a given code unit.

• Test case simplification: is a powerful feature of ScalaCheck. It is enabled by the fact that properties are abstract, and ScalaCheck therefore has control over the test data that is used. As soon as ScalaCheck finds a set of arguments that makes a property false, it tries to simplify those arguments.