This document contains notes, insight, and heuristics for the function of tests in engineering and the domain of software. I'm continually editing this document, so information will be out of place, unorganized.
The role of test can be described in many ways, here are a few that I find important:
- Tests should give a deeper understanding of the code's design
- Tests should be behavior-driven, and not implementation-driven
Testing gives us the ability to understand and think more clearly about code, it requires us to break our problem into smaller tasks, and it can complement documentation of what code does is or is supposed to do. Note: a higher percentage of test coverage doesn't mean safety.
Assume the implementation of a function or module with a fake version of that function or module.
We mock modules or functions when:
- we aren't concerned about testing how the function works, and are concerned about the value the function returns, or if functions call each other
- we want to assert that a function has been called
Stubbing is a technique that mocks requested data from a downstream request - and the data is in the format that an upstream service expects. Stubbing can be used for server and frontend development when we aren't sure about the underlying data structure - whether that's SQL or NoSQL. This technique can help a team's velocity because they don't have to depend on the downstream request to be operational.
Refactoring our code should not break our tests. If we're testing the changes to implementation details then we are testing the wrong things
A new behavior is the instigation of writing a test, not the addition of a new class. We want to test the API, and not the detail. We should test outside in. This doesn't mean testing our UI, it means to question what our public API is for a given module.
The problem that we get with refactoring is due to the fact that we are testing implementation details and not the API. We are coupling our tests with our implementation details. Put another way, if a test knows far too much about our implementation details then we are identifying the wrong stuff to test. Instead, we should be identifying the public API of our given module. For example, we have a number of classes that deliver a service, what's that API this service is implementing?
Defining the behaviors that our software needs to provide to consumers reveals the public exterior that we should expose. How we implement those behaviors should be encapsulated. If we refactor (change the implementation details) and we keep the API the same then the tests won't break. We'll also write significantly fewer tests. We're not talking about testing classes, we're talking about behavior. I don't want to know how the class is implemented, I want to know if it provides a behavior "When we write a test, we imagine the perfect API for our operations - we are telling ourselves a story about how the operation will look from the outside. This story won't always come true, but it's better to start from the best possible API and work backward than to make things complicated, ugly, and realistic from the get-go" - Kent Beck
Port and Adapter architectures help us understand where our tests should be
For frontend Most of the time when I'm talking about integration I'm talking about purely UI components and their integration with each other. Services is a whole different ballgame that I don't play very often.
When tests are a pain, it's not the test - it's the code.
When we evaluate what a unit of code should be doing, we should ask "is this code is delegating or specifying logic?" Our modules shouldn't tangle logic with delegation, and our tests should impose this rule. How can we know if a unit of code is specifying logic or it's delgating?
If there are no dependencies of the file, then the test specifies logic - a pure function. If the function calls change, we should expect the test to change, however; if the logic changes, that's an implementation detail
If isolated unit tests won't inform your design, then don't bother writing them. A commonly held assumption amongst developers is that we should write tests for everything - near 100% code coverage is good. I'd argue we don't gain anything by breaking code into small, testable units, simply because the virtue of code being smaller.
When do we use mocks? When writing tests, we must think - what value does this test give us? Will this test make me safe for a certain kind of change? We ned to thin about design upfront. functions either do something or delegate to someone, but never both. If we mock just to fake out a database, then we're doing it wrong.
Mocks mock out the implementation of a function. Capturing calls to the function
In the case of UI components - we should test how the component is used, not the implementation details. If we can replicate the component's usage, then we can build confidence a component will work exactly the way it was intended. This assumption relies on the fact that w know upfront how our components will be used - that's a problem of framing. We must do our best to frame our use cases by asking questions like:
- in what ways do we want the consumers of my component want to use it?
- does the API of our component describe the use cases we've defined?
- what ways can we envision this component to change - are we testing implementations details (that will change as our component changes), or are we testing larger use patterns? Meaning, our component may experience refactoring, wherein the implementation may change, but not the behavior itself - are we testing for behavior, or implementation? Hint: for component testing, we should be testing for behavior and not implementation.
- What sort of tests gives us false positives?