When you are working on a project, you normally know what code needs testing. For this example
I will show a way to figure out what is missing tests. Just keep in mind that not all of your
code needs or can be tested (e.g. in Java OutOfMemoryError or IOException blocks can be tricky
to test).
When running your code with tests, some tools keep track of what portion of the code was actually executed with the tests.
These tools provide a report for a metric called code coverage.
Cylc UI Server creates a code coverage report as part of its CI pipeline. The coverage is calculated
using pytest-cov and the coverage.py Python module. They produce an XML file with the metrics,
and in the CI pipeline a step uploads it to Codecov.io, a website that produces visualizations for
code coverage.
You can inspect the code coverage of Cylc UI Server at https://codecov.io/gh/cylc/cylc-uiserver.
There are several types of tests. Unit tests, functional tests, integration tests, performance tests, feature tests, regression tests, smoke tests, etc.
In some contexts or projects, integration and functional tests may be the same thing. There is no consensus on what each type of test must cover exactly.
Unit test is the easiest to identify or define. They are normally smaller, and run without any runtime requirements. (But a unit test for a telecom middleware system, and a gaming engine, may be completely different.)
Their only requirement is normally the source code and the test code.
There are several books and tutorials on how to write good unit tests.
- https://docs.microsoft.com/en-us/visualstudio/test/unit-test-basics?view=vs-2019
- https://www.martinfowler.com/bliki/UnitTest.html
IMHO, you should see what works best for you when writing tests. Some of the most popular coders may have an opinionated view, saying that you must isolate your unit test, or that you must use a single assert, or that unit tests are not effective for finding bugs and only regression tests really work. In practice I think YMMV :shrugs: