Inverting the testing pyramid

Why avoid bugs?

Annoyance, money loss, on AWS, missing out on customers, data loss, damages, lawsuites

What is a testing pyramid?

Horizontal axis: Amount of tests.

Vertical axis? Abstraction level, cost, runtime. Order of running?

Levels of the pyramid

Parser, syntax highlighting

Formatter

Linter

Static analysis, Type Check

Code review

Run code in your head.

Manual effort, systemic issues.

Unit test

Test harness, mocks

Intially cleaner design, then harder refacotring.

Fuzzing, performance testing, snapshot testing

More then a unit, abstract, costly, hight amount of data

Integration test

Cypress, playwright, BE mocks/casettes/contract testing

End-to-end test / full stack test

deployment, highest level of automated tests

Manual testing / acceptance test

pulling production data, or even on production, explorative, complex, makes sense, scrolling, synthetic performance

Separate team, discussions, level of detail.

Monitoring / SRE

DataDog, Pingdom, Post mortems morbid

Banana software / beta test / alpha test

OTA Updates, feedback, tracking and AB tests, performance measurement

UFC, Focus group

Effective testing: Effort vs Confidence

Effort

Writing, runtime, maintenance, cost

opportunity cost

tech debt

Confidence

Risk, probability multiplied by the severity.

Severity, medical appliances, cutting a cabel, switches, LBs, openSSL bug, central libraries.

Probabiliy: Complex code, bad/unknown code, often changed code, configuration, unexperienced workers, unforseen dependencies/"JENGA".

bxt/inverting-the-testing-pyramid.md