Unit tests vs Meaningful tests

convo.md

That dev told me:

This too is to abstract the test from things i don't want to really test, that sould be tested in the calendar's model test. Abstracting from others components and models behavior is especially useful when you don't know ALL the behavior of everything in the app, and because that behavior shouldn't be tested in this model, so you save time by not having to know how that component/model exactly works and by not setting a lot of enviroment variables.

And I answered:

You are right, if writing unit tests is what you want.

I'm asking, please, do not, go further, and write tests that assume less.

We as a team are following a strategy with tests which is to write meaningful tests.

Unit tests are necessary, but not the only kind of tests to focus on.

Because we cannot afford investing in all levels of testing (integration, unit, usability, performance, system, penetration, etc) we need to be wise about what we test and how. So maybe you can get some good degree of coverage, and above all things, confidence that the app is working by writing tests that look like the actual code that is ran when the app is being used by real users.

Mocks are shortcuts that hurt this objective and can backfire when you have 1000 unit tests that "asume" too much without a real reason other than "as a dev I want to write pure unit tests".

With the objectives mentioned above, I don't see the reason why stub everything if by almost cost you can write tests that actually check the app is running correctly.

Example.

calendar.set({require_foo})

instead of

stub(calendar, getRequiredFoo).returns(false).after(bla)

Yes, this is not a unit test anymore, but from my perspective better investment in my test suite:

• I don't assume getRequiredFoo returns true/false or is called getRequiredFoo or is being used by this piece of code I'm testing. I'm focusing on an initial state that changes to another state, I don't care how.
• When we update backbone for example, i run the tests and see that i'm not introducing regressions anywhere (isNew() has changed? does it break my code? stubs will hide this details from you).
• It wasn't too hard to write, it doesn't take much longer to run and gives me more confidence the app is working.
• You tested something that is not tested. Yes, turns out there were no tests for that stubbed model :(

tl;dr: like anything in this world, testing is affected by the law of tradeoffs, and as a team, we are ok with paying the cost of not writing unit tests, for now...