gistfile1.irclog

15:55 smathy: terrellt, is the tl;dr that you're replacing capybara tests with controller and/or model tests?
15:57 smathy: Assuming so, capybara should be used only when it's impossible to test other ways.
15:57 pipework: I use feature tests for important workflows.
15:58 pipework: But I keep them pretty minimal and sometimes run them on-demand and just in CI, not in the default test suite rake task.
15:58 terrellt: smathy: Controller/model/view tests - effectively the tipping point for me is I'm deciding that there being an interface that I have to touch my tests for should I change it is a -good- thing.
15:59 terrellt: Whereas before I used feature specs to give me complete flexibility independent of architecture.
16:00 terrellt: pipework: I think where I'm landing now says that I'd avoid them ESPECIALLY for important workflows, because if I add that crutch I'm less likely to think critically about the architecture.
16:00 pipework: During development, I'll often write feature specs to drive implementation, but then I push tests as far down the test suite as possible.
16:01 pipework: terrellt: Why can't you have both?
16:01 terrellt: pipework: Test code is still code, duplication of tests is bad because maintainability gets worse.
16:01 pipework: I always test lower stuff, but if I feel like it's not an important workflow in the user's UI, then I sometimes drop the feature test. It's something I haven't figured out entirely yet.
16:02 pipework: terrellt: I don't feel like feature tests ever duplicate other tests.
16:02 pipework: Integration and unit tests are non-functional style tests, and feature is a functional, and blackbox test.
16:03 terrellt: The argument I'm making to myself is that integration testing as done by Capybara is SO high that it masks the important decisions one makes when defining the boundary of components, and that testing those -boundaries- is not only sufficient, but doing that and an integration test is duplicative.
16:04 pipework: terrellt: Calling capybara tests as integration tests is only true if you reach into the system to set up state.
16:04 terrellt: s/integration/feature
16:05 pipework: Doing both isn't duplicative because the lower level can't assert on the functional level like the higher level can.
16:05 terrellt: /g
16:05 pipework: The opposite is true as well.
16:05 terrellt: Can you elaborate on the functional level point?
16:06 pipework: terrellt: There's two sorts of functional level, and two sets of rules for either. The two sorts are flow tests and focused feature tests. The two sets of rules are 1) No internal muddling about 2) You can set state inside the app for the test. But most of all, the functional level can only assert on the interface of the black box, your web application as the user 'sees' it.
16:07 pipework: For lower level, you can't assert on the behaviour of the outside of the box, otherwise it becomes one of those pseudo-functional integration tests anyways.
16:09 terrellt: Your argument is that feature specs are for testing flow through the application, IE "mouse clicks"?
16:11 pipework: terrellt: Well, they're for either flow through the system /or/ for testing a feature without the flow to arrive at it.
16:11 terrellt: I...might be able to buy feature specs as GET flows through an application
16:11 rhizome: i think of mouse clicks as an integration thing
16:11 terrellt: I think as soon as you POST you've gone too far.
16:11 pipework: Why GET?
16:12 pipework: terrellt: Functional tests are about inputs and outputs at the blackbox level.
16:12 pipework: For a completely contrived example, a flow is a user signup from start to finish, a focused feature test is "after the user filled out the signup form, this test asserts that the user is sent to the 'getting started' page."
16:13 pipework: I've tried, but not been able to accomplish nicely, a way to be able to specify a whole flow based off of multiple distinct focused features.
16:14 terrellt: At the same time, you architect that into clearly defined boundaries with transition points. My form view has these params, my controller action accepts those same parameters and redirects, my service object takes those parameters and creates a user
16:14 terrellt: you can*
16:15 pipework: terrellt: But if you don't exercise the user interface, you're integration testing.
16:15 pipework: Unless you don't consider the browser runtime interpreting your web application as part of the user interface.
16:15 terrellt: No, I'm unit testing, and I think I'm arguing that I don't need an integration test for that.
16:16 pipework: terrellt: I often 'push' feature specs down into lower level tests, if that's what you're getting at.
16:16 terrellt: Maybe I am.
16:16 terrellt: No, I'm not.
16:17 pipework: I really hate functional tests, they're slow. I try to balance having enough valuable tests with not bogging down the suite.
16:17 terrellt: If I push down then that means I designed the system to make a flow work, which means I have no reason to think about what my architecture is like until I start pushing down and it hurts.
16:17 pipework: terrellt: Do you TDD?
16:17 terrellt: Yes
16:18 pipework: I write a flow's first step, then the focused feature test, then I move to the inner parts as they're needed to satisfy the test 'above' the current part.
16:18 pipework: Some people prefer to make small closed loops, I prefer many loops within each other that drives implementation down to the lowest part, and when that's done entirely, all the tests pass at once.
16:19 pipework: Then I remove functional tests that aren't as valuable, keeping the lower level stuff that it drove the implementation of.
16:20 terrellt: That approach is going to take some thought and research.
16:20 terrellt: This has been useful to me, thank you pipework.
16:20 pipework: terrellt: Thank you too, good sir.
16:20 terrellt: Would you mind if I put this in a gist and stick it in that PR?
16:20 pipework: terrellt: Not at all.
16:21 pipework: Please, if you find a nice way to not have to have one test be a flow that retests the focused parts that are also their own tests, let me know.
16:21 terrellt: Will do.
16:21 pipework: I'd love to have a flow just be a container of focused parts that cna be run together as a representation of the flow.