Capybara that is cucumber-like

gistfile1.md

From what I understood this morning, the concern Steve had was to be able to use the acceptance specs as a communication tool with the clients, whether directly (the clients read it) or indirectly (the clients do not read it).

The concern from me (at least) is the maintainability of the Cucumber steps. Here are some example:

• Projects where there were so many cucumber steps that it became hard to figure out what each step did
• Projects that tried to be too DRY with the steps the regexps became difficult to handle
• Projects where a lot of set up was required, and the steps to create the users like became long and too descriptive (Given an active user exists that last logged in 2 years ago)
• Projects whose Cuke steps shared World-wide variables (@project) that were used across steps

For me, using Capybara directly solves this. But, I must agree that only developers can read them. Why not have the best of both worlds?

Given this rspec file:

feature 'User can disable their own account' do

  background do
    given 'a user Bob exists' do
      @user = FactoryGirl.build_stubbed(:activated_user,
                                        username: 'Bob')
    end
  end

  scenario 'Disables own account' do
    given 'I am logged in as Bob' do
      sign_in_as @user
    end

    when 'I go to my account page' do
      visit account_path(@user)
    end

    when 'I click on the button to disable my account' do
      click_button 'Disable my account'
    end

    then 'I should see the message that I am logged out' do
      within '.flash-messages' do
        should_see 'You have been logged out. We will miss you.'
      end
    end
  end

end

The output would contain something like this:

Given I am logged in as Bob
When I go to my account page
And I click on the button to disable my account
Then I should see the message that I am logged out
  expected page to contain 'You have been logged out. We will miss you.' but
  it did not

Won't you come across these?

• Projects where a lot of set up was required, and the steps to create the users like became long and too descriptive (Given an active user exists that last logged in 3 years ago)
• Projects whose Cuke steps shared World-wide variables (@project) that were used across steps

Thank you Ramon, for suggesting what might be a good compromise. I need to think about it more.

Here is one thing that cucumber buys, that his compromise might now address: A change in the requirement produces a diff in the spec that can be associated with the changes made to support that, all legible from reading the git log. When you are going back over the history of a feature, it does help to see clearly how the spec was changed at some point vis a vis other related changes. The more the specification looks like code, the less easy this is to do. If the cucumber is essentially a comment or a label then the requirement could be changed or the test could be changed without necessarily updating the comment and it would be hard to tell if the spec was supposed to change or if the test was changed for another reason solely from looking at the git log/diff.

I am also thinking about how a tool like tracker or our own version of tracker, would better integrate with the source control and timekeeping system. Cucumber provides a valuable hook into the source code for higher level changes. I know that sounds vague.

To a couple of your points following up keikun17:

I think the general idea is not to "reuse" or refactor your cucumber steps to make them dry or whatever. I think in cucumber steps, it is permissible in some cases to not be so dry or at least its okay to repeat yourself as soon as dryness becomes a problem.

I think the performance penalty during setup comes from the fact that you need a database cleaner because transactions do not work across the server/client boundary of capybara/rack, esp when javascript gets involved and capybara launches a browser. This means rebuilding database state from scratch. However, I think this is amenable to technical improvement and solution and we may be able to make a contribution to the state of the art.

I agree sharing world-wide variables is bad. We ought to be able to avoid this.

As a company, we need a process and there is a role for someone to create the cucumber working directly with the client. Right now I am filling that role but eventually it will be someone else, maybe even one of you. There are other ways to do it such has having specification documents that live outside of source control, but these have a lot of variability in format and quality and it is hard to say when a feature is "ready" to be estimated or to start developing. Imagine you are in my position and you are writing the spec for your associates to follow in developing features. How would you express it? You can't think in code, that is too involved. Anyway, its not up to you, you're not going to do the development, just write the spec. What are you going to write? I'll bet you write something like cucumber.

I think the cucumber level of analysis leads to a certain confidence factor that the feature has been thought through enough that it is ready to enter the backlog. Without cucumber we would need to become very good at spec-writing and maintaining and versioning those documents separately and frequently problems would remain buried until work begins and then estimates blow up. Or worse, we rely on the customer's "specs" that are almost never detailed enough or consistent. Either way, quality will suffer.

Ergo, someone is going to write cucumber.

I think Cucumber has an important role in the process and is a powerful tool for formalising requirements and bridging the gap between developers and clients. Now, I know it takes about 20 hours of me writing cucumber with a client to lay out 2-4 months of work for a pair. Ergo, if we were fully booked with 5 pairs, I could expect to spend approximately 30% of my time writing cucumber with clients. If I cannot afford to spend that much time or we want to scale beyond 5 pairs, I need to hire someone. I don't want to become a bottleneck, so it has to be a job that someone else can learn to do, and soon. I have no grasp on how long the spec process would take without cucumber or what the other gotchas are. I can train someone sufficiently able to do this job with cucumber. The only other process I know is something like what HH does and I am curious to know which process our team prefers.

I know this is a process/tool thing. Cucumber is a tool that facilitates a process. Until now it is the best tool that I have seen. If you want to propose a replacement, or even invent one, then we should address how the replacement fits into the process and what the implications will be.

Today it seems the client has made my point by rejecting all three features that were submitted for acceptance because they did not follow the cucumber precisely. Each word counts. This is correct. Without actually using cucumber it seems quite likely that we will miss things and have a lot of rejections. Especially on a greenfield app like this.

My two cents:

Aside from the global variables, what ultimately added complexity in Cucumber is the regex. When doing BDD (presumably with a client), I think the two indispensable components that we are looking for are:

1. Connextra Format/Gherkin syntax - Gherkin is the part of cucumber that deals with the .feature files. Based from Steve's reasons for choosing Cucumber, I suspect that it's really Gherkin that we really want. Basically, we want business analysts (spec/product/domain guy) to articulate specs without dealing with the internals of the app they want us to create.
2. Step Definitions - The code that will be driven by #1 (Capybara and party). In Cucumber's case, I don't like that fact sometimes regex step defs can be a pain to use. I don't want to debug some regex everytime I create a step definition with parameters to match. Frankly, they're not readable by humans.

I think it's overkill to use regex in step defs just for the sake of DRYness and being able to accept parameters. Like what Steve said, steps don't have to be as DRY as our codebase. The right way of describing the scenarios should be in declarative way anyway. Ergo, less regex needed.

Gladly, there are already a number of available alternatives to Cucumber that may address our specific needs. Here are my first impressions of each option based on their docs/description:

1. Steak - basically this is RSpec + Capybara niceties. It throws away Gherkin. We might use this internally, but I'm pretty sure that for client-based work, we wouldn't.
2. Turnip

   • it has Gherkin. +1!

   • step definitions are just in this format (easier than regex):

     module SomeSteps
     step "there is/are :user_count users" do |:user_count|
     # some ruby code here e.g. capybara
     end
     end

   • You control the scope of specific steps via RSpec.configure, since steps are defined in modules. At the very least, this can easily solve step conflicts.

3. Spinach
   • it has Gherkin too!
   • Based on their blog post, the guys who made this basically had the same complaint we had for Cucumber: the way steps defs are implemented in Cucumber suck.
   • Step definitions for a Feature: Some Feature are either in that associated SomeFeature < Spinach::FeatureSteps class, or in the include-d modules. No surprises. Less chances for conflict.
   • Want to encapsulate/expose certain step defs/variables/methods? Do it the Ruby way: private-ize methods,include Modules. Whatever floats your boat.

Personally, I would like to try out Spinach first, simply because it was borne out of the same frustration we had for Cucumber (step definitions). This is the tool that satisfies our minimum requirements.

What do you think?

(This discussion is worthy of a tech talk, a blog post, and/or topic for the upcoming meetup.)

Corrected Markdown format: https://gist.github.com/ec125767b0ac9ddb3093

Why can't we edit gist comments even for a few minutes?