Step definition hell. We’ve all heard of it. We’ve all experienced it. The question is, why?
This hell is borne from a simple, yet fundamental, misunderstanding.
When I first learned Cucumber, I instinctively thought of a step definition as a method. I could squint and imagine I was looking at a method. The regex looked roughly like a method “name”. The block arguments were basically like method arguments. The body of the block looked like a method body.
This led me to treat my step definitions like a basic unit of organization within my test suite. A typical step definition body might look like this:
Given /^I have created a widget named "([^"]+)$"$/ do |widget_name|
visit widgets_path
fill_in "Name", with: widget_name
select "sortable", from: "Type"
check "Fizzable"
check "Buzzable"
within(".widget_form .actions") do
click_on "Submit"
end
within(".widget_form .confirmation") do
choose "Widget Administrator", from: "Approver"
click_on "Confirm"
end
endThen, I began calling one step definition from another step definition:
Given /^I configured a widget named "([^"]+)$"/ do |widget_name|
step "Given I have created a widget named \"#{widget_name}\""
step "Given I have changed the fizbuzz property of my widget \"#{widget_name}\" to \"wuzbang\""
step "Given the widget administrator has approved the widget \"#{widget_name}\" configuration"
endAnd then I nearly killed myself. Because it turns out that step definitions are not methods. Let’s start with the step definition method “name”. It’s not a method name. When you “call” it, you mix in the arguments with the “name”, making the “name” change depend on the arguments. This makes it nearly impossible to do something as simple as an automatic refactor/rename of these “methods” (unless you like writing really complicated regular expressions for find and replace), much less any more complicated refactorings.
Also, there’s a reason real method names are concise. It’s so that we can remember them. With a test suite of more than a couple feature files, you simply can not remember the names of cucumber steps with any satisfying degree of accuracy (which is the same reason you shouldn’t attempt to force your product owner to remember all the exact wordings of existing steps, and instead, let them write the same “step” many different ways).
If you pursue this path of treating step definitions like methods, you will create such a tangled mess that you’ll be left with little choice but to either abandon cucumber entirely or burn your cukes to the ground and start over.
Step Definitions are Teleportation Devices
The only way I’ve found to do sustainable acceptance testing (whether or not it’s cucumber, rspec feature specs, or minitest integration tests) is to create an underlying system of helper methods that represent actions in your application. For example, if you were building Twitter, I would expect to open up your acceptance testing suite and find a module or set of modules with methods that represent the Twitter application domain. That might look something like this:
module Helpers
def authenticate(user)
visit root_path
fill_in "Username", with: user.username
fill_in "Password", with: user.password
submit
end
def tweet(message)
visit tweets_path
fill_in "Tweet", with: message
#...
end
def reply(tweet, message)
#...
end
def dm(message, recipient)
#...
end
#... etc
endA system of underlying helper methods like this make it really easy to spin up new features. It makes it really easy to let your product owner write the same step 5 different ways. Instead of tracking down the exact wording of the step already in the system and rewriting the feature file your product owner wrote, just take the feature “as is”, spin up new step definition and use your DSL to fill it out (creating any new DSL methods to match new concepts as you go).
If you want to make these module methods available to your cucumber step definitions, you can use the World method:
World Helpers
If you’re using RSpec request specs, use the RSpec configuration:
RSpec.configure do |c|
c.include Helpers, type: :request
end
Then you’re left with step definitions that transport you from the feature file to your underlying DSL:
Feature: Tweet
Scenario: Valid tweet
Given Bob has authenticated
When Bob submits a valid tweet between 1 and 140 characters
Then his followers should receive his tweet
And Bob should see his tweet in his timeline
Given /^I have authenticated$/ do
authenticate bob
end
When /^Bob submits a valid tweet between 1 and 140 characters$/ do
tweet valid_message
end
#...That’s why I think of them as teleportation devices. They transport me from the written world (the feature file) to a world of code (my application’s acceptance DSL).
I’ve been there. Must be a lesson we all have to learn the hard way: low-level tests should only interact with low-level details; high level tests should only interact with a facade of the system.
February 1, 2013 at 3:50 pm
Using Spinach instead of Cucumber solves a ton of problems.
github.com/codegram/spinach
February 1, 2013 at 4:28 pm
Pingback: Embracing Test Driven Development for SpeedAdventures in HttpContext | Adventures in HttpContext
The thing that gets me about this solution is the weird handling of state. Is “bob” a method wrapping an instance variable (state) set by something else? Worse, is it a method that creates a record in the database and then memoizes itself, changing the program’s state with what may have been meant to just be a query? I’ve been bitten by this stuff and have yet to figure out a good way to manage state clearly in Cucumber suites.
February 9, 2013 at 12:44 pm
Why is it weird, and how were you bitten?
February 13, 2013 at 10:15 am
I’ve had the same experience with Cucumber. It seems to me that the misunderstanding is all to common which is why am now of the opinion that Cucumber is partly to blame for not being opinionated enough. An excerpt from one of my blog post:
“Cucumber’s introduction to the world shares a lot of similarities with Rails’ debut: many folks got excited and some started to come up with their own ideas for fitting it into their context. The difference is that, at the onset, the Rails community was very opinionated about how it should be used. While that might have slightly turned off some folks, it didn’t stop the framework from growing to what it is now. Better to have a very limited, but well functioning tool than one that promises many things but doesn’t exactly meet anyone’s expectations.” from http://www.relaxdiego.com/2012/04/on-cucumbers-opinionatedness.html
So I took it one (probably two) steps further: http://www.relaxdiego.com/2012/05/how-we-use-cucumber-the-sequel.html
February 13, 2013 at 12:50 am
Can’t figure out how to reply, so I’ll just post a new comment.
What’s weird about making a method called ‘bob’ is that it is mixing retrieving a value and setting a value. Calling it modifies state in a way that is not obvious, because it has the semantics of a ‘getter’.
I’ve been bit by this in scenarios where a step uses something that appears to be an innocuous getter, and actually is modifying instance variables and creating new records in the database. Later steps relying on instance variables or a known state of the database then fall apart as the world shifts underneath them.
Was just googling for opinions on this, and found my own :)
March 4, 2013 at 12:57 pm