“Rails is slow, but Rails tests are slower.” Rails may be slow, but I’m here tell you that it’s likely you have only yourself to blame for your slow test suite.

I’ve seen some bad test suites in my day. I was once pulled onto a rescue project that had a total build time of over 24 hours. The health and future of your production code depends on a fast, reliable test suite. On my current project, we believe in fast feedback and we’ve focused a lot of detail and attention to our test suite. It’s paid off in dividends.

We’re six months into a project, and we have a one-minute test suite. That includes both rspec and cucumber (and the time to boot up rails for both). Here’s how we’ve accomplished it:

1. Pyramid

Your test suite is a pyramid. Skinny at the top (journey acceptance tests), fat at the bottom (unit tests). Conceptually, this sounds right, and it’s easy to think that it should just naturally fall out of the BDD outside-in process.

But in reality, it’s not a given. Tools like cucumber make it so easy to spin up new high-level acceptance tests for every edge case you can think of. Taken to an extreme, this can lead to unit-testing at the browser level. That doesn’t mean you were wrong to drive out exceptional paths at the acceptance level. But you should ask yourself this question before you check in: “Will I have any less confidance in my test suite if I don’t check in this acceptance test?” If you have already covered a happy path of a feature, and at least one exceptional path of a feature at the acceptance level, do you then need to cover every other exceptional case at the acceptance level? If you’ve added tests down at lower levels in your stack, then you might already have all the verification value you need.

2. Grooming

We groom our test suite. Not every day. But we watch it and keep it in order. We listen to pain in our test suite (“Why did that change cause so many other tests to break? And why did I have to go to every single test and fix each one of them individually?”). When we revisit old tests, we reconsider it in light of what we know about our application. We keep an eye out for duplicate tests.

3. Refactoring

Our fast test suite practically begs us to refactor. And it turns out, refactoring often leads to simpler designs, objects with fewer methods (and therefore fewer tests), as well as objects that are easier and more natural to test in isolation. Great tests will lead you to refactor your production code, which will lead to even better tests.

4. ActiveRecord Containment

How many tests require you to create data in the database? Ideally, that’s limited to just your acceptance tests and your activerecord model tests. But the reality is never that simple. Although it’s technically possible to accomplish this with any Rails application, it’s not always feasible, or even desired. Libraries you use may make this difficult in some circumstances (e.g., devise).

You might also find yourself in a situation where testing an object in isolation would lead to a very brittle test that knows the entire implementation via stubs and mocks. Worse, you may not see a way to untangle these dependencies. That’s OK. Test the object’s behavior, even if it means integrating with other objects. As your understanding of your application’s problem domain crystallizes, and as more patterns begin to emerge, you will eventually find a way to simplify it, to untangle the dependencies.

5. GC.disable

At one point on our project, our build time crept up to nearly two minutes. We saw ourselves slipping down a very slippery slope. We started running it less, refactoring less, and even finding less motivation to keep our test suite clean. So we threw a chore at the top of our backlog to bring the test suite time back down to a minute. Our PM was naturally wary, asking us to timebox the chore to an hour or two, but we made the case that this was of critical importance to the future of the project, and that the pair working on it would check with the rest of the team after a day.

Near the end of the day, the pair stopped and told the team that they thought they had done all they could, but it only got the test suite down to 1:20. Shaving forty seconds off a test suite in a day is a fantastic feat, and when we looked at what they’d done, we were really impressed. I jokingly suggested disabling GC. And then they actually did it. The test suite time dropped to 48 seconds. I’m not recommending you do this. This is a bit of a nuclear option. If this is the first thing you try to improve your test suite time, then you are missing the point of bringing your test suite time down. But if you feel like you’ve done everything you can to legitimately bring down the time of your test suite, then consider disabling garbage collection. Ruby GC is a beast, with the potential of turning a linear algorithm that creates N objects into a quadratic algorithm. Weigh the pros and cons, and decide for yourself if you can live with the dirty dirty feeling this will give you.

“I like to make my views so dumb, there’s no reason to test them.”

Uncle Bob Martin said that at some point, or something very close to it. I’d like to take that a step further: I like to make my (Rails) controllers so dumb, there’s no reason to test them (unless there’s no higher level acceptance test that would exercise the actions in them).

Complicated controllers are painful. Each controller action is like a mini main() function. That’s a lot main()s. The more each action knows about your underlying application, the more brittle your application gets. Think of an action as a launching point into your business domain.

I have some rules (that I occasionally break). It goes something like this:

1) Controllers should be RESTful. No custom actions. new, create, update, destroy, edit, show. That’s it.
2) Controllers should manage a single resource. If you’re instantiating more than a single object in your controller, you’re probably going to regret it.

There’s some great gems out there that can support this restrictive approach to Rails development:

1) Responders (https://github.com/plataformatec/responders). Responders are built into Rails, but the FlashResponder in the responders gem is essential. Replace all of your tedious flash message management code with defaults that can be overriden in your localization files. You can create your own responders to replace any tedious bookkeeping in your controllers.

2) Informal::Model (https://github.com/joshsusser/informal). Since you’re limited to instantiating a single object in your controllers, it’s likely that you won’t get by with just ActiveRecord models. You’ll need to create higher level models that can coordinate the work of all your lower level database models (and action mailers, etc.). Heads up, Rails 4 will obsolete this gem with ActiveModel::Model.

3) ActiveModel::Serializer (https://github.com/rails-api/active_model_serializers). This only applies if you’re creating a JSON API. But if you are, consider this gem. It’s convention over configuration for your API. It works with responders and makes all of the JSON format choices for you so that you can focus on more important things.

I’ll leave with you this:

= simple_form_for @widget_search do |f|
  = f.input :query
  = f.submit

= render @widget_search.results

class WidgetSearchController
  def new
    @widget_search = WidgetSearch.new
  end

  def create
    @widget_search = WidgetSearch.create(query: request.query_parameters)
  end
end

class WidgetSearch
  include Informal::Model

  attr_accessor :query

  validates_presence_of :query

  def self.create(options={})
    new(options).tap &:save
  end

  def save
    return false unless valid?
    @results = Widget.periscope query
  end  

  def results
    @results || []
  end
end

You might have noticed a “periscope” method in there. Yet another fantastic gem.

While working on AwesomeResource, I needed to implement functionality that would make the following test pass:

  it "creates readers and writers for any attributes passed in during initialization" do
    article_class = Class.new do
      include AwesomeResource
    end

    article = article_class.new("title" => "Fun")
    article.title.should == "Fun"

    article.title = 'Fun Times!'
    article.title.should == 'Fun Times!'

    expect { article.unknown_attribute }.to raise_exception
  end  

Simple enough. If you initialize an instance of a class that includes AwesomeResource, then you get attribute readers and writers for any hash keys passed in during initialization.

My first attempt at implementation looked something like this:

module AwesomeResource
  attr_reader :awesome_attributes

  def initialize(attributes={})
    @awesome_attributes = AwesomeResource::Attributes.new attributes
  end

  def method_missing(method_name, *args)
    if method_name["="]
      if awesome_attributes.has_key?(method_name[0...-1])
        awesome_attributes[method_name[0…-1]] = args.first
      else
        super
      end
    else
      if awesome_attributes.has_key?(method_name)
        awesome_attributes[method_name]
      else
        super
      end
    end
  end
end

Yuk. All those nested if/else’s didn’t sit well with me. Let me try to clean that up:

  def method_missing(method_name, *args)
    if method_name["="] && awesome_attributes.has_key?(method_name[0...-1])
      awesome_attributes[method_name[0...-1]] = args.first
    elsif awesome_attributes.has_key?(method_name)
      awesome_attributes[method_name]
    else
      super
    end
  end

The best I can say for this code is that it’s more compact. But is it any more readable? Hardly. In fact it’s worse.

Let’s take another look at the first implementation. The nested if/else blocks look awfully similar. Perhaps there’s a polymorphic model lurking there? But what are we modeling? If we were to wrap a class around those blocks, we’d have to be modeling a method. What would that look like? Let’s extract some classes:

  class AttributeWriter
    attr_reader :attributes

    def initialize(attributes)
      @attributes = attributes
    end

    def call(attribute_name, attribute_value)
      raise "Unknown Attribute" unless attributes.has_key?(attribute_name)
      attributes[attribute_name] = attribute_value
    end
  end

  class AttributeReader
    attr_reader :attributes

    def initialize(attributes)
      @attributes = attributes
    end

    def call(attribute_name)
      raise "Unknown Attribute" unless attributes.has_key?(attribute_name)
      attributes[attribute_name]
    end
  end

  def method_missing(method_name, *args)
    if method_name["="]
      AttributeWriter.new(awesome_attributes).call(method_name[0...-1], *args)
    else
      AttributeReader.new(awesome_attributes).call(method_name)
    end
  end

OK, I at least feel like I’m trying to write OO code at this point. There’s a bit of duplication between the AttributeReader and AttributeWriter classes. We could clean that up with template methods:

  class AttributeAccessor
    attr_reader :attributes, :attribute_name

    def initialize(attributes, attribute_name)
      @attribute_name = attribute_name
      @attributes = attributes
    end

    def call(*args)
      raise "Unknown Attribute" unless attributes.has_key?(attribute_name)
      execute(*args)
    end

    def execute(*)
    end
  end

  class AttributeWriter < AttributeAccessor
    def attribute_name
      super[0...-1]
    end
    
    def execute(attribute_value)
      attributes[attribute_name] = attribute_value
    end
  end

  class AttributeReader < AttributeAccessor
    def execute
      attributes[attribute_name]
    end
  end

  def method_missing(method_name, *args)
    if method_name["="]
      AttributeWriter.new(awesome_attributes, method_name).call(*args)
    else
      AttributeReader.new(awesome_attributes, method_name).call
    end
  end

Notice that we've moved the responsibility of stripping off the "=" off the method_name from method_missing down into AttributeWriter.

Hmmm... The base class is pretty abstract. How easy it this code to understand now?

Also, the body of the method missing now looks a lot like a factory method. While in Rome...

  class AttributeAccessor
    attr_reader :attributes, :attribute_name

    def self.from_method_name(attributes, method_name)
      if method_name["="]
        AttributeWriter.new(attributes, method_name)
      else
        AttributeReader.new(attributes, method_name)
      end
    end

    def initialize(attributes, attribute_name)
      @attribute_name = attribute_name
      @attributes = attributes
    end

    def call(*args)
      raise "Unknown Attribute" unless attributes.has_key?(attribute_name)
      execute(*args)
    end

    def execute(*)
    end
  end

  class AttributeWriter < AttributeAccessor
    def attribute_name
      super[0...-1]
    end

    def execute(attribute_value)
      attributes[attribute_name] = attribute_value
    end
  end

  class AttributeReader < AttributeAccessor
    def execute
      attributes[attribute_name]
    end
  end

  def method_missing(method_name, *args)
    AttributeAccessor.from_method_name(awesome_attributes, method_name).call(*args)
  end

I'm now starting to question this code. Is there a good trade off here between indirection and maintainability? I don't think the number of accessors are likely to grow (accessors have consisted solely of getters and setters since the dawn of OO). Between all of the refactorings, I think the first (with its minor duplication) was the easiest to understand.

Yet something still doesn't feel right. Let's look back at the very first code snippet. Notice the `AwesomeResource::Attributes.new` in the `initialization` method? Those are our bag of attributes (they're basically a hash with indifferent access). We keep passing it around to all of our Attribute* classes… perhaps some of this code should have lived there in the first place?

module AwesomeResource
  attr_reader :awesome_attributes

  def initialize(attributes={})
    @awesome_attributes = AwesomeResource::Attributes.new attributes
  end

  def method_missing(method_name, *args)
    awesome_attributes.accessor_for_method_name(method_name).call(*args)
  end
end

module AwesomeResource
  class Attributes < SimpleDelegator
	#...

    def accessor_for_method_name(method_name)
      if method_name["="]
        ->(attribute_value) { self[method_name[0...-1]] = attribute_value }
      else
        -> { self[method_name] }
      end
    end

    def [](key)
      validate_key_exists(key)
      attributes[standardized_key(key)]
    end

    def []=(key, value)
      validate_key_exists(key)
      attributes[standardized_key(key)] = value
    end

    private
    def validate_key_exists(key)
      raise "Unknown attribute" unless has_key? key
    end
    
    #...
  end
end

This feels right. We're now modeling methods with lambdas (Ruby's built in method objects). We've eliminated a tangle of nested if/else blocks without introducing too much indirection. We've maintained high-cohesion within the AwesomeResource::Attributes class despite the new methods.

Let’s talk about cargo culting.

When I started doing BDD, I cargo-culted it. I basically had no idea what I was doing. I worked at a company that had never done it. I worked with developers who had never tried it. I read blogs posts and watched screencasts, and I thought I got it. I didn’t. I cargo culted it, and I got it wrong. Result? Pain.

Lots of developers have had a similar experience, and they reached a crossroads: fight or flight. I fought the pain. Did you?

Let’s talk about history.

I want a museum of code. I want to see the first program that had a test written in it. Maybe it’s in C. Algol. Lisp. Maybe it’s a punchcard. Who knows?

Testing started as a movement with Smalltalk. They created SUnit. Smalltalk Unit. It worked. They unit tested their code. That was the easiest type of test to write. It didn’t require any fancy tooling or extra frameworks. Load up an object, assert on it.

Over the years, higher levels of testing emerged. Functional tests, integration tests, acceptance tests. Tooling grew up along side these higher level tests to make them possible.

What was Agile doing?

Agile had conversations. They were talking to their product owners, asking them questions, drawing out concrete examples. Today we call this “Specification By Example.”

Dan North came along. He said TDD was backwards. Why do we starting writing tests at the unit level? We have this process of specification by example with our product owners, and our development process doesn’t line up with that. What if we turned our development process on its head and TDD’ed outside-in? What if that converastion was truly a launching-point into our development process, what if that conversation was a spec that told us when we were done?

BDD is about drawing a line from a conversation with a product owner through the development of a feature and back to the product owner. BDD tools make this possible.

If your BDD process doesn’t start with a conversation with your product owner, you will feel pain.

Step definition hell. We’ve all heard of it. We’ve all experienced it. The question is, why?

This hell is borne from a simple, yet fundamental, misunderstanding.

When I first learned Cucumber, I instinctively thought of a step definition as a method. I could squint and imagine I was looking at a method. The regex looked roughly like a method “name”. The block arguments were basically like method arguments. The body of the block looked like a method body.

This led me to treat my step definitions like a basic unit of organization within my test suite. A typical step definition body might look like this:

Given /^I have created a widget named "([^"]+)$"$/ do |widget_name|
  visit widgets_path
  fill_in "Name", with: widget_name
  select "sortable", from: "Type"
  check "Fizzable"
  check "Buzzable"

  within(".widget_form .actions") do
    click_on "Submit"
  end

  within(".widget_form .confirmation") do
    choose "Widget Administrator", from: "Approver"
    click_on "Confirm"    
  end
end

Then, I began calling one step definition from another step definition:

Given /^I configured a widget named "([^"]+)$"/ do |widget_name|
  step "Given I have created a widget named \"#{widget_name}\""
  step "Given I have changed the fizbuzz property of my widget \"#{widget_name}\" to \"wuzbang\""
  step "Given the widget administrator has approved the widget \"#{widget_name}\" configuration"
end

And then I nearly killed myself. Because it turns out that step definitions are not methods. Let’s start with the step definition method “name”. It’s not a method name. When you “call” it, you mix in the arguments with the “name”, making the “name” change depend on the arguments. This makes it nearly impossible to do something as simple as an automatic refactor/rename of these “methods” (unless you like writing really complicated regular expressions for find and replace), much less any more complicated refactorings.

Also, there’s a reason real method names are concise. It’s so that we can remember them. With a test suite of more than a couple feature files, you simply can not remember the names of cucumber steps with any satisfying degree of accuracy (which is the same reason you shouldn’t attempt to force your product owner to remember all the exact wordings of existing steps, and instead, let them write the same “step” many different ways).

If you pursue this path of treating step definitions like methods, you will create such a tangled mess that you’ll be left with little choice but to either abandon cucumber entirely or burn your cukes to the ground and start over.

Step Definitions are Teleportation Devices

The only way I’ve found to do sustainable acceptance testing (whether or not it’s cucumber, rspec feature specs, or minitest integration tests) is to create an underlying system of helper methods that represent actions in your application. For example, if you were building Twitter, I would expect to open up your acceptance testing suite and find a module or set of modules with methods that represent the Twitter application domain. That might look something like this:

module Helpers
  def authenticate(user)
    visit root_path
    fill_in "Username", with: user.username
    fill_in "Password", with: user.password
    submit
  end

  def tweet(message)
    visit tweets_path
    fill_in "Tweet", with: message
    #...
  end

  def reply(tweet, message)
    #...
  end

  def dm(message, recipient)
    #...
  end

  #... etc
end

A system of underlying helper methods like this make it really easy to spin up new features. It makes it really easy to let your product owner write the same step 5 different ways. Instead of tracking down the exact wording of the step already in the system and rewriting the feature file your product owner wrote, just take the feature “as is”, spin up new step definition and use your DSL to fill it out (creating any new DSL methods to match new concepts as you go).

If you want to make these module methods available to your cucumber step definitions, you can use the World method:

World Helpers

If you’re using RSpec request specs, use the RSpec configuration:

RSpec.configure do |c|
  c.include Helpers, type: :request
end

Then you’re left with step definitions that transport you from the feature file to your underlying DSL:

Feature: Tweet

  Scenario: Valid tweet
    Given Bob has authenticated
    When Bob submits a valid tweet between 1 and 140 characters
    Then his followers should receive his tweet
    And Bob should see his tweet in his timeline

Given /^I have authenticated$/ do
  authenticate bob
end

When /^Bob submits a valid tweet between 1 and 140 characters$/ do
  tweet valid_message
end

#...

That’s why I think of them as teleportation devices. They transport me from the written world (the feature file) to a world of code (my application’s acceptance DSL).