Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker
Andrew Bruce

Smelling with your ears: TDD techniques to influence your design

Andrew Bruce
Sunday, May 12, 2013

Test Driven Development can be a hard sell. The first pitch is often designed to entice the buyer with safety features, like:

  • “How will you ensure that those bugs don’t creep back in?”
  • “Wouldn’t it be nice to know that one change doesn’t break another?”

In conversations between practiced test drivers, though, design topics tend to pop up:

  • “What is this test telling us about the design of our code?”
  • “Why is this test boring to write?”
  • “Why is this test so slow?”

Then there are really exciting questions, when getting close to a design breakthrough, like:

  • “Is this test telling us we’re lacking polymorphism in our design?”
  • “I’m tired of constructing this thing. How can we group this set of arguments into an object with a name?”

One distinguishing factor between these types of questions is the level of trust in TDD. Someone with little trust might be predisposed to abandon testing before implementation, instead choosing to test afterwards, or not at all. To such a person yet to be sold on the benefits of TDD, the safety questions make more immediate sense, while design questions are often met with blank stares. However, the safety concerns are easily brushed off: it’s a prototype. My team is so smart we don’t need tests. We need to move fast, so we’ll worry about tests later.

Explaining the basic advantages of TDD doesn’t always work as a sales pitch, because those explanations don’t reveal why testing can be difficult, much less why testing sometimes ought to be difficult. Take someone who has never let the design of their code be influenced by tests: they dislike testing for being difficult or boring. Encountering resistance in the TDD process, they choose to forgo the safety advantages of testing, and the design advantages haven’t been made clear.

As you may have gathered, I’m more excited by the design aspect of TDD and related tools than by the safety aspect. I’d like to think that if we sold how TDD can improve the design of code that’s yet to be written, we’d have an easier time tricking our friends into writing code with regression protection.

Learning to listen

There is much talk about “listening to the tests” among TDD practitioners. The listening analogy is apt. Like listening with our ears, the ability to understand what a test tells us about code quality can improve with practice. It’s a subtle concept to grasp, and one I frequently find is not well understood by otherwise experienced developers. This is unfortunate, because it’s a crucial part of getting rapid feedback on the quality of production code. By quality, I’m referring primarily to the ability to cope with changing requirements, as opposed to good coverage of features and edge-cases.

If you can’t hear what your tests are trying to say, there are tools for cranking up the volume. Below are a couple of my favorites. They’re not intended as hard-and-fast rules, but as exercises to try out when you’re frustrated with a test or wondering why it’s getting difficult to test something.

If you haven’t already, you should read about known test smells and their solutions, because we can apparently smell with our TDD ears.

Use your testing framework’s convenience helpers sparingly

In the RSpec world, this often comes down to writing readable examples without using ‘subject’, ‘let’ or ‘before’. It turns out that straightforward assignment is usually OK.

As this Thoughtbot post argues, the let helper effectively introduces Mystery Guests (implicit, hidden fixtures), and overuse results in slow and fragile tests.

I like to avoid lets, subjects and other test helpers for another reason: if I can’t stand to repeat myself in examples, I think about how the code that uses my code will feel. A boring, repetitive test setup might be telling me that my code has too many dependencies. If I’m frantically stuffing things into the database and stubbing out web service requests just to allow myself to construct an object, perhaps the object’s scope is too broad.

If you come across a test that is apparently repetitive, consider tidying the implementation of the system under test before the test itself. You may find that the noise in the test can be dramatically reduced with some production code tweaks.

Avoid stubbing methods to return values

I owe this one to Greg Moeck, who introduced something like it at the San Francisco eXtreme Tuesday Club.

First, a reminder of the definition of stubs versus mocks (to paraphrase Gerard Meszaros):

  1. A stub is a test double that allows you to control the indirect inputs of the system under test.
  2. A mock is a test double that allows you to test the indirect outputs of the system under test.

If you return a value from a stubbed method, you force your production code to depend on a blocking, synchronous call. If you could otherwise send a message and not expect an immediate response, you permit your design to (now or in the future) be asynchronous.

Further to that, if you instead use a mock to expect an output to the collaborator you were previously stubbing, you can more cleanly divide your testing into inputs and outputs of the system under test. It’s the difference between:

it "ensures user is authentic before performing the action" do
  user = stub('user')
  authenticator = stub('authenticator')
  authenticator.stub(:authentic_user?).with(user) { true }
  action = Action.new(user)
  action.perform
  expect(action).to be_complete
end

and:

it "ensures the user is authentic when action is requested" do
  user = stub('user')
  authenticator = mock('authenticator') # assume the player of this role knows who to tell when authentication succeeds or fails
  authenticator.should_receive(:authenticate_user).with(user)
  action = Action.new(user)
  action.perform
end

it "performs an action once a user has been authenticated" do
  action = Action.new(stub('unauthenticated user'))
  authenticated_user = stub('user')
  action.user_successfully_authenticated(user)
  expect(action).to be_complete
end

The code that passes the second set of examples is in better shape for when you need to queue requests to the authenticator and complete the action asynchronously. It uses a “tell, don’t ask” style. The fact that an explicit message is sent to the system under test (‘user_successfully_authenticated’) makes it clear to the reader that the request for authentication and the triggering of the action are separate bits of work. It’s someone else’s business whether I get told about the successful authentication, and how many steps are taken before I’m told.

There are several more techniques I’d like to tell you about, but this post is getting a bit long in the tooth. Maybe next time. Happy listening!

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Andrew Bruce

Fighting test pollution with an RSpec custom ordering strategy

Andrew Bruce
Thursday, April 25, 2013

Test pollution manifests itself as seemingly false negatives or false positives in a test suite. It occurs when some shared state is unintentionally modified, or unintentionally read and used in a test.

When test pollution builds up, it can mean that a project’s build fails unpredictably, which can stop a whole team from shipping code regularly. This is an expensive way to not build software.
Continue reading →

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Robbie Clutton

Thoughts on Simple BDD

Robbie Clutton
Saturday, March 30, 2013

A small number of projects here in New York have adopted my extremely simple behaviour driven development library, SimpleBDD, and I thought I’d share some of the emerging patterns those teams have developed while using it.

SimpleBDD, is a way of using a Gherkin like specification language and turning into a method call with Ruby. It takes the approach of tools like Cucumber but reduces it down the smallest set of features. Essentially taking a method call:

Given "a logged in user"

and calling an underlaying function:

a_logged_in_user

If that method is in the scope of the executing test, that method is executed, if the method isn’t in scope or doesn’t exist the standard method not found exception is raised. It enables a developer to produce Gherkin like specifications while staying in Ruby, using the test framework of choice.

I generally try and follow the advise of my colleague, Matt Parker, with his excellent post on steps as teleportation devises. We try and create a reusable and stateless domain specific language (DSL) for our tests and our steps call into the DSL and hold state pertinent to that particular test run.

At first on my current project we have had three separate areas for our request specs. We have the request spec itself which used SimpleBDD to describe the behavior of the application. We then had a ‘steps’ file which had the methods calls from the SimpleBDD and translated those into the reusable DSL for the application. The steps file was reused across all request specs and was becoming big pretty quickly.

Dirk, on another project which is also using SimpleBDD, skipped the ‘steps’ file and placed those methods straight into the rspec feature files underneath the scenario blocks. Then after some discussions with JT on where to keep the state our tests depended on, Brent and our team started using rspecs ‘let’ methods and the ‘steps’ within the scope of the feature block to keep the intention of the test in one place.

By also putting more responsibility onto the DSL, these methods are pretty dumb, leaving the test describing what the test is attempting to achieve through SimpleBDD method calls and the how through calling the DSL via the ‘step’ methods within the feature block in the same file.


require 'spec_helper'

feature 'homepage' do

  scenario 'happy path' do
    Given "an existing user with one widget"
    When "the user visits the homepage"
    Then "the user can see their widget"
  end

  let(:user) { FactoryGirl.create(:user) }
  let(:widget) { FactoryGirl.create(:widget) }

  def an_existing_user_with_one_widget
    user.widgets << widget # Using the application code to create initial state
    login user # Application DSL
  end

  def the_user_visits_the_homepage
    visit root_path # Capybara DSL
  end

  def the_user_can_see_their_widget
    can_see_widget widget # Application DSL
  end

end

This has been working out fairly well for us and if you're interested in a simple version for BDD for your project I hope you check this out.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Mark Rushakoff

What happened to stdout on CI?

Mark Rushakoff
Thursday, February 14, 2013

We were struggling for a bit yesterday trying to figure out why the few puts statements in our tests weren’t being displayed in Jenkins’ console output.

It turns out the ci_reporter gem that we were using (so that Jenkins could parse our test results) swallows stdout and stderr by default — unless you set the CI_CAPTURE environment variable to the string "off", like it tells you to do in the readme.

That was a confusing default setting. Since my pair and I weren’t involved in setting up Jenkins, and we didn’t install the ci_reporter gem, we assumed something was misconfigured in Jenkins before we started digging through the code.

Hopefully this saves someone else some time!

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Steve Ellis

BetterReceive: A More Assertive Mock

Steve Ellis
Friday, February 8, 2013

One thing I’ve always liked about TDD is the ease with which it guides you through development. Given an outside-in approach, one failing test will lead you into another, and as you develop you dive deeper. Each time you make a new set of tests pass you step back to the earlier context and continue until you’ve driven out your story one test at a time. This all works seamlessly, with the exception of adding new methods to a class. For example:

class Developer; end

class Office
  attr_accessor :developers
end

Now, we want to tell the office how to run. Testing it with RSpec would look something like this:

describe Office do
  describe "#run" do
    it "gives the developers a break to play ping pong" do
      dev = Developer.new
      office = Office.new(developers: [dev])

      dev.should_receive :break_for_ping_pong
  
      office.run
    end
  end
end

This test fails because dev does not receive break_for_ping_pong. From here making our tests pass is easier than actually making our code work:

class Office
  attr_accessor :developers

  def run
    developers.each(&:break_for_ping_pong)
  end
end

We run the tests, and we’re done! Everything passes. Time for a ping pong break. Oh, wait… dev still doesn’t respond to break_for_ping_pong, but the tests pass because dev does receive break_for_ping_pong.

Due to the dynamic nature of Ruby and all its meta-programming goodness/complexity, RSpec does not try to predict whether or not objects respond to the methods that are mocked out. Sometimes this is for the best, but in cases like the one above it would be nice to recognize the missing behavior.

Enter BetterReceive. BetterReceive works like any other RSpec mock, with the exception that before mocking/stubbing a method there is an extra assertion that the object under test responds to the specified method.

So, after including ‘better_receive’ in the Gemfile, we can update our test to be more assertive:

      dev.better_receive :break_for_ping_pong

Now we have a properly failing test, so we put off the ping pong break and make the code pass:

class Developer
  def break_for_ping_pong
    # ...
  end
end

More important than driving out new functionality, now that we have working code BetterReceive will catch bugs later on. Imagine break_for_ping_pong was called in other places in the code.

class Pair
  attr_accessor :developers

  def disagree
    developers.each(&:break_for_ping_pong)
  end
end

Changing the implementation of Pair may lead to a refactoring in the Developer model, for example moving break_for_ping_pong onto Pair. The other places in the code base that use break_for_ping_pong may go unnoticed due to oversight, lack of discoverability, or however your bugs slip in. Had we left should_receive in our test for Office the suite would continue improperly passing, harboring a new bug. Since dev still receives break_for_ping_pong, the should_receive assertion passes even though dev doesn’t respond to break_for_ping_pong. BetterReceive catches these regressions and alerts you before letting new changes get too far.

 
#responds_to? Considered Helpful

Ruby’s method_missing feature is the main thing that stood in the way of tools like RSpec from being able to consistently predict whether an object responds to a method. For this reason, Ruby 1.9.2 introduced responds_to_missing?. The folks over at Thoughtbot have a great post about why you should always define respond_to_missing? when overriding method_missing.

Without updating what an object responds to you are pitting two of Ruby’s biggest strengths against each other: Meta-Programming vs. Duck Typing. (Spoiler: Duck typing loses.) This tradeoff doesn’t have to be made. Many libraries do a good job of updating responds_to? when changing method_missing, but they don’t all. BetterReceive can help you determine which parts of your own code and which libraries you use that have not yet updated responds_to_missing?. Those libraries better receive some pull requests.

Ideally, I would like to see the responds_to? assertion become the default when mocking/stubbing. While there are other obstacles to overcome in making this possible, consistently updating responds_to? is the most important.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
George Dean

Test automation for iOS and smooth jazz

George Dean
Wednesday, February 6, 2013

Interestings

Appium – test automation tool for native and hybrid mobile apps

Does anyone have any experience with it?

Appium is an open source test automation tool for native and hybrid iOS apps.

  • Use Ruby, Java, Javascript etc.. to write your tests.

  • You don't have to recompile your app or modify it in any way because Appium's automation is based on Apple's UIAutomation library

http://appium.io

Smooth Jazz Nyan Cat Formatter hits 0.1.3

http://rubygems.org/gems/smooth-jazz-nyan-cat-formatter

With some display fixes. Coming eventually, configure your own music and cat!

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Travis Grathwell

Running tests with Zeus in RubyMine

Travis Grathwell
Tuesday, February 5, 2013

(Adapted from a thread on the RubyMine support forums)

Zeus is a self-described “language-agnostic application checkpointer for non-multithreaded applications.” As a Rails developer, you can use Zeus to keep your rails environment loaded so your tests run super fast. Spork does this job as well, but Zeus is newer/shinier and requires less explicit configuration. Zeus’ standard operating mode requires you to keep a Zeus server running (started with zeus start) to which you can issue commands (like zeus rspec).

To get Zeus installed on your system, follow the readme or this lovely post at ThoughtBot. The issue they mention with require ‘rspec/autorun’ should be fixed in the latest Zeus, but you can safely remove it from your spec_helper anyway.

In your Gemfile, add zeus:

group :test do
  gem 'zeus'
end

Yes the Zeus manual wants you to only have one global install, but RubyMine won’t be able to see it when running tests, because it always runs them under bundler. (You’ll get a cannot load such file — zeus or zeus is not part of the bundle error.)

Add this script to your Rails project as script/rspec_runner.rb:

#!/usr/bin/env ruby

# Suppress zeus' whining about how it won't use your RAILS_ENV
ENV.delete('RAILS_ENV')

# Zeus 0.13.2 parses options badly. RubyMine will invoke this file like this:
# rspec_runner.rb spec/my_cool_file.rb --require teamcity/spec/runner/formatter/teamcity/formatter --format Spec::Runner::Formatter::TeamcityFormatter
#
# ...but Zeus will parse those options thinking --require is meant for it, and die.
# If the test file is moved to the end, it dies less.
ARGV.push(ARGV.shift)

# Add rspec to the beginning of the commands sent to Zeus
ARGV.unshift 'rspec'

require 'rubygems'
require 'zeus'
load Gem.bin_path('zeus', 'zeus')

In RubyMine’s ‘Edit Configurations’, under Default -> Rspec:

Check ‘Use Custom Rspec Runner Script’
Set it to [your-absolute-app-path]/script/rspec_runner.rb

Your ‘Edit Configuration’ window should look like this: zeus_setup

When starting the Zeus server, use the following line:

env RUBYLIB=/Applications/RubyMine.app/rb/testing/patch/common:/Applications/RubyMine.app/rb/testing/patch/bdd zeus start

(I made this a script in my project called script/startzeus.)

These directories are where RubyMine keeps its favorite test formatters, and if you don’t tell Zeus about them at server load time, it may never find them. (you’ll get a cannot load such file — teamcity/spec/runner/formatter/teamcity/formatter error). If you’re not on OSX, you should replace these directories with something relevant to your system.

Now you can run a test in RubyMine just like you would normally, and it should be LIGHTNING FAST. Get it? Lightning? Zeus? You get it.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Robbie Clutton

Testing strategies with RSpec, NullDB and Nosql

Robbie Clutton
Sunday, February 3, 2013

Recently I had posted about a few testing strategies that can be applied with RSpec. One of the patterns I mentioned was using something like NullDB to ensure your unit tests were not hitting the database. I had a few conversations about what I’d written, notably from my colleague Ian Lesperance. We discussed, and I conceded, that it’s preferable to have tests related to one class in one spec file. In particular I had split out the tests for the unit level and the integration with the database tests. So, here are my experiments on how I brought those tests back while keeping the same integrity of using a database for some test and forcing the null object pattern on other tests.

I had some issues having those tests in the same file, but with a little help from another colleague, JT Archie, we managed to figure it out.

Consider this rspec test:

  describe Widget do
    describe "#higest_selling", :db do
      it "uses the 'highest_selling' scope" do
        ...
      end
    end

    describe "#display_name" do
      it 'concats the widget name and manufacturer' do
        ...
      end
    end
  end

The ‘higest_selling’ method is a scope and has the ‘:db’ tag associated to the block, while the ‘display_name’ test has no tags applied. I wanted this to be the case, no tags means no database but if you want to hit the database, you need to explicitly call it out.

One trick you might have missed above was no longer needing to do ‘db: true’ in the RSpec tag. With the following setting in the spec helper, you can apply a symbol directly like ‘:db’.

config.treat_symbols_as_metadata_keys_with_true_values = true

Testing with NullDB

To get this working, I had to use the HEAD revision of NullDB:

gem 'activerecord-nulldb-adapter', git: 'git://github.com/nulldb/nulldb.git'

Using NullDB within the same file, we can use the ‘nullify’ and ‘restore’ helpers, but I found it worked best using the ‘around’ configuration. Using ‘before’ and ‘after’ I was having issues with changing the connection adapter during a transaction. This way, it appears to get around that issue.

We run the configuration block around each test that has the ‘type: :model’ tag. RSpec-Rails applies these automatically to any tests in the ‘spec/models’ directory. We look to see if the example has the ‘:db’ tag and if it does, we restore the default connection adapter, and run the example. If the example does not have the ‘:db’ tag applied, we apply the NullDB adapter, run the example and then restore the default adapter.

Within the ‘spec_helper.rb’ file:

  config.around(:each, type: :model) do |example|
    if example.metadata[:db]
      NullDB.restore
      example.run
    else
      NullDB.nullify
      example.run
      NullDB.restore
    end
  end

Testing using stubs

There are other options and with a sizable amount of help from JT, we created a simple way to achieve a similar outcome. Under ActiveRecord there are two methods which actually hit the database, ‘exec’ and ‘exec_query’. These methods can be stubbed out much like any method on any object in an application codebase.

In the ‘spec_helper’ file, we replace the NullDB configuration with the following. We again check for the ‘db’ tag and if it’s not there we stub ‘exec’ and ‘exec_query’.

  config.around(:each, type: :model) do |example|
    unless example.metadata[:db]
      ActiveRecord::Base.connection.stub(:exec).
        and_raise("You're not allowed to do that")
      ActiveRecord::Base.connection.stub(:exec_query).
        and_raise("You're not allowed to do that")
    end
  end

Testing using Nosql

We took this concept one step further and created a Gem that wasn’t RSpec specific. We couldn’t believe our luck when RubyGems showed there was no Gem called ‘nosql’, so with that problem solved we created the Nosql gem. When included in a test suite, any call to the database will raise an exception.

With the around configuration block Nosql is disabled and enabled accordingly.

  config.around(:each, type: :model) do |example|
    if example.metadata[:db]
      Nosql::Connection.disable!
      example.run
    else
      Nosql::Connection.enable!
      example.run
      Nosql::Connection.disable!
    end
  end

All three of these options force unit tests to not hit the database. Database calls will either be ignored (NullDB), or will raise an error (Nosql). This should result in decreased execution time for tests as it will encourage the developer to stub out those interactions with the database.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Robbie Clutton

Introducing Simple BDD

Robbie Clutton
Saturday, February 2, 2013

Simple BDD is a way to bring structured natural language BDD syntax into any test framework, but why is this necessary?

Have you ever looked back a test you wrote several months ago, or even the several hours ago and wondered what you were thinking and what you were trying to achieve? Behaviour Driven Development (BDD) attempts to resolve this with the structured language or ‘given’, ‘when’ and ‘then’ and natural language describing how an application should behave.

There are a growing number of tools to assist using BDD in Ruby and Rails today, notably Cucumber. Cucumber extracts the behaviour of the application into plain text files using natural language and uses a parser to translate these into methods which should be executed in relation to the natural language.

What if you’ve already got a good test suite, but you just want some of the structured language that something like Cucumber gives you? Well, recently I created a tiny Ruby library to do just this, Simple BDD. Simple BDD allows use of natural language within any executable code, be that RSpec or any flavour of test library.

Given "an authenticated user"
When "that user submits a message less than 140 characters"
Then "that message is displayed to the users followers"
And "is also shown in the users timeline"

These natural language statements are translated into method calls which are executed in the current scope of the test.

an_authenticated_user
that_user_submits_a_message_less_than_140_characters
that_message_is_displayed_to_the_users_followers
is_also_shown_in_the_users_timeline

My colleague Matt Parker recently wrote about how Cucumber steps as teleportation devices and that same thinking applies here. To make the most of this these wordy methods should be considered jumping points into your testing DSLs.

For a more detailed example, check out the Simple BDD example project.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Robbie Clutton

Testing strategies with RSpec

Robbie Clutton
Tuesday, January 8, 2013

These are some tricks, tips, strategies, lessons learnt from a year of working with RSpec on Rails projects. Rails convention over configuration has proved fruitful for adoption and standardisation but there times when you do want to tweak certain things.  We’ll highlight some ways how that can be done such as changing the directory structure of your tests.

RSpec Directory Structure

Coming from a Java background, I was used to see all of a projects tests in the ‘test’ directory with subdirectories off of those for certain types of tests such as ‘unit’ and ‘integration’.  Using libraries like rspec and rspec-rails sets certain expectations on where certain types of tests will be. Model tests will be in ‘spec/models’, controllers in ‘spec/controllers’. This has been great importing useful helpers for those given situations, but it doesn’t tell me if those are unit tests or something else.

From rspec-rails 2.4 tags such as “type: :controller” can be used to import the appropriate helpers and the tests can be placed where they fit best for the application. I would recommend the following:

spec/
  ./unit
    ./models
    ./controllers
    ./etc...
  ./integration
    ./models
    ./etc...
  ./acceptance

The spec files themselves would look like:

describe FooModel, type: :model do
…
end

RSpec 2.12 (current latest stable) supports tags for model, controller, helper and routing.

Unit testing without a database

Why is it important to do unit testing without hitting a database? If a test hits a database then the execution path is through your business logic, through a persistence library, perhaps a database adapter and then the database. A unit test should be focused on the smallest unit of execution, namely the response and messages sent by a class method. Besides testing this whole stack, there typically tends to be an order of magnitude more unit tests in a project than any other type of test and if the tests are hitting the database hundreds or thousands of times this will eventually lead to a slow test suite.

One way of ensuring unit tests do not hit a database in Rails is to use a null object pattern database adapter. The best known is nulldb.  Although the latest stable release doesn’t support Rails 3.1+ adding the HEAD sha as a dependency will support the very latest Rails projects. The only place an application should really be hitting the database is within the application models so the nulldb adapter can be set for those types of tests.

In Gemfile:

gem 'activerecord-nulldb-adapter', git: 'git://github.com/nulldb/nulldb.git'

In spec/spec_helper.rb:

RSpec.configure do |config|
  ...
  config.before(type: :model) do
    require 'nulldb_rspec'
    ActiveRecord::Base.establish_connection :adapter => :nulldb
  end
  ...
end

If an application is integrating with a software as a service the result is the same. Unit tests should not be hitting that API, stubs should be used to mock the dependency. One way to ensure the tests are not hitting an API would be to open the class that hits the API and overwrite any methods that the application uses and raise an exception.

class ApiClient
  def do_something
    raise “Whoa there, you shouldn’t be here!”
  end
end

If an application does need to sometimes hit that API and sometimes not, RSpec allows models to be included depending on the tag. This would allow some test groups to use the API and others to be intercepted and raise the unexpected visitor error.

config.include ApiInterceptors, group: :model

What’s in an integration test

The integration tests are the best place for testing database integrations like scopes, or complex queries. They are also a good place for testing integrations with other libraries or services such as a message queue, background worker or remote API. An application may also want to test that the layers of an application are integrated correctly. After all with all that stubbing in the unit tests it can mislead a developer as to the correctness of an application. If an application has observers or is event driven this would be a good point to see if those layers are integrated successfully.

Sharing behaviours for integration and acceptance testing

If a project is using the BDD language in stories, the application may be reusing those in its acceptance tests. These stories are often the application codified in human language. Where they are used in the application they are often used in the acceptance testing stage but it doesn’t have to be constrained to there. If those stories are written without specifying the how it can be easier to share, e.g. “a user creates a foo widget” rather than “a user clicks on the ‘create foo widget’; a user fills in ‘bar’ for the foo widget; a user presses the create button’.

RSpec has a concept of ‘shared examples’ which can be used to share behaviours between testing stages. If the test reads in such a way where it could be run against a browser, command line or API, all of those test layers could use the behaviour.

In spec/features/foo_example.rb:

shared_examples "foo" do
  describe "user creates a foo” do
    it "lists new foos" do
      given_user_exists
      when_user_creates_a_foo
      then_foo_is_listed_against_user
    end
  end
end

Then in the target test file, including the shared example will bring that test in and be run in that context with associated tags.

in spec/integration/foo_spec.rb

require 'spec_helper'

def given_user_exists
end

def when_user_creates_a_foo
end

def then_foo_is_listed_against_user
end

describe 'Foo', type: :foo do
  include_examples "a foo"
end

The method calls can be defined so they are inline and in scope, this way each file has its own implementation of the requirement methods for the test. The methods can also be extracted into a module and included through RSpec configuration.

in spec/spec_helper.rb

config.include(IntegrationHelpers, type: :foo)

in spec/support/integration_helpers.rb

module IntegrationHelpers
  def given_user_exists
  end
  def when_user_creates_a_foo
  end
  def then_foo_is_listed_against_user
  end
end

Following this pattern means the business language can be added to code and used in multiple places, and therefore stages of the testing.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (780)
  • rails (113)
  • testing (88)
  • ruby (83)
  • ruby on rails (70)
  • jobs (62)
  • javascript (55)
  • techtalk (44)
  • rspec (38)
  • ironblogger (32)
  • productivity (30)
  • activerecord (29)
  • gogaruco (29)
  • git (28)
  • nyc (27)
  • rubymine (26)
  • bloggerdome (23)
  • mobile (22)
  • process (21)
  • pivotal tracker (20)
  • cucumber (20)
  • jasmine (19)
  • design (18)
  • ios (18)
  • webos (17)
  • objective-c (17)
  • android (16)
  • palm (16)
  • "soft" ware (16)
  • fun (15)
  • tracker ecosystem (15)
  • ci (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • bdd (14)
  • gem (13)
  • css (13)
  • tdd (13)
  • selenium (12)
  • goruco (12)
  • bundler (12)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
  • mojo (10)
  • chef (10)
  • api (10)
Subscribe to rspec Feed
  1. 1
  2. 2
  3. 3
  4. 4
  5. →
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >