Smelling with your ears: TDD techniques to influence your design

Sunday, May 12, 2013

Test Driven Development can be a hard sell. The first pitch is often designed to entice the buyer with safety features, like:

“How will you ensure that those bugs don’t creep back in?”
“Wouldn’t it be nice to know that one change doesn’t break another?”

In conversations between practiced test drivers, though, design topics tend to pop up:

“What is this test telling us about the design of our code?”
“Why is this test boring to write?”
“Why is this test so slow?”

Then there are really exciting questions, when getting close to a design breakthrough, like:

“Is this test telling us we’re lacking polymorphism in our design?”
“I’m tired of constructing this thing. How can we group this set of arguments into an object with a name?”

One distinguishing factor between these types of questions is the level of trust in TDD. Someone with little trust might be predisposed to abandon testing before implementation, instead choosing to test afterwards, or not at all. To such a person yet to be sold on the benefits of TDD, the safety questions make more immediate sense, while design questions are often met with blank stares. However, the safety concerns are easily brushed off: it’s a prototype. My team is so smart we don’t need tests. We need to move fast, so we’ll worry about tests later.

Explaining the basic advantages of TDD doesn’t always work as a sales pitch, because those explanations don’t reveal why testing can be difficult, much less why testing sometimes ought to be difficult. Take someone who has never let the design of their code be influenced by tests: they dislike testing for being difficult or boring. Encountering resistance in the TDD process, they choose to forgo the safety advantages of testing, and the design advantages haven’t been made clear.

As you may have gathered, I’m more excited by the design aspect of TDD and related tools than by the safety aspect. I’d like to think that if we sold how TDD can improve the design of code that’s yet to be written, we’d have an easier time tricking our friends into writing code with regression protection.

Learning to listen

There is much talk about “listening to the tests” among TDD practitioners. The listening analogy is apt. Like listening with our ears, the ability to understand what a test tells us about code quality can improve with practice. It’s a subtle concept to grasp, and one I frequently find is not well understood by otherwise experienced developers. This is unfortunate, because it’s a crucial part of getting rapid feedback on the quality of production code. By quality, I’m referring primarily to the ability to cope with changing requirements, as opposed to good coverage of features and edge-cases.

If you can’t hear what your tests are trying to say, there are tools for cranking up the volume. Below are a couple of my favorites. They’re not intended as hard-and-fast rules, but as exercises to try out when you’re frustrated with a test or wondering why it’s getting difficult to test something.

If you haven’t already, you should read about known test smells and their solutions, because we can apparently smell with our TDD ears.

Use your testing framework’s convenience helpers sparingly

In the RSpec world, this often comes down to writing readable examples without using ‘subject’, ‘let’ or ‘before’. It turns out that straightforward assignment is usually OK.

As this Thoughtbot post argues, the let helper effectively introduces Mystery Guests (implicit, hidden fixtures), and overuse results in slow and fragile tests.

I like to avoid lets, subjects and other test helpers for another reason: if I can’t stand to repeat myself in examples, I think about how the code that uses my code will feel. A boring, repetitive test setup might be telling me that my code has too many dependencies. If I’m frantically stuffing things into the database and stubbing out web service requests just to allow myself to construct an object, perhaps the object’s scope is too broad.

If you come across a test that is apparently repetitive, consider tidying the implementation of the system under test before the test itself. You may find that the noise in the test can be dramatically reduced with some production code tweaks.

Avoid stubbing methods to return values

I owe this one to Greg Moeck, who introduced something like it at the San Francisco eXtreme Tuesday Club.

First, a reminder of the definition of stubs versus mocks (to paraphrase Gerard Meszaros):

A stub is a test double that allows you to control the indirect inputs of the system under test.
A mock is a test double that allows you to test the indirect outputs of the system under test.

If you return a value from a stubbed method, you force your production code to depend on a blocking, synchronous call. If you could otherwise send a message and not expect an immediate response, you permit your design to (now or in the future) be asynchronous.

Further to that, if you instead use a mock to expect an output to the collaborator you were previously stubbing, you can more cleanly divide your testing into inputs and outputs of the system under test. It’s the difference between:

it "ensures user is authentic before performing the action" do
  user = stub('user')
  authenticator = stub('authenticator')
  authenticator.stub(:authentic_user?).with(user) { true }
  action = Action.new(user)
  action.perform
  expect(action).to be_complete
end

and:

it "ensures the user is authentic when action is requested" do
  user = stub('user')
  authenticator = mock('authenticator') # assume the player of this role knows who to tell when authentication succeeds or fails
  authenticator.should_receive(:authenticate_user).with(user)
  action = Action.new(user)
  action.perform
end

it "performs an action once a user has been authenticated" do
  action = Action.new(stub('unauthenticated user'))
  authenticated_user = stub('user')
  action.user_successfully_authenticated(user)
  expect(action).to be_complete
end

The code that passes the second set of examples is in better shape for when you need to queue requests to the authenticator and complete the action asynchronously. It uses a “tell, don’t ask” style. The fact that an explicit message is sent to the system under test (‘user_successfully_authenticated’) makes it clear to the reader that the request for authentication and the triggering of the action are separate bits of work. It’s someone else’s business whether I get told about the successful authentication, and how many steps are taken before I’m told.

There are several more techniques I’d like to tell you about, but this post is getting a bit long in the tooth. Maybe next time. Happy listening!

Fighting test pollution with an RSpec custom ordering strategy

Andrew Bruce

Thursday, April 25, 2013

Test pollution manifests itself as seemingly false negatives or false positives in a test suite. It occurs when some shared state is unintentionally modified, or unintentionally read and used in a test.

When test pollution builds up, it can mean that a project’s build fails unpredictably, which can stop a whole team from shipping code regularly. This is an expensive way to not build software.
Continue reading →

Making testing visible in the Tracker workflow

Lisa Crispin

Friday, April 5, 2013

As a feature story progresses through the Tracker workflow, a lot of testing activities are also underway. Team members are collaborating to turn examples of desired behaviors into business-facing tests that guide development. Testers are performing manual exploratory testing on stories as they are delivered. Performance or security testing may be underway at some point in the development process.

A testing workflow?

To keep things simple, Tracker’s states are limited to Not Started, Started, Finished, Delivered, Accepted and Rejected. Only the “Accepted” and “Rejected’ states seem directly related to testing. Testing activities such as specification by example, acceptance testing, exploratory testing, load testing, and end-to-end testing aren’t reflected in the Tracker workflow, but they’re going on nevertheless. Testers, coders, product owners and other team members continually talk about how a feature should work, and what to verify before accepting a story as “done”. But details can still be overlooked or lost. If stories are rejected multiple times because of missed or misunderstood requirements, or problems slip by and aren’t discovered until after production release, testing activities need to get more attention.

We’re working on enhancing collaboration and communication in Tracker, with increased flexibility that will help with tracking testing activities. Meanwhile, how can Tracker users follow testing along with other development activities? It would be helpful to have a place to specify test cases, note plans for executing different types of tests, and make notes about what was tested. Accomplishing this requires a bit of creativity, but it’s possible to keep testing visible in the current Tracker workflow. Here are some ways we do this on our own Pivotal Tracker team.

Testable stories

First of all, we work hard to slice and dice our features into stories small increments that are still testable. We read the stories in the backlog to make sure we understand what each one should deliver, and how it can be tested. If I have questions about an upcoming story when we’re not in a planning meeting, I note it in a task or comment to make sure we talk about it. Iteration planning meetings are a good place for the team to start discussing how each story will be tested. Some teams get together with their business experts to help write the stories with this in mind.

We make sure we know how we’ll test all the stories in the upcoming iteration. There are a couple of different ways to get enough of this information into the story .

Using tasks and comments

Test cases and testing notes can be added to a feature story as tasks. They’re easy to see in the story, and can be marked as completed when done. We often include links to additional details documented in a wiki page, or to automate-able functional tests used for acceptance test-driven development (ATDD). As teammate Joanne Webb points out, sharing test cases before implementing a story clarifies requirements, and gives developers clues on problems to avoid introducing. In our experience, this shortens the accept/reject cycle for stories.

Comments are another good place to add information about requirements and test cases, especially since you can also attach files with additional information, screenshots, pictures of diagrams, and mockups. And if team members have questions they can’t get answered in person right away, comments provide a place to record a written conversation, and email notifications can alert the story owner and requester so they can answer questions.

Visibility and workflow through labels

We can find ways to record conversations about requirements, but how do we incorporate a testing workflow into the larger development workflow for a Tracker story?

Labels are a handy way to keep stories progressing through all coding and testing tasks. In our Tracker project, automating functional tests is part of development. The story isn’t marked finished until both unit tests and functional tests are checked in, along with the production code. Once a feature story is delivered, someone (usually a tester or the product owner, but it could be a programmer who didn’t work on coding the feature) picks up the story to do manual exploratory testing.

To make this visible, we put a label on it with our name, for example, “lisa_testing”. Not only do we conduct exploratory testing, we verify that there are adequate automated regression tests for the story, and that necessary documentation is present and accurate. Once we’re done testing a feature story, we put a brief description of what we tested in a comment, remove the “testing” label, and add another label to show the story is ready for the product owner to verify. This might be “lisa_done_testing” or “ready_for_dan”. Sometimes the product owner gets to the story first, and uses similar labels to show he’s in the process of testing or finished with his own acceptance testing. Once all involved parties are happy with the story, we can accept it. Using labels is a bit of extra overhead, but it gives us flexibility to continually improve our acceptance process.

Putting together a bigger picture

Some testing activities extend beyond one story, especially since we usually keep our stories small. It’s possible to write a feature story or chore for the testing activity. For example, you might write a story for end-to-end testing of an epic that consists of many stories and extends to more than one iteration. Writing a chore for performance testing, security testing, or usability testing may be useful.

However, as my teammate Marlena Compton points out, there are advantages to making sure testing is integrated with the feature stories themselves. If a story remains in delivered state for several days while we complete system testing related to it, the labels we put on the story convey the testing activities underway. Completing all testing before accepting a story helps ensure the stories meet customer expectations on the first day. As Elisabeth Hendrickson says, testing isn’t a phase, it’s an integral part of software development, along with coding and other work. Having our Tracker stories reflect that helps keep us on target.

As we do exploratory testing on a feature story, we might discover issues or missing requirements that don’t make the story un-shippable, but may need to be addressed later. We can create separate feature stories, bugs or chores for those, and link back to the original story via links or labels.

We track some testing information outside of Tracker, for example, on our team wiki. However, we find that tracking testing activities in Tracker helps ensure that they get done in a timely manner, and keeping tests visible helps ensure that stories meet customer expectations the first time they’re delivered. Integrating testing activities with coding tasks keeps our testing efforts aligned with other development efforts.

While we work to make Tracker more flexible for teams and testers, we hope these ideas help you make your testing more visible in the Tracker workflow right now. Check out our blog post http://pivotallabs.com/2013-update-new-features-new-api-new-design/ to get an overview of some of the plans for Tracker this year, and come back periodically for the latest news. We’d also love to hear how your team incorporates testing in agile development. Please leave a comment, or write to us at tracker@pivotallabs.com.

Thoughts on Simple BDD

Robbie Clutton

Saturday, March 30, 2013

A small number of projects here in New York have adopted my extremely simple behaviour driven development library, SimpleBDD, and I thought I’d share some of the emerging patterns those teams have developed while using it.

SimpleBDD, is a way of using a Gherkin like specification language and turning into a method call with Ruby. It takes the approach of tools like Cucumber but reduces it down the smallest set of features. Essentially taking a method call:

Given "a logged in user"

and calling an underlaying function:

a_logged_in_user

If that method is in the scope of the executing test, that method is executed, if the method isn’t in scope or doesn’t exist the standard method not found exception is raised. It enables a developer to produce Gherkin like specifications while staying in Ruby, using the test framework of choice.

I generally try and follow the advise of my colleague, Matt Parker, with his excellent post on steps as teleportation devises. We try and create a reusable and stateless domain specific language (DSL) for our tests and our steps call into the DSL and hold state pertinent to that particular test run.

At first on my current project we have had three separate areas for our request specs. We have the request spec itself which used SimpleBDD to describe the behavior of the application. We then had a ‘steps’ file which had the methods calls from the SimpleBDD and translated those into the reusable DSL for the application. The steps file was reused across all request specs and was becoming big pretty quickly.

Dirk, on another project which is also using SimpleBDD, skipped the ‘steps’ file and placed those methods straight into the rspec feature files underneath the scenario blocks. Then after some discussions with JT on where to keep the state our tests depended on, Brent and our team started using rspecs ‘let’ methods and the ‘steps’ within the scope of the feature block to keep the intention of the test in one place.

By also putting more responsibility onto the DSL, these methods are pretty dumb, leaving the test describing what the test is attempting to achieve through SimpleBDD method calls and the how through calling the DSL via the ‘step’ methods within the feature block in the same file.


require 'spec_helper'

feature 'homepage' do

  scenario 'happy path' do
    Given "an existing user with one widget"
    When "the user visits the homepage"
    Then "the user can see their widget"
  end

  let(:user) { FactoryGirl.create(:user) }
  let(:widget) { FactoryGirl.create(:widget) }

  def an_existing_user_with_one_widget
    user.widgets << widget # Using the application code to create initial state
    login user # Application DSL
  end

  def the_user_visits_the_homepage
    visit root_path # Capybara DSL
  end

  def the_user_can_see_their_widget
    can_see_widget widget # Application DSL
  end

end

This has been working out fairly well for us and if you're interested in a simple version for BDD for your project I hope you check this out.

Start small and compose: A strategy for using FactoryGirl

John Barker

Tuesday, March 5, 2013

While I’m still not entirely sold on FactoryGirl, I often see it being used in a particularly lazy way. Imagine your basic factory:

factory :project

Before long you’re adding relations to projects, and the first thing people do is this:

factory :project do
  association :user
end

This immediately means that every single time you instantiate a project, you’re getting a user as well. In most cases this is more than you want, and if you continue to follow down this path you end up with a huge slow test suite. I prefer a slightly different strategy: start small and compose.

factory :project do
  ...
end

trait :with_manager do
  association :user, factory: :manager
end

This defines a very simple factory :project which gives you only a project and allows you to build a project with an associated user like so:

FactoryGirl.create(:project, :with_manager)

The result is a couple more arguments when you use the factory, but the overall code is more intention revealing.

If this is too much, you could always create a more descriptive factory:

factory :managed_project, parent: :project, traits: [:with_manager]

If you stick with this strategy, you’ll find that tests are more concise, factories are more useful and your test suite run time won’t grow as fast.

Broken permalinks and narrative, the pitfalls of moving content.

Rasheed Abdul-Aziz

Wednesday, February 20, 2013

Lately I’ve been thinking about the risks associated with:

moving content
refactoring domain terms within content.

I’ve identified the key areas of risk as narrative and permalinks.

Narrative

The narrative of your site/app can break in subtle and unexpected ways. It is difficult to guard against side-effects upon the narrative. This makes changes to domain terms high-risk. Specifically, a developer focused on the code can easily miss these side effects.

Here’s an obvious, and well guarded story:

Given I am a user
When I visit the user's profile
Then I see their latest activity

That one’s simple, if you move the latest activity to a sub-nav, this story will break (that’s a good thing). In this case, you could ask the product owner, “In the past, having a user’s activity visible on their profile page was desirable. Are you sure you want to lose that?”.

However not all narrative is this cut and dry. Consider this story:

As a user
When I visit a member's profile
I can see a list of pages the member is interested in.

Then, along comes a new unit of work, “Rename all pages to interests”. This seems reasonable, and so you finish the story. The test for the ‘pages’ the member is interested in continues to work, however the member profile ought to lose any reference to ‘pages’. The new domain term ‘interests’ should be used instead.

Now your tests are using misleading or confusing language. You need to update all your stories to use the new term. This can get onerous quickly, but it’s vital for consistency.

I’m yet to find a satisfying technique for mitigating this risk. It’s hard enough keeping the modelling clear of inconsistencies.

This same change to narrative also affects ‘curators’ of ‘pages’. They no longer curate ‘pages’, they now provide ‘interests’. Do we want the curators to go through this mental hurdle, or the strangeness of ‘managing an interest’ versus ‘my interests’? Suddenly a curator has a list of ‘interests which they maintain’ and a list of things they are ‘interested in’.

The best way to mitigate the ‘broken narrative’ risk is to stay strong on short, testable stories. Fast turnaround between development and acceptance is also valuable. The fast feedback cycle is less overwhelming for both developer and product. The chance to think about risks is stronger when the story is as uncomplicated as possible.

The risk of broken narrative is just another reason to avoid this internal desire to take ‘big bites’.

Permalinks

Permalinks and long lasting links, are any URLs for which permanence matters. E.g., the permalink to an article on your blog from some outside source. The risk of broken permalinks can be readily mitigated in your intrgration tests. To achieve this, you need to make sure that product is thinking actively about permalinking. Impart to the product owner that permalinks must be (as much as is possible) explicitly called out in stories.

Permalinks in cucumber tests

Here is an example.

A story in your backlog calls for content that must be available in a permalink. You write a scenario like this:

Given I am a visitor
And there is an article
When I click on a permalink for the article
Then I should see the article

The important term in this story is ‘permalink’ – whenever a step definition calls for a permalink, it needs to use a constructed link:

visit ('/articles/#(article.slug}')

Do not use Rail’s (or your framework of choice) magic:

visit(article_url(article)) # won't capture intent

This way, if the permalink goes away, the test fails, and the intent is clearly captured.

The many unseen dependencies

To complicate matters, a moderately complex site usually has a swathe of references to the content you are about to move/refactor. The term is often used in abundance, E.g. ‘Post’ or ‘Job’ or ‘Activity’. The page might be accessed from several places, E.g. messages are available from the ‘User -> Messages’ nav item, the ‘”My Favorite Lolcats” group messages’ page etc..

Key take-aways:

Fast feedback loops. (small testable and deliverable chunks of work)
Oodles (lots) of product owner/ux designer/copywriter/developer collaboration.
Changes in domain terms need to be kept consistent throughout your tests.

BetterReceive: A More Assertive Mock

Steve Ellis

Friday, February 8, 2013

One thing I’ve always liked about TDD is the ease with which it guides you through development. Given an outside-in approach, one failing test will lead you into another, and as you develop you dive deeper. Each time you make a new set of tests pass you step back to the earlier context and continue until you’ve driven out your story one test at a time. This all works seamlessly, with the exception of adding new methods to a class. For example:

class Developer; end

class Office
  attr_accessor :developers
end

Now, we want to tell the office how to run. Testing it with RSpec would look something like this:

describe Office do
  describe "#run" do
    it "gives the developers a break to play ping pong" do
      dev = Developer.new
      office = Office.new(developers: [dev])

      dev.should_receive :break_for_ping_pong
  
      office.run
    end
  end
end

This test fails because dev does not receive break_for_ping_pong. From here making our tests pass is easier than actually making our code work:

class Office
  attr_accessor :developers

  def run
    developers.each(&:break_for_ping_pong)
  end
end

We run the tests, and we’re done! Everything passes. Time for a ping pong break. Oh, wait… dev still doesn’t respond to break_for_ping_pong, but the tests pass because dev does receive break_for_ping_pong.

Due to the dynamic nature of Ruby and all its meta-programming goodness/complexity, RSpec does not try to predict whether or not objects respond to the methods that are mocked out. Sometimes this is for the best, but in cases like the one above it would be nice to recognize the missing behavior.

Enter BetterReceive. BetterReceive works like any other RSpec mock, with the exception that before mocking/stubbing a method there is an extra assertion that the object under test responds to the specified method.

So, after including ‘better_receive’ in the Gemfile, we can update our test to be more assertive:

      dev.better_receive :break_for_ping_pong

Now we have a properly failing test, so we put off the ping pong break and make the code pass:

class Developer
  def break_for_ping_pong
    # ...
  end
end

More important than driving out new functionality, now that we have working code BetterReceive will catch bugs later on. Imagine break_for_ping_pong was called in other places in the code.

class Pair
  attr_accessor :developers

  def disagree
    developers.each(&:break_for_ping_pong)
  end
end

Changing the implementation of Pair may lead to a refactoring in the Developer model, for example moving break_for_ping_pong onto Pair. The other places in the code base that use break_for_ping_pong may go unnoticed due to oversight, lack of discoverability, or however your bugs slip in. Had we left should_receive in our test for Office the suite would continue improperly passing, harboring a new bug. Since dev still receives break_for_ping_pong, the should_receive assertion passes even though dev doesn’t respond to break_for_ping_pong. BetterReceive catches these regressions and alerts you before letting new changes get too far.

#responds_to? Considered Helpful

Ruby’s method_missing feature is the main thing that stood in the way of tools like RSpec from being able to consistently predict whether an object responds to a method. For this reason, Ruby 1.9.2 introduced responds_to_missing?. The folks over at Thoughtbot have a great post about why you should always define respond_to_missing? when overriding method_missing.

Without updating what an object responds to you are pitting two of Ruby’s biggest strengths against each other: Meta-Programming vs. Duck Typing. (Spoiler: Duck typing loses.) This tradeoff doesn’t have to be made. Many libraries do a good job of updating responds_to? when changing method_missing, but they don’t all. BetterReceive can help you determine which parts of your own code and which libraries you use that have not yet updated responds_to_missing?. Those libraries better receive some pull requests.

Ideally, I would like to see the responds_to? assertion become the default when mocking/stubbing. While there are other obstacles to overcome in making this possible, consistently updating responds_to? is the most important.

Test automation for iOS and smooth jazz

George Dean

Wednesday, February 6, 2013

Interestings

Appium – test automation tool for native and hybrid mobile apps

Does anyone have any experience with it?

Appium is an open source test automation tool for native and hybrid iOS apps.

Use Ruby, Java, Javascript etc.. to write your tests.
You don't have to recompile your app or modify it in any way because Appium's automation is based on Apple's UIAutomation library

http://appium.io

Smooth Jazz Nyan Cat Formatter hits 0.1.3

http://rubygems.org/gems/smooth-jazz-nyan-cat-formatter

With some display fixes. Coming eventually, configure your own music and cat!

Testing strategies with RSpec, NullDB and Nosql

Robbie Clutton

Sunday, February 3, 2013

Recently I had posted about a few testing strategies that can be applied with RSpec. One of the patterns I mentioned was using something like NullDB to ensure your unit tests were not hitting the database. I had a few conversations about what I’d written, notably from my colleague Ian Lesperance. We discussed, and I conceded, that it’s preferable to have tests related to one class in one spec file. In particular I had split out the tests for the unit level and the integration with the database tests. So, here are my experiments on how I brought those tests back while keeping the same integrity of using a database for some test and forcing the null object pattern on other tests.

I had some issues having those tests in the same file, but with a little help from another colleague, JT Archie, we managed to figure it out.

Consider this rspec test:

  describe Widget do
    describe "#higest_selling", :db do
      it "uses the 'highest_selling' scope" do
        ...
      end
    end

    describe "#display_name" do
      it 'concats the widget name and manufacturer' do
        ...
      end
    end
  end

The ‘higest_selling’ method is a scope and has the ‘:db’ tag associated to the block, while the ‘display_name’ test has no tags applied. I wanted this to be the case, no tags means no database but if you want to hit the database, you need to explicitly call it out.

One trick you might have missed above was no longer needing to do ‘db: true’ in the RSpec tag. With the following setting in the spec helper, you can apply a symbol directly like ‘:db’.

config.treat_symbols_as_metadata_keys_with_true_values = true

Testing with NullDB

To get this working, I had to use the HEAD revision of NullDB:

gem 'activerecord-nulldb-adapter', git: 'git://github.com/nulldb/nulldb.git'

Using NullDB within the same file, we can use the ‘nullify’ and ‘restore’ helpers, but I found it worked best using the ‘around’ configuration. Using ‘before’ and ‘after’ I was having issues with changing the connection adapter during a transaction. This way, it appears to get around that issue.

We run the configuration block around each test that has the ‘type: :model’ tag. RSpec-Rails applies these automatically to any tests in the ‘spec/models’ directory. We look to see if the example has the ‘:db’ tag and if it does, we restore the default connection adapter, and run the example. If the example does not have the ‘:db’ tag applied, we apply the NullDB adapter, run the example and then restore the default adapter.

Within the ‘spec_helper.rb’ file:

  config.around(:each, type: :model) do |example|
    if example.metadata[:db]
      NullDB.restore
      example.run
    else
      NullDB.nullify
      example.run
      NullDB.restore
    end
  end

Testing using stubs

There are other options and with a sizable amount of help from JT, we created a simple way to achieve a similar outcome. Under ActiveRecord there are two methods which actually hit the database, ‘exec’ and ‘exec_query’. These methods can be stubbed out much like any method on any object in an application codebase.

In the ‘spec_helper’ file, we replace the NullDB configuration with the following. We again check for the ‘db’ tag and if it’s not there we stub ‘exec’ and ‘exec_query’.

  config.around(:each, type: :model) do |example|
    unless example.metadata[:db]
      ActiveRecord::Base.connection.stub(:exec).
        and_raise("You're not allowed to do that")
      ActiveRecord::Base.connection.stub(:exec_query).
        and_raise("You're not allowed to do that")
    end
  end

Testing using Nosql

We took this concept one step further and created a Gem that wasn’t RSpec specific. We couldn’t believe our luck when RubyGems showed there was no Gem called ‘nosql’, so with that problem solved we created the Nosql gem. When included in a test suite, any call to the database will raise an exception.

With the around configuration block Nosql is disabled and enabled accordingly.

  config.around(:each, type: :model) do |example|
    if example.metadata[:db]
      Nosql::Connection.disable!
      example.run
    else
      Nosql::Connection.enable!
      example.run
      Nosql::Connection.disable!
    end
  end

All three of these options force unit tests to not hit the database. Database calls will either be ignored (NullDB), or will raise an error (Nosql). This should result in decreased execution time for tests as it will encourage the developer to stub out those interactions with the database.

Introducing Simple BDD

Robbie Clutton

Saturday, February 2, 2013

Simple BDD is a way to bring structured natural language BDD syntax into any test framework, but why is this necessary?

Have you ever looked back a test you wrote several months ago, or even the several hours ago and wondered what you were thinking and what you were trying to achieve? Behaviour Driven Development (BDD) attempts to resolve this with the structured language or ‘given’, ‘when’ and ‘then’ and natural language describing how an application should behave.

There are a growing number of tools to assist using BDD in Ruby and Rails today, notably Cucumber. Cucumber extracts the behaviour of the application into plain text files using natural language and uses a parser to translate these into methods which should be executed in relation to the natural language.

What if you’ve already got a good test suite, but you just want some of the structured language that something like Cucumber gives you? Well, recently I created a tiny Ruby library to do just this, Simple BDD. Simple BDD allows use of natural language within any executable code, be that RSpec or any flavour of test library.

Given "an authenticated user"
When "that user submits a message less than 140 characters"
Then "that message is displayed to the users followers"
And "is also shown in the users timeline"

These natural language statements are translated into method calls which are executed in the current scope of the test.

an_authenticated_user
that_user_submits_a_message_less_than_140_characters
that_message_is_displayed_to_the_users_followers
is_also_shown_in_the_users_timeline

My colleague Matt Parker recently wrote about how Cucumber steps as teleportation devices and that same thinking applies here. To make the most of this these wordy methods should be considered jumping points into your testing DSLs.

For a more detailed example, check out the Simple BDD example project.