Active Record scopes are an interesting thing to test. In projects I’ve worked on, I have seen many different patterns of testing, some much better than others.

A little over two years ago I wrote the gem pg_search, which provides a sort of Domain-Specific Language (DSL) for creating Active Record scopes that take advantage of PostgreSQL’s built-in full-text search.

For example, the following code will set up a scope that accepts query as a string and returns records whose name matches that query. The records will be ordered by relevance.

class Book
  include PgSearch
  pg_search_scope :search_by_name, :against => :name
end

Book.search_by_name("catch") 
# => [#<Book id: 3, name: "Catch 22">, 
#     #<Book id: 7, name: "The Catcher in the Rye">]

In pg_search’s test suite, I found myself needing to test the results of scopes over and over again. In doing so, I believe I have developed a reasonable approach. But first, let’s look at a common pitfall.

Antipattern: Stub the object under test

describe ".chronological" do
  it "should call order('year ASC')" do
    expected_result = double("expected result")
    Book.stub(:order).with("year ASC").and_return(expected_result)

    Book.chronological.should == expected_result
  end
end

In this example, we use an RSpec test double to simulate what would happen if we were to call Book.order("year ASC"). We then call our Book.chronological method to see if it returns the same value.

There are a few problems with this approach. First off, we are testing a class method on the Book class, and also stubbing the class method Book.order. By modifying part of how Book works, we cannot be sure that the object will work when the mocks are absent.

Secondly, our test code is tightly coupled to our implementation. What if we decide that the .chronological scope should always rank records with NULL year first? In PostgreSQL, this would work:

def self.chronological
  order("year ASC NULLS FIRST")
end

This is effectively a new feature, but you have to go back and change the test for an old feature in order to make everything pass.

Better solution: Set up a failing scenario

describe ".chronological" do
  it "orders records by year" do
    later = Book.create!(year: "2005")
    earlier = Book.create!(year: "2002")
    middle = Book.create!(year: "2004")

    results = Book.chronological

    results.index(earlier).should be < results.index(middle) 
    results.index(middle).should be < results.index(later) 
  end
end

In this example, we set up a much simpler situation. We create three records, then expect the chronological method to return them in order. Note that we use Enumerable#index to compare the positions of the records in the output Array.

Now we can experiment with different solutions.

order("year ASC")
order("year")
order("year DESC").reverse_order
joins(:author).order("year ASC")

All of these code examples should pass the test. And our test no longer cares about that joins(:author) part. The original stubbed version would have failed because .order is not called directly on Book anymore.

Avoiding brittleness

We could also have written something like this:

results.should == [earlier, middle, later]

But now that the database is involved, we would need to be more careful to allow for other records that might be there, such as test fixtures. We could solve that by deleting all Book records at the beginning of the spec.

describe ".chronological" do
  it "orders records by year" do
    Book.delete_all

    later = Book.create!(year: "2005")
    earlier = Book.create!(year: "2002")
    middle = Book.create!(year: "2004")

    results = Book.chronological
    results.should == [earlier, middle, later]
  end
end

My personal preference is to avoid the Book.delete_all solution. In my opinion it’s better to have a test that works regardless of the internal state of other objects and systems.

Also, if later down the road a test fixture happens to be present and somehow breaks my test, I have a chance of noticing and doing something about it at that point. If I always delete all of the records, then this free informal fuzz testing goes away.

This is similar to the oft-repeated Robustness Principle (aka Postel’s Law)

Be conservative in what you do, be liberal in what you accept from others.

In other words, I want my test to tread lightly, but also not to fail when a bunch of garbage is thrown at it. It should still pass when I do this:

describe ".chronological" do
  it "orders records by year" do
    later = Book.create!(year: "2005")
    earlier = Book.create!(year: "2002")
    middle = Book.create!(year: "2004")

    1_000_000.times do
      random_year = 2000 + rand(10)
      Book.create!(year: random_year.to_s)
    end

    results = Book.chronological

    results.index(earlier).should be < results.index(middle) 
    results.index(middle).should be < results.index(later) 
  end
end

But that’s not a unit test!

This example fully integrates the test against the database. Some would argue that this is no longer a unit test, and is thus not as good.

I agree that this is not a pure unit test. It has a dependency on the database. For example, if the database is not set up properly, this test will fail, while the stubbed example would still pass.

Scopes are an interesting concept that I believe evade easy unit testing; they are methods that return instances of ActiveRecord::Relation.

The strange thing about these Relation objects is that they behave somewhere in-between a class and an instance. For example, they are clearly not a class.

Book.chronological.is_a?(Class) # => false

But they behave much like the Book class, delegating many of the common things you might do.

Book.chronological.new # => #<Book id: nil, name: nil>
Book.where(year: 2007).destroy_all # (destroys all records with year 2007)

And any class method you define on Book is callable directly on the Relation instance. It’s almost as if it were a subclass of Book. Even silly methods that have nothing to do with the database work almost as normal.

class Book < ActiveRecord::Base
  def self.name_in_french
    "Livre"
  end
end

Book.where(:year => "2000").name_in_french # => "Livre"

And yet Relation objects also act like an Enumerable and have all the methods like #each, #map, #select, and so on. In fact, there is the crazy implementation of ActiveRecord::QueryMethods#select which goes out of its way to guess whether you wanted the Active Record class method or the Enumerable method.

In closing

So scopes are these strange methods that have a fluent interface that chains, create a set of virutal subclasses of your model, and build an abstract syntax tree (AST) of your SQL queries using the Arel gem.

This last point is the nail in the coffin for me. An AST is not that meaningful until it is compiled into runnable code and executed. And with scopes and Relation objects, that means generating SQL code. And SQL code itself is not that interesting until it is executed against a database. For most of its useful operations, a Relation is actually a code generator for another language!

So my vote is to embrace the database, get something working quickly, and move on. That way, you know that your true goals are going to work against your actual application.

I’d love to hear feedback and debate. I don’t believe this is a closed issue. Feel free to join the discussion in the comments!