Duplicate test name detection

Ruby does not throw any exceptions or warnings if an object defines two methods with the same name. The second definition always wins. While this provides great flexibility for many Ruby tasks it can be problematic when writing tests. In particular, if you define two tests with same name in the same test class, one will get run and the other will not. If you have been writing tests for a while you understand that there is nothing worse than writing a test that never gets used!

Recently Matthew O’Connor and I set out to fix this problem for Test::Unit by alias method chaining :method_added for TestCase classes and their subclasses. We use the inherited hook on Ruby classes to dynamically define a method_added hook for every test case. This is required because method_added does not get inherited between classes, so it can’t be defined only in Test::Unit::TestCase.

The following patch raises an exception if it detects a duplicated test name.

class Test::Unit::TestCase
  class << self
    def known_test_methods
      @known_test_methods ||= Array.new
    end

    def record_test_method(method)
      if method.to_s.starts_with?("test")
        if known_test_methods.include? method
          raise "Duplicate test #{self}##{method}"
        else
          known_test_methods << method
        end
      end
    end

    def inherited(subclass)
      class << subclass
        def method_added_with_duplicate_check(method)
          record_test_method(method)
          method_added_without_duplicate_check(method)
        end
        alias_method_chain :method_added, :duplicate_check unless method_defined?(:method_added_without_duplicate_check)
      end
    end
  end
end

When we first tried this against a legacy code base we found almost a dozen duplicated tests that were not running.

8 Comments

Josh Susser says:

Very useful check! But where did you hear that `method_added` wasn’t inherited? I’ve used it before and just double-checked again right now, and it inherits just fine. I also don’t see the need for doing an alias method chain. Just override method_added in TestCase and call super at the end.

class Test::Unit::TestCase
class << self
def known_test_methods
@known_test_methods ||= Array.new
end

def method_added(method)
if method.to_s.starts_with?(“test”)
if known_test_methods.include?(method)
raise “Duplicate test #{self}##{method}”
else
known_test_methods << method
end
end
super
end
end
end

January 14, 2009 at 5:43 am

Adam Milligan says:

This is great stuff; the sort of thing that testing frameworks should have built in.

January 14, 2009 at 6:31 am

Pat Nakajima says:

Great check! I added a similar hook to my [fixjour](http://github.com/nakajima/fixjour) project to make sure people don’t define redundant builder methods: [http://github.com/nakajima/fixjour/blob/master/lib/fixjour/redundant_check.rb](http://github.com/nakajima/fixjour/blob/master/lib/fixjour/redundant_check.rb).

January 14, 2009 at 7:19 am

grosser says:

just switch to rspec/should/anything better then pure test and do yourself+everybody reading your code a favor + no need to worry about such wacky details

January 14, 2009 at 8:47 am

Chris Heisterkamp says:

Josh,

You’re absolutely right about method_added inheriting in Ruby. We initially tried your method but something in our code base (Ruby 1.8.6 / Rails 2.0.2) seems to break/override this inheritance for test cases.

The alias method chain was added because we didn’t want to make assumptions about whether or not these classes might already have a method_added, potentially blowing them away.

January 14, 2009 at 5:49 pm

Chris Heisterkamp says:

Grosser,

We would love to use alternate frameworks, but we’re working with a large legacy code base and it’s not worth the time right now to introduce a new testing framework.

January 14, 2009 at 5:53 pm