Recently I was about to check in some changes and did a last minute
click through of the application. All of a sudden I’m staring at a stack
trace. My tests were green and I had functional tests for the failing
controller/action.
Tests are like pants — they cover your backside while you focus on other
things like adding features to your application. Suddenly I felt a
breeze on my cheeks. Something was amiss.
I soon discovered the action and its associated tests had diverged
over time. Some of the parameters were renamed in the action but not
in the functional test. Since some of the work of the action was
conditional on the presence of certain parameters, that work was no
longer being tested.
This exposed weaknesses in the tests and code, such as expected
side effects in the tests that are never checked. If they had been
checked the tests would have failed and the parameter name mismatch
would have been discovered.
Most functional tests provide specific parameters that should at least
be examined during the processing of the action. Reporting unread
parameters would strengthen those tests. It was conceivable to me that
some of the other functional tests had similar unused parameters. I
wanted all of my functional tests to report all unused parameters.
The first step was to instrument the params hash. I wanted to track
access to the params hash and report parameters that were not read
during the processing of the action. I don’t know what all is done to
params during the lifecycle of a test. I’m only interested in access
from the time the action starts till it returns so I need to be able
to turn the tracking on and off at specific times.
It turns out that Rails uses a subclass of Hash called
HashWithIndifferentAccess.
I added my changes to
HashWithIndifferentAccess in test/test_helper.rb:
class HashWithIndifferentAccess
def [](key)
@accessed_keys ||= {}
@accessed_keys[key] = true
super
end
def start_logging
@accessed_keys = {}
end
def end_logging
@accessed_keys['action'] = true
@accessed_keys['controller'] = true
never_accessed = []
self.each_key do |key|
never_accessed << key unless @accessed_keys.include?(key)
end
raise "Some keys never accessed: #{never_accessed.join(', ')}" unless never_accessed.empty?
end
end
With these changes an exception will be raised if any first level keys
are not read between start_logging
and end_logging.
In each of my functional tests I added code similar to this (from account_controller_test.rb):
class AccountController
around_filter :check_params
private
def check_params
params.start_logging
yield
params.end_logging
end
end
The around filter starts and ends the logging in the context of the
action.
With these changes in place my tests no longer passed and my
backside was warm and protected again.
Webrat, a plugin I wrote to do acceptance testing with Rails, protects your functional tests from a parameter name breakage.
http://agilewebdevelopment.com/plugins/webrat
January 23, 2008 at 8:10 pm
very clever
January 23, 2008 at 8:15 pm
I really like the idea of expanding the test framework this way — it’s great information for the developer, it doesn’t bog down production code, and the errors are presented at just the right time in the workflow (during testing).
Do you run into problems with some of the extra params that rails inserts that you don’t routinely use in your actions? I see the exclusions for “action” and “controller”, but what about “format” and “commit”? (For those who are wondering, “format” contains the file-like extension on the end of a rails 2.0 URL, such as “xml”, and “commit” contains the text of the form button, if the action was invoked via a form).
On a side note: If you’re interested in actually enforcing your params/protocol interface to your controllers in all environments (not just test), our company wrote a plugin called assert_request that allows you to do just that:
http://validaterequest.rubyforge.org/
Of course, our plugin serves a slightly different (more paranoid) purpose. It also has a performance penalty (unmeasured, as of yet) since it runs in production, and you have to declare which params are permitted at the beginning of your actions. Good stuff for writing a strict, robust application, but certainly not for everyone.
Thanks for the great post and idea. This will scratch a great itch for us, for those times when we don’t need the tight security of assert_request, but still want to know about orphaned or extra params. It seems like just about any Rails developer would benefit from including this in their test helper.
January 30, 2008 at 6:48 am
Scott,
Thanks for the kind words.
Regarding: “Do you run into problems with some of the extra params that rails inserts…”
As you pointed out, I added ‘action’ and ‘controller’. I’ve only used this technique when updating an existing, simple project and I didn’t run into any other rails added params but you are correct that these will have to be addressed.
Thanks again for checking in.
January 31, 2008 at 5:45 pm