I've been a C++ developer ever since I discovered the language in the early 90's and I realized that my beloved Pascal had nothing on objects. I've spent plenty of time working with other languages, of course, and over the past year or so I've written almost exclusively Ruby. But, one thing I've missed about C++ is the ease with which you can make objects act like functions.
In case your C++ is a little rusty, here's an example:
class Fibonacci {
public:
Fibonacci(): n1_(1), n2_(1) {}
int operator()() {
int result = n1_;
n1_ = n2_;
n2_ = result + n2_;
return result;
}
private:
int n1_, n2_;
};
What you're looking at there is an overload of the function call operator. No, I'm not kidding; that will compile and run. Instances of the Fibonacci class are called function objects, or functors.
Now, you're wondering to yourself why anyone would care. The answer is, this function can now carry state around with it:
Fibonacci fibonacci;
fibonacci(); // 1
fibonacci(); // 1
fibonacci(); // 2
Fibonacci()(); // 1
vector<int> v(5);
generate(v.begin(), v.end(), Fibonnaci()); // [1, 1, 2, 3, 5]
generate(v.begin(), v.end(), fibonnaci); // [3, 5, 8, 13, 21]
This may not seem particularly compelling for generating Fibonnaci numbers, but consider generators that may carry more complex state, or references to state owned by other objects. Consumers can also set initial state, such as a seed value for a random number generator, via the ctor.
Also, consider the generate method above. It expects a third parameter that supports function call semantics with arity of zero. And nothing else. That parameter could be a function pointer (should you desire statelessness and impenetrable syntax), a functor of any type, or anything else that supports operator (). That's duck-typing, my friends. In a statically-typed language. Dogs and cats sleeping together, and all that.
Again, you cry, why would anyone care? Well, blocks in Ruby carry around the state of the context in which they were created, but sometimes you want more. For instance, if you pass your Proc object around your code may be clearer with the state explicitly encapsulated. The initial state may simply not make sense as local variables when you create the Proc. You may want to save some secondary value that a consumer can query the functor for (how many Fibonacci numbers has this generator generated). Or, perhaps you want consumers to be able to mutate the state in some way.
In any case, this functor approach wacked me over the head recently while I was looking at some code that used the Rails Symbol#to_proc. We all know that Rails adds voodoo to symbols so that
User.find(:all).collect(&:name)
is equivalent to
User.find(:all).collect { |u| u.name }
And, we all know that this works because the & operator, when applied to an object in a parameter list, will implicitly call #to_proc on that object and then convert the result to a block. This is vanilla Ruby functionality, Rails just adds #to_proc to the Symbol class.
So, duh. Check it out:
class Fibonacci
def initialize
@n1 = @n2 = 1
end
def to_proc
@proc ||= Proc.new do
result = @n1
@n1, @n2 = @n2, @n1 + @n2
result
end
end
end
fibonacci = Fibonacci.new
fibonacci.call # 1
fibonacci.call # 1
fibonacci.call # 2
Fibonacci.new.call # 1
(1..5).collect(&Fibonacci.new) # [1, 1, 2, 3, 5]
(1..5).collect(&fibonacci) # [3, 5, 8, 13, 21]
Voilà, a functor. I'd love to hear from anyone who has used this technique to do something really cool.
I'm pleased to announce the 0.6.0 version of RR. The changes include:
- Declaring Double subject objects without having to pass it in via the mock!, stub!, dont_allow!, instance_of!, and proxy! methods
- Revised Double chaining API
- satisfy matcher
- hash_including matcher
Declaring Double Subjects (The bang methods)
In previous versions of RR, you always needed to pass in the subject of the double. For example:
subject = Object.new
mock(subject).does_something {:and_returns_me}
subject.does_something # :and_returns_me
Now you can have RR automatically create the subject object for you by using the ! method:
subject = mock!.does_something {:and_returns_me}.subject
subject.does_something # :and_returns_me
Now the bang methods by themselves don't really add a whole lot, but when used in the context of Double chaining, they become a powerful addition.
Double Chaining
Nick Kallen presented the use case for Double chaining and contributed a patch for the 0.5.0 release of RR. It has proved useful and is now more fully incorporated into RR. Now you can pass in your subject or use the subject provided by RR by using the ! method. Here are some examples of Double Chaining:
mock(subject).first(1) {mock(Object.new).second(2) {mock(Object.new).third(3) {4}}}
subject.first(1).second(2).third(3) # 4
mock(subject).first(1) {mock!.second(2) {mock!.third(3) {4}}}
subject.first(1).second(2).third(3) # 4
mock(subject).first(1).mock!.second(2).mock!.third(3) {4}
subject.first(1).second(2).third(3) # 4
Of course you have access to the proxy facilities:
mock.proxy(User).find('1').mock.proxy!.children.mock.proxy!.find_all_by_group_id(10)
User.find('1').children.find_all_by_group_id(10) # Makes verifications pass and returns the actual children
You can also do branched Double chaining:
mock(subject).first do
mock! do |expect|
expect.branch1.mock!.branch11 {11} # or expect.branch1 {mock!.branch11 {11}}
expect.branch2.mock!.branch22 {22} # or expect.branch2 {mock!.branch22 {22}}
end
end
o = subject.first
o.branch1.branch11 # 11
o.branch2.branch22 # 22
Satisfy Matcher
Matthew O'Conner submitted a patch that added the satisfy matcher. This adds the ability to add arbitrary argument expectation matchers.
mock(object).foobar(satisfy {|arg| arg.length == 2})
object.foobar("xy")
Hash Including Matcher
Matthew O'Conner also submitted a patch that added the hash_including matcher. This adds a convenient way to assert that the passed-in hash includes certain key/value pairs.
mock(object).foobar(hash_including(:red => "#FF0000", :blue => "#0000FF"))
object.foobar({:red => "#FF0000", :blue => "#0000FF", :green => "#00FF00"})
Mailing list
RR has a mailing lists at:
Also, RR's rubyforge page is at http://rubyforge.org/projects/double-ruby and of course the github page is at http://github.com/btakita/rr.
Yes, and there is more to come
There are many interesting ideas floating around. Joseph Wilk has been playing around with adding Spies into RR. I'm also thinking about adding Double validation scoping into RR. Also, I'm impressed by Mocha's warning of unused stubs. Josh Susser also proposed having a mode where a warning would occur if a mocked method is not implemented on the subject being mocked.
If you have any feature requests, please send an email to the mailing list or add it to the rubyforge tracker.
This Sunday I am co-presenting at toorcon. The talk that I am a part of is titled Owning telephone entry systems (aka why you shouldn't sleep so well), and will be presented by Josh Brashars and myself. If you're in the greater San Diego area and have time to kill at a security conference, stop by.
I'm pleased to introduce a new Test Double (or mock) framework named RR, which is short for Double Ruby.
Why a Double framework and not a Mock framework?
A mock is a type of test double. Since RR supports mocks, stubs, and proxies, it makes sense to refer to RR as a double framework. The proxy is a new usage pattern that I will introduce later in this article, and in more detail in future articles.
Unfortunately, the terminology over doubles has been contradictory depending on the framework. RR's terminology tries to be as faithful as possible to Gerald Meszaros' definition of test doubles. You can read more about test doubles in XUnit Test Patterns and Martin Fowler's article, Mocks aren't Stubs. Regretfully, this does mean that RR will have slightly different terminology than other double frameworks.
How does RR compare to other Mock frameworks?
Most double frameworks focus mainly on mocks (hence the categorization "mock framework"). RR's focus is on enabling more double test patterns in a terse and readable syntax.
RR also does not have dedicated mock objects. It primarily uses the technique called 'double injection'. Names that other frameworks use are 'stub injection', 'mock object injection', 'partial mocking', or 'stubbing'. The term I'll use for this is a double injection, since one or many doubles are being injected into an object's method.
I'll use trivial Rails examples to highlight the syntactical differences between RR, Mocha, Rspec's mocks, and Flexmock. They may or may not be appropriate situations for mocks. The right situations for mocks is an entirely different discussion.
If there is better way to do any of the examples, please post a comment and I will gladly replace it.
Mocks
Here are the ways to mock the User.find method. The expectation is the User class object will receive a call to #find with the argument '99' once and will return the object represented by the variable user.
RR
mock(User).find('99') { user }
Mocha
User.expects(:find).with('99').returns(user)
spec/mocks
User.should_receive(:find).with('99').and_return(user)
Flexmock
flexstub(User).should_receive(:find).with('99').and_return(user).once
Stubs
Here are the ways to stub the User.find method. When the User class object receives a call to find with the argument '99' it will return user1. When User receives find with any other arg, it returns user2.
RR
stub(User).find('99') { user1 }
stub(User).find { user2 }
Mocha
User.stubs(:find).with(anything).returns(2)
User.stubs(:find).with('99').returns(1)
spec/mocks
users = {
'99' => user1,
'default' => user2
}
User.stub!(:find).and_return do |id|
users[id] || users['default']
end
Flexmock
users = {
'99' => user1,
'default' => user2
}
flexstub(User).should_receive(:find).and_return do |id|
users[id] || users['default']
end
Proxy
A proxy used with a mock or stub causes the real method to be called. Expectations can be placed on the invocation and the return value can be intercepted. The main rationales are test clarity and you can ensure that the methods are being called correctly, even after you refactor your code. I will delve more into proxies and their usage patterns in my next article.
Mock Proxy
The following examples set an expectation that User.find('99') will be called once. The actual user is returned.
RR
mock.proxy(User).find('99')
Mocha
You cannot implement this in Mocha. You can do an approximation in this situation however. This technique is not always the solution you need, though.
user = User.find('99')
User.expects(:find).with('99').returns(user)
spec/mocks
find_method = User.method(:find)
User.should_receive(:find).with('99').and_return(&find_method)
Flexmock
find_method = User.method(:find)
User.should_receive(:find).with('99').and_return(&find_method)
Stub Proxy
The following examples intercept the return value of User.find('99') and stub out valid? to return false.
RR
stub.proxy(User).find('99') do |user|
stub(user).valid? {false}
user
end
Mocha
Again, this is an approximation, since you cannot use proxies in Mocha.
user = User.find('99')
user.stubs(:valid?).returns(false)
User.stubs(:find).with('99').returns(user)
spec/mocks
find_method = User.method(:find)
User.stub!(:find).with('99').and_return do |id|
user = find_method.call(id)
user.stub!(:valid?).and_return(false)
user
end
Flexmock
find_method = User.method(:find)
flexstub(User).should_receive(:find).with('99').and_return do |id|
user = find_method.call(id)
flexstub(user).should_receive(:valid?).and_return(false)
user
end
instance_of
instance_of is method sugar than allows you to mock or stub instances of a particular class. The following examples mock instances of User to expect valid? with no arguments to be called once and return false.
RR
mock.instance_of(User).valid? {false}
Mocha
User.any_instance.expects(:valid?).returns(false)
spec/mocks
new_method = User.method(:new)
User.stub!(:new).and_return do |*args|
user = new_method.call(*args)
user.should_receive(:valid?).and_return(false)
user
end
Flexmock
new_method = User.method(:new)
flexstub(User).should_receive(:new).and_return do |*args|
user = new_method.call(*args)
flexmock(user).should_receive(:valid?).and_return(false)
user
end
More to come
This concludes the introduction to RR. RR enables some techniques, like proxying, that will make your tests clearer and less brittle. In the next article I will describe into patterns and techniques that will make mocks a more feasible tool for more situations.
The Confreaks site has released a slew of videos from the 2007 RubyConf. The videos are in the perfect format: side by side streams with the projection material on one side and the stream of the speaker on the other.
Many great sessions are covered, including a good one on the Treetop parser presented by Pivotal Labs' own Nathan Sobo. As a bonus, in the audience you can spot other Pivots you may be familiar from this weblog (Nick Kallen and Brian Takita).
I have a few gems on Rubyforge and I want to track how many of them were downloaded. I found Firefox's search tools lacking to find my gem rr.
To fix this issue, I made a sake task, named gems:downloads:list, that prints the gem downloads in text.
The source is on caboo.se.
You can install it by using:
sudo gem install sake sake -i http://pastie.caboo.se/79547.txt gems:downloads:list sake gems:downloads:list | less
This will give an output like:
------------------------------------------------ | Gem | Downloads | ------------------------------------------------ | rails | 1194471 | | activerecord | 1121778 | | actionpack | 1054718 | | activesupport | 990851 | | actionmailer | 960759 | | actionwebservice | 948640 | | rake | 860824 | | mysql | 593476 | | fcgi | 230394 | | mongrel | 220370 | | daemons | 167443 | | rmagick | 164537 | | gem_plugin | 153505 | | RedCloth | 147182 | | rubygems-update | 119615 | | net-ssh | 114369 | | sqlite3-ruby | 105796 | | fastthread | 95534 | | cgi_multipart_eof_fix | 95399 | | needle | 87718 |
Sake is way cool. It was just too easy to implement and deploy this. Have fun making your own sake tasks.
The setup:
I'm told file uploading is a pain to test. We needed to. So we cruised through the tubes over to ruby-doc.org to check out the Net::HTTP rdoc -- only to find that Net:HTTP::Post does not support multipart uploading and files. What to do, what to DO?!?
The research:
Some googling later, we find this article showing how to do it. A little copy-paste, a small spike later, and we have an external script capable of uploading files into our web-apps. But, lets brain-storm a little...
- How can we make it better?
- What would be a nice interface?
Well, the first step is to change the script such that it can be more easily integrated into rake test:functionals: make it less script-y; more library. The interface is somewhat inspired by the basic_auth method. All you have to say is Net::HTTP::Post.new().multipart_params = {}? You give it a hash, and it takes care of the rest. Huzzah! So lets open up Net::HTTP::POST and give it some new methods. Time for some CODE!!!
The Code
require 'net/https'
require "rubygems"
require "mime/types"
require "base64"
require 'cgi'
class Net::HTTP::Post
def multipart_params=(param_hash={})
boundary_token = [Array.new(8) {rand(256)}].join
self.content_type = "multipart/form-data; boundary=#{boundary_token}"
boundary_marker = "--#{boundary_token}\r\n"
self.body = param_hash.map { |param_name, param_value|
boundary_marker + case param_value
when String
text_to_multipart(param_name, param_value)
when File
file_to_multipart(param_name, param_value)
end
}.join('') + "--#{boundary_token}--\r\n"
end
protected
def file_to_multipart(key,file)
filename = File.basename(file.path)
mime_types = MIME::Types.of(filename)
mime_type = mime_types.empty? ? "application/octet-stream" : mime_types.first.content_type
part = %Q|Content-Disposition: form-data; name="#{key}"; filename="#{filename}"\r\n|
part += "Content-Transfer-Encoding: binary\r\n"
part += "Content-Type: #{mime_type}\r\n\r\n#{file.read}"
end
def text_to_multipart(key,value)
"Content-Disposition: form-data; name=\"#{key}\"\r\n\r\n#{value}\r\n"
end
end
Oh the utility:
Now that's more like it. Hackish, since you have to stick headers into the request body, but effective. Notice the bit in there about MIME::Types. Did you see that? Yeah, we went there. Say it with me... Automatic mime type detection with a safe default. The absurd thing in there is that the MIME::Types gem (as of today) does not know about .rb files.
irb(main):007:0> MIME::Types.of('something.rb')
=> []
So now that you have that, it's just a simple use of Net::HTTP with a blizzock to upload a file in a functional test.
File.open(File.expand_path('script/test.png'), 'r') do |file|
http = Net::HTTP.new('localhost', 3000)
begin
http.start do |http|
request = Net::HTTP::Post.new('/your/url/here')
request.basic_auth 'lonely_user', 'really_long_password'
request.multipart_params = {'file' => file, 'title' => 'title'}
response = http.request(request)
response.value
puts response.body
end
rescue Net::HTTPServerException => e
p e
end
end
The questions:
So what do you think? How can this be made even better?
We all like a good oxymoron, like redefining constants. There are times where we need to redefine a constant to test an edge case in the application code. Before I go into this example, please note that redefining constants is generally not a good way to have maintainable software. If you find yourself needing to redefine a constant, it may be an indication that refactoring is needed.
Given that, lets get into an example where you may need to redefine a constant. Lets say an app has does file uploads to Amazon's S3 service. A common practice to upload to a real S3 account made for the production, development, or demo environment.
When in the test environment, a fake S3 service would be used instead. The fake service is useful to keep your tests fast and running predictably.
To get a different File Upload service object in each of your environments, one can have the S3 configuration in the environment files:
test.rb
STORAGE_SERVICE = FakeStorageService.new
development.rb
STORAGE_SERVICE = S3StorageService.new("development_service", "access_key", "secret_access_key")
production.rb
STORAGE_SERVICE = S3StorageService.new("production_service", "access_key", "secret_access_key")
The File Upload service objects can be set to constants in the environment file. This works great when testing the logic of the objects that use the File Upload service. However it is a good idea to run an integration test that does a real upload.
Since the tests are running in the test environment, a fake File Upload service is being used. Well now we want to use a real service that points to a test S3 account. An easy trick is to redefine the constant to the S3 service in setup and then redefine the constant back to the fake service on teardown.
There are a few ways of doing this...
Just Reset the Constant
context "A real S3 call" do
setup do
STORAGE_SERVICE = S3StorageService.new("test_service", "access_key", "secret_access_key")
end
teardown do
STORAGE_SERVICE = FakeStorageService.new
end
end
This is the simplest approach, but it produces an error:
warning: already initialized constant STORAGE_SERVICE###Use silence_warnings
context "A real S3 call" do
setup do
silence_warnings do
STORAGE_SERVICE = S3StorageService.new("test_service", "access_key", "secret_access_key")
end
end
teardown do
silence_warnings do
STORAGE_SERVICE = FakeStorageService.new
end
end
end
This solution removes the warning, but now a certain section of your code will not have warning at all. Also, one could argue that you lose semantic meaning. It also feels like a hack.
Redefine the Constant
class Module
def redefine_const(name, value)
__send__(:remove_const, name) if const_defined?(name)
const_set(name, value)
end
end
context "A real S3 call" do
setup do
Object.redefine_const(
:STORAGE_SERVICE,
S3StorageService.new("test_service", "access_key", "secret_access_key")
)
end
teardown do
Object.redefine_const(
:STORAGE_SERVICE,
STORAGE_SERVICE = FakeStorageService.new
)
end
end
Calling redefining the constant does not generate a warning. Also it does provide semantic value because you are actively declaring that you are redefining the constant. If there are other warnings, you will also see them.
Its all Dirty
Redefining constants is a non-standard tatic, especially for those new to Ruby. Since this is unconventional and is often contrary to assumptions, it may lead to unpredictable behavior.
Maybe the storage service can be an attribute that can be changed for individual tests.
I was just sitting around my living room listening to NPR, and heard the following Car Talk puzzler:
I want you to get a pencil and write down the numbers, 1 - 9, inclusive, and leave enough space between them. At your disposal you have one plus sign and two minus signs. You can insert those plus and minus signs wherever you want, to make the total come out to 100.
Naturally, I thought, "Gee, that would be tedious to solve it by hand. But it would be fun to write a Ruby program to solve it!" 9 minutes later I was sending the result (and the source code) to Car Talk Plaza.
So here's your challenge: can you write a program to solve this puzzle? And can you beat my time?
My solution is below the fold... don't click "more" until you've taken a stab at it yourself.
As the righteous wave of Intel iMacs surges into the Pivotal Labs offices, more Pivots are finding themselves working with multiple OS X Terminal windows. The opening and positioning of terminal windows often follows the same pattern: cd into project directory, run mongrel, open next window, cd into project directory, tail the test log, etc.. To avoid violating DRY, I've hacked up some simple ruby scripts that automate the process. See my original post for details and links to the scripts.
Now it's just a matter of running one command:
$ terminals.rb myproject
This opens up all of the standard windows from the project in their specified positions and running the right processes.