MVC in Rails and Thick Views

The default way of using ActionController and ERB in Rails is to pass models into your views, and let the views figure out what to render and where. There’s been some pushback in the community on this, people are talking about getting rid of “thick views” with logicless templates like Mustache. Mustache is awesome, but if thick views are the problem, we don’t need to go that far if we just change the way we think about controllers a little bit.

Thick views are a controller problem

Consider this ERB, using a loop instead of partials for illustration:

<%@posts.each do |post|%>
  <%=post.author.name%>
<%end%>

If we write our controller like this, the controller will do two queries:

@posts = Post.all(:include => :author)

If we write our controller like this, it will do only one query in the controller:

@posts = Post.all

And the view will do as many queries as there are posts. I’m sure you’ve seen code like this in projects you have worked on. In the standard Rails model, we pass ActiveRecord objects to our views, they might or might not be pre-populated, and thin views can be hard to tell from thick views.

We can explicitly test this by mocking Post and counting queries, but that is a lot of extra work, and you have to remember to do it everywhere.

APIs are unit testable

In contrast to standard controller tests, the result of a JSON API call can be declaratively described. The difference between

{:posts => [:id => 4, :title => "Jerusalem", :author_id => 33]}

and

{:posts => [:id => 4, :title => "Jerusalem", :author => {:name => "Josephus, :id => 33}]}

Is pretty clear, and easy to assert on. If we TDD our controller methods this way it protects us from a lot of the n-query problems in views.

API work comes too late

I’ve seen serveral teams experience pain because Rails does not make it easy to develop a JSON API alongside your HTTP controllers. Rails 3 is better than Rails 2 on this, but you still have to handle the cases separately, particularly as things get more complex.

Twitter is a good example of this. The HTML endpoint recently started consuming its own JSON API over HTTP for generating pages:

One of the most important architectural changes is that Twitter.com is now a client of our own API. It fetches data from the same endpoints that the mobile site, our apps for iPhone, iPad, Android, and every third-party application use. This shift allowed us to allocate more resources to the API team, generating over 40 patches. In the initial page load and every call from the client, all data is now fetched from a highly optimized JSON fragment cache.

This is a very cool pattern in general for a web application. Consuming your own API keeps your business logic in one place, separate from your display logic. And caching on JSON API calls can be easier than caching after building HTML. Obviously you don’t want to run two processes when you’re getting started, but we can take inspiration from this as we organize our code.

The Presenter pattern

A Presenter is a controller that delivers data rather than objects. People like Martin Fowler have been talking about this for a while, and Twitter recently announced a major technical change along these lines.

From the Wikipedia article on “Presenter First”, which is mostly aimed at desktop design, but is applicable to the web, too:

When used in GUI applications, this approach allows the presentation logic and business logic of the application to be developed in a test first manner decoupled from on-screen widgets. Thus, the vast majority of the application programming can be tested via unit tests in an automated test suite. In so doing, the reliance on GUI testing tools to perform extensive system testing can be reduced to verifying basic GUI operation or eliminated entirely.

Unlike standard Rails TDD, if you construct your page data as an API call, your controller tests can now assert directly on that data, instead of the objects that your templates can query with. Testing on a data dictionary is more declarative, and it helps you write thinner views.

Make your view a logical client of your API

We can use the Presenter pattern in Rails by having our controllers call the methods that generate our JSON API at the moment right before serialization to a string. That way there is only one code path that does business logic, which solves the divergence problem.

Vanna

Vanna is an exploration of how MVP can work in Rails. It’s not fully functional (in particular it needs a way to do non-200 returns), but it is enough to illustrate the point. Mixing Vanna into ActionController::Metal restructures controller flow to mimic JSON API calls. Your controllers wind up looking like this:

class VillainsController < ApplicationController
  def index
    {:main => {"villains" => Villain.all}}
  end

  def show(opts = params)
    villain = Villain.named(opts["villain"]).first
    sidebar = catchphrases("villains" => villain["partners"])
    {:main => {:villain =>villain, :sidebar => sidebar}}
  end

  def catchphrases(opts=params)
    names = opts["villains"]
    Villain.named(names).map{|p| p["catchphrase"]}
  end
end

How does this differ from a standard Rails controller? For one thing there’s a silly “catchphrases” method, which is a placeholder for an API method that gives a set of fields for a set of records:

def catchphrases(opts=params)
  names = opts["villains"]
  Villain.named(names).map{|p| p["catchphrase"]}
end

This is structured like an API call. It expects to take in a set of keys, and return some subset of fields on the objects with those keys. Exactly the sort of thing an API does, but not something that you usually see in web apps in their early stages. There’s also an explicit (opts=params) in the signature. Let’s take a look at the show method to see why that is there:

def show(opts = params)
  villain = Villain.named(opts["villain"]).first
  sidebar = catchphrases("villains" => villain["partners"])
  {:villain =>villain, :sidebar => sidebar}
end

On the second line of the method we are explicitly calling one method in a controller from another, passing a params hash (suggested by Richard Crowley). We construct a data dictionary for these smaller calls and pass it through to our template.

Here’s what the view looks like:

<div id=main style="width:70%;float:left;border:black 5px solid;">
  <ul>
    <li><%=villain[:name]%></li>
    <li><%=villain[:catchphrase]%></li>
  </ul>
</div>
<div id=sidebar style="width:20%;float:right;border:black 5px solid;">
  <% sidebar.each do |catchphrase| %>
    <%=catchphrase%> <br/>
  <% end %>
</div>

This looks almost exactly like a standard Rails ERB template, except that the top level objects available to you are accessed as locals, not with an @.

API for free

Vanna makes all your controller methods available as API calls automatically. Because you are explicitly returning a data dictionary, there’s no need to have a different code path for your HTML and JSON. The only difference is whether you go to ERB, or call to_json on the dictionary. So now we can do our controller tests as API tests (the tests in Vanna itself are a little different):

 def test_catchphrases
   header "Accept", 'application/json'
   get "/villains/catchphrases?villains=luis"
   assert{ JSON(last_response.body) == ["Hungry like the volcano!"] }
 end

So OK, we can call our catchphrases method as a JSON call.

def test_show_has_catchphrases
  header "Accept", 'application/json'
  get "/villains/show?villain=luis"
  assert{ JSON(last_response.body)["main"]["sidebar"] == ["You're gonna get punted!"] }
end

And the same data is available inside the larger call which includes it.

def test_html_has_sidebar
  get "/villains/show?villain=luis"
  assert{last_response.body.match(/div id=sidebar/) != nil }
end

And the template actually renders out the sidebar.

Now we have a single data retrieval code path, that breaks off into HTML right before we actually render HTML.

The code for Vanna is on Github, but here’s the meat of it:

module Vanna
  def self.included(klass)
    raise "#{klass.name} does not inherit from  ActionController::Metal" unless  klass.ancestors.include?(ActionController::Metal)
    klass.send(:include,  AbstractController::Layouts)
    klass.send(:include,  AbstractController::Callbacks)
    klass.append_view_path "app/views"
    klass.class_eval("def logger; ActionController::Base.logger; end;")
  end

  def process_action(method_name, *args)
    run_callbacks(:process_action, method_name) do    dictionary = send_action(method_name, *args)    dictionary = @layout_pieces.merge(dictionary) if @layout_pieces  && dictionary.is_a?(Hash)    self.response_body = request.format.symbol == :json ?
 dictionary.to_json : html_render(dictionary)
    end
  end
  def html_render(dictionary)
    render(nil, :locals => dictionary)
  end
end

It’s an example more that a real tool – it’s a demonstration of a valuable way to think about building web applications.

Try it out

It’s extremely rough still, but it serves pages. Here’s the setup:

Gemfile

gem 'vanna', :git => 'git://github.com/MikeSofaer/vanna.git'

ApplicationController

Change ‘ActionController::Base’ to ‘Vanna::Base’

layouts/application.html.erb

Remove the javascript_include line. (and if you know what to do so this isn’t necessary, tell me.)