Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker
Robbie Clutton

Stop leaky APIs

Robbie Clutton
Wednesday, May 15, 2013

There are many blogs about how to expose an API for a Rails application and many times I look at this and am concerned about how these examples often leak the application design and the schema out through the API. When this leak occurs a change to the application internals can ripple out and break clients of an API, or force applications to namespace URI paths which I feel is unnecessary and ugly.

When the only consumer of application data models are the views within the same application then the object design can be fluid and malleable. Once an application exposes an API to more than one client, and especially if that client is on a different release cycle to the server, such as iPhone application, data models become rigid. Rails tends discouraged N-tier architecture to the benefit of development speed but APIs are contracts between a server and it’s client and can be difficult to change once they start being used.

Passing an object into the Rails JSON serialisation methods will work for a time, but relying on this will only get you so far. At some point a refactor will take place that will cause a breaking change. It could be something simple such as renaming a column, moving responsibilities from one class to another or adding extra meta-data to a response. Either way, adding this information into your model class starts to place more responsibilities into one place.

There are a few ways out of this potential issue. Let’s take a look at the classic blog application and its Post object. The Rails rendering engine will call as_json on an object if the request has sent the content-type of application\json to the server. Here we override the implementation from ActiveRecord to provide a stable, known version:

def as_json(options={})
    {
        author_id: author.id
        title: title
    }
end

A second option is to model the object explicitly and serialise the internal model into a public representation. We can duck-type the object to respond how ActiveRecord objects behave during a serialisation call. Although this can be seen as a step towards a N-tier architecture, it’s also a step towards service dependent abstraction:

class Api::Post
  attr_reader :post

  def initialize(post)
    @post = post
  end

  def as_json(options={})
    {
      author_id: post.author.id
      title: post.title
    }
  end
end

The benefit of doing this is a separation of concerns between your data model and the data presentation. An application model doesn’t need to know how it’ll be represented by an API, command line interface or any other outside communication mechanism. If an application were tending more towards HATEOAS for instance this separation could help resolve hyperlinks relevant to the interface. You may lose some of the Rails respond_with goodness with this:

respond_to :html, :json

def show
  post = Post.find(params[:id])
  respond_to |format| do
    format.html { @post = post }
    format.json { render json: Api::Post.new(post) }
  end
end

That can be regained with the help of a presenter:

respond_to :html, :json

def show
  post = Post.find(params[:id])
  @presenter = PostPresenter.new(post)
  respond_with @presenter
end

Where PostPresenter may look something like:

class PostPresenter < SimpleDelegator
  def as_json(options={})
    Api::Post.new(self).as_json(options)
  end
end

What’s the difference between this and putting the as_json method into Post directly? More control, separation of concerns with application modeling vs presentation and the big win is when breaking changes occur within the API. Now we can put version relevant information into new objects, or into the serialised class itself.

class Api::Post
  attr_reader :post, :version

  def initialize(post, version)
    @post = post
    @version = version
  end

  def as_json(options={})
    send("v#{:version}")
  end

  private
  def v20130505
    # version specific JSON
  end

  def v20121206
    # version specific JSON
  end
end

Through this we have versioning information in one place and through a request parameter of something like v=20130506 the application can handle multiple versions in one object. For me, this ultimately removes URIs like /v1/posts, but why is that important? The URI is an identifier which points to a resource and having v1 or v2 in the URI muddies the fact that the two identifiers are pointing to the same resource. Using a request parameter, much like pagination is handled, means we can ask for a representation of that resource rather than having to specify different resources. Then we can do away with needing controllers such as Api::V1::PostsController and just deal with Api::PostsController or even just PostsController and deal with the versioning within the object instead of the URI path.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Robbie Clutton

Single resource REST Rails routes

Robbie Clutton
Tuesday, April 16, 2013

REST principles by default is a fantastic convention within Rails applications. The documentation for how to route HTTP requests are comprehensive and give examples about photo resources within an application. If you’ve got photo and tag as first class resources of your application, Rails has you covered. But what if you are building an application with a focus on one type of resource, do you really want /resource_type as a prefix to all of your application paths? I certainly don’t and I’ll show you how to remove that without diverging from Rails core strenghts.

For better or worse, I’m always conscience of making sure applications I’m involved in have Cool URIs and sometimes that does mean fighting the Rails conventions. However Rails routing is very flexible and can provide me with the application paths that make me happy.

Take Twitter as an example. Every user has their username as a top level path, so instead of having /users/robb1e, they simply have /robb1e. When dealing with an application where there is one core resource it can make a lot of sense to strip the resource prefix. This can be achieved through scopes in the routing configuration.

Your::Application.routes.draw do
  scope ":username" do
    get '', to: 'users#show'
  end
end

Gives you routes which look like

           GET  /:username(.:format)                users#show

If you wanted to see the followers and followees of that user, you have two options. Return to the default resource or use HTTP verb contraints. I’ll show you both.

Your::Application.routes.draw do
  scope ":username" do
    get '', to: 'users#show'
    resource :following, only: [:show]
    resource :followers, only: [:show]
  end
end

This adds the routes

following GET  /:username/following(.:format)      followings#show
followers GET  /:username/followers(.:format)      followers#show

Alternatively HTTP verb constrains can be used to achieve a similar result.

Your::Application.routes.draw do
  scope ":username" do
    get '', to: 'users#show'
    get '/following', to: 'user#following'
    get '/followers', to: 'user#followers'
  end
end

This gives the paths

          GET  /:username/following(.:format)      user#following
          GET  /:username/followers(.:format)      user#followers

If you are trending into paths unknown, you always have the safety of tests to help you out. Both Rails and RSpec have ways to test your application routes.

One gotcha which using the default resource routing removes is clashing paths. If you decide to build an admin page and want to put that at /admin, that needs to be in the routes config before the scoped block and if a user has given themselves the name of admin then you may be in for some fun.

So the next time a need arises for an unconventional route, check the documentation, it’s probably possible although almost always warants thinking about.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Access Control & Permissions in Rails

Pivotal Labs
Thursday, July 26, 2007

Access Control is a simple idea. We want company employees to be able to delete inappropriate content; but random Users cannot. Here I propose one way to implement Access Control that has the particular advantage of being very general, very concise, and unlikely to be violated. I call it RESTful Access Control.

What is RESTful Access Control?

When we model RESTfully the Application Interface (i.e., the HTTP interface), we construe Resources as responding to a simple set of Verbs: Get, Put, Post, Delete. To the extent that our entire application is designed this way, we can Control all Access to our Application with perfect granularity: as being a set of Permissions to perform these four Verbs on a corresponding Resource. Let’s start with an example.

Suppose a User can add Members to a Group. How do we determine if this or that User can create a Membership? This logic doesn’t belong in the controller — call me a prig, but Business Logic belongs in the Model. Anything else is unbecoming. Therefore, we have a Controller like:

class MembershipsController < ApplicationController
  before_filter :load_group

  def create
    membership = @group.memberships.build(params[:membership])
    raise SecurityTransgression unless current_user.can_create?(membership)
    ...
  end

  def load_group
    @group = Group.find(params[:group_id])
  end
end

Let’s implement the can_create? method. Despite the above Formulaic code, the User Class isn’t really the proper place for the Logic: different kinds of resources are likely to have different rules. Thus, we implement the Logic in the Model being created:

class Membership < ActiveRecord::Base
  def can_be_created_by?(user)
    ...
  end
end

I call the above “the Passive Voice.” I prefer, for various reasons, to use the “Active Voice”, and therefore we implement a simple Proxy in the User Class:

class User < ActiveRecord::Base
  def can_create?(resource)
    resource.can_be_created_by?(self)
  end
end

Of course, this “Active Voice” implementation requires that we follow the Null Object Pattern for Users who are not Logged In, otherwise we’ll produce Errors when we ask if a Logged Out User has Permission to do something:

class AnonymousUser < User
  ...
end

Why we Always Create An Instance Before Checking Permission

In our Controller Action above, we first build an instance of a Model in memory, then ask for Permission.

membership = @group.memberships.build(params[:membership])
raise SecurityTransgression unless current_user.can_create?(membership)

But why even create the Model in memory it shall not be persisted? Well, for some advanced Access Control Rules, we need all of the data available to an Instance in order to proceed. Suppose, for example, that only the Owner of a Group can create Memberships. The simplest way to enforce this rule is to reference the Group Owner in the can_be_created_by? method:

class Membership
  belongs_to :group

  def can_be_created_by?(user)
    group.owner == user
  end
end

Similarly, can_be_destroyed_by?, can_be_updated_by?, and can_be_read_by? all must be instance methods. This provides the maximum flexibility and a consistent Interface.

This is a robust Pattern for Access Control. It extends brilliantly to User Roles:

def can_be_destroyed_by?(user)
  user.admin? or ...
end

Even a per-user Access Control scheme where we grant individual Users permission to manipulate individual objects can elegantly fit in. Consider something like the following:

class Permission
  belongs_to :user
  belongs_to :resource, :polymorphic => true
end

class CreatePermission < Permission
end

class Membership
  has_many :create_permissions, :as => :resource

  def can_be_created_by?(user)
    create_permissions.find_by_user_id(user)
  end
end

Handling Access Control in the Views

Access Control has View consequences as well. For example, we shouldn’t show a destroy button if the User lacks permission to perform that Action. Given the above implementation, this is simple enough:

 <%= link_to_unless current_user.can_destroy?(resource), 'destroy', ... %>

Why Exceptions?

Above, I’ve proposed using Exceptions to handle Security Transgressions. (At Pivotal, we call these Exceptions SecurityTransgressions because it has a more romantic air than the banal AccessDenied). Normally, these Transgressions cannot happen: in your Templates you will not even display links to modify an object if the User lacks Permission. So Transgressions occur only when a User is fiddling about with nefarious intent. So we can handle misbehavior consistently across the site, by (for example) rendering a HTTP 403 Forbidden Response. Exceptions are a neat way to do this, because Rails gives us a systematic way to handle exceptions with rescue_action:

 class SecurityTransgression < StandardError; end

 def create
   raise SecurityTransgression unless ...
 end

class ApplicationController < ActionController::Base
  def rescue_action(e)
    case e
    when SecurityTransgression
      head :forbidden
    end
  end
end

Ensuring Programmers Remember to do Access Control And How to Avoid Over-Engineering

The biggest danger with Access Control rules is the possibility that a careless Programmer might forget to implement them. Having a consistent, terse pattern for Access Control goes some way towards mitigating this problem. When all Access Control is as simple as adding a can_create? check to your Actions, Access Control is unlikely to be overlooked.

The Second biggest danger in Access Control is over-engineering. Suppose we want to enforce Access Control logic. Rails gives us some tools to do this. Well, Rails gives us just enough rope to hang ourselves. There are some serious disadvantages to the approach I am going to propose now, but we’ll get to that shortly.

Rails has something called an Around Filter. An Around Filter, like a Before and After Filter, is a Filter applied to every Action. But this Around filter is like the Heavenly Union of the Before and After Filters: both great tasting and less filling, it can do work before and after invoking the Action.

class ApplicationController
  around_filter :ensure_permission_to_create, :only => :create

  def ensure_permission_to_create
    class_name = ModelsController.to_s.demodulize.gsub(/Controller$/,'').singularize
param_name = class_name.downcase
    instance_variable_name = "@#{class_name.downcase}"

    instance_variable_set instance_variable_name, class_name.constantize.new(params[param_name])
    if current_user.can_create?(instance_variable_get instance_variable_name)
      yield # the create acction is invoked here.
    else
      head :forbidden
    end
  end
end

Whew. In the above implementation, control is yielded to the Action only if the User has permission. The instance of the Model is also built for us automatically; so, our Formulaic create Action becomes:

def create
  if @model.save
    ...
  end
end

Very DRY, But this is, again, just enough rope to hang ourselves with. Our Controllers need to be named exactly right, our parameters without an inconsistency, and so forth. This is not a problem in the World of Forms, but in Reality it may require some coding contortions. The tricky metaprogramming and impertinent assumptions in ensure_permision... only get nastier when we want to make a Nested Resource, and build our Model using a proxy (as in the previous example: @group.memberships.build(params([:membership])). Finally, where we have Controllers that do not map directly to ActiveRecords, we would need to introduce a Model layer that responded to the Interface defined in ensure_permission... We often want to do this, but not always. Being forced to might suck. So I offer this design strategy with much caution.

The Exception based approach, while a bit more verbose, is battle-tested and works in Practice. In any case, pushing the logic to the Model with can_be_created_by? will certainly work, regardless of whether you take the Around Filter approach or the Exception-Based one.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

REST: Theory to practice

Pivotal Labs
Thursday, May 24, 2007

REST. What is it, and how can it be used to design better web applications?

A presentation at RailsConf did me a great service by first pointing out all the things REST is not. It isn’t CRUD. It isn’t pretty URLs. It is neither a protocol nor an architecture, but it can play a role in your implementation of all of the above. REST itself though, is less concrete than all of that. It is a theoretical framework, a way of thinking about designing distributed software systems. For me, the first step in absorbing its principles is to forget about the database and focus on the fundamentals. This article will start there, then drill down to show how these ideas can help organize the development of your Rails applications.

Noon: Rest From Work (After Millet) by Van Gogh

(Noon: Rest From Work (After Millet) by Vincent Van Gogh)

REST encourages a focus on resources. A resource is anything that can be named, and your system can have as many resources and corresponding names as you want. Conversely, there is a limited set of operations defined on those resources. Unlike objects in object-oriented programming languages, which support very diverse, rich interfaces, resources in a RESTful system are relatively uniform. So how can a sophisticated API be developed if REST requires a fixed and limited number of operations that resources can support? The answer: add more resources!

Lets play with an example. Say I’m writing a web application that has a collection of shapes that can be moved to new locations. In an object oriented program, I might have a move method defined on each shape, but in this RESTful API, I have assigned consistent semantics to a limited set of operations supported by the HTTP protocol. I can show GET, create POST, update PUT, and destroy DELETE my resources. Nowhere in the HTTP specification is the a MOVESHAPE method defined. A naiive approach to remedying this limitation is to shoehorn this operation into the protocol by abusing URLs.

http://againstgrain.com/shapes/move/1

I call this abuse because URL stands for uniform resource locator, and it’s very hard to see this imperative-style command encoding as much of a resource. This is an API decision that fights the nature of the protocol it uses. How can the API go with the flow?

There are many potential solutions. Let me outline two, the first very simple, the second more sophisticated.

The first solution is to recognize that a move operation is just a change to the location of an object. If we expose this location as a resource and allow it to be updated, we’ll implement movement in a natural way without contorting URLs to name things that aren’t actually resources:

So we combine one of the four standard operations:
Update, represented by an HTTP PUT

With a resource:
http://withgrain.com/shapes/1/location

Updating the location of the shape will naturally equate to moving it.

But what if we want the movement to be relative to the objects current position, so that the client can say that they want a shape to move 5 pixels up and 10 pixels to the right without needing to know the objects current position or do any computation? To solve that problem, we apply a technique I learned doing computational semantics: reification.

Reification means that we give solid form or objecthood to something formerly fleeting or ephemeral. Anything can be reified. The fact that I am named Nathan Sobo can be thought of as my NathanSoboness, which is an (albeit abstract) conceptual object. Here we’ll apply the technique in a more conservative fashion, and say that shapes are associated with a history of movements. This movement history is a collection, which is itself a resource.

http://withgrain.com/shapes/1/relative_movements

Now say we want to move the shape. We combine the above resource locator with one of our standard operations, create, implemented as an HTTP POST. By posting a new movement to a shapes history, we cause the shape to move.

Now we’re working with HTTP rather than around it.

So how does this transfer to the design of Rails applications? Embracing resource oriented application development means you’ll be writing more controllers with fewer, more consistent methods. Lets work through a potential implementation of the shape API in Rails. It will all start in the routes file, with map.resources…

Lets say we want to expose both the relative and absolute means of moving a shape. First we’ll start with a shapes resource.

map.resources :shapes

This represents the collection of all shapes in our system. It assumes the existince of a corresponding ShapesController and will set up a series of routes and url-generating methods to help reference the actions therein. Note the controller and its standard complement of methods below.

class ShapesController < ActionController::Base
  def index
  end

  def show
  end

  def create
  end

  def edit
  end

  def update
  end

  def destroy
  end
end

But with map.resources, the actions on the controller do not play a critical role in the url. They merely name the operations to which a given pairing of HTTP request method and URL will map. GETting /shapes will execute index. POSTing to /shapes will execute create. GETting /shapes/:id will execute show. PUTting to /shapes/:id will execute update, and DELETEing /shapes/:id will execute destroy. So even though there are 5 actions, there are really only two url patterns, one referencing the resource that is the collection of all shapes, and another referencing resources that are members of that collection. We can reference these URLs with automatically defined methods:

shapes_url ---> /shapes
shape_url(@square) or shape_url(@square.id) ---> /shapes/1

By pairing these with the correct HTTP method, we can access every operation we need.

Now lets add relative movement:

map.resources :shapes do |shape|
  shape.resources :relative_movements, :name_prefix => 'shape_'
end

class RelativeMovementController < ActionController::Base
  ...
end

This again assumes the existence of a RelativeMovementController with all of the standard methods defined on it. Except the resources supported by this controller are nested within shape resources, so the URL patterns look like this:

/shapes/1/relative_movements
/shapes/1/relative_movements/4

Because of the :name_prefix supplied (which will no longer be needed at some release of Rails in the future), we can refer to these URLs with helper methods that look like this:

shape_relative_movements_url(@triangle) ---> /shapes/1/relative_movements
shape_relative_movement_url(@triangle, 4) ---> /shapes/1/relative_movements/4

All of the same rules about HTTP method choice allow access to the RelativeMovementController‘s actions.

Now a cool twist: Singleton resources. Lets add the nested position resource to shapes.

map.resources :shapes do |shape|
  shape.resources :relative_movements, :name_prefix => 'shape_'
  shape.resource :position, :name_prefix => 'shape_'
end

And a corresponding controller, this time with a different complement of methods:

class PositionController < ActionController::Base
  def show
  end

  def edit
  end

  def update
  end
end

Because position is a singleton resource nested inside of shape, this controller is designed to deal with a single resource rather than a collection of them, so there is no need for an index action. The HTTP verb / URL combination mappings are also different. So PUTting to /shapes/1/position will invoke the update action.

None of these changes are Earth-shattering, but the simple act of focusing on resources is a force that will organize your application. Rather than growing a hodgepodge of actions on ever fattening controllers, you’ll instead create a greater number of controllers that are more circumscribed in their responsibilities.

What does this say about your model? Not much. I used to think that it was important to have a controller for every model object, and I no longer do. Controllers are responsible for supporting the exposure of resources to a remote API. This collection of resources is, in a sense, your remote client’s model. Whether your resources map precisely onto your underlying data model is your business. For example, you might expose resources that have no direct correspondence in the model layer. Or you might have model objects that you don’t choose to expose as resources.

But regardless, REST finally provides an organizing principle for the controller layer. Even if you don’t plan on exposing a RESTful API as a service, thinking in terms of resources will help you build more consistent applications and help you make fewer decisions.

I realize that this article has by no means covered every aspect of REST, but I hope it fills a gap that I felt as I was learning all of this.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (781)
  • rails (113)
  • testing (88)
  • ruby (83)
  • ruby on rails (70)
  • jobs (62)
  • javascript (55)
  • techtalk (44)
  • rspec (38)
  • ironblogger (32)
  • productivity (30)
  • activerecord (29)
  • gogaruco (29)
  • git (28)
  • nyc (27)
  • rubymine (26)
  • bloggerdome (23)
  • mobile (22)
  • process (21)
  • pivotal tracker (21)
  • cucumber (20)
  • design (19)
  • jasmine (19)
  • ios (18)
  • webos (17)
  • objective-c (17)
  • android (16)
  • tracker ecosystem (16)
  • palm (16)
  • "soft" ware (16)
  • fun (15)
  • ci (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • bdd (14)
  • gem (13)
  • css (13)
  • tdd (13)
  • selenium (12)
  • goruco (12)
  • bundler (12)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
  • mojo (10)
  • chef (10)
  • api (10)
Subscribe to rest Feed
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >