Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker

Monthly Archives: November 2007

rake query_trace

Alex Chaffee
Saturday, November 17, 2007

QueryTrace is a great Rails plugin (which I learned about from ErrTheBlog) for pinpointing where in your Rails application that slow query is running. Once you have it installed, your logs won’t just tell you that you have a problem, they will pinpoint the exact location of that problem for you. This is invaluable when doing load & performance testing or just trying to understand what the hell ActiveRecord is doing.

Unfortunately, even though it only logs query traces in DEBUG log mode, it still clutters up your test and development logs, and actually slows things down a bit too. So you don’t want to leave it in your project’s vendor/plugins directory after you’re done using it. So I wrote a pair of rake tasks to enable and disable it.

rake query_trace:on                     # Enables the query_trace plugin. Must restart server to take effect.
rake query_trace:off                    # Disables the query_trace plugin. Must restart server to take effect.

The “on” task actually checks out the plugin from query_trace’s subversion repository, makes a tarball in vendor/query_cache.tar.gz, then leaves the tarball around so it doesn’t have to keep going to the network all the time. You can even check the tarball in to your project and it’ll never go to the query_trace repository again. The “off” task just removes the whole vendor/plugins/query_cache directory.

namespace :query_trace do
  desc "Enables the query_trace plugin. Must restart server to take effect."
  task :on => :environment do
    unless File.exist?("#{RAILS_ROOT}/vendor/query_trace.tar.gz")
      Dir.chdir("#{RAILS_ROOT}/vendor") do
        url = "https://terralien.devguard.com/svn/projects/plugins/query_trace"
        puts "Loading query_trace from #{url}..."
        system "svn co #{url} query_trace"
        system "tar zcf query_trace.tar.gz --exclude=.svn query_trace"
        FileUtils.rm_rf("query_trace")
      end
    end
    Dir.chdir("#{RAILS_ROOT}/vendor/plugins") do
      system "tar zxf ../query_trace.tar.gz query_trace"
    end
    puts "QueryTrace plugin enabled. Must restart server to take effect."
  end

  desc "Disables the query_trace plugin. Must restart server to take effect."
  task :off => :environment do
    FileUtils.rm_rf("#{RAILS_ROOT}/vendor/plugins/query_trace")
    puts "QueryTrace plugin disabled. Must restart server to take effect."
  end
end
  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Installing Freeimage/ImageScience on OS X 10.5 Leopard

Alex Chaffee
Wednesday, November 7, 2007

Gleaned these instructions from a rubyonrails-talk thread.
Another thing is mysql is a bit funky,
you can’t use the built in control panel, but following these
instructions
makes it work.

Full installation instructions (full credit goes to: Michael Steinfeld for figuring this out)

  1. I started with a clean install of Leopard.
  2. Install macports for 10.4
  3. Install the xcode dev tools from the Leopard disk – *be sure to
    also install the 10.3 sdk from the xcode dev tools install*
  4. sudo port install freeimage
  5. cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_graphics_freeimage/work/FreeImage

    and change this:

    LIBRARIES_PPC = -Wl,-syslibroot /Developer/SDKs/MacOSX10.3.9.sdk
    LIBRARIES_I386 = -Wl,-syslibroot /Developer/SDKs/MacOSX10.4u.sdk

    to this

    LIBRARIES_PPC = -Wl,-syslibroot /Developer/SDKs/MacOSX10.3.9.sdk/usr/lib
    LIBRARIES_I386 = -Wl,-syslibroot /Developer/SDKs/MacOSX10.4u.sdk/usr/lib

  6. sudo port install freeimage

  7. sudo gem install -y imagescience
  8. cd /Library/Ruby/Gems/1.8/gems/RubyInline-3.6.4/lib
  9. edit inline.rb

    look for the line

    flags = @flags.join(’ ‘)

    and change it to

    flags = @flags.join(’ ‘) + ‘ -lruby’

  10. remove ~/.ruby_inline

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Bush Violates Standup Rules

Alex Chaffee
Sunday, November 4, 2007

Courtesy of Steve C and Alex Tabarrok

There is no "W" in "team"

There is no “W” in “team”

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Making Rails Wicked Fast: Pagecaching Highly Personalized Web Pages

Pivotal Labs
Sunday, November 4, 2007

Consider the following snippet for a page showing blog articles. Notice how content on the page differs based on who is viewing it:

<% if current_user.nil? %>
  You are logged out
<% elsif current_user.admin? %>
  You are an admin
<% elsif @article.author == current_user %>
  You are the author of this blog article
<% end %>

Pagecaching such a page is difficult because all of this conditional logic would need to be translated to Javascript. The appropriate data (whether the user is logged in, etc.) needs to be available to the client–usually this is stored in a cookie or comes from an Ajax request (presumably the Ajax request is much faster than having Rails generate the entire page).

While we can translate this conditional logic to Javascript, a much simpler approach is to use CSS:

<style>
  .logged_out, .admin, .author { display: none; }
  body.logged_out .logged_out { display: block; }
  body.admin .admin
</style>
<body class="">
  <div class="logged_out">
    You are logged out
  </div>
  <div class="admin">
    You are an admin
  </div>
  <div class="author">
    You are the author...
  </div>
</body>

By default, anything of class admin, logged_out, etc. is invisible. But simply by adding a class to the body tag, we can “unlock” these hidden parts of the page:

<body class="admin author">
</body>

And voila! both the admin and author sections are visible to the end user.

Implementation Details

So how do we add classes to the page? And where do we get the appropriate data for the end user? Use Javascript to add classes to the body tag:

for(var i = 0; i < classes.length; i++) {
  $$('body').first().addClassName(classes[i]);
}

Data then comes from one of three places.

Constant Data about the Current User

Set constant data about the current user at the start of a session (for example, we know whether the person is logged in and we know whether she is an admin):

class ApplicationController < ActionController::Base
  after_filter :set_classes
  def set_classes
    cookies[:user_classes] = current_user.classes
  end
end
class User
  def classes
    [admin?? 'admin' : 'not_admin', ...]
  end
end

Personalized Data

Set data about the current user’s relationship to the presently displayed content (for example, whether she the author of the article she is currently looking at) using an Ajax request.

new Ajax.Request('#{url_for(:format => :js)}', {
  method: 'post',
  onComplete: function(transport, json) {
    if (json.classes) {
      for(var i = 0; i < json.classes.length; i++) {
        $$('body').first().addClassName(json.classes[i]);
      }
    }
  }
}

class ArticlesController < ApplicationController
  caches_page :show
  def show
    @article = Article.find(params[:id])
    respond_to do |format|
      format.html {}
      format.js do
        headers['X-JSON'] = @article.to_json_for(current_user)
      end
    end
  end
end

class Article < ActiveRecord::Base
  def to_json_for(user)
    {
      :classes => [
          [user == author ? :is_author : :is_not_author
          ...
          ]
    }.to_json
  end
end

A couple gotchas here. First, You must override Rails pagecaching functionality to ensure it doesn’t cache requests for Json. Put this in ApplicationController:

def self.caches_page(*actions)
  return unless perform_caching
  actions.each do |action|
    class_eval "after_filter { |c| c.cache_page if c.action_name == '#{action}' && c.params[:format] != 'js' }"
  end
end

(Some details are missing in the above implementation since respond_to seems to delete the :format parameter from the params hash.)

Second, since we’re making an Ajax request, we are hitting the Rails stack; nevertheless, this is still wicked fast because the Ajax request returns only that data that is essential–very few objects need be instantiated. Also, you can almost always avoid making this Ajax request for logged out users, which should take enormous load off the server.

Stateful Session Data

Some data related to the current user is not constant–it lasts only for some finite part of the session. Still it should persist longer than just the current page. An example is flash[:error] content, but many sophisticated web sites utilize this kind of personalization extensively (think wizards and contextual help). The easiest way to populate this data is as part of the Ajax request but rather than return it in the Json, return it in a cookie.

class ArticlesController < ApplicationController
  caches_page :show
  def show
    @article = Article.find(params[:id])
    respond_to do |format|
      format.html {}
      format.js do
        cookies[:temporary_classes] += ...
      end
    end
  end
end

You may want to remove these “temporary” classes from the cookie as you use them on the client side.

CSS is Easy to Use

The reason this technique works is that CSS selectors permit the ability to do complex Boolean expressions. Though far from Turing Complete, CSS is powerful enough to express and and or, and it expresses it in an elegant way. It’s far easier and more elegant than translating your conditional logic to inlined Javascript.

Disadvantages

There are a few downsides to this approach. One is security. Because we render content for all possible people into the page, there is a potential security violation. Though no sensitive data appears in the browser, it is visible in the source code. For most applications, these security concerns are unimportant because the “real” security rules are enforced during write operations. But your mileage may very.

Some Cache Design Principles

There are a few principles to bear in mind when implementing a caching strategy.

  • Distinguish data a) independent of the current user (e.g., the title of an article) and data b) dependent on the current user.
  • In the latter category, distinguish data b1) concerning the current user only (e.g., whether the user is an admin) and data b2) concerning the relationship between the current user and some other object (e.g., whether the user is the author of an article).
  • Data concerning the relationship between the current user and some other object (b2) can usually be segmented into axes of variability; that is, we can reduce the “space” of the data from the number of users, to some smaller set of criteria. For example, we can usually categorize the kinds of relationships into is_author, is_not_author, is_friend, is_not_friend, etc.

Data of type (a) and (b2) are usually pre-rendered into the pagecached page. Data of type (b2) is then shown and hidden using the CSS technique. Data of type (b1) is usually set into the cookie when the user logs in, and alternatively any time the user hits the Rails app.

Designing Cachable URLs

It is very important to know when to use and when to avoid putting a user id in your url. For example, if you model the current user’s profile page with the following url:

http://www.mysite.com/profile

You’ve effectively made that page uncachable, since all of the content on that page depends upon the current user. A better URL would be:

http://www.mysite.com/profiles/nick

If my profile looks differently to me than others, model that as data of type (b2), that is data concerning the relationship between the current user and some other object.

That said, never put the current user into the url:

http://www.mysite.com/users/nick/articles/1

If this isn’t an article written by Nick (data of type a), but is rather Article 1 as seen by Nick (data of type b2), you’ve just pagecached a page with a cache hit ratio of zero. So consider carefully how you model a resource like the following:

http://www.mysite.com/account/password vs. http://www.mysite.com/users/nick/account/password

The latter format ensures that the password page has a zero cache-hit ratio, so screw that. Furthermore, a page that links to the latter url (for example, an “Account Settings” link in the site’s header) that happens to be cached must now generate that link using Javascript since the url differs based upon the current user. To reiterate this point:

Ensure that any pages that concern only the current user do not have the user identifier in the url. Examples include: the logged-in home page, account settings page, edit my profile page, my message inbox page, etc.

Hope you guys find these techniques helpful. Along with a little bit of scriptaculous templates, this technique will make it easy to make your highly personalized Rails apps scale massively.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (780)
  • rails (113)
  • testing (88)
  • ruby (83)
  • ruby on rails (70)
  • jobs (62)
  • javascript (55)
  • techtalk (44)
  • rspec (38)
  • ironblogger (32)
  • productivity (30)
  • activerecord (29)
  • gogaruco (29)
  • git (28)
  • nyc (27)
  • rubymine (26)
  • bloggerdome (23)
  • mobile (22)
  • process (21)
  • pivotal tracker (20)
  • cucumber (20)
  • jasmine (19)
  • design (18)
  • ios (18)
  • webos (17)
  • objective-c (17)
  • android (16)
  • palm (16)
  • "soft" ware (16)
  • fun (15)
  • tracker ecosystem (15)
  • ci (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • bdd (14)
  • gem (13)
  • css (13)
  • tdd (13)
  • selenium (12)
  • goruco (12)
  • bundler (12)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
  • mojo (10)
  • chef (10)
  • api (10)
Subscribe to Community Feed
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >