Interesting Things

• Neat Plugin: Caio Chassot suggested a patch to rails that makes rails template finder traverse the controller inheritance chain when looking for templates. This would make the view system work "correctly" with inheritance, which one of our projects needed. The patch wasn't applied, but the code was released as a plugin called inheritable_templates, which we are now using and enjoying.
• What's the opposite of {:a => 1, :b => 2}.to_a? It's Hash[:a, 1, :b, 2].
• Rails 2.3 is going to be awesome! We're most looking forward to
  • Nested model assignment and views
  • Nested transactions, even on MySQL!
  • Default Scopes, no more adding :order => "position" on every acts_as_list model
  • Smarter rendering of partials
  • Rack support
  • Bringing of Engines back. Pivotal is still going to support Desert at this time. Desert is similar to engines, but loads every class that matches in the load path, not just the first one. This allows you to build plugins that extend previous plugins. Using engines, however, we are hoping to make the source code for desert even more trivial.

In order to accomplish some advanced search functionality, we've added a lot of named_scopes to our User model. This seems like a good idea, and well within the intended use for named_scopes. Unfortunately, we ran into issues with our :joins. We have a separate User and Profile model, but our advanced search scopes often needed both to make decisions. So we had some scopes that look like this:

class User
  named_scope :verified {
    :conditions => {:email_verified => true}
  }

  named_scope :answered_questions {
    :join => "INNER JOIN profiles ON profiles.user_id = users.id " +
                 "INNER JOIN answers ON answers.profile_id = profiles.id"
  }

  named_scope :with_name { lambda { |name| 
    :join => "INNER JOIN profiles ON profiles.user_id = users.id",
    :conditions => ["profiles.name LIKE ?", "%#{name}%"]
  } }
end

Using these named_scopes, we wanted to dynamically construct a finder that would return the results the user was interested, such as: User.verified or User.answered_questions or even User.verified.answered_questions.with_name('Joseph'). The last scope caused issues, unfortunately, with table aliasing. The query ended up joining in the profiles table twice, in exactly the same way without renaming the table, so mysql rejects the query.

The easiest solution to this problem was to use only the hash form for :join clauses, such as :join => :profile. Rails correctly merges multiple consecutive join scopes that use hashes. If you need to use string joins (such as a LEFT JOIN rather than an INNER JOIN) or put a condition directly on your join, then merging goes out the window and the hashed form is immediately converted to a string and all consecutive joins are "merged" by appending them together.

We started by manually aliasing our scopes, but in some cases we were concerned about the amount of duplicate data this was causing in our queries.

We thought about creating a dependency framework for named_scopes, such that you could have a single :profile scope that other scopes were dependent on and it would only ever get added once. This seemed really difficult because of the way the with_scopes are constructed by named_scopes, there was no good place to keep track of these dependencies, and it would still cause problems if you had a manual with_scope, or :join in your find.

Finally we decided that rails fundamentally lacked the capability to deal with duplicate joins, and that we should solve this problem. It seemed a good solution was to allow :join options to take an array of strings as follows:

  named_scope :answered_questions {
    :join => ["INNER JOIN profiles ON profiles.user_id = users.id",
                 "INNER JOIN answers ON answers.profile_id = profiles.id"]
  }

Now calling User.answered_questions.with_name('Joseph') will create three values in a :join array, two of which are identical and will be uniq'd out. The downside to this approach is that each value in the :join array has to be string identical, or it will not be properly uniq'd.

So if you are mixing hash style :profile joins with string joins of the same table you need to be careful you match the rails generated syntax. We mostly use string style joins to avoid this issue.

Here's the ticket the we filed and patched: 1077-chaining-scopes-with-duplicate-joins-causes-alias-problem

It has been commited and will roll out with rails 2.2. Since then we have filed two more issues related to :join and :include:

• 1078-using-include-assoc-and-join-assoc-leads-to-alias-issue
• 1104-references_eager_loaded_tables-should-search-tables-in-join-clauses

We hope to patch these two as well!

Joseph & David

Interesting Things

• When defining an extension to an association, you can access the loaded association data through proxy_target. If the data hasn't been preloaded/loaded when you call this method, it will return []. If you'd like to manually load the target, you can call load_target, and you can call loaded to determine if the proxy data has been loaded. For most situations, however, you can rely on the association to load itself when necessary by calling methods on self as follows:

has_many :people do
  def bad_people
    self.select {|person| person.bad? }
  end

  # exact same situation as 'bad_people', but 2x worse code
  def good_people
    load_target unless loaded?
    proxy_target.select {|person| !person.bad? }
  end
end

• There's no good way to use CSV fixtures and has_and_belongs_to_many associations, in such a way that they are easily understandable and editable by non-technical people. Foxy fixtures solved a lot of issues with fixtures, but those advantages only work with YAML fixtures. Hence, if you have a HABTM situation, you're stuck building a lot of rows of CSV referencing arbitrarily chosen IDs across several different files.

Interesting from yesterday

The difference between new-style-includes (rails 2.1+) and old-style-includes in rails is the size of the query. In the old style, rails selects all the data from all the tables in a single query, using some crazy renames that look like this:

SELECT users.id AS t1_r1, users.name AS t1_r2,
profiles.id AS t2_r1, ...
FROM users
LEFT OUTER JOIN profiles ON profiles.user_id = users.id

This can get really bad if you :include multiple has_many associations, because the number of rows multiplies rapidly! In the new-style-includes, ActiveRecord does one SELECT per table like so:

SELECT * FROM users
SELECT * FROM profiles WHERE user_id IN (1,2,3,4,5,6)

More queries, but each one returns a small number of rows, and overall is a big performance improvement. The problem comes when you add :conditions that reference tables you :include. The new style will attempt to write this query:

SELECT * FROM users WHERE profiles.gender = 'M'
# ERROR - no table profiles!

So, you can make all your includes faster as long as you don't have any :conditions, :order, or :select clauses that select from tables other than the base finder table. In our case, we hardcoded this check to always use the new-style-includes, manually ensuring that we don't fall into these failing situations.

Ask for Help

"Why does my JVM seg. fault when running SOLR?"

Virtual machines should never segmentation fault! It's probably a JVM/OS/library issue, so check try a different version of the JVM and check that it has all it's proper dependencies. Alternatively, try a different VM entirely.

"Is there a way in Excel to 'reshape' 2D data?"

If you have an NxM matrix in Excel, you can transpose it to a MxN matrix easily. But if you want to convert it to a (M/2)x(N*2) through a reshaping you're probably on your own. You could open it in ruby and reshape the arrays that way...

Interesting Things

• If your HTTP header's HTTP_CLIENT_IP is not equal to HTTP_X_FORWARDED_IP, then rails 2.1 and above will consider it an IP spoofing attack and throw an exception! This is bad news for some traditional Apache->Mongrel setups. Solution is probably to change the apache HTTP headers, but we're wondering exactly why this is a security problem for rails (and why they would break compatibility with the default apache setup from way back when)?
• Be careful when using validates_uniquess_of with :case_sensitive => true AND a unique index at the database level. If your database is case insenitive, then rails will approve the uniqueness, but the database will fail the insert. Solution: be sure to use a collation type for the unique column that is case sensitive (such as binary in mysql).
• Rails 2.1+ :includes are way better than pre-2.1, but they are less compatible with conditions. Hence, rails falls back on the old style. Here's when it might legitimately fall back:

User.find(:all, :include => :profile, :conditions => "profiles.gender = 'M'")

Because we reference the included table profiles in the :conditions, rails has no choice but to construct one giant query to fetch Users and their profiles, rather than a separate query. Here's a case when it guesses wrong:

User.find(:all, :include => :profile, 
  :joins => "INNER JOIN comments ON comments.user_id = users.id",
  :conditions => "comments.approved = 1")

Because the conditions references a table that is not users, rails thinks it has to fall back to the old include style... but it's wrong! Here's how we tricked ActiveRecord into always using rails 2.1+ includes (note that we had to fix up a few queries that were referencing :inlcuded tables in :conditions to make this work):

module ActiveRecord::Associations::ClassMethods
  private
  def references_eager_loaded_tables?(options)
    false
  end
end

Interesting Things

• Google caches all favicons, and will serve them to you as PNG files! The url is simply google.com/s2/favicons?domain=xxx. Here's tracker's favicon: . We recently used this to update our social bookmarking plugin to include new icons for facebook, myspace, digg, etc.