Interesting Things

• svn log takes a long time when you have recently committed a large number such as 5000 files.
  Specifically, this happens when you have path based security. Here’s a link from the subversion documentation that mentions this

  All of this path-checking can sometimes be quite expensive, especially in the case of svn log. When retrieving a list revisions, the server looks at every changed path in each revision and checks it for readability. If an unreadable path is discovered, then it’s omitted from the list of the revision’s changed paths (normally seen with the –verbose option), and the whole log message is suppressed. Needless to say, this can be time-consuming on revisions that affect a large number of files

• Check out the new links at pivotallabs.com

  • Talks lists all the tech talks that happen here at Pivotal such as Blaine Cook’s Fire Eagle talk. If you have some cool technology and would like to give a talk, contact us at techtalks@pivotallabs.com
  • Who lists all the great people who work here

Interesting Things

• When defining an extension to an association, you can access the loaded association data through proxy_target. If the data hasn’t been preloaded/loaded when you call this method, it will return []. If you’d like to manually load the target, you can call load_target, and you can call loaded to determine if the proxy data has been loaded. For most situations, however, you can rely on the association to load itself when necessary by calling methods on self as follows:

has_many :people do
  def bad_people
    self.select {|person| person.bad? }
  end

  # exact same situation as 'bad_people', but 2x worse code
  def good_people
    load_target unless loaded?
    proxy_target.select {|person| !person.bad? }
  end
end

• There’s no good way to use CSV fixtures and has_and_belongs_to_many associations, in such a way that they are easily understandable and editable by non-technical people. Foxy fixtures solved a lot of issues with fixtures, but those advantages only work with YAML fixtures. Hence, if you have a HABTM situation, you’re stuck building a lot of rows of CSV referencing arbitrarily chosen IDs across several different files.

Interesting from yesterday

The difference between new-style-includes (rails 2.1+) and old-style-includes in rails is the size of the query. In the old style, rails selects all the data from all the tables in a single query, using some crazy renames that look like this:

SELECT users.id AS t1_r1, users.name AS t1_r2,
profiles.id AS t2_r1, ...
FROM users
LEFT OUTER JOIN profiles ON profiles.user_id = users.id

This can get really bad if you :include multiple has_many associations, because the number of rows multiplies rapidly! In the new-style-includes, ActiveRecord does one SELECT per table like so:

SELECT * FROM users
SELECT * FROM profiles WHERE user_id IN (1,2,3,4,5,6)

More queries, but each one returns a small number of rows, and overall is a big performance improvement. The problem comes when you add :conditions that reference tables you :include. The new style will attempt to write this query:

SELECT * FROM users WHERE profiles.gender = 'M'
# ERROR - no table profiles!

So, you can make all your includes faster as long as you don’t have any :conditions, :order, or :select clauses that select from tables other than the base finder table. In our case, we hardcoded this check to always use the new-style-includes, manually ensuring that we don’t fall into these failing situations.

Ask for Help

“Why does my JVM seg. fault when running SOLR?”

Virtual machines should never segmentation fault! It’s probably a JVM/OS/library issue, so check try a different version of the JVM and check that it has all it’s proper dependencies. Alternatively, try a different VM entirely.

“Is there a way in Excel to ‘reshape’ 2D data?”

If you have an NxM matrix in Excel, you can transpose it to a MxN matrix easily. But if you want to convert it to a (M/2)x(N*2) through a reshaping you’re probably on your own. You could open it in ruby and reshape the arrays that way…

Interesting Things

• If your HTTP header’s HTTP_CLIENT_IP is not equal to HTTP_X_FORWARDED_IP, then rails 2.1 and above will consider it an IP spoofing attack and throw an exception! This is bad news for some traditional Apache->Mongrel setups. Solution is probably to change the apache HTTP headers, but we’re wondering exactly why this is a security problem for rails (and why they would break compatibility with the default apache setup from way back when)?
• Be careful when using validates_uniquess_of with :case_sensitive => true AND a unique index at the database level. If your database is case insenitive, then rails will approve the uniqueness, but the database will fail the insert. Solution: be sure to use a collation type for the unique column that is case sensitive (such as binary in mysql).
• Rails 2.1+ :includes are way better than pre-2.1, but they are less compatible with conditions. Hence, rails falls back on the old style. Here’s when it might legitimately fall back:

User.find(:all, :include => :profile, :conditions => "profiles.gender = 'M'")

Because we reference the included table profiles in the :conditions, rails has no choice but to construct one giant query to fetch Users and their profiles, rather than a separate query. Here’s a case when it guesses wrong:

User.find(:all, :include => :profile,
  :joins => "INNER JOIN comments ON comments.user_id = users.id",
  :conditions => "comments.approved = 1")

Because the conditions references a table that is not users, rails thinks it has to fall back to the old include style… but it’s wrong! Here’s how we tricked ActiveRecord into always using rails 2.1+ includes (note that we had to fix up a few queries that were referencing :inlcuded tables in :conditions to make this work):

module ActiveRecord::Associations::ClassMethods
  private
  def references_eager_loaded_tables?(options)
    false
  end
end