Ask for Help

"We keep getting webrat thread exceptions running our integration specs with the rails integration runner: Thread tried to join itself. The error message varies with different versions of ruby 1.8.6 vs 1.8.7."

Anyone had this problem or know why?

"How do I skin an iphone mobile site to be the correct width so it's not 980px wide?"

<meta name="viewport" content="width = device width" />

*"We're trying to deploy some nginx configuration changes to EngineYard Cloud, what's the right way to do that?"

We've tried building custom chef recipes to solve this problem, but they run after nginx has already restarted, so are a poor solution to this problem. The better solution might be to check in configuration files into the application and symlink them into the nginx configuration directory using a before_symlink.rb hook in the /deploy directory.

*"We've got a has_many association where some of the child records are originally saved in an invalid state. When we later load the parent and ask it if it's valid, it returns true even with validates_associated. How can we get the desired validation behavior?"

Turns out that unloaded associations are not validated. Solution: load the association before calling .valid? on the parent. In general, you should also not create invalid objects, instead using a state variable to put them into a "draft" or "incomplete" state where they are still valid but not complete. Then remove that state and you'll see the errors required to finish that object.

Interesting Things

• When RubyMine 2.0.1 won't run your focused specs, try attaching rspec 1.2.9 to it rather than 1.3.x. It fixed this issue for one of our teams.
• Rubymine 2.0.2 came out today: can finally run focused contexts?! Also including bundler support! What's new
• We tried our Unicorn on EngineYard cloud: so far so good. It's still "experimental" but seems to work.

Interesting Things

• Be careful when extending classes from structs. Their superclasses are essentially anonymous classes, so reopening them can be difficult. If you attempt to reopen them by extending them from "the same" struct, it will actually be a different anonymous class.

• Ever had an STI model but wanted the views and controllers to pretend like it all extended the base class? You can have rails change the params[] namespace that it uses like so:

  form_for :user, @admin_user, :url => user_path(@admin_user)

Or you can be super-cool and use polymorphic routes:

form_for @admin_user.becomes(User)

The becomes method is part of ActiveRecord, and it actually creates a 2nd copy of the object with the same attributes and a different class (shallow copy). Due to this implementation, it has limitations so use it carefully.

• Upgrading to rails 2.3.3 breaks HopToad. This is related to filter_parameter_logging, and it's technically rails' fault. It has been fixed in 2.3.3 stable (which I assume will be released as 2.3.4). You can also fix it yourself with a one-line-patch. Personally, I'd wait to upgrade till 2.3.4 comes out.

Interesting Things

• One of our projects is using mechanical turk to match and normalize their ugly data. After building some automatic matching with little success, the group is simply using the template web interface. The task can be trivially decomposed into tiny tasks, and accuracy does not need to be 100% perfect, so it's a great fit. They only spent a few hours and have results already!

Interesting Things

• pgrep is a sweet tool for finding processes. You can find all your mongrels, for example, without having the problems of running ps aux | grep ruby. It's in the proctools package on most linux/unix operating systems. For example, on osx use sudo port install proctools
• kill -482 kills all processes in the group 482. This is great for killing all the children of a daemon like mysql or backgroundrb. ps shows the process group id next to the process id.
• monit doesn't have great support for figuring out what happens when a start/stop command runs. It can fail silently, for example. One (bad) way of debugging this is to add echo to dump debugging info to a temporary file before and after these commands. Rumor has it that god doesn't have these problems...
• we've heard a rumor that Marshal.dump(object) uses a temporary file on disk! This would be slower than it needed to be. Perhaps this is to deal with dumping objects to large to fit in RAM?

Interesting things

• When we used localhost in our solr.yml configuration, we couldn't run tests on our OSX 10.5.6 machines. Commenting out the IPv6 localhost entries in /etc/hosts fixed the problem. The better solution would probably be to use 127.0.0.1 in SOLR configuration.

Interesting Things

• When you call user.purchases.sum(), you are invoking ActiveRecord::ClassMethods#sum rather than Enumerable#sum. If you want to invoke Enumerable#sum (which takes a block and is more powerful though less performant), you'd have to call user.purchases.target.sum() {|p| p.price * p.quantity}.
• NewRelic sometimes makes our app servers malfunction. Several of us reported having these sorts of problems on different projects. It's always fixed by the NewRelic team with a new version or a configuration change, but we wish that we felt safer about our production server stability. Some projects feel that the value is certainly worth it, and Engineyard uses NewRelic data when discussing scaling, so it's worth hanging in there.

Ask for Help

• What's the deal with using the OSX terminal and bash/readline messing with the terminal? We're always typing some ridiculously long command and bash starts writing over itself. Especially when we use Ctrl+A and Ctrl+R and edit the line. Anyone know how we can stop/fix this once it happeneds?

Interesting Things

• Neat Plugin: Caio Chassot suggested a patch to rails that makes rails template finder traverse the controller inheritance chain when looking for templates. This would make the view system work "correctly" with inheritance, which one of our projects needed. The patch wasn't applied, but the code was released as a plugin called inheritable_templates, which we are now using and enjoying.
• What's the opposite of {:a => 1, :b => 2}.to_a? It's Hash[:a, 1, :b, 2].
• Rails 2.3 is going to be awesome! We're most looking forward to
  • Nested model assignment and views
  • Nested transactions, even on MySQL!
  • Default Scopes, no more adding :order => "position" on every acts_as_list model
  • Smarter rendering of partials
  • Rack support
  • Bringing of Engines back. Pivotal is still going to support Desert at this time. Desert is similar to engines, but loads every class that matches in the load path, not just the first one. This allows you to build plugins that extend previous plugins. Using engines, however, we are hoping to make the source code for desert even more trivial.

Interesting Things

• When defining an extension to an association, you can access the loaded association data through proxy_target. If the data hasn't been preloaded/loaded when you call this method, it will return []. If you'd like to manually load the target, you can call load_target, and you can call loaded to determine if the proxy data has been loaded. For most situations, however, you can rely on the association to load itself when necessary by calling methods on self as follows:

has_many :people do
  def bad_people
    self.select {|person| person.bad? }
  end

  # exact same situation as 'bad_people', but 2x worse code
  def good_people
    load_target unless loaded?
    proxy_target.select {|person| !person.bad? }
  end
end

• There's no good way to use CSV fixtures and has_and_belongs_to_many associations, in such a way that they are easily understandable and editable by non-technical people. Foxy fixtures solved a lot of issues with fixtures, but those advantages only work with YAML fixtures. Hence, if you have a HABTM situation, you're stuck building a lot of rows of CSV referencing arbitrarily chosen IDs across several different files.

Interesting from yesterday

The difference between new-style-includes (rails 2.1+) and old-style-includes in rails is the size of the query. In the old style, rails selects all the data from all the tables in a single query, using some crazy renames that look like this:

SELECT users.id AS t1_r1, users.name AS t1_r2,
profiles.id AS t2_r1, ...
FROM users
LEFT OUTER JOIN profiles ON profiles.user_id = users.id

This can get really bad if you :include multiple has_many associations, because the number of rows multiplies rapidly! In the new-style-includes, ActiveRecord does one SELECT per table like so:

SELECT * FROM users
SELECT * FROM profiles WHERE user_id IN (1,2,3,4,5,6)

More queries, but each one returns a small number of rows, and overall is a big performance improvement. The problem comes when you add :conditions that reference tables you :include. The new style will attempt to write this query:

SELECT * FROM users WHERE profiles.gender = 'M'
# ERROR - no table profiles!

So, you can make all your includes faster as long as you don't have any :conditions, :order, or :select clauses that select from tables other than the base finder table. In our case, we hardcoded this check to always use the new-style-includes, manually ensuring that we don't fall into these failing situations.

Ask for Help

"Why does my JVM seg. fault when running SOLR?"

Virtual machines should never segmentation fault! It's probably a JVM/OS/library issue, so check try a different version of the JVM and check that it has all it's proper dependencies. Alternatively, try a different VM entirely.

"Is there a way in Excel to 'reshape' 2D data?"

If you have an NxM matrix in Excel, you can transpose it to a MxN matrix easily. But if you want to convert it to a (M/2)x(N*2) through a reshaping you're probably on your own. You could open it in ruby and reshape the arrays that way...

Interesting Things

• If your HTTP header's HTTP_CLIENT_IP is not equal to HTTP_X_FORWARDED_IP, then rails 2.1 and above will consider it an IP spoofing attack and throw an exception! This is bad news for some traditional Apache->Mongrel setups. Solution is probably to change the apache HTTP headers, but we're wondering exactly why this is a security problem for rails (and why they would break compatibility with the default apache setup from way back when)?
• Be careful when using validates_uniquess_of with :case_sensitive => true AND a unique index at the database level. If your database is case insenitive, then rails will approve the uniqueness, but the database will fail the insert. Solution: be sure to use a collation type for the unique column that is case sensitive (such as binary in mysql).
• Rails 2.1+ :includes are way better than pre-2.1, but they are less compatible with conditions. Hence, rails falls back on the old style. Here's when it might legitimately fall back:

User.find(:all, :include => :profile, :conditions => "profiles.gender = 'M'")

Because we reference the included table profiles in the :conditions, rails has no choice but to construct one giant query to fetch Users and their profiles, rather than a separate query. Here's a case when it guesses wrong:

User.find(:all, :include => :profile, 
  :joins => "INNER JOIN comments ON comments.user_id = users.id",
  :conditions => "comments.approved = 1")

Because the conditions references a table that is not users, rails thinks it has to fall back to the old include style... but it's wrong! Here's how we tricked ActiveRecord into always using rails 2.1+ includes (note that we had to fix up a few queries that were referencing :inlcuded tables in :conditions to make this work):

module ActiveRecord::Associations::ClassMethods
  private
  def references_eager_loaded_tables?(options)
    false
  end
end