If you are using Bundler to lock and package your gem dependencies in your app (which you should), here's some tips on making everything automatic in your Capistrano deploy.

Refer to the Bundler Documentation for instructions on how to use Bundler to properly package your gems and check everything in.

Once this is done, however, you still must ensure that two things are done on every machine to which you will deploy:

1. Bundler is installed
2. You run 'bundle install' on every deploy to install the packaged gems on the local machine (and compile any gems with native dependencies)

Here's the Capistrano magic to accomplish these two tasks automatically on every deploy:

before "deploy:bundle_install", "deploy:install_bundler"
after "deploy:update_code", "deploy:bundle_install"

namespace :deploy do
  desc "installs Bundler if it is not already installed"
  task :install_bundler, :roles => :app do
    sudo "sh -c 'if [ -z `which bundle` ]; then echo Installing Bundler; sudo gem install bundler; fi'"
  end

  desc "run 'bundle install' to install Bundler's packaged gems for the current deploy"
  task :bundle_install, :roles => :app do
    run "cd #{release_path} && bundle install"
  end
end

Oh, and for you GemInstaller users out there - here's an easy way to generate a Bundler Gemfile from your geminstaller.yml config:

geminstaller --bundler-export > Gemfile

You'll probably still need some tweaks, but this will get you started. Just make sure you upgrade to GemInstaller 0.5.5 first (0.5.4 forgot to put the 'source' line in the Gemfile).

Happy Bundling! -- Chad

P.S. There is a similar article here, which includes tasks to symlink your .bundle dir into the Capistrano shared directory, but my deploy was pretty fast anyway, so I didn't worry about it. YMMV.

This fixes several bugs that people have complained about for quite a while. Please let me know if anything is broken.

GemInstaller 0.5.3 has been released!

GemInstaller

• by Chad Woolley
• http://geminstaller.rubyforge.org

CHANGES

• 0.5.3 / 2009-08-25
• Many long overdue bugfixes and patches, see http://tinyurl.com/geminstaller-0-5-3-release for details.
• Thanks to Greg Fitzgerald, Britt Crawford, John Trupiano, Gabriel Gironda, and Eric Hodel for patches and assistance.
• Issues with case statement under Ruby 1.9
• GemInstaller cannot distinguish between gems that have the ame name but capitalized differently.
• add ./ci as default location for config file
• Disable GemInstaller install in default rails preinitializer.rb, but fork if it is used
• autogem() fails when run for newly-installed gem
• Sometimes installing fails due to RubyGems cache not being cleared between multiple API calls

DESCRIPTION

Automated Gem installation, activation, and much more!

FEATURES

GemInstaller provides automated installation, loading and activation of RubyGems. It uses a simple YAML config file to:

• Automatically install the correct versions of all required gems wherever your app runs.
• Automatically ensure installed gems and versions are consistent across multiple applications, machines, platforms, and environments
• Automatically activate correct versions of gems on the ruby load path when your app runs ('require_gem'/'gem')
• Automatically reinstall missing dependency gems (built in to RubyGems > 1.0)
• Automatically detect correct platform to install for multi-platform gems (built in to RubyGems > 1.0)
• Print YAML for "rogue gems" which are not specified in the current config, to easily bootstrap your config file, or find gems that were manually installed without GemInstaller.
• Allow for common configs to be reused across projects or environments by supporting multiple config files, including common config file snippets, and defaults with overrides.
• Allow for dynamic selection of gems, versions, and platforms to be used based on environment vars or any other logic.
• Avoid the "works on demo, breaks on production" syndrome
• Find lost socks.

Quick Start

See http://geminstaller.rubyforge.org/documentation/index.html

INSTALL

• [sudo] gem install geminstaller

In a recent thread on the East Bay Ruby Meetup list, several people chimed in with Ruby IDE suggestions. I suggested RubyMine, which we use at Pivotal. Several people mentioned NetBeans and Aptana RadRails, so I decided to have a little contest.

Now, if I am going to work in an IDE and sacrifice the speed of a text editor, I want to see it work for me ('cuz RubyMine can chew through all your CPU and RAM and then some faster than you can say Moore's Law). That means understanding Ruby, and using that information to save me some significant thinking and work.

I don't mean code-generation macros or dumb context-aware keyword-completion, I mean something useful like knowing where my classes and methods are. In my book, that leaves out Emacs, Vi, and even TextMate, regardless of their other merits (sorry people, I like text editors too, but I'm making up this test - flame away, the comments section is below).

So, here's the smackdown scenario:

• Download the latest NetBeans, RadRails and RubyMine.
• Open a Ruby project, this example is one of mine (I wrote this three years ago to learn Ruby, so don't make fun of me for doing dumb stuff...)
• Test the ability of the IDE to navigate through Ruby language constructs. This should be easy, it is a command-line app using a Dependency Injection architecture, no metaprogramming curveballs!
  • Open the root Ruby class for the project (lib/geminstaller.rb)
  • Pick a variable (app in self.install)
  • Try to work back to the class declaration (GemInstaller::Application) using IDE navigation (Command-click in all the IDEs, although sometimes they ignore you)

Result? RubyMine succeeded, NetBeans and RadRails failed miserably. Here's what happened in each:

NetBeans

1. Click create_application class method reference in app = create_application. NetBeans takes me to the method declaration in same class.
2. Click the app method in registry.app.

FAIL! ANGRY BEEP! app is an attr_accessor on another class, NetBeans can't find it.

RadRails

1. Click create_application class method reference in app = create_application. RadRails takes me to the method declaration in same class.
2. Click app method in registry.app. RadRails takes me to the attr_accessor on the Registry class (without a prompt, and highlighting the symbol, which is even better than RubyMine).
3. Click (and F3) on the :app symbol argument to attr_accessor.

FAIL! ANGRY BEEP! RadRails can't figure out the symbol parameter to attr_accessor. It says "Current text selection does not resolve to a Ruby element".

RubyMine

1. Click create_application class method reference in app = create_application. RubyMine takes me to the method declaration in same class.
2. Click app method in registry.app. RubyMine pops up a dialog asking if I mean the attr_accessor on the Registry class, or the local variable I'm declaring. That's rather silly, I admit, but the point is it followed the reference to another class.
3. Click on the attr_accessor choice. RubyMine takes me to the attr_accessor line in Registry.
4. Click on the :app symbol argument to attr_accessor. RubyMine takes me to the point where the @app instance variable is initialized.
5. Click on the Application class name in the GemInstaller::Application.new constructor invocation. RubyMine pops up a dialog asking if I mean the Application class in my application, or two other Application classes that happen to be in my Ruby installation. This is also a silly question - it should have known the correct choice because of my namespacing, but it still found the class.
6. Click on the GemInstaller:Application choice. RubyMine takes me to the class declaration.

SUCCESS! RubyMine drilled all the way to the class declaration, even through an attr_accessor, albeit with a couple of stupid questions.

Summary

I personally think this is a Big Deal. In the past, I've mocked Ruby IDE functionality as a poor simulacrum of the vast power in Java IDEs. When using Eclipse in Java, I could perform epic refactorings, extracting superclasses and adding parameters to method signatures; refactoring scores of classes across multiple projects in a few mighty keystrokes. Yes, I'm fully aware that this is because Ruby is a dynamic language, but that doesn't make me miss a real refactoring IDE any less, and others have long lamented these shortcomings, as well.

So, for years, even though I'd always indulge my pairs if they wanted to use an IDE, I've done all my personal hacking with a fast, lightweight text editor and command line tools. To me, the benefits of a memory- and processor-sucking IDE with tons of unnecessary, unconfigurable, resource-eating tiny-ass-fonts and chrome did not justify giving up the speed and responsiveness of a great text editor.

However, RubyMine can now navigate code for me. I don't have to think and manually find that class, RubyMine knows where it is. Granted, that ain't no Extract Superclass, but it saves me a lot of thought and time, both of which are increasingly rare commodities for me.

To be fair, this is really just a problem related to parsing and interpreting attr_accessor declarations, and I expect that NetBeans and RadRails will pass this test as well in another release or two. That's all great news, because it means that Ruby IDEs are finally, slowly, coming of age. I think I'll still be waiting a long time for an automated Modify Method Signature refactoring, though...

From the Too Useful Not to Blog Department:

open_gem from Adam Sanderson is a new RubyGems Plugin to automatically open a gem's source in your favorite $EDITOR.

gem update --system
sudo gem install open_gem
export EDITOR=mate
gem open rails

NOTE: If you have RubyGems 1.1 or 1.2, 'gem update --system' may not work. See the RubyGems Release Notes for more info.

By popular demand, we're working on making the Pivotal Tracker API ActiveResource-compliant.

However, there are some quirks that are required to make ActiveResource happy. For example, when you are doing a 'create' or 'update' request, ActiveResource wants the response location to point to the 'show' URL for the new or updated record. For example, here's an ActiveResource 'create' call:

new_story = Story.create(
  :name => "New Story", 
  :requested_by => "Dan",
  :description => "Make API ActiveResource compliant")

On the controller, you must add the :location option to the render - you can't redirect:

render  :xml => xml, 
        :location => service_project_story_url(service_id, project_id, @story),
        :status => status

...otherwise, you get this helpful error from ActiveResource:

/Library/Ruby/Gems/1.8/gems/activeresource-2.2.2/lib/active_resource/base.rb:1006
        :in `id_from_response': undefined method `[]' for nil:NilClass (NoMethodError)
        from /Library/Ruby/Gems/1.8/gems/activeresource-2.2.2/lib/active_resource/base.rb:993:in `create'

This is the type of error which you will only catch through end-to-end testing with a real ActiveResource client hitting the running app. When I did the initial spike to see what problems we would run into, I wrote a simple manual script to run against the local development environment, hacking my way to a point which didn't blow up and I could visually inspect the output:

#!/usr/bin/env ruby
require 'rubygems'
require 'activeresource'
require 'pp'
class Story < ActiveResource::Base
  self.site = "http://localhost:3000/services/v1/projects/1"
  headers['TOKEN'] = '6cfc2055d1df5605241759014b06b232'
end

p "========================== Stories#create ====================================="
new_story = Story.create(:name => "New Story", :requested_by => "Dan", :description => "Make API ActiveResource compliant")
pp new_story

# etc for all other supported API actions...

However, now that we are doing the real non-spike implementation, we want to automate this end-to-end integration testing as part of our Continuous Integration. That way, we'll ensure that we are fully ActiveResource-compliant (against current and future versions), and that we don't have any inadvertent regressions due to future API bugfixes/enhancements.

Digging through the internets and rubyonrails-talk list archives turns up some discussion, but no good answers:

• Thoughtbot blog post on ActiveResource and Testing
• rubyonrails-talk post on "Testing XML over HTTP in a Rails app"
• rubyonrails-talk post on "Testing ActiveResource models with HttpMock"

All of these mention using ActiveResource::HttpMock. However, as Eric and Xavier point out, there seem to be drawbacks to this approach. Plus, even if we get it to work, I'm worried the usage of HttpMock might mask some other issues related to authentication handling, or who knows what else. That's what real integration tests are for. Finally, HttpMock is an undocumented internal method that seems to exist in order to support Rails' test suite, so it's probably not a great idea to depend on that long term.

So, we don't have a great answer yet, but it seems clear that the highest-value, least-risk approach is to hit a real running app over HTTP with a real ActiveResource client.

The current plan is to leverage our existing Selenium RC test environment, which already has support for spinning up and managing a Rails server with the test environment. We can then port the manual spike tests above to automated ones which run as part of the selenium suite under Continuous Integration, and add appropriate assertions. This isn't optimal, though, because they won't actually use Selenium RC at all, which may confuse people. However, there's no sense reinventing the wheel (and adding time to the overall CI build) by spinning up a separate test server instance when we already spin one up for our selenium suite.

Let us know if you have any clever solutions.

-- Chad

I just released cinabox. It is intended to be The Simplest Thing That Could Possibly Work to set up a Continuous Integration (CI) server, using cruisecontrolrb (CCRB).

Watch the Screencast!

In addition to being a (hopefully) useful tool to help people easily set up CI systems for various platforms and languages, it is also an experiment in simplicity and minimalism:

• The project consists of only two simple scripts, one shell script to bootstrap ruby, and one ruby script to set up cruisecontrolrb.
• In the script, readability and simplicity are favored over clever abstractions and DRYness. Hopefully, even people who don't know shell scripting or Ruby can read the scripts and easily understand the commands it is executing.
• A standardized environment is assumed: A dedicated Ubuntu 8.04 system, Ruby 1.8.6, and latest dependencies via aptitude. PCs and Virtual Machines are cheap, and Linux and CCRB are free. There's really no reason you shouldn't be able to run a dedicated CI box. If this environment doesn't work for you for some reason, the scripts should be self-explanatory enough that you can easily hack it up to work for your environment (and contribute your version back to to the project!).
• I use the magic fairy dust of GitHub to eliminate build scripts, release scripts, packaging, versions, and pretty much all the regular boring overhead of a project. The README.txt is my only documentation. The GitHub "Download Tarball" link automatically provides packaging and uniquely-named packages (by the git hex commit id) for each "release".

I'm pretty pleased with how this turned out. I hope it will lower the barrier for people to start trying out Continuous Integration, as well as provoke some thought about simplicity and minimalism. I've tried it out on a few flavors of Ubuntu VMs and my personal box, and it works for me. Please let me know what you think, and feel free to offer any suggestions for improvement.

I just had a discussion with a co-Pivot about the resentment that many teams develop about Continuous Integration - especially when the release process requires a green tag from CI, and a broken build is standing in the way.

As anyone who has worked with me will attest, I'm hardcore on CI and consider any team which leaves a build red for longer than a workday to be sorely lacking in discipline.

OK, OK, there are always extenuating circumstances, but I still believe that most resentment of CI stems from underlying antipatterns and smells, rather than problems with CI itself. For example:

• "The Customer Has To See This Feature RIGHT NOW": Frequent releases are a great thing, but if you cannot wait for a green build to deploy, you have some deeper problems. Often, this is because a team doesn't manage customer expectations well. The customer should understand that CI is a critical part of the Agile process which ensures that only reliable, quality releases get pushed to staging or production. Any problem which is preventing a green build should be fixed before the release is deployed. If the customer is not willing to allow you that time and flexibility, perhaps they are too addicted to new features, and the entire team needs to have a heart-to-heart about Code Debt in the next retrospective.
• "It Works For Me, But Fails On CI: The important question is which environment is more like production - your development environment or CI? If you are developing on Windows or Mac, and your production box is some other flavor of Xnix, then your CI box should be as close a possible to production. Ideally, you should be able to log on to the CI instance and debug the failing test there. Usually, your CI box is not configured correctly. If it is hard to keep your CI environment in sync with production, then perhaps you should look into automation (because you KNOW you or your sysadmin will probably forget to do the same thing when you push to production, right?). If the problem is that your development environment is not the same as production, and it is a legitimate problem, then CI just saved you some stress on the next deploy.
• "Intermittent" Failures: Same deal as the prior point. CI runs your tests much more than you do. For web apps, it hopefully runs them in more browsers than you do. In my experience, many "intermittent" bugs are real bugs which are just very hard to isolate. It could be an AJAX bug that only happens when the site is run remotely, not via localhost. It could be a performance problem which only shows up on a slower system, not your fastest-on-the-market dev box. It could be a dependency on an external resource that happens to be unavailable sometimes, such as a web service, remote storage, etc. Again, just being aware of these issues puts you ahead of the game. For browser bugs, dig in and find out WHY it is failing intermittently. It may be a real bug. For intermittent outages of external resources, you may just have to live with it, but you don't have to live with the intermittent failures in CI. Mock out the resource or disable the tests in the CI environment. Yes, this is OK, especially if you leave them enabled in your development environment. Another option is to automatically repeat these tests a few times with a delay, and only fail the entire build if they fail repeatedly. Big services like Amazon or Google might drop a request occasionally, but still respond to a subsequent request.
• Slow Test Suites: This is an insidious problem, because once your suite is slow, it is often a monumental effort to make it fast again. It is much better to be proactive, and monitor any slow-running tests like a hawk, relentlessly mocking out slow resources or replacing broad functional tests with faster, more targeted unit tests. You can also always split your tests into different suites, running your fastest tests continuously, and the entire slow deploy-test suite only nightly or periodically. As long as your customer isn't addicted to immediate features, it should be fine to only deploy from nightly builds.
• The Failing Test That "Doesn't Matter": This is my pet peeve. Whenever I break CI, I fix it ASAP. If I ignore a "minor" broken test, the next thing I check in may be a major FUBAR which gets past my local tests for some reason (see prior points). Some who know me might even say it is LIKELY to be a major FUBAR. The point is, I don't trust myself or my local box, I trust CI. Now, if ANOTHER developer breaks the build, and tries to tells me they are not going to fix it because it is a "minor" problem, that really chaps my hide. They are ripping huge holes in my nice safety net, forcing me to expend much more time and attention on the tests that I run on my local environment, and causing me more stress and work in general. Stop making excuses, and fix the damn build NOW, or comment out the failing test.

Now, I'm sure that all of the above points can be debated or shown to be inapplicable in a specific situation. Plus, if you are dealing with imperfect CI and development tools (which is always the case), you will have some degree of pain which is directly attributable to CI. It would be great to hear about some of these situations in the comments.

Bottom Line: Integration is always one of the most painful parts of software development. Doing integration with high quality and low risk is even harder. Most developers who have been on a non-Agile project of any significant size have experienced days-long integration hell and ulcer-inducing all-night production deployments. Continuous Integration doesn't make that pain and stress go away, but it does break it down into small, bite-sized pieces that can be easily handled on a daily basis. All for the low, low cost of being proactive and disciplined, which makes you a better developer anyway.

This question came up at standup. I've put some thought into it, so I thought I'd throw it out for discussion and see what other people think.

At Pivotal, we use svn:externals and the Third-Party Branch Pattern (from the SCM Patterns Book) to manage easy, reliable cross-project updates of common plugins which live elsewhere (rubyforge, etc), without having to manually update in every project or be at the mercy of the RubyForge svn repo going down or being slow. When everything was on SVN, this worked well; we had some rake tasks which made it easy to update the branch from the latest trunk version in the external vendor repository. However, with more stuff like Desert is moving to GitHub.

One option is to just check the entire Git repo into subversion. This is labor-intensive, though. It also loses some of the benefits of the Third-Party branch pattern, such being able to easily preserve local patches while still pulling in changes from a new vendor version. We could probably update our Rake tasks to handle Git as well as SVN.

Another option is to switch all our projects to Git, but we aren't quite ready for that across the board.

Plus, this is a problem even if you move your parent project to Git. For example, there is no easy Git equivalent to having a svn:external which automatically updates to the latest version of an external repository, which is a useful approach for Continuous Integration or automatically pulling the latest Edge Rails into your project on every update.

Here's some links from people working on related issues, but please comment if you have any ideas or something that works well for you:

• git-svn-dcommit-workaround-for-git-submodules
• using-git-submodules-to-track-vendorrails
• Nazar's git-rails-plugins

I just attended (and thoroughly enjoyed) the Mountain West Ruby Conf, where Evan Phoenix gave a powerful keynote speech.

He talks about the status of Rubinius, and makes some profound observations on modern open source culture and community.

Here's some highlights, but if you participate in open source, and especially if you help run an open source project, I highly recommend that you watch the video:

• Community: Rubinius' Giant Spec Suite, and its value not only as a living language specification for different implementations of Ruby itself, but as a "gateway drug" which provides a low barrier to get new contributors addicted to open source.
• Trust: Asking for forgiveness vs. permission, and the Rubinius commit policy, where any accepted patch gets you commit rights. You can always roll back a change, and debate is healthy.
• Worth: The impact of annoying fifteen year olds who make a lot noise versus "core" contributors.
• Ego: You are not the project, Mr. Ego! The importance of being wrong and admitting your faults in public.
• Innovation: Fostering innovation and debate vs closely holding the mythical "Keys to the Castle".

Many people (including me) have complained about the lack of a good GUI debugger for Ruby. Now that some are finally getting usable, I've found I actually prefer IRB-style ruby-debug to a GUI.

There's good tutorial links on the ruby-debug homepage, and a very good Cheat sheet, but I wanted to give a bare-bones HOWTO to help you get immediately productive with ruby-debug.

Install the latest gem

$ gem install ruby-debug

Install the cheatsheet

$ gem install cheat
$ cheat rdebug

Set autolist, autoeval, and autoreload as defaults

$ vi ~/.rdebugrc
set autolist
set autoeval
set autoreload

Run Rails (or other app) via rdebug

$ rdebug script/server

Breakpoint from rdebug

(rdb:1) b app/controllers/my_controller.rb:10

Breakpoint in source

require 'ruby-debug'
debugger
my_buggy_method('foo')

Catchpoint

(rdb:1) cat RuntimeError

Continue to breakpoint

(rdb:1) c

Next Line (Step Over)

(rdb:1) n

Step Into

(rdb:1) s

Continue

(rdb:1) c

Where (Display Frame / Call Stack)

(rdb:1) where

List current line

(rdb:1) l=

Evaluate any var or expression

(rdb:1) myvar.class

Modify a var

(rdb:1) @myvar = 'foo'

Help

(rdb:1) h

There are many other commands, but these are the basics you need to poke around. Check the Cheat sheet for details.

This can also be used directly from any IDE that supports input into a running console (such as Intellij Idea).

That should get you started. So, before you stick in another 'p' to debug, try out ruby-debug instead!