So I didn't go to whatever was going on in Baltimore this week, but I did do a whole bunch of open source coding over the past week or two:

• TestFirst has a totally revamped design, including the downloadable (or cloneable) student exercises, and Learn JavaScript is now a first-class citizen
• Wrong's expect alias now plays nicer with RSpec's expect
• Rerun got a few new command-line options, including --clear and --exit so you can now easily rerun regular scripts (like rerun -cx rake test) when their files change
• I submitted a patch to RubyGems to make the warningitis less ZOMG and more KTHXBY
• Fonzie is a bookmarklet that tells you what font you're looking at
• Twitter RSS is a bookmarklet that brings back the RSS link to the New Twitter UI
• Showoff works better with nested bullets and missing showoff.json files -- so go nag Scott if you want him to accept my patches ;-)
• I also finally got my git pre-commit hook correctly stripping whitespace, and

A lot of these projects, especially TestFirst, are aching for improvement, so if you feel like contributing code or courseware, or even just feature requests, please get in touch!

You should be doing automated configuration, period. Chef is a great automated configuration tool.

It has to be said, however, that chef has lots of parts, arguably an excess. If you google around for chef intros you see chef-solo referenced as a simple first step into "full" or "real" chef - chef client/server.

On our project we've built a mature web application, we've been using chef for over a year, and have never once felt the need for the client/server model, and we have no reason to expect to.

Here's how we run chef manually:

cd ~/projectroot
git pull
chef/run.sh

(that's it)

run.sh contains:

sudo sh -c "RAILS_ENV=$RAILS_ENV chef-solo -c chef/config/solo.rb -j chef/config/$RAILS_ENV/`hostname -s`.json"

We have capistrano (multi-server ssh tool) do the equivalent on deploy:

sudo [
  "cd #{app_root}",
  "export RAILS_ENV=#{self.variables[:rails_env]}",
  "chef/run.sh"
].join(" && ")

We deploy our code and update system config at the same time.

And that's all we need or want.

Links:

• Cooking with Chef 101 - a chef-solo-centric chef introduction
• Chef wiki

The Pivotal News Network has been going strong for six months (Pivots: talk to me if you'd like to share into the feed). Here are some highlights from May:

• from Startup Product Development: Simple Now vs. Complex Later

When starting any software project, there’s an age old argument: should we build something simple that solves our current problem or should we use an existing product that’s more complex, but more feature rich, since we know that’s where we’re going to end up in the future?

...

an oft neglected repercussion of building too much too quickly is that the extra functionality can calcify your product and make it very rigid. Releases become more complex, new features take longer to implement and bugs take longer to fix. You can find yourself a prisoner of your product, maintaining functionality and features that no one ( or very few ) people use. It can demoralize a engineering team, making them more and more susceptible to the nuclear option: the big rewrite.

I think the tendency to lean towards a more exhaustive solution upfront comes from a time when the effort require to change software was much higher than it is today. When systems were written in C, C++, Perl or even Java, making changes was a large undertaking. The thought of possibly throwing away chunks of code was nerve racking. It represented a huge investment in time and money. However, with todays rapid development languages and frameworks like Ruby/Rails & Python/Django, the investment required to create something, both in time and money, is rapidly shrinking.

• from Are you punching your users in the face?

Jeff [Patton]’s reply shocked me:

“The Ruby community cares about building high-quality apps, but doesn’t necessarily care about shipping high-value apps.”

Jeff went on to say that the Ruby community is obsessive about craftsmanship. This is a good thing, of course. We test. We write clean code. We take the time and care to build applications that are beautiful and do what our customers ask for.

Therein lies the rub: what customers ask for is rarely what they want, and almost never what they need. As Henry Ford put it, “If I had asked what people wanted, they would have said faster horses.” Or as I put it, your customer may pay you $1000 to deliver him a knuckle sandwich, but no amount of precision or strength training is going to leave you with a happy customer.

It turns out that constructing a high-quality application is not enough – you have to conceptualize and design an application that users will actually find useful. Doing this is every bit as difficult as constructing the software, if not harder. It requires a combination of research – generating new ideas from asking questions & identifying problems – and feedback – testing out ideas you’ve created. The Ruby & Agile worlds have been primarily focused on getting user feedback, without doing the all-important research.

• from Agile Anti-Patterns: Democratic Design

Weeks ago, some people in the Ubuntu community got a bit disappointed with the distribution’s core team:

We are supposed to be a community, we all use Ubuntu and contribute to it, and we deserve some respect regarding these kind of decisions. We all make Ubuntu together, or is it a big lie?

We all make Ubuntu, but we do not all make all of it. In other words, we delegate well. We have a kernel team, and they make kernel decisions. You don’t get to make kernel decisions unless you’re in that kernel team. You can file bugs and comment, and engage, but you don’t get to second-guess their decisions. We have a security team. They get to make decisions about security. You don’t get to see a lot of what they see unless you’re on that team. We have processes to help make sure we’re doing a good job of delegation, but being an open community is not the same as saying everybody has a say in everything.

• from Velocity as a Goal

  From my experience having velocity as a goal doesn't make any difference to the motivation of the team which is often cited as the reason for referring to it as a target. In all the teams I've worked on people are giving their best effort anyway so they can only really have an impact on the velocity by doing one of the following:

  • Working longer hours
  • Cutting corners on quality (by less testing perhaps)
  • Finding a smarter way of working

  ... In reality I haven't noticed that people on the teams I've worked on pay that much attention to whether velocity is considered a target or not. People just do their job and we pretty much always have the same velocity each week regardless.

• "If you split test every idea, you'll end up with either porn or gambling."

• "Every time you convince a client to leave their Excel spreadsheets behind and use @PivotalTracker an angel gets his wings."

More popular shared items:

• 7 LESSONS LEARNED WHILE BUILDING REDDIT TO 270 MILLION PAGE VIEWS A MONTH

• The Case for Git Rebase

• "The best way to handover work is to leave a broken test for your colleague to fix."

• Nonblocking ActiveRecord & Rails

• HTML5 vs Flash performance

• How many lines of code does it take to create the Android OS?

• The unimportance of product names (which got smacked down in the Google Reader comments)

If you are using Bundler to lock and package your gem dependencies in your app (which you should), here's some tips on making everything automatic in your Capistrano deploy.

Refer to the Bundler Documentation for instructions on how to use Bundler to properly package your gems and check everything in.

Once this is done, however, you still must ensure that two things are done on every machine to which you will deploy:

1. Bundler is installed
2. You run 'bundle install' on every deploy to install the packaged gems on the local machine (and compile any gems with native dependencies)

Here's the Capistrano magic to accomplish these two tasks automatically on every deploy:

before "deploy:bundle_install", "deploy:install_bundler"
after "deploy:update_code", "deploy:bundle_install"

namespace :deploy do
  desc "installs Bundler if it is not already installed"
  task :install_bundler, :roles => :app do
    sudo "sh -c 'if [ -z `which bundle` ]; then echo Installing Bundler; sudo gem install bundler; fi'"
  end

  desc "run 'bundle install' to install Bundler's packaged gems for the current deploy"
  task :bundle_install, :roles => :app do
    run "cd #{release_path} && bundle install"
  end
end

Oh, and for you GemInstaller users out there - here's an easy way to generate a Bundler Gemfile from your geminstaller.yml config:

geminstaller --bundler-export > Gemfile

You'll probably still need some tweaks, but this will get you started. Just make sure you upgrade to GemInstaller 0.5.5 first (0.5.4 forgot to put the 'source' line in the Gemfile).

Happy Bundling! -- Chad

P.S. There is a similar article here, which includes tasks to symlink your .bundle dir into the Capistrano shared directory, but my deploy was pretty fast anyway, so I didn't worry about it. YMMV.

We've pooled some Pivot shared tech news feeds and made this feedburner feed:

http://feeds.feedburner.com/pivotal-news-network

The content is in the spirit of Blabs, so we hope readers here might find it to be useful. See what you think.

I have an RSS feed, and I'd like to make a little news page out of it with the ability to post comments. Having done a quick survey of what's available, I'm thinking of doing nothing. But I'd like to find out what others have to say about tools available.

General requirements:

• I'd like to take an RSS feed and make a nice news page out of it.

• No mangling or truncating of articles in the feed.

• It'd be great if I could tie comments in. That might mean, for instance, dropping in disqus.

• I'd rather not have to write, deploy, or maintain any code. But if there's some set of tools/apis I could easily tie together with code, I'd consider it (some Heroku-type setup, for instance).

Examples of sites I consider to be at least partially what I'm driving at:

• ycombinator Hacker News
• Tabbloid
• Any blog

If I had to write code, I could imagine using something like feedzirra, sticking it on a many-times-per-hour cron job, and writing out a page and dropping in disqus.

Thoughts?

No, seriously. Why wouldn't you use Erector? Cause I think it's a pretty awesome view framework, but for some reason it hasn't caught fire yet. So if you think writing actual Ruby to emit HTML, with a clean, nestable syntax with full support for Ruby features like inheritance, delegation, and yield is neat, but there's something holding you back, then please let us know what it is. At best we can fix it, and at worst, at least we'll know why.

Here are some reasons I think you might not use Erector:

You love angle brackets. If this is the case then I can't help you. I don't think anybody can.

You like typing every tag name twice. Since Erector elements are Ruby statements, every open tag gets automatically closed.

You like invalid HTML. Since Erector elements are Ruby statements, every open tag gets automatically closed. (See how that works?)

You always remember to call 'h'. Rails 3.0 is going to HTML-escape all output by default. Erector's been doing this the whole time. Cause, you know, why wouldn't you?

You like having to rewrite your code when you extract a partial, and then again when you extract a helper method. In ERB, templates, partials, and helpers all have slightly (and annoyingly) different syntax for things like referring to variables and calling other code. Erector is all Ruby, so you can use your favorite refactoring browser, or just cut and paste, to move your code around. Check out this excerpt from Jeff Dean's RailsConf talk to see this in action, or read the slides from the whole talk on SlideShare.

You hate encapsulation. You think that your views should have direct access to all the instance variables of your controller. Unless they're partials, in which case you shouldn't, even though you can, although the names might be different. Confused yet? So am I.

You like putting code for one component in three separate files. Erector's new "externals" feature allows you to put all the code -- HTML, CSS, and JavaScript -- inside a single Ruby class. The CSS and JavaScript then get output inside the HEAD, once per HTML page, while the HTML gets rendered in the BODY as usual, as many times as necessary. This follows the OO paradigm of cohesion, otherwise known as "put similar stuff together," which is the complement of loose coupling, which means, "keep different stuff apart."

Okay, so those were sarcastic reasons. Here are some more possible reasons why you wouldn't use Erector. I suspect that these next ones hit closer to the mark. But I believe that they're all specious, if not downright false.

Your site contains a whole lot of complex HTML and a few inserted Ruby variables. OK, this makes sense. Erector's not great for static sites. But I've never personally worked on a web application where the code inside the views didn't quickly get complex enough to require codey things like loops and functions. And if you're writing code, then why not do it in a programming language?

Your designers don't know Ruby. I've heard this complaint a lot, but I have yet to meet this mythical designer who's smart enough to understand modern HTML, CSS, JavaScript, ERB, and partials, but is not smart enough to learn that "div 'foo', :class=>'bar'" outputs "<div class='bar'>foo</div>". On the contrary, I've worked with several designers who, after a few tutorial pairing sessions, were comfortable checking code in and out and editing Erector view code at will. Like any junior coder, they need to stay away from the tough stuff, but they're pretty good at knowing what they don't know and asking for help when they need it. (Which they would also do if working inside ERB.)

View code needs to look as similar to HTML as possible. Well, I hear this, but have you looked at HAML? That language is hella popular, and it doesn't look anything like HTML. Its structure is similar, in the abstract, but so's Erector's, and at least in Erector the method for emitting a div is called, you know, "div". And it's a method. And I don't want to turn this into a war between HAML and Erector -- I think HAML is gorgeous -- but HAML suffers from the same design flaw as every templating technology: views are not objects, and markup isn't code. After a certain point of complexity, HAML's elegance breaks down and you'd be better off doing loops and functions in code.

You've already got a bunch of stuff in ERB and it'd take too long to convert it. Yes, legacy code is a pain, but we have a command-line tool that converts ERB (or HTML) to Erector to make it a bit smoother. And you don't have to convert your whole app to Erector at once. Erector views can interoperate with ERB or HAML in Rails and Sinatra.

You're stuck on an old version of Erector. Yes, legacy code is a pain, but we have an upgrade guide for getting to 0.6.0, and people on the mailing list ready to help.

Erector's too slow. Lies! Erector is faster than a greased rattlesnake going downhill. Check out these benchmarking results. Erector is about 2x as fast as ERB and 4x as fast as HAML about the same speed as ERB and HAML(*) under typical conditions. We make sure to use the same output stream to minimize string copy or realloc, and using Ruby objects means much lower parsing overhead.

(*) Update: the "2x/4x" figure was based on a benchmark program that didn't use template caching, which speeds things up for both ERB and Haml. With template caching, Erector and Haml are about the same speed; Haml is about 20% faster when rendering a page with no partials. See this ongoing thread on the Erector list.

There's no documentation. More lies! We have a whole bunch of documentation at http://erector.rubyforge.org, including a FAQ and a user guide.

You got burned by Markaby. Underneath the elegant facade of Markaby lay a confusing and often counter-intuitive engine. Its use of instance_eval and other tricks made simple things break in weird ways and made debugging a real chore. Erector was born out of those frustrations, and one of its main design goals is "no magic." Also, there was a long time where Markaby wasn't being maintained (although that's changed recently); we have a core group of developers committed to responding on the mailing list and github, and we run integration tests against the latest stable Rails release (and soon, against Edge) to catch incompatibilities early on.

Rails has all these great helpers and I want to keep using them. Okay, go right ahead! Erector's Rails integration allows you to call any helper, either directly through a proxy method, or indirectly through the helpers object. If you find a helper that doesn't work, let us know and we'll add it to the list of supported helpers. (We haven't done all of them yet because it's a pain in the neck to look at each function and figure out what its input and output semantics are. Does it return a string or emit directly onto the output stream? Does it take a block? An options hash? An html_options hash? Etc.) We're also slowly putting some Rails functionality into Erector, either in the base class or in custom widgets. If there's something you need, ask on the mailing list, or better yet, send us a patch.

Its name is a dirty word. I've heard this more from people who didn't grow up in the United States, where the Erector Set was a popular toy among the 6-to-12-year-old DIY set in the 70s and 80s. (Apparently it was called Meccano in the UK.) Erector is a normal word, used all the time in the news and in business names. And as the name of a view library it's evocative in a way that's relevant and interesting, in that it's a builder, and you build a view up out of parts.

But we have heard this complaint, and in sympathy, changed the name of the command-line tool (oh, sorry, guess I can't say "tool" either)-- uh, executable-- from "erect" to "erector" even though the former is a venerable English verb that's grammatically appropriate ("I asked him to erect the scaffolding."). If you introduce the library and your coworkers get all giggly then I think if you just say the name with a straight face and then roll your eyes and mock your bawdy buddies when they snicker then all will be well. After a few repetitions it won't sound odd at all.

You've never heard of it. Help spread the word! Post a review on your blog! Ask your favorite app framework whether they support it! Post code samples in Erector and when people say "What's that?" then point them at http://erector.rubyforge.org! Give a talk at a meetup! Write your congressman and ask if she supports the Erector Mandate Bill of 2009! Buy ad space on the moon!

So, in conclusion, and despite my somewhat snarky tone throughout, I am honestly and desperately curious to know why the world has not yet beat a path to Erector's door. Anybody got any more ideas?

sudo gem install remixr

We at Pivotal like that incantation. Thanks to the Squeegee crew for putting Remixr together.

# find stores within 50 miles of ZIP 76227 and products over three G's

stores = client.stores({:area => ['76227', 50]}).products({:salePrice => {'$gt' => 3000}}).fetch.stores

Beautiful.

• Remixr intro

• http://github.com/squeejee/remixr

• Best Buy Remix API site

• Remix has some exciting upgrades and additions coming soon, keep up with it all via the Remix API blog.

This fixes several bugs that people have complained about for quite a while. Please let me know if anything is broken.

GemInstaller 0.5.3 has been released!

GemInstaller

• by Chad Woolley
• http://geminstaller.rubyforge.org

CHANGES

• 0.5.3 / 2009-08-25
• Many long overdue bugfixes and patches, see http://tinyurl.com/geminstaller-0-5-3-release for details.
• Thanks to Greg Fitzgerald, Britt Crawford, John Trupiano, Gabriel Gironda, and Eric Hodel for patches and assistance.
• Issues with case statement under Ruby 1.9
• GemInstaller cannot distinguish between gems that have the ame name but capitalized differently.
• add ./ci as default location for config file
• Disable GemInstaller install in default rails preinitializer.rb, but fork if it is used
• autogem() fails when run for newly-installed gem
• Sometimes installing fails due to RubyGems cache not being cleared between multiple API calls

DESCRIPTION

Automated Gem installation, activation, and much more!

FEATURES

GemInstaller provides automated installation, loading and activation of RubyGems. It uses a simple YAML config file to:

• Automatically install the correct versions of all required gems wherever your app runs.
• Automatically ensure installed gems and versions are consistent across multiple applications, machines, platforms, and environments
• Automatically activate correct versions of gems on the ruby load path when your app runs ('require_gem'/'gem')
• Automatically reinstall missing dependency gems (built in to RubyGems > 1.0)
• Automatically detect correct platform to install for multi-platform gems (built in to RubyGems > 1.0)
• Print YAML for "rogue gems" which are not specified in the current config, to easily bootstrap your config file, or find gems that were manually installed without GemInstaller.
• Allow for common configs to be reused across projects or environments by supporting multiple config files, including common config file snippets, and defaults with overrides.
• Allow for dynamic selection of gems, versions, and platforms to be used based on environment vars or any other logic.
• Avoid the "works on demo, breaks on production" syndrome
• Find lost socks.

Quick Start

See http://geminstaller.rubyforge.org/documentation/index.html

INSTALL

• [sudo] gem install geminstaller

In a recent thread on the East Bay Ruby Meetup list, several people chimed in with Ruby IDE suggestions. I suggested RubyMine, which we use at Pivotal. Several people mentioned NetBeans and Aptana RadRails, so I decided to have a little contest.

Now, if I am going to work in an IDE and sacrifice the speed of a text editor, I want to see it work for me ('cuz RubyMine can chew through all your CPU and RAM and then some faster than you can say Moore's Law). That means understanding Ruby, and using that information to save me some significant thinking and work.

I don't mean code-generation macros or dumb context-aware keyword-completion, I mean something useful like knowing where my classes and methods are. In my book, that leaves out Emacs, Vi, and even TextMate, regardless of their other merits (sorry people, I like text editors too, but I'm making up this test - flame away, the comments section is below).

So, here's the smackdown scenario:

• Download the latest NetBeans, RadRails and RubyMine.
• Open a Ruby project, this example is one of mine (I wrote this three years ago to learn Ruby, so don't make fun of me for doing dumb stuff...)
• Test the ability of the IDE to navigate through Ruby language constructs. This should be easy, it is a command-line app using a Dependency Injection architecture, no metaprogramming curveballs!
  • Open the root Ruby class for the project (lib/geminstaller.rb)
  • Pick a variable (app in self.install)
  • Try to work back to the class declaration (GemInstaller::Application) using IDE navigation (Command-click in all the IDEs, although sometimes they ignore you)

Result? RubyMine succeeded, NetBeans and RadRails failed miserably. Here's what happened in each:

NetBeans

1. Click create_application class method reference in app = create_application. NetBeans takes me to the method declaration in same class.
2. Click the app method in registry.app.

FAIL! ANGRY BEEP! app is an attr_accessor on another class, NetBeans can't find it.

RadRails

1. Click create_application class method reference in app = create_application. RadRails takes me to the method declaration in same class.
2. Click app method in registry.app. RadRails takes me to the attr_accessor on the Registry class (without a prompt, and highlighting the symbol, which is even better than RubyMine).
3. Click (and F3) on the :app symbol argument to attr_accessor.

FAIL! ANGRY BEEP! RadRails can't figure out the symbol parameter to attr_accessor. It says "Current text selection does not resolve to a Ruby element".

RubyMine

1. Click create_application class method reference in app = create_application. RubyMine takes me to the method declaration in same class.
2. Click app method in registry.app. RubyMine pops up a dialog asking if I mean the attr_accessor on the Registry class, or the local variable I'm declaring. That's rather silly, I admit, but the point is it followed the reference to another class.
3. Click on the attr_accessor choice. RubyMine takes me to the attr_accessor line in Registry.
4. Click on the :app symbol argument to attr_accessor. RubyMine takes me to the point where the @app instance variable is initialized.
5. Click on the Application class name in the GemInstaller::Application.new constructor invocation. RubyMine pops up a dialog asking if I mean the Application class in my application, or two other Application classes that happen to be in my Ruby installation. This is also a silly question - it should have known the correct choice because of my namespacing, but it still found the class.
6. Click on the GemInstaller:Application choice. RubyMine takes me to the class declaration.

SUCCESS! RubyMine drilled all the way to the class declaration, even through an attr_accessor, albeit with a couple of stupid questions.

Summary

I personally think this is a Big Deal. In the past, I've mocked Ruby IDE functionality as a poor simulacrum of the vast power in Java IDEs. When using Eclipse in Java, I could perform epic refactorings, extracting superclasses and adding parameters to method signatures; refactoring scores of classes across multiple projects in a few mighty keystrokes. Yes, I'm fully aware that this is because Ruby is a dynamic language, but that doesn't make me miss a real refactoring IDE any less, and others have long lamented these shortcomings, as well.

So, for years, even though I'd always indulge my pairs if they wanted to use an IDE, I've done all my personal hacking with a fast, lightweight text editor and command line tools. To me, the benefits of a memory- and processor-sucking IDE with tons of unnecessary, unconfigurable, resource-eating tiny-ass-fonts and chrome did not justify giving up the speed and responsiveness of a great text editor.

However, RubyMine can now navigate code for me. I don't have to think and manually find that class, RubyMine knows where it is. Granted, that ain't no Extract Superclass, but it saves me a lot of thought and time, both of which are increasingly rare commodities for me.

To be fair, this is really just a problem related to parsing and interpreting attr_accessor declarations, and I expect that NetBeans and RadRails will pass this test as well in another release or two. That's all great news, because it means that Ruby IDEs are finally, slowly, coming of age. I think I'll still be waiting a long time for an automated Modify Method Signature refactoring, though...