At Pivotal Labs we take Continuous Integration (CI) seriously. Every project has a dedicated machine that serves as a CI environment. Each checkin on the project causes a build to be kicked off. A “build” means checking out the code from scratch and running of all the project’s tests, which, for a Rails project, means unit and functional tests, JavaScript tests and Selenium tests. For the JavaScript and Selenium tests, we run multiple browsers on multiple OS’s (e.g. IE 7 on Windows XP, FF 3 on OS X, etc).
We consider it critically important to keep each project’s build green (i.e. successful) at all times. A build is the heartbeat of the project: if it’s green, everything is healthy; if it turns red (i.e. fails), the team is encouraged to jump on the problem and get it back to green right away. We want red builds to go away quickly; the longer a build stays red, the longer it takes to track down the problem and the more likely it is that additional tests will be broken (the “broken windows effect”).
In order to facilitate this level of discipline, we’ve learned over the years that making the status of our CI environments obvious and visible to the team is critical. If a team isn’t acutely aware of the status of its build, it’s unlikely that a red build will get noticed and fixed quickly. You can have the CI server email the team, but that doesn’t work very well when the whole team is pairing all day: it might be a few hours before someone notices the email. You can install plugins in your browser or system tray that show build status, which helps, but still, they’re not always obvious enough. The best way we’ve found to keep the team informed is to display the status of the build high on a wall near the team as a big red or green indicator. That way, even when you’re busy coding, it’s easy to notice the build going red. These days we use 2 wide screen TVs, positioned in the office so that one is easily seen from any developer station.
When there’s only a single project going on, we’ve found that a screen that’s simply all red or all green is effective. At Pivotal Labs, though, we have many projects going on at once. Rather than putting numerous TVs up on the wall, we’ve created an application that aggregates each project’s CI status into one page. It’s only visible internally, of course. It displays the build status of each of our projects – all the client projects, internal projects, and open source projects that Pivotal Labs is involved with.
Recently we decided to bring up an external instance of the aggregator that shows the status of our open source projects. We’ve also pulled in the status of some of the open source projects that Pivotal depends on (e.g. Rails, CCrb). It’s available at ci.pivotallabs.com. The idea is to provide the same level of visibility into build status for open source developers, or teams that rely on their products, as we have internally at Pivotal Labs. Feel free to display the page on a monitor/TV/projector in your office! It refreshes itself every 30 seconds.
If you have an open source project and you’d like us to run your build and display its status, or if you already have a build and you’d like us to add it to the page, there’s no charge – just let us know (email contact@pivotallabs.com).
