At Pivotal, our default choice for CI is Jenkins. I feel that Jenkins does a fine job running builds and reporting on a pass/fail. It also has nice plugins and plenty of features like build labeling and clustering.

JetBrains’ TeamCity has all that, and more!

To be fair, TeamCity is missing some things that Jenkins has. However, I don’t remember what they are because I’ve never used some of the deeper features of Jenkins. One thing TeamCity doesn’t support well is running builds for multiple branches dynamically.

That said, I’m a huge fan of TeamCity. Here’s why.

TeamCity understands your tests. It uses custom formatters for RSpec, Cucumber, TestUnit and Shoulda. This means you get more than just a pass/fail on the build. You get a count of the tests that have run. You also get a count of the tests that failed. Seeing these numbers gives me a warm and fuzzy feeling that I don’t get with any other CI tool I’ve seen.

Because it understands the tests it’s running, it can keep track of them. For each spec, or cucumber step, it knows how many times it’s passed, and how many times it’s failed. And, it shows you the relevant stacktrace for each failure. You can also get statistics on a particular spec or step, like so:

It also knows what commit introduced a new failure. Here’s a shot of TeamCity showing a spec failure. The spec description is shown. Below that is the stacktrace in maroon. And to the right, it is showing that the spec first failed in build #532 with andre’s commit. The spec is still failing 105 builds later in build #637. It’s easy to see the stacktrace from both the first failure and the current failure. And yes, that spec has been broken since September 2011.

TeamCity makes it super easy to investigate what is making a build slow. Here’s a list of tests for the bundler gem, sorted by duration. You can filter, sort, and search the tests to analyze what’s going on.

You also get some interesting statistics at the build level:

The insight and analysis that TeamCity makes possible is extremely compelling. After using TeamCity for a few years, I used Jenkins for a a few months. I really missed all of the features above, and felt a lot less connected to my tests.

Other things I find to be very useful:

• Immediate feedback if a test fails during a build. The build will go red as soon as the first test does.
• If a failing test is fixed in a newly running build, it’ll let you know.
• Besides the awesome insight into your tests, TeamCity has support for larger teams. Users can set or take responsibility for a broken build or individual test. So, it’s easy to keep your team informed about who is working on what.
• RubyMine integration, including pre-tested commits.
• TeamCity is free for 20 build configurations and 3 slave agents.

Full list of features: www.jetbrains.com/teamcity/features

Here’s the demo environment: teamcity.jetbrains.com

DISCLAIMER: I do not work for JetBrains, nor are they sponsoring this post. I do, however, love their products.