TeamCity is pretty cool, you should totally check it out

Tuesday, March 13, 2012

At Pivotal, our default choice for CI is Jenkins. I feel that Jenkins does a fine job running builds and reporting on a pass/fail. It also has nice plugins and plenty of features like build labeling and clustering.

JetBrains’ TeamCity has all that, and more!

To be fair, TeamCity is missing some things that Jenkins has. However, I don’t remember what they are because I’ve never used some of the deeper features of Jenkins. One thing TeamCity doesn’t support well is running builds for multiple branches dynamically.

That said, I’m a huge fan of TeamCity. Here’s why.

TeamCity understands your tests. It uses custom formatters for RSpec, Cucumber, TestUnit and Shoulda. This means you get more than just a pass/fail on the build. You get a count of the tests that have run. You also get a count of the tests that failed. Seeing these numbers gives me a warm and fuzzy feeling that I don’t get with any other CI tool I’ve seen.

test counts

Because it understands the tests it’s running, it can keep track of them. For each spec, or cucumber step, it knows how many times it’s passed, and how many times it’s failed. And, it shows you the relevant stacktrace for each failure. You can also get statistics on a particular spec or step, like so:

test detail

It also knows what commit introduced a new failure. Here’s a shot of TeamCity showing a spec failure. The spec description is shown. Below that is the stacktrace in maroon. And to the right, it is showing that the spec first failed in build #532 with andre’s commit. The spec is still failing 105 builds later in build #637. It’s easy to see the stacktrace from both the first failure and the current failure. And yes, that spec has been broken since September 2011.

TeamCity makes it super easy to investigate what is making a build slow. Here’s a list of tests for the bundler gem, sorted by duration. You can filter, sort, and search the tests to analyze what’s going on.

You also get some interesting statistics at the build level:

test success chart
test count chart

The insight and analysis that TeamCity makes possible is extremely compelling. After using TeamCity for a few years, I used Jenkins for a a few months. I really missed all of the features above, and felt a lot less connected to my tests.

Other things I find to be very useful:

Immediate feedback if a test fails during a build. The build will go red as soon as the first test does.
If a failing test is fixed in a newly running build, it’ll let you know.
Besides the awesome insight into your tests, TeamCity has support for larger teams. Users can set or take responsibility for a broken build or individual test. So, it’s easy to keep your team informed about who is working on what.
RubyMine integration, including pre-tested commits.
TeamCity is free for 20 build configurations and 3 slave agents.

Full list of features: www.jetbrains.com/teamcity/features

Here’s the demo environment: teamcity.jetbrains.com

DISCLAIMER: I do not work for JetBrains, nor are they sponsoring this post. I do, however, love their products.

Standup 01/09/2012: Lightning and Rakes

Grant Hutchins

Tuesday, January 10, 2012

Help

“RSpec 2.8 is out. The Rake runner in TeamCity isn’t yet working.”

The suggestion was to update to an EAP release of TeamCity.

Events

Tuesday, January 10, 2012: NYC.rb lightning talks, pizza, and beer after work at Pivotal NYC.

[Standup][NY] Wed 5/18/2011: TeamCity + git + symlinks == fail

Ian Zabel

Wednesday, May 18, 2011

Interesting Things

A standup of pivots (Sam Coward, Sean Moon, Peter Jaros, and Brent Wheeldon) have recently run into a bug in TeamCity that causes trouble with symlinks when using git.

If you commit symlinks into your repo, TeamCity will not properly transfer these to the build agents. They end up being copied over to the agents as plain text files.

The workaround for this issue is to use Agent-side Checkouts instead of Server-side Checkouts.

Radiating your TeamCity Builds

Ian Zabel

Friday, April 8, 2011

If you’re using a Continuous Integration tool, you should also be using an information radiator like Pivotal Labs’ CiMonitor. CiMonitor is designed to be displayed on a screen that the entire team can see. If any of your builds go red, it shows up as a big red square and the team can quickly respond.

CiMonitor has long had support for CruiseControl.rb and Jenkins (formerly Hudson). However, my CI tool of choice is TeamCity from JetBrains. We’ve recently added native support in CiMonitor for displaying your TeamCity build status, and I want to show you how to get it going. It’s as easy as configuring a new project and pointing to TeamCity’s REST API. Let’s walk through it step by step.

Install CiMonitor

For native TeamCity support, you’ll need to be on the latest version of CiMonitor. Follow the installation instructions on the CiMonitor GitHub page. If you already have CiMonitor installed locally, you’ll have to update to the latest version.

Create a New Project

Choose “Team City Rest Project” as the Project Type.

For the Feed URL, you’ll be hitting the REST API provided by TeamCity 5.0 and newer. The correct format for the URL is as follows, with the # representing the TeamCity project number:

http://teamcity:8111/app/rest/builds?locator=running:all,buildType:(id:bt#)

You can find the correct project number by clicking onto the project in TeamCity and looking in the URL. Grab the number from the buildTypeId parameter.
TeamCity Project ID

You will also need to fill in a TeamCity username and password that CiMonitor will use to access the REST API.

Example Settings

New Project

Once you’ve set up the new project, save it and you’re all set.

Watch Your Builds Run!

You can now easily see the status of your TeamCity builds.

CiMonitor will also show that a build is currently in progress.
Running TeamCity Build

Wrapping Up

The integration is pretty solid, but there are one or two things that may seem to be odd behavior.

CiMonitor keeps track of build start times to control the size of the build dots for each project. The older the build, the smaller the dot. Currently, TeamCity’s REST API doesn’t publish the start time of a running build. Since this data is unavailable, CiMonitor just uses the current time when it fetches the feed. This is a minor issue, and JetBrains has already committed a fix for their REST API which will be released in the next version of TeamCity. CiMonitor will use the new field if it’s present in the feed.

Also, TeamCity has a great feature that will tell you that a build is failing before it even finishes. This has an interesting effect on CiMonitor. If a TeamCity build is red, the next time it begins running, CiMonitor will show it as green until the first test fails. Once TeamCity picks up on a failure, CiMonitor will show the build as red. This can be a little confusing at first because it may look like a build-in-progress is green before it has finished.

Hopefully you find this new functionality useful!