I’ve been working with my client, Unpakt, for a while now. One of their core values is making people’s lives easier. They’re specifically focused on making it easy for people moving to a new home or office to find a mover, compare prices and book their move online.

As a development team, we’ve taken that same core value of making things easier and applied it to our software development & deployment workflow. Over time, we’ve progressively improved our process. We’re now at the point where we’re happy with it, and I wanted to share what we’ve been up to.

The Typical Flow

Let’s assume I’m using GitHub and Pivotal Tracker and I’ve just completed a story. What’s next? Push my changes up to GitHub, click Finish on my story, wait for the build to go green, deploy to staging. Then, click Deliver on all the stories that have just been deployed. Other stories were most likely ready to be delivered as well, so I’ll check that they were actually included in the deployment, and then Deliver them in Tracker.

The typical flow goes like this:

1. Commit the changes to my story
2. Push to GitHub
3. Click Finish on story in tracker
4. Wait for green build
5. Deploy to staging
6. Click Deliver in Tracker

That’s not really that bad. I’m lazy though. If something can be automated, I want it to be automated. For instance, the deployment process should be automatically handled by my Continuous Integration server.

If we take this to the next step and automate our build and deployments as described in Robbie Clutton’s Building Confidence post, we can reduce that flow to this:

1. Commit the changes to my story
2. Push to GitHub
3. Click Finish on story in tracker
4. Wait for green build
5. Click Deliver in Tracker

Let’s assume we’ve done that. That’s a nice improvement, as deploys will happen automatically whenever there’s a green build. That in itself is amazing.

But, what if we could reduce the steps in my everyday flow to this?

1. Commit the changes to my story
2. Push to GitHub

I want that!

A Streamlined Flow

Here’s the basic idea: tie the commit message to the Tracker Story ID. This enables all kinds of functionality that will get us down to that two step process.

GitHub & Pivotal Tracker Integration

GitHub supports integration with Pivotal Tracker. Dan Podsedly blogged about this back in 2010. You just tag your commits with Tracker Story IDs, and GitHub automatically comments on your story with the commit message. And, most importantly, it will click Finish on the story for you.

First, configure the Pivotal Tracker service hook on GitHub.

Once this is enabled, you can start tagging your commits to use the hook. The syntax for your commit message is pretty simple. Add the following at the end of your commit message:

[(Finishes|Fixes|Delivers) #TRACKER_STORY_ID]

So, if you just fixed a CSS bug for story #35192613:

When you push to GitHub, the post-receive hook will then call back to Tracker and put a comment on the story with a link to the commit on GitHub:

At this point, we’ve eliminated step 3 from above. Now our workflow looks like this:

1. Commit the changes to my story
2. Push to GitHub
3. Wait for green build
4. Click Deliver in Tracker

Honestly, that’s a pretty nice workflow. If I’m diligent about tagging each commit with the Tracker Story ID, GitHub will handle Finishing the story for me. It’ll also add a comment to the story, so others can take a look at the commit related to the story. Technical PMs especially appreciate this, since their curiousity about what changed is easily satisfied by just clicking on the comment in the story.

But I want more. I want to automate steps 3 and 4.

Deliver Stories Automatically

Pivotal Tracker has an API hook to deliver all finished stories. However, it’s not what we’re looking for. If our CI server calls that API hook after a green build, it’s possible for other stories to have been finished in the meantime that will not be included in the deploy. So, calling this API endpoint will just cause false positives.

A side-effect of tagging all your commits with Tracker Story IDs is that the git log now contains a nice little list of stories that have been finished. This makes it trivial to automate delivery of only the stories that have been deployed.

Unpakt was keen to share the script that makes this happen. They’ve thrown it up as a gist:

#!/usr/bin/env ruby

# gem 'pivotal-tracker'
require 'pivotal-tracker'

TRACKER_TOKEN = "..."
TRACKER_PROJECT_ID = "..."

PivotalTracker::Client.token = TRACKER_TOKEN
PivotalTracker::Client.use_ssl = true

unpakt_project = PivotalTracker::Project.find(TRACKER_PROJECT_ID)
stories = unpakt_project.stories.all(:state => "finished", :story_type => ['bug', 'feature'])

staging_deploy_tag = `git tag | grep staging | tail -n1`

stories.each do | story |
  puts "Searching for #{story.id} in local git repo."
  search_result = `git log --grep #{story.id} #{staging_deploy_tag}`
  if search_result.length > 0
    puts "Found #{story.id}, marking as delivered."
    story.notes.create(:text => "Delivered by staging deploy script.")
    story.update({"current_state" => "delivered"})
  else
    puts "Coult not find #{story.id} in git repo."
  end
end

Using the pivotal-tracker gem, the script finds all Finished stories in your Tracker project, and looks for a commit in the git log with that Story ID. If it finds one, it marks the story as Delivered.

It also adds a note to the story:

Our New Development Workflow

So, here’s what our development workflow now looks like:

1. Commit the changes to my story (including the Tracker Story ID in the commit message)
2. Push to GitHub

GitHub will mark my story as finished, and add a comment. CI will then run. If there’s a green build, it will deploy to staging. The deploy script will then run the deliver_stories script shown above, and all stories that have been deployed will be marked as delivered.

This is really, really nice. Having this much of the process automated for me just makes my life easier. It also gives our PM much quicker feedback about what’s happening. It’s very easy to forget to click Finish on a story, or deliver stories that weren’t deployed, or forget to click deliver on stories that were. This workflow eliminates these concerns.

Caveat

For this to work properly, the script that deploys to staging must checkout the same SHA as the green build from CI. If the git log shows commits from after the green build, some stories may be marked as Delivered when they haven’t actually made it through a green build and been deployed.

Caution

This workflow is not for every team. For example, it may be that your team functions better with developers verifying that each story is actually available and working on the staging environment before clicking Deliver in Tracker. No Product Manager wants to try to accept a story only to find that it’s not actually available on the staging environment. It may also be that on your project “Deployed” is not the same as “Delivered”.

You should only consider adopting a flow like this if your project is of a nature where deploys are easy, everyone on the team is playing along, and you can agree that “Deployed” and “Delivered” are one and the same.

The workflow described in this post has been working perfectly for months at Unpakt. We initially ran into the problem described above in the Caveat section, because our CI server running our deploy script while HEAD was checked out instead of the green build’s SHA. We haven’t had any stories accidentally marked as Delivered since we resolved that with TeamCity’s Snapshot Dependency feature.

Unpakt is hiring!

Unpakt is a small team with solid financial backing, led by a highly successful entrepreneur. They are creating an intense but fun and respectful culture and are looking for talented people to join their team. Check out their job listings!