Next Tuesday, July 14, at 10pm PST, Chris Sepulveda from Pivotal Labs will be broadcasting an O'Reilly WebCast on automated unit testing with the Palm Mojo SDK. Developer testing is at the heart of Pivotal Labs's development practices, and we're excited to be involved in bringing testing to Mojo development. The WebCast will cover the following:

• Introduce BDD & Jasmine
• Installing Jasmine & add related code to the app to support BDD
• Discuss how to write a failing test first, then add functionality to make a test pass
• Develop a simple webOS application test first, with the Mojo SDK

There's more information about the WebCast on webOSHelp.net.

Date: Tuesday, July 14th at 10 am PT
Price: Free
Duration: Approximately 60 minutes
To register: http://oreilly.com/go/palmmojo

At Pivotal Labs we take Continuous Integration (CI) seriously. Every project has a dedicated machine that serves as a CI environment. Each checkin on the project causes a build to be kicked off. A "build" means checking out the code from scratch and running of all the project's tests, which, for a Rails project, means unit and functional tests, JavaScript tests and Selenium tests. For the JavaScript and Selenium tests, we run multiple browsers on multiple OS's (e.g. IE 7 on Windows XP, FF 3 on OS X, etc).

We consider it critically important to keep each project's build green (i.e. successful) at all times. A build is the heartbeat of the project: if it's green, everything is healthy; if it turns red (i.e. fails), the team is encouraged to jump on the problem and get it back to green right away. We want red builds to go away quickly; the longer a build stays red, the longer it takes to track down the problem and the more likely it is that additional tests will be broken (the "broken windows effect").

In order to facilitate this level of discipline, we've learned over the years that making the status of our CI environments obvious and visible to the team is critical. If a team isn't acutely aware of the status of its build, it's unlikely that a red build will get noticed and fixed quickly. You can have the CI server email the team, but that doesn't work very well when the whole team is pairing all day: it might be a few hours before someone notices the email. You can install plugins in your browser or system tray that show build status, which helps, but still, they're not always obvious enough. The best way we've found to keep the team informed is to display the status of the build high on a wall near the team as a big red or green indicator. That way, even when you're busy coding, it's easy to notice the build going red. These days we use 2 wide screen TVs, positioned in the office so that one is easily seen from any developer station.

Kevin Matheny, Senior E-Biz Architect at Best Buy, has an excellent article today on BusinessWeek.com about Best Buy's take on Agile software development and Best Buy's experiences as a client of Pivotal Labs. As he mentions in the article, Pivotal Labs has been helping Best Buy build "Remix", an API for the BestBuy.com product catalog. Kevin describes the agile methods that Pivotal Labs uses and how they've helped with what he calls "Corporate Agility", which he describes as "working components instead of complete solutions, expecting and responding to change instead of trying to eliminate it, and trust rather than control." He also describes how Pivotal Tracker fits into Pivotal's agile process:

For example, I recently added a story to the tracker for Remix that read simply "flag products as new if their start display date is less than 30 days in the past." That's all the up-front documentation needed for Pivotal Labs, a development company that specializes in agile software development, to code that function into Remix. Any additional information can be gathered in the daily 15-minute team meetings or in a longer follow-up if more time is required.

Thanks for the mention, Kevin, and we're very glad that the project is proving to be successful. Pivot Steve Conover is at the helm.

How my outlook on coding in Rails has changed over the past few months!

When I made the switch from Java to Rails a few years back, I, like many of my fellow Pivots making that same well-chronicled transition, delighted in the ease with which we could suddenly knock out a web app. How we cheered when our object-relational mapping took zero lines of code! How we applauded when we declared our model object validations in near-English! How we roared with laughter when convention viciously slapped the face of configuration! And how we shook our heads in dismay when we realized that our new development environment appeared to be from the mid-, if not early, nineties.

For, while we had arrived in a brave new world of minimalist declarative meta-programming, rapid prototyping, and an new-found sense of productivity that made even the most nimble forms of Java development look like wading through a morass of slimy boilerplate code and endless XML, we soon realized that the IDE situation was less than awesome. Our productivity was overall much improved, but we had taken a huge step backwards when it came to the act of writing - and especially changing - code and tests. Overnight, we went from living it up in a paradise of automated refactorings, seamlessly inbuilt test runners and powerful debuggers to roughing it with a text editor that, to our spoilt eyes, appeared to offer barely more than code highlighting and support for homemade macros.

Ian McFarland and I attended a 2-day Development Forum hosted by OpenView Venture Partners last week in Boston. It's the second year that Pivotal Labs has participated in the event. Open View has a portfolio of companies from all over (Europe, Australia, the US), each of which has been working on implementing Scrum over the past year. The engineering staff from 10 portfolio companies attended the event. Jeff Sutherland (amongst other things the co-creator of Scrum) is a Senior Adviser to OpenView; he provides advice and guidance to the portfolio companies as they progress through their Scrum adoption, and he gave a talk at the Forum. Pivotal Labs was invited to speak and lead a discussion of on 2 core topics: Developer Testing and the Principles of Build.

Looks like JsUnit gets a few pages devoted to it (about a dozen, actually) in the new book "Foundations of Agile Python Development":

http://www.apress.com/book/view/1590599810

Thanks to Dave Smith for the pointer.

Suppose we have some behavior attached to a button's onclick event: when clicked, the button should append "foobar" to the value in the text field with ID "output_field". We might do it like this:

<input type="text" id="output_field">
<button onclick="var textField = document.getElementById('output_field'); var currentValue = textField.value; textField.value=currentValue + 'foobar';">

Inline event handler code such as this is extremely common. It's easy to write, but it's not testable, reusable, or readable, and it's mixed up in the HTML world. The way it is currently, it's hard to think of it as code that can be refactored, added to, abstracted, generalized, etc. Of course, this is a contrived, simple example, and if it were real code, probably not worth worrying too much about. However, we can use it to demonstrate some techniques that we can apply to real-world situations.

Extracting the handler to a function

Let's attack it by considering it from the point of view of testability with JsUnit; very often this can the best way to move forward because it creates a second client to the code that is relatively independent of the HTML DOM. The first step is to extract the onclick event into a function, rather than having it inline:

<script language="javascript">
  function appendFoobar() {
    var textField = document.getElementById("output_field");
    var currentValue = textField.value;
    textField.value = currentValue + "foobar";
  }
</script>

<input type="text" id="output_field">
<button onclick="appendFoobar()">

Notice that even ignoring testability, this is a dramatic improvement. First, we don't a fragment of JavaScript floating around in an HTML element. Second, we have a name for our behavior (appendFoobar) that makes the code more readable: it's now much clearer that clicking the button should write the current date. Third, we can now reuse this code from more than just our onclick handler.

Writing a test page

The second step is to move our function to an external .js file, say appender.js, so that we can write a Test Page:

<script language="javascript" src="appender.js"></script>

<script language="javascript">
function testAppendFoobar() {
  assertEquals("initialvalue", document.getElementById("output_field"));
  appendFoobar();
  assertEquals("initialvaluefoobar", document.getElementById("output_field"));
}
</script>

<input type="text" id="output_field" value="initialvalue">

Our test is for our extracted function: our test, rather than the button element, calls our extracted function.

Injecting DOM dependencies

Now that we have a green test, and code that we can actually read, we might want to consider the following refactoring. The code and test both go out and grab an element from the DOM with a certain ID. The code would be more self-contained and reusable if rather than going out and finding the DOM element, we instead passed it in to appendFoobar. That is, we could inject the dependency on the DOM element:

function appendFoobar(textField) {
  var currentValue = textField.value;
  textField.value = currentValue + "foobar";
}

function testAppendFoobar() {
  var textField = document.createElement("input");
  textField.value="initialvalue";
  appendFoobar(textField);
  assertEquals("initialvaluefoobar", textField.value);
}

<button onclick="appendFoobar(document.getElementById('output_field'))">

Extracting an object

Let's do one more refactoring: objectifying our code. Obviously, at this point our simple example doesn't warrant this refactoring, but let's keep going to illustrate the point. We'll use prototype.js to keep things simple:

Appender.prototype = {
  initialize: function(textField) {
    this.textField = textField;
  },

  appendFoobar() {
    var currentValue = this.textField.value;
    this.textField.value = currentValue + "foobar";
  }
}

function testAppender() {
  var textField = document.createElement("input");
  textField.value="initialvalue";
  var appender = new Appender(textField);
  appender.appendFoobar();
  assertEquals("initialvaluefoobar", textField.value);
}

<button onclick="new Appender(document.getElementById('output_field')).appendFoobar()">

Summary

Our code has come a long way from being an inline event handler, muddled up with HTML code. In its new form, it's the kind of code that many developers are more comfortable working with. By aggressively refactoring even simple event handlers with these techniques, we can make working with our JavaScript a far more pleasant experience.

The explosion of client-side JavaScript in Web 2.0 applications has taken many developers by surprise. I've often found that more senior members on web development teams shy away from JavaScript coding, preferring the known quantity (and perceived purity) of server-side development. After all, isn't JavaScript that flaky, inconsequential language that we used to use to make pretty pull-down menus, but that can't be relied on for any serious purpose? Since when did it become something that I can't leave to my UI designer? Can't I just Google for a script kiddy's website any more? Do I really have to start taking JavaScript seriously? Well, yes. These days, it's becoming inevitable that the majority of the team comes into contact with client-side JavaScript. But still some of us resist it, largely due to painful past experiences of long nights spent trying to get browser X to work the same as browser Y.

As a consequence, I consistently observe two patterns over the course of a project. First, the client-side effort doesn't get the benefits of the skill and experience held by seasoned object-oriented developers. Instead the JavaScript code is left to newer, less experienced members of the team; the rigorous practices that the more senior members of the team usually insist on aren't advocated for to the same degree on the client side. Second, because of the fear of JavaScript, opportunities to make the application slicker, more user-friendly and generally more modern are missed.

I don't mean to downplay the difficulty of JavaScript coding. There's no doubt about it: it's eccentric, it's not consistent between browsers, it's slow where you don't expect it, it does objects in a weird way, and the list goes on. But here's where we can turn the situation on its head. What does our experience tell us about how to attack a difficult area in our code? It certainly doesn't tell us to ignore it and hope it works out in the end. No, it tells us to shine a light on the painful area and apply our development practices even more strictly than usual: test-drive it, cover it with unit and functional tests, aggressively refactor it, separate concerns, extract the right objects, apply design patterns where they can help, and so on.

What I've seen work when it comes to JavaScript is to fight fire with fire. Rather than responding to the difficulties of JavaScript by shying away from it and rationalizing a lack of quality in our client-side code, we can choose to react by applying our practices more strictly, not less. We can ask ourselves how to better test their JavaScript, how to expose complexity more clearly, how to objectify concepts more naturally. We can break our JavaScript into layers, and apply typical MVC patterns as we might do in server-side Java or Ruby on Rails code. We can abstract our AJAX requests and responses in order to better simulate them in our tests. We can run our tests in a continuous build on all our target browser/platform combinations. Why not try to hold our JavaScript to the same standards as any other part of our code? It's not always easy, and it certainly takes practice and strict discipline. I've found that the learning curve is steeper and longer than with other languages. But I've also found that there's really no reason why we can't be as rigorous with our client-side development as we are anywhere else in our codebase as. We end up with all the usual benefits: better stability, fewer bugs, more malleability.

Getting started

Amongst other topics, over the next few weeks I'll be posting concrete ideas for ways to approach JavaScript development. Most of them will center around JsUnit, because in my experience it's the introduction of unit tests and test-driving that drive the most dramatic improvements in not only the software but how we think about our work.

So, what does it look like when a team attacks JavaScript in this way? As someone who has been through his fair share of pain coding JavaScript, it's thrilling to observe. I've seen teams at Pivotal Labs take a more disciplined and rigorous approach to JavaScript than I could have imagined even a couple of years ago. And the best part is that besides the benefits I already mentioned, there's a huge payoff to all this added discipline: rather than being afraid of JavaScript development, we sometimes enjoy it and look forward to it. It's simple, really: JavaScript is hard, so let's throw everything we know at the problem. I've found that if we do, we can tame JavaScript to the point where our progress becomes as predictable as we expect in the rest of the codebase.