Agile and Trust

Tuesday, December 30, 2008

Edward pointed out the great article by Kevin Matheny, featured in BusinessWeek, on Agile, and our experience on BestBuy Remix.

I’d like to highlight this passage:

Trust is tied closely to how you deal with change. Often, extending trust is hard for businesspeople working on technology projects, because we don’t know how to do the work. We often look to the documentation — requirements, design specifications, and the like — to give us the feeling of control over the outcome. Don’t bother. If you can’t trust your team to deliver, you have the wrong team. Find people you can trust, and then let them do the work. Talk every day, and make sure that the development team has direct access to someone who will be using the product every day after release. For Remix, we’ve never had a formal project plan, never had a requirements-gathering session, never created a requirements document. We chose the right partners, told them what we needed, and got to work. We have control over the outcomes, but we’re not worried about trying to control the details of how we get there.

Of course without trust, any project – “agile” or not – is at risk. But practices typically associated with agile let you go further with trust: everyone is in close communication*, and all levels of the project – from test-driven code written by developers, to regular demos to the client of the latest features – are oriented around fast feedback.

In other words, you the customer trust us in part because what we’re doing is visible and tangible to you. If we’re going in a direction you didn’t intend, or what the team planned a few weeks ago just doesn’t seem relevant anymore, we all talk and we correct course.

* for close communication, see Pivotal Tracker

Best Buy & Pivotal Labs

Edward Hieatt

Monday, December 29, 2008

Kevin Matheny, Senior E-Biz Architect at Best Buy, has an excellent article
today on BusinessWeek.com about Best Buy’s take on Agile software development and Best Buy’s experiences as a client of Pivotal Labs. As he mentions in the article, Pivotal Labs has been helping Best Buy build “Remix”, an API for the BestBuy.com product catalog. Kevin describes the agile methods that Pivotal Labs uses and how they’ve helped with what he calls “Corporate Agility”, which he describes as “working components instead of complete solutions, expecting and responding to change instead of trying to eliminate it, and trust rather than control.” He also describes how Pivotal Tracker fits into Pivotal’s agile process:

For example, I recently added a story to the tracker for Remix that read simply “flag products as new if their start display date is less than 30 days in the past.” That’s all the up-front documentation needed for Pivotal Labs, a development company that specializes in agile software development, to code that function into Remix. Any additional information can be gathered in the daily 15-minute team meetings or in a longer follow-up if more time is required.

Thanks for the mention, Kevin, and we’re very glad that the project is proving to be successful. Pivot Steve Conover is at the helm.

Tim Berners-Lee: Principles of Design

Pivotal Labs

Sunday, December 21, 2008

Here’s a good, quick read. It got its start about 10 years ago:

http://www.w3.org/DesignIssues/Principles.html

Topics:

Simplicity
Modular Design
Tolerance (”Be liberal in what you require but conservative in what you do”)
Decentralization
Test of Independent Invention (”If someone else had already invented your system, would theirs work with yours?”)
Principle of Least Power

When you’re heads-down doing Agile or OOP sometimes you find yourself accidentally assuming that certain useful general principles are special to what you practice – when the truth is they’re probably not even unique to your discipline, and some Greeks wrote them down around 500 BC.

Agile Tools: Simplify, Instead of managing complexity

Dan Podsedly

Friday, December 19, 2008

Great article comparing Pivotal Tracker to Mingle, as an example of a tool that simplifies things, rather than trying to manage complexity (and getting in the way):

Agile Tool Vendors: Please don’t try to manage complexity – simplify my life!

Thanks for the positive feedback, Matthias!

Best Remote Pairing Settings 2008

Chad Woolley

Thursday, December 18, 2008

About a year and a half ago, I made two posts on the “Best Remote Pairing Settings”.

A Remote Pair

The world has moved on, and so has the state of the art in remote pairing. I work remote 75% of the time, so this is an important topic for me. The setup we have now is working pretty good, so I wanted to describe it for the benefit of other remoties. Also, I’m only going to describe one specific setup – the one that is currently working best for me. So, here it is:

Computer and OS

Macintoshes running OS X 10.5 (Leopard) and maxed out on ram (at least 2 gig+), with a second monitor, ideally a 24″.

Pivotal uses 24″ iMacs exclusively. This lets you have your remote pair’s screen up on the second monitor, while still having the primary iMac screen available for local work (configuration, looking stuff up, temporary soloing, etc). You must be very disciplined and up-front – you should be explicit about when you are paying attention to your pair and when you are not, but that’s a whole other topic.

Also, at my home office, I have an iMac which I use for screen sharing, but I usually run the Skype audio/video on my MacBook. This takes the CPU load off of the iMac, which is important, because Skype is a CPU hog (more on that later).

Screen Sharing Software

Apple Screen Sharing which comes with Leopard.

Apple has done a nice job making the screen sharing app perform well, and the most important feature for any remote pairing screen sharing tool is performance. In my opinion, it performs better than any other VNC client on any platform, assuming you have a good connection (possibly excluding some windows native-video-hook solutions like UltraVNC or Remote Desktop, but there’s no way I would ever use Windows for development). All VNC clients which use the standard RFB Protocol can only be tweaked so much, and will only give you mediocre performance. However, there are several annoying bugs and gotchas with Apple Screen Sharing:

It is hard to find. Look under /System/Library/CoreServices/Screen Sharing.app. It is easiest to use Spotlight/Quicksilver or drag it to your dock. Supposedly you can start it with iChat but this never worked for me, I had to run the app directly.
Most of the useful features are disabled by default with no way to access them via menu or toolbar buttons. This is an amazingly annoying decision by Apple, but it is fairly easy to hack the toolbar buttons back into existence. Here is a Macworld tutorial showing two options.

In general, it seems that the best remote-control tools are those with some sort of native/low-level GUI integration: Leopard Screen Sharing, NoMachine NX, UltraVNC, Windows Remote Desktop, etc. Higher-level platform agnostic tools (like standard VNC/RFB protocol) just don’t perform as well – no matter how much you tweak the available color/depth/etc settings.

Audio/Video Hardware

Plantronics GamePro USB Headset

This is a great headset, and you need a really good headset if you are going to wear it all day, every day. Cloth earpieces, mic cover, very long cord, and I believe it also has some echo cancellation built in (there’s a huge box inline in the cord that does something). Unfortunately, I don’t see the exact model on the Plantronics website anymore. It may be replaced by the “GameCom” model, but I haven’t tried this.

Built-in iMac Microphone/Speaker

The built-in microphone and speaker on iMacs is really good. If you want to talk to a group of people remotely (for example, project standup), this is the way to do it. However, if the ambient noise gets too much, you can switch back to the headset.

You can even combine the two. For example, if you want to hear the surrounding conversations, but your pair is having trouble hearing you over the noise, then can wear the headset, but still keep the input set to the built-in iMac mic.

Sometimes you will need to adjust the input/output levels to reduce echo, and the remote pair should handle this themselves – they know what it sounds like.

Built-in iMac camera

Just like the built-in iMac mic and speaker, the iSight is a great camera. A detached iSight is just as good, if you want to be able to move it around or aim it without moving the computer.

Audio/Video Software

Skype

Skype is the best I’ve found. It does have drawbacks: it crashes rather frequently, it sucks a lot of CPU, it can do bad things to your network if you become a supernode, and it doesn’t support video in conferences.

However, it has great echo cancellation, it is free, and easy to use. The echo cancellation is really the most important thing – all other audio conferencing tools I’ve tried seem to have much more issues with echoes – even when you are using echo-cancelling hardware devices or speakerphones.

Some people seem to like iChat, but I have not had good luck with it. It takes longer than Skype to connect, the echo cancellation is not as good (sometimes it is, sometimes not), and most annoyingly, it doesn’t always close. I often end up having to force quit it – which is even more annoying when it is stuck on a freeze-frame of me making a stupid face or scratching my nose. Skype never does this – video always goes away when you shut your video or kill the call.

iChat has video conferencing, though, which is a benefit. You can sort of work around this by putting up the video preview in iChat, and having multiple remote people connect to view it via screen sharing, if you only want to see video for one of the participants (e.g. a couple of remote people calling in to a company meeting).

Network, Network, Network

This is the last but most important component to usable remote pairing. A fast, low-latency network connection is critical. I don’t have any numbers, but I believe that low latency is at least as important as high bandwidth. I also (without proof) believe that ping is not necessarily a good indicator of latency – I bet it is possible to have a good ping (ICMP) but still have issues with TCP/UDP latency. Who knows what’s going on in the tubes between you and your pair? Any data, tools, or insight on this would be very welcome.

As empirical evidence, for the first year or two at Pivotal I had DSL, which was pretty fast with low ping, but had continual problems with performance. Then, I switched to corporate-grade cable with a significantly higher bandwidth limit. My experience improved dramatically and my problems decreased greatly. This was about the same time I switched to Leopard screen sharing, so I think that had something to do with it, but the better connection definitely made a huge difference. Again, sorry I don’t have more concrete numbers, but I will guarantee that the better your connection, the better your experience will be.

Also, if you are in a corporate network, this may cause you problems. Even if there is a big pipe to your location, there may be saturation on your local LANs or intranet. Again, no hard data, but this is backed up by experiences of having consistently better performance when connecting to another remote at-home pair with a good connection as opposed to connecting to the Pivotal office which has a much larger pipe.

Remote Pairing Presentation at RailsConf 2008

At RailsConf 2008, Michael Buffington and Joe O’Brien did a good presentation on Remote Pairing. This is a very good presentation which covered many important aspects of remote pairing, as well as presenting some innovative ideas. Unfortunately, I don’t see a link to the preso, please post one in the comments if you have it.

Summary

This isn’t meant to be the be-all, end-all set of recommendations, it’s just what is working pretty well for me now. By “pretty well”, I mean that I can be an efficient pair, even when I’m driving the remote machine.

However, I’ve learned to cope with a lot, and adapted my work habits. It has forced me to become much better at communication, and describing what, why, and how I am programming. In general, though, I believe that remote pairing is physically, emotionally, and intellectually taxing. Regardless, I personally deal with it because Pivotal is such an awesome place to work and Pivots are such incredible developers. Most importantly, I come out in person for a week every month, attend retrospectives and brownbags, have some beers, and generally stay “entangled” with the rest of the team in person. If I was 100% remote, I don’t think I could handle it long-term.

So, I hope this helps out all the other remoties out there. Please let me know what you think, your experiences, and what works well for you.

Back in the land of the living (or: How RubyMine makes me happy)

Edward Hieatt

Wednesday, December 17, 2008

How my outlook on coding in Rails has changed over the past few months!

When I made the switch from Java to Rails a few years back, I, like many of my fellow Pivots making that same well-chronicled transition, delighted in the ease with which we could suddenly knock out a web app. How we cheered when our object-relational mapping took zero lines of code! How we applauded when we declared our model object validations in near-English! How we roared with laughter when convention viciously slapped the face of configuration! And how we shook our heads in dismay when we realized that our new development environment appeared to be from the mid-, if not early, nineties.

For, while we had arrived in a brave new world of minimalist declarative meta-programming, rapid prototyping, and an new-found sense of productivity that made even the most nimble forms of Java development look like wading through a morass of slimy boilerplate code and endless XML, we soon realized that the IDE situation was less than awesome. Our productivity was overall much improved, but we had taken a huge step backwards when it came to the act of writing – and especially changing – code and tests. Overnight, we went from living it up in a paradise of automated refactorings, seamlessly inbuilt test runners and powerful debuggers to roughing it with a text editor that, to our spoilt eyes, appeared to offer barely more than code highlighting and support for homemade macros.

Not only didn’t our favorite Java IDE, IntelliJ, not function well with Ruby, but nor did Eclipse, and nor did NetBeans. Early on, each had some nominal support for Ruby, it’s true, but it was mostly just code highlighting and some basic navigation. If we wanted to run a test, we had to (horror of horrors) leave our IDE, go to a shell and run a command (mapping a key to an “external tool” was cold comfort, it seemed to me). If we wanted to rename a variable, we had to do it manually. And if we wanted to debug something, well, the only option was to use a tool that appeared so prehistoric that we simply didn’t do it.

In fact, things were so bad that if we were honest with ourselves, TextMate sometimes looked like a better option that the IDE we knew and loved. We pleaded with Pivots who didn’t come from the world of Java to use IntelliJ, if only (for them) for its awesome global search, and if only (for us) to make ourselves feel comfortable in the new world of Rails. But IntelliJ had been rendered so impotent by its Ruby Kryptonite that it felt at times as if we were simply dragging around a comfort pillow that had lost its stuffing. All the progress that had been made in the world of Java IDEs for so many years seemed to have been lost. Woe was certainly us.

But then came some signs of life. NetBeans and Eclipse (in its various forms) were making progress in Ruby-land: they had test runner integration and debuggers that worked. Big steps forward indeed. While I suspected that I would probably miss IntelliJ, I nevertheless gallantly tried to commit to one of them: if they were going to offer me something even close to the power I used to have in my Java IDE, I was willing to put up with a lot. But despite my best attempts to be patient, I was disappointed time and time again. Eclipse still felt clunky compared to IntelliJ. NetBeans still suffered from a problematic global search and less-than-perfect VCS integration. They are gallant attempts, and they deserve credit. They’re both great products and they both serve communities that no doubt find them invaluable. But in that very unquantifiable, personal, emotional way, they weren’t what I really wanted. I wanted that IntelliJ feel.

So where was JetBrains on this? There was the Ruby plugin for IntelliJ, which was making some great strides forward, but that always felt like a bolt-on solution at best. What was more, it often didn’t work with new IntelliJ updates.

Imagine my surprise and delight, then, when Christmas came several months early this year. Suddenly, in late 2008, we are quietly presented with RubyMine. IntelliJ for Ruby? Surely not! I dared not hope for too much when with some trepidation I downloaded an early pre-EAP candidate a few months ago. But, joy of joys, it turned out that it was, or rather, is gradually becoming, (almost) all I was hoping for. Built-in test runner, debugger, pretty good refactorings – all things that other IDEs provide, but in that special JetBrains way that is so much more intuitive. Great success!

Now, I realize that all this probably sounds like a gush of praise that is not yet due, and perhaps that’s true. It’s certainly true that there’s a long way to go: RubyMine is lacking in many ways. But the signs are there that things are moving hastily in the right direction. Recently, each build that has come out has been noticeably better to me than the last (additional useful features, stability, performance). But more importantly, I feel like JetBrains is on the right trajectory with RubyMine. We now have a path towards a great IDE for Rails. I think we’re back in the land of the living, and I’m hopeful that between Rails and RubyMine our productivity will soon jump higher than we could have imagined just a few years ago.

OK, start the “All you need is Emacs”, “VIM is the best thing ever”, and “TextMate rocks, what’s this IDE nonsense?” comments below :)

Standup 12/03/2008: Instance variables in Selenium

David Goudreau

Wednesday, December 3, 2008

Help!

Instance variables in Selenium? We’re setting up a User model instance in a setup method. We want to use that User instance in our test, naturally. But when we get into the test, the User instance has no methods on it. It’s a standard Test::Unit Selenium test. Any ideas?
hoe/rake/rspec dependencies and CI? The following combination will break CI. The issue is that RSpec 1.1.11 depends on Hoe 1.8.2 and Hoe 1.8.2 requires Rake 0.8.3, which will break our CI environment. RubyGems may also need to get upgraded to get this combination to work. Sounds like a bit of work…
Uploaded file encodings? We’re getting truncated content when uploading files encoded with ISO-8859-1 Windows. Any ideas why this is happening? Can we check the encoding when the file is uploaded?
Clock.now with a time zone in a named scope in the TEST Rails environment does not use the MockClock class (which lets us manipulate time in our tests). Why isn’t it being used?
Demo environment DNS propagation & Apache configuration. The expected demo URL isn’t working. It sounds like Ops just recently set the name in DNS so it’s probably still getting propagated around.

Interesting Things

Apparently Treetop does not let you assume equality in the usual Ruby ways. Just something to look out for.
Last night we hosted a panel in conjunction with our partners Venture Archetypes called Project Startup. It’s a regular panel we’ve been hosting for a while now. It happens roughly every 2 months.

Standup 12/01/2008: Fun with libxml

David Goudreau

Monday, December 1, 2008

Interesting Things

Libxml has been giving us some more strange behavior on Linux. If you do

parser = XML::Parser.new
parser.string = '<foo></foo>'
document = parser.parse

# Now watch me fail, but only on Linux!
parser.string = '<bar></bar>'
document = parser.parse

We’re hosting a MagLev tech talk today compliments of Martin McClure.
Joseph Palermo has won the annual Pivotal Labs Mustache Competition. Granted, he was the only entry. But don’t let that affect your admiration of his work. Photo to follow.

Standup 11/21/2008: Pro Bono Airwaves

Joe Moore

Saturday, November 22, 2008

Interesting Things

Rails reminder: flash[:notice] = "Good Job" will survive a redirect, while flash.now[:notice] = "Good Job" will not. In general, flash.now is used when you render a template without a redirect, such as when a form submit has validation errors.
Good Books: Several folks have recommended JavaScript: The Good Parts.
Pro bono: Would anyone like to help out KUSF for free? Their new website project has been stalled for a year.

Ask for Help

“How do you get Selenium to work with Firefox 3?”

If you know how, pull the jar files out of a later release and use those. Good luck!

Standup 11/20/2008: Engine Yard ssh key changes

Joe Moore

Friday, November 21, 2008

Interesting Things

Engine Yard has made some changes to their ssh-key setup:
…any non-approved keys will be removed from the root user’s
authorized_keys file. It should be noted that customers should not log
in directly as root but rather should log in as their user and use
sudo for any commands that need super user privilege. If you are
currently using the root user to log in and have your key in roots
authorized_keys file it will be removed when this change is made.
If you are having any ssh problems, contact them.