Interestings
Google's favicon service
A reliable way to get cached favicons (e.g. https://plus.google.com/_/favicon?domain=pivotallabs.com )
My Parade Costume’s a Kitty, What’s Yours?
Tuesday, May 21, 2013
Community
HQ
Explore Blog posts about everything we are up to, Tech Talk videos covering a huge range of timely topics and Event listings to keep you current on happenings at the Labs.
Showing and hiding conditional HTML without Javascript
Tuesday, May 21, 2013
Have you ever filled out an address form that had a checkbox for “my shipping address differs from my mailing address”? When you click that box a conditional form part gets revealed that allows you to enter another address. We had to build something very similar the other day and stumbled on a neat way to make the conditional part show and hide with CSS only.
Continue reading
Finding Pivotal
Monday, May 20, 2013
The year is 2005. I’m one year out of school, and a year into a job doing PHP web development at a small development firm in Dallas. A co-worker tells me jokingly about extreme programming. He laughs about the absurdity of pair programming and writing tests. Another developer goes rogue and develops an application in something called “Ruby on Rails.” He’s learning the framework at the same time he’s developing the application. I boast that I could have done it in half the time in PHP. That developer takes a job for a company in Seattle doing Ruby on Rails. I spend another frustating year at the company in Dallas.
Fast forward four years to 2009. I’m now living in Manhattan. Burned out on PHP, I’ve spent the last year doing free-lance Ruby on Rails development. I love it. It’s everything I wanted out of a language and framework. It solves all of the common, tedious problems that I faced developing in PHP, and lets me focus on developing my applications.
I land a job at a giant multi-national corporation. The team I’m working with is agile. They hold standups. They have a certified scrum master. They do two week iterations (on projects with fixed scope and hard deadlines). They estimate stories (in hours) (that they write themselves). They write tests (sometimes) (after they write their production code).
It was the best thing that had ever happened to me. And it was hard. And it was painful. We attempted to be agile within an organization of six-sigma blackbelts that would spend 18 months defining a process for defining processes. We didn’t know what we were doing half the time. We got a lot of stuff wrong. But we cared. And we learned. And we got better.
Fast forward three years. I’m still working for that giant company. I get an email from Pivotal Labs. A co-worker warns me, “You know they pair all the time.” I’m a little frightened. I go in for a day-long pairing interview. It’s amazing. I learn more in that one day than I had learned in the last year of work. I pair with developers smarter than me, better than me, and more experienced than me. I take the job.
Every day at Pivotal is like that first day, but even better. I learn something new every single day. I work with other engineers absolutely committed to developing great code. We give new meaning to the word “consultant”, giving our clients not just advice, but pairing with them to act on that advice (and course-correcting when things go wrong). We pair program. We test drive. We collaborate on story specification. We start each week looking at our priorities and estimating our stories. We end each week reflecting in a retrospective, talking about what’s working, what’s not, and what we should do about it. We take breaks throughout the day. We play ping-pong. We’re relentless about self-improvement, team-improvement, project-improvement, and company-improvement. And I’ve never been happier. I’ve never had the privilege of working with such a talented, passionate group of people.
We want to change the way the world develops software. You can too. Do the right thing. Do what works. Be kind.
Simple Test Parallelization
Monday, May 20, 2013
Let’s look at a simple approach to parallelizing a test suite for a Ruby app. Parallelizing your specs can be a good strategy to get a speedup on an existing slow suite. It can also be employed early on a greenfield project as part of a commitment to fast tests. The same caveats that Andrew mentions in that article post here as well, namely that parallelization might mask more important design changes you need to make in you suite.
While you could use a gem like parallel_tests, let’s look at what it would take to achieve this without needing to pull in another dependency.
The only requirement to employ this approach is that the parts of your build that you want to parallelize do not share a database, or if they do, that it will not cause test pollution if your specs run at the same time. These parallelizable portions of your build could be Rails engines, a library in lib, an unbuilt gem, or any other isolated piece of your app. If your app doesn’t meet that requirement, something like parallel_tests will likely be more useful.
Let’s assume that you are using engines to organize functionality in your app. In this case you likely are already using a separate database for each engine’s test suite, so let’s use that as a basis for our example. Assuming that you have two engines (engine1 and engine2) and they are both in the engines directory, you could write a rake task that parallelizes your build that looks something like this:
task :build do
build_pids = []
%w{engine1 engine2}.each do |engine_name|
build_pids << fork { exec "cd engines/#{engine_name} && rspec spec" }
end
trap(:INT) do
build_pids.each do |pid|
begin
Process.kill(:INT, pid)
rescue Errno::ESRCH
end
end
end
Process.waitall.each do |pid, status|
unless status.success?
puts "Build failed"
exit 1
end
end
puts "Build successful"
exit 0
end
This rake task does three things:
- Uses fork+exec to kick off child processes to run each engine’s build
- Collects the child processes after they have completed and exits non-zero if any of the child build processes were unsuccessful
- Captures INT so that all child build processes will be killed when you Ctrl-C in the terminal
The output can be ugly but may be worth the time savings, especially if you only are going to be running this task as a last check before CI.
Is there something funny about your Standup?
Monday, May 20, 2013
Agilish
Software development teams that aim to be more “Agile” often pick and choose the pieces of an agile methodology that suit them. For some reason standup is usually picked first, way before addressing their waterfall ways. I guess it’s because it’s hard to do “retrospective” but easy to stand up during a meeting (despite teams that sit through their standup).
I’m a big fan of standups but have also witnessed ones that are poorly executed or even detrimental. A traditional standup is a five to fifteen minute meeting where each team member stands together and answers three fundamental questions:
What did I accomplish yesterday?
What will I do today?
What obstacles are impeding my progress (blockers)?
A standup is not:
A status report for your managers
The forum for discussing details at length
A replacement for healthy and timely communication with your product team
A way to trick developers into coming in early
Walking the Wall
The intent of standup is to share work in progress and highlight blockers — why not use the tool where you manage your agile development process? This is typically referred to as “Walking the Wall,” and in my opinion creates a quicker and healthier conversation. Here’s how:
Open Pivotal Tracker (or your agile tracker of choice) on a screen that’s easily viewed by the team. Standup stations work well for this, otherwise you can huddle around the monitor with the most space. Close the Icebox and zoom in on Current.
Step through each story in the Accept/Reject state. Is it clear how a story should be accepted? Is it clear [for larger teams] who is doing the acceptance for a story? Side note for story requesters — do your best to accept stories right away. Nothing is worse for a developer than seeing a rejection at noon for a story that was delivered yesterday.
Moving onto the first story in progress, let the story owner describe the work and give a gut check on how it’s progressing. This is a great opportunity to raise concerns about an estimate, e.g., “This was pointed as a 2 but we had to do a lot of refactoring of the css. We’ve probably got another day of refactoring before we deliver.” Context is important, and highlighting this discrepancy gives the PM a chance to course correct if need be.
Once you’ve covered WIP, move on to the next set of stories in the Backlog. If any have blockers like a “needs-discussion” label, drill into the detail and identify if you can resolve them on the spot. If something needs design/assets, hopefully your designer (along with the rest of the core Product team) is there to confirm that they’ll upload the right files, etc.
Lastly, work out your pairs for the day and cover any housekeeping (“we’ll be reticulating the spines at 3pm, no pushing commits”)
If some important topic comes up and can’t be resolved within fifteen minutes, schedule a follow-up meeting with the people needed to resolve the issue. You don’t get points for dragging out a standup until every problem has been solved!
This is the format for standup that I default to now. I encourage you to give it a try if you’re finding standups have lost their value, or if you don’t think product stakeholders are in sync with the development process. As always, ask your doctor if continuous improvement is right for your team.
Debugging in RubyMine
Monday, May 20, 2013
Before using RubyMine, my debugging workflow went something like this:
Why isn’t this test passing?. It should’ve passed. Let me add a few
Kernel#putscalls to see what’s going on. Hmmm, ok, it’s still failing. I’m going to need some more output. Man, these tests take forever to run. Syntax error?!. Ugh, typo…
Debugging like this isn’t fun; especially when pairing. Fortunately, RubyMine can help. RubyMine includes a standard debugger that has minimal setup and works out of the box. I still don’t enjoy debugging, but RubyMine has made it much less painful. In this post, we’ll look at debugging in RubyMine on OS X.
Setting Breakpoints
Add or remove breakpoints with command + F8.
View existing breakpoints with command + shift + F8.
Starting the Debugger
Debug the current program with control + shift + D. This will automatically open the Debug tool window (command + 5).
Use control + D to re-run the last debug session. View recent debug sessions with control + alt/option + D.
Examining a Program
Examine variables in the Debug tool window’s variables pane.
Use command + N in the Debug tool window’s Watches pane to watch a specific variable. backspace will delete a watch.
Use alt/option + F8 to evaluate arbitrary expressions. control + space triggers autocompletion.
Instead of using the Debug tool window, examine a variable inline by placing your cursor on it and pressing command + alt/option + F8.
Stepping through a Program
Step into a method with F7. Step over a method with F8. Step out of a method with shift + F8.
Use your cursor as a temporary breakpoint with alt/option + F9. Continue your debug session with F9.
Stop #puts’ing
If you’re using an IDE, take time to learn its debugger. Don’t litter your code with Kernel#puts and friends. Instead, set a breakpoint, start the debugger, examine some objects, and slowly step through your problems.
A Responsible Recipe for the Fewest Possible Meetings
Monday, May 20, 2013
Meetings are crucial to healthy team communication. But they’re also opportunities for waste, occasionally dull, and always expensive. Every team is different, but continuing the theme of “Convention over configuration for process”, I’ve found the following structure keeps meetings to ~7.5% of your week. This minimizes waste, maximizes making-time, and makes for a nice scheduling default for projects.
1. Daily Standup
5x weekly, 10 minutes each
Standup is a short, team-wide orientation meeting. Each morning, the team meets for a the Team Stand-Up meeting. As it’s name implies, everyone should be standing up—even if they’re remote and parked behind a screen. Meetings tend to run short when everyone’s on their feet, and there isn’t much to cover anyway, just three canonical questions:
- What did you do yesterday?
- What are you doing today?
- Are you blocked on anything?
Since the team is pairing, most of the time the second person will say “pass” since their pair already mentioned the stories they worked on, the issues they had, and what they’re doing next: sticking together or seeking a new pairing partner.
2. Iteration Planning Meeting (“IPM”)
1x weekly, 60 minutes each
The IPM is a weekly tactical planning meeting. It’s aim is to ensure that the backlog is in good shape, and that all the team members have a shared understanding of what the stories mean. The product manager leads the meeting by reviewing the backlog. Each story should be reviewed; questions can be answered, clarifications can be issued, and developers can estimate the stories. Ideally, the meeting is on Monday or Tuesday and the team will get through the current iteration’s undone stories and a healthy chunk of next iteration’s stories. On some teams, it helps to take an hour before the IPM to groom the backlog, prioritize, and flesh out poorly-worded stories in a pre-IPM meeting with a small group (usually the Product Owner and a single developer).
3. Friday Afternoon Meeting
1x weekly, 60 minutes each
A useful pattern on recent projects has been to block out an hour Friday afternoon and rotate between three meetings: the Team Retro, the Tech Retro, and the Release Planning. It’s nice to have a regular time, and every-three-weeks is about the right frequency for each of these.
Week 1: Team Retro
The Team Retro is a chance for the day-to-day members of the team to reflect on how things are going, celebrate successes, voice frustrations, and suggest Action Items. While there are plenty of variations, the classic Retro format is to throw three columns on a whiteboard:
- “Smileys”, or things that are going well, represented by the expected emoticon :-)
- “Frownies”, or things that are going poorly, represented by :-(
- “Mehs”, or things that don’t fit cleanly in either category, but bear mentioning :-\
After ~25m of going around the room and throwing out Smileys, Frownies, and Meh’s, the team might spend a few minutes identifying items which have a common theme, and then spend the rest of the time suggesting action items, either for each item (starting with the Frownies) or, if time is short, on a theme-by-theme basis. The action items are recorded and assigned to individuals, and their progress is followed (often at the start of the next retro). For more about Team Retros, see 7 Best Practices for Facilitating Agile Retrospectives.
Week 2: Tech Retro
A Tech Retro is for and by developers. While pair-programming is essentially continuous code review, it can still be useful to take some time to step back and look at the codebase. Tech Retros often take the traditional “Smilely / Frownie / Meh” format, but focus exclusively on the codebase. This is a great time to talk about modeling decisions and suggest refactors. Often the action items resulting from tech retros are a long list of chores. Try to keep each one to a manageable size, and make sure at least a few of them make it into the backlog for the next iteration.
Week 3: Release Planning
“Release Planning” means different things to different Agilists, but in this context it’s a loose title for a long-range planning meeting. While the IPM is a short-term tactical meeting, it’s important to occasionally step back and look at the big picture. What are the product’s medium- and long-term goals? Are we on track? What should our next milestone be? The Release Planning should be attended by the whole team, and by the stakeholders like clients and CEOs who may not be part of the day-to-day workings of the team. It’s a chance to review epic story tracks, dates and deadlines, and to make sure that everyone has a shared vision of where the team is going—and that it’s a place worth going to.
Why Meetings Matter
Communication is crucial, but so striking a balance between waste and on keeping everyone on the same page. I’ve found that teams which hew to this schedule tend to have a healthy rhythm. While circumstances occasionally require additional meetings, especially on the part of the Product Owner or the outward-facing members of the team, this structure is sufficient to maintain healthy intra-team communication without taking developers away from code any more than necessary.
Do you have another default meeting structure that you like? Let’s hear about it in the comments!
Put the User in User Stories
Monday, May 20, 2013
I’ve often been in situations where people see user stories in Pivotal Tracker as just a punch list of to-dos, line items for snippets of code we can write that does one little thing in a product owner’s head. Even that much is useful for its prioritization and clarity, but thanks to a great chat last week with Lean User Experience wizard Josh Wexler, I’ve found myself paying closer attention to the reason they’re called User Stories in the first place: the users.
Complicated business logic becomes much easier to grok when you have a clear scenario in mind for the person who’s using it. I was recently working on a piece of particularly thorny logic with a new client developer for Case Commons. It had to do with subtle requirements for when and how to get permission to update a foster family’s license. We read and reread the story and the explanation, but it was hard to communicate why we were doing the thing we needed to do. We were talking in circles, and as soon as that happened, we knew we were doing it wrong.
So I pulled out the blog post I wrote a month ago, and it turned out that my friend’s family and newly adopted children were in an almost identical scenario to the one we were working on. I pulled up pictures of my friends and their adopted kids to show to the guy I was pairing with, and described the social worker, Tania, who had connected them with their new family. From then on, it wasn’t some abstract user we were talking about, it was Tania. My pair and I discussed what would happen if my friends became able to adopt more kids, the process Tania would have to go through to make that happen, and where that information would go on the physical, printed foster family license we were implementing. Having faces and names to go with our code made all the pieces click together, when we see as the story title in tracker: “Foster Family Licensing worker sees checklist only if capacity has changed”
I’m looking forward to putting more users into more user stories to make them come to life. When each story is complete, what new awesome thing will that person be able to do, and what need of theirs will it fulfill?
RDS can’t fail
Monday, May 20, 2013
Helps
Making snapshots fail on AWS
We're trying to help our PM accept a story that requires a snapshot of a persistent disk to fail on AWS. Does anyone know a good way to make this happen or fake making this happen?
Events
Monday: Muse user group
This Tuesday is our second muse monthly user group at adobe sf.
Free: get badge: http://www.mugsf.com/
Blog:
http://mugsf.tumblr.com/
Org Chart Growth and Keeping Our “Flatness”
Monday, May 20, 2013
We’re growing. You may have heard that we were acquired in 2012. You might also have heard that we’ve been spun out again as a core part of a new company that shares part of our name. From when I started at Pivotal Labs back in 2009, we’ve grown by a factor of six just within ‘Labs and we’re now part of an organization that’s twelve hundred strong. If you were watching our org chart during this time you might have wondered where we got the magic beans for that growing bean stalk.
When I started four years ago we all “reported” to one person, and we talked a lot about being a “flat” organization. Then we got multiple offices, and added a local director layer. Then office head count grew beyond one director having a close relationship with all of the pivots and keeping up with director duties, so we added managers. Then we got enough managers and head count still growing that we needed associate directors. At the same time we got two more levels added on top of the head of our company when we were spun out. Given all of this change, you can relate to my astonishment when just last week I heard a pivot say, “We’re a flat organization.” Double woah.
For a pivot, one who is eight degrees of separation away from the head of the company, to say “we’re flat” means he’s feeling something that isn’t reflected in an org chart. As he talked more about it, he clarified that in terms of day-to-day individual, pair, and team activities there is still the same huge amount of autonomy. There’s no boss or manager in the traditional sense telling you what to do. You get to decide as a team what the right priorities are. You and your pair get to decide if you should do this refactoring now, or raise it up to the rest of the team as part of a larger effort. In terms of how you work on your project, you have as much influence as anyone else. The flatness is still the same in many ways as it was four years and a thousand people ago.
Moving up, let’s say you want to affect some change at the office level. Your first option is to go one step up the on-paper org chart, but really it’s a lateral move: Talk to your manager. At Pivotal Labs, your manager is more of a buddy or coach. Your manager is not on your project, your manager does not assign you work. He’s there to help you process feedback, help you find new ways to use your skills that also help the company, and navigate the company infrastructure so that you don’t have to keep it all in your head. Your manager might give you some advice on things to try, people to talk to, or green light some pizza budget so you can host an event after hours. Otherwise he might encourage you to talk directly to the office director who always has a door open for pivots who want to make their workplace better – and note that you’re skipping right over the manager and associate director levels.
From there, if you’re looking to make some cross office improvements, you’re just one door away from the COO of Pivotal Labs. At this point you probably can’t just drop in and instead you’ll have to schedule something on his calendar, but he’s the kind of guy that’s easy to trust. What I always find fascinating is that I have yet to surprise him – like you might expect a COO to do, he’s always thinking about making the company better and he’s good at finding ways to put passionate pivots to use. Similarly, you can get lunch or coffee with our founder who is a great example of an active listener.
Moving up to the company level I haven’t had any direct experience with those folks, but I know they expect the Pivotal Labs Way to become the Pivotal Way the same that we’ve always expected the Pivotal Labs Way to become the way our clients do software development. The way we’ve accomplished that in the past is by pairing with them on their projects. It is for this reason that I’ve asked to do a rotation on CloudFoundry, one of our core projects from Pivotal. By sharing my practices, I hope they carry them forward as they engage with other parts of Pivotal and those teams with the teams they overlap with, and so on.
Yes, the org chart is a lot deeper now than it once was. It’s clear though that pivots still have a voice and they still have all the autonomy to manage their own day-to-day. What’s interesting is that now my lever arm is potentially even longer than it was before because I’m now at the center of an organization that is designed around the way I work and I have an opportunity to shape the way more people make software than ever before.