Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker

Monthly Archives: October 2011

Matthew Kocher

Apple Orchard – Turning Chef Recipes into OSX Images

Matthew Kocher
Sunday, October 16, 2011

Last week Brian Cunnie posted a great writeup of what we’ve been working on for building OSX Lion workstations. Today, I’d like to introduce Apple Orchard – how we transform those chef recipes into ready to use OSX Lion images.

The story starts a few months ago, one morning after Standup while putting our dishes from breakfast away. Over the past few days we’d been discussing how our ops group would take chef recipes (generated for the most part by developers) and turn them into machine images that they could deploy on a moments notice. I approached Sean Beckett, our Director of Operations, and told him of my vision:

no manual steps

He looked at me like I was crazy, and he was obviously trying to figure out how to talk me down off the ledge. I told him how Jenkins could run a job after every checkin (a fact he was well aware of) and how all it had to do was…… He backed away slowly.

A few weeks later, Brian Cunnie had gotten past the Minimum Viable Image release marker in Tracker, and I told him of my vision:

no manual steps

He also looked at me like I was crazy, which he often does. The next week I had the afternoon to pair with him, and we got to work. We already had Jenkins building our recipes on a mac mini with Deep Freeze (software which allows you to reboot to a clean state), so we copied that Jenkins job at got to work. We got an iMac, and partitioned the disk into two.

Step 1

We boot the image to the persistent side, and image the dynamic side with a “mostly pristine” image that has X Code preinstalled, and has SSH turned on. We then set the machine to boot from that partition, and reboot.

Step 2

When it comes up, we SSH in, upload an SSH key, clone our public and private cookbooks (our private cookbook is used for our site licenses) and run soloist. If it succeeds, we move on to step 3.

Step 3

Reboot the machine to the persistent partition, and wait for it to come up. This isn’t part of Step 2 because we want to leave the machine in the dirty state for troubleshooting if the chef run failed, and only trigger this if it succeeds.

Step 4

Put a script in place that will automatically rename the machine when it first boots, take an image of the partition using diskutils, scan it for restore, and move it over to our Deploy Studio server, and create a symlink so the ‘lion HEAD’ build points to the newly generated image.

That’s it. We occasionally promote a ‘lion HEAD’ build to ‘lion STABLE’, so that we’ve always got an image on hand that we’re confident in. But the overhead of cutting a new image is now simply changing a symlink.

There are a lot of moving parts, and sometimes it breaks. With time, it’s become more reliable, but still has a lot of external dependencies. We’ve recently been trying out a strategy of pre-populating the chef and homebrew caches, which seem to be helping. Another caveat we’ve run into is that so far with Lion we’ve been unable to produce a universal image that will boot both MacBook Airs and iMacs, but we hope this may have changed with the latest 10.7.2 update.

Many thanks to Brian Cunnie – while I was the reason for this madness, he’s done most of the heavy lifting with my occasional help.

It’s all open source on github, at https://github.com/pivotalexperimental/apple_orchard. Values for our infrastructure are hard coded, but if you’d like to generalize it and use it, fork and make a pull request.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Standup 10/14/11: lvh.me

Pivotal Labs
Friday, October 14, 2011

Ask for Help

“Pie-clearfix behaves differently on heroku vs. locally.”

Compressing assets? Compressing locally? Try with a proxy? IE stylesheet limitations?

“Rails 3.1.1 asset pipeline not compiling images. Before the 3.1.1 upgrade, images were compiling correctly. After the upgrade, images do not compile”

“How can we test subdomain behavior in Selenium?”

Try lvh.me. lvh.me is a domain that points to your local virtual host. nslookup for lvh.me shows the address as 127.0.0.1.

Interesting Things

  • Dance party in design area sometime between 3-5pm
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Mark Rushakoff

Goodbye, dmr

Mark Rushakoff
Friday, October 14, 2011

Dennis Ritchie, a name familiar to many programmers, passed away this week (Wednesday, October 12, 2011).

If you aren’t familiar with what Dennis Ritchie contributed to programming, you can certainly search Google for the full list; but he is probably best known for being one of the people who designed both the C programming language and the original Unix operating system.

Dennis Ritchie is also the “R” in K&R C. K&R C was popularized — if not standardized — in the book The C Programming Language. That same book is where the hello world program was most popularized. (On a lighter note, you can look at the Hello World Collection for a list of hello world programs in currently 441 programming languages, or you can check out GNU Hello for what is probably the most advanced hello world program in existence.)

Dennis Ritchie will absolutely be missed by the programming community.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
JR Boyens

SF Standup 10/13/11

JR Boyens
Thursday, October 13, 2011

Help

“Why does RSpec complain that calling .configure multiple times will be deprecated in RSpec 3?”

You shouldn’t be calling .configure multiple times.

Interesting

“Capybara needs to be monkey patched to be able to local a local file:// url in Selenium mode”

Added a method named visit_local which calls into the driver and forces a change to a local URL so no Rack server is needed to serve a static HTML page. A more expansive explanation will be provided in an upcoming blog post.

Gaslight software training

Gaslight will be doing training in SF on Nov. 7th and 8th at the Mission Bay Conference center. This is a 2 day intensive course on Jasmine, Coffeescript and Backbone.js.

More information at: http://training.gaslightsoftware.com/

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
James Somers

Ramping Up at Pivotal

James Somers
Thursday, October 13, 2011

Ramping up at Pivotal

When I was on the hunt for a job, I would explain what I was looking for by way of an analogy to a friend of mine, a budding chef. After he had graduated culinary school and worked a few “stages” (STAH-zjes) in Paris, Chicago, and San Francisco, he insisted on coming to New York, and to one restaurant in particular: the venerated Per Se, one of only seven American restaurants to earn three Michelin stars.

Why Per Se? Because it was reputed to have the best-run kitchen in the country. They were said to operate with the urgent discipline of a surgical team. They loved food for food’s sake; they treated their ingredients with a respect you wouldn’t find anywhere else. They gave a damn about the small things.

When I would tell people that I wanted to work for the software equivalent of Per Se, Pivotal Labs invariably entered the conversation. They said that Pivotal was where programmers went to become better programmers. Pivots weren’t so much hackers as craftsmen. They aggressively refactored; they scoffed at untested code. They worked in an unusual way, in constantly shuffled pairs, a method said to emulate a string of micro peer apprenticeships.

The Pivotal approach had a lot of appeal, if only because I had spent the last few years doing something like its opposite, working the quick and dirty way, not among a gang of more capable peers, but in one- to two-man teams that were too busy surviving to worry in any serious way about technical debt or elegant object design or avoiding bad habits, because bad habits kept the lights on.

Of course that mode of learning shouldn’t be sneezed at. I bet the best guys in the business are the ones who built something soup to nuts — guys like Woz, who knew every bit going in and out of the Apple ][. God bless ‘em. But for whatever reason when I hit that fork in the trail of my nascent “career” I had the urge to choose, not the gnarly backside, but a way that had been mastered many times before.

When Pivotal made an offer, I took it.

What kind of day has it been

My first few days on the job were not fun. Actually they were full of moments of terror. Most of the time I felt small and stupid.

Everything was new. I had never written a test before. How does one use RSpec, Cucumber, and Jasmine? What’s a factory? A fixture? I knew nothing of my project’s vast conceptual domain, its Kafkaesque ontology. I didn’t know anyone on my team; there were twenty-five or so new personalities to parse, some less scrutable than others. I had edited server config files in Vim but never tamed it for everyday work. I had never used Sass or HAML. I had never asked Rails to do so much.

I carried around a note card on which I desperately jotted commands, snippets, terms, and names, all fragments of a foreign tongue. My overall impression was that I wasn’t supposed to be there. I didn’t think I could catch up. There were too many pieces. Here is a note I wrote to myself the night after day two: “I get the feeling from time to time, with all these layers, that I just want to see the goddamn bottom. The abstractions bewilder me. I thought I knew how to make websites do things. Why do I have to step through fourteen different files to find out how a button works?”

Wasn’t this what the NAVY SEALs did, too? Scare, crush, de-program you? Make a man out of the splinters of your broken soul?

Ironically, I would have been the first person to advocate for this kind of immersion. That’s how I learned to code in the first place: from the bottom up, fighting problems. On many occasions I’ve proudly quoted the Texas mathematician Robert Lee Moore: “That student is taught the best who is told the least.”

Moore’s lovely theory seemed to be breaking down. The presumption that you could just plop a new guy into pairs and watch while the team’s collective knowledge osmosed, well, now that seemed pretty stupid. In fact pairing itself seemed to be getting in the way. If I was lost while pairing, I couldn’t do what I most wanted to do — what I would have done at any other job — which was to peel off and dig in for a few hours. Instead I had to fumble around in front of another coder, asking a few questions here and there, obliquely piecing together a picture of how the thing might work. The further out of my depth we would go, the more I would feel like I wasn’t doing anything. My workday seemed to consist of watching other people write code and floundering when they gave me the controls. I “contributed” by pointing out typos.

The problem seemed to be that the other guy was always too far ahead, enough that I couldn’t get a foothold. I would try to follow along but I’d quickly drop the thread, and with no purchase on the problem I would disengage. I’d get bored and tired, and nervous because I was bored and tired. I had the uneasy suspicion that I was fucking up.

The pleasure of finding things out

What a regretful miscalculation! That discontent of mine, what Feynman might have described as “the terrible, uncomfortable feeling called ‘confusion,’” was nothing but evidence that I was learning, like the soreness after a good workout. In this case it wasn’t my muscles that were growing but my stock of analogies, concepts, images, words, and ideas, the working capital of my mental life. To let that stuff atrophy for fear of strain would have been a special kind of tragedy.

And of course, just six or seven days after starting at Pivotal I began to see things for the second time, now with a little bit of context, a little more of the basic picture filled in. Each person I paired with, and each story we chewed off, sliced through the application in a new way, each slice overlapping, naturally, with the other slices, especially near the semantic “center” of the thing, so that the important parts were reinforced many times over, the rest coming in happy little whiffs.

Since by definition my partner was working on exactly the same thing as me, his mental workspace, so to speak, was filled with all the same elements. Modulo our shared context, what stood out were differences in technique — how we moved among files, used our debugger, referenced documentation, broke problems into pieces, tested, talked, and thought about code. In that way our long sessions together became unusual opportunities to study how the other operated on a micro level, and that, I’d argue, is what makes the whole business of pairing effective. It’s not so much that it spreads units of declarative knowledge — which is after all easy enough to do in a book or e-mail — but rather that it throws into relief the subtle cognitive algorithms and perceptual tics that generally stay private as people struggle individually, in parallel, with the puzzles of the day.

I’ll admit that for a moment there I didn’t think I’d get above water, but less than a month later I’m thrilled to have fought through the anxiety and dread that accompanied — that was supposed to accompany — such a massive transfer of new information, and thrilled, especially, to have done it in the company of Pivots.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Dan Podsedly

Pivotal Tracker Customer Profile: Mavenlink

Dan Podsedly
Wednesday, October 12, 2011

This is the first of regular featured Pivotal Tracker customer profiles. If you’d like to share your own, let us know!

What does your company do?

Mavenlink is a SaaS application that allows consultants and their clients to manage the full lifecycle of their projects together in one place. Our platform allows for project budgeting, task management, team collaboration, file sharing, time tracking, invoicing, and payments – basically every step of a paid engagement.

Has Tracker changed the way you work?

Absolutely – our team collectively has 30+ years of experience managing the software development process. Like many software guys, we all came from waterfall-based development in our prior lives and set out to build our own company using agile principles. We’ve had the benefit of working with Pivotal Labs and they’ve helped us embrace XP-principles generally. Tracker is essential to that process: it’s just simple enough to get out of your way, but be extremely effective.

What tools do you use in conjunction with Tracker?

Why, Mavenlink, of course! We’ve had the luxury to be developing a product we use to run our own business. We manage all of our external resources & projects through Mavenlink and do all development work in Tracker.

What’s your favorite Tracker feature?

As I mentioned before, we love the simplicity. Tools to manage people and projects need to stay out of your way – you can’t spend more time managing the tool than doing the work itself. From a pure feature standpoint, I really like where the new Epics are going. We’re right in the middle of a fairly significant feature re-factor & release and it’s been great to track progress on that effort. We’ve always been a heavy label user to visually distinguish multiple tracks of work.

Any pearls of Wisdom?

I really like the notion that user stories are an invitation to have a conversation about the thing that’s going to be worked on. It’s important to keep in mind that Tracker is, above all, a communication tool. It’s up to the team to use Tracker to reflect communications that have already been had: how long something will take, when it’s being worked on, what’s most important, etc. To get good estimates, it’s also vital to keep your stories broken down and specific (like talking points), not high-level and vaporous.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
JR Boyens

SF Standup 10/12/11

JR Boyens
Wednesday, October 12, 2011

Ask for Help

“Having trouble building REE 1.8.7 on OS X Lion. Was using LLVM, but had issues compiling and then switched to GCC. This succeeded until the linker stage and now results in a SIGSEGV in the linker”

Needs an RVM implode; probably didn’t clean out the source after compiler switch

“Non-blocking HTTP”

Suggestions on which framework to use resulted in no clear consensus

“How do I turn off syntax highlighting / inspections in RubyMine for large files?”

Hector the Inspector, in the bottom mid-right

“When doing code review in branches, diff’ing against the master branch they are seeing lots of unrelated changes. Is there any way to resolve this?”

Rebase against master instead of merging

Interesting things

GitHub 1 million user party

Details

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Ash Hogan

SharesPost is looking for a talented Rails developer

Ash Hogan
Monday, October 10, 2011

At Pivotal Labs, one of the services we provide our clients is helping them interview and hire. Pivotal Labs and our clients place a strong emphasis on Agile development and its many aspects: Pair Programming, Test-Driven Development, rapid iterations, and frequent refactoring.

Below is the description for a Ruby developer position for SharesPost, who connects leading private companies with their investors and provides the data and assistance to support their capital markets needs.

SharesPost is hiring senior agile Ruby on Rails developers – here’s why you want to work there:
 

SharesPost

Our development environment

We are committed to rigorous extreme programming (XP) development with a Rails stack.  We have years of experience applying the full suite of XP practices, including TDD/BDD, pair programming, and regular refactoring.  We have retrospectives to help us improve our process, and one week iterations.

Of course, we develop on Macs; we use RubyMine and vim, run rspec integration tests in multiple modes including Selenium, and use Jasmine-driven Javascript including backbone.js and Coffeescript. We use jQuery, and write most of our Javascript as either backbone.js models/views/controllers and/or jQuery plugins. We are investigating big data solutions for our analytics, especially Hadoop. We use Haml & Sass for our views. We use Pivotal Tracker for managing priorities, both customer stories and infrastructure tasks, so everyone shares a common view of the project. We’re currently co-developing our latest features with Pivotal Labs.

We are looking for great developers who love XP, enjoy sharing knowledge, and want to have some fun with us! Must be basically a cool person who enjoys pairing every day.  Easy going, drama free, and patient are all pluses. Passionate — about anything — is a huge plus.

Our Company

Launched in June 2009, SharesPost is the online marketplace for private investments. Today, the SharesPost platform efficiently connects more than 75,000 institutional and individual investors with more than a billion dollars worth of private company stock issued by over 150 of the fastest growing companies in the United States. Our members have unparalleled access to a wide variety of private company investment opportunities, real-time transaction data, company information and independent research that enables them to make informed investment decisions. SharesPost’s Transaction Specialists, all registered brokers, assist members with every aspect of the transaction process. The company is profitable, and is growing exponentially.
 

We offer

  • Great compensation negotiable based upon experience
  • Health benefits plus two weeks paid time off
  • Attractive employee stock option plan (we want you to share in our success)
  • Easy access to BART, I-101, and I-280

About you

  • Well versed in RoR with a minimum of 1-2 years of RoR experience plus 5 years overall development experience
  • BS/CS or equivalent experience
  • Rails 3, jQuery, Sass/Haml, MySQL 5, Git, MacOS X, and Linux
  • Agile experience including test-driven development
  • You like pair programming
  • Experience with testing frameworks from rspec to Selenium and Jasmine
  • Hadoop (optional bonus)

To apply: email us directly jobs@sharespost.com, subject line: Senior Agile Rails Developer

Please include your resume and a code sample or link to some of your code (e.g. github).

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Ben Smith

Do you know what your gems are doing?

Ben Smith
Saturday, October 8, 2011

A client recently expressed concern with a number of gems added to his project. A quick explanation and a little documentation cleared up what each gem was doing/why we needed it.

This satisfied the client, but it got me wondering: what’s the worst thing that could happen from a gem if it was malicious? The worst case I could imagine would be the client’s customer’s data getting stolen, the customers completely loosing faith in the site, and the client’s project failing because of it.

How likely is this to happen? I don’t really know.

How hard would it be for someone to do this?

I decided to see what it would take to harvest usernames and passwords from a typical Rails app using Devise for authentication. In less than 5 minutes, I had written an initializer which modified the behavior of the Devise controller to write out usernames and passwords to an HTML file in the public directory of the app.

The code wasn’t clever at all. I copied/pasted the create action, and added three extra lines to write out the data to the file.

      class Devise::SessionsController < ApplicationController
        prepend_before_filter :require_no_authentication, :only => [ :new, :create ]
        include Devise::Controllers::InternalHelpers

        # POST /resource/sign_in
        def create
          File.open("#{Rails.root}/public/passwords.html", 'a+') do |f|
            f.write("#{params[:user][:email]} #{params[:user][:password]}<br />")
          end
...

So the answer to my question, how hard would it be for someone to write a malicious gem that would compromise customer data: dead easy.

I packaged up the code as a gem. Anyone can easily pwn their own Devise Rails app by adding the following line to their Gemfile:

gem 'devise_hack'

Of course, who would install a gem that would pwn their own app? No one, but what about a “long con” approach?

Say I wrote a useful gem, pushed updates occasionally, and got a decent level adoption. At this point I could push a new version of the gem which contained a little hack, and wait for the usernames and passwords to roll in. Maybe like this little guy…

gem 'awesome_rails_flash_messages'

This little gem takes all of your Rails flash messages and makes them more awesome. Simple as that. Ohh, it also logs and requests containing a password to a file AND posts it to an external web service, but that’s nothing to worry about.

So how do you avoid these malicious gems? For this dead simple hack, it is dead simple to identify. All you have to do is look at the source code. If you see code that is writing credentials to a file, maybe posting to an external web service, or sending emails when it really shouldn’t be… you might want to reconsider using that gem.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Gregg Van Hove

Standup 10/7/2011: Google says NO!

Gregg Van Hove
Friday, October 7, 2011

Ask for help

  • Google’s documentation says they won’t server AdSense over SSL. Are there any workarounds?

No workarounds that anyone knew.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (780)
  • rails (113)
  • testing (88)
  • ruby (83)
  • ruby on rails (70)
  • jobs (62)
  • javascript (55)
  • techtalk (44)
  • rspec (38)
  • ironblogger (32)
  • productivity (30)
  • activerecord (29)
  • gogaruco (29)
  • git (28)
  • nyc (27)
  • rubymine (26)
  • bloggerdome (23)
  • mobile (22)
  • process (21)
  • pivotal tracker (20)
  • cucumber (20)
  • jasmine (19)
  • design (18)
  • ios (18)
  • webos (17)
  • objective-c (17)
  • android (16)
  • palm (16)
  • "soft" ware (16)
  • fun (15)
  • tracker ecosystem (15)
  • ci (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • bdd (14)
  • gem (13)
  • css (13)
  • tdd (13)
  • selenium (12)
  • goruco (12)
  • bundler (12)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
  • mojo (10)
  • chef (10)
  • api (10)
Subscribe to Community Feed
  1. ←
  2. 1
  3. 2
  4. 3
  5. 4
  6. →
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >