Abstract

When we moved to a new office, we faced a problem: how do we give printer access to everyone even though we had segregated machines to different networks? And how do we make it transparent to the user?

The solution we found was to add a new VLAN (i.e. network segment) for the printers (and other common resources, e.g. license servers), and use DNS Service Discovery (dns-sd) and a handful of crafted records to our DNS server.

This blog post is directed at Operations staff at companies which have the following characteristics:

• primarily use Apple workstations
• have network-attached printers
• use Bonjour for printer discovery
• need to print from multiple VLANs

The Problem

The short version: people couldn't print from the WiFi network.

Ask for Help

"Is there an easy way to white-label a domain when using SSL?"

Short answer: not really. One way would be to get a different IP and SSL certs. There are still some hurdles when sharing sessions across domains. Cookies can't be read across sites, either.

Interesting Things

• When deploying to Heroku using the Cedar Stack, BUNDLE_WITHOUT is not supported, but you can detect heroku in your Gemfile with ENV['HOME'].gsub('/','') == "app", and then substitute any group like staging/cucumber with "test" so they are not installed via the default bundle task on heroku.
• Rails 3 Asset Pipeline allows you to extend the compilation of additional files. Registering your own extension is easy but poorly-documented: you create a class that handles the compilation and register it with the asset pipeline.

Ask for Help

"The Twitter API has callbacks which return tweet IDs; however, we're not seeing the IDs."

"nginx upload module: does it buffer to memory and then stream to disk?"

The upload module by itself streams to disk; have used it to upload multi-gigabyte files, which would have caused problems if it was buffering to memory.

"Is there a way to undelete stories in tracker?"

Look at the history, it will show what stories were recently deleted. Export/import might help, too. Try contacting the Tracker team.

Interesting Things

• Webmock + RubyMine == fail. This applies both to RubyMine-stable and RubyMine-EAP. You can monkeypatch around it by writing your own initializer. It may be a require-order problem.

Ask for Help

"Does anyone really understand character encoding in Ruby? Certain gmails are causing exceptions in our applications, and it seems to be a problem with gmail specifying the wrong encoding. Whatever is causing it, it's fixed in 1.9.2."

"When calling render_to_string in an AJAX request, rails does not find partials ending in .html.erb. Is there anyway to force it to find them other than requesting a filename.html.erb?"

"What's better to use to handle attachments—paperclip or CarrierWave?"

They're both good. It seems that CarrierWave is better and more recently updated.

So you have a nice new Apple machine running Lion, but you don't want to spend the next few hours installing software.

What to do?

The short answer: use chef/soloist in conjunction with a slew of recipes developed at Pivotal Labs to help install the most common set of components.

Here's what to do:

1. make sure you have Xcode installed; if not, you can install it through the App Store. Don't worry--it's a free application.

2. Do the following:

   sudo gem install soloist
   mkdir -p ~/workspace
   cd ~/workspace
   git clone https://github.com/pivotal/pivotal_workstation.git
   cat > ~/soloistrc <<EOF
   cookbook_paths:
   - $HOME/workspace
   recipes:
   - pivotal_workstation::meta_osx_base
   - pivotal_workstation::meta_osx_development
   - pivotal_workstation::meta_ruby_development
   EOF
   soloist
   

3. It typically takes an hour for the chef run to complete. You can do other things while it's running (but if you reboot or logout you'll need to restart the chef run).

[Chef is a framework written by OpsCode to help configure in maintain one or more machines using 'recipes' (ruby scripts, more or less).]

[Soloist is an application which makes running chef-solo (the version of chef which runs without any centralized server) easier. It was written by my co-worker Matthew Kocher.]

What Is Installed by Default??

At the end of the chef run [as of this writing, and if all goes well] the following software will be installed:

• java
• Skype
• SizeUp
• Firefox
• DropBox
• Chrome
• Mouse Locator
• Homebrew
• Pivotal Git scripts
• Gitx
• Vim
• CCMenu
• TextMate
• KeyCastr
• rvm
• MySQL
• postgres
• RubyMine
• ImageMagick
• node.js

The following services will be enabled:

• sshd
• screen sharing
• locate

The following preferences will be set:

• better bash (history, PS1, inputrc)
• faster key repeat
• git (settings, a few scripts)
• TextMate preferences (e.g. soft tabs)
• RubyMine preferences (e.g. keymaps)

What if I don't want all that software? I just want TextMate and node.js!

Want to change the software that is installed? It's simple: just change your ~/soloistrc file. Here's a soloistrc that will only install TextMate & node.js:

cookbook_paths:
- ./workspace
recipes:
- pivotal_workstation::textmate
- pivotal_workstation::node_js

If you're interested in seeing all the recipes available (and there are quite a few), just browse the recipes in the pivotal_workstation repo.

Why Did You Choose That Set of Software?

Early in June, several pivots (Sean Beckett, Matthew Kocher, and David Goudreau, and I) met to decide on the bare minimum set of software and features that our developers would need to function on a new Lion Machine.

This set of chef recipes is the result of that meeting. There have been some changes (we have had great difficulty writing recipes to install firefox addons, so we iceboxed the story; some of our developers contributed recipes for things they wanted, so we added those).

Why Did You Choose Chef?

We chose chef/soloist partly because felt that our previous process had reached the end of its usefulness and were familiar with chef from our work automating server configuration.

Here's how our previous process worked:

1. For minor releases (e.g. 10.6.7 → 10.6.8), we would take the previous golden image (a golden image is a snapshot of the disk drive of a machine with the applications, preferences, and settings that we wanted), install it on a workstation, upgrade the OS and possibly upgrade some of the applications. We would then use DeployStudio to take an image of the workstation, and that image would become the new golden image.
2. For major releases (e.g. 10.5 → 10.6), we would re-create the golden image by hand, manually installing & configuring the individual software. For the Leopard/Snow Leopard transition, my co-worker Kevin Fitzpatrick spent a week painstakingly configuring the new machine. We then took an image using DeployStudio, and that image became the golden image.

This approach had several shortcomings:

• It was monolithic: if you were a developer, there was no choice: there was only one image. This wasn't so bad when we were strictly a ruby shop, but when we expanded into android and iphone development, the monolithic approach began to show some shortcomings.
• There was cruft in the image: the golden image had been built up over years.
• It wasn't clear that the Golden Image would make the jump to Lion: Lion introduced some big changes (e.g. no PPC executables).
• We were hesitant to approach the developers to ask them what they would like to see on the new image; we worried of re-igniting a Holy War.

We looked for alternatives. We wanted the following features:

• We wanted to be able to install all (or almost all) of the features automatically (with minimal user intervention)
• We wanted to be able to pick-and-choose which features were installed; the needs of an Android developer were different than those of an iPhone developer.
• We want our developers to be happy; sure, they could install the features that they wanted or fix a problem with their workstation, but we want to go a step further: we want to provide the resources they need to write a recipe to minimize the effort the next developer has to go through. [This has been a success: several of our recipes were written in conjunction with developers.]

Testing

Integration tests for the cookbook took several days to set up. We use Faronic's Deep Freeze on a fairly pristine mac mini to ensure that we have a clean machine when we run our chef scripts. Continuous integration has proven invaluable for collaboration, for we quickly learn if a commit has unintended consequences.

In the more complex chef recipes, we attempt to write tests to test that they [the recipes] have succeeded; sshd_on.rb is a good example of testing that a service (sshd) was correctly started.

Expectations

The chef runs, especially the initial one, are flaky. Our current chef run must download software from over 40 different servers, any one of which being down or having changed the download location can cause a failure. For example, Little CMS, a dependency of ImageMagick, resided on littlecms.com, which was down for a few days. Our integration tests failed during that period.

If you encounter a server being down or a file that has moved, please send us a pull request with an updated download location, or just comment-out the broken recipe.

Target Audience

Our target audience is developers, which is great: they understand errors, and often contribute code fixes. Our goal is for Pivotal Ops to provide a framework for Pivotal Engineers to write the recipes that build the workstations they want.

Acknowledgements

I am grateful to Matthew Kocher, who more than anyone helped me write the bulk of the ruby scripts. Also to Sean Beckett, without whose support this would never have happened. And to the many pivots who offered suggestions & help.