So you have a nice new Apple machine running Lion, but you don’t want to spend the next few hours installing software.
What to do?
The short answer: use chef/soloist in conjunction with a slew of recipes developed at Pivotal Labs to help install the most common set of components.
Here’s what to do:
- make sure you have Xcode installed; if not, you can install it through the App Store. Don’t worry–it’s a free application.
Do the following:
sudo gem install soloist mkdir -p ~/workspace cd ~/workspace git clone https://github.com/pivotal/pivotal_workstation.git cat > ~/soloistrc <<EOF cookbook_paths: - $HOME/workspace recipes: - pivotal_workstation::meta_osx_base - pivotal_workstation::meta_osx_development - pivotal_workstation::meta_ruby_development EOF soloist
It typically takes an hour for the chef run to complete. You can do other things while it’s running (but if you reboot or logout you’ll need to restart the chef run).
[Chef is a framework written by OpsCode to help configure in maintain one or more machines using ‘recipes’ (ruby scripts, more or less).]
[Soloist is an application which makes running chef-solo (the version of chef which runs without any centralized server) easier. It was written by my co-worker Matthew Kocher.]
What Is Installed by Default??
At the end of the chef run [as of this writing, and if all goes well] the following software will be installed:
- Mouse Locator
- Pivotal Git scripts
The following services will be enabled:
- screen sharing
The following preferences will be set:
- better bash (history, PS1, inputrc)
- faster key repeat
- git (settings, a few scripts)
- TextMate preferences (e.g. soft tabs)
- RubyMine preferences (e.g. keymaps)
What if I don’t want all that software? I just want TextMate and node.js!
Want to change the software that is installed? It’s simple: just change your ~/soloistrc file. Here’s a soloistrc that will only install TextMate & node.js:
cookbook_paths: - ./workspace recipes: - pivotal_workstation::textmate - pivotal_workstation::node_js
If you’re interested in seeing all the recipes available (and there are quite a few), just browse the recipes in the pivotal_workstation repo.
Why Did You Choose That Set of Software?
Early in June, several pivots (Sean Beckett, Matthew Kocher, and David Goudreau, and I) met to decide on the bare minimum set of software and features that our developers would need to function on a new Lion Machine.
This set of chef recipes is the result of that meeting. There have been some changes (we have had great difficulty writing recipes to install firefox addons, so we iceboxed the story; some of our developers contributed recipes for things they wanted, so we added those).
Why Did You Choose Chef?
We chose chef/soloist partly because felt that our previous process had reached the end of its usefulness and were familiar with chef from our work automating server configuration.
Here’s how our previous process worked:
- For minor releases (e.g. 10.6.7 → 10.6.8), we would take the previous golden image (a golden image is a snapshot of the disk drive of a machine with the applications, preferences, and settings that we wanted), install it on a workstation, upgrade the OS and possibly upgrade some of the applications. We would then use DeployStudio to take an image of the workstation, and that image would become the new golden image.
- For major releases (e.g. 10.5 → 10.6), we would re-create the golden image by hand, manually installing & configuring the individual software. For the Leopard/Snow Leopard transition, my co-worker Kevin Fitzpatrick spent a week painstakingly configuring the new machine. We then took an image using DeployStudio, and that image became the golden image.
This approach had several shortcomings:
- It was monolithic: if you were a developer, there was no choice: there was only one image. This wasn’t so bad when we were strictly a ruby shop, but when we expanded into android and iphone development, the monolithic approach began to show some shortcomings.
- There was cruft in the image: the golden image had been built up over years.
- It wasn’t clear that the Golden Image would make the jump to Lion: Lion introduced some big changes (e.g. no PPC executables).
- We were hesitant to approach the developers to ask them what they would like to see on the new image; we worried of re-igniting a Holy War.
We looked for alternatives. We wanted the following features:
- We wanted to be able to install all (or almost all) of the features automatically (with minimal user intervention)
- We wanted to be able to pick-and-choose which features were installed; the needs of an Android developer were different than those of an iPhone developer.
- We want our developers to be happy; sure, they could install the features that they wanted or fix a problem with their workstation, but we want to go a step further: we want to provide the resources they need to write a recipe to minimize the effort the next developer has to go through. [This has been a success: several of our recipes were written in conjunction with developers.]
Integration tests for the cookbook took several days to set up. We use Faronic’s Deep Freeze on a fairly pristine mac mini to ensure that we have a clean machine when we run our chef scripts. Continuous integration has proven invaluable for collaboration, for we quickly learn if a commit has unintended consequences.
In the more complex chef recipes, we attempt to write tests to test that they [the recipes] have succeeded; sshd_on.rb is a good example of testing that a service (sshd) was correctly started.
The chef runs, especially the initial one, are flaky. Our current chef run must download software from over 40 different servers, any one of which being down or having changed the download location can cause a failure. For example, Little CMS, a dependency of ImageMagick, resided on littlecms.com, which was down for a few days. Our integration tests failed during that period.
If you encounter a server being down or a file that has moved, please send us a pull request with an updated download location, or just comment-out the broken recipe.
Our target audience is developers, which is great: they understand errors, and often contribute code fixes. Our goal is for Pivotal Ops to provide a framework for Pivotal Engineers to write the recipes that build the workstations they want.
I am grateful to Matthew Kocher, who more than anyone helped me write the bulk of the ruby scripts. Also to Sean Beckett, without whose support this would never have happened. And to the many pivots who offered suggestions & help.