Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker
Chad Woolley

Automating Bundler In Your Deploy

Chad Woolley
Thursday, March 25, 2010

If you are using Bundler to lock and package your gem dependencies in your app (which you should), here’s some tips on making everything automatic in your Capistrano deploy.

Refer to the Bundler Documentation for instructions on how to use Bundler to properly package your gems and check everything in.

Once this is done, however, you still must ensure that two things are done on every machine to which you will deploy:

  1. Bundler is installed
  2. You run ‘bundle install’ on every deploy to install the packaged gems on the local machine (and compile any gems with native dependencies)

Here’s the Capistrano magic to accomplish these two tasks automatically on every deploy:

before "deploy:bundle_install", "deploy:install_bundler"
after "deploy:update_code", "deploy:bundle_install"

namespace :deploy do
  desc "installs Bundler if it is not already installed"
  task :install_bundler, :roles => :app do
    sudo "sh -c 'if [ -z `which bundle` ]; then echo Installing Bundler; sudo gem install bundler; fi'"
  end

  desc "run 'bundle install' to install Bundler's packaged gems for the current deploy"
  task :bundle_install, :roles => :app do
    run "cd #{release_path} && bundle install"
  end
end

Oh, and for you GemInstaller users out there – here’s an easy way to generate a Bundler Gemfile from your geminstaller.yml config:

geminstaller --bundler-export > Gemfile

You’ll probably still need some tweaks, but this will get you started. Just make sure you upgrade to GemInstaller 0.5.5 first (0.5.4 forgot to put the ‘source’ line in the Gemfile).

Happy Bundling!
– Chad

P.S. There is a similar article here, which includes tasks to symlink your .bundle dir into the Capistrano shared directory, but my deploy was pretty fast anyway, so I didn’t worry about it. YMMV.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Chad Woolley

[ANN] GemInstaller 0.5.3 Released

Chad Woolley
Tuesday, August 25, 2009

This fixes several bugs that people have complained about for quite a while. Please let me know if anything is broken.


GemInstaller 0.5.3 has been released!

GemInstaller

  • by Chad Woolley
  • http://geminstaller.rubyforge.org

CHANGES

  • 0.5.3 / 2009-08-25
  • Many long overdue bugfixes and patches, see
    http://tinyurl.com/geminstaller-0-5-3-release for details.
  • Thanks to Greg Fitzgerald, Britt Crawford, John Trupiano, Gabriel
    Gironda, and Eric Hodel for patches and assistance.
  • Issues with case statement under Ruby 1.9
  • GemInstaller cannot distinguish between gems that have the ame name
    but capitalized differently.
  • add ./ci as default location for config file
  • Disable GemInstaller install in default rails preinitializer.rb, but
    fork if it is used
  • autogem() fails when run for newly-installed gem
  • Sometimes installing fails due to RubyGems cache not being cleared
    between multiple API calls

DESCRIPTION

Automated Gem installation, activation, and much more!

FEATURES

GemInstaller provides automated installation, loading and activation
of RubyGems. It uses a simple YAML config file to:

  • Automatically install the correct versions of all required gems
    wherever your app runs.
  • Automatically ensure installed gems and versions are consistent
    across multiple applications, machines, platforms, and environments
  • Automatically activate correct versions of gems on the ruby load
    path when your app runs (’require_gem’/'gem’)
  • Automatically reinstall missing dependency gems (built in to RubyGems > 1.0)
  • Automatically detect correct platform to install for multi-platform
    gems (built in to RubyGems > 1.0)
  • Print YAML for “rogue gems” which are not specified in the current
    config, to easily bootstrap your config file, or find gems that were
    manually installed without GemInstaller.
  • Allow for common configs to be reused across projects or
    environments by supporting multiple config files, including common
    config file snippets, and defaults with overrides.
  • Allow for dynamic selection of gems, versions, and platforms to be
    used based on environment vars or any other logic.
  • Avoid the “works on demo, breaks on production” syndrome
  • Find lost socks.

Quick Start

See http://geminstaller.rubyforge.org/documentation/index.html

INSTALL

  • [sudo] gem install geminstaller
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Standup 08/07/2009: Rubygems 1.3.5 fail?

Pivotal Labs
Friday, August 7, 2009

Interesting Things

  • One project had trouble installing the Rails 2.3.3 with the most recent version of Rubygems (1.3.5) on a Gentoo box (EY Solo instance). Installing the gem on an OSX box worked fine with the same versions. No obvious solution presented itself, other than to roll back to Rubygems version 1.3.1.

  • If you do update your Rubygems version to 1.3.5 and you use Geminstaller you must update your Geminstaller gem to 0.5.2; Rubygems made some changes that break Geminstaller’s implementation. If you use an older version of Rubygems then keep using Geminstaller 0.5.1.

  • One project wanted to find all places where they needed to escape user input on their site, so they injected <script>alert('foo');</script> into every text field in the database, ran through their site, and looked for every place that a pop up box appeared.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Take a Looksee at how a Ruby Object got its Methods

Pivotal Labs
Monday, July 27, 2009

One property of the Ruby object model and object oriented programming in general is that a subclass of an object automatically inherits all of the methods of its superclass. Classes can further expand the number of methods available by mixing in a Module, or several.

Because of mixins and subclassing even a class that has declared just a few methods can actually have hundreds of methods on it. In Ruby, all classes subclass Object by default which declares a hefty 45 methods, guaranteeing you to have at least that many. Out of the box in 1.8.7, a Ruby String object has 176 instance methods. If you are programming on top of the Rails framework, ActiveSupport adds 98 methods bringing the total to 274!

On numerous occasions I have needed to see what methods are available on an object I am working with I will type the following in irb.

myobject.methods - Object.instance_methods

This prints out a large array of instance methods with the methods inherited from Object removed from the list. This is useful but what if the object I am working with mixed in several modules and I am left with a list of over a hundred methods? It would be great to view which Class or Module each method came from. Well, actually there’s a gem for that.™

Looksee

Looksee is a new gem by George Ogata that examines the method lookup path of any object. To use it add require 'looksee/shortcuts' to your ~/.irbrc. This will add a lp (”lookup path”) method to your irb environment. When passed an object lp prints out a colored display showing where each of an object’s methods lives.

looksee output for a string object

  • public methods are show in green
  • protected methods are show in yellow
  • private methods are show in red
  • overwritten methods are show in gray

Go ahead and install Looksee and play around with it for a moment. Run lp on a String in vanilla irb and then open script/console in a Rails project and do the same thing. It is quite eye-opening to see the additions that the Rails framework makes.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Introducing ActiveApi – A sane way to translate your data to xml

Pivotal Labs
Sunday, July 5, 2009

ActiveApi allows you to define a schema in Ruby, and use that schema to convert ruby objects to xml. An example looks like this:

Schema.version(:v1) do |schema|
  schema.define :article do |t|
    t.attribute :id
    t.string :title
    t.date :published_on
    t.has_many :comments
  end
end

On one of the apps I’m working on now, we have to expose our data as xml. This XML will be used as the datasource for reports, and as a way for external clients to import data into a data warehouse. The clients who will be importing the data are enterprisey – they have lots of tools that work with DTDs and XSD’s, and they’ve never heard of ActiveResource. (Many barely know what Rails is.)

On this app the data model is quite large and complex, and changes will be inevitable. We may want to generate reports using the latest and greatest xml, but some clients may take longer to update their data import code, so it’s very likely that we’ll have to maintain overlapping versions for short periods of time.

Through this process I’ve come to think that exposing your data via XML is not the job of the model. Instead, it’s the job of a separate class that is specifically designed to translate your model schema into a schema appropriate for public consumption.

A good api tool will have built-in support for:

  • XSD or DTD generation
  • Versioning
  • The ability to represent your model in a way that is not tightly coupled to the model itself – so you’re models can change at a different rate than your xml schema

ActiveApi attempts to provide this functionality.

Installation

sudo gem install zilkey-active_api

Usage

You define a schema like so:

Schema.version(:v1) do |schema|
  schema.define :article do |t|
    t.attribute :id
    t.string :title
    t.date :published_on
    t.has_many :comments
  end

  schema.define :comment do |t|
    t.belongs_to :user
    t.string :article_title, :value => proc { |element|
      element.object.article.title
    }
  end

  schema.define :user do |t|
    t.string :username, :value => :user_name
  end
end

To create xml from this schema, you could write code like this:

ActiveApi::Schema.find(:v1).build_xml(@articles, :node => :article).to_xml

Which will give you xml output that looks like this:

<?xml version="1.0"?>
<articles>
  <article id="1">
    <title>target efficient applications</title>
    <published_on>2004-08-22</published_on>
    <comments>
      <comment>
        <article_title>target efficient applications</article_title>
        <user>
          <username>foo</username>
        </user>
      </comment>
    </comments>
  </article>
  <article id="2">
    <title>recontextualize viral e-services</title>
    <published_on>2004-12-05</published_on>
    <comments>
      <comment>
        <article_title>recontextualize viral e-services</article_title>
        <user>
          <username>foo</username>
        </user>
      </comment>
    </comments>
  </article>
</articles>

Extending

ActiveApi is highly extensible. The general pattern used to extend ActiveApi is to subclass the default ActiveApi implementation, and specify that you’d like to use your subclass instead.

For example, if you are working with a database that has audit columns such as timestamps, you might want to do this:

class MyDefinitionClass < ActiveApi::Definition
  def timestamps
    date_time :created_at
    date_time :updated_at
  end
end

@schema = Schema.version(:v1, :definition_class => MyDefinitionClass) do |xsl|
  xsl.define :article do |t|
    t.timestamps
  end
end

Schema.find(:v1).build_xml([@article], :node => :article).to_xml

Which will produce the following xml:

<?xml version="1.0"?>
<articles>
  <article>
    <created_at>1945-12-21T00:00:00+00:00</created_at>
    <updated_at>1992-04-05T00:00:00+00:00</updated_at>
  </article>
</articles>

NOTE: when specifying custom definition classes, those classes must be loaded before calling Schema.version

You can also create custom classes for any element you define, like so:

@schema = Schema.version(:v1) do |xsl|
  xsl.define :article, :builder_class => "MyCustomClass"
end

class MyCustomClass < ActiveApi::ComplexType
  def build(builder)
    builder.send :foo, :bar => "baz" do |xml|
      xml.send :woot, "lol"
    end
  end
end

Schema.find(:v1).build_xml([@article], :node => :article).to_xml

Which will produce the following xml:

<?xml version="1.0"?>
<articles>
  <foo bar="baz">
    <woot>lol</woot>
  </foo>
</articles>

NOTE: since the builder classes are evaluated at runtime, you can specify a string name for the class, and the class does not have to be loaded before calling Schema.version

Features

You define your schema completely separately from your data. So you could in theory render multiple types of objects with the same schema, provided that they have the same interface. You could also render a single object in any number of ways.

You can choose to have the builder send methods on your object, or provide more complex values by using the :value => proc{} syntax. Since you can define the value of the elements separately from the names, aliasing your objects field names is built in.

The element keeps track of all of it’s ancestors, so you can access objects that were rendered as ancestors, even if those objects aren’t ancestors in your object graph.

The Schema definition just creates an array of Ruby objects, which you could use to create documentation or XSD files. One of my major goals for this gem is to create valid XSD from the schema itself. Given how extensible it is, I’m not sure how that will work yet, but I’m psyched to give it a shot.

You define your schema versions using whatever versioning scheme you want – could be a string, symbol or any other object. You can render the same objects with different schemas easily. This library is totally agnostic as to how or if you version your schemas – but if you decide to version, it makes it easy.

The schema defining DSL allows you to define any valid XSL data format – it formats dates, times and datetimes in XSD compliant formats, it URI escapes any element tagged as anyURL and has helper methods for all XSD data types in both their native form (anyURL) and a more ruby-friendly form (any_url) – so you can write:

  schema.define :article do |t|
    t.anyURL :foo
    t.any_url :foo
  end

And they will be emitted identically.

ActiveApi also allows you to define polymorphic relationships via the XSL “choice” instruction:

      xsl.define :comment do |t|
        t.belongs_to :commentable, :choice => {
          "Article" => :article,
          "User" => :user
        }
      end

This allows your API to determine which elements to render at runtime, while still giving you complete control over the schema to use for that element. This allows you to have more than one representation of your objects – one that gets rendered if the object is the root, and a different one that gets rendered if it’s being rendered as a sub-element of another object.

Integration with Rails

ActiveApi is framework agnostic. While the belongs_to / has_many syntax is Rails-like in name, it does not depend on ActiveRecord. It also does not modify or interfere with AR in any way. To use xml generated from ActiveApi in a controller, you can do this:

def index
  @articles = Article.all

  respond_to do |format|
    format.html # index.html.erb
    format.xml  do
      render :xml => ActiveApi::Schema.find(:v1).build_xml(@articles, :node => :article)
    end
  end
end

Implementation

ActiveApi uses’s Nokogori::XML::Builder to create the xml nodes. As such, the creation of the xml is fast. The rest of the code likely needs major refactoring to be performant and have a small footprint with large datasets. In particular, it creates a new object for every collection, element and value. That’s a lot of objects. I’d love to hear any comments about how to improve this aspect of the design.

Authors

While I wrote all of the code in this repo, the code was inspired by pairing on a similar project with:

  • Mike Dalessio
  • Peter Jaros
  • Ben Woosely

Development

http://github.com/zilkey/active_api/tree/master

See the issue tracker on github for the list of features and bugs I know about.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Chad Woolley

"open_gem" Gem Plugin

Chad Woolley
Saturday, May 9, 2009

From the Too Useful Not to Blog Department:

open_gem from Adam Sanderson is a new RubyGems Plugin to automatically open a gem’s source in your favorite $EDITOR.

gem update --system
sudo gem install open_gem
export EDITOR=mate
gem open rails

NOTE: If you have RubyGems 1.1 or 1.2, ‘gem update –system’ may not work. See the RubyGems Release Notes for more info.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Introducing AutoTagger – easy multi-stage deployment tagging with git

Pivotal Labs
Monday, March 30, 2009

AutoTagger is a gem that helps you automatically create a date-stamped tag for each stage of your deployment, and deploy from the last tag from the previous environment.

Let’s say you have the following workflow:

  • Run all test on a Continuous Integration (CI) server
  • Deploy to a staging server
  • Deploy to a production server

You can use the autotag command to tag releases on your CI box, then use the capistrano tasks to auto-tag each release.

Installation

gem sources -a http://gems.github.com
sudo gem install zilkey-auto_tagger

To contribute, you can fork the github repository.

You can also visit the tracker project

The autotag executable

Installing the gem creates an executable file named autotag, which takes two arguments: the stage, and optionally the path to the git repo:

$ autotag demo  # => creates a tag like demo/200804041234 in the current directory
$ autotag demo . # => same as above
$ autotag demo /Users/me/foo # => cd's to /Users/me/foo before creating the tag

Running autotag does the following:

$ git fetch origin --tags
$ git tag <stage>/<timestamp>
$ git push origin --tags

Capistrano Integration

AutoTagger comes with 2 capistrano tasks:

  • release_tagger:set_branch tries to set the branch to the last tag from the previous environment.
  • release_tagger:create_tag runs autotag for the current stage

Example config/deploy.rb file:

require 'release_tagger'

# The :stages variable is required
set :stages, [:ci, :staging, :production]

# The :working_directory variable is optional, and defaults to Dir.pwd
# :working_directory can be an absolute or relative path
set :working_directory, "../../"

task :production do
  # In each of your environments that need auto-branch setting, you need to set :current_stage
  set :current_stage, :production
end

task :staging do
  # If you do not set current_stage, it will not auto-set your branch
  # set :current_stage, :staging
end

# You need to add the before/ater callbacks yourself
before "deploy:update_code", "release_tagger:set_branch"
after  "deploy", "release_tagger:create_tag"

Assume you have the following tags in your git repository:

  • ci/01
  • staging/01
  • production/01

The deployments would look like this:

cap staging deploy    # => sets branch to ci/01
cap production deploy # => sets branch to staging/01

You can override with with the -Shead and -Stag options

cap staging deploy -Shead=true      # => sets branch to master
cap staging deploy -Stag=staging/01 # => sets branch to staging/01

Known Issues

  • DOES NOT work with capistrano ext multi-stage
  • It will accept invalid tag names (if you specify a tag name with a space, it will blow up when you try to create the tag)

I have this working on a single project, and there is a fairly complete spec suite, but use at your own risk. It’s probably worth setting up on a demo project before adding it to a production repo.

Over time it would be nice to add other scms like subversion as well, so that you have a single way of tagging, regardless of scm.

Acknowledgments

Special thanks to Brian Takita, who gave me an initial implementation. Another useful link is http://codeintensity.blogspot.com/2008/06/changelogs-and-deployment-notification.html.

Happy tagging!

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

New York Standup 10/7/2008

Pivotal Labs
Wednesday, October 8, 2008

Interesting Things

When you specify a gem from a custom source, and it has dependencies on a separate source, you need to list both sources in geminstaller.yml.

This comes up when you are installing a gem from github and that gem depends on other gems from rubyforge. You can specify multiple sources by adding more –source attributes.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Chad Woolley

Removing Old Ruby Source Installation After a Leopard Upgrade

Chad Woolley
Friday, February 22, 2008


Removing Ruby

I just upgraded to Leopard on my Mac. Previously, on Tiger, I had installed Ruby from source, in the default /usr/local/lib prefix. After reading the discussion on the Apple-provided Ruby installation, I decided to try it – mainly to ensure that my apps, such as GemInstaller, play well with it (on Pivotal’s Mac pair workstations, we still install Ruby from source, so everything matches our demo/production environments as closely as possible, and things are in consistent locations).

So, I wanted to uninstall the old Ruby source installation, and only have the Apple-provided Ruby on disk. Googling for a few minutes did not provide exact instructions for this, so I’m writing up what I did, in hopes that it will help you!

I didn’t use the “–prefix” option when I originally installed Ruby from source, so it was in the default location of /usr/local/lib/ruby, with binaries in /usr/local/bin.

WARNING: Use ‘rm -rf’ at your own risk – a sleep-deprived encounter with ‘rm -rf’ and a stray file named ‘~’ is what “motivated” my Leopard upgrade in the first place…

First, I deleted the old ruby libraries/gems, which was easy enough, because they all lived under /usr/local/bin/ruby:

sudo rm -rf /usr/local/lib/ruby

However, this left all the old ruby/gems executables in /usr/local/bin. This resulted in errors when trying to run executable gems that I had not yet installed under the Apple Ruby installation:

$ cheat
/usr/local/bin/cheat:9:in `require': no such file to load -- rubygems (LoadError)
from /usr/local/bin/cheat:9

Instead of a cryptic rubygems error, I should get a ‘file not found error’:

$ sudo rm /usr/local/bin/cheat
$ cheat
-bash: /usr/local/bin/cheat: No such file or directory

So, I want to purge everything ruby-releated from my /usr/local/bin folder. I whipped up a quick ruby one-liner which just prints out (almost) all ruby-related files in /usr/local/bin:

ruby -e "old_ruby_execs = `egrep 'rubygems|bin/ruby|env ruby' /usr/local/bin/*`; require 'pp'; pp old_ruby_execs.split("n").collect{|line| line.split(':').first}.uniq"

Yeah, I know, ugly and obtuse, but one-liners are kind of fun, and help me remember that Ruby is great tool for sysadmin scripts. Feel free to put it in a class and test it if you are so inclined.

Even though I tried to make a fairly specific regexp for egrep, when inspecting that list, I did find a ‘jgem’ file, which was part of JRuby. I’m planning on reinstalling JRuby anyway, so I didn’t care if that got deleted along with the other ruby stuff.

Anyway, if the output of that looks like everything you want to delete, then run this one-liner to do the actual deed (the ‘sudo echo’ is to ‘prime’ the sudo auth, so you don’t get a noninteractive password prompt):

sudo echo; ruby -e "old_ruby_execs = `egrep 'rubygems|bin/ruby|env ruby' /usr/local/bin/*`; old_ruby_execs.split("n").collect{|line| line.split(':').first}.uniq.each { |exec| p 'removing ' + exec; `sudo rm #{exec}`}"

After that, the only thing that I saw left was the ‘ruby’ executable itself, which I whacked as well:

$ sudo rm /usr/local/bin/ruby

That seems to be about it, as least good enough to get all the old invalid executables off my path. I’m sure this could have been done cleaner if I had taken more care with the original source install. However, a good brute-force approach never hurt anyone. Much. Feel free to post links to relevant and helpful stuff.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (780)
  • rails (113)
  • testing (88)
  • ruby (83)
  • ruby on rails (70)
  • jobs (62)
  • javascript (55)
  • techtalk (44)
  • rspec (38)
  • ironblogger (32)
  • productivity (30)
  • activerecord (29)
  • gogaruco (29)
  • git (28)
  • nyc (27)
  • rubymine (26)
  • bloggerdome (23)
  • mobile (22)
  • process (21)
  • pivotal tracker (20)
  • cucumber (20)
  • jasmine (19)
  • design (18)
  • ios (18)
  • webos (17)
  • objective-c (17)
  • android (16)
  • palm (16)
  • "soft" ware (16)
  • fun (15)
  • tracker ecosystem (15)
  • ci (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • bdd (14)
  • gem (13)
  • css (13)
  • tdd (13)
  • selenium (12)
  • goruco (12)
  • bundler (12)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
  • mojo (10)
  • chef (10)
  • api (10)
Subscribe to rubygems Feed
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >