Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker
Mark Rushakoff

They practically couldn’t make it easier to contribute to the Rails docs

Mark Rushakoff
Monday, April 30, 2012

If you ever spot room for improvement or an error in the Rails documentation — and that includes the Rails Guides and the API docs — they’ve made the process extremely easy. If you’ve been waiting for “the right moment” to start contributing to open-source software, this might be it.

The docrails project on Github, has public write-access enabled so that anyone can push documentation fixes.

This Rails blog post goes into full detail on the docrails project, but here’s the short and sweet version:

  • Never touch any source code in commits that get pushed to docrails. If you need to change code, make a pull request to the main Rails project. docrails is for changes to RDoc, guides, and README files.
  • docrails will occasionally be merged into rails, and vice versa. docrails has maintainers who ensure that no “poor” commits to docrails make it into the rails repo.
  • If you’re confident that your documentation fix is an improvement, just push to docrails!

If you want to expand (or just start) your “open-source portfolio,” docrails is a great place to begin.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Mark Rushakoff

Reversible migrations in Rails 3.1+

Mark Rushakoff
Thursday, April 26, 2012

In Rails 3.1 and newer, when you write a migration by hand, you can (usually) just define a change method instead of an up and a down method.

This StackOverflow answer from user Readonly demonstrates the use of change nicely:

class RenameOldTableToNewTable< ActiveRecord:Migration
  def self.up
    rename_table :old_table_name, :new_table_name
  end
  def self.down
    rename_table :new_table_name, :old_table_name
  end
end

becomes

class RenameOldTableToNewTable< ActiveRecord:Migration
  def change
     rename_table :old_table_name, :new_table_name
  end
end
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Ken Mayer

TDD Action Caching in Rails 3

Ken Mayer
Wednesday, March 28, 2012

On my current project, we needed to prove that an action cache was working as expected. Alas, the blogosphere had either out-of-date or unhelpful information. So, after many experiments, we came up with an RSpec test that does what we want. It seems ugly to me, and I hope there’s a better way. The names have been changed to protect the guilty. Any resemblances to actual classes and methods are purely coincidental.

We needed to confirm that a certain action was cached. This action is preview in the brands controller. Using the usual Rails url helpers, we construct some fixture data.

describe BrandsController do
  describe "caching" do
    let(:brand) { Factory.create(:brand) }
    let(:preview_cache_path) {'views/test.host' + preview_brand_path(brand)}
  end
end

Then we wrote our first test:

it "should action cache #preview" do
  Rails.cache.clear
  get :preview, :brand_id => brand.to_param
  ActionController::Base.cache_store.exist?(preview_cache_path).should be_true
end

This won’t work at all, however; because, in the test environment, caching is turned off.

$ cat config/environments/test.rb
Activator::Application.configure do
...
  config.action_controller.perform_caching = false

So, we need an around block to temporarily turn caching on:

around do |example|
  caching, ActionController::Base.perform_caching = ActionController::Base.perform_caching, true
  example.run
  ActionController::Base.perform_caching = caching
end

That’s great, but the default cache store is the :null store, which, as its name implies, does nothing.

around do |example|
  caching, ActionController::Base.perform_caching = ActionController::Base.perform_caching, true
  store, ActionController::Base.cache_store = ActionController::Base.cache_store, :memory_store
  example.run
  ActionController::Base.cache_store = store
  ActionController::Base.perform_caching = caching
end

Better. But our tests still won’t run because while ActionController uses the cache_store, Observers and Sweepers
use Rails.cache and that is only updated at boot time.

around do |example|
  caching, ActionController::Base.perform_caching = ActionController::Base.perform_caching, true
  store, ActionController::Base.cache_store = ActionController::Base.cache_store, :memory_store
  silence_warnings { Object.const_set "RAILS_CACHE", ActionController::Base.cache_store }

  example.run

  silence_warnings { Object.const_set "RAILS_CACHE", store }
  ActionController::Base.cache_store = store
  ActionController::Base.perform_caching = caching
end

Did I mention that Rails.cache is an accessor for the global, constant, RAILS_CACHE. Ugh.

So, now, we can implement our method

class BrandsController < ApplicationController
caches_action :preview
  def preview
  end
end

But that is still not enough. caches_action
has an interesting performance enhancement; it doesn’t actually set up the action caching unless caching is enabled at class load time. Since we’re not turning caching on until test time, the caches_action method call in the controller class does nothing. We need to re-add it in our test spec.

it "should action cache #preview" do
  Rails.cache.clear
  BrandsController.caches_action :preview # must be recapitulated to get around load time weirdfullness

  get :preview, :brand_id => brand.to_param

  ActionController::Base.cache_store.exist?(preview_cache_path).should be_true
end

This is ugly; it doesn’t test very much (except the underlying caching module, and why bother testing the framework). At least it proves to ourselves that the action is cached and the cache key is what we expect.

Now that we’ve got caching under control, let’s check cache expiration (using a Sweeper).

it "should clear the cache on #update" do
  ActionController::Base.cache_store.write(preview_cache_path, 'CACHED ACTION')

  put :update, id: brand.to_param, brand: {one: 'attribute', after: 'another'}

  ActionController::Base.cache_store.exist?(sign_up_cache_path).should be_false
end

First, I create a cached object, in this case, just the string ‘CACHED ACTION’ and then I invoke the action, and then, I hope, the cache will be expired.

It doesn’t really matter what happens in the #update method of the BrandsController as long as it updates a Brand object. A sweeper in Rails is a mix of Observer & controller filters, so all you need to do is “declare” it in the controller

class BrandsController < ApplicationController
caches_action :preview
cache_sweeper :brand_sweeper
def update
  ...
  @brand.save

Awesome sauce! Now our tests are red and I’m ready to implement the sweeper

class BrandSweeper < ActionController::Caching::Sweeper
  observe Brand # Observers will introspect on the class, but Sweepers don't

  def after_update(brand)
    expire_action :controller => "brand", :action => :preview, :brand_id => brand.to_param
  end
  ...

And voilà! We have greenness.

So what have we learned from this? The Rails source is still your best friend when exploring a sticky problem. Caching is hard, and testing caching is even harder.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Ken Mayer

Dry DevOps with heroku_san

Ken Mayer
Sunday, March 25, 2012

Quiz time!

  1. How many times (each day) have you typed this at your console?
    git push heroku master and then forgotten to run
    heroku run rake db:migrate --app yellow-snow-3141 or
    heroku ps:restart
  2. Does your script support a multi-stage environments?
  3. Do you remember how to get to the application’s console process?
  4. Is your application’s configuration consistent across all stages?
  5. Are you deploy scripts tested?

Yeah, me too.

Enter heroku_san, the 85% solution, which allows me to do this:

$ rake staging deploy

And this is what happens: Maintenance mode is turned on; the latest green build is deployed to my staging server; all pending migrations are run, the app restarted, and then maintenance mode disabled. All while I go get a fresh cup of joe.

The value for us is that we don’t repeat ourselves, that our app will deploy, and we are confident that extending the toolkit is easy with a clear API and good test coverage. That’s a big win when you have dozens of apps deploying all day long.

Installation is pretty simple; add gem 'heroku_san' to your Gemfile (you are using Bundler, aren’t you?). This gem is only used locally, so you can add it to your group :development, :test block. Then create and edit a configuration file; config/heroku.yml[1]. The README file has more details and there’s a wiki on github with handy tips on how to extend the basic functionality in common ways.

So, why?

The gem has been around since 2010 and supported by a loyal core of maintainers. Version 2.1.1 brings changes large and small.

The biggest change since v1.3 is the creation of 2 new classes: Project and Stage, and a test suite to drive out the features. The task library is now almost entirely simple method calls, so there’s a lot of confidence that they will run correctly. Just to make sure, however, I’ve written an integration test, remote.feature which exercises the whole stack by creating an app on Heroku and run it through its paces. It’s worth a read.

There are some other new features worth highlighting:

  • Heroku stack support (aspen, bamboo and cedar)
  • Direct use of the Heroku::Client (instead of calls out to the shell)
  • tags — By adding an (optional) tag configuration value to your heroku.yml file, heroku_san will deploy only the most recent revision with that tag to your stage. This is great for creating a multi-stage environment, and when combined with auto_tagger and a continuous integration server (CI), you get a lot of bang for a little bit. See the wiki for more complete instructions.

tl;dr

Why write your own deploy scripts, again. Use heroku_san (or another gem like kumade)


Notes

[1]: I am working hard to remove the Rails dependencies.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Stephan Hagemann

Migrating from a single Rails app to a suite of Rails engines

Stephan Hagemann
Tuesday, March 13, 2012

TL;DR

We moved a Rails app into an unbuilt engine of a new blank slate container app to allow new parts of our app to live next to it as engines as well. It has been working great for us!

I have a sample app rails_container_and_engines of the result’s structure on github.

Skip to the pitfalls and discoveries section to read about some of the speed bumps we during our transition. Interested in the why and how? Read on!

Rationale

As part of the project we had built a web app, a mobile app that talks to the web app’s API, two ETL tools to 1) do the initial data import from the app we were replacing and 2) get data into another of our client’s apps. At this point we knew that we would create one more big web app and several auxiliary apps.

Running up to the decision to using unbuilt engines we had several intense discussions on how to build loosely-coupled, highly-cohesive systems with Rails applications. We saw basically three choices:

  • All-in-one app that could get more structure through namespacing of its interals.
  • Services (REST or whatever, but) running as separate apps and communicating via APIs.
  • Engines running within a mostly feature-less container app.

The two big web apps share several models within the data access layer and use the same data. Because of this we chose the third option and left all of the apps within the same rails code base.

Internal discussions highlighted the costs/benefits of having all of these apps live within their own Rails project versus an engines approach. We feel that by using engines we are getting many of the benefits of a component based architecture without breaking Rails patterns. In addition, we feel that the cost of maintaining individual applications that share a central database or one giant application with less defined components would have been very high.

In order to make day to day development easier, and to avoid the “where do migrations live…” conversation and top level Rails deployment patterns, there is one twist to the architecture: everything resides in one git repo and engines are referenced from a single container application (similar to old school enterprise archive files). Each application is exposed via a unique context or resource identifier (each engine/app could also be isolated per instance via Apache). Here is the directory structure we ended up with:

container_rails_app/
  ...
  app
  config
  engines/
    etl/
    shared_modules/
    web_app_1/
    web_app_2/
  ...

Mike described this pattern in his recent blog posts Unbuilt Rails Dependencies: How to design for loosely-coupled, highly-cohesive components within a Rails application and Rails Contained: A Container for Web Application Development and Deployment. On how to make RubyMine work seamlessly with engines, read my post on IntelliJ Modules in Rubymine.

The steps we took

  • Generate a new, empty Rails app
  • Within the engines folder, create a new, mountable Rails engine
  • Copy all the tests from the original Rails app into the test directory and make them green.

    Ok. This step is a bit more involved. Essentially that’s it thought. Find some of the pitfalls we ran into described below. Here are a couple of highlights of what needs to happen:

    • Copy the files you need for a test to pass and namespace its class
    • Start with the model tests and work your way up towards integration and acceptance tests
    • Namespace everything with the name of the engine (either by fully qualifying every name – don’t do that) or with the lexical scope trick explained below. This includes tests and all classes.
    • Load needed gems explicitly: engines don’t load as much automatically as a Rails app
    • Namespace tables and assets
  • Copy the old .git directory into the new root folder in order to not loose any history. For us, git was not able to detect the changes as moves in many cases. This is certainly an area where you can improve on our solution!

For us, the real work started after this, when we started pullling out common code into a common models engine and began work on the second app.

We got all the tests to pass before we had namespaced any of the assets or rake tasks. That was an additional search-and-replace heavy step after the actual transition to an engine. It is not necessary right away if you do not have multiple applications at first, but to achieve the full effect, you will need to namespace the things as well.

Pitfalls and discoveries

Namespaces

Modules in your enige are not automatically loaded: make sure you reference them yourself before they are needed in other files.

Asset pre-compilation

Rails creates a default app/assets/stylesheets/application.css file which contains these lines:

/* ...
*= require_self
*= require_tree .
*/

If you have this file in your main app, all css files will be compiled into one file. For us, this made almost everything look right. Almost. Little things broke here and there. Our app contained a couple of sections for which the stylesheets were meant to be loaded separately. Add files that you want to have precompiled as individual files to config.assets.precompile list to have them precompiled into separate files and solve this problem.

Are all your references and associations breaking?

Try this

class M::Y
  def use_y
    M::Y.do_it!
  end
end
class M::Y
  def self.do_it!
  end
end

instead of

module M
  class X
    def use_y
      Y.do_it!
    end
  end
  class Y
    def self.do_it!
    end
  end
end

The first way sets the lexical scope to the module M as well as to the class Y allows you to references other classes in M without their full name. The second way sets the scope only to class M::Y and you have to fully qualify every class name to find it.

Don’t fight conventions

We were using fixture builder and while we were namespacing classes in modules we were trying to override the default table names to not be namespaced… Fixture builder didn’t like that at all. There may or may not be other dependencies that make leaving the conventions hard. So, save yourself the trouble and do the migrations (and stay consistent!) and namespace your tables as well!

HABTM

They seem to be falling out of favor, which is probably a good thing. With engines they don’t seem to work (well). We ended up getting rid of our last two habtm relationships instead of trying to make them work. Creating the join as a class and adding the necessary has many :through relationships is straight forward enough.

We ended up with one production performance bug due to this: Rails

Engines depending on engines

We used kaminari for our pagination requirements. When our app first became an engine, kaminari stopped picking up our custom views. Instead it used its standard views. The load order got screwed up to which there are basically two solutions:

  • require => nil on both engines in the Gemfile and force the correct load order in your app, or…
  • avoid name clashes by namespacing your views (which you were going to do anyways, right?)
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Georg Apitz

[Standup] [Boulder] heroku and the crickets

Georg Apitz
Friday, March 2, 2012

help

  • anybody seen old content showing up on heroku at various points in time (especially at night)?
    • crickets …

interesting

  • rake execute takes a hash, rake invoke takes an ordered list
  • http://join.me is a cool tool for interactive screen sharing
  • don’t use anonymous iframes for ajax file upload, use a class or id,
    • we saw weird issues with a chat provider who uses an anonymous iframe which was getting all our file upload info
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Jacob Maine

Standup 2012/1/31: The bleeding edge

Jacob Maine
Tuesday, January 31, 2012

Interesting Things

  • There’s a new release of Backbone – 0.9.0. It’s billed as a release candidate for 1.0, and seems to be a bit buggy, as RCs can be. However, it’s exciting to see that Backbone is getting close to that milestone.
  • You should default boolean fields to true or false, at the database layer. Otherwise your queries have to deal with three-valued logic.
  • Rails 3.2 has some unexpected behavior. First, the generated form ids have changed … what used to be id=user_new is now id=new_user. Second, if your routes file is missing an entry, you will no longer get errors in controller tests. If you liked that behavior, try out request specs.

Ask for Help

  • “Anyone seen problems with the latest REE and iconv?”

Everything works on the Linux machines, but on Macs, there’s an error about an unrecognized target encoding. Iconv works on the command line, so it’s something about REE.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Verifying hosts are active in the load balancer pool

Pivotal Labs
Wednesday, January 11, 2012

Most load balancers have a heart beat monitor, that allows the load balancer to decide whether or not to send traffic to a given host.

For example:

http://server1.host.com/check.txt

If the load balancer gets a 200, it will send traffic to server1, if it gets a 404 or a timeout, it will not.

Today we were trying to figure out how to find what servers were up and running in the load balancer pool.

Here’s the capistrano task we came up with:

desc "Curls the check.txt file to see if the host is in the load balancer"
  task :check_load_balancer do
    roles[:web].map(&:host).each do |hostname|
      value = %x{/usr/bin/curl -fs 'http://#{hostname}/check.txt'}
    puts "Curling http://#{hostname}/check.txt: #{value}"
  end
end

This gives you output like the following:

$ cap production deploy:check_load_balancer
  * executing `production'
  * executing `deploy:check_load_balancer'
Curling http://server2.dc1.domain.com/check.txt:
Curling http://server3.dc1.domain.com/check.txt: OK
Curling http://server4.dc1.domain.com/check.txt: OK
Curling http://server5.dc1.domain.com/check.txt:
Curling http://server6.dc1.domain.com/check.txt: OK
Curling http://server7.dc1.domain.com/check.txt: OK
Curling http://server8.dc1.domain.com/check.txt: OK
$

Now we can easily see there are issues with server 2 and 5.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Glenn Jahnke

Standup 1/9/2012

Glenn Jahnke
Monday, January 9, 2012

Helps

CCMenu + Hudson w/o Basic Auth

“Has anyone figured out how to use CCMenu with Hudson and not have Basic Auth?”

Hudson has its own authorization mechanism, unlike Basic Auth, so it can’t be used with nice desktop tools like CCMenu which shows the red/green square in your system tray.

Consider using Jenkins.

Using Symlinks with Dropbox

Using the Linux Dropbox client seems to allow symlinks to be uploaded, but they will not behave as expect anywhere else. Someone was trying to have a “latest” folder point to the latest versioned folder.

The recommendation was just to have two copies of the files as a workaround.

Interestings

Rails Bridge Outreach for Women Workshop has space

Looking to learn Rails? There’s a great meetup to get you up and running. As of this writing there is still space available. Come check the event out.

Yammer Javascript Meetup on Tuesday

Yammer will be hosting a Javascript meetup, expect the usual snacks and beer, and good talks about our favorite client-side language.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Davis W. Frank

Giving Rails 2 the Asset Pipeline

Davis W. Frank
Thursday, December 29, 2011

We had a client project, a Rails 2 app, that needed some cleanup around JavaScript and CSS management. They were using both Sprockets 1 and Jammit, as well as a bunch of plugins and gems to help make all this work together. They had a set of problems that the Rails 3.1 Asset Pipeline solved. But due to gem dependencies we couldn’t go to Rails 3 yet.

How hard could it be to put Sprockets 2 and the Asset Pipeline into a Rails 2 app?

1. Add Sprockets 2

In the Gemfile:

gem sprockets, "~> 2.0"

2. Working with SASS

If you need/want the Sass gem for SASS or SCSS stylesheets, then this goes in the Gemfile:

gem 'sass'
group :development, :test do
  gem 'sprockets-sass'
end

Rails 2 doesn’t include Sass by default. So you need to add it. Yes you might already have Sass if you’re using HAML for your view templates, but since they’re about to be broken apart, add it now.

The sprockets-sass gem adds support of watching individual .sass files in development and recompiling the parent stylesheet asset as needed. It’s not going to be used in production, but more on that later.

For your SASS files you’ll want to use @import to declare SASS dependencies instead of Sprockets’s require.

3. Rack it up in config.ru

Sprockets runs as Rack middleware, but since it’s difficult to map a path in Rails’ environment.rb 2 and make a rackup file. Name it config.ru so that the rest of Rails’s scripts (e.g., script/server) work with it transparently. For example:

What’s going on in this rackup file -

  • Sprockets with the same paths as Rails 3.1′s defaults – add more as necessary for your app
  • Guard against mapping /assets in production.
  • Guard against multiple loads if Rails is already initialized (a fix for testing)
  • Use Sprockets::Helpers to get asset_path and asset_url helpers (and others, see sprockets-helpers) for your stylesheets and views
  • Map / to the rest of Rails, guarding the middleware you don’t need in test

4. More Templates

Using EJS or Mustache? Then drop in the additional gems that work with Sprockets 2′s templating via Tilt:

gem 'ejs'
gem 'sprockets_spacely'

5. Move Your Assets

Now that configuration is done, move your assets under app/assets, just like the Rails Guide suggests. Don’t forget to change the url calls in your stylesheets to asset_url. Run the app, checking your JS console for 404s until all the includes and paths are correct.

6. Capybara

Capybara now needs to forget that it’s running Rails and instead just a Rack app. This technique (WARNING: eval ahead.), meant for using Cucumber/Capybara in a Sinatra app, works.

7. Compile your assets on deploy

All of this work so far makes your assets available from your /app directory in development and test. In production, they will get served from /public/assets. Rails 3.1 provides the assets:precompile to be used as part of the deploy process Generate an empty Rails 3.1 app and copy it into lib/tasks. Copy the Sprockets configuration from your config.ru into this task. Or yes, come up with a way to DRY this configuration up. Then you need to make sure it’s run at the right time during your deploy.

8. What’s left?

This was all we needed and it’s working well for this app. There are a few things that this guide doesn’t cover.

Assets from gems or /vendor are not loaded. Feel free to add explicit paths to Sprockets as needed. The Sprockets Helpers gem doesn’t yet support a Proc for an asset host, but there’s now an open issue on Github. This project didn’t need the former and we threw something together based on Sasset, which appears to be abandoned, and use it only on the asset compile step. We’re not using CoffeeScript on this project. These we’ll leave as exercises for the reader.

The Asset Pipeline is still a bit new and we’re all getting used to it. It’s not just Sprockets, SASS, etc. – there is a lot of functionality that Rails puts on top of Sprockets, but it’s easy enough to add the equivalents.

That wasn’t so bad, was it?

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (778)
  • rails (113)
  • testing (86)
  • ruby (83)
  • ruby on rails (70)
  • jobs (62)
  • javascript (54)
  • techtalk (44)
  • rspec (38)
  • activerecord (29)
  • productivity (29)
  • gogaruco (29)
  • ironblogger (29)
  • git (28)
  • nyc (27)
  • rubymine (25)
  • mobile (22)
  • bloggerdome (20)
  • cucumber (20)
  • process (19)
  • pivotal tracker (19)
  • jasmine (19)
  • design (18)
  • ios (18)
  • webos (17)
  • objective-c (17)
  • android (16)
  • palm (16)
  • "soft" ware (16)
  • fun (15)
  • tracker ecosystem (15)
  • ci (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • bdd (14)
  • gem (13)
  • selenium (12)
  • css (12)
  • goruco (12)
  • bundler (12)
  • tdd (12)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
  • mojo (10)
  • chef (10)
  • rubygems (9)
Subscribe to rails Feed
  1. ←
  2. 1
  3. 2
  4. 3
  5. 4
  6. 5
  7. 6
  8. 7
  9. ...
  10. 12
  11. →
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >