Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker

Monthly Archives: September 2010

Will Read

Standup 2010-09-17: Why Aren't You at GoGaRuCo Edition

Will Read
Friday, September 17, 2010

Ask for Help

“How do I upgrade Passenger on the EngineYard Cloud with a unsupported account?”

Jump in to EY’s IRC to get ‘er done.

“Using the Vimeo gem, uploading fails with ‘Unknown upload error’”

No answer, but the gem hasn’t been updated in a while and the working theory is that it isn’t compatible with the new OAuth authentication. Suggestions included using HTTP Scoop to see what the traffic is doing and get more info.

“Ngnix 0.6 + Rails 3 isn’t sending the Content-Length header for 201 responses”

Since there’s no content length specified, some browsers grab on to the keep-alive and appear to take a very long time to load the page. In Rails 2.x the Content-Length is supplied. Does anyone know if this was an intentional choice in Ngnix/Rails 3?

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Mike Gehard

Testing Rails3 Generators using Cucumber and Aruba

Mike Gehard
Wednesday, September 15, 2010

In an effort to continue my contributions to the open source Ruby/Rails ecosystem, I decided to help the factory_girl_rails team move the Rails3 generators from the rails3-generators project into the factory_girl_rails project.

Like all good Ruby/Rails developers, they asked to make sure that I had tests written around the generators. I thought for a bit on how I was going to do this and then I wandered across the Cucumber feature files in the rspec-rails repo and found my answer.

Rspec-rails (and RSpec2 as well) uses a gem called Aruba to easily write Cucumber features around things that happen from the command line.

If you’d like to check out the result of using Cucumber and Aruba to test Rails3 generators, head over to my fork of the factory_girl_rails gem and check out the features/generators.feature file.

Hopefully the changes will be merged into the official factory_girl_rails repo soon and the generators will live closer to home.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Pivotal Tracker GoGaRuCo Haiku Contest Winner!

Pivotal Labs
Tuesday, September 14, 2010

As already tweeted, we have a winner in the Pivotal Tracker GoGaRuCo Haiku contest. It was a tough choice, with lots of awesome submissions. The ultimate call went to Sarah Gray, with her entry below:

Bloom and iterate.
Each story a fragment of
the pivotal whole.

Below are the three runners-up:

Icebox, Backlog, Done
Righteous Fall stories sorted
Tracker guiding flow

Don Smith

Tracker has arrived,
transparency is blooming,
the team uniting.

Gustin Prudner

heisenberg was wrong
I know both where I am and
my velocity

Dav Yaginuma

The runners-up will be receiving a Pivotal Tracker mug. Thanks to everyone who participated!

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Davis W. Frank

Jasmine 1.0 Released

Davis W. Frank
Tuesday, September 14, 2010

Pivotal Labs and the Jasmine team are pleased to announce the 1.0 release of Jasmine, our JavaScript BDD framework.

This release is largely about stabilization. We’ve fixed some bugs, stabilized our API, and improved support for JRuby and Ruby 1.9. There are several deprecations (especially around asynchronous ‘waits’ blocks and some matchers) to go along with the fixes and features. See the release notes for full details.

This release is available in standalone and Ruby gem versions.

Our upcoming priorities include improving the documentation, properly supporting Rails 3 and RSpec 2 and a number of other environments, and looking at all of your pull requests and issue reports. Jasmine’s backlog lives in Pivotal Tracker.

Big thanks to the Jasmine community for the pull requests; the majority of changes between 0.11.1 and 1.0 are based on user-submitted patches.

See you on the mailing list and make sure to follow us on Twitter: @jasminebdd…

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Sam Coward

Identify memory abusing partials with GC stats

Sam Coward
Monday, September 13, 2010

Recently we had to investigate a page that had very poor performance. The project used newrelic, but neither that, the application’s logs nor render traces gave clear results. The database wasn’t the bottleneck, but there were well over 100 partials in the render trace. These partials seemed to perform well, except sometimes the same partial would take hundreds to thousands of milliseconds.

As it turns out, the cause of these random delays was actually garbage collection triggered by abusive memory consumption within many of the partials – millions of objects and heap growth in excess of 100mb. We discovered this by using the memory and GC statistics features of Ruby Enterprise Edition. You can get an idea of how much memory is being used and garbage collected during rendering by wrapping parts of your page in a block passed to this helper:

def gc_profile
  raise "Dude, you left GC profiling on!" unless Rails.env.development?
  allocated_objects_before = ObjectSpace.allocated_objects
  GC.enable_stats
  GC.clear_stats

  yield if block_given?

  growth = GC.growth
  collections = GC.collections
  time = GC.time
  mallocs = GC.num_allocations
  allocated_objects = ObjectSpace.allocated_objects - allocated_objects_before
  GC.disable_stats

  concat content_tag :span, "GC growth: #{growth}b, collections: #{collections}, time #{time / 1000000.0}sec, #{mallocs} mallocs, #{allocated_objects} objects created."
end

Because there were so many partials on the page, moving the helper around to identify the most egregious memory abusers got tiring so I wrote a monkeypatch to the venerable Rack::Bug plug-in which shows you memory consumption and garbage collection statistics for each template shown in the Rack::Bug template trace, as well as on the memory panel.

Example template trace

Memory panel example

First, install Rack::Bug and make sure it’s working properly, then add the code below in an initializer. Keep in mind this was written to work with Ruby Enterprise Edition 1.8.7 2010.02. Various 1.8 patches and Ruby 1.9 also provide some GC statistics reporting tools which you could probably customize this patch to use instead.

if Rails.env.development?
  require 'rack/bug'

  Rack::Bug::TemplatesPanel::Trace.class_eval do
    alias_method :old_start, :start
    def start(template_name)
      old_start(template_name)
      @initial_allocated_objects = ObjectSpace.allocated_objects
      @initial_allocated_size = GC.allocated_size
      @initial_num_allocations = GC.num_allocations
      @initial_gc_count = GC.collections
      @initial_gc_time = GC.time
    end

    alias_method :old_finished, :finished
    def finished(template_name)
      @current.allocated_objects = ObjectSpace.allocated_objects - @initial_allocated_objects
      @current.allocated_size = GC.allocated_size - @initial_allocated_size
      @current.num_allocations = GC.num_allocations - @initial_num_allocations
      @current.gc_count = GC.collections - @initial_gc_count
      @current.gc_time = (GC.time - @initial_gc_time)/1000.0
      old_finished(template_name)
    end
  end

  Rack::Bug::TemplatesPanel::Rendering.class_eval do
    attr_accessor :allocated_objects
    attr_accessor :allocated_size
    attr_accessor :num_allocations
    attr_accessor :gc_count
    attr_accessor :gc_time

    def memory_summary
      %{<strong>%.2fms</strong><small>in</small><strong>%d</strong><small>GCs</small>
      <strong>%d</strong><small>new objects</small>
      <strong>%d</strong><small>bytes in</small><strong>%d</strong><small>mallocs</small>} % [gc_time, gc_count, allocated_objects, allocated_size, num_allocations]
    end

    def html
      %{<li>
          <p>#{name} (#{time_summary}) [#{memory_summary}]</p>
          #{children_html}
        </li>}
    end
  end

  Rack::Bug::MemoryPanel.class_eval do
    alias_method :old_before, :before
    def before(env)
      old_before(env)
      GC.enable_stats
      GC.clear_stats
      @initial_allocated_objects = ObjectSpace.allocated_objects
      @initial_allocated_size = GC.allocated_size
      @initial_num_allocations = GC.num_allocations
    end

    alias_method :old_after, :after
    def after(env, status, headers, body)
      old_after(env, status, headers, body)
      @gc_count = GC.collections
      @gc_time = GC.time / 1000.0
      @allocated_objects = ObjectSpace.allocated_objects - @initial_allocated_objects
      @allocated_size = GC.allocated_size - @initial_allocated_size
      @num_allocations = GC.num_allocations - @initial_num_allocations
    end

    def heading
      "#{@memory_increase} KB &#916;, #{@total_memory} KB total, %.2fms in #{@gc_count} GCs" % @gc_time
    end

    def has_content?
      true
    end

    def name
      "memory_panel"
    end

    def content
      %{<style>#memory_panel dd { font-size: large; }</style>
        <h3>Garbage Collection Stats</h3>
        <dl>
          <dt>Garbage collection runs</dt>
          <dd>%d</dd>
          <dt>Time spent in GC</dt>
          <dd>%.2fms</dd>
          <dt>Objects created</dt>
          <dd>%d</dd>
          <dt>Bytes allocated</dt>
          <dd>%d</dd>
          <dt>Allocation calls</dt>
          <dd>%d</dd>
      </dl>} % [@gc_count, @gc_time, @allocated_objects, @allocated_size, @num_allocations]
    end
  end
end
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Mike Barinek

Nesting and authenticating resque-web on ey-cloud

Mike Barinek
Monday, September 13, 2010

Wouldn’t it be nice to include the resque-web interface underneath your current application’s url?

Wouldn’t it then be nice to use the same authentication mechanism that your web application uses?

Here’s a solution that we used on a recent Rails 2.3 ey-cloud project that accomplished both goals.

First, we created a simple resque_web.ru file within our Rails 2.3 project. In this example we used the same Warden SSO authentication mechanism and fired up the resque-web server from the new mapping.


# Set up Resque environment
resque_config = YAML.load_file(File.join(File.dirname(__FILE__), 'config', 'resque.yml'))

env = ENV['RACK_ENV'] || ENV['RAILS_ENV'] || 'development'
Resque.redis = resque_config[env]

map '/resque-web' do
  use Rack::Session::Cookie, :key => 'rack.session',
    :path => '/',
    :expire_after => 60 ** 2,
    :secret => 'supersecretresquekey'

  use CasrackTheAuthenticator::Simple, :cas_server => "http://sso.your_sso_server.com"
  use CasrackTheAuthenticator::RequireCAS
  use Rack::ShowExceptions
  run Resque::Server.new
end

The important piece here is that we use ‘/resque-web’, we’ll use this same path in a bit within our Nginx configuration. We also used the rack session cookies with our SSO authenticator.

Another interesting piece here is that we’re using Castronaut and Casrack the Authenticator, which makes SSO pretty straight forward.

Next we created an upstream server in Nginx to resolve the path. We’re just using Thin here.

Here’s our Chef template for the upstream server.


upstream resque_web_upstream {
   server unix:/var/run/engineyard/<%= @app_name %>/<%= @app_name %>.thin.1.sock;
}

The important piece here is that the upstream server lives outside of our web application config area.

Here’s our Chef recipe for creating the upstream server file.


template "/etc/nginx/servers/resque_web.conf" do
  owner "root"
  group "root"
  mode 0755
  source "resque_web.conf.erb"
  variables({
    :app_name => app_name
  })
end

And then finally, we created a new custom location within our Nginx server configuration.

Here’s our Chef template for the custom Nginx Location.


location /resque-web {
  proxy_set_header Host $http_host;
  include /etc/nginx/common/proxy.conf;
  proxy_pass http://resque_web_upstream;
}

And here’s our Chef recipe for creating the custom location file.


template "/etc/nginx/servers/#{app_name}/custom.locations.conf" do
  owner "root"
  group "root"
  mode 0755
  source "custom.locations.conf.erb"
  variables({
    :app_name => app_name
  })
end

That’s it!

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Sean Beckett

New Tech Talk: Sencha Touch

Sean Beckett
Friday, September 10, 2010

Sencha‘s Sr. Software Architect Ed Spencer describes and demonstrates their Touch framework. Sencha Touch is an open-source javascript app framework that helps developers build web apps for HTML5-capable touch devices like Android, iPad and Blackberry. It has an object-oriented design with a rich event-handling system and pre-built UI components like carousels and scrollable lists. Ed gives a quick conceptual introduction for orientation and then shows sample code, demos, and how to wrap your web app for app store/marketplace distribution. He also shows some of the amazing stuff you can do with CSS3 these days.

See all our talks at http://pivotallabs.com/talks

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Sencha Touch

Friday, September 10, 2010 | Run time: 29:24

Sencha Touch is an open-source javascript app framework for building web apps for HTML5-capable touch devices. It has an object-oriented design with a rich event-handling system and pre-built UI components like carousels and scrollable lists.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Mike Barinek

Dependency musings

Mike Barinek
Friday, September 10, 2010

Recently there has been a lot of discussion around project dependencies. As a result, it has been pretty hard to decide when and if to include a new gem into a project.

Overall, I think I tend to weigh in heavier on the side of exclude everything possible except what is absolutely necessary. That being said, here’s the mental check list I tend to work through when deciding whether or not to include a dependency.

Is it something that I really need? Is it something that I can write myself?

Rails or MySQL might be something I’d rather not write because it might take a while versus an authorization component that would take only a few hours or days. It might be just as quick to write an authorization component then attempt to understand and debug another developers intentions.

Am I ready to support the Gem if the maintainers won’t?

Is this something I want to continue to maintain and grok a year from now, or will the Gem be obsolete?

Will it help me achieve a deliverables drop dead date?

Will it help me ensure that a deadline will not slip?

Can I accelerate toward a deliverable by including a Gem and then revisit the dependency?

What happens when I need to modify the Gem to tailor it to my needs? How do I keep up with the fixes from the original branch?

I tend to error on the side of Gems lasting for 6-18 months, then they’re essentially dead or re-written by the same author.

Is DRYing up my code really that important?

I’ve found that it is easy for people to ramp up on verbose code, especially verbose test code versus ramping up on code that includes a hierarchy of dependencies that one would need to sort through.

Continuing this thought, is there a ramp up time? And how many people will need to ramp up on the dependent Gem?

Further more, do I need total control over the component that I’m thinking about including?

Am I worried about a rogue check in that might impact the end-user experience when I rebase?

Is there a performance implication that I would like better to manage?

In summary, I guess I tend to get a bit uncomfortable with a large number of dependencies and wind up continually try to reduce the number of project dependencies.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Ryan Richard

Standup 9/10/10

Ryan Richard
Friday, September 10, 2010

Interesting Things

  • Using gem install with the soap4r gem doesn’t work. Fear not! It can be installed by downloading the tar file and manually unpacking it.

  • There is a node.js meetup at Dogpatch Labs tonight at 6:30.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (781)
  • rails (113)
  • testing (88)
  • ruby (83)
  • ruby on rails (70)
  • jobs (62)
  • javascript (55)
  • techtalk (44)
  • rspec (38)
  • ironblogger (32)
  • productivity (30)
  • activerecord (29)
  • gogaruco (29)
  • git (28)
  • nyc (27)
  • rubymine (26)
  • bloggerdome (23)
  • mobile (22)
  • process (21)
  • pivotal tracker (21)
  • cucumber (20)
  • design (19)
  • jasmine (19)
  • ios (18)
  • webos (17)
  • objective-c (17)
  • android (16)
  • tracker ecosystem (16)
  • palm (16)
  • "soft" ware (16)
  • fun (15)
  • ci (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • bdd (14)
  • gem (13)
  • css (13)
  • tdd (13)
  • selenium (12)
  • goruco (12)
  • bundler (12)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
  • mojo (10)
  • chef (10)
  • api (10)
Subscribe to Community Feed
  1. ←
  2. 1
  3. 2
  4. 3
  5. 4
  6. →
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >