Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Tools
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker

Database.yml files using Chef Server

Colin Shield
Sunday, February 24, 2013

I’ve been spending an awful lot of time lately deploying rails applications using chef server. I’ll be blogging some of the more interesting ways we’re using chef server. One of the most useful features of chef server is search. This allows you to write recipes that reference nodes in your stack that match particular roles. In this post I’ll describe how we use search to generate a rails database.yml. This becomes particularly useful when creating and deploying the same rails application to multiple stacks.

First of all we added a simple file read in our database.yml file in place of the connection details.


<% if File.exist?('/opt/apps/fbfcats/shared/database_include.yml') %>
<%= File.read('/opt/apps/fbfcats/shared/database_include.yml') %>
<% end %>

In our chef project we have a recipe that searches for the database server node and writes out the database_include.yml file using a template resource.


master = search(:node, "role:db_master AND chef_environment:#{node.chef_environment}").first
template "/opt/apps/ms/#{app_name}/shared/database_include.yml" do
  owner "gemini"
  group "gemini"
  source "database_include.yml.erb"
  mode "0600"
  variables(:hostname => master,
  :database_name => "fbfcats" + "_" + node[:rails_env],
  :user => 'fbfcats',
  :password => 'mittens',
  :rails_env => node[:rails_env])
end

Finally the template.


<%= @rails_env %>:
  host: <%= @hostname %>
  adapter: mysql2
  encoding: utf8
  reconnect: false
  database: <%= @database_name %>
  pool: 5
  username: <%= @user %>
  password: <%= @password %>

 

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Standup Oct/19/2010: Cedar, a BDD Testing Framework for Objective C

Colin Shield
Wednesday, October 20, 2010

Interesting Things

Cedar, a BDD Testing Framework for Rails”, Saturday, October 23, 2010 11:00 AM!

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Donkey & Goat Open House

Colin Shield
Saturday, March 6, 2010

Donkey & Goat Winery in Berkeley is having an open house Saturday March 20 2010. I’m personally a big fan of their wine. I’m enjoying a glass of their 2006 Syrah, The Recluse right now. Jared, one of the owners, has been kind enough to let us at Pivotal sample his wines on a few occasions. I’d heartily recommend going along to sample their wine, eat some food and learn about their wine making.

DONKEY & GOAT SPRING OPEN HOUSE PARTY SAT 3/20 1-5PM, AT THE WINERY IN BERKELEY
7 NEW WINES, EATS FROM BERKELEY’S SOON TO OPEN LOCANDA DA EVA AND THE MICHAEL LAMACCHIO TRIO, OUR FAVORITE BRAZILIAN JAZZ TRIO IS BACK!
SAVE $10 WITH ADVANCE TICKETS

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Standup 1/14 actionmailer content type issue

Colin Shield
Monday, January 18, 2010

We were bitten by a Rails 2.3 bug related to ActionMailer today. It took us a good part of the day to hunt down due to the fact that it only happened in production and even then only occasionally.
Basically ActionMailer occasionally sends your multipart emails as text/plain with html content.

Here’s the ticket we found:

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Standup 1/13 Successful completion with SQS Internal Error

Colin Shield
Thursday, January 14, 2010

The RightScale SQS gem returned an exception from SQS multiple times, including retries. Not an unusual event. This could have been caused by the SQS service being unavailable. However, the team noticed that despite the failure the message was actually successfully added to the queue and processed as normal.

ActiveSupport logger appears to open the default ruby logger and remove everything except the basic log message passed through. This is done for all subsequent uses of the logger. Perhaps this is done so that the log message could be passed to a syslog service which will add timestamps.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Standup Jan 12 2010 DateJS timestring parsing

Colin Shield
Tuesday, January 12, 2010

Whilst trying to parse differently formatted date strings from rss feeds a pivot found that date.now is overridden by DateJS to return a new date.
There was a suggestion, that later proved useful, to use google’s rss reader to first clean up the different rss feeds to ensure that they all can be parsed in much the same way.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Marshal.dump vs YAML::dump

Colin Shield
Sunday, August 16, 2009

We find ourselves with a project with a very large dataset, more than 2 million items. This dataset changes frequently. The changes need to be transported to their respective servers ready to be served out to clients.
We decided to use a queuing architecture to distribute data. Objects are serialized and pushed to a queue. The large size of the dataset requires us to optimize as much as possible. There are only so many hours in a day and there is a lot of data to transport.
A question was raised in standup as to what was the fastest serialization method: YAML::dump or Marshal.dump. It seemed appropriate to write a quick script and work out which would be appropriate for our particular situation.
The objects we are serializing are simple hashes. I thought I’d write something that was representative of our situation in order to present a nice clear decision.
Here’s some code:

require 'yaml'
obj = {:a => "hello", :b => "goodbye", :c => "new string", :d => {:da => 1, :db => 2}, :e => 1}
start = Time.now
(0..10000).each do
  ser_obj = YAML::dump(obj)
  new_obj = YAML::load(ser_obj)
end
puts "YAML::dump time"
puts Time.now - start
start = Time.now
(0..10000).each do
  ser_obj = Marshal.dump(obj)
  new_obj = Marshal.load(ser_obj)
end
puts "Marshal.dump time"
p Time.now - start

I think we all knew how the results would look. It was nice to see that for our particular case there was a clear winner.

YAML::dump time
5.397909
Marshal.dump time
0.280292

Seems fairly cut and dried to me.
I personally prefer YAML for test result comparison. Maybe we’ll put something in our spec_helper to use YAML for testing and Marshal for production.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Standup 6/9/2009 Spring Cleaning. rspec should == useless

Colin Shield
Tuesday, June 9, 2009

Ask for help

Using Rspec 1.2.6 and an expression of some desired behavior such as this:

it "should score 0" do
  score = 0
  score.should == 0
end

Will sometimes result in a ruby warning:

warning: useless use of == in void context

There was a little confusion as to why this warning was sometimes appearing.
It turns out that the warning appears when the command ruby -w is used to run the spec rather than spec.
This seems to be the case when rake is run for a non rails project, say a plugin.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

KCachegrind OS X 10.5.6

Colin Shield
Thursday, May 7, 2009

Installing KCachegrind in order to profile output from ruby-prof turned out to be quite a time consuming task.
This was all performed on an iMac with a 2.16 GHz Intel Core 2 Duo CPU and 4GB 667 MHz DDR2 SDRAM.
The initial command:

sudo port install kcachegrind

ran for several hours only to fail with :

checking for XPROTO... configure: error: Package requirements (xproto >= 7.0.13) were not met:

Requested 'xproto >= 7.0.13' but version of Xproto is 7.0.11

I manually installed the correct version of xorg-xproto:

sudo port deactivate xorg-xproto # deactivates version 7.0.11_1
sudo port install xorg-xproto # installs version 7.0.14_1

Restarted the KCachegrind install:

sudo port install kcachegrind

Then waited again.
The next error encountered:

--->  Activating xorg-renderproto @0.9.3_0
Error: Target org.macports.activate returned: Image error: /opt/local/include/X11/extensions/render.h is being used by the active render port.  Please deactivate this port first, or use the -f flag to force the activation.

I did what the error message suggested:

sudo port -f activate xorg-renderproto

It looks as though some other install had installed files

/opt/local/include/X11/extensions/render.h
/opt/local/include/X11/extensions/renderproto.h

the install process copied the old files to a tmp directory. A quick diff of the files showed that the installed files were the newer files.
Restarted the install of kcachegrind:

sudo port install kcachegrind

The install continued for several more hours. kdelibs3 taking 2 hours.
When everything was finally installed I do have a working version of kcachegrind running.
In total the install ran for approximately 8 hours.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Standup 12/19/2008 Enumeration mixin and Rails 2.2

Colin Shield
Saturday, December 20, 2008

One project reported a problem with the enumeration mixin when upgrading to Rails 2.2. The problem was with the all method.

The enumerations mixin allows you to treat instances of your ActiveRecord models as though they were an enumeration of values.

The proposed solution is to switch to in memory enumeration instead of using ActiveRecord.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Colin Shield

Colin Shield
San Francisco

Subscribe to Colin's Feed

Author Topics

chef server (1)
agile standup (2)
performance (1)
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Tools
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >