Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker

Standup 2010-08-20: Database Woes and Eval Nuances

Will Read
Sunday, August 22, 2010

Help!

A Pivot is working on a project where 1,500 records living in a CSV are iterated over (loaded in to ActiveRecord objects) and loaded in to a Postgres database… or at least that was the intention. The database just crashed on him in a repeatable way. Anyone seen anything like this before?

On another project, there’s a has_many association on a table with 40,000 records in MySQL. It seems that pulling these in through standard ActiveRecord means is not releasing memory (you can watch the memory grow as it executes). The work around was to use ActiveRecord::Base.connection.select_all(), but it would be nice to know why GC didn’t kick in and clean up house. Maybe AR is holding on to object references somewhere outside of scope?

Interesting Things

Speaking of scope, there’s a subtle difference in Ruby 1.9.2 that a pivot ran into – previously you could set variables inside a call to eval() and have that variable defined in the same scope as the call. That is no longer the case in 1.9.2 where eval has its own scope inside the call.

And lastly, on this day in 19..(mumble mumble), NASA launched the Voyager 2. Later in its career, Voyager was given it’s own drama series and a female captain to command the vessel.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

One comment

  1. Nick Gauthier says:

    I’ve been loading ~50k records from CSV into PG reliably for a while for a project.

    Using FasterCSV in combination with AR-Extensions ActiveRecord::Base#import.

    http://fastercsv.rubyforge.org/

    http://www.continuousthinking.com/are/import

    AR#import does the insert without having to instantiate the AR objects. So less memory and also much much faster.

    -Nick Gauthier
    SmartLogic Solutions

    August 23, 2010 at 6:35 am

Add New Comment Cancel reply

Your email address will not be published.

Will Read

Will Read
San Francisco

Recent Posts

  • Org Chart Growth and Keeping Our “Flatness”
  • Releasing When It’s Ugly
  • “Drinking the Kool-Aid™” How We Create Value Alignment
Subscribe to Will's Feed

Author Topics

agile (31)
bloggerdome (3)
process (1)
"soft" ware (16)
feedback (1)
engines (1)
documentation (1)
tdd (2)
bdd (1)
breakfast (1)
pairing (1)
testing (3)
api (2)
bug (1)
chore (1)
velocity (1)
oauth (1)
queues (3)
version control (1)
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >