Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Tools
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker

Monthly Archives: May 2009

Monkey Patch Du Jour

Alex Chaffee
Sunday, May 24, 2009

The following monkey patch gives a bit more information in the ActiveRecord SQL logs. Instead of just saying “User Load” it also says the file and line number in your code that asked AR to perform the operation. That way you can have a hope of tracking it down and optimizing it away if at all possible.

  User Load (0.2ms) views/main_page.rb:107:in `filters_box'   SELECT * FROM `users`

Code after the jump. I guess you put it in environment.rb with all the other monkeys.

    module ActiveRecord
      module ConnectionAdapters
        class AbstractAdapter
          def log_info(sql, name, ms)
            if @logger && @logger.debug?
              c = caller.detect{|line| line !~ /(activerecord|active_support|__DELEGATION__)/i}
              c.gsub!("#{File.expand_path(File.dirname(RAILS_ROOT))}/", '') if defined?(RAILS_ROOT)
              name = '%s (%.1fms) %s' % [name || 'SQL', ms, c]
              @logger.debug(format_log_entry(name, sql.squeeze(' ')))
            end
          end
        end
      end
    end
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Jonathan Barnes

Title: Standup 05/22/2009: RubyMine – CentOS – Symlink Testing – JSON gem – EY Solo

Jonathan Barnes
Friday, May 22, 2009

Ask for Help

“I am having trouble installing Mechaize on CentOS any tips and tricks?”

None from the pivots…

“Any good tips on testing the creation of symlinks from ruby?”

shell out and do an ls -l then parse the string to see if it’s target is correct.

Interesting Things

  • RubyMine‘s changes -> repository pane only seems to show the current users changes (SVN) instead of all changes for the repository. Showing history of a file still shows all user changes.
  • The JSON gem overrides active supports to_json method. So if you are using the built in rails JSON support and you install the gem be aware that there are differences between them.
    • Word is this is better on Edge and the gem no longer conflicts with the built-in version
  • We have started moving some of our servers to Engine Yards cloud solution, and while it is still in the early stages it is looking like a promising solution.
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Jonathan Barnes

Title: Standup 05/21/2009: Slow dependent destroy's

Jonathan Barnes
Thursday, May 21, 2009

Ask for Help

“I have a dependent destroy that is taking 5sec+, What can I do to speed that up?”

  • try dependent delete on the leaves of the chain if you don’t have any after destroy hooks there that need running.
  • try cascade delete in your database (again as long as you don’t have any after destroys to worry about)
  • try marking the element at the top of the tree as deleted (using acts as paranoid or similar), then run an offline process to look for those records and destroy them and their dependent objects.
  • don’t worry about orphaned records and manually clean them up every once in a while (unrecommended but least development effort)
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Jonathan Barnes

Standup 05/20/2009: "Merging Large Data Sets – after_commit"

Jonathan Barnes
Wednesday, May 20, 2009

Ask for Help

“What are some good ways to merge large business location data sets?”

There was a bunch of input including the following:

  • You should create a scoring of how close the matches are.
  • Good admin merge tools are worth the effort to create.
  • Normalizing the data prior to the merge (i.e. pass the addresses through the USPS API to turn [Av Ave Avenue] => Ave)
  • Humans do this best, outsource or Mechanical Turk it.

Interesting Things

  • The after_commit plugin allows you to hook events to after the transaction commits. This is really useful when kicking off threads that expect to have access to the data in the database. Note: using after_save can cause you to have a race condition if the other thread attempts access to the data before the original thread has a chance to commit the transaction.

  • If you are storing a marshaled object in the database, you should make that field a blob type, it is smaller to store and if you leave it as a text or varchar you can corrupt the binary data you are storing in there. If you don’t have a choice about field types you should at least base64 encode the marshaled data before storing it.

ctrl+z

RE: “NewRelic Side Effects?” from 05/19/2009 Standup

  • It seems that NewRelic was not the cause of the problem but helped in exacerbating the problem by holding the transaction open long enough to create a race condition that still shows up when the system is put under enough load. To fix our problem we moved the trigger that launches the background process from and after_save to an after_commit see plugin. We also re-added NewRelic.
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Javascript Snacks, Nibble #1: RTE/WYSIWYG is Built in to your Browser

Pivotal Labs
Wednesday, May 20, 2009

The Label

“Tasty! No Trans-Fats.”

I’ve now worked with a number clients who have a taste for a “rich text” editor
experience. Each time, the team initially looked in to using an
off-the-shelf package, e.g.:

  • TinyMCE
  • FCKeditor
  • WysiHat

Each of these client projects had fairly basic needs.
Pretty much everyone wants “bold”, “italic”, “underline”. Throw in bulleted
and numbered lists and perhaps a few other trinkets and you’ve pretty much
covered the use cases.

It turns out that the existing packages are generally tricky to integrate and
modify or configure per your particular needs. This makes sense. A full-blown
Javascript RTE package needs to have capabilities far in excess of anything
a given project requires, simply in order to meet every project’s
requirements. You end up with a massive library that is not trivially
integrated into your site. What to do some TDD/BDD on the RTE interaction
with your page? Good luck.

So, I’ve generally suggested that we roll our own.

I know, crazy. Why write something new and custom when so many others have
already solved all of the issues with, for example, cursor movement and text
editing and command handling (e.g., making the text “bold”) and image
placement, and…?

Well, it turns out that all of that
behavior is already built in to the browser. Yes, my browser, your browser.
Pretty much every browser you need to care about1.
On top of that, the basics are extremely simple.

To many, this may not be news at all. The basic capability has been around since MSIE 5.5 was released in 2000. On the other hand, I’ve worked with a number of very smart and seasoned web developers who were unfamiliar with the extent to which it is the browser, rather than the libraries, that has taken care of the hard parts.

Ingredients

What you’ll need to make this work2:

// enable editing:
document.contentEditable = true;
document.designMode      = 'on';

// make something bold:
document.execCommand('bold', false, []);

It’s pretty much as simple as that. Give it a try: paste the following
snippet into your browser’s address bar, hit enter, and edit text or resize
images on this page to your tummy’s delight.

javascript: function enable_DM() { document.contentEditable = true; document.designMode = 'on'; }; enable_DM();

Preparation

So, with those ingredients we can make quite a mess of the page we’re visiting,
but we’ve hardly created a delicious RTE. To go the next step, we need a way
grab the HTML representation of our edited content to send with a form
submission.

The basic idea here, which is what every editor out there employs,
is to embed an <iframe />, the document of which is editable, and when
appropriate copy the HTML of that document to a (likely hidden) textarea within
your form.

Why an iframe? There are two important reasons:

  1. You probably want to protect the look and behavior of your edited document
    from the styles and scripts of the rest of your site.
  2. Until version 3, Firefox only supported designMode, which is applicable
    to the document, but not to an element within your page.

There are slight browser differences that will come in to play, e.g. MSIE’s
IFrameElement.document property versus Firefox’s (and the W3C standard)
IFrameElement.contentDocument. As long as your editor remains fairly simple
these intricacies will cause very little trouble.

I’ll leave it to you to hook things up to your liking, but will toss in this
morsel: On a current project, we’re using a home-baked RTE which, from 100
very-nicely-formatted lines of Javascript (not counting the supporting
jQuery and Disco
libraries), is providing…

  • DOM building of the iframe and command toolbar (<ul /> with “buttons”), etc.
  • Document editability
  • Registration and handling of 8 commands (e.g., “bold”, “italic”, “indent”)
  • Content transfer to-and-from an associated textarea
  • Additional helpers (e.g., get the currently-selected text as HTML)

Tasting Notes

  • 1 My apologies to visitors using a
    specialized browser for accessibility.
  • 2 Setting document.designMode
    should be enough, but there are reports of discrepancies in MSIE’s handling
    of the two.

Shopping List

  • An MDC article on Mozilla’s Implementation
  • Another MDC article, highlighting some differences with MSIE
  • An MSDN article on MSIE’s Implementation
  • Mark Finkle on contentEditable in Firefox 3
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Jonathan Barnes

Standup 05/19/2009: Tool Annoyances; Better Firebug SASS Support

Jonathan Barnes
Tuesday, May 19, 2009

Ask for Help

“Who is getting svn: Server sent unexpected return value (403 Forbidden) in response to PROPFIND in SVN?”

Lots of people! This is a known bug when a SVN project has to traverse the repository using more than one user. Check out subversion: Issue 3242.

“Has anyone gotten ImageMagic to work with Phusion Passenger on CentOS?”

It seems to work with Mongrel, but not Passenger.

“Who is getting svn: Server sent unexpected return value (403 Forbidden) in response to PROPFIND in SVN?”

Interesting Things

  • Firebug 1.4 – Alpha

    • It has some cool new features, such as showing you CSS comments — handy for SASS debugging! Works in conjunction with enabling SASS line numbers in the CSS output.
    • It also supports FireDiff, which “records all of the changes made by firebug and the application itself to CSS and the DOM”.
  • NewRelic Side Effects? We have some background processes that get kicked off from after saves of some of our ActiveRecords. These processes use Active Resource to post back updates to these records. Unfortunately the transaction that creates the record initially doesn’t seem to have completed before the first post-backs attempt to update them. (race condition). After trying a bunch of stuff we randomly removed the NewRelic plugin and all of the sudden we had no more race conditions. Is the NewRelic plugin holding the DB transactions open too long?

  • MySQL index sample size: We discovered that MySQL’s ANALYZE TABLE uses a fixed sample size from the index regardless of the size of the table and thus can have really incorrect cardinality and which could lead to it choosing the wrong index. Also ANALYZE TABLE gets run often (opening new connections, first read of table) and due to the problem mentioned above can cause the system to switch from using the right index to the wrong index each time you look at it.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Jonathan Barnes

Standup 05/18/2009: Ruby Profiling

Jonathan Barnes
Monday, May 18, 2009

Ask for Help

“Does time spent in the C code show up in the ruby-prof output?”

Yes it does — There are two stats for method time: “Self” [time in method] – [time spent in called children]), “total” [total time in method]. The second should include time spent in the C code

It was also mentioned that it is much more useful to look at the ruby prof output as HTML or even better using KCachegrind (though it was mentioned that it can be a bit of a pain to get it installed see KCachegrind OS X 10.5.6

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Davis W. Frank

Standup 5/13/2009: Whole Lotta Blog Edition

Davis W. Frank
Thursday, May 14, 2009

Interesting Things

  • Webrat + Selenium + IE7 => FAIL – The problem is in the file label.js, which is custom matcher code that Webrat injects into Selenium at run-time (EEK –ed.). That code was not IE7 compliant. The team decided to fork Webrat, pull out selenium-rc and use their own selenium-rc gem to solve the problem. There’s a Lighthouse ticket for this this issue.
  • For those of you upgrading to Rails 2.3, one team had a large chunk of issues that made the upgrade take over a day. They had myriad issues with: Polonium, RSpec, & Cucumber. Another team reports that ActiveScaffold also made things difficult. Before you point to DHH’s RailsConf 2009 Keynote where he said “Don’t upgrade if things are working”, the former team wanted to use…
  • default_scope – This is a great idea, but it’s not yet ready for prime time. It can’t be arbitrarily chained with ActiveRecord associations (much like acts_as_paranoid was originally implemented) – so you need to add calls it to the end of your chained association & scoping calls. Which doesn’t really make it default, does it?
  • Think that your brand new download of RubyMine is a little slow? Make sure you have Java 1.6 installed, then open the plist file (locations vary by host OS) and change the value that says 1.5* to 1.6*. The Mine will now require Java 1.6, which is known to have snappier garbage collection. (I tried this today and found fewer beach balls of tranquility. YMMV. –ed.).
  • Stack Overflow DevDays were announced. It’s a set of one-day conferences with San Francisco’s being October 19th.

Help

Is Rails’ built-in protect_from_forgery worth using? Does anyone turn it off?

The overall consensus is, “Of course we use it! Why would you turn off this default?” Well, It’s a little bit of pain when you’re constructing your own Ajax requests – it breaks perf tests, but it’s turned off for other tests. You may get Invalid Authenticity Token exceptions. (Joseph?)

How do you aggregate results of background tasks in a nightly report?

At least turn on the cron flag for emailing you results of cron jobs. Or, look at Delayed Job. Since tasks live in the database, you could make a status page or have another job to mail a report.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Davis W. Frank

Standup 5/11/2009: IE Extreme Edition

Davis W. Frank
Tuesday, May 12, 2009

Interesting Things

  • Using Eric Meyer’s reset.css in an <IFRAME> on an undetermined patch level (but not up-to-date) version of IE6 results in a horrible browser freeze – the kind where it doesn’t chew CPU cycles & can’t seem to update it’s window at all. The team solved the problem by dropping reset.css and (GASP! –ed.) inlining the necessary styles to make the <IFRAME> look as required.
  • IE8.js made IE7 kind of horrible. It’s a known issue with this library that big DOMs and/or a lot of CSS can slow the browser down enough as to make it unusable. While IE8.js is configurable (dropping unneeded features), the team wasn’t using enough of its features and moved on.
  • If you’ve been playing with Compass, a (cool –ed.) CSS framework in SASS that we’ve talked about before, you’ve had to build your own HAML gem from scratch since Compass has depended on HAML edge. The open source elves have been working a bit of overtime and have made the haml-edge gem that removes this step. Check out this Compass commit for more detail.

Help

Who Want’s to Buy My Mom’s House in Salmon, Idaho?

Pivot Joe has launched his own Internet reality show. Contestant applications are being accepted now.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Capturing Standard Out In Unit Tests

Alex Chaffee
Monday, May 11, 2009
    def capturing_output
      output = StringIO.new
      $stdout = output
      yield
      output.string
    ensure
      $stdout = STDOUT
    end

then…

    it "exits immediately from --version" do
      output = capturing_output do
        lambda {
          Erector.new(["--version"])
        }.should raise_error(SystemExit)
      end
      output.should == Erector::VERSION + "n"
    end
  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (783)
  • rails (117)
  • testing (90)
  • ruby (86)
  • ruby on rails (71)
  • jobs (62)
  • javascript (59)
  • techtalk (44)
  • ironblogger (42)
  • rspec (39)
  • bloggerdome (34)
  • productivity (34)
  • activerecord (30)
  • rubymine (30)
  • git (29)
  • gogaruco (29)
  • nyc (27)
  • design (24)
  • mobile (23)
  • pivotal tracker (22)
  • process (21)
  • cucumber (21)
  • jasmine (19)
  • ios (18)
  • tracker ecosystem (17)
  • webos (17)
  • objective-c (17)
  • fun (16)
  • android (16)
  • palm (16)
  • ci (16)
  • "soft" ware (16)
  • bdd (15)
  • tdd (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • css (14)
  • gem (13)
  • mouse-free development (12)
  • selenium (12)
  • goruco (12)
  • bundler (12)
  • api (12)
  • keyboard (11)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
Subscribe to Community Feed
  1. ←
  2. 1
  3. 2
  4. 3
  5. 4
  6. 5
  7. 6
  8. 7
  9. 8
  10. →
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Tools
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >