Pivotal Labs

Main menu

Skip to primary content
Skip to secondary content
  • About
  • Case Studies
  • Team
    • Executives
    • Locations
      • San Francisco (HQ)
      • Boston
      • Boulder
      • Denver
      • London
      • Los Angeles
      • New York
  • Community
    • Blogs
    • Tech Talks
    • Events
  • Careers
    • Lifestyle
    • Principles & Practices
    • Benefits
    • FAQ
    • Apply
  • Contact
    • Press Room
    • Press Releases
    • In The News
    • Press Kit
  • All
  • Labs
  • Standup
  • Tracker

Monthly Archives: February 2008

Java Stink

Alex Chaffee
Monday, February 11, 2008

After about two years in which the only Java I wrote had a “Script” after it, I’ve recently started working in my old favorite language again. It was clear to me long before I made the leap that somewhere along the line Java took a sharp turn towards Scarytown. (Maybe the writing was on the wall when the “Hello World” program comprised five lines, two declarations, and a static reference, but back in 1995 we were all so excited about getting objects without C++ that our judgement was clouded.)

Anyway, I will always have a place in my heart for the old bird (picture a portly English matron with flower dress and pocketbook and floppy hat), but Stu at Relevance Blog points out why coding in Java now feels like trying to sprint with 30-pound weights strapped to my ankles.

Java is a high-ceremony language. At every turn, Java enforces a high busy-work/real-work ratio. Specifically:

  1. Java’s checked exceptions bloat code, make components harder to use and maintain, and lead to tons of boilerplate code, each line of which is a bug-in-waiting.
  2. Java’s new operator/constructors cannot pick a return type. The amount of code that exists only to work around this is staggering. Two entire cottage industries have sprung up to deal with this single issue: factory patterns and dependency injection.
  3. Java has no metaprogramming features to automate common tasks such as field accessors, standard constructors, and simple delegation.
  4. Primitives, functions, and classes are not first-class objects, leading to huge code bloat to deal with these types specially.
  5. Java’s core reflection and interception capabilities are clunky, requiring tons of bolt-on technologies to make them workable, including AOP, annotations, and code generators.

That’s a pretty big stink, but if you are used to it you probably can’t smell it anymore.

(And that’s not even mentioning the prevalent idioms of programming with massive amounts of indirection and wrappers and statics and service locators and and BigLongClassNamesThatIncludeTheirAncestry (I always say, “Do we call it a DogMammalVertebrateAnimal? No, we call it a dog!”) and redundant JavaDoc on every method and…)

I ranted and spoke and even blogged about some of these issues before, but now that I’m a visitor in that world I just feel vaguely amused and sad when I see all the hoops Java programmers still have to jump through. Yeah, control-space completion is nice, but gotapi works pretty well, and at the end of the day, no matter how many curly braces my IDE inserts for me, I’d rather have my code look like this:

parse_args(["--topping", "pepperoni"])

than this:

String[] args = {"--topping", "pepperoni"};
parseArgs(new ArrayList<String>(Arrays.asList(args)))


Wouldn’t you?

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Parker Thompson

Screen Scrape No More…Seriously!

Parker Thompson
Sunday, February 10, 2008

This week I had the pleasure of attending Dapper Camp put on by the folks at Dapper for their user community. Mitch Kapor kicked it off with a talk on disruptive technology, openness, and innovation. We then got to hear both from the Dapper team about new and upcoming features, and folks like Aaron Fulkerson of MindTouch about using Dapper to repurpose data. All around it was pretty interesting.

What I love about Dapper is that it helps solve one of the big issues I see our clients have: data. We can build just about anything, but if an application needs some specific data (and many do), products must be launched with sub-par (but available) data, or worse launches can end up being delayed. In many cases, we can end up spending a large amount of time (aka money) getting/munging data rather than developing features. Note: I also think the ease if pushing data out of apps via instant gadgets makes Dapper very interesting but that’s a whole separate post.

The “Dapp Factory,” a Rhino-based server application and a web front-end that deals with just about any site by proxying your requests and modeling the DOM on the proxy, then recording your actions for later replay. But, their secret sauce is a super-cool algorithm that figures out the structure of pages in such a way that your API can withstand changes to the target site, making your feed resilient to all but massive site overhauls. You then simply consume an XML or JSON feed, or use a simple API to dynamically construct paramaterized feeds.

There are other companies trying to make data less painful. Metaweb, for example, provides an incredibly fast graph engine and relational schemas (think RDF) that makes real-time use interesting. But, if the data you need isn’t in Freebase (a likelihood until they get larger), or your data is continually being updated, you will still be stuck scraping and relating the data, and that’s generally where most of the work is.

Take as a small example Dav’s awesome Vacation Planner. The concept is simple, the feature set is small, but getting the data is a pain (see article). Some sites don’t have APIs, and those that do provide unstansardized, sometimes buggy, and are often often missing the data you need.

I could imagine writing a dapp parser akin to ActiveResource pretty easily (I hear a ruby SDK came out of DapperCamp, but I can’t find it). With a little more work, it would probably be easy to add cache_fu support, and ruby modules that could be mixed into models (for asynchronous data gathering) and controllers (for serve now vs. polling) to easily support Dav’s polling mechanism.

This would leave Dav with pretty simple model (data) code, and the luxury of focussing on whether to add wikipedia integration for population figures or the Big Mac Index, rather than tweaking his Mechanize xpaths all weekend. I vote for the Big Mac Index.

So, the next time someone suggests you screen scrape to get that data you need, tell them to give Dapper a shot. And if anyone wants to write ‘ActiveDapp’ let me know. It could be really fun.

Note: In the spirit of full disclosure, Jon Aizen (Dapper CTO) is a friend of mine and they gave me a free t-shirt…and a sandwich (thanks).

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

User Research As A Commodity

Alex Chaffee
Saturday, February 9, 2008

The intersection of development and UI is an area I’ve got some interest in.
Arena just shared this amazing article about a team’s experience merging Agile project management with User Research and testing.

User Research as a Commodity (part 3 of 7) by Tim Kieschnick

Here (below the fold) are some of the things in the article I found eye-opening.

  • “When asked, the sponsors and most of the team thought the best thing to do would be to crank out an initial site on a tight timeline, doing the best we could based on intuition and a few heuristics, and then test it with users after we went live. …Clearly, the UX people didn’t understand agile, and the software people didn’t understand UX.” It’s interesting to me, standing mostly on the software side of the fence, that — much like there are developers who don’t get agile, or, getting it, don’t make time for it — there are UI designers who don’t get User Testing or don’t make time for it.
  • Despite initial fears over wasting time or slowing progress, what mattered in the end was that they set up a regular time once a week to meet with users and do user testing. “We virtually eliminated arguments within the team about what would work best for users. Instead of pressing the point, the Product Manager could just say, “I’ll ask them this Thursday.”” This is the key to the agile process: if you care about something, pay attention to it, and then (the hard part) keep paying attention to it. Have a standup every day, a demo every week, a retrospective every release — and a user test every Thursday.
  • Leveraging slack. “What if teams don’t generate enough tasks to fill up the time this week? No problem–our “Platform UI Team” will maintain a backlog of non-urgent areas to test with each audience, so they can fill in as needed.” This is something I see a lot with teams that are successful at what I will somewhat pompously call Journeyman Agile: they are so good at pumping out features that that’s all they do. They’re addicted to points. They often forget to take time for chores like large-scale refactorings or infrastructure improvements or even retrospectives or water-cooler time or “sustainable pace” (i.e. actually leaving the office). And since Agile developers are such nice guys, they rarely push back when their managers — who aren’t confronted directly with the pain of crusty code or communication — keep asking for higher and higher velocity.
  • At one point (in the speculative conclusion) the author mentions the metrics mantra: “What gets measured gets managed.” I am struck by how insidiously similar this is to one of my Alexisms: “If you pay attention to something, it gets better.” Why insidious? Because if you happen to pick the wrong metric, you’re screwed. But “pay attention”
    implies you’re not a slave to the digits, but are actually having intelligent conversations about the thing, and maybe picking or discarding metrics as you go.
  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Standup 02/07/2008

Pivotal Labs
Thursday, February 7, 2008

Interesting Things

  • There’s an IE PNG transparency bug
    fix as a plugin for jQuery.
    It looks fairly complete, with more coverage than many others out there and the usage is trivial (once installed):

    <script type="text/javascript">
      $(document).ready(function(){
        $(document).pngFix();
      });
    </script>
    

    The major downside so far: it scales background images.

  • Rails Bug:

    In a failing/rolled back transaction, those ActiveRecord
    objects that were created prior to the exception still have IDs and respond
    false to object_name.new_record? (which would seem to indicate that a record for the object has been saved).

    For more info, here are a couple bug tickets.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Standup 02/05/2008

Pivotal Labs
Tuesday, February 5, 2008

Interesting Things

  • In exploring options for RSpec testing of XML responses, one project decided
    a custom XPath matcher
    would do the trick nicely:

    items = Item.find(:all)
    get :index
    
    
    response.body.should     have_nodes("//items/item", items.size)
    response.body.should     have_xpath("//items/item[ position() = 1 and @id = '0001' ]")
    response.body.should_not have_xpath("//bogus")
    

    Some other possibilities:

    • Hashes…

      items = Item.find(:all)
      get :index
      
      
      hash = Hash.from_xml(response.body)
      hash['ancestor']['parent']['items']['item'].size.should == items.size
      hash['ancestor']['parent']['items']['item'][0]['id'].should == '0001'
      hash['ancestor']['parent']['bogus'].should be_nil
      
    • Hpricot…

      items = Item.find(:all)
      get :index
      
      
      doc = Hpricot(response.body)
      doc.search("//items/item").size.should == items.size
      doc.at("//items/item[ @id = '0001' ]").position.should == 1
      doc.at("//bogus").should be_nil
      
    • assert_select from Rail’s Test::Unit (using CSS-style selectors)…

      items = Item.find(:all)
      get :index
      
      
      assert_select("items > item", :count => 5)
      assert_select("items > item:nth-child(1)[id=?]", '0001')
      assert_select("bogus", false)
      

    What are your favorite techniques for asserting XML/XPath?

  • Something to consider when test-driving controller code:

    You’re working with ActionController::TestResponse. So, response.success?,
    response.redirect?, etc. are available for you there (since they are
    defined on TestResponse), but not in your actual controllers. That is,
    things blow up if you try to use @response.success? in your application,

    e.g. to determine whether or not to store the current URL and redirect there
    after a login.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter
Pivotal Labs

Happy Path Testing With Selenium RC Fu

Pivotal Labs
Tuesday, February 5, 2008

Selenium RC Fu is a fantastic system for testing Ruby On Rails
applications. It is the blending of xUnit testing with Selenium.
Selenium is a cool system that operates your browser as if a human were sitting
there moving the mouse pressing buttons and keys.

Selenium RC Fu is also a remarkable example of the power of open
source. It’s selenium remotely controlled by rails and
ruby. You can learn more about it by viewing the slides for
Full-stack webapp testing with Selenium and Rails
presented by my colleagues Alex Chaffee and Brian Takita at the
SDForum Silicon Valley Ruby Conference.

Now that you are excited about Selenium RC Fu, by law I
must inform you that this wonderful testing tool comes with some
costs. First, this is the daisy cutter of testing — problems will be detected,
but it won’t be too specific about those problems. A failed selenium
test will likely only tell you some expected text
was not present on the page — you have to do some digging
to discover the real problem.

It’s also slow. To be fair, a lot of software is running to do this testing.

So use selenium testing sparingly. A good strategy is to restrict
selenium testing to “happy path” testing. These happy path tests
become a compliment to other more focused and faster unit and integration
tests.

Getting Started With Selenium RC Fu

Selenium RC Fu is hosted at rubyforge.org in the “pivotal.rb”
project. It’s not well documented and a little hard to find. The
primary documentation is the
README File.

Step 1 is to add it to your project:

script/plugin install svn://rubyforge.org/var/svn/pivotalrb/seleniumrc_fu/trunk

Building Your Tests

Create your tests in app/tests/selenium. The basic structure mirrors
the other test types:

require File.dirname(__FILE__) + "/selenium_helper"
class HappyPathsTest < MyProject::SeleniumTestCase

  def test_nav_bar
      ...
  end

  def test_about
      ...
  end

  # more tests
end

Note that HappyPathsTest extends something called
MyProject::SeleniumTestCase. Selenium RC Fu provides a sample
selenium helper file that suggests this convention for name spacing
your tests.

For the happy paths tests I simply wanted to go to a page and know
that the page loads. A pattern I learned from my colleague
Shifra is:

  1. Pick something you know is on the target page, but not on the
    current page. Let’s call it ‘evidence’. Assert that evidence is not
    present on the current page.
  2. go to the target page
  3. assert that evidence is present on the target page.

This process ensures that you are moving about as you expect. Here’s
an example from my project:

  def test_tasks
    ...

    assert_element_not_present "xpath=//h1[text()='The Daily Planet']"
    click_and_wait "link=The Daily Planet"
    assert_element_present "xpath=//h1[text()='The Daily Planet']"

    ...
  end

I repeated this simple pattern for every path.

Selenium Functions/Assertions/Commands

Here are the functions and assertions I used in my tests:

  • click_and_wait locator
  • go_back
  • wait_for_page_to_load
  • type locator, text
  • assert_text_present text
  • assert_text_not_present text
  • assert_equal value1, value2

Many of the functions require a ‘locator’ argument. This needs to
identify a single element on the page. Often the name or id of an
element is sufficient, but you may need to use an XPath. Check
the docs
for more information on element locators.

These functions were sufficient for me. Look for more in
vendor/plugins/seleniumrc_fu/lib/seleniumrc_fu/selenium_dsl.rb.

Selenium RC Fu comes with some rake commands too (from rake -T selenium):

rake selenium:restart_servant           # Stop and start the selenium servant (the server that launches browsers) on localhost
rake selenium:run_server                # Run the selenium servant in the foreground
rake selenium:server                    # Run the selenium remote-control server
rake selenium:start_servant             # Start the selenium servant (the server that launches browsers) on localhost
rake selenium:stop_servant              # Stop the selenium servant (the server that launches browsers) on localhost
rake selenium:test                      # Run a selenium test file (default test/selenium/selenium_suite)
rake selenium:test_with_server_started  # Run the selenium tests in Firefox

Running Your Tests

You can run your selenium tests individually or as part of a
suite. Selenium RC Fu comes with a suite script in the
vendor/plugins/seleniumnrc_fu/sample directory. Drop that in your
test/selenium directory.

The selenium server must be running before running individual
tests. You can start the server with the rake command rake
selenium:server
. Once the server is running you run your tests from
the command line or via the suite.

You can also run the suite with the command rake selenium:test. This
command is smart enough to start the server if it’s not already
running. However, when the server is started by this mechanism it’s a
little more work to stop it. You can stop the server with a simple ^c
if it was started via the rake selenium:server command.

Make It So

That’s really all you need to build your tests, but here are few more
things that may help.

Site Diagram

A site diagram is handy for building your happy paths tests. There
will be a lot of paths through the site and marking the paths on the
diagram as you visit them is an easy way to track your progress.
On the diagram you need to indicate both pages and the paths to those
pages.

The later is really important. It’s the paths that you will be
testing with your happy path selenium testing. Often there are
buttons, links, or other controls such as check boxes that may touch
the server and reload the current page, or just update the current
page via ajax. Be sure to add these to your
navigation diagram.

Here’s the diagram I used for my site.

site diagram

Selenium IDE

You can code your tests by hand using Selenium RC Fu functions but
it’s easier if you use
Selenium IDE. This is a
Firefox plugin that records your activity. That activity is available
in various selenium dialects and can be pasted into your
tests. Unfortunately, it does not have output for Selenium RC
Fu. Nonetheless I still found it useful. I simply clicked through my
application while Selenium IDE was recording, pasted the
resulting selenium commands into my tests, then converted those
commands to equivalent Selenium RC Fu commands. For example:

Output from Selenium IDE:

@selenium.click "link=My Account"
@selenium.wait_for_page_to_load "30000"
assert @selenium.is_text_present("Update Account")

Becomes:

click_and_wait "link=My Account"
assert_text_present "Update Account"

Test Data

Your selenium tests will be exercising your application just like
standard unit or integration tests. These tests require data and there
are a variety of techniques to make data available to your
tests. Building your happy path tests is easier if you have fixture data and that fixture data is loaded in your development environment via rake
db:fixtures:load
. If you use fixture scenarious, it
is handy to have a scenario for selenium testing, which you would load
with something like rake db:scenario:load SCENARIO=selenium.

With your fixtures loaded in your development environment you can see
exactly what is present during testing and the selenium IDE will
record exactly what is played back by selenium.

XPath Checker

Usually Selenium IDE can assign a locator that simply works but there
are times when it can’t.
Using XPath Checker you can
right click on an element and it will display an XPath to that
element. You can also experiment with varitions and XPath Checker will
list all of the elements on the page that can be identified by that XPath.

That’s it. Let me know how it goes.

  • 0 Shares
  • Share on Facebook
  • Share on Twitter

Topics

  • agile (780)
  • rails (113)
  • testing (88)
  • ruby (83)
  • ruby on rails (70)
  • jobs (62)
  • javascript (55)
  • techtalk (44)
  • rspec (38)
  • ironblogger (32)
  • productivity (30)
  • activerecord (29)
  • gogaruco (29)
  • git (28)
  • nyc (27)
  • rubymine (26)
  • bloggerdome (23)
  • mobile (22)
  • process (21)
  • pivotal tracker (20)
  • cucumber (20)
  • jasmine (19)
  • design (18)
  • ios (18)
  • webos (17)
  • objective-c (17)
  • android (16)
  • palm (16)
  • "soft" ware (16)
  • fun (15)
  • tracker ecosystem (15)
  • ci (15)
  • cedar (15)
  • rails3 (14)
  • performance (14)
  • bdd (14)
  • gem (13)
  • css (13)
  • tdd (13)
  • selenium (12)
  • goruco (12)
  • bundler (12)
  • meetup (11)
  • railsconf (11)
  • nyc-standup (11)
  • capybara (10)
  • mac (10)
  • mojo (10)
  • chef (10)
  • api (10)
Subscribe to Community Feed
  1. ←
  2. 1
  3. 2
  • About
  • Case Studies
  • Team
  • Community
  • Careers
  • Contact
  • Labs
  • Events

Contact Us

contact@pivotallabs.com
+1 415-77-PIVOT
TwitterLinkedInFacebook

Pivotal Tracker

Tracker is the award-winning agile project management tool that enables real-time collaboration around a shared, prioritized backlog.
Visit pivotaltracker.com >