A while back Matt Williams created a Java API for Remix:

Remix.java at Google Code

Thanks Matt!

• Best Buy Remix Homepage
• Kevin Matheny's article on Agile Software Development in Businessweek

We made a code change and deployed to demo, and all the sudden some of our ruby processes were eating a ton of CPU against our full dataset.

In the java world you can send a SIGQUIT to any running java process and get a thread dump. Go ahead, run a java process and kill -3 it.

You can get this in the ruby world by using xray:

sudo gem install xray

Drop this into your code:

require "xray/thread_dump_signal_handler"

Now:

kill -3 <ruby pid>

Look in the log file where you're sending stdout. You'll see something like:

=============== XRay - Done ===============
/usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.0/lib/eventmachine.rb:224:in `call'
    _ /usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.0/lib/eventmachine.rb:224:in `run_machine'
    _ /usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.0/lib/eventmachine.rb:224:in `run'
    _ /usr/lib64/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/backends/base.rb:57:in `start'
    _ /usr/lib64/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/server.rb:150:in `start'
    _ /usr/lib64/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/controllers/controller.rb:80:in `start'
    _ /usr/lib64/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/runner.rb:173:in `send'
    _ /usr/lib64/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/runner.rb:173:in `run_command'
    _ /usr/lib64/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/runner.rb:139:in `run!'
    _ /usr/lib64/ruby/gems/1.8/gems/thin-1.0.0/bin/thin:6
    _ /usr/bin/thin:19:in `load'
    _ /usr/bin/thin:19

(that's thin patiently waiting to service the next request)

A one-line code drop-in results in a powerful new inspection tool. Pretty neat.

For bonus points:

ps ax | grep "thin server" | grep -v grep | awk '{print $1}' | xargs kill -3

For more bonus points, stick this in a capistrano task and grab the thread dumps from your logs, and you'll have a cluster-wide snapshotting tool.

We kill -3'd our CPU-eating thins and discovered a directory scan problem introduced by a recent code change - totally obvious from the thread dump. Now we're nailing it down with a failing perf unit test and fixing the problem.

I'm here as part of the Best Buy Remix crew, hanging out in Mashery's Circus Mashimus all weekend. Come by, have a beer, and check out Remix and other interesting API stuff if you're at SXSW.

We're in a room near the front of the convention center, not far from the Pepsi booth.

-Steve

It's getting hard to keep up with all the Remix activity.

Matt Williams has put together a PHP library for Remix.

Code sample:

require_once 'BestBuy/Service/Remix.php';

$apiKey = '12345678'; // Your API key
$remix  = new BestBuy_Service_Remix($apiKey);

// Retrieve a list of stores within 10 miles of a zip code
$result = $remix->stores(array('area(10006,10)'))->query();

// Result objects may be implicitly cast as strings
echo $result;

// Retrieve a list of Movies containing the text "Bat"
$result = $remix->products(array('name=bat*','type=Movie'))->query();

if(!$result->isError())
{
    echo $result;
}
else if(403 != $result->http_code)
{
    // API errors result in an error document with detailed info
    echo current($result->toSimpleXml()->xpath('/error/message'));
}
else
{
    // 403 errors do not contain a full error document, only an h1 message
    echo $result->data;
}

// Retrieve fields from a list of Movies starting with "Bat" in JSON format
$result = $remix->products(array('type=Movie', 'name=bat*'))
            ->show(array('name','regularPrice','url', 'sku'))
            ->format('json')
            ->query();

echo $result;

// Check for store availability of a Playstation 3 in a given area
$result = $remix->stores(array('area(10006,10)'))
            ->products(array('sku=8982988'))
            ->sort('distance')
            ->query();

echo $result;

Omar is on fire. Just a few days after publishing his Guide to creating a Google App Engine / Remix API / Facebook app, he's released Remix.NET, a C# wrapper for the Best Buy Remix API.

Code sample:

Using Remix;

Product p = null;
try
{
    Server remix = new Server("username", "password");
    String filter = "iphone";
    String postalcode = "94102";
    String radius = "25";
    bool tersemode = true;
    int pagenum = 0;

    // Get all Hardgoods that match "iphone" in the name attribute and
    // are in stores in the 94102 area code in a 25 mile radius.
    // When "tersemode" is true, only return a handful of information. 
    // (See method signature for details.)
    Products list = remix.GetHardGoods(filter, tersemode, pagenum, postalcode, radius);

    if (list.Count > 0) p = list[0];
}
catch (Exception e)
{
}
return p;

Check out Omar Abdelwahed's* guide to using Google App Engine with Best Buy's Remix API to build a Facebook app.

Omar is also the author and maintainer of the bbyremix Twitter app.

Sign up for an api key at the Remix API website.

* friend of Pivotal

1. A great little Ruby tutorial using HTTParty with the Remix API.

2. Slashdot mention.

3. Finance and Commerce: Best Buy API Aims To Expand Store's Reach Online.

For more on Remix, check out the Remix API website (where you can sign up for a developer key).

This is suddenly more relevant after Adam's excellent post on Aristotle and software (and while you're at it don't miss There Is No Agile). The following is just an application of virtues to writing...and in my opinion, very relevant to the practice of making software.

Following from the assertion that you can apply writing advice to code...

Zen and the Art of Motorcycle Maintenance (book, wikipedia) is an exploration of Quality (wikipedia) (not in the sense of QA or building cars). This is a long passage from Chapter 17 - Phaedrus is teaching a writing course at a local college - one of my favorite parts of the book, it's a model for examination that I keep coming back to. My comments follow at the bottom.

"I think there is such a thing as Quality, but that as soon as you try to define it, something goes haywire. You can't do it."

(continued after the jump)

Edward pointed out the great article by Kevin Matheny, featured in BusinessWeek, on Agile, and our experience on BestBuy Remix.

I'd like to highlight this passage:

Trust is tied closely to how you deal with change. Often, extending trust is hard for businesspeople working on technology projects, because we don’t know how to do the work. We often look to the documentation — requirements, design specifications, and the like — to give us the feeling of control over the outcome. Don’t bother. If you can’t trust your team to deliver, you have the wrong team. Find people you can trust, and then let them do the work. Talk every day, and make sure that the development team has direct access to someone who will be using the product every day after release. For Remix, we’ve never had a formal project plan, never had a requirements-gathering session, never created a requirements document. We chose the right partners, told them what we needed, and got to work. We have control over the outcomes, but we’re not worried about trying to control the details of how we get there.

Of course without trust, any project - "agile" or not - is at risk. But practices typically associated with agile let you go further with trust: everyone is in close communication*, and all levels of the project - from test-driven code written by developers, to regular demos to the client of the latest features - are oriented around fast feedback.

In other words, you the customer trust us in part because what we're doing is visible and tangible to you. If we're going in a direction you didn't intend, or what the team planned a few weeks ago just doesn't seem relevant anymore, we all talk and we correct course.

* for close communication, see Pivotal Tracker

net/http is slow. (and so are libraries that depend on it, like open-uri)

Performance Disclaimer: this ought to matter in your app, measurably, before you do anything about it. If you profile and ruby-prof is showing a bunch of classes like BufferedRead and Timeout at the top of the list, your app qualifies. And in addition if you know that your app is dependent on data transfer over http (let's say you're interacting with a Solr server, and you're storing sizable documents in Solr), you should be aware of the problem.

Otherwise net/http or open-uri might be just fine for you.

The problems with net/http, and benchmarks of ruby http client lbraries are nicely written about in An analysis of Ruby 1.8.x HTTP client performance.

Some good alternatives:

• curb (libcurl wrapper)
• rfuzz
• httpclient / http-access2

Our findings matched the article referenced above - the alternatives have pros and cons but each was at least 10x faster than net/http for transfers of 50-300k response bodies.

The fastest solution we found was curb, reusing the Curl::Easy object:

require "curb"

curl = Curl::Easy.new

2.times do
    curl.url = "http://www.pivotaltracker.com"
    curl.perform
    puts curl.body_str
end