SEO-friendly single-page apps in Rails

Thursday, May 9, 2013

TL;DR version

To make all this process as simple as possible, a variation of the third approach (Rack middleware + Selenium Webdriver + no caching) is available here as a Gem. Drop it in your project, have the dependencies installed, and may the SEO gods bless you!

The whole story

Much has been said about how hard it is to build a single-page app that responds well to crawlers. Most search crawlers don’t support Javascript, which means those applications which rely completely on client-side rendering will serve blank pages.

Luckily, there are several approaches one can take to circumvent the lack of faith from certain crawlers – there’s obviously no “one size fits all” approach, so let’s take a minute to go through three of the most commonly used approaches, highlighting pros and cons of each one of them.

Render everything in two “modes” (the no script approach)

This strategy consists of rendering your app normally, BUT with pieces of “static” content already baked in (usually inside a <noscript> block). In other words, the client-side templates your app servers to be rendered will have at least some level of server-side logic on them – hopefully, very little.

Although it’s a somewhat natural approach for a developer used to rendering templates and content on the server side, it leads to a scenario where everything has to be implemented twice – once in the javascript templates/views, once in the pages, making everything hard to maintain (and potentially out of sync) real quick. That Product detail view now includes a “featured review”? No problem, have its text rendered on the server-side (and returned by the JSON endpoint your client app will use to render it again in a Javascript template).

This DOES work well for mostly-static, single-content apps (eg.: blogs), where even the javascript app itself would benefit from having the content already pre-loaded (the javascript views/snippets/segments would effectively fiddle with blocks of content, instead of fetching them from the server).

It’s worth noting that you should NOT rely on rendering just bits of content when serving pages to bots, as some of them state that they expect the full content of the page. It’s also worth pointing that Google also the snapshots it takes when crawling to compose the tiny thumbnails you see on search results, so you want these to be as close to the real thing as possible – which just compounds on the maintenance issues of this approach.

The hash fragment approach

This technique is supported by Google bot alone (with limited support by some other minor search bots – Facebook’s bot works too, for instance) and is explained in detail here.

In short, the process happens as follows:
The search bot detects that there are hash parameters in your URL (eg.: www.example.com/something#!foo=bar)
The search bot then makes a SECOND request to your server, passing a special parameter (_escaped_fragment_) back. Eg.: www.example.com/something?_escaped_fragment_=foo=bar – it’s now up to your server-side implementation to return a static HTML representation of the page.

Notice that for pages that don’t have a hash-bang on their URL (eg.: your site root), this also requires that you add a meta tag to your pages, allowing the bot to know that those pages are crawlable.

<meta name="fragment" content="!">

Notice the meta tag above is mandatory if your URLs don’t use hash fragments (which is becoming the norm these days, due to the amazing adoption of html5 across browsers) – analogously, this is probably the only technique of these three that will work if you depend on hash-bang urls on your site (please don’t!).

You still have to figure out a way to handle the _escaped_fragment_ requests and render the content these are supposed to return (like the previous approach), but at least it takes that content away from the body of the pages served to regular users (reducing, but not eliminating, the duplication issue). This works somewhat well on sites which part of the content is dynamic – not so much on single-page apps. No universal search bot support is also an obvious downside. Plus, you still have to pick a strategy to render the content without Javascript when the _escaped_fragment_ request arrives. Which leads us directly to the third approach…

Crawl your own content when the client is a bot

Although this seems counterintuitive at first, this is one of the approaches Google suggests when you’re dealing with sites whose majority of the content is generated via javascript (all single-page apps fall on this category).

The idea is simple: if the requester of your content is a search bot, spawn a SECOND request to your own server, render the page using a headless browser (thankfully, there are many, many options to choose from in Ruby) and return that “pre-compiled” content back to the search bot. Boom.

The biggest advantage of this approach is that you don’t have to duplicate anything: with a single intercepting script, you can render any different page and retrieve them as needed. Another positive point of this approach is that the content search bots will see is exactly what a final user would.

You can implement this rendering in several ways:

A before_filter on your controllers checks the user-agent making the request, then fetches the desired content and return it. PROS: all vanilla-Rails approach. CONS: you’re hitting the entire rails stack TWICE.
Have a Rack middleware detect the user-agent and initiate the request for the rendered content. PROS: still self-contained on the app approach. CONS: need to be careful on which content will be served, since the middleware will intercept all requests.
Have the web server (nginx, apache) handle the user-agent and send requests to a different server/endpoint on your server (eg.: example.com/static/original/route/here) that will serve the static content. PROS: only one request hits your app, CONS: requires poking around the underlying server infrastructure.

As for how to store the server-side rendered content (again, from worst to best):

Re-render things on every request. PROS: no cache-validation hell, CONS: performance.
Cache rendered pages, optimally with a reasonable expiration time PROS: much faster than re-fetching/rendering pages every time, CONS: cache maintenance might be an issue.
Cache rendered pages in a way that the web server can fetch them directly (eg.: save them as temp files). PROS: INSANELY FAST. CONS: huge cache maintenance overhead.

There are a few obvious issues you have to keep in mind when using this approach:

Every request made by a search bot will consume two processes on your web server (since you’re calling yourself back to get content to return to the bot)
The render time is a major factor when search engines rank your content, so fast responses here are paramount (caching the static versions of pages, therefore, is very important).
Some hosting services (I’m looking at you, Heroku) might not support the tools you use to render the content on the server side (capybara-webkit, selenium, etc). Either switch servers or simply host your “staticfying” layer somewhere else.

leave your migrations in your Rails engines

Ben Smith

Wednesday, May 8, 2013

If you are using Rails engines to break up a single app into modular pieces, migrations (as they are currently implemented in Rails 3.2.13) become clumsy.

There are three options for migrations within an engine (spoiler: #3 is the best):

1) You can use the your_engine_name:install:migrations rake task, which copies the migrations out of the engine and into the wrapping Rails app where they can be run normally. This works fine if your migrations in your engine never change, but if you’re actively developing your engine you need to run this rake task each time you add a migration.

2) You can put all your migrations in your wrapping Rails app. This works if you’re using your engines as a way to break up your app, but it doesn’t feel right. If your models, views, and controllers all live within the engine (and depend on migrations), shouldn’t your migrations live within the engine as well? If your migrations live in the wrapper Rails app, you actually create a weird upward dependency where the engine is actually dependent on the wrapper app. This is bad.

3) You can monkey patch Rails so all of your engine’s migrations automatically get run in the wrapper Rails app. Everything just works, and migrations live where they should: in the engine. If you’re breaking up your large Rails app into engines, this is the way to go. Here’s how you do it….

Within your Rails engine, there should be a file called engine.rb here’s an example of it for an engine I called EngineWithMigrations:


module EngineWithMigrations
  class Engine < ::Rails::Engine
    isolate_namespace EngineWithMigrations
  end
end

All you need to do is tell Rails to add your engine’s migration directory to its list of places it looks for migrations. Like so:


module EngineWithMigrations
  class Engine < ::Rails::Engine
    isolate_namespace EngineWithMigrations
    
    initializer :append_migrations do |app|
      unless app.root.to_s.match root.to_s
        app.config.paths["db/migrate"] += config.paths["db/migrate"].expanded
      end
    end
  end
end

app.config is the config of your wrapper Rails app, config is the config of your engine. The above line adds the engine's migration directory to the wrapper Rails app's migration directory list. The unless wrapping it is to keep your migrations from running twice in your testing dummy app (which already runs migrations fine). Now when you run rake db:migrate from your wrapper app, your engine's migrations just work!

Launching Focused Jasmine Specs From RubyMine

Charles Hansen

Wednesday, May 8, 2013

RubyMine is great for launching focused rspec tests, but is a little trickier for launching Jasmine specs, but we have had it working on my current project using a shell script and RubyMine external tools . The script relies on using sed to parse the first line of your spec file, so this actually only runs the describe block at the top of your file. The steps for our setup below.

Create a shell script somewhere in your project:

#!/bin/sh JASMINE_URL="http://localhost:8888/?spec=" if [ -z "$1" ]; then echo "Opening $JASMINE_URL to run all the tests" open "$JASMINE_URL" exit 0 fi # is this a spec file? echo $1 | grep 'spec.js$' if [ $? -eq 0 ]; then # ends with 'spec.js', probably a spec file SPECFILE=$1 echo "Detected $SPECFILE as matching spec for $1" else echo "Could not locate a spec file to go with $1" >&2 exit 1 fi DESCRIBE_LINE=$(grep describe $SPECFILE | head -n1) if [ -z "$DESCRIBE_LINE" ]; then echo "This does not appear to be a spec file." >&2 exit 1 fi # to escape the single quotes in sed, you do: '\'' - escaping your single quotes, and wrapping that in single quotes # see: http://muffinresearch.co.uk/archives/2007/01/30/bash-single-quotes-inside-of-single-quoted-strings/ SPEC=$(echo $DESCRIBE_LINE | sed 's/.*["'\'']$.*$["'\''][^"'\'']*$/\1/') echo "Opening $JASMINE_URL$SPEC" open "$JASMINE_URL$SPEC"

Go to RubyMine -> Preferences -> External Tools -> +
This should launch a new tool dialog. Fill in name, descriptions, etc
For program, you should link the script you just created.
For parameters, you should use “$FilePath$”
Bind this external tool under keymaps (look for the name you just gave it). We use command-option-control-f8

Stop leaking ActiveRecord throughout your application

Robbie Clutton

Monday, May 6, 2013

Extending ActiveRecord::Base leaks a powerful API throughout an application which can lead to tempting code which breaks good design. Take the classic blog example where you may want to retrieve the latest posts by a given author. You may have seen, or even written code that gets the dataset you need straight into the controller or view:

Post.where(author_id: author_id).limit(20).order("created_at DESC").each { ... }

For me this is a design violation ~~as well as breaking the “Law of Demeter”~~[Edit: Current Pivot Adam Berlin and former Pivot John Barker pointed out that chaining with the same object was not a Demeter violation]. The example above tells me structure of the schema that the calling class has no business knowing. It also makes testing using stubs ugly and encourages testing against the database directly. A test would have to chain three methods to stub a return value. It’s brittle, as in it’s susceptible to breaking due to changes outside of the class. For me it also fails from a narrative perspective in that it doesn’t succinctly reveal the intent of this part of the application.

If we were testing this and attempting to use stubs, we’d have to write something like the below. You can see how this is at best cumbersome, but also fragile.

where = stub(:where)
limit = stub(:limit)
order = stub(:order)

Post.stub(:where).with(author_id: author_id) { where }
where.stub(:limit).with(20) { limit }
limit.stub(:order).with("created_at DESC").and_yield(post1, post2, post3)

You may be forgiven for thinking you could chain the stubs like below, but the arguments are ignored ~~and this just serves to highlight the breaking of the ‘Law of Demeter’.~~

Post.stub_chain(:where, :limit, :order).and_yield(post1, post2, post3)

I’d much rather see that as a message to the Post class.

def self.latest_for_author id
  where(author_id: id).limit(20).order("created_at DESC")
end

Post.latest_for_author(1)

If there were variations of the limit and perhaps offset, they can be passed as option parameters of as an options hash:

def self.latest_for_author id, limit = 20, offset = 0
  where(author: id).limit(limit).offset(offset).order("created_at DESC")
end

Post.latest_for_author(1)
Post.latest_for_author(1, 20, 0)

def self.latest_for_author id, options
  limit = options[:limit] || 20
  offset = options[:offset] || 0
  where(author: id).limit(limit).offset(offset).order("created_at DESC")
end

Post.latest_for_author(1, offset: 20)

In order to get the dataset the call looks like the following, and I think is more informative than using the ActiveRecord DSL directly.

Post.latest_for_author(author_id).each { ... }

Testing is also easier, as it puts more emphasis on the messages being sent to objects rather than a chain of calls having to be correct.

Post.should_receive(:latest_for_author).with(1).and_yield(post1, post2, post3)

There are a few advantages to this refactor:

Only the Post class knows about the schema
Any changes to the implementation of what latest_for_author are encapsulated in one place
The method describes the intent more than the implementation
Stubbing in the tests are easier as there is one clear dependency
Testing the database is encouraged only in the class hitting the database

One further refactor could be done here, and that is to move the query logic out of the Post class once more, but this time into a purpose built query Object:

class LatestPosts
  attr_reader :author_id

  def initialize author_id
    @author_id = author_id
  end

  def find_each(&block)
    Post.where(author_id: author_id).limit(20).order("created_at DESC").find_each(&block)
  end

end

Where using the class looks like:

LatestPosts.new(author_id).find_each { ... }

Here’s what Bryan Helmkamp has to say on query objects in his excellent write up on fat ActiveRecord models. Bryan here rightfully points out that once in a single purpose object, they warrant little attention to unit testing. Now is the right time to use the database to ensure the right data set is being returned and that N+1 queries are not being performed. This means that database testing would only occur within the class actually hitting the database and not the rest of application which has a dependency on the database.

All of these techniques discussed serve to improve the design of an application by preventing leaking responsibilities from one class throughout the rest of the application. I’m also not saying that developers shouldn’t be using ActiveRecord or even Rails, but to use the tools responsibly.

Access your database’s best features with Sequel

Brandon Liu

Monday, May 6, 2013

Sequel is a wonderful library for interacting with relational databases. Some of my favorite aspects:
1. Out-of-the box support for foreign key constraints.
2. Straightforward migration DSL.
3. Support for all major free RDBMSes and even some proprietary ones. Also JRuby support through JDBC. (although JDBC doesn’t necessarily
give you all the features a native driver would. Projects like jruby-pg are making progress though.)
4. “One way to do it” for common operations such as inserts and updates. Parsimonious API (compared to ActiveRecord).
5. bin/sequel executable gives you an interactive ruby session directly from a database URL.
6. Extremely powerful Dataset abstraction lets you write general query code.
7. The source code itself is a pleasure to work with and the maintainer (jeremyevans) is extremely diligent about pull requests.

One feature i’d like to expand upon is the ability to call raw SQL functions from Sequel’s DSL.

Here’s a migration where I create a speeches table with some full-text search columns:


Sequel.migration do
  up do
    create_table :speeches do
      primary_key :id
      String :text
      String :speaker
      Integer :year
    end

    run "ALTER TABLE speeches ADD COLUMN ts_text tsvector;"
    run "CREATE INDEX ts_text_idx ON speeches USING gin(ts_text);"
    run "CREATE TRIGGER tsvectorupdate BEFORE INSERT OR UPDATE
         ON speeches FOR EACH ROW EXECUTE PROCEDURE
         tsvector_update_trigger(ts_text, 'pg_catalog.english', text, speaker);"
  end
end

And here’s some sample code to query it:


require 'sequel'

def search(dataset, term)
  results = dataset.select do
    [ts_headline('english',
                 :text,
                 to_tsquery('english', term),
                 'MaxFragments=2').as(headline),
     id,
     speaker]
  end
  results.filter("ts_text @@ to_tsquery('english', ?::text)", term)
end

DB = Sequel.connect('postgres://localhost/blogpost')

DB[:speeches].insert(:year => 1865, :speaker => 'President Lincoln', :text => 'Fourscore and seven years ago')
DB[:speeches].insert(:year => 1963, :speaker => 'President Kennedy', :text => 'I am a Jelly Donut')

p search(DB[:speeches], 'President').all # returns both speeches
p search(DB[:speeches].where("year > ?", 1900), 'President').all # returns only the Kennedy speech

The search function takes in a dataset and a query term, and returns a new dataset filtered with a body or speaker name matching the query term,
as well as a “headline”, or the context around the search term to display in search results. This takes full advantage of a full-text search index.

Sequel is also great as glue code for calling PostGIS functions. I’ve found it especially useful for DRY-ing up long sets of raw SQL queries into maintainable scripts.

Responsive web design with GroundworkCSS: Column stacking

David Varvel

Monday, May 6, 2013

Recently, I’ve started using GroundworkCSS to implement responsive design on a personal project. Simply put, it’s been an absolute pleasure to use. Not only is Groundwork highly flexible, but it also feels remarkably lightweight in comparison to something like Bootstrap.

There’s a few challenges, however. The Groundwork website has quite a few live demos, but there’s very little text to explain exactly what’s going on. There’s also not much of a community to glean knowledge from. While teaching myself how to use the framework, I’ve had to spend a lot of time in the Chrome web inspector reverse-engineering the live examples and then experimenting.

As an example, one of the more difficult pieces when initially working with Groundwork is figuring out how things will stack on top of each other when your layout shrinks. The default behavior is simple, and reasonably intuitive. Elements will be stacked top-to-bottom in the order that they’re declared in markup. Here’s an example:

<div class='row'>
  <div id="left_thing" class="three fifths"></div>
  <div id="right_thing" class="two fifths"></div>
</div>

When rendered in desktop mode, the ‘left_thing’ will show up on the left and take up three fifths of the viewport, while the ‘right_thing’ takes up the rest. When the viewport shrinks for mobile, then the left_thing gets stacked on top. Simple enough.

However, what if you want the right_thing to render to the right in desktop mode but stack on top of the left_thing on mobile devices? That’s when it starts to get tricky.

In order to get the “right_thing” to stack on top, it has to be declared first in the markup, like so:

<div class='row'>
  <div id="right_thing" class="two fifths"></div>
  <div id="left_thing" class="three fifths"></div>
</div>

However, this screws up the ordering in desktop mode! Fortunately, there’s a fix.

It turns out that Groundwork will allow you to shift around the relative ordering of your declared elements. So, if you want to swap the “right_thing’ and “left_thing” into their proper places for desktop browsers, we’ll need to add some more classes.

<div class='row'>
  <div id="right_thing" class="two fifths right-three></div>
  <div id="left_thing" class="three fifths left-two"></div>
</div>

Note that the “right_thing” needed to be nudged over for the number of columns that “left_thing” was taking up, and we had to do a similar dance for “left_thing”. It turns out that Groundwork behaves quite differently from the standard CSS box model, and shifting an element using the “right-xxx” or “left-xxx” directives does NOT cause anything else to automatically move. Much like CSS relative positioning, you’ll have to shift the other elements yourself to get the effect you want. Be careful, or you’ll get overlapping elements or breaks that you don’t want.

You can also omit columns or skip columns to create negative space. For example, here’s two equal sized columns with 20% margins on the sides.

<div class='row'>
  <div id="right_thing" class="one fourth skip-one"></div>
  <div id="left_thing" class="one fourth"></div>
</div>

Note that we didn’t account for the fourth (and rightmost) column in the markup. Since we declared “fourth” in one of the row elements, Groundwork will automagically create four columns, no matter how few we provide elements for.

There’s some other tricky bits, but figuring out the nuances of column stacking has been by far my biggest hurdle. After getting that sorted, it’s been pretty smooth sailing. Overall, I’m impressed by the power and simplicity of Groundwork, and would strongly recommend anyone who’s interested in a responsive grid system to give it a shot.

Running Programs in RubyMine

Jared Carroll

Sunday, May 5, 2013

A typical Rails development environment includes an editor, a terminal for running a web server, and a utility terminal for managing files, using version control, running tests, etc. During development, you’re constantly switching between your editor and these external terminals. RubyMine, an Integrated Development Environment, can eliminate this tedious back and forth workflow. In this post, we’ll learn how to run programs in RubyMine on OS X; allowing you to stay in RubyMine all day long.

Running Files

To run a file, open the file in the editor or select the file in the project tool window, then press control + shift + R. Rerun a file with control + R.

These commands are commonly used to run test files, but they could also be used to run a simple Ruby script.

Run Tool Window

All running programs are displayed in the Run tool window. Press command + 4 to open the Run tool window.

run tool window

Stop a running program with command + F2. Use command + shift + [ and command + shift + ] to navigate between multiple running programs.

Running Tests

In a test file, press control + shift + R outside of any individual test to run all the tests in the file. Press control + shift + R within an individual test to run just that test.

Run Dialog

Open the run dialog with control + alt/option + R. The run dialog lists recently run programs. This is useful for when you want to rerun a test you ran several tests ago.

run dialog

By default, this dialog also includes commands to run a development server, and your entire test suite.

Ruby/Rails Quick List

Use command + option + R to open the Ruby/Rails quick list. The Ruby/Rails quick list includes several useful commands such as, starting a Rails console, and starting an IRB session.

Ruby Rails quick list

You can run a file or a code selection in an existing IRB or Rails console with alt/option + shift + L. View IRB history with command + E.

IRB history

Running Rake Tasks

Run a Rake task with alt/option + R.

run Rake task

If a custom Rake task doesn’t appear in the list, reload Rake tasks from this dialog or the Ruby/Rails quick list (command + alt/option + R).

Running Rails Generators

Run a Rails generator with command + alt/option + G.

run Rails generator

If a custom Rails generator task doesn’t appear in the list, reload Rails generators from this dialog or the Ruby/Rails quick list (command + alt/option + R).

Take Advantage of Your IDE

IDEs increase your productivity by combining all of your development tools into one program. Frequent context switching to external tools not only slows you down, but also requires more in-depth knowledge of each tool. Try gradually replacing external tools with their IDE equivalents. Over time, your knowledge of shell command options and obscure Git commands will no longer seem very important to you.

Procrastination, considered.

Andrew Bruce

Sunday, May 5, 2013

Last week I blogged about a new project for aiding in the hunt for test pollution, Scrubber. This is a personal side project that I began recently. It’s in the very early stages, like many of my other pet projects. It’s something I’ve worked on entirely alone, though I’d also love to collaborate with others and/or pair on it.

I was going to blog this week about the latest improvements I’d made to the code, and how it was going to improve the end-user’s experience despite being a mere refactor and a minor user interface tweak. I got bored after writing the first few paragraphs, and instead got distracted by the code, refactoring and tweaking it more than was necessary. A couple of hours in, I’d totally gold-plated the code in a way that I might have deemed unacceptable whilst on client time. This made me a little angry with myself: why was I procrastinating from blogging, and was I a bad developer who’d lost the ability to work alone?

Something we’re encouraged to do at Pivotal Labs is reflect on our behavior. Thinking about my boredom and procrastination a little, I realized there was something more interesting going on. The process went a bit like this:

Start blogging about the new features. Paste the visual output of the program into the blog post. Realize that the output didn’t meet the requirement of improving the user experience.
Start to implement the missing parts of the feature. Spend hours enjoying the freedom to refactor, delete and re-implement with no time limit, far more than when pairing on client time.

When we pair, we have a safety net to catch us when we get distracted by neat language features or elegant implementation tricks. Our partner will often remind us that we’ve spent, say, two hours making no discernible feature changes and no improvement to maintainability of the code. When soloing, however, especially on pet projects, we are free to choose a goal for the activity. Sometimes we decide to perform code katas, but other times we don’t consciously choose a goal: the goal could become apparent during the activity itself.

So it seemed that I’d accidentally found validation in what I was doing, and a potentially more interesting blog post at the same time. I’d managed to get some programming exercise in instead of banging out a not-quite-earth-shattering new feature. Specifically, I:

Thought like a user until it became obvious that changes were still needed. Switched to developer mode by accident.
Allowed myself to try out several approaches to the Presenter pattern, applying them to a trivial example that would not need a presenter in ‘real world’ programming.
Used the Introduce Null Object refactor in a situation that didn’t call for it. Yet, it felt good and might even prove useful for the project later.
Practiced several coding techniques that might become useful in the work environment.
Temporarily inverted my work-based insecurities about performance and timeliness, providing stress relief as an unexpected benefit of my brain switching itself off from the task at hand.

Reflection is a useful technique for improving well-being. Your mileage may vary. If you’d have preferred to read about the changes made to Scrubber this week, you can read the commit history!

Sencha Touch BDD – Part 3 – Testing Views and Mocking Stores

Ken Mayer

Sunday, May 5, 2013

Sencha Touch BDD

tl;dr

A multi-part series of articles on how to test Sencha Touch applications. It uses Jasmine for unit testing and Siesta for integration testing.

Part 3 – Testing Views and Mocking Stores

In Part 1 I showed you how to set up your Sencha Touch development environment to use the Jasmine JavaScript test framework. In Part 2 I showed you to unit test Sencha model classes in Jasmine.

I don’t normally test views, but when I do

There’s an old MVC mantra: “Fat models, skinny controllers and stupid views.” We don’t want complex logic in our views; that makes them hard to maintain. We don’t want any business logic in our controllers, either. That’s why we have models. Views should be so simple that they don’t require tests. There is a gray area with Sencha, however, where we found testing to be useful in our design process. It has to do with how views interact with stores.

Stores are essentially collections of models. They also encapsulate the persistence layer logic, separate from the business logic of the model. DataViews are a special class of Views in Sencha Touch that will consume a collection to generate a list-type view via a template.

Here are two goals for testing views and stores:

Does the view consume the right fields from the store, without hitting the back-end for data?
Does the store organize the data from the back-end, i.e. create the interface that is used by the view? Again, without hitting the back-end for data.

Small steps and iterate

Let’s define a very simple view via tests, we’ll make it work using an in-line store, then we’ll refactor the store into its own class. Next, we’ll refactor the storage class to use a remote back-end.

Ext.require('SenchaBdd.view.MyView');

describe('SenchaBdd.view.MyView', function () {
  it("has a list of colors", function () {
    var view = Ext.create('SenchaBdd.view.MyView', {
      renderTo: 'jasmine_content',
      store:    {
        fields: ['color'],
        data:   [
          {color: 'red'},
          {color: 'green'},
          {color: 'blue'}
        ]
      }
    });

    expect(Ext.DomQuery.select('.favorite-color').map(function (el) {
      return el.textContent
    }).join(', ')).toEqual('red, green, blue');

  });
});

Sencha does not come bundled with jQuery, so if you were expecting a DOM query like, $(“.favorite-color”), you might be surprised by the expectation. Ext has its own flavor of querying the DOM, using Ext.DomQuery.select.

Try implementing the view on your own to make the test pass. It should look remarkably similar to this:

Ext.define('SenchaBdd.view.MyView', {
  extend: 'Ext.dataview.DataView',
  xtype:  'myview',
  config: {
    itemTpl: '<div class="favorite-color">{color}</div>'
  }
});

In your application, you probably won’t hardwire a store into the view. In fact, you will probably embed this little view inside a larger container, like so (this adds another tab to the sample app):

--- a/app/view/Main.js
+++ b/app/view/Main.js
@@ -10,6 +10,11 @@ Ext.define('SenchaBdd.view.Main', {
 
         items: [
+            {
+              title: 'Favorites',
+              iconCls: 'star',
+              xtype: 'myview',
+              store: 'mystore',
+              styleHtmlContent: true
+            },
             {
                 title: 'Welcome',
                 iconCls: 'home',

Let’s create a store so our view will show us something:

Ext.define('SenchaBdd.store.MyStore', {
  extend: 'Ext.data.Store',
  config:    {
    storeId: 'mystore',
    fields: ['color'],
    data:   [
      {color: 'red'},
      {color: 'green'},
      {color: 'blue'}
    ]
  }
});

In order to see this view, we’ll need to add it to our app.js file:

--- a/app.js
+++ b/app.js
@@ -31,7 +31,11 @@ Ext.application({
     ],
 
     views: [
-        'Main'
+        'Main', 'MyView'
     ],

+    stores: [
+            'MyStore'
+    ],
 
     icon: {

Now, let’s go back and refactor our test so it uses MyStore instead of one that was hard-wired.

--- a/spec/javascripts/view/MyViewSpec.js
+++ b/spec/javascripts/view/MyViewSpec.js
@@ -2,17 +2,16 @@ Ext.require('SenchaBdd.view.MyView');
 
 describe('SenchaBdd.view.MyView', function () {
   it("has a list of colors", function () {
+    var store = Ext.create('SenchaBdd.store.MyStore', {
+      data:     [
+        {color: 'red'},
+        {color: 'green'},
+        {color: 'blue'}
+      ]
+    });
     var view = Ext.create('SenchaBdd.view.MyView', {
       renderTo: 'jasmine_content',
-      store:    {
-        fields: ['color'],
-        data:   [
-          {color: 'red'},
-          {color: 'green'},
-          {color: 'blue'}
-        ]
-      }
-
+      store:    store
     });
     expect(Ext.DomQuery.select('.favorite-color').map(function (el) {
       return el.textContent

All of our tests should remain green, but we’ve removed the “fake” store from our test.

Let’s get dynamic

Static data stores are boring. The fun starts when you start talking to a back-end API. Let’s say that we have a server that responds to an end point of ‘/colors.json’ with a list of favorite colors. We can even “fake” it by placing a file in the appropriate place. Even so, we don’t want our tests to make network calls to the back-end. That’s not appropriate for unit testing. We’ll use Jasmine’s AJAX mocking helper, jasmine-ajax. At the time of this writing, the 2.0 branch had not been merged into the main line, and we need the 2.0 branch in order to work with Ext.

cd spec/javascripts/helpers
curl -O 'https://raw.github.com/pivotal/jasmine-ajax/2_0/lib/mock-ajax.js'
git add ./mock-ajax.js

And we’ll create our first store spec in spec/javascripts/store/MyStoreSpec.js:

describe('SenchaBdd.store.MyStore', function () {
  var store;
  beforeEach(function () {
    jasmine.Ajax.useMock();
    clearAjaxRequests();
    store = Ext.create('SenchaBdd.store.MyStore')
  });

  it('calls out to the proper url', function () {
    store.load();
    var request = mostRecentAjaxRequest();
    expect(request.url).toEqual('/colors.json');
  });
});

Notice that I call jasmine.Ajax.useMock() and clearAjaxRequests() in the set up block. This is because I want to wait until the very last moment to turn on ajax mocking. The Ext class loader might still be trying to load a class (via xhr), and the mocker will prevent that from happening. I also clear all previous requests (in case there were any left over, to prevent test polution).

When you run Jasmine, you’ll get a test failure, “TypeError: Cannot read property ‘url’ of null” because we haven’t set up the proxy in the store, yet. Let’s do that.

$ cat app/store/MyStore.js
Ext.define('SenchaBdd.store.MyStore', {
  extend: 'Ext.data.Store',
  config: {
    autoLoad: true,
    storeId:  'mystore',
    fields:   ['color'],
    proxy:    {
      type:       'ajax',
      url:        '/colors.json'
    }
  }
});

Now, when you run the test suite, you’ll get a different error! This is because the default settings for the ajax proxy enables caching, paging, etc. Things we don’t need, so we have to turn them off:

    proxy:   {
      type:       'ajax',
      url:        '/colors.json',
      noCache:    false,
      pageParam:  false,
      startParam: false,
      limitParam: false
    }

Now our api test is green! You can even test this in the application by dropping a file, ‘colors.json’ into the public directory.

Let’s add one more test to finish things off.

  it('populates the collection', function () {
    store.load();
    var mockedRequest = mostRecentAjaxRequest();

    mockedRequest.response({
      status:       200,
      responseText: [
        {color: 'red'},
        {color: 'green'},
        {color: 'blue'}
      ]
    });

    expect(store.getCount()).toEqual(3);
    expect(store.getAt(0).get('color')).toEqual('red');
    expect(store.getAt(1).get('color')).toEqual('green');
    expect(store.getAt(2).get('color')).toEqual('blue');
  });

Every mocked ajax request has a response() method that you can use to inject your own response, synchronously, to your tests. This confirms that MyStore properly parses and arranges the received data so that it can be presented by the view.

Validate your mocks

We now have 2 tests that depend on a back-end to respond in a specified way. How do we keep these tests from drifting out of sync? Since we’ve mocked the store in our view test, we might never know if the back-end API changes!

You can wrap Jasmine expectation in functions so that they are reusable. Then we can mix this matcher into our view test to confirm that the store we use there is the same.

Let’s add this function to our SpecHelper.js file:

function myStoreDataIsValid(store) {
  expect(store.getCount()).toEqual(3);
  expect(store.getAt(0).get('color')).toEqual('red');
  expect(store.getAt(1).get('color')).toEqual('green');
  expect(store.getAt(2).get('color')).toEqual('blue');
}

And we’ll replace our existing tests with a single line:

  myStoreDataIsValid(store);

We’ll also modify the MyViewSpec.js:

--- a/spec/javascripts/view/MyViewSpec.js
+++ b/spec/javascripts/view/MyViewSpec.js
@@ -9,6 +9,7 @@ describe('SenchaBdd.view.MyView', function () {
         {color: 'blue'}
       ]
     });
+    myStoreDataIsValid(store);
     var view = Ext.create('SenchaBdd.view.MyView', {
       renderTo: 'jasmine_content',
       store:    store

Similarly, we can refactor our colors array into var:

--- a/spec/javascripts/helpers/SpecHelper.js
+++ b/spec/javascripts/helpers/SpecHelper.js
@@ -16,6 +16,15 @@ afterEach(function () {
     domEl.setAttribute('style', 'display:none;');
 });
 
+var colorsJSON;
+beforeEach(function() {
+  colorsJSON = [
+    {color: 'red'},
+    {color: 'green'},
+    {color: 'blue'}
+  ];
+});

And then use colorsJSON in our tests.

--- a/spec/javascripts/store/MyStoreSpec.js
+++ b/spec/javascripts/store/MyStoreSpec.js
@@ -18,11 +18,7 @@ describe('SenchaBdd.store.MyStore', function () {
 
     request.response({
       status:       200,
-      responseText: [
-        {color: 'red'},
-        {color: 'green'},
-        {color: 'blue'}
-      ]
+      responseText: colorsJSON
     });
     myStoreDataIsValid(store);
   });

and

--- a/spec/javascripts/view/MyViewSpec.js
+++ b/spec/javascripts/view/MyViewSpec.js
@@ -3,12 +3,9 @@ Ext.require('SenchaBdd.view.MyView');
 describe('SenchaBdd.view.MyView', function () {
   it("has a list of colors", function () {
     var store = Ext.create('SenchaBdd.store.MyStore', {
-      data:     [
-        {color: 'red'},
-        {color: 'green'},
-        {color: 'blue'}
-      ]
+      data:     colorsJSON
     });
+    myStoreDataIsValid(store);
     var view = Ext.create('SenchaBdd.view.MyView', {
       renderTo: 'jasmine_content',
       store:    store

What we’ve managed, so far

The toy app is coming along. We’ve test driven a DataView that consumes a back-end API. We’ve added the mock-ajax library so we can unit tests our stores in isolation. We’ve even seen a few techniques for keeping our mocks from getting out of sync (although, if you’ve been paying attention, I’ve still left a gaping hole, that needs to be plugged).

Hash#fetch with confidence

Abhijit Hiremagalur

Sunday, May 5, 2013

Hash#fetch is stricter about key presence than Hash#[]

 {}[:foo]
 => nil

 {}.fetch(:foo)
 KeyError: key not found: :foo

If you forget to set an ENV variable, would you rather your application fail late or immediately?

 ENV['TWITTER_OAUTH_TOKEN']
 => nil

 ENV.fetch('TWITTER_OAUTH_TOKEN')
 KeyError: key not found

It’s especially interesting when a key with a truthy default is explicitly set to a falsy value.

options = {on: false}

@on = options[:on] || true 
=> true # yikes, ever fallen into this trap?

@on = options.fetch(:on, true) 
=> false