Bundler 0.8.1 is out. There have been some significant changes around how the vendor directory is organized, so you'll want to rm -rf vendor/gems and re-run bundle.

A team tried swapping in Sqlite to see if it made any difference in test suite runtime. It was actually slightly slower than MySql. In-memory Sqlite didn't help either.

A gotcha when using cap and bundler:

"If deploy.rb does a require 'auto_tagger' and the auto_tagger gem is in the app's bundle but not the system, running the system cap won't find the auto_tagger gem. Using bin/cap runs the bundled cap and thus has access to all the gems in the bundle."

• XSS #1: There's a huge cross-site scripting hole if you use the meta refresh tag...it has a "data" attribute into which you can insert arbitrary javascript.

• XSS #2: Cross-site scripting resources, from an internal mailing list:

  • "I've gained a new appreciation for the importance of carefully thinking through security and escaping in RoR there's more than just h()'ing all your user entered data."

  • XSS vulnerabilities - http://ha.ckers.org/xss.html.
    Very useful catalog of different XSS vectors. Includes some utilities to base64-, URL- and hex- encode attacks so you can test out your apps.

  • General OWASP wiki - http://www.owasp.org/index.php/Main_Page. Lots of useful data information here. OWASP is a nonprofit group charted to improve the security of webapps in general.

  • Security Guide for RoR - http://www.lulu.com/product/download/owasp-ruby-on-rails-security-guide/4489819 general guidelines/things to think about for securing RoR apps.

  • Loofah - http://github.com/flavorjones/loofah is supported by a fellow Pivot and provides fast and good sanitization built on Nokogiri, albeit slightly slower on short strings than brittle regular expressions. It's in production at several companies.

    "Loofah excels at HTML sanitization (XSS prevention). It includes some nice HTML sanitizers, which are based on HTML5lib’s whitelist, so it most likely won’t make your codes less secure."

• Happy New Year

We've pooled some Pivot shared tech news feeds and made this feedburner feed:

http://feeds.feedburner.com/pivotal-news-network

The content is in the spirit of Blabs, so we hope readers here might find it to be useful. See what you think.

I have an RSS feed, and I'd like to make a little news page out of it with the ability to post comments. Having done a quick survey of what's available, I'm thinking of doing nothing. But I'd like to find out what others have to say about tools available.

General requirements:

• I'd like to take an RSS feed and make a nice news page out of it.

• No mangling or truncating of articles in the feed.

• It'd be great if I could tie comments in. That might mean, for instance, dropping in disqus.

• I'd rather not have to write, deploy, or maintain any code. But if there's some set of tools/apis I could easily tie together with code, I'd consider it (some Heroku-type setup, for instance).

Examples of sites I consider to be at least partially what I'm driving at:

• ycombinator Hacker News
• Tabbloid
• Any blog

If I had to write code, I could imagine using something like feedzirra, sticking it on a many-times-per-hour cron job, and writing out a page and dropping in disqus.

Thoughts?

I just got the chance to listen to this interview with Arlo Belshee about Naked Planning ("Kanban Simplified"):

http://cdn2.libsyn.com/agiletoolkit/Agile2008_Arlo_Belshee.mp3

Highly recommended. My summary of Naked Planning:

• Teams work toward delivering Minimum Marketable Features - usually chunkier than User Stories but more relevant to customers. Think "administrator can set user permissions" vs what you might break that down to in terms of finer-grained stories.
• The dev team breaks MMFs into tasks as appropriate.
• Teams work on MMFs in a ratio of about one MMF to every two pairs.
• There is no estimation (it's usually wrong, there's no particular reason why the time before you start a story - when you know very little - ought to be when you assess cost). Instead a pure yesterday's weather approach is taken: you write down the date you start the MMF, and the date you finish it, and keep a rolling average days to complete an MMF - that becomes your "velocity" (no need for that word though).
• "Done" means everyone - customers, developers - determine that the MMF is done. That means development is finished, any follow-on discovery or bug fixing is done, support staff are trained up, and the feature is really ready to go out into the world and be used by users and create value.
• The MMF backlog is somewhere between 5 and 9, because beyond that prioritization (remember, we're talking about something chunkier than a User Story) is mostly useless. If it takes, say, 15 days to finish an MMF, on a two pair team with a 7 MMF backlog, you're out about 6 weeks - this is when any backlog typically starts to degrade a bit. At the outside you're planning for an entire quarter.
• One slot, that you try to keep empty, is reserved for fires. The customer can put in a fire card that the team immediately works on. You can only work on one fire at a time - others go into the backlog with the MMFs.

This is a sort of Lean/XP/Kanban fusion that Arlo created and refined on a project he heads up. It drops some of the weirder parts of XP-style planning and distills down to its more interesting ideas, and addresses many of the beefs I have with point estimates, velocity, and iterations in practice. At the very least it's food for thought.

And here's a video in which Arlo explains the planning board.

This thread on the Agile Systems Administration group is particularly good:

http://groups.google.com/group/agile-system-administration/browse_thread/thread/7c32b729aaa1079b

There's some great stuff in here about planning in a highly interrupt-driven environment, and I particularly like Allspaw's breakdown of ops work at Flickr (the "MumbleMumble" process). Anyone who's wondering what's generally involved in making webops go ought to have a look.

sudo gem install remixr

We at Pivotal like that incantation. Thanks to the Squeegee crew for putting Remixr together.

# find stores within 50 miles of ZIP 76227 and products over three G's

stores = client.stores({:area => ['76227', 50]}).products({:salePrice => {'$gt' => 3000}}).fetch.stores

Beautiful.

• Remixr intro

• http://github.com/squeejee/remixr

• Best Buy Remix API site

• Remix has some exciting upgrades and additions coming soon, keep up with it all via the Remix API blog.

Theo Schlossnagle: "I don't care what you use: puppet, chef, bcfg2, cfengine - choose one and automate your configuration." (Theo's presentation was a highlight, check out the slides here.)

Allspaw and Hammond from Flickr: "If there's only one thing you do, automate your infrastructure."

There were lots of Puppet users at the conference, and Luke Kaines gave a good talk on it.

Interested in a Chef intro? Watch the video of Adam and Ezra's talk:

From the Chef BoF:

• If you're just doing application deployment of a typical webapp then running chef-solo on deploy might be just fine for you (it is for the Remix team).
  • Ezra points out that he has a chef subproject on git that even strips out cap - chef-deploy (vs cap sudo'ing out to chef).
• Adam gave a good explanation of the Chef run model.
  • It's important to understand the basics of what's going on behind the scenes - otherwise you'll shell out right in the middle of your Chef scripts and be surprised by the run order.
  • In short, all your Chef Resource statements are evaluated in a first pass. Then they're executed.
  • This gives Chef the opportunity to evaluate what's to be done and optimize. Adam noted that included recipes are only executed once (analogous to ruby require's)
  • Adam also mentioned something kind of cool - if I have this right - Resource statements return the associated Resource object. And when you refer to the Resource the same way twice, you're getting back the same Resource instance. So your included Recipes (for example) each have the opportunity to decorate a Resource defined earlier - if, say, it ought to be configured differently based on a combination of what else is getting installed on the box. Nice.

My favorite talk at Velocity was by Paul Hammond and John Allspaw from Flickr, who are doing real lowercase-a agile:

UPDATE

Here's the video, highly recommended:

"If there's one thing you do, it should be automated infrastructure". This was a refrain through the conference - as Theo Schlossnagle put it, it doesn't matter if it's chef, puppet, bcfg2, cfengine - whatever works for you, just do it.

Some of their techniques:

• One-step build. They literally go to a web page and click a button and watch the build take the full site from soup to nuts.
• Deploys: Who. What. When. You want to make all the meta-details of a deploy easily visible to anyone. Deploy logs are readily accessible.
• Always ship trunk. In a webapp this is possible. It vastly simplifies - everyone knows where to look for what's going out, and what's live.
  • Flickr does their branching in the code (take note git people), and these become natural ops levers (i.e. uh-oh turn that feature off / turn it down).
• "Dark launches". Facebook does this too: launch the guts of a new feature with the UI turned off so you can see how the technology works in real production.
  • Which results in anticlimactic launches, the best kind
• They roll forward (not back) to turn features off that aren't working.
• "Gather shitloads of metrics". System metrics, app metrics, everything. John has a great writeup on their Ganglia setup in his book.
  • Developers watch the metrics just as obsessively as ops. Visualization is a powerful tool used day-to-day by the whole group.
  • Developers have a way of putting in their own metrics via a little framework.
• They're all on IRC. This kept coming up at the conference - teams are on some chat tool like IRC, Skype chat LINK, or Campfire LINK.
  • Important events are piped into IRC, so you'll be right in the middle of a conversation and an alert will pop in.
• Logs are piped into a search engine so they can find things in the log history, easily.

Culture, philosophy:

• There's an ongoing conversation between dev and ops. They're learning to solve the Flickr problem together. Each side's way of thinking informs the other (major conference theme as well)
• Failure will happen. Develop your ability to respond. Like ER doctors you practice on failures, that makes you better/competent at handling what comes along next.
• In addition to the ops people on call, there's always a developer who has a pager

More:

• John Allspaw's blog
• Web Ops Visualizations at Flickr