We've pooled some Pivot shared tech news feeds and made this feedburner feed:

http://feeds.feedburner.com/pivotal-news-network

The content is in the spirit of Blabs, so we hope readers here might find it to be useful. See what you think.

I have an RSS feed, and I'd like to make a little news page out of it with the ability to post comments. Having done a quick survey of what's available, I'm thinking of doing nothing. But I'd like to find out what others have to say about tools available.

General requirements:

• I'd like to take an RSS feed and make a nice news page out of it.

• No mangling or truncating of articles in the feed.

• It'd be great if I could tie comments in. That might mean, for instance, dropping in disqus.

• I'd rather not have to write, deploy, or maintain any code. But if there's some set of tools/apis I could easily tie together with code, I'd consider it (some Heroku-type setup, for instance).

Examples of sites I consider to be at least partially what I'm driving at:

• ycombinator Hacker News
• Tabbloid
• Any blog

If I had to write code, I could imagine using something like feedzirra, sticking it on a many-times-per-hour cron job, and writing out a page and dropping in disqus.

Thoughts?

No, seriously. Why wouldn't you use Erector? Cause I think it's a pretty awesome view framework, but for some reason it hasn't caught fire yet. So if you think writing actual Ruby to emit HTML, with a clean, nestable syntax with full support for Ruby features like inheritance, delegation, and yield is neat, but there's something holding you back, then please let us know what it is. At best we can fix it, and at worst, at least we'll know why.

Here are some reasons I think you might not use Erector:

You love angle brackets. If this is the case then I can't help you. I don't think anybody can.

You like typing every tag name twice. Since Erector elements are Ruby statements, every open tag gets automatically closed.

You like invalid HTML. Since Erector elements are Ruby statements, every open tag gets automatically closed. (See how that works?)

You always remember to call 'h'. Rails 3.0 is going to HTML-escape all output by default. Erector's been doing this the whole time. Cause, you know, why wouldn't you?

You like having to rewrite your code when you extract a partial, and then again when you extract a helper method. In ERB, templates, partials, and helpers all have slightly (and annoyingly) different syntax for things like referring to variables and calling other code. Erector is all Ruby, so you can use your favorite refactoring browser, or just cut and paste, to move your code around. Check out this excerpt from Jeff Dean's RailsConf talk to see this in action, or read the slides from the whole talk on SlideShare.

You hate encapsulation. You think that your views should have direct access to all the instance variables of your controller. Unless they're partials, in which case you shouldn't, even though you can, although the names might be different. Confused yet? So am I.

You like putting code for one component in three separate files. Erector's new "externals" feature allows you to put all the code -- HTML, CSS, and JavaScript -- inside a single Ruby class. The CSS and JavaScript then get output inside the HEAD, once per HTML page, while the HTML gets rendered in the BODY as usual, as many times as necessary. This follows the OO paradigm of cohesion, otherwise known as "put similar stuff together," which is the complement of loose coupling, which means, "keep different stuff apart."

Okay, so those were sarcastic reasons. Here are some more possible reasons why you wouldn't use Erector. I suspect that these next ones hit closer to the mark. But I believe that they're all specious, if not downright false.

Your site contains a whole lot of complex HTML and a few inserted Ruby variables. OK, this makes sense. Erector's not great for static sites. But I've never personally worked on a web application where the code inside the views didn't quickly get complex enough to require codey things like loops and functions. And if you're writing code, then why not do it in a programming language?

Your designers don't know Ruby. I've heard this complaint a lot, but I have yet to meet this mythical designer who's smart enough to understand modern HTML, CSS, JavaScript, ERB, and partials, but is not smart enough to learn that "div 'foo', :class=>'bar'" outputs "<div class='bar'>foo</div>". On the contrary, I've worked with several designers who, after a few tutorial pairing sessions, were comfortable checking code in and out and editing Erector view code at will. Like any junior coder, they need to stay away from the tough stuff, but they're pretty good at knowing what they don't know and asking for help when they need it. (Which they would also do if working inside ERB.)

View code needs to look as similar to HTML as possible. Well, I hear this, but have you looked at HAML? That language is hella popular, and it doesn't look anything like HTML. Its structure is similar, in the abstract, but so's Erector's, and at least in Erector the method for emitting a div is called, you know, "div". And it's a method. And I don't want to turn this into a war between HAML and Erector -- I think HAML is gorgeous -- but HAML suffers from the same design flaw as every templating technology: views are not objects, and markup isn't code. After a certain point of complexity, HAML's elegance breaks down and you'd be better off doing loops and functions in code.

You've already got a bunch of stuff in ERB and it'd take too long to convert it. Yes, legacy code is a pain, but we have a command-line tool that converts ERB (or HTML) to Erector to make it a bit smoother. And you don't have to convert your whole app to Erector at once. Erector views can interoperate with ERB or HAML in Rails and Sinatra.

You're stuck on an old version of Erector. Yes, legacy code is a pain, but we have an upgrade guide for getting to 0.6.0, and people on the mailing list ready to help.

Erector's too slow. Lies! Erector is faster than a greased rattlesnake going downhill. Check out these benchmarking results. Erector is about 2x as fast as ERB and 4x as fast as HAML about the same speed as ERB and HAML(*) under typical conditions. We make sure to use the same output stream to minimize string copy or realloc, and using Ruby objects means much lower parsing overhead.

(*) Update: the "2x/4x" figure was based on a benchmark program that didn't use template caching, which speeds things up for both ERB and Haml. With template caching, Erector and Haml are about the same speed; Haml is about 20% faster when rendering a page with no partials. See this ongoing thread on the Erector list.

There's no documentation. More lies! We have a whole bunch of documentation at http://erector.rubyforge.org, including a FAQ and a user guide.

You got burned by Markaby. Underneath the elegant facade of Markaby lay a confusing and often counter-intuitive engine. Its use of instance_eval and other tricks made simple things break in weird ways and made debugging a real chore. Erector was born out of those frustrations, and one of its main design goals is "no magic." Also, there was a long time where Markaby wasn't being maintained (although that's changed recently); we have a core group of developers committed to responding on the mailing list and github, and we run integration tests against the latest stable Rails release (and soon, against Edge) to catch incompatibilities early on.

Rails has all these great helpers and I want to keep using them. Okay, go right ahead! Erector's Rails integration allows you to call any helper, either directly through a proxy method, or indirectly through the helpers object. If you find a helper that doesn't work, let us know and we'll add it to the list of supported helpers. (We haven't done all of them yet because it's a pain in the neck to look at each function and figure out what its input and output semantics are. Does it return a string or emit directly onto the output stream? Does it take a block? An options hash? An html_options hash? Etc.) We're also slowly putting some Rails functionality into Erector, either in the base class or in custom widgets. If there's something you need, ask on the mailing list, or better yet, send us a patch.

Its name is a dirty word. I've heard this more from people who didn't grow up in the United States, where the Erector Set was a popular toy among the 6-to-12-year-old DIY set in the 70s and 80s. (Apparently it was called Meccano in the UK.) Erector is a normal word, used all the time in the news and in business names. And as the name of a view library it's evocative in a way that's relevant and interesting, in that it's a builder, and you build a view up out of parts.

But we have heard this complaint, and in sympathy, changed the name of the command-line tool (oh, sorry, guess I can't say "tool" either)-- uh, executable-- from "erect" to "erector" even though the former is a venerable English verb that's grammatically appropriate ("I asked him to erect the scaffolding."). If you introduce the library and your coworkers get all giggly then I think if you just say the name with a straight face and then roll your eyes and mock your bawdy buddies when they snicker then all will be well. After a few repetitions it won't sound odd at all.

You've never heard of it. Help spread the word! Post a review on your blog! Ask your favorite app framework whether they support it! Post code samples in Erector and when people say "What's that?" then point them at http://erector.rubyforge.org! Give a talk at a meetup! Write your congressman and ask if she supports the Erector Mandate Bill of 2009! Buy ad space on the moon!

So, in conclusion, and despite my somewhat snarky tone throughout, I am honestly and desperately curious to know why the world has not yet beat a path to Erector's door. Anybody got any more ideas?

sudo gem install remixr

We at Pivotal like that incantation. Thanks to the Squeegee crew for putting Remixr together.

# find stores within 50 miles of ZIP 76227 and products over three G's

stores = client.stores({:area => ['76227', 50]}).products({:salePrice => {'$gt' => 3000}}).fetch.stores

Beautiful.

• Remixr intro

• http://github.com/squeejee/remixr

• Best Buy Remix API site

• Remix has some exciting upgrades and additions coming soon, keep up with it all via the Remix API blog.

This fixes several bugs that people have complained about for quite a while. Please let me know if anything is broken.

GemInstaller 0.5.3 has been released!

GemInstaller

• by Chad Woolley
• http://geminstaller.rubyforge.org

CHANGES

• 0.5.3 / 2009-08-25
• Many long overdue bugfixes and patches, see http://tinyurl.com/geminstaller-0-5-3-release for details.
• Thanks to Greg Fitzgerald, Britt Crawford, John Trupiano, Gabriel Gironda, and Eric Hodel for patches and assistance.
• Issues with case statement under Ruby 1.9
• GemInstaller cannot distinguish between gems that have the ame name but capitalized differently.
• add ./ci as default location for config file
• Disable GemInstaller install in default rails preinitializer.rb, but fork if it is used
• autogem() fails when run for newly-installed gem
• Sometimes installing fails due to RubyGems cache not being cleared between multiple API calls

DESCRIPTION

Automated Gem installation, activation, and much more!

FEATURES

GemInstaller provides automated installation, loading and activation of RubyGems. It uses a simple YAML config file to:

• Automatically install the correct versions of all required gems wherever your app runs.
• Automatically ensure installed gems and versions are consistent across multiple applications, machines, platforms, and environments
• Automatically activate correct versions of gems on the ruby load path when your app runs ('require_gem'/'gem')
• Automatically reinstall missing dependency gems (built in to RubyGems > 1.0)
• Automatically detect correct platform to install for multi-platform gems (built in to RubyGems > 1.0)
• Print YAML for "rogue gems" which are not specified in the current config, to easily bootstrap your config file, or find gems that were manually installed without GemInstaller.
• Allow for common configs to be reused across projects or environments by supporting multiple config files, including common config file snippets, and defaults with overrides.
• Allow for dynamic selection of gems, versions, and platforms to be used based on environment vars or any other logic.
• Avoid the "works on demo, breaks on production" syndrome
• Find lost socks.

Quick Start

See http://geminstaller.rubyforge.org/documentation/index.html

INSTALL

• [sudo] gem install geminstaller

In a recent thread on the East Bay Ruby Meetup list, several people chimed in with Ruby IDE suggestions. I suggested RubyMine, which we use at Pivotal. Several people mentioned NetBeans and Aptana RadRails, so I decided to have a little contest.

Now, if I am going to work in an IDE and sacrifice the speed of a text editor, I want to see it work for me ('cuz RubyMine can chew through all your CPU and RAM and then some faster than you can say Moore's Law). That means understanding Ruby, and using that information to save me some significant thinking and work.

I don't mean code-generation macros or dumb context-aware keyword-completion, I mean something useful like knowing where my classes and methods are. In my book, that leaves out Emacs, Vi, and even TextMate, regardless of their other merits (sorry people, I like text editors too, but I'm making up this test - flame away, the comments section is below).

So, here's the smackdown scenario:

• Download the latest NetBeans, RadRails and RubyMine.
• Open a Ruby project, this example is one of mine (I wrote this three years ago to learn Ruby, so don't make fun of me for doing dumb stuff...)
• Test the ability of the IDE to navigate through Ruby language constructs. This should be easy, it is a command-line app using a Dependency Injection architecture, no metaprogramming curveballs!
  • Open the root Ruby class for the project (lib/geminstaller.rb)
  • Pick a variable (app in self.install)
  • Try to work back to the class declaration (GemInstaller::Application) using IDE navigation (Command-click in all the IDEs, although sometimes they ignore you)

Result? RubyMine succeeded, NetBeans and RadRails failed miserably. Here's what happened in each:

NetBeans

1. Click create_application class method reference in app = create_application. NetBeans takes me to the method declaration in same class.
2. Click the app method in registry.app.

FAIL! ANGRY BEEP! app is an attr_accessor on another class, NetBeans can't find it.

RadRails

1. Click create_application class method reference in app = create_application. RadRails takes me to the method declaration in same class.
2. Click app method in registry.app. RadRails takes me to the attr_accessor on the Registry class (without a prompt, and highlighting the symbol, which is even better than RubyMine).
3. Click (and F3) on the :app symbol argument to attr_accessor.

FAIL! ANGRY BEEP! RadRails can't figure out the symbol parameter to attr_accessor. It says "Current text selection does not resolve to a Ruby element".

RubyMine

1. Click create_application class method reference in app = create_application. RubyMine takes me to the method declaration in same class.
2. Click app method in registry.app. RubyMine pops up a dialog asking if I mean the attr_accessor on the Registry class, or the local variable I'm declaring. That's rather silly, I admit, but the point is it followed the reference to another class.
3. Click on the attr_accessor choice. RubyMine takes me to the attr_accessor line in Registry.
4. Click on the :app symbol argument to attr_accessor. RubyMine takes me to the point where the @app instance variable is initialized.
5. Click on the Application class name in the GemInstaller::Application.new constructor invocation. RubyMine pops up a dialog asking if I mean the Application class in my application, or two other Application classes that happen to be in my Ruby installation. This is also a silly question - it should have known the correct choice because of my namespacing, but it still found the class.
6. Click on the GemInstaller:Application choice. RubyMine takes me to the class declaration.

SUCCESS! RubyMine drilled all the way to the class declaration, even through an attr_accessor, albeit with a couple of stupid questions.

Summary

I personally think this is a Big Deal. In the past, I've mocked Ruby IDE functionality as a poor simulacrum of the vast power in Java IDEs. When using Eclipse in Java, I could perform epic refactorings, extracting superclasses and adding parameters to method signatures; refactoring scores of classes across multiple projects in a few mighty keystrokes. Yes, I'm fully aware that this is because Ruby is a dynamic language, but that doesn't make me miss a real refactoring IDE any less, and others have long lamented these shortcomings, as well.

So, for years, even though I'd always indulge my pairs if they wanted to use an IDE, I've done all my personal hacking with a fast, lightweight text editor and command line tools. To me, the benefits of a memory- and processor-sucking IDE with tons of unnecessary, unconfigurable, resource-eating tiny-ass-fonts and chrome did not justify giving up the speed and responsiveness of a great text editor.

However, RubyMine can now navigate code for me. I don't have to think and manually find that class, RubyMine knows where it is. Granted, that ain't no Extract Superclass, but it saves me a lot of thought and time, both of which are increasingly rare commodities for me.

To be fair, this is really just a problem related to parsing and interpreting attr_accessor declarations, and I expect that NetBeans and RadRails will pass this test as well in another release or two. That's all great news, because it means that Ruby IDEs are finally, slowly, coming of age. I think I'll still be waiting a long time for an automated Modify Method Signature refactoring, though...

Jeff is Chief Scientist at Cloudera, which helps enterprises with Hadoop implementations.

Hadoop consists of three separate modules, which are apparently in the process of being split into separate Apache projects:

• Hadoop Distributed File System (HDFS)
• MapReduce
• Common (aka Hadoop Core)

I'll just mention some of the interesting little tidbits from the presentation:

• Standard box spec is 1U 2x4core, 8gb ram, 4x1TB SATA 7200rpm.

HDFS:

• Stores 128mb blocks, replicates the block
• Good for large files written once and read many times
• Throughput scales nearly linearly

Some examples of Hadoop-based projects:

• Avro - cross-language data serialization
• HBase - like BigTable
• Hive - SQL interface, an interesting open-source data warehouse solution
• Zookeeper - coordination service for distributed applications

Hadoop @ Yahoo: 16 clusters, each cluster is 2.5PB and 1400 nodes

Cloudera maintains convenient, stable Hadoop packages - it's all open-source - so you don't have to go around figuring out what version of what subproject works best with others.

Testing: Hadoop has a standalone mode, which uses a single reducer in one JVM.

Jeff mentioned that they use Facebook's Scribe for distributed logging.

And last but not least, Cloudera has a GetSatisfaction page.

Quick report from Steve Sounders' workshop at Velocity 2009, current Googler, author of High Performance Websites.

Short version: he has a new brand new book out, and if you're interested in any of the following tips you should probably buy it: Even Faster Web Sites

Resources:

• cuzillion - model your page and see how various browsers load it using Firebug's Net tab or...

• httpwatch works in IE and Firefox

• pagespeed - A little like (YSlow)[http://developer.yahoo.com/yslow/] (Steve uses a combo of YSlow and Pagespeed day-to-day) but gives you a different set of perf information, notably what % of functions in your script are actually invoked in the header, vs afterwards.

• spriteme A tool that Steve developed and just released, which looks to be a major leap in css sprite-generation technology - i.e. it doesn't just do the (easy) part where all the images get combined together. You get css help, etc.

• smush.it Uses non-lossy image optimization methods to reduce the number of bytes your images take.

Some tips (I'm assuming these all get better/more elaborate treatments in his book):

• For over 95% of websites, the vast majority (80%) of the time spent in page load is spent on the front end (i.e. only 10-20% is spent transferring html).

• Scripts block other elements from downloading. So while js is downloading and executing, nothing else can be downloaded.

• Typically only 25% of js functions are called before body onLoad (pagespeed helps you see what % this is for you). So one thing to consider when optimizing is lazy-loading the other 75%.

• There are tricks you can use to pull down scripts in parallel, for instance by creating script tags through document.createElement and attaching to the dom. But there are other techniques, and pitfalls for many of them in different browsers. He goes through the strategy decision tree in the new book.

• Bad: stylesheet tag followed by an inline script. This stops all the parallel resource loading and forces the browser to only download the js, then continue.

• Using different domains for assets. A well-known trick. Steve adds that returns diminish around 2-4 domains. Also points out that the browser doesn't care about whether these are actually separate hosts, just that the actual names are different, so you could use a simple CNAME record to make this work with one server.

• Flush the document early. Particularly header sections (some common images + html). In addition to the raw speed benefit, Google user testing shows this is very positive for user perception - they get visual feedback earlier and have a perception that it's a "fast page".

• Note that FF 3.5+ contains an interesting new event: MozAfterPaint - a great way to see when the browser decides to repaint parts of the page. See John Resig's post on MozAfterPaint for more.

And don't miss stevesounders.com

by Russell Edens. He has a great take on why Erector is interesting, complete with code examples:

With erector [views] are first class plain old ruby objects. Why is this good? It gives you all the tools of inheritance and mixin's for your views. That is cool. Especially for an application with multiple views of the same underlying models. You can refactor your views into base classes that derive and render the same data in different ways. This is object oriented design for views. Nice.

I've seen object oriented view code in other languages and it leads to some very powerful re-use that all OO programmers can understand. The most ambitious of these attemps was by an HR company ...[that] created their own markup language that was object oriented. The nature of HR data is that it has very complicated rules regarding who can see what data and when. The OO design of the language allowed that to be abstracted to the base classes and a functional programmer simply focused on the problem at hand. They took it further, as all commercial enterprise applications do, and they allowed the customer to define new models and views. Those views were very easy to write with this advanced data access logic abstracted out. Their customers loved it. They wrote very advanced business applications on top of this abstraction.

Views as simple classes, methods, and objects in Ruby - perfect!

Erector Hello World:

class Hello < Erector::Widget
  def content
    html do
      head do
        title "Hello"
      end
      body do
        text "Hello, "
        b "world!"
      end
    end
  end
end

For more see the Erector user guide.

• There was a problem uploading files to s3 through Paperclip with # characters in the name (s3 doesn't like # characters). There's a fix on Paperclip trunk, but that hasn't been packaged into a gem. Perhaps the Paperclip people could be convinced to cut a release?

• One team is seeing files on s3 disappear occasionally. They're using v2 of the s3 api, where the s3 gem uses v1. The team has now turned on s3 logging (which is off by default) - which they recommend everyone turn on as a general good practice.