Interesting
Selenium removes the If-Modified-Since header. This gets in your way if you are testing ETags.
Speaking of ETags - they are awesome. See Ryan Daigle's article for code.
Help
Just how slow is Mongrel when serving static images?
The answer is - pretty slow. The reason for the question was that Google's crawler tends to keep a single connection open and fire lots and lots of http requests over a period of several hours. With a standard Nginx/Mongrel setup, this would tie up a mongrel for this entire period.
A proposed solution to this problem is to use HAProxy between Nginx and Mongrel.
Wouldn't a standard nginx/mongrel setup have it so all static content is handled by Nginx only? Requests for /images/, /stylesheets/, /javascripts/*, etc would be handled by Nginx before ever hitting Mongrel.
remove
Dan,
You are correct, there were a few issues and they all got combined into this one standup. Google actually gets locked onto a single Nginx for the entire period (assuming you have Nginx on each host in front of the Mongrel's on that host). This was our problem.
One idea was to use HAProxy to solve this, but then there was a question of do we still need Nginx. It seems like it, since HAProxy can't serve the static files, and you don't really want Mongrel doing that (although it doesn't tie up a Mongrel since it doesn't need to get into the Rails mutex, everybody agrees it's still much slower).
remove