Ask for Help
A pivot asked “what is the current state of the art in scheduling recurring processes?”
The first-order answer was simply “cron”, but then the conversation got interesting.
Cron has a few downsides-
- Each task execution has to re-load the entire ruby/rails runtime, so, you pay a significant penalty in terms of startup time
- Crontabs often don’t get checked into source control, so there ends up being little visibility into which jobs are running when
One suggestion to solve the visibility problem was to use the whatever gem, which allows you to express your cron schedules in a ruby DSL that can easily be kept in source control.
A suggested alternative that eliminates cron altogether is resque-scheduler with resque.
The upsides with resque-based scheduling are that all your schedule logic is expressed in ruby, and you don’t pay the ruby/rails startup penalty for each worker.
The downside is that it adds additional operational infrastructure for you to manage (the resque workers and the redis server(s)).
We’ll be making some DNS changes that affect Pivotal Tracker this Saturday, May 29, at 10:00am PDT. We expect a fairly short outage, but for some users, Tracker may be unavailable for up to 6 hours.
Ask for Help
Identify and uploaded files
Running identify on uploaded files sometimes fails, because by the time identify gets hold of the file, the file extension has been lost (it has been written to
/tmp or the like).
The suggested workaround was to see if the filename specified in the
Content-Disposition header, if present, could be carried through to the temporary file.
Amazon has recently introduced a reduced-durabilty pricing tier for S3 storage, with significant discounts available to those who can live with 4 9′s versus 11 9′s of reliability. This seems like a win for things like thumbnails, which can easily be re-created from the originals on the off chance that they were actually lost by S3.
Rails 2.3.7 is out, but it doesn’t play nice with HAML just yet. Projects using HAML should avoid upgrading until those incompatibilities are ironed out.
- One of our clients had a large production issue due to a long-standing bug in Rails with case sensitivity.
Here’s the situation: Rails validates_uniqueness_of has a flag called :case_sensitive. This flag defaults to ‘true’, but can be flipped.
MySQL’s default collation is case-insensitive. As a result, queries will, in general, ignore case unless specifically overridden.
So one might imagine that setting :case_sensitive to false would be completely harmless in a standard MySQL application.
One would be wrong. Setting case_sensitive to false changes the query to lowercase the field in question, causing the MySQL database to ignore any indices it may have and turning the validates_uniqueness_of operation from something cheap and quick to something requiring a full table scan.
The open Lighthouse ticket on this issue is:
Ask for Help
“Any clever ways to catch out of bounds exceptions from Solr?”
This is a follow-up to yesterday’s Solr question. After some investigation, it looks like none of the major providers catch out of bound exceptions for very large numbers. Rather than instrumenting every Ruby call with validations to prevent these numbers from getting into Solr, are there any other brilliant ideas?
Goto File + line #: If you use ctrl-shift-N to go to a file, try typing in a line number after a colon, something like “my_file:30″. You’ll end up on that line.
Analyze stack trace: This tool lets you paste in an external stack trace, and gives you the ability to browse to all of the pieces of that stack trace.
At Pivotal Labs, one of the services we provide is helping clients interview and hire. Pivotal Labs and our clients place a strong emphasis on Agile development and its many aspects: test-driven development, rapid iterations, frequent refactoring, etc.
Here is a job posting from Revelation, a former Pivotal client looking for a lead Rails developer.
Rails Developer – Portland, OR
Revelation creates tools that help people understand other people. Our offerings are used for market research – helping companies better understand their customers to make better things. Our customers include some of the world’s leading research and consumer companies. Our culture is one that encourages the intellectually curious and intensely creative.
Are you entrepreneurial in spirit? Do you have the desire to not only improve product, but improve process and tools? Are you passionate about test-driven and agile development? And most important of all, are you ready to make the leap from skilled developer to leading a rails development team?
Come add your experience and thought-leadership to an incredible team. We are looking for someone with great code and people skills, who wants to take on a leading role in the development of our product and our practices. Reporting to the CTO, this position will be responsible not only for developing a great product, but also helping to create a solid foundation for an expanding team.
Candidates need to come to the table with the following:
- 3-5 years of experience building web applications
- Strong background with ruby and rails
- Experience with relational and non-relational data modeling
- Experience with virtualized environments (Xen/EC2)
- Experience with test-driven development and agile software practices
- Strong interpersonal communication skills
Even better if you have:
- An active github account with contributions to open-source projects
- Experience with Objective-C/iPhone and mobile development
We offer generous stock options, health insurance with vision coverage and a great work environment. If being a key part of the core team of a young, growing company is appealing to you, then let’s talk.
Please email resume and cover letter to firstname.lastname@example.org
Ask for Help
“Our site requires crafting URLs in a very particular SEO-friendly way. Rails doesn’t seem to give us a good solution for our URLs. Any suggestions?”
One of our clients needs to make their app accept and generate compound URLs that look something like the following:
where author, series, and book are all different domain concepts. Rails RESTful resources don’t really support this format. There wasn’t an immediate solution, but among the peanut gallery of ideas:
Hyphens are better than slashes in URL crafting, but Rails doesn’t separate on slashes at all
to_param solutions – Overriding to_param to something that starts with an integer ID generates URLs that look very slug-like, but can use standard Rails Domain.find mechanisms. For example, a book.to_param might be overridden to be “1-bookname”, which works for all purposes. The problem with this solution is that it doesn’t quite fit the requirements here, and doesn’t cover the compound needs.
Custom routes are always a possibility. You can hook up a special (non-resource) controller that understands flexible browse-y routes like the one above, parses them, and delegates to the more standard resource controllers. The problem here is that you have to figure out a decent delegate pattern and route generation pattern.
In general, URL crafting is a separate art from domain model crafting, and Rails doesn’t really cater to this. You will have to design URL-centric code to suit your URL crafting.
“Any ideas on ways to performance test IE7?”
No immediate ideas, but potentially more later.
“When users enter very large search parameters for numbers we get the following exception out of RSolr:”
RSolr::RequestError: Solr Response: For_input_string_11111111111111111111__javalangNumberFormatException
Is there an elegant solution to this aside from validating that all input parameters aren’t larger than max int?
When using named scope methods that refer to other named scope methods, you may discover that your SQL has some redundant condition clauses. This is a bug in Rails 2, and has been true for several versions. However, it’s a harmless bug – MySQL will understand the extraneous condition clauses just fine, without performance implications.
Mocking Paperclip for tests is a careful art. See our other blog post: Stubbing out Paperclip ImageMagick in Tests
Many people use the ultra popular Paperclip library to handle file attachments in Rails. Unfortunately the Paperclip documentation does not cover how to stub out calls to ImageMagick in your test suite. Without the proper stubs in place a test suite that uses Paperclip will take much, much longer to run.
In the grease your suite presentation by Nick Gauthier it has a slide titled Quickerclip that describes what needs to be done to spend up Paperclip in tests, basically you need to keep it from shelling out to ImageMagick. Alas, the presentation does include code for how to achieve Quickerclip.
As the presentation shows Paperclip.run is the method that needs to be changed. The first parameter passed to Paperclip.run is the ImageMagick command be executed. Paperclip uses the identify and convert commands. The
identify command is used to determine the dimensions of an image. The
convert command is the really heavy one that does image manipulation and thumbnail generation. Here is a redefinition of Paperclip.run with sensible behavior for tests.
def self.run cmd, params = "", expected_outcodes = 0
post_process in Paperclip::Attachment is an optional additional optimization. In Paperclip,
post_process eventually calls
Paperclip.run("convert") and by short-circuiting the method earlier in the chain we save a few cycles.
- Keep your OAuth Nonce values simple.
The Twitter Api, which is requiring all clients to move to OAuth for authentication by June 30, 2010, like all OAuth systems requires a nonce value for every call. This value is supposed to be random and unique for each request you make.
While there are many ways to generate a random ASCII value, our recent experience with Twitter’s OAuth system shows that a nonce value should not include a ‘%’ character – which would happen if your value has any non-URL-safe character. Twitter will return you a 401 error and tell you that your signature and token cannot be verified.
We’ve filed a bug with Twitter. But until then, keep your nonce value to ASCII letters & numbers and the calls will work just fine.
If you use auto_tagger in conjunction with a continuous integration build you may have noticed that the autotag executable exits with a status code of “1″ when it shouldn’t. Thanks to Sarah Mei I just pushed version 0.1.5, which fixes this bug. Thanks Sarah!
If you use active_model_listener, you are probably pretty tired of that active record deprecation notices. After Todd Persen pointed this out to me I just pushed version 0.2.5 which fixes this.
Pivotal Labs is hosting a happy hour the Friday before GoRuCo at the Blue Owl.
We’re looking for brilliant Rails developers to join our team. If you’re looking for somewhere to do serious agile development, and serious Rails development, this is the place to go.
196 2nd Ave, New York, NY 10003
5/21/2010, 7:00 pm – 9:00 pm
Version 1.6 has been submitted and should be in the App Catalog soon.
- Character counter didn’t work if “Post on Enter” was enabled
- Couldn’t post a photo tweet without text
Features and Changes
- geo tagging of tweets
- configurable notification sound
- (hopefully) improved German translation
- Copy Tweet has been added to popup menu
- preference to enable/disable metatap copy for tweets
- Email / SMS / IM tweets
- preference to enable geo tagging
- tweets will include location/geo tag metadata if Tweed preference is enabled and Twitter setting for adding location is enabled
- Tweets with geo tags will display link to location (in popup menu and in tweet)
Our poll results for changing the popup menu resulted in 46% for A, 54% for B. Only about 100 people responded. So while B won by a small majority, we’ve implemented option A, as that was the smallest change to Tweed and we felt the least disruptive to the larger Tweed userbase.
We may change this based on feedback.