Help
“Anyone have experience dealing with failures in Amazon SQS?”
- Amazon Simple Queue Service is a reliable, highly scalable, hosted queue for storing messages as they travel between computers. You can place tasks in a queue and have worker systems extract and process them. However processing a queue item does not delete that item; you have to explicitly delete a queue item. The problem occurs when a persistent catastrophic failure occurs during processing of an item, e.g. a process core dump (say RMacick encounters an Image of Death) that does not produce an exception. Eventually SQS will timeout and mark the task as requiring processing again. Eventually all yor workers will try to process the queue item and die. There was a lot of discussion (e.g. communicate task status back to app server – too chatty, not scalable) but no obvious solutions arose that will work in the particular environment.
Interesting
SF.TUG: The first Tracker User Group will meet tomorrow. This one is somewhat exploratory, to see what the community wants out of it, so it is being kept purposefully small.
should_not is not the same as !=. Apparently, it is a feature of Ruby that you can override == but you cannot override !=. Rspec implements == but cannot implement !=. Sometimes you will get objects that will give different results when compared using should != as opposed to should_not. Use should_not.
Aviary.com is an online Photoshop-like too. They also have a vector editor. It is free, however they may reuse images you store on their site.
Pivotal Library:
We are getting a tool from Delicious Monster to automate our library. It will scan barcodes, categorize books and allow people to check books in and out.
> The problem occurs when a persistent catastrophic failure
> occurs during processing of an item, e.g. a process core dump
> (say RMacick encounters an Image of Death) that does not
> produce an exception.”
Ruby child processes return an exit signal. This signal can be used to sense a segfault. With this in mind, you could move the image processing to a child process, and then check the signal:
identify_output = `identify #{file_path}`
identify_exit_status = $?
if identify_exit_status.termsig == 11 # seg fault
raise “segmentation fault while examining file with identify”
end
The parent ruby process now knows about the seg fault and can take appropriate action on SQS.
April 29, 2009 at 11:48 pm