Abhijit HiremagalurAbhijit Hiremagalur
Standup 01/13/2009: daemons, encoding
edit Posted by Abhijit Hiremagalur on Tuesday January 13, 2009 at 09:39PM

Ask for Help

"Daemon best practices in Ruby?"

  • We haven't tried DaemonKit
  • SimpleDaemon is what we currently use, which we suspect of interfering with monit (had problems with multiple instances starting, and process not starting upon reboot).
  • A couple of people suggested looking at Daemonize
  • Always monitor daemons with sanity checks (e.g. memory usage); use Monit or God
  • Roll your own?

"cut doesn't handle strange characters in large (5GB) text file, are there other unix commands for text file manipulation that are utf-8 compliant?"

  • Try awk/sed maybe
  • Try using od/hexdump to figure out what the weird characters are

UPDATE 01/14/2009: Chad's corrections

Comments

  1. Kyle Maxwell Kyle Maxwell on January 14, 2009 at 11:45PM

    Cut should be locale sensitive. Sed and Awk certainly are. Have you tried anything like:

    LC_ALL=en_US.utf8 cut -n ....

    I'm in the office in row closest to the ping pong table. Find me if you want to ask questions.

    Cheers!

  2. Kyle Maxwell Kyle Maxwell on January 14, 2009 at 11:46PM

    In other news, what's wrong with the daemons gem? I've used it in the past, and http://blog.rapleaf.com/dev/?p=19 resolves my key complaint.

  3. Chad Woolley Chad Woolley on January 15, 2009 at 02:19AM

    Couple of corrections/comments:

    • We didn't try DaemonKit
    • SimpleDaemon is what we currently use, which we suspect of interfering with monit (had problems with multiple instances starting, and process not starting upon reboot).

    @Kyle: I had tried the Daemons gem in the past - I forgot about it. However, it does seem too complex - multiple layers of forked out of concern over security, which we don't care about in this case.

    Our current plan (at the suggestion of Brian Takita) is to write a simple loop in ruby that just drops a pid. He claims this will work fine with monit.

    -- Chad

  4. Abhijit Hiremagalur Abhijit Hiremagalur on January 15, 2009 at 03:36AM

    @Chad: Updated the post, sorry :)

  5. Joseph Palermo Joseph Palermo on January 15, 2009 at 06:04AM

    The problem with "cut" turned out to be we were IN utf8 mode. Our data provider apparently ran out of standard ASCII characters for a single byte field of theirs and started using values above 127. So all the unix tools were seeing these as multi-byte characters and then eating the next character or so.

    Switching the terminal to ASCII mode solved our problems (go figure)

  6. Strass Strass on January 15, 2009 at 10:45AM

    Depending on the complexity of the daemon, EventMachine could be a good choice as well.

  7. Tj Holowaychuk Tj Holowaychuk on January 15, 2009 at 12:19PM

    I cant remember what went wrong with daemonize but I remember I could not seem to get it working correctly

Add a Comment (MarkDown available)