In this session, David Stevenson explores how to run untrusted code inside a ruby application using a sandbox. With this powerful technique, users can upload code that integrates as part of a larger application, making it ideal for custom business rules, dynamic games (think SecondLife), and science/math applications. Ruby’s english-like syntax and ease of creating DSLs makes it a good scripting candidate for non-technical people.
Sinatra has been getting a lot of attention lately as the next great (micro-)framework. In writing apps, diving in, and contributing the reasons for its existence have become more clear. Sinatra is not just a toy or a neat trick, its the best way to create simple and non-obtrusive web interfaces to sit on top of a new or existing Ruby codebase. I’ll walk through the whats, whys, and tools for getting started with Sinatra.
Over lunch with Jake, I’d wondered aloud “where are all the wide-eyed optimistic presentations?” and Jake starts by saying he’s sorry that this will not be one of those talks.
We’ve been programming in Rails for several years now, and now more than ever we’re left with the problem of how to deal with, maintain and correct projects which may be riddled with out-dated thinking, mistaken ideas and problematic implementations.
In other skilled enterprises there are core ideas which are repeated by for practice and for their general utility. In martial arts, these are called “katas,” in programming, we have “patterns.” Patterns are general, reusable solutions to common problems in software engineering, which are often arrived at through emergent design. Anti-patterns, likewise, emerge in the general work on a project, but their presence is harmful. They’re the mistakes we make again and again in our projects.
Jake then laid out a few patterns and anti-patterns for your consideration:
1. Know your APIs
Some of the most egregious anti-patterns come from simple ignorance about the APIs available, and the consequential re-implementation of basic Ruby or Rails functionality. The Rails and Ruby APIs are both well-documented and important to your work. There’s no excuse not to know them.
2. script/plugin and gem installitus
While we must install a plugin or gem to use it, often we move on without doing the important job of cleaning up after ourselves.
- if not self.value.nil?
People litter ‘self’ all over the place. The only place you need self is on assignment. And all this could be replaced with the simple line:
- @value = 600 if @value.nil?
@value ||= 600
- some_value == nil?
What does this even mean? ‘nil?’ is false unless inside NilClass
- IRB-driven Design
Occasionally we hack away in IRB and, once we get the results we’re looking for, drop the code into our app without a second thought. This can produce some very buggy, obtuse, far-from-minimal code. Once we discover a mechanism, better to revisit the problem and produce the code in a more stable context.
Confusing. Not to be used outside of a predicate method.
4. Planning is Hard
Will you remember what your boss tells you to do a year ago? Will you unknowingly violate these past decisions and understandings? To avoid doing so it’s important to maintain active specs against your app.
Jake recommends Cucumber + Webrat for carrying this out.
5. How Bad Is It?
There’s a gem called metric_fu, a gem full of analytics. One in particular is known as Reek, which sniffs out code smells.
Run reek against your app, and it will find long methods, for example, and return all of the methods that are longer than 5 lines. And you can make your tests fail on the existence of these methods, to keep yourself and your team in check.
Orphans are methods, models, views or controllers, or any other bit of functionality which is disconnected from the rest of your app. Like un-used plugins or gems, orphan code is noise which keeps your code from being easily maintainable.
We can look to the more modular, merb-style project organization as a way of better structuring our projects. Likewise the Fat-model, Thin-controller design pattern helps us to create easily-testable, well-organized apps.
- The valueless test
assert_equal 2, User.count
This test fixtures, and activerecord, and its very presence hinders us by interfering with our view of more meaningful tests.
- The slow test
Any slow test has a cost of both slowing the testing feedback loop and making it less likely that we’ll run the tests as often as we should.
- The coupled test
Coupled tests mean that a test depends on the side-effects of other tests, or on the effects of an outside service or application. Instead we can mock out the outside web service or operation
- Complex tests mean complex code
9. Make Decisions and Document Them
Discuss your code conventions and style with the team, settle on a set of best practices, and write them down. Store them in a style guide in the project root or document directory.
10. Continuous Integration
Jake recommends using Integrity Continuous Integration Server. Whether you use that or Cruise Control, or any other option, continuos integration insures your specs and tests are frequently and consistently run.
11. Build Artifacts
Generating and maintaining build artifacts, such as statistics reports from your reek run or performance tests can enable you to track your progress over time. In addition, you put minimal expectations on these
12. Reviews and Retrospectives
Taking time every few months to review your recent progress, and discuss problems and progress.
Q & A
Someone mentions a particularly egregious example of a test which deleted its own application code in the process of running.
Jake responds that in cases where tmp deletion and file generation, it may be appropriate to have a safe version of the test which runs locally, and a separate exhaustive test which runs in a sandbox staging environment. This allows you to exercise the more dangerous tests in a safe environment.
Yehuda mentions one way to do this is to mock in the safe version, and turn of that mock in the sandbox environment.
Jake then shows a 45-line login method and points out how difficult it is to immediately grok. Newer-code built in the REST-ful style is much simpler to work with.
By discussing these approaches with old-hands and newbies alike, we can all improve the quality of our code, spend less time dealing with bugs or code comprehension, and more time building out the next great Rails projects.
A Brief History
Ruby web development came of age with MVC and Rails. Later, people who didn’t need a full MVC invented Sinatra and other frameworkes. Which brings us to today, and …
Waves can do simple apps in just a few lines of code. And by using “foundations”, developers can build more advanced apps with MVC-like functionality. You can build your own foundation for whatever web framework you envision (there are several for MVC and REST).
Waves supports rack::cache and JRuby. It’s Actually In Production(tm)!
Web as Services
As more rich browser apps use AJAX and COMET, server-side APIs are becoming more important. This is where REST shines.
“HTTP isn’t MVC, but our frameworks think in MVC.”
REST and ROA
“REST” shouldn’t be applied to things that are “REST-influenced” (just ask Roy). Dan likes to use “Resource-Oriented” for these situations.
HTTP-based ROA uses the existing infrastructure, and has proven scalability. HTTP defines a protocol for a distributed hash table:
- put(key, value)
Q: “What about post?” A: “Post is for ‘everything else’.” Some things aren’t clearly RESTful, and post is the catch-all for other operations.
What’s in the hash? Resources, and keys are the URIs.
What’s the point? Platform-neutral distributed objects! RDF can be used to describe discoverable resources.
ROA in action:
rss/atom. It’s one link to a resource describing your blog. “Boom! Podcasts for free.” Dan describes this as the law of “unintended consequences,” in a good way.
Edge caching is another big win for HTTP-based ROA.
How Does Waves Help?
Waves makes it easier to write resourceful applications like this today. New foundations will make it even easier going forward.
You can check out Waves at http://rubywaves.com, and on their Google Group.
Eleanor McHugh, a physicist by training, will be talking about how to make *nix systems work naturally within the Ruby environment.
The Unix Way
Eleanor actually hates Unix, but recognizes that it’s a very effective operating system for getting things done. It’s DRY: build little things, build them well, don’t build them twice. There’s a natural marriage between agile Ruby and the Unix philosophy.
Unix provides basic services which make it a very useful OS to “muck about with”:
- virtual memory
- process management
- hierarchical file system
- user permissions
- interprocess communication
Ruby provides some “really lovely” utilities:
However, if you’re doing a lot of IO, you end up doing a lot of
select()s and keeping a lot of file descriptors open.
System Calls with Ruby DL
DL, which Ruby delivers out of the box, is a way to wrap a C library with a Ruby call. This is a nice way to access the underlying kernel system calls without relying on the Ruby IO implementations.
This is superior to Ruby’s
syscall, in that you can actually get results back from the function call.
mmap allows you to do much faster memory reads, rather than do slower file reads.
inotify allows you to build evented ruby (like EventMachine, but without EventMachine).
Using pipes allows you to build efficient IPC.
The drawback is that using DL means more verbose code, and more error prone code. (Pointer math FTL!) So, for things like sockets, use the Ruby API unless you specifically need kernel-level eventing.
The lack of real thread support in Ruby can be addressed by using multiple processes, held together with IPC (sockets, pipes, memory mapped files). This is the traditional “Unix way” for handling multiple processes.
Moving to 1.9
On Ruby 1.8, strings are sequences of bytes. On Ruby 1.9, strings are proper characters (not bytes!). Even if your app only speaks “American”, you still need to be aware of this to handle data properly. Plus, some of the new syntax in 1.9 is not backwards compatible with 1.8.
Recommended steps for upgrading from 1.8 to 1.9:
- make sure you have good test coverage!
- make sure your test are checking the output (some end-result validation)
- run on 1.9
- hammer on your code until the tests pass
- decide whether to continue to support 1.8
Prawn only officially supports 1.8.6 and 1.9.1 to make life easier, but if support more versions is necessary for your project, check out ZenTest’s multiruby features.
Greg recommends using conditional-execution blocks to make version-dependent code look nicer:
if RUBY_VERSION < "1.9" def ruby18 yield end else def ruby18 end end
Greg opines that moving to Ruby 1.9 is not a magic bullet, but has lots of advantages, so try it out!
Ruby 1.8.6 is a workhorse (insert image of beat-up pickup truck). Ruby 1.9 is a Lamborghini (we think). “What the hell is 1.8.7?”
Answer: 1.8.7′s patch set is largely 1.9 backports. It’s a platypus!
However, this doesn’t mean that code written for 1.9 will magically work on 1.8.7. Or that code written for 1.8.7 will work on 1.8.6.
What should authors be doing? Should we release for 1.8.6 or 1.8.7? Greg recommends releasing for 1.9, especially if you’re writing a Ruby book (wink wink).
FFI (Foreign Function Interface) is supported “all over the place”, and is an alternative to writing a C extension. FFI works across implementations (JRuby, Rubinius, and MRI).
On Windows, Greg proclaims that JRuby is the easiest way to wrap a C library. “WTF?”
Oversimplified Explanations of Ruby Variations
According to Greg. (Not all of the nuance may be captured here, since Greg was moving pretty quickly. Blame me, not him.)
- 1.8.6 is ubiquitous, and may be slowing adoption of other, better interpreters.
- YARV (1.9) is faster than Matz’s implementation and is the only complete m17n implementation of Ruby.
- Ruby Enterprise has a great installer!
- JRuby is great and new, but requires C extensions to be rewritten
- Rubinius is what created the RubySpec project and FFI, and is very innovative.
- MacRuby is, um, Ruby for Macs.
We have a winner in the Pivotal Tracker GoRuCo Haiku Contest.
Collin Miller submitted the winning entry:
Rivers of Action
Clearly Sculpt a Way Forward
My Path is Now Clear
We thank everyone for participating. There were a number of great entries, but this piece really stood out.
Collin, come on down tomorrow and say hi. I’ll be there all day enjoying the conference with some other folks from Pivotal Labs and demoing Pivotal Tracker for anyone who doesn’t know it already. Congratulations on your win!
One day only… We’re giving away one ticket to GoRuCo, the Gotham Ruby Conference happening here in New York tomorrow, Saturday, May 30th.
We’ll pick one winner, at our sole discretion, to be announced here. All submissions are licensed to Pivotal Labs to use however we see fit, and you retain the ability to use them as you see fit, under the Creative Commons commercial attribution share alike license. The winner will be granted one ticket to GoRuCo, to be used tomorrow.
Valid entries must include name, telephone, and email. Name will be used in attribution, but neither the telephone number or email will be made public.
So have at it! Give us your Tracker haikus. Don’t know Tracker yet? Check it out!
MacRuby is an implementation of the Ruby language that runs on the Objective-C runtime under OS X. MacRuby is based on Ruby 1.9 but contains substantial modifications including the merging of object models, using the Objective-C 2.0 generational garbage collector, moving core types atop their Objective-C counterparts and replacement of standard libraries to more optimally integrate with OS X. MacRuby also includes a new library, HotCocoa, a thin, idiomatic Ruby layer that sits above Cocoa and other frameworks. This talk introduces MacRuby and HotCocoa and demonstrates how to use them to quickly build OS X desktop applications with Ruby.
At Pivotal Labs, one of the services we provide is bootstrapping startups, including helping them interview and hire. We currently have clients looking for skilled engineers to build their development teams. This is an excellent opportunity to learn Extreme Programming by working side-by-side with Pivotal’s talented and experienced developers while at the same time getting in on the ground floor of a small and dynamic product team.
Here’s a short description of Honk and Mavenlink, two Pivotal Labs clients currently looking for developers. Their full job postings follow at the end of this post.
Mavenlink is a funded startup that is changing the way people find experts who can help them and is providing the necessary tools to get their work done online. We are founded on the principle that virtually everybody needs qualified professional services that are readily accessible, affordable, and there when they need them. We’ve been working with Pivotal Labs to get our product launched, so we’re serious about being agile and we’ve got the right engineering process in place. This is a unique opportunity to join the Mavenlink team and contribute significantly to the direction of the company. We’re looking for someone who is not only passionate about development, but also shares our vision for the tools and capabilities necessary for making remote work better than working in person for both the client and the maven.
Honk.com is a new online automotive website that will make car shopping fun and social. We will enable consumers to experience a new way to explore new cars. We have partnered with a top social website to deliver this new way of car shopping and are funded by one of the largest media companies in the world. Our small team is made up of an experienced group of humble, efficient, and hyper-passionate individuals who are veterans of the automotive industry and social media space. We are proud of our ego-less culture, one that promotes team thinking, not individual accolades. If you’re interested in helping prove that social media and car buying go hand in hand, social networks serve a bigger purpose than keeping up with one’s day, and a small team can outdo the work of an army – then we may have a seat waiting for you.
If you are interested or for more information please contact each company directly. This is an exclusive service provided to our clients, no external companies or recruiters please.
Full job postings follow.
Mavenlink is a funded startup that is changing the way people find experts who can help them and is providing the necessary tools to get their work done online. We are founded on the principle that virtually everybody needs qualified professional services that are readily accessible, affordable, and there when they need them.
We are looking for a Ruby on Rails developer to join our team. We’ve been working with Pivotal Labs to get our product launched, so we’re serious about being agile and we’ve got the right engineering process in place. This is a unique opportunity to join the Mavenlink team and contribute significantly to the direction of the company. We’re looking for someone who is not only passionate about development, but also shares our vision for the tools and capabilities necessary for making remote work better than working in person for both the client and the maven.
- Developing Mavenlink’s Ruby on Rails application
- Test Driven Development
- Pair Programming
- Aggressive Refactoring
- Using Pivotal Tracker to estimate and knock down stories
- Participating in stand-ups and retrospectives to improve
Mavenlink’s product, process, and culture
Additional Skills desired:
- Experience with rSpec, Webrat, and other testing frameworks
- Rails application deployment experience
Compensation and benefits:
- Competitive salary
- Equity stake
- Full medical and dental
Interested? Send your resume to firstname.lastname@example.org
Honk.com is a new online automotive website that will make car shopping fun and social. We will enable consumers to experience a new way to explore new cars, focusing on what other real people actually think, not product specifications or biased editorial. Our site will be 100% consumer driven with no journalists or former race car drivers telling you what minivan or sedan you should purchase. Instead, users will find real people like yourself sharing their opinions and experiences. We have partnered with a top social website to deliver this new way of car shopping and are funded by one of the largest media companies in the world. Thankfully, our partners allow (and encourage) us to remain financially independent, unpolitical, and fast-moving… a true start up.
Our small team is made up of an experienced group of humble, efficient, and hyper-passionate individuals who are veterans of the automotive industry and social media space. We are proud of our ego-less culture, one that promotes team thinking, not individual accolades. If you’re interested in helping prove that social media and car buying go hand in hand, social networks serve a bigger purpose than keeping up with one’s day, and a small team can outdo the work of an army – then we may have a seat waiting for you.
Honk is developing a platform of distributed applications and a destination website that will engage consumers’ existing social networks. To be clear, we are not building yet another community or social network. Many of our social applications will reside on our partners’ sites with the intent to drive users to honk.com for a richer experience, including unique content, interaction, and transaction-oriented tools. Our integration with our core partner is currently underway and our beta launch is scheduled for 5/27. We will continue to expand our product over the next twelve months. In addition to deep knowledge of Ruby on Rails and Agile / Test-Driven Development precepts, we hope you have a thorough understanding / are comfortable with:
- Amazon S3/SQS/EC2
- CSV and XML data feed integration
Previous experience working in online automotive or social media is desired, but definitely not required. Honk is currently co-located in San Francisco and Los Angeles. Honk headquarters is currently located in West Los Angeles, right on the border of Santa Monica. Our ideal candidates should reside in one of these two major metro areas, although we are open to “off site” developers who have the right skills and background.
Please send inquiries to Bruce Krysiak, CTO: email@example.com
- Congratulations to the latest Pivotal client to go live! KODA is a new breed of career-building site, using the power of rich media and social networking to make the job search less about checklists and resumes and more about personality, talents, and networking.
A laundry list of stuff I’ve come across / been pointed to lately. What are your favorites (i.e. comments, please)?
Book: Scalable Internet Architectures by Theo Schlossnagle
Book: Release It! by Michael Nygard
Book: The Art of Capacity Planning by John Allspaw
Conference: Velocity 2009 in SJ – it’s only a month away.
Presentation: Operational Efficiency Hacks by John Allspaw
John Allspaw’s blog, “Kitchen Soap – Thoughts on Capactiy Planning and Web Operations”…(excerpt from latest post: “I can’t tell you how ripped I get when people say things like this: ‘cloud computing means getting rid of ops’ “)
WebOps Visualizations Flickr Group
Apis / tools:
Chef. This is step one, or as Allspaw puts it “if there’s only one thing you do, automated configuration and
deployment management should be it”. We run chef-solo in our cap deploy. Don’t miss Cooking with Chef 101.
God for process management.
Xray for ruby process inspection.
Elif is Perl File::ReadBackwards ported to ruby.
Book: Think Unix