Want to get on the NoSQL bandwagon but don’t know where to start? Riak is one of the lesser known entrants into the NoSQL horserace. Learn about what Riak is and how you can design Riak schemas to store your data.
Monthly Archives: January 2011
Standup 1/11/2011: 10,000
Ask for Help
“Does Active Record have batched insert? I need to dump 10,000 rows into the database in process and I don’t want it to take 45 seconds.”
The upshot: use a raw SQL INSERT statement.
Interesting Things
For a day as interesting as today (1/11/11 anyone?) there were surprisingly few interesting things of note.
Standup 1/10/2011: Monday Mix
Ask for Help
A pivot asks: “Anyone know anything about data warehousing & ETL?”
A handful of pivots raise their hands.
Another pivot asks: “How about Radiant CMS?”
More pivots raise their hands.
Interesting things
Should a
before(:all)in Rspec raise an exception the return code coming back is 0. This breaks CI.Rspec supports
before(:suite)— this runs, just once, before all other befores.In MongoMapper if you have a key (field) with type Hash, and set a default value, that default value points to a reference instantiated at class loading time. Instances inherit a reference to the same object (instead of a copy) which means any changes to to an instance with the default value propagate to all future instances using default. The immediate work-around is to wrap the value in a lambda, but within hours of reporting the bug there was a fix on HEAD.
splunk users meetup here (at Pivotal Labs in SF) on Wednesday January 12th. More details on meetup.com.
A global Rspec
before(:each)defined inspec_helper.rbapparently does not have access to fixtures. One admittedly klunky workaround involvesconfig.includea Module, and then aclass_evalinside the included hook of the module. This has the unfortunate property that thebefore(:each)will run multiple times, once at every test level, instead of just once at the top level — though this can be fixed by looking at the length of ancestors in the included hook in the module and only doing thebefore(:each)if you are on the first level. Others argued that fixtures should be available in abefore(:each)and that the problem is a load-order issue.
ModCloth.com Seeks Ruby Engineers in San Francisco
At Pivotal Labs, one of the services we provide our clients is helping them interview and hire. Pivotal Labs and our clients place a strong emphasis on Agile development and its many aspects: Pair Programming, Test-Driven Development, rapid iterations, and frequent refactoring.
Modcloth.com is looking for Ruby on Rails developers to join their team in San Francisco. The full job posting follows.
ModCloth.com, rated the 2nd fasted growing retail company in the nation by Inc. Magazine, is a distributor of independent designer fashion and décor.
Our San Francisco office is looking for Software Engineers ideally proficient in Ruby and Rails. Our team utilizes agile practices, paired programming, and test-driven development to build new exciting features for our rapidly growing site that attracts over one million unique visitors every month! We strive for a healthy work/life balance, play ping-pong during breaks, and are always looking to learn something new.
Please visit our Career Page to apply and be sure to include a cover letter letting us know why you can’t wait to join the ModCloth Engineering team.
Are you an expert Java Developer without production Ruby experience? That’s okay! We’re interested in speaking with you too.
TRUECar Seeks Senior Ruby Developer in San Francisco
At Pivotal Labs, one of the services we provide our clients is helping them interview and hire. Pivotal Labs and our clients place a strong emphasis on Agile development and its many aspects: Pair Programming, Test-Driven Development, rapid iterations, and frequent refactoring.
TRUECar is looking for senior Ruby and Rails/Sinatra developers to join their team in San Francisco. The full job posting follows.
We are TRUECar, and we’re changing the way cars are sold.
Who Are We? TRUECar, Inc. is a revolutionary Internet company whose mission is to reinvent how cars are bought and sold. Driven by industry leaders, successful entrepreneurs and superior technologists, we are growing quickly, and we’re looking for energetic new talent.
Why TRUECar? We ranked 209 on the 2010 Inc. 500 list of Fastest Growing Companies in America. We use lean startup / agile development processes. We leverage our speed and ideas to be the David who dances with the automotive Goliaths. Lone wolves No: great teammates Yes.
Who Are We Looking For? Right now, we’re looking for a Senior Ruby (Rails/Sinatra) Developer to build the technology used to power our production websites, APIs, widgets, and internal tools. We’re lucky enough to have an amazingly great (and fun) team and are smart enough to know that only by keeping our team’s DNA at the top of the chart can we continue to grow into the game-changing company we feel destined to be.
Us:
- Highly energetic team committed to a great product and culture
- Fully funded company – built-in financing & exit strategy PLUS great upside
- Highly experienced industry and start up veterans
- Growing, viable revenue streams
You:
- Possess a burning internal goal of creating game-changing product
- Will leave your ego at the door and thrive in healthy debate
- Have passionate integrity and a healthy sense of work/life balance
- Can work productively together with and learn from other top-level players
- Are a world-class developer looking for the right team & opportunity to really make an impact
What we need from you:
- Expert-level knowledge of Ruby on Rails (2 years+), Sinatra is a plus
- Have delivered great technology quickly and understand how to ship in a fast-changing environment
- Well versed in Agile processes / TDD / Continuous Integration
- Good communicator and team member
- Smart and gets things done – you have a can-do attitude for anything from server setup and build maintenance through to QA and feature delivery
- Thorough experience & understanding of:
- Rails up to 3 / Ruby 1.9.2, MySQL
- RSpec 1.x / TDD
- Javascript / JQuery / AJAX / CSS
- Git / Capistrano
- Bonus points for these – else an interest and ability to learn quickly:
- Sinatra / Unicorn
- MongoDB / Redis / Thinking Sphinx
- Linux and/or OSX Development (we use RubyMine)
- Webrat / Selenium / ScrewUnit / Jasmine / Cucumber
- SASS
- Facebook Connect-JS API
- Chef
- Amazon AWS EC2/S3/RDS
- Data feed management and integration
- High-availability issues (deployment, monitoring)
- Anything mobile (iPhone / iPad, Android, etc)
We get that this skill set doesn’t come cheap. So of course we offer a highly competitive salary and a wealth of great benefits including company equity, 100% paid family medical, dental, vision, and a healthy 401k. Add that to an environment with world class leadership, explosive growth, passion for fun, and great work. We are constantly evolving our employee perks, like gym reimbursements, referral and patent bonuses and incentive programs.
Interested? Let’s connect! Please send your resume, cover letter, and links to your portfolio to Molly Sigworth, msigworth@zag.com. Principals only, no recruiters.
Standup 2011.01.07: Pivots Make Gems
Interesting Things
- Check out the new gem from Tim, benchmark-timed_each. This handy gem lets you find out how long it takes to process each item in a collection.
Embedding Mongoid documents and data migrations
When first starting out with mongodb, it’s easy to make the wrong decision on whether to embed a document or not. Even if you made the correct decision at that moment, changing requirements may force you into a migration. So how do you migrate existing data when transitioning from a standalone document to an embedded document? This is what I came up with.
Initial Data Structure
class User
include Mongoid::Document
field :name
references_many :sales
end
class Sale
include Mongoid::Document
field :price, :type => Integer
referenced_in :user
end
Now with Sale embedded in User
class User
include Mongoid::Document
field :name
embeds_many :sales
end
class Sale
include Mongoid::Document
field :price, :type => Integer
embedded_in :user, :inverse_of => :sales
end
Migrating Sales Data
class EmbedSalesInUsers < Mongoid::Migration
def self.up
# pull your existing data into memory
# consider batching for large data sets
# Note that you must call query methods on the object you are migrating
# for this method to work (i.e. you can not pull via User#sales)
sales_attributes = while_stand_alone_doc(Sale) do
Sale.all.map(&:attributes)
end
# now when you save your data, your fields will be embedded
sales_attributes.each do |attributes|
user = User.find(attributes[:user_id])
user.sales << Sale.new(:price => attributes[:price])
end
# remove all the documents from the original collection
while_stand_alone_doc(Sale) do
Sale.destroy_all
end
end
def self.while_stand_alone_doc(klass)
# by changing the Mongoid::Document.embedded you can temporarily
# modify which collection Mongoid looks to for your model's data store
begin
klass.embedded = false
yield
ensure
klass.embedded = true
end
end
end
There are a couple things to note here.
- The embedded flag in Mongoid::Document is not documented so it could easily change. This was working as of 2.0.0.beta.20
- When you create the new embedded document, make sure you pass only the attributes you care about. Passing all attributes will add things that you no longer need like user_id in this case. (For clarity, attributes you assign will be persisted, though you will only have setters and getters for the fields you explicitly define in your document.
- I am using mongoid_rails_migrations in this example
Testing Service Integrations with Bash and cURL
One of the most important parts of testing a system is “finding a seam”. Testing a whole system can be a foolhardy and unproductive first step. Find your seam, and begin spreading tests in each direction from it. External services almost always present us with an excellent and obvious seam. I was inspired to take testing my services more seriously after talking to pivots Jeff and Rachel about their recent work testing and developing Facebook integration for a client, and also after hearing from Paul Dix and reading his Rails Services book. Based on the words of Jeff, Rachel and Paul, I’ve drawn up the following strategy for testing services:
- Test at the point of integration downwards, that the live service returns what you expect for various inputs.
- Using the same tests to drive your development, write a fake-out of the service that passes the same tests.
- Then use the faked out service as you test upwards, that functionality in your app can utilize the features of the service.
- Once you’re ready to integrate, write a very high level smoke test that can pass against both the real service and the fake service.
After some analysis with our client and their resident WordPress expert, we determined we’d need to create users in WordPress when they sign up for rails, and update those users later when they change their information. We knew we’d prefer to use a simple HTTP REST API with a /users resource we could handle the C and U in CRUD. We’d also like to help the developer test drive his API, without having to learn much about Ruby. We chose to write some simple smoke tests in the most pervasive scripting language in world, Bash [1]. Luckily, Bash has an awesome standard distribution, usually including one of the best http wrappers ever written, cURL. So we began with the following script:
#!/usr/env bash
RESULT=`curl -ipost -s http://testsite.boonewebdev.com/api/users -pname=Evan+Farrar -pid=12`
if [[ $RESULT =~ 'evan-farrar' ]]
echo "Test Create: Success. Got: $result"
else
echo "Test Create: Failure. Got: $result"
endif
There it was in a few lines of Bash: a failing test. The WordPress developer used our suite of tests, ran them as he refactored, extended our script and corrected it where we had made incorrect assumptions about his service. As we worked with him, it was much easier to simply run the test to see what had changed or what was failing than to try to communicate the subtle differences by email. Of course, once I first saw the script I went a little overboard with Bash hackery; I’ll have to save my resulting Bash unit testing framework for another blab…
[1] This statement is wholly unfounded and without research. I didn’t even ask my friend Google if this was true.
Standup 1/6/2011: 'Parker Has a Posse' Edition
Ask for Help
“Rubymine seems to have lost my RSpec stack trace? Spec from the command line is fine.”
Check the run command & try reverting to the default options?
“What’s our favorite ‘copy to clipboard’ code these days?”
Zero Clipboard – jQuery version – is the one that lets you bind the Flash code to any HTML element. That’s the one you want.
“Can you use Jasmine to test a MongoDB map/reduce?”
Yes! It just uses JS objects, so you can unit test your code with Jasmine no problem. We recommend you add a couple of integration tests to make sure everything is connected properly.
Interesting Things
- The cmdparse gem has now been released under LGPL. This gem gets pulled in as a dependency for some other gems, but was GPL before. It should now be easier to deal with if you’re license-wary
- Having memory leaks in your JavaScript? Try Leak Helper! It’s a simple enumeration of objects off of
window, not catching objects inside closures, but it still helped a team find a set of circular references that were confounding the garbage collector.
Standup 1/4/2011: Consultant Cookies Edition
Ask for Help
“How do you handle jQuery, new HTML5 elements, and IE6/7?”
Other than “I don’t”, how do you deal with building large article elements and then attach them to the DOM? It’s icky since older IE’s don’t handle elements it doesn’t know about. Do you have any suggestions?
“Are you using AuthSTMP on an app hosted at EngineYard? Is it working?”
Something changed and our configuration is no longer working. Not sure why.
Interesting Things
- RubyGems 1.4 is out. There are tips on how to upgrade nicely with RVM. But you might want to wait if you’re using Ruby 1.9.
- Our friends at Irrational Design have launched Secret Goals and want you to share your good & horror stories about New Years’ Resolutions.