- Check out the new gem from Tim, benchmark-timed_each. This handy gem lets you find out how long it takes to process each item in a collection.
When first starting out with mongodb, it’s easy to make the wrong decision on whether to embed a document or not. Even if you made the correct decision at that moment, changing requirements may force you into a migration. So how do you migrate existing data when transitioning from a standalone document to an embedded document? This is what I came up with.
class User include Mongoid::Document field :name references_many :sales end class Sale include Mongoid::Document field :price, :type => Integer referenced_in :user end
class User include Mongoid::Document field :name embeds_many :sales end class Sale include Mongoid::Document field :price, :type => Integer embedded_in :user, :inverse_of => :sales end
class EmbedSalesInUsers < Mongoid::Migration def self.up # pull your existing data into memory # consider batching for large data sets # Note that you must call query methods on the object you are migrating # for this method to work (i.e. you can not pull via User#sales) sales_attributes = while_stand_alone_doc(Sale) do Sale.all.map(&:attributes) end # now when you save your data, your fields will be embedded sales_attributes.each do |attributes| user = User.find(attributes[:user_id]) user.sales << Sale.new(:price => attributes[:price]) end # remove all the documents from the original collection while_stand_alone_doc(Sale) do Sale.destroy_all end end def self.while_stand_alone_doc(klass) # by changing the Mongoid::Document.embedded you can temporarily # modify which collection Mongoid looks to for your model's data store begin klass.embedded = false yield ensure klass.embedded = true end end end
There are a couple things to note here.
One of the most important parts of testing a system is “finding a seam”. Testing a whole system can be a foolhardy and unproductive first step. Find your seam, and begin spreading tests in each direction from it. External services almost always present us with an excellent and obvious seam. I was inspired to take testing my services more seriously after talking to pivots Jeff and Rachel about their recent work testing and developing Facebook integration for a client, and also after hearing from Paul Dix and reading his Rails Services book. Based on the words of Jeff, Rachel and Paul, I’ve drawn up the following strategy for testing services:
After some analysis with our client and their resident WordPress expert, we determined we’d need to create users in WordPress when they sign up for rails, and update those users later when they change their information. We knew we’d prefer to use a simple HTTP REST API with a /users resource we could handle the C and U in CRUD. We’d also like to help the developer test drive his API, without having to learn much about Ruby. We chose to write some simple smoke tests in the most pervasive scripting language in world, Bash . Luckily, Bash has an awesome standard distribution, usually including one of the best http wrappers ever written, cURL. So we began with the following script:
#!/usr/env bash RESULT=`curl -ipost -s http://testsite.boonewebdev.com/api/users -pname=Evan+Farrar -pid=12` if [[ $RESULT =~ 'evan-farrar' ]] echo "Test Create: Success. Got: $result" else echo "Test Create: Failure. Got: $result" endif
There it was in a few lines of Bash: a failing test. The WordPress developer used our suite of tests, ran them as he refactored, extended our script and corrected it where we had made incorrect assumptions about his service. As we worked with him, it was much easier to simply run the test to see what had changed or what was failing than to try to communicate the subtle differences by email. Of course, once I first saw the script I went a little overboard with Bash hackery; I’ll have to save my resulting Bash unit testing framework for another blab…
 This statement is wholly unfounded and without research. I didn’t even ask my friend Google if this was true.
“Rubymine seems to have lost my RSpec stack trace? Spec from the command line is fine.”
Check the run command & try reverting to the default options?
“What’s our favorite ‘copy to clipboard’ code these days?”
“Can you use Jasmine to test a MongoDB map/reduce?”
Yes! It just uses JS objects, so you can unit test your code with Jasmine no problem. We recommend you add a couple of integration tests to make sure everything is connected properly.
window, not catching objects inside closures, but it still helped a team find a set of circular references that were confounding the garbage collector.
“How do you handle jQuery, new HTML5 elements, and IE6/7?”
Other than “I don’t”, how do you deal with building large
article elements and then attach them to the DOM? It’s icky since older IE’s don’t handle elements it doesn’t know about. Do you have any suggestions?
“Are you using AuthSTMP on an app hosted at EngineYard? Is it working?”
Something changed and our configuration is no longer working. Not sure why.
…but it’s definitely for me.
Will Sargent posted an excellent writeup of the issues he encountered pair programming. I’d like to respond to some of his points. His experience is his, and is inherently valid. I’d like to offer some thoughts, though, as a counterexample.
In general, I love pairing, and never want to give it up. I pair full time and that’s the way I like it. I end up soloing a couple of days each month, and I miss having a pair to help me keep the flow going.
I do have some issues with the full-time pairing lifestyle, chiefly that it’s harder to take breaks, both to refresh yourself and to attend to administrivia, such as email. It’s easy to just write code all day and let other responsibilities slip. But when I’m writing code, I wouldn’t do it any other way.
I found that in order to pair, I had to act as if I was in a continuous meeting. I had to not just listen to my pair, but appear to be listening; I had to nod in the right places, repeat back what my pair said in active listening fashion. I had to pick the right moment to interject. I tried to model my partner’s mental state in my head so I could see his viewpoint better.
This has not been my experience at all. There are moments when I feel the need to practice active listening, but those moments are not the same moments as when we’re writing code. They’re thinking moments, when we discuss what we’re doing.
They also don’t feel awkward or stressful to me. Partly, I think that this sort of interaction is something I’m good at and which appeals to me. Partly, though, I suspect it’s the type of people I work with. Pivotal only hires people who are great at making close personal connections with each other. That’s why we pair with people for a day during the interview process: we mostly want to see how quickly they can become comfortable with their pair. I feel at home working with every Pivot in the office (as far as I’ve experienced). It doesn’t feel like a meeting with a client, it feels like hanging out with a friend. But then, working with clients at Pivotal usually feels the same way.
Will says that pairing failed for him because he’s an introvert. I’m an introvert, and I love pairing. Introverts take comfort and restoration in being alone or with close friends, while extroverts are more energized by meeting new people and being in larger groups. If I were pairing with the same disconnected energy that Will encountered here, I’d collapse by the end of the day too. The way we fit together at Pivotal, though, I’m energized by pairing with my colleagues.
And if I was writing code, I found something very interesting: I don’t think in English when I write code. I think in code first, and I can translate written code to English.
I’ve found it extremely useful as a design tool to learn to think in English—that is, in the domain—rather than in code. This is what I love about Cucumber. You clarify what you mean in English, and then you translate it into code. In my experience, the design of the code and the user experience improves from developing that strong domain language.
The large part of pair programming was doing things in “The Simplest Possible Way That Could Work.” In practice, this meant doing it the way that it had been done previously, with the least possible amount of change.
When I got to the point where I understood some of the framework, I saw that there were some bugs inherent in the existing design which would make the system act unpredictably in production. In other situations, the design could improved by eliminating some classes and moving some other classes to have a single responsibility.
To me, this was “refactor mercilessly” and “once and only once.” But to my partner, this was a violation of “do the simplest thing that could possibly work.”
This is just a shame; no wonder Will hated his time doing this. First of all, to be clear, the principle of “Do The Simplest Thing That Could Possibly Work” is not part of pair programming, it’s another tenet of XP and other agile practices. It also doesn’t mean what Will’s pair told him it means.
As I’ve always practiced it, DTSTTCPW means two things, because “simplest” has two meanings. First, it means to write a test and then make it pass by expending the least effort possible; this is what Will is talking about here: write the special cases and use poor design to get it done quickly. Then it means to refactor your design to give it the least complexity possible. This cycle is known as “Red, Green, Refactor“.
DTSTTCPW is not a tool to improve your velocity of development. It is a tool to improve your quality of development.
It also went against the daily switching of partners. Whatever you start when you pair has to be finished, and checked in, by the end of the day. If you’re only looking at getting the story done by the end of the day, you’re not interested in hypothetical bugs or refactoring — you don’t have time for them and they’re not budgeted for.
This, to me, is a stupid rule. If it takes you two or three days to do the story right, take the time. It’s not worth sacrificing code quality to improve…actually I’m not sure what they’re trying to improve here.
Spikes only happen at the beginning of an iteration, and this iteration was budgeted at 8 weeks
Two more strange rules. At Pivotal, we work almost exclusively in 1 week iterations. I can’t imagine doing more that 2 weeks. And only spiking at the beginning of an iteration? I’ve never heard of that.
Furthermore, I can’t fathom the motivation behind the rule that you have to deliver your code by the end of the day if you’re not going to give it to the customer for another two months.
There is no ownership in pair programming. You’re working on a different piece every day, with a different partner. Everyone owns all the code equally. No-one really has responsibility for a single piece of it. Pair programming views this as a strength.
True, I view it as a strength myself. I enjoy not having single responsibility for any piece of code. I like knowing that any of my teammates can cover for me at any time, and that I can do the same in return. There is also joy for me in a Buddhist-like non-attachment to the code. I am not my code, and I don’t take changes to code I wrote (or co-wrote) personally. That means I don’t defend stupid choices for the sake of my ego.
There’s really one rule that comes out of all of this:
Hire great people who work well together.
That’s really the secret to any endeavor. Get a group of the best who work well in a particular way. That might be pairing, it might be daily code review, heck, it might be Waterfall. Polarize the team. Make sure you’re all pointing in the same direction—whatever that direction is. Everything else is dogma.
Pivotal Tracker has some updates for the new year: we’ve added a built-in integration with Bugzilla, a popular bug tracking tool, redesigned the My Projects page to make it easier to use, and made the Zendesk integration more flexible. More details on all of this below.
Bugzilla is great tool for easy bug reporting and organization, while Tracker enables focused team collaboration around a prioritized backlog of stories. That backlog may include stories for new features, as well as to fix existing bugs in your application, which may come from multiple places, including a bug reporting system. This integration allows you to use Tracker and Bugzilla to their strengths, side by side.
You can set up Bugzilla integration for your project in your project settings, based on the instructions here.
Once the integration is configured, you’ll see a new panel in your project, in the MORE menu:
This panel will allow you to see and import Bugzilla bugs into your Tracker backlog or icebox easily, via drag drop. As you collaborate around and complete stories linked to these bugs, Tracker will post notes about story comments and state changes to Bugzilla, allowing the entire development team to stay on the same page.
You can configure which bugs are shown in the Bugzilla import panel based on bug state, assigned product, and/or assigned component. You can also set up multiple “buckets” per projects, to allow you to see and import bugs for multiple product components separately, for example.
Notes: This integration supports Bugzilla versions 3.4.x to 3.6.x. Bugzilla 4.0RC has a number of changes in the API, and may not work with this integration (yet). Also, your Bugzilla instance has to be exposed on the web (outside of your company’s firewall) in order for the integration to work. Tracker must see the URL you specify in the integration settings.
We’ve re-organized the My Projects page to be easier to use. This is where you see all of the projects that you’re a member of, grouped by account. Hoving over the area with the project name, in the project list, will show links that allow you to visit various pages for that project, as well as to archive or leave the project.
It is also now easier to create pre-populated sample projects, which allow you to see what a typical software project looks like, and experiment with the various Tracker features. To create a sample project, click the Create Project button on the dashboard or on the My Projects page, and then click the ‘create a sample project’ link.
The Zendesk integration now allows you to specify the full URL of your Zendesk instance, rather than just the account name. This is useful if you’re using a vanity domain, for example “support.yourcompany.com”.
We’ve also fixed a number of cosmetic / CSS issues, including IE bugs. Please let us know if you’re still seeing any layout problems or other visual bugs that may have gotten introduced as part of the recent redesign.
Vintner Jared Brandt of A Donkey and Goat winery was a technical product manager before the first dot-com crash. He and his wife spent a year in France learning biodynamic wine production methods before returning to Berkeley to found their winery. Jared discusses their approach to winemaking in this evening talk at Pivotal Labs SF.
See all our talks at http://pivotallabs.com/talks
Vintner Jared Brandt was a technical product manager before the first dot-com crash. He and his wife spent a year in France learning biodynamic wine production methods before returning to Berkeley to found their winery.
If you’d like to display colorized output like this you can specify the appropriate custom reporter class using the CEDAR_REPORTER_CLASS environment variable. We do this in our Rakefiles, like so:
task :specs => :build_specs do ENV["DYLD_FRAMEWORK_PATH"] = BUILD_DIR ENV["CEDAR_REPORTER_CLASS"] = "CDRColorizedReporter" system_or_exit(File.join(BUILD_DIR, SPECS_TARGET_NAME)) end
You can set the environment variable in whatever way works for you. You can also set it to any reporter class you choose, so customize away.
And of course, Happy Holidays from Pivotal Labs!
“What tools can one use to track down JS memory leaks in IE 7/8?”
Is there a way to get MySql indexing to speed up queries involving greater and less than operators on date columns?
Postgres handles these operators a little bit better than MySql, but may not actually solve this problem.
Using millis instead of dates would give the DB the best chance of handling this scenario.
We are using Git’s subtree merge facility instead of submodules to stay synced to a different repository for part of our project. How do we push changes back to that repo?
See Tim Connor’s blog post “Git sub-tree merging back to the subtree for pushing to an upstream“. Early in that post is a pointer to an article describing the the subtree merge operation. Tim goes on to explain how to push your changes back through the chain.
Some cloud environments leave the names of temp files visible even when their contents are not accessible. Be sure to use obfuscated names for your temporary files!
The “Headless” gem allows you to easily set up an alternate “display” that allows programs to execute in a headless environment. See this blog post about how to use Headless to run Selenium tests on a CI box: http://www.aentos.com/blog/easy-setup-your-cucumber-scenarios-using-headless-gem-run-selenium-your-ci-server
Ccrb will bog down to painfully slow levels if more than a couple of CC Tray clients are pinging it repeatedly during a build.
Cron will not honor your .rvmrc file unless you do some work to set up the environment. If you set up your cron job like this:
0 6 * * * /bin/bash -l -c 'echo /home/someuser/.rvm/bin/rvm rvmrc trust ... && cd ...
the -l & -c parameters cause bash to load your environment as if your were logging in before running the specified commands. Someone also mentioned that rvm-shell can be used as a solution to this problem.