Open Source Roundup: Rails, Rubygems.org, and LicenseFinder

Friday, September 7, 2012

Having just come off a project, Ian Lesperance and I spent the week working on some open source initiatives. We also spent the week showing new pivot David Edwards the ropes.

We started the week with a feature request to rails for customized STI (Single Table Inheritance) type serialization. On the project we just finished, we needed to model a legacy database that serialized integers instead of class names for the STI type, and unfortunately we found that Rails had no real API for overriding the type serialization, so we added the following:

class Foo < ActiveRecord::Base
  self.inheritance_serializer = ->(klass) do
    # Map the class to the appropriate type identifier.
    # Defaults to `klass.name`.
  end

  self.inheritance_deserializer = ->(type_before_cast) do
    # Map the type identifier back into the appropriate class.
    # Defaults (approximately) to `type_before_cast.constantize`.
  end
end

Ian also contributed a bug fix to ActiveRecord, fixing the pluck method when the column name plucked is a reserved word.

We then continued with a pull request to rubygems.org for adding license information to gem version pages and version api requests. In our line of work, we have to keep a vigilant eye on the software licenses of our dependencies; adding more exposure of licenses can’t hurt.

This segued nicely into some much needed work on LicenseFinder, an open source project Pivotal Labs created and maintains to automate the process of keeping track of your project’s license dependencies. We’ve added detection for two more license types (”LGPL” and “ISC”), added proper integration tests to the project, put the project on Travis-CI, and refactored some of the code along the way. In the future, we’d like to create a proper command line interface for license_finder so that you can use it without rake.

Standup 5/27/2011 Hack For Change, Guiderails goes public, Jenkins & Jasmine jems

Phil Goodwin

Friday, May 27, 2011

Helps

*Is there a way to change the URL that CCRB pulls from when it builds?

“Use Jenkins” (we will be standardizing on Jenkins in the near future anyway)

Apparently the answer has been found successfully in the past by grepping through the Ruby portion of the CCRB source.

Interesting

Hack For Change, sponsored by Change.org is inviting engineers and designers to spend 24 hours to build a web or mobile app that can help advance positive change. Top-rated hacks will be awarded a total of $10,000 to ensure their continued success and will gain recognition through widespread media coverage and promotion. http://hackforchange.com
Guiderails: Pivotal’s Rails 3 Templates, has been made publicly available on GitHub. https://github.com/pivotal/guiderails
While there is not consensus on how hash tags in URLs that are being redirected should be handled, Safari stands apart from most other modern browsers by throwing them away entirely.
When configuring a new project for Jenkins, remember to specify the branch to build, otherwise Jenkins will try to pull and build all branches from the repo.
Jasmine has a bug in its “runs and waits for” construct that causes it to ignore changes to the defaults for the timer and message on the “waits for” block.

Public projects directory and feed coming to Tracker

Dan Podsedly

Monday, February 28, 2011

We’re launching a new public projects directory in Pivotal Tracker this Friday. If you own or use public projects in Tracker, continue reading, as these projects are about to get much easier to discover.

This new directory, linked to from the Pivotal Tracker front page, will show a featured public project at the top, a list of the most active projects, and a live feed of all public project activity. The directory will also allow you to search for public projects by name, project description, or project member name.

What are public projects?

Public projects are just like other projects in Tracker, but they can be viewed by anyone. If you’re using Tracker to manage and collaborate around an open source project, for example, a public project gives you a great way to increase visibility into your efforts, and give the world a live view of progress and priorities. Best of all, public projects are completely free, with an unlimited number of collaborators!

How can I tell if my project is public?

When you’re in a project that’s public, you’ll see (Public) at the end of the project name. You can also see which of your projects are public on the View All Projects page – there is an indicator next to the name of each project that is public.

How do I change my project to be public or private?

By default, all projects in Pivotal Tracker are private, and visible only to the people that you explicitly invite as members. To allow anyone to view your project, even people who are not signed in to Tracker, you can make your project public. To do that, or to make a public project private again, visit the Settings page for your project, and change the Public Access option.

There are many active public projects in Pivotal Tracker already! This new directory will allow you to explore them, and see what everyone else in the open source community is up to. We’ll be featuring a few projects at a time at the top of the page in this new directory, let us know if you’d like us to feature your project!

Automating Bundler In Your Deploy

Chad Woolley

Thursday, March 25, 2010

If you are using Bundler to lock and package your gem dependencies in your app (which you should), here’s some tips on making everything automatic in your Capistrano deploy.

Refer to the Bundler Documentation for instructions on how to use Bundler to properly package your gems and check everything in.

Once this is done, however, you still must ensure that two things are done on every machine to which you will deploy:

Bundler is installed
You run ‘bundle install’ on every deploy to install the packaged gems on the local machine (and compile any gems with native dependencies)

Here’s the Capistrano magic to accomplish these two tasks automatically on every deploy:

before "deploy:bundle_install", "deploy:install_bundler"
after "deploy:update_code", "deploy:bundle_install"

namespace :deploy do
  desc "installs Bundler if it is not already installed"
  task :install_bundler, :roles => :app do
    sudo "sh -c 'if [ -z `which bundle` ]; then echo Installing Bundler; sudo gem install bundler; fi'"
  end

  desc "run 'bundle install' to install Bundler's packaged gems for the current deploy"
  task :bundle_install, :roles => :app do
    run "cd #{release_path} && bundle install"
  end
end

Oh, and for you GemInstaller users out there – here’s an easy way to generate a Bundler Gemfile from your geminstaller.yml config:

geminstaller --bundler-export > Gemfile

You’ll probably still need some tweaks, but this will get you started. Just make sure you upgrade to GemInstaller 0.5.5 first (0.5.4 forgot to put the ‘source’ line in the Gemfile).

Happy Bundling!
– Chad

P.S. There is a similar article here, which includes tasks to symlink your .bundle dir into the Capistrano shared directory, but my deploy was pretty fast anyway, so I didn’t worry about it. YMMV.

[ANN] GemInstaller 0.5.3 Released

Chad Woolley

Tuesday, August 25, 2009

This fixes several bugs that people have complained about for quite a while. Please let me know if anything is broken.

GemInstaller 0.5.3 has been released!

GemInstaller

CHANGES

0.5.3 / 2009-08-25
Many long overdue bugfixes and patches, see
http://tinyurl.com/geminstaller-0-5-3-release for details.
Thanks to Greg Fitzgerald, Britt Crawford, John Trupiano, Gabriel
Gironda, and Eric Hodel for patches and assistance.
Issues with case statement under Ruby 1.9
GemInstaller cannot distinguish between gems that have the ame name
but capitalized differently.
add ./ci as default location for config file
Disable GemInstaller install in default rails preinitializer.rb, but
fork if it is used
autogem() fails when run for newly-installed gem
Sometimes installing fails due to RubyGems cache not being cleared
between multiple API calls

DESCRIPTION

Automated Gem installation, activation, and much more!

FEATURES

GemInstaller provides automated installation, loading and activation
of RubyGems. It uses a simple YAML config file to:

Automatically install the correct versions of all required gems
wherever your app runs.
Automatically ensure installed gems and versions are consistent
across multiple applications, machines, platforms, and environments
Automatically activate correct versions of gems on the ruby load
path when your app runs (’require_gem’/'gem’)
Automatically reinstall missing dependency gems (built in to RubyGems > 1.0)
Automatically detect correct platform to install for multi-platform
gems (built in to RubyGems > 1.0)
Print YAML for “rogue gems” which are not specified in the current
config, to easily bootstrap your config file, or find gems that were
manually installed without GemInstaller.
Allow for common configs to be reused across projects or
environments by supporting multiple config files, including common
config file snippets, and defaults with overrides.
Allow for dynamic selection of gems, versions, and platforms to be
used based on environment vars or any other logic.
Avoid the “works on demo, breaks on production” syndrome
Find lost socks.

Quick Start

See http://geminstaller.rubyforge.org/documentation/index.html

INSTALL

[sudo] gem install geminstaller

GoGaRuCo '09 – TrustTheVote: Open Source Digital Voting – Gregory Miller

Chad Woolley

Friday, April 17, 2009

Links

Open Source Digital Voting

Trust The Vote

What OSDV means to me personally

I personally am really excited about this talk. I worked on the OSDV prototype at Pivotal last year, when we made a small prototype in a few weeks. This was subsequently presented to congress. It was an incredible experience. As a programmer, you write a lot of code which isn’t that exciting, counting beans or Yet Another Social Networking Website.

OSDV, however, is something that is REALLY important. It has the potential to revolutionize the way Democracy works, and really change the world for the better.

Here goes the talk, with Matthew Douglass running the slides and Gregory Miller talking.

Intro Video

First is a video about how democracy used to work, when we trusted the outcome of votes. Now, after the 2000 Presedential Election, people lost confidence.

Now, states are getting funding to update their voting system. However, now that we are past the “Hanging Chad”, we are seeing MORE, not fewer problems. The companies that make proprietary digital voting do not make the required investment to make their machines trustworthy, and rely on PC technology and proprietary code.

Shouldn’t we be able to say “I count”? We should not expect the Government or Private Sector to fix this. It must be a Grass-roots movement, something big. We need to completely rethink the lifecycle of our ballots.

We have to shift away from companies guarding proprietary, black box voting to a world of “glass-box” voting. Blueprints and designs are freely available.

We need the Open Source Digital Voting Foundation.

it is not just another thinktank or group of lobbyists. It is technology professionals teaming up with volunteers. Everyone can see, touch, and try it out.

This is a digital public works project, calling people from all over the country and world to help out, take a hands-on approach, and do something.

We are the real stakeholders in our Democracy. We can all make our votes count. The time to begin is NOW.

Pop Quiz

Q: Federal guidleines for how votes are counted?
A: FALSE

Q: California’s absentee ballots always counted?
A: FALSE

Q: Major voting vendors system rely on commodity Hardware/Software
A: Sort Of. They use “Windows 95″.

He then shows “Clippy” helpfully offering to finish your vote for you…

A “Free Markets” Failure

No Competition
High Barriers to Entry
No Incentive to Innovate

Horribly dysfunctional market. There are FOUR vendors of voting systems in the US, there may be two by the end of year

Very high barriers to entry, hard to get it approved and legal.

When you have no competition and barriers to entry, there is no incentive to innovate. You end up with closed proprietary systems with inconsistencies and irregularities. There is a natural conflict of interest between shareholder interest and public interest.

Guess who wins every time when shareholder interest meets public interest?

Critical Democracy Infrastructure

The pillar of democracy is transparency, and the substance of the pillar is technology.

“Sunlight is the best disinfectant”

This stuff is so imperative and essential to our Democracy, it needs to be lifted up to the level of a public works project.

Why not commercial sector? They will do as little as possible, and have conflict of interest

Why not the government? Slow, and at risk of losing funding.

Our Solution

Bringing together two approaches – fault tolerance and high-availability computing, with the dynamics of open source community.

Rather than being a think tank, they have a group of people in Silicon Valley making things that we can see and touch.

Development Process

Core team
- partner with Mozilla Foundation.
RFC (Request For Comments) Service
- Send out requests for comment to community
Design Congress
- A virtual community to help drive requirements, so they know there is a possibility of adoption
Federally certified

Public Technology Repository – State and local govt, Fed govt, Commercial Vendors, test suites, dynamic continuous testing, everyone is giddy!

Two commercial vendors who are deploying with a commercial deployment license, and are being delivered open source solutions based on draft standards that the consortium is building.

Major work areas

Digital Voter Registration System
Ballot Design Studio
Ballot Casting and Counting Systems
Election Management Services
Operating System Platform

Rails is a major part of their work. They are assembling a great core team.

It has been below the radar, but it will be more public in the future.

Questions

Q: How do we advance or improve the system?
A: Yes, look over the horizon at what the future looks like – Instant runoff, etc. However, there is another half of the question. They DON’T want to build the ‘perfect’ system, and have it be a relic. They have to be driven by real requirements and real adoption. They have to take the EXISTING processes, and make them better. That will get their attention, and drive adoption.

Q: Are the Hardware and Interface designs open source?
A: Absolutely everything is open. Everything will be transparent and funneled through the RFC process. The goal is to build an entire software ecosystem that runs against a known, virgin, commodity hardware system. Then they will examine on a device-by-device basis to plug in new parts. “Open Source Hardware” has never been done, but they will try.

Q: What are the obstacles (e.g. politicians)
A: Lots of them, but their position is that they are technologists, making the best solutions. Senator Patrick Leahy said “please don’t waste time trying to change systems, make things that people can touch and try”.

There are “horrifying” ways the system is designed to preserve incumbency. If this works, it really changes the landscape in a big way.

Q: What percentage of elections are corrupt?
A: They have been doing due diligence, and have found “remarkable” inconsistencies, some of which have resulted in criminal elections. We may think that Obama got elected, things are great, but we dodged a bullet. We are 170 days into the congressional session, and no senator from Minnesota is seated. Politicians will no longer be able to hide and say “the box did it”.

Q: It seems like a huge complex problem to solve, shouldn’t it be bite-sized?
A: They thought about componentizing it, but the only way to do it right is to start with a clean slate. Forget incumbency, and legacy. We need open data and open processes. They are partitioning the process to different buckets, and have different teams working on them. They are laying the foundation for a pluggable, XML-based framework. They are going in a procedural fashion, and really focusing on the 2010 election.

Rapid prototyping, Agile Development approaches with Structured Approaches.

HUGE APPLAUSE AND WHISTLES!

Continuous Integration – in a Box: exploring TSTTCPW

Chad Woolley

Saturday, August 23, 2008

I just released cinabox. It is intended to be The Simplest Thing That Could Possibly Work to set up a Continuous Integration (CI) server, using cruisecontrolrb (CCRB).

Watch the Screencast!

In addition to being a (hopefully) useful tool to help people easily set up CI systems for various platforms and languages, it is also an experiment in simplicity and minimalism:

The project consists of only two simple scripts, one shell script to bootstrap ruby, and one ruby script to set up cruisecontrolrb.
In the script, readability and simplicity are favored over clever abstractions and DRYness. Hopefully, even people who don’t know shell scripting or Ruby can read the scripts and easily understand the commands it is executing.
A standardized environment is assumed: A dedicated Ubuntu 8.04 system, Ruby 1.8.6, and latest dependencies via aptitude. PCs and Virtual Machines are cheap, and Linux and CCRB are free. There’s really no reason you shouldn’t be able to run a dedicated CI box. If this environment doesn’t work for you for some reason, the scripts should be self-explanatory enough that you can easily hack it up to work for your environment (and contribute your version back to to the project!).
I use the magic fairy dust of GitHub to eliminate build scripts, release scripts, packaging, versions, and pretty much all the regular boring overhead of a project. The README.txt is my only documentation. The GitHub “Download Tarball” link automatically provides packaging and uniquely-named packages (by the git hex commit id) for each “release”.

I’m pretty pleased with how this turned out. I hope it will lower the barrier for people to start trying out Continuous Integration, as well as provoke some thought about simplicity and minimalism. I’ve tried it out on a few flavors of Ubuntu VMs and my personal box, and it works for me. Please let me know what you think, and feel free to offer any suggestions for improvement.

Evan Phoenix at Mountain West Ruby Conf

Chad Woolley

Monday, March 31, 2008

I just attended (and thoroughly enjoyed) the Mountain West Ruby Conf, where Evan Phoenix gave a powerful keynote speech.

He talks about the status of Rubinius, and makes some profound observations on modern open source culture and community.

Here’s some highlights, but if you participate in open source, and especially if you help run an open source project, I highly recommend that you watch the video:

Community: Rubinius’ Giant Spec Suite, and its value not only as a living language specification for different implementations of Ruby itself, but as a “gateway drug” which provides a low barrier to get new contributors addicted to open source.
Trust: Asking for forgiveness vs. permission, and the Rubinius commit policy, where any accepted patch gets you commit rights. You can always roll back a change, and debate is healthy.
Worth: The impact of annoying fifteen year olds who make a lot noise versus “core” contributors.
Ego: You are not the project, Mr. Ego! The importance of being wrong and admitting your faults in public.
Innovation: Fostering innovation and debate vs closely holding the mythical “Keys to the Castle”.