Close and Go BackBack to Viget

(re)-Introducing simple_importer

Justin Marney
Justin Marney, Web Developer, October 24, 2009 2

simple_importer has existed for some time as a lowly little gem that made it easy to create CSV import scripts using the Ruby CSV library. The API was usable but wasn't nearly as "simple" as it could be. Motivated by our upcoming Hackday event, Intro to Ruby Metaprogramming, I decided to rewrite simple_importer using some Ruby metaprogramming techniques. The end result is a gem that makes CSV import tasks easy to create and manage, and hopefully provides some real-world examples of basic Ruby metaprogramming.

The basic use case starts with defining importers using a small internal DSL. Each importer has a name, a csv file, and a block of code that is used to process each row in the file. The row instance that is passed into the block is a fastercsv row and all of the fastercsv configuration options are available.

importer :items do
  file 'items.csv'

  foreach do |row|
    Item.create(:name => row[:name])
  end
end

Once you have created an importer, save it into a directory named importers. This directory can be located in the same directory as your Rakefile, lib/importers or app/importers. Next, pull in the simple_importer rake tasks by adding the following line to your Rakefile.

require 'simple_importer/tasks'

Now, when you do a rake -T simple_importer you'll see a rake task for each importer as well as a rake task that will run all of the importers.

Pretty simple. Although, simple_importer does have a few other neat features such as before callbacks and multiple file processing capabilities. Be sure to check out the documentation which has plenty of examples. You can grab the gem from gemcutter, or check out the code on github.

Matt Jones said on 11/04 at 12:43 AM

The one thing that would be great in a lib like this is wrapping the whole iterator in an AR transaction; I’ve used that idiom (Model.transaction around FasterCSV.foreach) quite a bit…

Justin Marney said on 11/04 at 11:24 AM

Matt,

Good idea. One thought, we use this for lots of other import tasks besides ActiveRecord. In fact, ActiveRecord isn’t a dependency at all in this library. However, I think I could add that feature in as a configuration option that is off by default. Thanks for the suggestion!

Justin

Commenting is not available in this weblog entry.

We're the Developers

at Viget Labs. We write about web development trends, tips, best practices, industry events, and our projects — all with an emphasis on Ruby on Rails.

Recent Comments

Your post has made me think!

We people get used to that what we daily do. And normally we forget that we have to evolve.

Contact Us

Have any questions, comments, ideas, or secrets to share? Let us know.


How many hours in a day?

Sorry, you need to have Javascript enabled to use this form. (Don't blame us, blame the spammers!) If you'd like to contact us, please visit our Contact page.