Close and Go BackBack to Viget

(re)-Introducing simple_importer

Justin Marney
Justin Marney, Web Developer, October 24, 2009 2

simple_importer has existed for some time as a lowly little gem that made it easy to create CSV import scripts using the Ruby CSV library. The API was usable but wasn't nearly as "simple" as it could be. Motivated by our upcoming Hackday event, Intro to Ruby Metaprogramming, I decided to rewrite simple_importer using some Ruby metaprogramming techniques. The end result is a gem that makes CSV import tasks easy to create and manage, and hopefully provides some real-world examples of basic Ruby metaprogramming.

The basic use case starts with defining importers using a small internal DSL. Each importer has a name, a csv file, and a block of code that is used to process each row in the file. The row instance that is passed into the block is a fastercsv row and all of the fastercsv configuration options are available.

importer :items do
  file 'items.csv'

  foreach do |row|
    Item.create(:name => row[:name])
  end
end

Once you have created an importer, save it into a directory named importers. This directory can be located in the same directory as your Rakefile, lib/importers or app/importers. Next, pull in the simple_importer rake tasks by adding the following line to your Rakefile.

require 'simple_importer/tasks'

Now, when you do a rake -T simple_importer you'll see a rake task for each importer as well as a rake task that will run all of the importers.

Pretty simple. Although, simple_importer does have a few other neat features such as before callbacks and multiple file processing capabilities. Be sure to check out the documentation which has plenty of examples. You can grab the gem from gemcutter, or check out the code on github.

Matt Jones said on 11/04 at 12:43 AM

The one thing that would be great in a lib like this is wrapping the whole iterator in an AR transaction; I’ve used that idiom (Model.transaction around FasterCSV.foreach) quite a bit…

Justin Marney said on 11/04 at 11:24 AM

Matt,

Good idea. One thought, we use this for lots of other import tasks besides ActiveRecord. In fact, ActiveRecord isn’t a dependency at all in this library. However, I think I could add that feature in as a configuration option that is off by default. Thanks for the suggestion!

Justin

Name:

Email:

URL:

Not an infant or robot? Please prove it:
How many days in a non-leap year?

Some HTML (strong, a, em) is allowed.

Notify me of follow-up comments?

We're the Developers

at Viget Labs. We write about web development trends, tips, best practices, industry events, and our projects — all with an emphasis on Ruby on Rails.

Recent Comments

@rick: technically, you’re correct—backticks would be the idiomatic Markdown to format code for inline display. That said, we don’t want to force users to know Markdown in order to express themselves. This system is accessible for normal as well as power users.

Contact Us

Have any questions, comments, ideas, or secrets to share? Let us know.


How many days in a non-leap year?

Sorry, you need to have Javascript enabled to use this form. (Don't blame us, blame the spammers!) If you'd like to contact us, please visit our Contact page.