Close and Go BackBack to Viget

Introducing Garb: Access the Google Analytics Data Export API with Ruby

Tony Pitale
Tony Pitale, Web Developer, April 24, 2009 29

Since the moment Google announced that they would be releasing an API for their de-facto analytics tools, I was very excited to dive into the wealth of useful information within. Shortly thereafter, Josh and Jen from our Marketing Lab came to us to develop some tools that would help in collecting the data for our reporting.

To accomplish these goals, I first read through all of the documentation that Google provided with access to the analytics data export beta. After, with some brainstorming with Pat, we decided to create a new gem to handle much of the interaction with the new API. Garb is what we've come up with.

Thankfully, the waiting period is over. Google has announced the release of the API to the public, and Josh has written his take on the announcement.

The API and Design of Garb

Documentation

The documentation for the analytics API is pretty solid, though complex. Sections describe the authentication process, retrieving accounts and profiles, the metrics and dimensions available and how they could be combined, and details for sorting and filtering the data. It is very well written, and very concise. I suppose I should have expected nothing less from Google.

A Word on Dimensions and Metrics

The API provides access to all of the same dimensions and metrics (and more) that are familiar to anyone who has ever used Google Analytics and especially the custom reporting tools within. Through the API there is, however, a substantial set of rules surrounding the selection and combination of dimensions and metrics.

Basically, two dimensions and up to 50 metrics can be selected in any one request. Easy enough? Unfortunately, we hit snags trying to make (what seem like) commonplace combinations. For example, while we were developing Garb, you could not get the "visits" metric scoped to the "requested page URI" dimension. It took a good bit of studying to truly understand the boundaries of these combinations.

So, On To Using Garb in Your Project

There are three parts to using Garb. The first is getting a session, which requires your Google Analytics username and password . Second is the profile (or many profiles) that your authenticated account has access to. Last, given the first two parts, you can create reports and retrieve the data from the API.

Session

Sessions are simple enough to create and are currently stored globally. This is simpler, but limits us to one session at a time.


Garb::Session.login('username', 'password')

Profile


profile = Garb::Profile.all.first # or select from the list

Reports

You can create a simple report.


report = Garb::Report.new(profile, {:metrics => [:visits]})
report.all

Or, an advanced one.


class ExitsReport < Garb::Report
  def initialize(profile)
    super(profile) do |config|
      config.start_date = Time.now.at_beginning_of_month
      config.end_date = Time.now.at_end_of_month
      config.metrics << [:exits, :pageviews, :exit_rate]
      config.dimensions << :request_uri
      config.sort << :exits.desc
      config.max_results = 10
    end
  end
end

ExitsReport.new(profile).all

Note: the date/time related methods are from ActiveSupport.

The Google Analytics Data Export API has come along very quickly. It's already powerful enough to provide the data to generate reports that we've never before been able to create. Soon, we'll only be limited by our imaginations, which is exactly how Google wants it. There have always been many compelling reasons to use Google Analytics. Now, I struggle to find a reason not to.

I encourage everyone interested in using this library to read the documentation, check the code out on Github at http://github.com/vigetlabs/garb/tree/master and, of course, comment, question, and enjoy. If you'd like to go straight to installing it be sure to vigetlabs-garb sourced from Github until I can get it up on Rubyforge!

David Reese said on 04/29 at 04:33 PM

This is great—just settling down to play with it. Gem instructions might be helpful since the name is not immediately apparent:
# sudo gem install vigetlabs-garb

Tony Pitale said on 04/29 at 05:43 PM

Thanks David, I’ll update the post to clarify that. I also suggest reading the README and I’ll be posting documentation to the Github wiki soon.

Charlie Plexus said on 05/14 at 04:15 PM

Has anything changed with the gem itself recently? I installed the gem but when I try to require it or use it I get the following error:
dyld: NSLinkModule() error
dyld: Symbol not found: _htmlNewParserCtxt
Referenced from: /usr/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.9.8/lib/libxml_ruby.bundle
Expected in: flat namespace

Any suggestions?

Tony Pitale said on 05/14 at 04:34 PM

Libxml-ruby built without issue? My first guess is that the libxml C library is not installed. This would generally cause libxml-ruby gem to fail. My first action would be attempt to reinstall both of these components.

Charlie Plexus said on 05/18 at 09:34 AM

Thanks Tony,
Apparently my problem is specific to Mac OS X 10.5.6—I had to remove libxml2 and recompile it from source (v. 2.7.2), then re-install the libxml-ruby gem. That fixed it. Just putting it up here in case anyone else has the same problem.

Thanks for writing the gem, I’ve been messing around with it over the weekend and trying to familiarize myself with the Analytics API. Great job!

Philipp said on 07/01 at 07:22 AM

Thank for this article. But i have a problem with this GEM:

after i do:
profile = Garb::Profile.all.first
i become this error:
RuntimeError: “Accounts URL did not end in /default”

What is the error here? A few days ago it still worked?!

Hope you can help.
Thx

Tony Pitale said on 07/01 at 10:17 AM

Google changed their API for requesting accounts. We’ve updated the gem on github, and will be pushing a new gem to rubyforge shortly.

Philipp said on 07/02 at 08:04 AM

Please, can you post it, when the gem is up to date?…
Thank you very much!

Bruce Williams said on 07/05 at 06:51 PM

I’d love to see that new gem on Rubyforge too; also running into this issue.

Tony Pitale said on 07/05 at 07:34 PM

Hi folks, sorry it took so long! The gem should be up on Rubyforge momentarily.

Doug Hall said on 07/21 at 11:02 AM

Hi - first off - great library!  Now, can I suggest a quick change?  With the API being relaxed a little recently in terms of the number of rows that can be pulled I came across a few instances where the http.get call timed out.

For now I’ve just added a http.read_timeout = *larger value* tweak in data_request.rb.  This has helped a wee bit but I think a tidier optioanl timeout param might be a nice addition.

Cheers

Doug

Tony Pitale said on 07/21 at 11:18 AM

Thanks for the suggestion Doug. We haven’t come across this just yet. If you fork on Github, add the code and tests required, we’ll be happy to assess a pull request. Your other option is to create an issue on Github, suggesting your fix, and we’ll try and get to it when we can.

Thanks! Tony

John Clarke Mills said on 09/16 at 03:00 PM

I am getting a very frustrating error that I cant to the bottom of.  Every time I invoke ‘report.all’ I get ‘undefined method `all’ for #<Garb::Report:0x103059188>’.  Anyone else have this problem?

John Clarke Mills said on 09/16 at 03:10 PM

Also, is there any documentation anywhere?

Joshua Lippiner said on 09/17 at 11:07 PM

This is great, thanks!

Two quick questions:
1 - Do you plan on adding support for Authsub instead of simple login?
2 - How, if at all, is caching handled?  It seems that there are a lot of times when duplicate data would be pulled that caching could fix.

Thanks,

Josh

Tony Pitale said on 09/18 at 02:07 PM

@John: You should probably check the documentation available in the github wiki http://wiki.github.com/vigetlabs/garb as the code examples in this blog post are old.

@Joshua:
1. I doubt we would ever do authsub, we will probably add oauth at some point. We usually add stuff when we really need it on something. Anyone is more than welcome to add the functionality in a fork on Github and we will consider pull requests.

2. Caching is done at different places by memoization. Anything beyond that is up to the users of the library.

Ernesto Tagwerker said on 12/14 at 01:24 PM

Hi Tony,

Thanks for creating this gem.

I have a simple question. How should I go about using the gem if I want to get the total pageviews for a particular page/URI?

I got as far as getting my profile, but I don’t know how I should continue. Do I need to create a report for that particular URI?

- Ernesto

Tony Pitale said on 12/14 at 01:25 PM

@Ernesto

I would try this: http://wiki.github.com/vigetlabs/garb/building-a-report

Ernesto Tagwerker said on 12/14 at 01:50 PM

Thanks for your quick response.

Now I’m getting an error. This is what I did:

./script/console
>> require ‘garb’
>> Garb::Session.login(’email’, ‘password’)
>> profile = Garb::Profile.all.first
>> report = Garb::Report.new(profile)
>> report.metrics :pageviews
>> report.dimensions :request_uri
>> report.filter :request_uri.contains => ‘873’
>> report.results

And this is the error I get:

RuntimeError: “No such dimension(s): ga:requestUri”
from /usr/local/lib/ruby/gems/1.8/gems/garb-0.3.1/lib/garb/data_request.rb:23:in `send_request’
from /usr/local/lib/ruby/gems/1.8/gems/garb-0.3.1/lib/garb/resource.rb:83:in `send_request_for_body’
from /usr/local/lib/ruby/gems/1.8/gems/garb-0.3.1/lib/garb/report.rb:27:in `results’
from (irb):8

Which is weird because I can see this:
>> report.dimensions
=> #<Garb::ReportParameter:0x53a7d34 @name=:dimensions, @elements=[:request_uri]>

Any ideas?

Tony Pitale said on 12/14 at 02:18 PM

@Ernesto

This error means that GA changed their API. That dimension must not be available.

I’ll have to update the wiki.

You can see the current available Metrics and Dimensions (and valid combos) here: http://code.google.com/apis/analytics/docs/gdata/gdataReferenceDimensionsMetrics.html

Ernesto Tagwerker said on 12/14 at 02:19 PM

Never mind. The problem was that :request_uri was supposed to be :page_path

You might want to change that in your doc. I don’t see requestUri in the GA docs now. It might have been there in the past.

http://code.google.com/apis/analytics/docs/gdata/gdataReferenceDimensionsMetrics.html

Thanks!

Matteo Galli said on 01/07 at 04:17 AM

Hi, first of all great job with this library, just want to report that using

profile = Garb::Profile.all.detect{|p| p.table_id == ‘7654321’.to_ga}

as defined in the library wiki at github account will raise an error

irb(main):010:0> Garb::Profile.all.detect { |p| p.table_id == ‘24553770’.to_ga }
NoMethodError: undefined method `to_ga’ for “24553770”:String
from (irb):10:in `block in irb_binding’
from (irb):10:in `each’
from (irb):10:in `detect’
from (irb):10
from /opt/local/bin/irb:12:in `<main>’

by the way i found the same method defined in the module with Garb::to_google_analytics() that seems to work fine.

Tony Pitale said on 01/07 at 10:04 AM

@Matteo

Thanks, I removed that section from the wiki. Really, nobody used that anyway. That’s why I added Profile.first(’UA … ‘)

Matteo Galli said on 01/07 at 11:34 AM

I know but under the same web property I have two different reports and I need to access to just one of them, actually the other that is not returned when using Profile.first("UA..."), that’s why I was trying to use Garb::Profile.all.detect { |p| p.table_id == ‘24553770’.to_ga } (that now has been changed with either Garb::Profile.all.detect { |p| p.table_id == ‘ga:24553770’} or Garb::Profile.all.detect { |p| p.table_id == Garb::to_google_analytics(‘24553770’) }

Tony Pitale said on 01/07 at 12:13 PM

@Matteo
I’m not sure I understand. There should be a one-to-one mapping between web property id and table id. Garb uses table id internal as required by the GA api, but the web property id is the one that most people are familiar with.

“reports” should not be tied to any particular profile. What do you mean when you talk about “reports”.

Matteo Galli said on 01/07 at 01:12 PM

would you mind giving me your email address so that I can send you a screenshot of what I get on Analytics that probably will clarify the situation?

sale87 said on 01/18 at 05:55 PM

Hi,

i’m getting strange error when I try to use you’r gem. Here is sample code:

require “garb”
Garb::Session.login("user", “pass")
profile = Garb::Profile.first(’UA-xxxxxx-xx’)
report = Garb::Report.new(profile, :start_date => (Date.today - 30), :end_date => Date.today)
report.metrics :source
report.results

when I run above code the result is:

RuntimeError: “No such metric(s): ga:source”
from /usr/local/lib/ruby/gems/1.9.1/gems/garb-0.5.1/lib/garb/data_request.rb:23:in `send_request’
from /usr/local/lib/ruby/gems/1.9.1/gems/garb-0.5.1/lib/garb/resource.rb:75:in `send_request_for_body’
from /usr/local/lib/ruby/gems/1.9.1/gems/garb-0.5.1/lib/garb/report.rb:22:in `results’
from (irb):6
from /usr/local/bin/irb:12:in `<main>’

when I inspect response, i get:
#<Net::HTTPBadRequest 400 Bad Request readbody=true>

I already looked at http://code.google.com/apis/analytics/docs/gdata/gdataReferenceDimensionsMetrics.html, but as you can see ga:source is still valid. I also tried with ga:eventCategory (:event_category) but it also return above error…

Is this api problem, or I’m not using metrics right?

Tnx you in advance.

Best regards,
Sasha

Tony Pitale said on 01/18 at 07:08 PM

@Sasha: source and event_category are both dimensions. Use report.dimensions :source instead.

sale87 said on 01/18 at 07:29 PM

@Tony
Thank you very much, I didn’t notice, that part in Google documentation. My bad :)

Tnx you for excellent gem! Keep up with good work!

Regards,
Sasha

Name:

Email:

URL:

Not an infant or robot? Please prove it:
How many days in a non-leap year?

Some HTML (strong, a, em) is allowed.

Notify me of follow-up comments?

We're the Developers

at Viget Labs. We write about web development trends, tips, best practices, industry events, and our projects — all with an emphasis on Ruby on Rails.

Contact Us

Have any questions, comments, ideas, or secrets to share? Let us know.


How many hours in a day?

Sorry, you need to have Javascript enabled to use this form. (Don't blame us, blame the spammers!) If you'd like to contact us, please visit our Contact page.