Development Blog: Viget Labs

Clinton R. Nixon, Senior Developer, April 18, 2008 9

Like many organizations using Rails, we have caught the git wave, and are in a state of transition between git and Subversion. Our open-source work is stored in git repositories, but our client work is still stored in Subversion repositories, and probably will be for some time. While git is amazing, Subversion still has its good qualities, and makes an excellent centralized repository, especially with its ecosystem of user-friendly tools.

The integration between git and Subversion (git-svn) is so well done that several of us have been using git as our interface to all our Subversion repositories. Doing this is fairly simple, but there are some interesting tricks, and so I thought I would share a day in the Viget life with git-svn.

Continue reading "Effectively Using Git With Subversion"

Clinton R. Nixon, Senior Developer, April 17, 2008 8

I am working on a project where we are using Ultrasphinx as the search solution for the site. I had to install Sphinx to make this work, and I soon found out that the version of Sphinx in MacPorts is one release too old to work with Ultrasphinx. (You need to use the latest release of version 0.9.8.) I very much like using package management, but sometimes you have to bite the bullet and compile something yourself.

When I tried to compile Sphinx with the tried-and-true ./configure; make; make install, I ran into some serious problems. It wouldn’t compile, and gave me this error:

g++  -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG   
-o indexer  indexer.o libsphinx.a  -L/opt/local/lib 
-L/opt/local/lib/mysql5/mysql -lmysqlclient -L/opt/local/lib 
-lz -lm  -L/opt/local/lib -lssl -lcrypto  -liconv -lexpat  
-L/usr/local/lib
Undefined symbols:
  "_iconv_close", referenced from:
      xmlUnknownEncoding(void*, char const*, XML_Encoding*)in libsphinx.a(sphinx.o)
  "_iconv", referenced from:
      xmlUnknownEncoding(void*, char const*, XML_Encoding*)in libsphinx.a(sphinx.o)
  "_iconv_open", referenced from:
      xmlUnknownEncoding(void*, char const*, XML_Encoding*)in libsphinx.a(sphinx.o)
ld: symbol(s) not found
collect2: ld returned 1 exit status
make[2]: *** [indexer] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

It turns out that I needed to compile the latest versions of the expat XML parser and iconv. Here’s a quick guide to how I got everything working:

Download iconv from the GNU project and compile it.

curl -O http://ftp.gnu.org/gnu/libiconv/libiconv-1.12.tar.gz
tar zxvf libiconv-1.12.tar.gz
cd libiconv-1.12
./configure --prefix=/usr/local
make
sudo make install
cd ..

I’m installing everything in /usr/local, and I suggest you do, too. I don’t want to mix up the programs I compile with the OS X system libraries and binaries (/usr) or MacPorts’ libraries and binaries (/opt/local).

Download expat and compile it.

curl -O http://internap.dl.sourceforge.net/sourceforge/expat/expat-2.0.1.tar.gz
tar zxvf expat-2.0.1.tar.gz
cd expat-2.0.1
./configure --prefix=/usr/local
make
sudo make install
cd ..

Download Sphinx and compile it. There’s a new flag given to configure this time, specifying the location of my MySQL installation. I installed it via MacPorts; if you installed it differently, you will have to change this flag. Also, you may want to go to the Sphinx download page to see if there’s a new version of Sphinx out. The version current while I’m writing this is a release candidate, and a finished release of 0.9.8 may be out by the time you read this.

curl -O http://www.sphinxsearch.com/downloads/sphinx-0.9.8-rc2.tar.gz
tar zxvf sphinx-0.9.8-rc2.tar.gz
cd sphinx-0.9.8-rc2
./configure --prefix=/usr/local --with-mysql=/opt/local/lib/mysql5
make
sudo make install
cd ..

The install is quick and easy, but certainly wasn’t when I was trying to figure out why Sphinx wouldn’t compile.

Clinton R. Nixon, Senior Developer, April 16, 2008 3

I was incredibly excited to see the announcement of Google App Engine (GAE) recently. I can run web applications on Google’s grid, and have access to their authentication and back-end data store? Sign me right up!

Since it was announced, GAE has taken a lot of heat from pundits. For the most part, they are right, once you cut past the hyperbole. It is exceptionally limited: you cannot run spun-off processes on the server, there are no cron jobs, and the data store, BigTable, is not a relational database. It does threaten to lock in your application: if you use Google authentication, good luck migrating your users, and moving from BigTable to a relational database elsewhere would possibly be tough.

However, limitations can provoke creativity, and enhance applications built for that space. There are a lot of applications you can build without background processes, and if the end-game for your application is not to be bought, having Google’s authentication in place could vastly increase your user registration rate, as most users will not have to sign up for a new account. If you want to make a web application that does an interesting, simple task, GAE may be the best platform around once it gets out of beta.

From a development perspective, the fact that GAE only runs the Python programming language is definitely limiting. It is obvious why it was chosen: Guido van Rossum, the inventor of Python, works at Google, and they needed to limit the Python virtual machine to disable writing to the filesystem or other machine-specific tasks. Google’s announcement that they plan to support other languages, however, excites me. One of Google’s main languages is Java. The Java virtual machine (JVM) is a great piece of software; many languages have been ported to it including Ruby, our favorite here at Viget, and new languages have been written for it, such as Scala, one of my personal favorites. It is fairly trivial to use the JVM to sandbox these languages and limit their abilities to manipulate the machine as Google has done with Python. I expect to see JRuby supported soon, and a flood of languages thereafter. What this means for Rubyists is Google putting resources into developing JRuby, or even Rubinius, if they choose that route for a sandboxed Ruby.

The other exciting part about GAE to me is that they have a software development kit (SDK) to run GAE apps locally. This has already been ported to Amazon’s EC2, allowing you to run GAE apps there. BigTable, Google’s data store, is an interesting concept, and the development community has expressed interest lately in document databases, like CouchDB. With this SDK, we have been given a specification for the interface for BigTable. This is interesting: we may well see competing implementations of this non-relational database come out, allowing GAE apps to use any of them on non-Google machines. This can only result in innovation in database design, something which I have not seen a lot of in the last few years.

Patrick Reagan, Development Director, April 15, 2008 0

A little more than a year ago, we started something unprecedented in my career here at Viget: We made the decision to make some of our code freely available to the public.  The software was Tyrant, and while we haven’t put together an “official” release, this first effort was enough to spark an interest in releasing more of our work to the larger developer community.

A few months later, we reconvened to discuss ideas for Rails plugins that would be useful to other developers.  After a lively discussion and some good ideas, we left the meeting feeling invigorated with plans for a new VPS to host both a Trac installation and a public-facing Subversion repository.

Some of those original ideas died out, but others live on as active projects, and even more have been created as we encounter new challenges.  This is all part of the life-cycle of a healthy open source ecosystem, where new approaches to problems and changes in technology give way to better solutions. 

Continue reading "Open Source, Take Two."

Mark Cornick, Web Developer, April 04, 2008 0

A little over a year ago, the open_id_authentication plugin for Rails made its debut, intending to make OpenID authentication simple. And it did, for a while. But, like Rails itself, OpenID is a moving target and the OpenID specification, as well as the ruby-openid library, have both moved on to version 2.0 since then. The API changed enough that the plugin wouldn’t work with the newer library, effectively leaving applications that use it in the OpenID 1.0 world.

This all changed last Thursday. After much deliberation, a new version of open_id_authentication has landed, with support for ruby-openid 2.0.x, and OpenID 2.0 along with it. The upgrade process is pretty painless; a few database tables need attention (the new plugin will generate a migration to take care of those) and you’ll need to tweak any mocks you’re using in your tests/specs (you are using mocks, right?) It took me about half an hour last night. If you’re doing OpenID in your application, or if you think you want to, now’s a great time to take another look at this plugin.