Windmill and the Art of Automated Web Testing
I recently attended O’Reilly’s Open Source Conference in Portland and was very excited this year to see a tremendous focus on testing. There were sessions on unit testing and test-driven development; but, I was most excited to see people tackling one of the most complicated and least-advanced testing environments: web testing.
Testing web applications has been a bit of a black art. You’ve got a minimum of three languages to deal with (HTML, CSS, and Javascript), perhaps a fourth language for your tests, and a cornucopia of environments in which to test. Each web browser can be seen as a virtual machine for your application and, currently, there’s a bare minimum of four different virtual machines your application will have to run in (Internet Explorer 6, IE 7, Firefox 1.5, and Firefox 2.) That’s the fewest you can get away with, and you also have to worry about variations in browsers running on different operating systems. Yahoo!’s supported browser list has 22 separate browser/operating system combinations. It is no wonder to me why developers throw up their hands at browser compatibility issues.
Luckily, the team at the Open Source Applications Foundation has been hard at work. Their new testing platform, Windmill, is still rough around the edges but seems to be a great competitor to Selenium. Windmill’s focus is making it easier to debug tests and author them in any programming language, as well as fitting into continuous integration. It provides you with a nice test authoring environment and console with any of its supported browsers to help in the traditional test, fix, and refactor development cycle.
It is easy to forget that we are in the early days of web applications. The coming of Web 2.0 and the heavy use of Ajax is roughly analogous to the leap from text-based command-line applications to graphical applications on the desktop. I am very excited to see our testing tools begin to mature to meet the new challenges of this platform.