A Recorder/Code generator for FireWatir

gistfile1.txt

Abstract
========

This project aims at allowing users to record normal usage on a
website and then have a script generated that can reproduce such
actions.

This script is generated in Ruby and can be changed and mixed with
existing Ruby code to extend its behaviour, using regular Ruby
variables, loop constructs, etc.

A parser is going to be built for Firefox logs of user-activity
(extracted by existing free software extensions) with the ability to
generate code for the browser-driving tool FireWatir.



A Recorder/Code generator for FireWatir
======================================

# Synopsis
Log user interaction with a website on Firefox and generate Ruby code
using the FireWatir[1] API to reproduce such actions programmatically.


# Benefits to the community
The idea of a recorder for Watir[2] was suggested at the ruby-talk
mailing list and was enthusiastically embraced by many[3]. Watir is a
browser-driver API in Ruby that simulates user interaction with
websites. As Watir only works on Windows+IE, the idea was quickly
turned to FireWatir which works on Firefox, Windows and Linux. Such a
recorder would allow for easier creation of scripts for testing web
applications, allowing testers to focus on designing the test instead
of losing time boringly querying page elements by hand and writing
FireWatir code themselves. There is also a growing movement of people
trying to get better screen scraping techniques into the Ruby toolkit
and this recorder would facilitate information extraction from web
pages, the creation of mash-up applications, and general sweet hackery
=) The idea is: hit "record", click link, fill form, push button,
"stop recording", change literals for variables and you're all set.

# Deliverables
A user-action-to-FireWatir-code translator. This will most likely be
in the form of a parser/generator for action logs generated by already
existing Firefox logging extensions.

These extensions might have to be altered a bit in order to log the
whole scope of FireWatir-reproducible interactions. In this case such
changes are also part of the deliverables.

Fully documented code, documentation of use, a tutorial, a screencast
and whatever else the community sees fit.

# Project details
The initial idea was to use an approach similar to that of Perl's
HTTP::Recorder[4]. It uses a proxy to intercept HTTP requests, then
adds specific markers into the html tags of the returned page so that
it can have richer information about clicked links, etc. in further
requests. The idea was discussed with Angrez Singh [5], Amit Garde [6]
(developers of Watir and FireWatir), Midhun Chandran and in the
#ruby-lang freenode channel and the consensus was that the proxy
approach was a bad idea. It is much more limited in the range of
interactions it can capture with the only advantage being
browser-independence which is uneeded in an already browser-tied
system (FireWatir). So the winner idea was to go into the browser.

There are to my knowledge 3 Firefox extensions that log user activity
on a website: DejaClick[7], TestGen4Web[8] and Selenium-ide[9], the
last two being free software. Both free ones record user activity into
an intermediary language (xml for TestGen4Web and a homemade one for
Selenium-ide) and provide support for writing translators.

The project execution devides itself into the following parts:

1. Install TestGen4Web and Selenium-ide from svn, play with them, get
acquainted with their output format and study what kinds of things
each can record. Get a rough idea of how each one interfaces with the
browser to obtain the usage information and where in the code they do
that.

2. Heavy toying with FireWatir, see what kinds of interactions it can
simulate, get a feel for the edge cases.

3. Compare the expressiveness of what each extension can record to
what FireWatir can reproduce and choose the closest match. Here should
be taken into account not only how close each extension comes, but how
difficult it would be to adapt it to work with FireWatir and how big a
user base it already has. The community will have a major role in this
decision.

4. Chosen the extension, modify it to record the whole breadth of
FireWatir's functionality if needed.

5. Start writing the translator (in Ruby for TestGen4Web or in
Javascript for Selenium-ide). Release a first alpha version with basic
functionality (static-page behavior: click links, fill-out forms, push
buttons) to get feedback and help from the community.

6. Finish coding all features, release beta for feedback.

7. Write documentation while catching bugs.

8. Package and deploy final product.

# Project schedule
I'm from Brazil so here it will actually be a Winter of Code (no cold
though). Thus I'll have classes at the University until the end of
June. I also have a 20h/week job that I'll quit if I get accepted so
until June 30th I'll have some 25 weekly hours for the project. Then
comes vacation and I'll be on this 24/7.

April 11: get acquainted with the tools I'll use: version control,
editors, bug tracking, project management, deployment tools, etc.

April 26: start phase 1.

May 16: phase 1 completed.

June 6: phase 2 completed.

July 2: phase 3 and 4 completed.

July 15: phase 5 (alpha release) completed.

August 4: phase 6 (beta release) completed.

August 13: phase 7 completed.

August 18: phase 8 completed.

#Bio
I'm a 4th-year Computer Engineering student at the State University of
Campinas (UNICAMP). I've been involved with Free Software since I
entered in 2004, using and helping people out. I am part of the local
Free Software student group and currently involved in organizing
weekly seminars on various related subjects. This is basically my
first attempt at contributing code back, except for a tiny patch to a
Java library (JFreeChart) I sent last year and a few spell-checking in
method names for the bn4r gem.

My experience as a programmer is largely academic, writing some
Pascal, a bit more C and very little Lisp for class projects. The only
exception is that I have a 1 year+ experience as a Java programmer in
a large project with distributed teams for the Brazilian-equivalent of
IRS (this is the job I'll quit to work on this project if I get
accepted; I've learned all I could there).

I found out about Ruby late last year, got very interested, joined the
ruby-talk list early January this year and have since been studying to
ramp up so I could get my hands dirty in some real-world project. I am
a big fan of the community and I'm constantly amazed at how generous
and kind everyone is and has been to me. It seems like the culture of
users giving back to the community in libraries, tools and
infrastructure is also very strong and I want to be part of that.

I've already started discussing this project idea with other free
software developers through email and IRC and already have people
involved and willing to help me. Angrez Singh and Amit Garde have
agreed to co-mentor me on this.

UNICAMP is one of the best Computing schools in Latin America because
of its highly competent faculty and its difficult admission process. I
am proud to have studied here thus far and I think it has given me a
great capacity for solving problems and solid grounds on which I can
build additional knowledge quickly. I am completely capable of
delivering what I proppose.

#References
[1] FireWatir: http://wiki.openqa.org/display/WTR/FireWatir+-+WATiR+for+FireFox
[2] Watir: http://www.openqa.org/watir/
[3] 'Google Summer of Code -- Get Ready for the Proposal Window', M.
Edward Borawsky, Bret Pettichord, Peter Szinek. March 14h, 2007.
[4] HTTP::Recorder: http://www.perl.com/pub/a/2004/06/04/recorder.html
[5] Angrez Singh: http://angrez.blogspot.com/
[6] Amit Garde: [email protected]
[7] DejaClick: http://www.dejaclick.com/
[8] TestGen4Web:
http://developer.spikesource.com/wiki/index.php/Projects:TestGen4Web
[9] Selenium-IDE: http://wiki.openqa.org/display/SIDE/Home