Wanted: multi-page web robot for automated testing 
Author Message
 Wanted: multi-page web robot for automated testing

I'm looking for a perl module--LWP is a good start--that can spider
through a Web process, looking for links and sending new requests based
on HTTP responses (eg. automate setting up a Hotmail account) anyone
know of such a robot or code that would be a good model for writing one?

I've starting writing one of my own that basically sends one request,
looks for a link in the page using a regex, and sends another request,
until the process is complete. The multi-step process I'm testing logs a
user into my site, so I know the test was successful if a particular
cookie is set (I can just check the cookie jar that LWP uses). I could
also just grep the content for a "Success!" message that I know should
be there if the login was successful.

I have looked at HTTP::WebTest, but this seems specialized for testing
particular pages and their response times, not a sequence of pages and
requesting new pages by using links in the content of requested pages.
Any ideas?

--
  6:38pm  up 50 days, 12:32,  7 users,  load average: 0.09, 0.09, 0.08



Mon, 31 May 2004 10:39:28 GMT  
 Wanted: multi-page web robot for automated testing
Ovid on Perlmonks.org sent me a link to Roboweb, a very cool Perl script
that acts as a proxy, recording all your requests and making them
replayable!
http://sf.net/projects/roboweb/

(Sorry to posterbate... I would still like to know if anyone has found
cool tools like I'm trying to describe)

Quote:

> I'm looking for a perl module--LWP is a good start--that can spider
> through a Web process, looking for links and sending new requests based
> on HTTP responses (eg. automate setting up a Hotmail account) anyone
> know of such a robot or code that would be a good model for writing one?

> I've starting writing one of my own that basically sends one request,
> looks for a link in the page using a regex, and sends another request,
> until the process is complete. The multi-step process I'm testing logs a
> user into my site, so I know the test was successful if a particular
> cookie is set (I can just check the cookie jar that LWP uses). I could
> also just grep the content for a "Success!" message that I know should
> be there if the login was successful.

> I have looked at HTTP::WebTest, but this seems specialized for testing
> particular pages and their response times, not a sequence of pages and
> requesting new pages by using links in the content of requested pages.
> Any ideas?

> --
>   6:38pm  up 50 days, 12:32,  7 users,  load average: 0.09, 0.09, 0.08



Mon, 31 May 2004 13:07:40 GMT  
 Wanted: multi-page web robot for automated testing

Quote:

> I'm looking for a perl module--LWP is a good start--that can spider
> through a Web process, looking for links and sending new requests based
> on HTTP responses (eg. automate setting up a Hotmail account) anyone
> know of such a robot or code that would be a good model for writing one?

I just posted a namespace request for a web testing module I'm
writing.  The module is called WWW:AppTest and does just what you're
talking about.  I'll be happy to send you the modules (they aren't on
CPAN yet).

The module is used by writing a script that uses a browser object to
navigate through your web application.  Usually the best practice is
to identify critical elements in the web page with an ID attribute.
You can "click" on link elements, set the values of form fields, and
submit the forms (either with $form->submit or by using the click
method of the submit button objects).  It's a little more work to set
up than testers that follow you in your own web browser, but you get a
lot of flexibility because you can run any tests you want in the perl
script.  For example, in the script below, I test that certain files
have been created after filling out some forms.

The module is in alpha (I only finished a working version of it two
nights ago) so there's plenty of room for you to jump in and improve
it the way you want.  Viva la open source.

To give you an idea of what it does, here's a script that tests an
application I'm writing:

 #!/usr/local/bin/perl -wT
 use strict;
 BEGIN{$ENV{'PATH'} = ''}
 $ENV{'PATH'} = '';
 use lib '../PtLib';
 use WebTester::UserAgent;
 use WebTester::Tokenizer;
 use PtUtils::Strings;

 my ($browser, $req, $res, $parser, $form,
    $input, $new_inst_id, $form);
 my $testappinsts = '/home/users/idocs/pt/testapp/insts';

 #---------------------------------------------------------------------------
 # create a user agent object
 #
 $browser = WebTester::UserAgent->new;

 $browser->{'verbose'} = 1;
 $browser->{'hold_code'} = 1;
 #
 # create a user agent object
 #---------------------------------------------------------------------------

 #---------------------------------------------------------------------------
 # login
 #
 $browser->go_page('http://www.idocs.com/cgi-bin/pt2/ptcgi/admin/login.pl');
 $form = $browser->{'forms'}->[0];

 $form->{'elements_by_name'}->{'pw'}->set_value('edited');
 $form->submit();
 #
 # login
 #---------------------------------------------------------------------------

 #---------------------------------------------------------------------------
 # admin page
 #
 $browser->check_for_id('home_page_form', 'app_link_emailform',
'app_link_testapp');
 $browser->{'ids'}->{'app_link_testapp'}->click;
 #
 # admin page
 #---------------------------------------------------------------------------

 #---------------------------------------------------------------------------
 # inst list
 #
 $browser->check_for_id('new_inst_link');
 $browser->{'ids'}->{'new_inst_link'}->click;
 #
 # inst list
 #---------------------------------------------------------------------------

 #---------------------------------------------------------------------------
 # create new inst
 #
 $browser->check_for_id('create_inst_form');

 $new_inst_id = lc(PtUtils::randword(15));

 $form = $browser->{'ids'}->{'create_inst_form'};
 $form->{'elements_by_name'}->{'new_name'}->set_value($new_inst_id);
 $form->{'elements_by_name'}->{'new_id'}->set_value($new_inst_id);

 print "creating inst: [$new_inst_id]\n";

 $form->submit('xmycgi'=>1);

 # make sure the new instantiation directory exists
 unless (-d "$testappinsts/$new_inst_id")
        {die "did not create instantiation id=$new_inst_id"}

 print "successfully created instantiation id=$new_inst_id\n";
 #
 # create new inst
 #---------------------------------------------------------------------------

 #---------------------------------------------------------------------------
 # admin page for new instantiation
 #
 $browser->check_for_id('delete_instantiation');
 $browser->{'ids'}->{'delete_instantiation'}->click;
 #
 # admin page for new instantiation
 #---------------------------------------------------------------------------

 #---------------------------------------------------------------------------
 # admin page for new instantiation
 #
 $browser->check_for_id('delete_yes', 'delete_no', 'logout_link');
 $browser->{'ids'}->{'delete_yes'}->click;

 # make sure the new instantiation directory was deleted
 if (-d "$testappinsts/$new_inst_id")
        {die "did not delete instantiation id=$new_inst_id"}

 print "successfully deleted instantiation id=$new_inst_id\n";

 # logout
 $browser->{'ids'}->{'logout_link'}->click;
 #
 # admin page for new instantiation
 #---------------------------------------------------------------------------

 #---------------------------------------------------------------------------
 # logout page
 #
 $browser->{'ids'}->{'logout_success'} || die "unable to log out";
 #
 # logout page
 #---------------------------------------------------------------------------

 print "done\n";



Mon, 31 May 2004 23:52:50 GMT  
 
 [ 3 post ] 

 Relevant Pages 

1. Pub needs automated web page generator program

2. Automating Web Page Update

3. Automated Testing of a Web Site

4. Safely sending data between a multi-web paged perl based CGI Script

5. Safely sending data between a multi-web paged perl based CGI Script

6. Fetching a web page from a site that wants me to be logged in

7. composer wants web page designed - nyc

8. Suggestions Required - New Web Robot

9. My solution to LWP robot/ua make test failure

10. web robot

11. Making web robot ...(search engine)

12. problem with simple Web robot

 

 
Powered by phpBB® Forum Software