Programmatically Controlling Remote Search Engines and Consolidating Results 
Author Message
 Programmatically Controlling Remote Search Engines and Consolidating Results

There are a number of specialized search sites I use that I'd like to gather
the search results from programmatically. I currently scroll through the
search screens manually, writing information down as I go, repetitively
clicking on "Next Page" until I hit the end of the search list. I've been
trying to determine a Perl solution for a while. I've looked at WWW::Search,
WWW::LWP, WWW-Robot, and other Perl search engines and scripts.

Is there a generic perl function to automatically drive a web-based search
engine in a programmatic manner? It would be nice to use a generic perl
solution with general parameters to specify the:
1. Location or tags of the "# of Pages in the Search List" field
2. Location or tags of the "# of Search Results per Page" field
3. Location(s) or tag(s) of the Individual search results field(s)

I'd like to do the following:

For i=1 to #_of_Pages_in_the_Search_List;
    For j=1 to #_of_Search_Results_per_Page(i);
        Output specific search field(s) from Individual_Search_Results(j);
        Get next Search Result;
        }
    Get next page in the Search List;
    }

Sorry for such a detailed note. This should be easy to do in Perl, if I
could just find the right modules.



Wed, 12 Feb 2003 03:00:00 GMT  
 
 [ 1 post ] 

 Relevant Pages 

1. Tricky Local SQL Problem ?????

2. Printing

3. Remote Search Engine Results

4. Remote Search Engine Results

5. Retrieving Links from a Search Engine's Results Page

6. pics with search engine results?

7. search engine results with pic?

8. pics with search engine results?

9. Retrieving Links From a Search Engine Results Page???

10. How-To search engine on search engines/mlml

11. Saving TreeView with Data

12. Accessing Access 97 Tables from Delphi3

 

 
Powered by phpBB® Forum Software