Getting a web page content using perl 
Author Message
 Getting a web page content using perl

Hi,

I am new about using perl with the internet. I would like to use perl to
get the web page content using perl. For example, if I input
" http://www.*-*-*.com/ " to my perl program, my perl program will give me
the source content of that web page. Could this kind of thing be done by
perl? Any suggestion?

Thanks,

-- Chris



Wed, 11 Apr 2001 03:00:00 GMT  
 Getting a web page content using perl

Quote:

> I am new about using perl with the internet. I would like to use perl to
> get the web page content using perl. For example, if I input
> "http://www.perl.com" to my perl program, my perl program will give me
> the source content of that web page. Could this kind of thing be done by
> perl? Any suggestion?

This is in the PERL FAQ (you'll be hearing that a lot on this newsgroup)
at http://www.perl.com/CPAN-local/doc/FAQs/FAQ/PerlFAQ.html .  The FAQ is
very helpful and really does contain the most common questions.

You can use backtics (``) to just run Lynx and capture the output:

$html_code = `lynx -source $url`;
$text_data = `lynx -dump $url`;

This, of course, assumes you have lynx installed.  You can also do it in
100% perl.  The LWP::Simple perl module makes this very easy:

use LWP::Simple;
$content = get("http://www.perl.com/");

For full documentation, you can type "perldoc LWP::Simple".  If you don't
have the LWP::Simple module installed, you can get it with the CPAN.pm
module:

perl -MCPAN -e shell;
install LWP::Simple

For slightly more complicated requirements, you can use LWP::UserAgent.
Speaking of which, could anyone provide me with an example of fetching a
password-protected page with LWP::UserAgent? tnx.

And that's it.

Tobin Fricke



Wed, 11 Apr 2001 03:00:00 GMT  
 Getting a web page content using perl
: I am new about using perl with the internet. I would like to use perl to
: get the web page content using perl. For example, if I input
: "http://www.perl.com" to my perl program, my perl program will give me
: the source content of that web page. Could this kind of thing be done by
: perl? Any suggestion?

Check out the documentation for the LWP module.



Thu, 12 Apr 2001 02:00:00 GMT  
 Getting a web page content using perl

: I am new about using perl with the internet. I would like to use perl to
: get the web page content using perl. For example, if I input
: "http://www.perl.com" to my perl program, my perl program will give me
: the source content of that web page. Could this kind of thing be done by
: perl? Any suggestion?

   There are Perl modules for doing that.

      use LWP::Simple;

--
    Tad McClellan                          SGML Consulting

    Fort Worth, Texas



Thu, 12 Apr 2001 02:00:00 GMT  
 Getting a web page content using perl
Quote:

>OK, I downloaded libwww, and while trying to install, it says that I need
>4 more modules: HTML::Parser MIME::Base64 NET::FTP and ,MD5 (optional)

>I've found all but the FTP module -- where is it, or what is it called?

FTP is part of libnet, CPAN/modules/by-module/Net/libnet-1.0605.tar.gz

--
                Jason Kohles -- Internet Consultant

    Even if you aren't in doubt, consider the mental welfare of the person who
    has to maintain the code after you, and who will probably put parens in
    the wrong place.  --Larry Wall in the perl man page



Fri, 13 Apr 2001 03:00:00 GMT  
 
 [ 7 post ] 

 Relevant Pages 

1. getting web page content

2. web page contents into a perl script

3. Getting Perl value to insert in web page

4. opening a remote web page and saving the content

5. How to constantly get contents of a Web page

6. How to constantly get contents of a Web page

7. getting HTML pages with javascripts inside using perl?

8. getting HTML pages with javascripts inside using perl?

9. Getting IP address of user who has submitted information from a Web page

10. Getting the Current URL of a Web Page in IE5 and NN4.75

11. getting a web page from secure URL

12. Getting web page programmatically

 

 
Powered by phpBB® Forum Software