(Off Topic) HTTP/LWP protocol oddness challange 
Author Message
 (Off Topic) HTTP/LWP protocol oddness challange

This started out as a Perl issue, but now I'm scratching my head.

If you hate off topic questions, please excuse me and move on.  But if
you know HTTP protocol, or are a curious type....

[I did posted a similar question to the lower volume
comp.infosystems.www.authoring.cgi]

I use LWP::Simple and LWP::UserAgent to check that URLs are reachable
before redirecting to them in a script.

LWP is reporting pages at one site error 500s that I can reach fine via
Netscape or lynx.  The 500 error is probably a problem with their
scripts.  Fine.

But, since I can read the pages with lynx and Netscape but not with LWP,
I assumed LWP was the problem.  I switched to using telnet to see if I
could figure out what the difference was.

What's driving me crazy is how is it that Netscape and lynx can read the
pages, but I can't via telnet or LWP?

Is it that telnet and LWP are doing something wrong?  (Does Netscape and
telnet send crlf or just lf, and Netscape and lynx send the opposite?)

I though, "Well, their scripts are assuming they will get headers that
LWP isn't sending."  So, I used Tom & Nathan's local little web server
(from The Perl Cookbook) to capture what lynx and Netscape send on a
request.

I then cut-n-pasted those request to telnet and still got the 500 error.  
It only happens on some URIs at that site, too.

For example, here: what I'm sending via telnet (which I tried both from a
unix machine and from my Win 95 machine).

51) ~ %telnet www.library.ca.gov 80
Trying 206.40.21.202...
Connected to ferguson.library.ca.gov.
Escape character is '^]'.
HEAD http://www.*-*-*.com/
Accept: text/html, text/plain, text/sgml, */*;q=0.01
Accept-Encoding: gzip, compress
Accept-Language: en
Host: 209.24.234.230
User-Agent: Lynx/2.8.1rel.1 libwww-FM/2.14
Negotiate: trans

HTTP/1.1 500 Server Error
Server: Netscape-Enterprise/3.5.1G
Date: Tue, 09 Feb 1999 17:27:47 GMT
Content-length: 305
Content-type: text/html
Connection: close

Ok, but watch this:

51) ~ %telnet www.library.ca.gov 80
Trying 206.40.21.202...
Connected to ferguson.library.ca.gov.
Escape character is '^]'.
HEAD http://www.*-*-*.com/
Accept: text/html, text/plain, text/sgml, */*;q=0.01
Accept-Encoding: gzip, compress
Accept-Language: en
Host: 209.24.234.230
User-Agent: Lynx/2.8.1rel.1 libwww-FM/2.14
Negotiate: trans

HTTP/1.1 200 OK
Server: Netscape-Enterprise/3.5.1G
Date: Tue, 09 Feb 1999 17:02:53 GMT
Content-type: text/html
Last-modified: Fri, 22 Jan 1999 07:26:46 GMT
Content-length: 17671
Accept-ranges: bytes
Connection: close

So asking for '/' doesn't work, but '/index.html' does work (which by
Netscape return the same page).  Maybe '/' is a cgi script that redirects
to index.html???

These one doesn't work either:

HEAD /html/main.cfm HTTP/1.0

or
HEAD / HTTP/1.0\
Accept: */*
User-Agent: Mozilla/4.05 [en] (Win95; I ;Nav)

Any ideas?

--



Sat, 28 Jul 2001 03:00:00 GMT  
 (Off Topic) HTTP/LWP protocol oddness challange

in <URL::">
^^ This started out as a Perl issue, but now I'm scratching my head.
^^

It's clearly something that has to do with policies/configurations/bugs
at ferguson.library.ca.gov.  Please ask them why and how.

Abigail
--
perl -wle 'print "Prime" if (1 x shift) !~ /^1?$|^(11+?)\1+$/'



Sat, 28 Jul 2001 03:00:00 GMT  
 
 [ 2 post ] 

 Relevant Pages 

1. SMS Protocols and hosts (totally off topic)

2. use_alarm in LWP::Protocol::http?

3. LWP and HTTP protocol

4. taking on off-topic bootstrap discussions?

5. Off Topic - Linus Torvalds/Open Source code

6. email to the unknown ! - off topic

7. Off Topic Perl Email Question: Sending mail to AOL and Outlook Express with SENDMAIL

8. Anecdote - Off Topic

9. Your code (off topic but bear with me...)

10. Off topic (somewhat)

11. Viral matters [completely off-topic]

12. topmind - Appologies if this is off topic

 

 
Powered by phpBB® Forum Software