I've built a web-bot in VB .net that parses through web pages and
pulls out the links using WebResponse. It works great on regular
non-server pages, such as .HTM's, but for ASP pages it pulls in no
information for the page content. I tried playing with the Proxy
settings, but I basically was just taking a shot in the dark and it
didn't seem to help. Any ideas would be greatly appreciated. I may
just be overlooking something simple.

Thanks,
Ryan Hayes