Hi All,
I'm trying to grab a page query from google to parse, is this possible? Here's a snip of code to demonstrate my problem...
Code:
use LWP::Simple; my $url = qq|http://www.google.co.uk/|; ## This works okay...
#my $url = qq|http://www.google.co.uk/search\?q=cartoons|; ## This doesn't!
my $file = "/_google/google.htm";
&getfile($url,$file);
exit;
sub getfile {
my ($url,$file) = @_;
print "Grab: $url\n";
my $rc = mirror($url, $file);
if ($rc == 304) {
print "Up To Date.\n";
} elsif (!is_success($rc)) {
warn "Error: $rc ", status_message($rc), " ($url)\n";
return(0);
}
}
The problem is that a 403 error is received, my guess is that I'm going to have to send a spoof header set so the request looks like IE to google... or am I wandering off in totally the wrong direction and wasting my time?
moog
-- I've spent most of my money on beer and women... the rest I just wasted.