CODEWITHSUNDEEP
perllwp
user1249684
user1249684

Reputation: 11

Using LWP to get webpage contents

I'm trying to use LWP::UserAgent to gather contents of a webpage, but for some reason when I run the script I get a Windows pop-up saying:

Perl Command Line Interpreter has stopped working

I can't figure out why as I have simplified to the code to its bare essentials in the following:

#!C:/Perl/bin/Perl.exe
use LWP::UserAgent;

$URL = 'http://www.google.com';

my $oHTTPAgent = new LWP::UserAgent;
$oRequest = HTTP::Request->new('GET');
$oRequest->url($URL);
$sResponse = $oHTTPAgent->request($oRequest);
if ($sResponse->is_success) {
    $sPage = $sResponse->content;
}

print $sPage;

What could be the problem?

Upvotes: 0

Views: 8647

Answers (2)

Anil
Anil

Reputation: 3992

A few common things to be considered while scripting:

  • Add use strict;
  • Add use warnings;

Does the path C:/Perl/bin/Perl.exe exists? The shebang{#} should be pointed to the path where Perl has been installed.

A few variables used in the code are not declared, and after modification the code looks like:

use strict;
use warnings;
use LWP::UserAgent;

my $URL = 'http://www.google.com';
my $oHTTPAgent = new LWP::UserAgent;
my $oRequest = HTTP::Request->new('GET');
$oRequest->url($URL);
my $sResponse = $oHTTPAgent->request($oRequest);
if ($sResponse->is_success) {
    my $sPage = $sResponse->content;
    print $sPage;
}

Upvotes: 2

brian d foy
brian d foy

Reputation: 132914

You're hitting google.com, which doesn't allow robots. I don't know if that's your problem, but you don't have another branch if you get an unsuccessful response. At the end of your Perl program, your program shuts down without printing anything.

use LWP::UserAgent;

$url = 'http://www.perl.com';

my $ua = LWP::UserAgent->new;
my $response = $ua->get( $url );
print $response->content if $response->is_success;

print "I'm done!\n";

You might also be interested in Mojo::UserAgent:

use Mojo::UserAgent;

$url = 'http://www.perl.com';

my $ua = Mojo::UserAgent->new;
print $ua->get( $url )->res->body;

print "I'm done!\n";

Upvotes: 3

Related Questions