CODEWITHSUNDEEP
perl
adsdsa
adsdsa

Reputation: 667

Complete form with LWP::UserAgent in Perl

I am trying to complete a form by post using the module LWP::UserAgent in Perl of melissadata page. The problem is never complete well fields and the request never completes, showing "normal page" without the desired result.

The code:

use LWP::UserAgent;

my $nave = LWP::UserAgent->new();
$nave->timeout(5);
$nave->agent("Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.1.12) Gecko/20080201Firefox/2.0.0.12");

my $target = "5.135.178.142";

$re = tomar(
  "http://www.melissadata.com/lookups/iplocation.asp",
  { 'ipaddress=' => $target, 'submit' => 'Submit' }
);

if ( $re =~ /City<\/td><td align=(.*)><b>(.*)<\/b><\/td>/ ) {
  print "Found : $2\n";
}
else {
  print $re;
}

sub tomar {
  my ($web, $var) = @_;
  return $nave->post($web, [%{$var}])->content;
}

Can anyone help?

Upvotes: 0

Views: 100

Answers (1)

Degustaf
Degustaf

Reputation: 2670

The answer for why your scraper doesn't work is that the website you are interested in uses Javascript, and LWP::UserAgent doesn't support Javascript.

For how to handle this, you have a couple of options.

  1. Parse out the JS, and run it using the Javascript package

  2. Use WWW::Mechanize::Firefox and perl will automate firefox, and use it to process the JS

  3. This is Perl (TMTOWTDI) find another package that can parse/process the JS for you.

Upvotes: 1

Related Questions