CODEWITHSUNDEEP
perl
user1092042
user1092042

Reputation: 1295

Downloading the html of a page using perl

I am using the following code to get the html of a page.

 #!C:\Perl64\bin\perl.exe
    use strict;
    use warnings;
    use WWW::Mechanize;
    my $url=$ARGV[0];
    my $mech=new WWW::Mechanize();
    $mech->get($url,":content_file" => "C:\\path\\www\\page.html");

The above code works fine for certain sites but others like wikipedia require an agent_alias to be added. However whenever i add an agent_alias the page.html displays some nonsense.

$mech->agent_alias('Windows IE 6');

I have tried to add other aliases but it doesnt make a difference. What can i do to get the html(source) of all pages correctly.

Upvotes: 0

Views: 275

Answers (1)

Vytautas Vytas
Vytautas Vytas

Reputation: 131

I had better experience with libwww-perl:

require LWP::UserAgent;
$ua->agent('Mozilla/5.0');
$response = $ua->get('http://search.cpan.org/');

Upvotes: 1

Related Questions