CODEWITHSUNDEEP
perllwp
naf
naf

Reputation: 95

need to rewrite this perl function . as libperl UserAgent is blocked

One of the hosts I deal with is blocking me from downloading images from their site. I have spoken to them and they have asked me to change the user-agent libwww-perl/ to Mozilla/5.0. The image links are http and https.

I have tried various options like 

my $ua = LWP::UserAgent->new();
$ua->ssl_opts( verify_hostnames => 0 );

But I can't get it to work with getstore. Would appreciate any help.

    sub storeimage {
  my $image = shift;
  if ($image =~ m#^https?://.+\/(.+\.)([a-z]+)$#i) {
    my $ext = $2;
    my $filename = "$1$2";
    if (exists $wantedfiles{$ext}) {
      my $savepath = $localwantedpath.$wantedfiles{$ext};
      if (!-f $savepath.$filename) {
        unless (is_success(getstore($image, $savepath.$filename))) {
          _warn("Couldn't download file $image to $savepath.");
          return '';
        }

    if ( $ext =~ /jpg|jpeg/oi ) {
      system("mogrify -resize '800>' -quality 70 $savepath$filename");

      #mogrify -resize 800x800  -quality 70 -format jpg $imageloc
    }

      }
      return $wantedfiles{$ext}.$filename;

    }
  }
  return '';
}

Upvotes: 2

Views: 155

Answers (3)

simbabque
simbabque

Reputation: 54381

LWP::Simple has a package variable with the user agent.

You can use that to change your agent string and still use getstore. 

use LWP::Simple;

$LWP::Simple::ua->agent("Mozilla...");
getstore($url, $file);

Upvotes: 4

melpomene
melpomene

Reputation: 85897

I'm not sure why you're messing around with SSL options. Hostname verification has nothing to do with HTTP headers.

What you need is something like

my $ua = LWP::UserAgent->new(agent => 'Mozilla/5.0');

to set the agent attribute.

To replicate the getstore function (from LWP::Simple) with your $ua object, you need to do something like this:

unless ($ua->request(HTTP::Request->new('GET' => $image), $savepath.$filename)->is_success) {
    ...
}

See the request method.

Or maybe consider using mirror:

$ua->mirror($your_url, $your_filename)

It behaves a bit differently, though.

Upvotes: 4

ikegami
ikegami

Reputation: 386706

my $ua = LWP::UserAgent->new( agent => 'Mozilla/5.0' );

Upvotes: 2

Related Questions