CODEWITHSUNDEEP
perlsessioncookieslwp
nikonan
nikonan

Reputation: 166

Create a session with necessary cookies using Perl

I will try to give as many details as possible.

Using Perl, I am trying to download the list of Impact Factors of all Journals for the JCR (Journal Citation Reports) from the Web of Knowledge (WoS) (url: http://admin-apps.webofknowledge.com/JCR/JCR?RQ=SELECT_ALL&cursor=1). It should be noted that the link is only accessible using IPs from most academic institutions, like mine, or from paying users.

Whenever I visit the link (using the browser) and sometime has passed, the session has expired so I need to click on a link (http://admin-router.webofknowledge.com/?DestApp=JCR) to establish a new session. Using my rudimentary knowledge of Firebug I found that when I am authenticated I get a cookie named jcrsid with a random string value.

My question is the following: Using Perl's LWP module, what steps should I take in order to establish a session and download the web pages I need. (If I use LWP to download the page, i download exactly the same page that appears when I am asked to establish a new session using the browser).

Upvotes: 0

Views: 427

Answers (1)

simbabque
simbabque

Reputation: 54371

You need to set a cookie_jar on your LWP::UserAgent object. LWP::UA will then store all the cookies it encounters in a HTTP::Cookies object and handle them for you. They'll be sent with every request.

You can use an empty hash reference {} to store your cookie in memory. There is no need to use a file for this kind of job.

use strict;
use warnings;
use LWP::UserAgent;

my $ua = LWP::UserAgent->new(
    cookie_jar => {},
);

That's it. Then you would to get the link that gives you the login cookie first.

my $res_login = $ua->get('http://admin-router.webofknowledge.com/?DestApp=JCR');
die $res_login->status_line unless $res_login->is_success;

Afterwards you can go about loading your pages. The cookie will be sent automatically.

Upvotes: 1

Related Questions