CODEWITHSUNDEEP
perlhttp-headershttprequesthttpresponseuser-agent
Ωmega
Ωmega

Reputation: 43703

How to get full HTTP request (not response) headers

I have a simple code like this:

use LWP::UserAgent;
use HTTP::Cookies;
use HTTP::Request;

my $cookies = HTTP::Cookies->new();
my $browser = LWP::UserAgent->new();
   $browser->agent(' ... ');    
   $browser->cookie_jar($cookies);   
my $request = HTTP::Request->new();     
my $response;                           
my $url;                                
my $referer;                                        

$referer = '';
$url = 'https:// ...'; # url #1
$request->url($url);
$request->method('GET');
$request->header('Referer' => $referer);    
$response = $browser->request($request);    
print $response->request()->uri()     . "\n\n" .
      $response->headers()->as_string . "\n\n" .
      $response->content              . "\n\n";

$referer = $response->request()->uri();
$url = 'https:// ... '; # url #2
$request->url($url);
$request->method('GET');
$request->header('Referer' => $referer);
$response = $browser->request($request);
print $response->request()->uri()     . "\n\n" .
      $response->headers()->as_string . "\n\n" .
      $response->content              . "\n\n";

Now, I want to see full HTTP request headers as well, not just response headers.

How can I do it? What has to be added to this code?

Upvotes: 1

Views: 2357

Answers (4)

ikegami
ikegami

Reputation: 386696

$request->headers->as_string and $response->request->headers->as_string will you get you the headers of the first and last request passed to Net::HTTP by LWP[1], but these aren't quite what Net::HTTP sends. For example, Net::HTTP can add a Content-Length header, a TE header, and/or a number of others.

Net::HTTP doesn't keep a record of the headers it actually sends. You will need a wire sniffer (e.g. tcpdump) or a debugging proxy (e.g. Fiddler) for that. You could also use a debugger or trace statements to view the request prepared in Net::HTTP::Methods's format_request. The most convenient, however, might be to wrap Net::HTTP::Methods's format_request.

  1. These are the same unless the initial request was redirected. To get all the requests (and responses), you can use:

    while ($response) {
   my $request = $response->request;
   ...

   $response = $response->previous;
}

Upvotes: 0

oalders
oalders

Reputation: 5279

This will show you requests as well as responses.

use LWP::UserAgent;
use LWP::ConsoleLogger::Easy qw( debug_ua );

my $browser = LWP::UserAgent->new();

debug_ua( $browser );

Upvotes: 1

Borodin
Borodin

Reputation: 126772

print $response->request->as_string

Upvotes: 1

UncleCarl
UncleCarl

Reputation: 330

I think you almost have it in your existing code. You are accessing the request URI with $response->request()->uri(). The ->request() is your HTTP::Request object. I believe that you can use $response->request->headers->as_string to get what you want.

Upvotes: 3

Related Questions