CODEWITHSUNDEEP
phpcurl
Blank
Blank

Reputation: 201

How to use Curl with HEADERS?

I tried doing something like this but it doesn't work!

$ch = curl_init(); 
curl_setopt($ch, CURLOPT_URL, "http://google.com/"); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('GET /search?q=kk HTTP/1.1
Host: www.google.de
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Proxy-Connection: Close
Cookie: PREF=ID=2bb051bfbf00e95b:U=c0bb6046a0ce0334:
Cache-Control: max-age=0
Connection: Close
'));
$response = curl_exec($ch); 
curl_close($ch);   



    echo $response;

Also, Is it possible to make the entire request with just headers without setting the URL? I mean without this?

curl_setopt($ch, CURLOPT_URL, "http://google.com/");

Thanks!

Upvotes: 4

Views: 21234

Answers (5)

Mike Clark
Mike Clark

Reputation: 10136

I got it to work.

1) Change header Host: www.google.de to Host: www.google.com

Motivation: the host specified in the Host header should match the URL's host exactly.

2) Use "www.google.com" instead of "google.com"

Motivation: search requests made to google.com will not retrieve search results. You will be told to go to www.google.com.

3) Set the full URL into CURLOPT_URL, not just the hostname. E.g. change the CURLOPT_URL to curl_setopt($ch, CURLOPT_URL, "http://www.google.com/search?q=kk");

Motivation: correct usage of cURL API.

4) Remove GET /search?q=kk HTTP/1.1 from CURLOPT_HTTPHEADER -- it's misplaced.

Motivation: correct usage of cURL API.

5) The response will be gzip or deflate compressed. To stop this, remove the Accept-Encoding: gzip, deflate request header.

Motivation: if you tell Google you're capable of receiving a compressed response, they will send you one. Decompressing an HTTP response is an extra step that you may not want to undertake. It may be easier to deal with the response if it is in an uncompressed text form.

Upvotes: 5

fruchtose
fruchtose

Reputation: 1320

To add onto what the other posters have said, you also cannot stick a GET command in the CURLOPT_HTTPHEADER array, because that's specified in the other cURL options. cURL is meant to be operated using the curl_setopt function; you cannot bypass it by putting your HTTP message in the headers section. For instance, to ensure that your command is an HTTP GET operation, you set CURLOPT_HTTPGET to TRUE (although by default cURL will send a GET until you change it to something else).

To address your question of why you can't get to the right URL, that's because you need to specify the entire pathname in CURLOPT_URL, not just the host. So, you should really be writing curl_setopt($ch, CURLOPT_URL, "http://google.de/search?q=kk HTTP/1.1"); to set the URL.

Further, I have no idea why you put Connection: Close in he HTTP headers for a GET request. In that header, you're telling Google that you're closing the connection you have.; this is handled by curl_close($ch);, so forget about that header. In fact, half the items in your HTTP headers have no place being there. For instance, why are you sending a cookie in a request to get search results? Make sure you know what each header does before you send it out. Otherwise, you have absolutely no way to tell if you are sending the correct headers or not.

Upvotes: 4

David Souther
David Souther

Reputation: 8174

You've got a few problems, but they should be easy to sort out. First, you're setting the host in the header different than the host in the URL request, but since you're doing HTTP1.0, that isn't needed anyways.

Second, you need each line in the HTTPHEADER as its own thing in the array, and you don't include the GET line.

curl_setopt($ch, CURLOPT_HTTPHEADER, array(
   'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2',
    'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Language: en-gb,en;q=0.5',
    'Accept-Encoding: gzip, deflate',
    'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7',
    'Proxy-Connection: Close',
    'Cookie: PREF=ID=2bb051bfbf00e95b:U=c0bb6046a0ce0334:',
    'Cache-Control: max-age=0',
    'Connection: Close'
));

(You clearly stole this from Firefox, and an old version at that, but we'll let it slide.) Finally, yes, you must specify CURLOPT_URL, that's just the way the cURL API is designed.

Upvotes: 2

Mārtiņš Briedis
Mārtiņš Briedis

Reputation: 17752

You should read the fine manual more carefully next time. There is an example how to add the fields: array('Content-type: text/plain', 'Content-length: 100'), not everything in a one string, but fields are separate array elements.

Upvotes: 1

Another Code
Another Code

Reputation: 3151

If you need such a high level of control over the resulting HTTP request, I would recommend using raw socket functions to manually send the request instead. The manual even has an example of doing a HTTP request with fsockets:

$fp = fsockopen("www.example.com", 80, $errno, $errstr, 30);
if (!$fp) {
    echo "$errstr ($errno)<br />\n";
} else {
    $out = "GET / HTTP/1.1\r\n";
    $out .= "Host: www.example.com\r\n";
    $out .= "Connection: Close\r\n\r\n";
    fwrite($fp, $out);
    while (!feof($fp)) {
        echo fgets($fp, 128);
    }
    fclose($fp);
}

Upvotes: 1

Related Questions