CODEWITHSUNDEEP
phpapicurl
Miguel
Miguel

Reputation: 1897

File_get_contents, curl not working

Something strange is going on, and I would like to know why.

On this url: http://api.promasters.net.br/cotacao/v1/valores?moedas=USD&alt=json, which works well in the browser, but when I tried to retrieve the content with php:

echo file_get_contents('http://api.promasters.net.br/cotacao/v1/valores?moedas=USD&alt=json');

printed nothing, with var_dump(...) = string(0) "", so i went a little further and used:

function get_page($url) {
    $curl = curl_init();
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, True);
    curl_setopt($curl, CURLOPT_URL, $url);
    $return = curl_exec($curl);
    curl_close($curl);
    return $return;
}

echo get_page('http://api.promasters.net.br/cotacao/v1/valores?moedas=USD&alt=json');

Also printed nothing, so i tried python (3.X):

import requests
print(requests.get('http://api.promasters.net.br/cotacao/v1/valores?moedas=USD&alt=json').text)

And WORKED. Why is this happening? What's going on?

Upvotes: 5

Views: 3694

Answers (2)

Progrock
Progrock

Reputation: 7485

I experienced the same behaviour.

Fetching the URL using the CLI Curl worked for me.

I then wrote a script with a file_get_contents call to another script that dumped all request headers to a file using getallheaders:

<?php
file_put_contents('/tmp/request_headers.txt', var_export(getallheaders(),true));

Output of file:

array (
  'Host' => 'localhost',
)

I then inspected the curl request headers,

$ curl -v URL

And tried adding one at a time to the file_get_contents request. It turned out a User agent header was needed.

<?php
$opts = array(
    'http'=>array(
    'method'=>"GET",
    'header'=>
      "User-Agent: examplebot\r\n"
    )
);
$context  = stream_context_create($opts);
$response = file_get_contents($url, false , $context);

This gave me a useful response.

Upvotes: 2

Jeff Puckett
Jeff Puckett

Reputation: 41021

It looks like they're blocking the user agent, or lack thereof, considering that php curl and file_get_contents doesn't seem to set the value in the request header.

You can fake this by setting it to something like Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0.1) Gecko/20100101 Firefox/7.0.1

<?php
function get_page($url) {
    $curl = curl_init();
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, True);
    curl_setopt($curl, CURLOPT_URL, $url);
    curl_setopt($curl,CURLOPT_USERAGENT,'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0.1) Gecko/20100101 Firefox/7.0.1');
    $return = curl_exec($curl);
    curl_close($curl);
    return $return;
}

echo get_page('http://api.promasters.net.br/cotacao/v1/valores?moedas=USD&alt=json');

Upvotes: 3

Related Questions