CODEWITHSUNDEEP
curlwget
Cesare
Cesare

Reputation: 1749

Getting a file using curl or wget from command line doesn't work (in php it's working ....)

I'm on Ubuntu 15 ....

I need to download this open data CSV http://www.sviluppoeconomico.gov.it/images/exportCSV/prezzo_alle_8.csv

from this page

http://www.sviluppoeconomico.gov.it/index.php/it/open-data/elenco-dataset/2032336-carburanti-prezzi-praticati-e-anagrafica-degli-impianti.

I'd like to use a simply wget or curl command line but if I try for example

curl http://www.sviluppoeconomico.gov.it/images/exportCSV/prezzo_alle_8.csv pippo.csv

the result is 

<html><head><title>Richiesta Rifiutata</title></head><body>La URL Richiesta e' stata riufiuta. Contattare l'amministratore di sistema.<br><br>The requested URL was rejected. Please consult with your administrator.<br><br></body></html>    
<html>
 <head><title>302 Found</title></head>
  <body bgcolor="white">
   <center><h1>302 Found</h1></center>
   <hr><center>nginx/1.4.6 (Ubuntu)</center>
  </body>
</html>

the same result if I use 

wget http://www.sviluppoeconomico.gov.it/images/exportCSV/prezzo_alle_8.csv

I've tried to use a simply php progam

<?php
 $url = 'http://www.sviluppoeconomico.gov.it/images/exportCSV/prezzo_alle_8.csv';

 print $url;
 echo '<br>';
 echo '<br>';

 //#Set CURL parameters: pay attention to the PROXY config !!!!
 $ch = curl_init();
 curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
 curl_setopt($ch, CURLOPT_HEADER, 0);
 curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
 curl_setopt($ch, CURLOPT_URL, $url);
 curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
 $data = curl_exec($ch);
 curl_close($ch);

 echo $data;
?>

and in this case the data of the CSV file is printed on my browser page (i've to wait for it but it's printed ....).

So, I think that should be possible to download the data using curl or wget from command line and probably there are some parameters that I should set but I don't have the solution now ...

Any suggestions / examples?

Thank you in advance!!

Cesare

Upvotes: 0

Views: 1137

Answers (1)

Sabuj Hassan
Sabuj Hassan

Reputation: 39355

You need to use -L switch with your curl command as you are using curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); from your php code to follow the redirection.

More information about this -L switch can be found here: http://curl.haxx.se/docs/manpage.html#-L

Also override the default curl user agent string (i.e. User-Agent: curl/7.40.0) with following option

 -A "Opera"

For wget command add the following switch for useragent string.

-U "Opera"

Upvotes: 2

Related Questions