CODEWITHSUNDEEP
phpfilecopyphp-stream-wrappers
illinois
illinois

Reputation: 508

File copied from a URL is truncated

I have a problem copying a file. My code:

$file = "https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml";

$newfile = $_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';

if(copy($file, $newfile)) {
    echo "salvato<br>";
} else {
    echo "ERROR inport file PO_Offerte_".$data.".".$ext."<br>";
    die;
}

copy() give true, the file is created but some lines at the end of the file are missing... the file is 3.6MB and 0.3 at the end of the file are missing...

If I download the file manually all is fine so the source is complete...

I actually have the same problem if I get the file content with file_get_contents() and I try to save it in a file using file write function...

I do not think upload_max_filesize and post_max_size actually are involved in copy() but they are 20MB setted

any tip?

thanks

Upvotes: 2

Views: 724

Answers (8)

Professor Abronsius
Professor Abronsius

Reputation: 33813

As the url is SSL enabled typcially additional information needs to be packaged and sent with the request - for the copy function there is a context argument. The context allows you to specify method,protocol and much more to support the request.

$url='https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';

/* download a copy from: https://curl.haxx.se/ca/cacert.pem */
$cacert=__DIR__ . DIRECTORY_SEPARATOR . 'cacert.pem';
/* define where files are to be stored */
$dir='c:/temp/downloads/';

Using the stream_context_create() function to build the context for the request and copy to actually make the request.

$filepath=$dir . basename( $url );

$args = array(
    'http'  =>  array( 'method' => 'GET', 'protocol_version' => '1.1' ),
    'ssl'   =>  array( 'verify_peer' => true, 'verify_peer_name' => true, 'allow_self_signed' => false, 'cafile' => $cacert )
);
$ctxt = stream_context_create( $args );
$status = copy( $url, $filepath, $ctxt );

if( $status && file_exists( $filepath ) ){
    printf( 
        'The file "%s" downloaded successfully. %sMb written to disk.',
        $filepath,
        round( filesize( $filepath ) / pow( 1024, 2 ),2 )
    );
}

Another, and perhaps better option, is curl:

function downloadfile( $url=false, $dir=false, $cacert=false ){
    if( $url && $dir ){

        /* define the save path */
        $filepath = $dir . basename( $url );

        /* time how long the download takes */
        $start=time();

        /* open a file pointer for use by curl */
        $fp = fopen( $filepath, 'w+' );

        /* create the curl request - write file directly */
        $ch = curl_init( $url );
        curl_setopt($ch, CURLOPT_HEADER, 0 );
        curl_setopt($ch, CURLOPT_TIMEOUT, 10 );
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
        curl_setopt($ch, CURLOPT_BINARYTRANSFER, true );
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true );
        curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36' );
        if( parse_url( $url, PHP_URL_SCHEME )=='https' ){
            curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true );
            curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2 );
            curl_setopt($ch, CURLOPT_CAINFO, $cacert );
        }
        curl_setopt($ch, CURLOPT_ENCODING, '' );
        curl_setopt($ch, CURLOPT_FILE, $fp );# write to file

        /* the response */
        $obj=(object)array(
            'response'  =>  curl_exec($ch),
            'info'      =>  (object)curl_getinfo($ch),
            'error'     =>  curl_error($ch),
            'filepath'  =>  $filepath
        );

        /* tidy up */
        curl_close($ch);
        fclose($fp);

        /* calculate time operation took */
        $obj->duration=round( time() - $start, 2 );

        return $obj;
    }
}



/* run the function */
$obj = downloadfile( $url, $dir, $cacert );

if( $obj->info->http_code==200 ){
    printf( 
        'The file "%s" downloaded successfully in approximately %ss. %sMb written to disk.',
        $obj->filepath,
        $obj->duration,
        round( filesize( $obj->filepath ) / pow( 1024, 2 ),2 )
    );
} else {
    printf(
        'Error: A problem was encountered downloading %s. The response code is: %d and error message: "%s"',
        $url,
        $obj->info->http_code,
        $obj->error
    );
}

The above two methods and also a manual download resulted in a file of 3.19Mb

Upvotes: 0

Mohamad Nowruzi
Mohamad Nowruzi

Reputation: 56

The problem is from Http 1.0

This should fix it:

$sc = stream_context_create(['http' => ['protocol_version' => '1.1']]);

copy($file, $newfile, $sc);

Upvotes: 0

Salman Arshad
Salman Arshad

Reputation: 272106

I was able to use file_get_contents and forcing HTTP/1.1 protocol:

$context = stream_context_create([
    'http' => [
      'protocol_version' => '1.1',
      'header' => 'Connection: Close'
    ],
]);
$content = file_get_contents('https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml', false, $context);
file_put_contents('document.xml', $content);

Having said that, I would recommend using CURL:

$ch = curl_init();
$curlopts = array();
$curlopts[CURLOPT_RETURNTRANSFER] = true;
$curlopts[CURLOPT_VERBOSE] = true;
$curlopts[CURLOPT_URL] = 'https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';
curl_setopt_array($ch, $curlopts);
$content = curl_exec($ch);
file_put_contents('document.xml', $content);
curl_close($ch);

Upvotes: 7

alxndrbauer
alxndrbauer

Reputation: 1

Looks like memory_limit. Try to set it to a higher value at the beginning of your script.

ini_set('memory_limit' '1024m')

Upvotes: 0

Millar248
Millar248

Reputation: 401

http://php.net/manual/en/function.copy.php

I see this: exec("xcopy $source $destination");

Try that!

Upvotes: -1

Ajay Katariya
Ajay Katariya

Reputation: 459

Plea try this connection timeout function,Hope this thing will work for you.

$url=curl_init("https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml"); 
$set_timeout=400; 
$newfile=$_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';
curl_setopt($url, CURLOPT_CONNECTTIMEOUT, $set_timeout); 
$content = curl_exec($url);//execute request
if($content)
{
    $copied_file = fopen($newfile, "w"); 
    if(fwrite($copied_file , $content)){
        echo "Done successfully";
    }
    else{
        echo "unable to write file";
        fclose($copied_file ); 
    }
}
else {
    echo "Something Went wrong";
}

Upvotes: 1

Maksym Fedorov
Maksym Fedorov

Reputation: 6456

You have a problem because you use HTTPS protocol. copy() and file_get_content() functions have some problems with working by HTTPS. The more reliable way to load remote file is using of CURL. For example:

$url = 'https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';    
$filePath = $_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';

$ch = curl_init($url);
$fp = fopen($filePath, 'wb');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($fp);

Upvotes: 0

Andrea Rossi
Andrea Rossi

Reputation: 9

Take a look at your php.ini. Probably you have to increase these two parameters

upload_max_filesize and post_max_size

Upvotes: -1

Related Questions