CODEWITHSUNDEEP
phpzlibdeflate
Havenard
Havenard

Reputation: 27894

gzdeflate() and large amount of data

I've been building a class to create ZIP files in PHP. An alternative to ZipArchive assuming it is not allowed in the server. Something to use with those free servers.

It is already sort of working, build the ZIP structures with PHP, and using gzdeflate() to generate the compressed data.

The problem is, gzdeflate() requires me to load the whole file in the memory, and I want the class to work limitated to 32MB of memory. Currently it is storing files bigger than 16MB with no compression at all.

I imagine I should make it compress data in blocks, 16MB by 16MB, but I don't know how to concatenate the result of two gzdeflate().

I've been testing it and it seems like it requires some math in the last 16-bits, sort of buff->last16bits = (buff->last16bits & newblock->first16bits) | 0xfffe, it works, but not for all samples...

Question: How to concatenate two DEFLATEd streams without decompressing it?

Upvotes: 3

Views: 2459

Answers (2)

Naveed
Naveed

Reputation: 1221

PHP stream filters are used to perform such tasks. stream_filter_append can be used while reading from or writing to streams. For example

    $fp = fopen($path, 'r');
    stream_filter_append($fp, 'zlib.deflate', STREAM_FILTER_READ);

Now fread will return you deflated data.

Upvotes: 1

Dooltaz
Dooltaz

Reputation: 2463

This may or may not help. It looks like gzwrite will allow you to write files without having them completely loaded in memory. This example from the PHP Manual page shows how you can compress a file using gzwrite and fopen.

http://us.php.net/manual/en/function.gzwrite.php

function gzcompressfile($source,$level=false){
    // $dest=$source.'.gz';
    $dest='php://stdout'; // This will stream the compressed data directly to the screen.
    $mode='wb'.$level;
    $error=false;
    if($fp_out=gzopen($dest,$mode)){
        if($fp_in=fopen($source,'rb')){
            while(!feof($fp_in))
                gzwrite($fp_out,fread($fp_in,1024*512));
            fclose($fp_in);
            }
          else $error=true;
        gzclose($fp_out);
        }
      else $error=true;
    if($error) return false;
      else return $dest;
}

Upvotes: 0

Related Questions