CODEWITHSUNDEEP
c++czlibgzip
towi
towi

Reputation: 22267

How to create a gz-compatible file with zlib?

I want to use the zlib to produce a gz-compatible output file with C++.

I installed the developer package for zlib, which can be used -- as I understand it -- to create gz-compatible files both on Unix and on Windows.

sudo aptitude install libz-dev

Although I write a C++-program, I quite I followed the usage example in the relevant points, I think. I also compiled the example to zpipe.c unchanged.

Alas, what I get is not a gz-compatible output.

$ ./zpipe.x < data.txt > x.gz
$ file x.gz
x.gz: data
$ gunzip x.gz 
gzip: x.gz: not in gzip format

I thought that the reason here might be, because deflateSetHeader is not called. So I added that into my own source code, i.e. (excerpt, you can find the full code here):

struct DeflateWrap { // RAII wrapper
  z_stream strm_ ; // C-Struct from zlib.h
  explicit DeflateWrap() : strm_{} {
    strm_.zalloc = Z_NULL;
    strm_.zfree = Z_NULL;
    strm_.opaque = Z_NULL;
    auto ret = deflateInit2(&strm_, LEVEL,
                 Z_DEFLATED, 15, 9, Z_DEFAULT_STRATEGY); 
    if(ret != Z_OK) throw std::runtime_error("Error ZLib-Init");
  }
  // ...more, eg. operator-> and *...
};

void pack(const string& infn) {
  DeflateWrap dwrap {};
  //...
  dwrap->avail_in = indata.size();
  dwrap->next_in = reinterpret_cast<unsigned char*>(indata.data());
  gz_header header {0}; // <<< HEADER HERE
  header.name = const_cast<unsigned char*>(
    reinterpret_cast<const unsigned char*>(infn.c_str()));
  header.comment = Z_NULL;
  header.extra = Z_NULL;
  bool first = true;
  do {
    dwrap->avail_out = outdata.size();
    dwrap->next_out = reinterpret_cast<unsigned char*>(outdata.data());
    if(first) {
      cerr << deflateSetHeader(&(dwrap.strm_), &header); // <<< SET HDR HERE
      first = false;
    }
    deflate(&(dwrap.strm_), Z_FINISH); // zlib.h: this packs
    auto toWrite = outdata.size() - dwrap->avail_out;
    outf.write(outdata.data(), toWrite);
  } while (dwrap->avail_out == 0);
}

To my interpretation I followed the manual for deflateSetHeader:

...and still I get a -2, i.e. Z_STREAM_ERROR from the deflateSetHeader call. Although, the output I produce can be uncompressed with zpipe.c, therefore it can't be totally wrong, can it?

Any idea how to set a gz-compatible header?

Update:

As I see it I use the C++-pendant to 

SET_BINARY_MODE(stdin);
SET_BINARY_MODE(stdout);

by opening the files like this:

ifstream inf{ infn, ifstream::binary };
ofstream outf { infn + ".gz", ofstream::binary };

Also, I wonder why the zpipe.c example I produced also does not make a gunzip-compatible file, as I described before. From what I read here it should.

Upvotes: 4

Views: 4122

Answers (2)

kevin.guo
kevin.guo

Reputation: 46

windowBits can also be –8..–15 for raw deflate. In this case, -windowBits determines the window size. deflate() will then generate raw deflate data with no zlib header or trailer, and will not compute an adler32 check value.

windowBits can also be greater than 15 for optional gzip encoding. Add 16 to windowBits to write a simple gzip header and trailer around the compressed data instead of a zlib wrapper. The gzip header will have no file name, no extra data, no comment, no modification time (set to zero), no header crc, and the operating system will be set to 255 (unknown). If a gzip stream is being written, strm->adler is a crc32 instead of an adler32.

Upvotes: 3

towi
towi

Reputation: 22267

Although I read the documentation of deflateSetHeader that the output file is gz-compatible, a bit further down there is a hint that it may be not so.

This library supports reading and writing files in gzip (.gz) format with an interface similar to that of stdio, using the functions that start with "gz". The gzip format is different from the zlib format. gzip is a gzip wrapper, documented in RFC 1952, wrapped around a deflate stream.

Thus, when I use the different set of functions gz... I get gz-compatible output and simpler code:

struct GzWrite { // RAII-Wrapper
    gzFile gz_ ; // C-Struct aus zlib.h
    explicit GzWrite(const string& filename)
        : gz_{gzopen(filename.c_str(),"wb9")}
    {
        if(gz_==NULL) throw std::runtime_error(strerror(errno));
    }
    ~GzWrite() {
        gzclose(gz_);
    }
    int write(const char* data, size_t len) {
        return gzwrite(gz_, data, len);
    }
    GzWrite(const GzWrite&) = delete; // keine Kopie
    GzWrite& operator=(const GzWrite&) = delete; // keine Zuweisung
};

void packe(const string& infn) {
    vector<char> indata = lese(infn); // lese Eingabe
    GzWrite gz{infn+".gz"}; // initialisiere Ausgabe
    auto res = gz.write(indata.data(), indata.size());
    if(res==0) throw std::runtime_error("Fehler beim Schreiben");
}

Upvotes: 3

Related Questions