CODEWITHSUNDEEP
rubycurlzlibgzipstream
Veeraa
Veeraa

Reputation: 29

Zlib decompression throws header error - Ruby

I need help on decompressing chunk data from a stream api. I am connecting to Gnip stream api which returns json output using gzip compression. When I try to decompress the output data, it throws me the following error "Zlib::DataError: incorrect header check".

It might be very close to this issue - http://groups.google.com/group/nodejs/browse_thread/thread/32b9ee7f691a68d9

Here I attached my code snippets for your reference:

require 'rubygems'
require 'curl'
require 'stringio'
require 'zlib'
url = "https://stream.gnip.com:443/accounts/SomeGroup/publishers/twitter/streams/track/Prod.json"
crl = Curl::Easy.new(url)
crl.headers={"Authorization"=>"Basic dmVlcmFzd5kYXJhdmVsLRoaX1Z25hbmFzd5kYhbU4ZXJeC5b26GpbFnW0MzIy", "Accept-Encoding" => "deflate, gzip"}
zstream = Zlib::Inflate.new
crl.on_body { |data| zstream.inflate(data);}
crl.http_get

The above code always returns "Zlib::DataError: incorrect header check". I know the gnip returns the data chunk by chunk so the required gzip'ed output will not be in the first chunk. So how can I collect all required chunk of gzip'ed outputs and decompress them to get required single json output.

Thanks in Advance. Veeraa.

Upvotes: 2

Views: 2614

Answers (1)

Mark Adler
Mark Adler

Reputation: 112374

By default zlib is looking for a zlib header, not a gzip header. So a gzip header would cause an incorrect header check. I don't know what the Ruby interface to zlib is like, but you should see if you can specify the type of stream to inflate. zlib's inflate supports zlib, gzip, and raw deflate streams. It also optionally provides an auto-detect of zlib and gzip streams.

Upvotes: 3

Related Questions