CODEWITHSUNDEEP
pythonpython-3.xpython-requestsgzip
AdrianBR
AdrianBR

Reputation: 2588

Unzip downloaded gzipped content on the fly

I am downloading a gzipped CSV with python, and I would like to write it to disk diretly as csv.

I tried several variations of the following:

url ='blabla.s3....csv.gz'
filename = 'my.csv'

compressed = requests.get(url).content
data = gzip.GzipFile(fileobj=compressed)
with open(filename, 'wb') as out_file:
    out_file.write(data)

but I am getting various errors - I am not sure I am passing the right part of the response to the gzip method. If anyone has experience with this, input is appreciated.

Upvotes: 1

Views: 868

Answers (1)

cs95
cs95

Reputation: 402363

You should be able to use zlib to decompress the response.

import zlib

res = requests.get(url)
data = zlib.decompress(res.content, zlib.MAX_WBITS|32)

Now, write to a file:

with open(filename, 'wb') as f:
    f.write(data)

Upvotes: 2

Related Questions