CODEWITHSUNDEEP
pythonjsongzipasp.net-web-api
static_rtti
static_rtti

Reputation: 56242

Python: what is the correct way to handle gzipped json?

I've found this snippet, which seems to do the job, but I can't understand why it uses StringIO. Isn't f already a file-like object? What is the need to read it, then make it look like a file again, only to read it again? I've tested it (well, a slightly modified version of it), and it doesn't work without StringIO.

Upvotes: 3

Views: 3171

Answers (3)

John Machin
John Machin

Reputation: 82924

It's possible that the gunzip code needs a file-like object that has a seek method, which a HTTP library is very unlikely to provide. What does "doesn't work" mean? Error message?

If efficiency is your real concern, slightly modify the code so that it uses cStringIO, not StringIO.

Upvotes: 1

log0
log0

Reputation: 10917

Seems to be a flaw in python standard library which is fixed in Python 3.2.
see http://www.enricozini.org/2011/cazzeggio/python-gzip/

urllib and urllib2 file objects do not provide a method tell() as requested by gzip.

Upvotes: 1

pvoosten
pvoosten

Reputation: 3287

The way I read the relevant part of the code says:

  • Open an url
  • Download it completely into memory (with the read method)
  • Store the content in a StringIO object, so that it's available as a file-like object
  • Do the gzip and json stuff with it.

Upvotes: 0

Related Questions