CODEWITHSUNDEEP
google-app-enginegoogle-app-engine-pythonurlfetch
Valentin Coudert
Valentin Coudert

Reputation: 1919

App engine urlfetch: how to get the full response with allow_truncate = true ?

I've faced ResponseTooLargeError when using AppEngine's urlfetch.fetch because my response was several Mb big. (doc)

I see there is a allow_truncate param I can pass and it will truncate the response if too big. Is there any way then to request the rest of the response ? Something like making new calls to the same URL with an offset ?

Otherwise I don't really understand how this param can be useful (just checking if it returns without error ?)

Thanks

Upvotes: 0

Views: 210

Answers (1)

Dan Cornilescu
Dan Cornilescu

Reputation: 39824

There is no standard way of requesting the "rest of the response", it all depends on the particular server-side implementation of the service you're working with: if it offers support for transferring data in smaller chunks, and, if it does, how exactly that works, i.e. the exact protocol for performing such transfers. Some may even consider such capability a part of the service itself.

Just to get an idea of what the implementation might look like you can see some possible options for a particular service discussed in Very large HTTP request vs many small requests

Yes, the usefulness of the option is to be able to get a partial response vs just getting a ResponseTooLargeError (which might be sufficient in some cases).

Upvotes: 1

Related Questions