hoju
Reputation: 29452
How to process large files on gae
I want my app to download an XML file and process it daily. The XML file comes in a ~100MB zip file - ~1GB when extracted. The processed output is less than 1MB.
I noticed the URL fetch (and urllib) response size quota is 32MB: http://code.google.com/appengine/docs/python/urlfetch/overview.html#Quotas_and_Limits
Is there a workaround?
Upvotes: 0
Views: 243
Answers (1)
Related Questions
- How to serve files larger than 32 MB from GCS using GAE and Python 3
- How to handle processing of large file on GAE?
- Processing a large (>32mb) xml file over appengine
- processing large cloud storage files in app engine
- GAE Python: Parsing compressed XML exceeds memory
- Serving large generated files using Google App Engine?
- Overcoming 1mb file restriction in GAE Datastore
- Importing a large amount of XML data into Google App Engine
- Google app engine parsing xml more then 1 mb
- App engine - Uploading a large file and parsing data from it onto the datastore