CODEWITHSUNDEEP
pythongoogle-app-enginegoogle-cloud-datastorecsv
Niklas Rosencrantz
Niklas Rosencrantz

Reputation: 26637

Writing a file with properties from the datastore

I want a comma-separated list of all the email addresses my application stores. It's too large for a regular request and writing it to the blobstore is also a too large request:

class CSVHandler(webapp2.RequestHandler):
    def get(self):
        entities = Entity.all().fetch(10000)
        s = ''
        for entity in entities:
            s= s+","+str(entity.email)       
        file_name = files.blobstore.create(mime_type='application/octet-stream')     
        with files.open(file_name, 'a') as f:
          f.write(s)    
        files.finalize(file_name)    
        blob_key = files.blobstore.get_blob_key(file_name)

Can I do it as a task, a queue, a backend or something else instead?

Thank you in advance for any suggestion

Upvotes: 0

Views: 125

Answers (2)

Nick Johnson
Nick Johnson

Reputation: 101149

Instead of trying to save the entire file in a single write operation, write it to the blobstore in chunks. Using the csv module will make this easier, allowing you to write it progressively.

Upvotes: 1

Daniel Roseman
Daniel Roseman

Reputation: 599778

You should probably use the built-in bulkloader tool, which can download all objects via the remote API and save selected fields into CSV format.

Upvotes: 1

Related Questions