CODEWITHSUNDEEP
pythongoogle-app-engine
Chris McKee
Chris McKee

Reputation: 4387

Appengine Python Bulk Export error

This seems to run for ~42,200 records then fails:

import datetime
import time

from google.appengine.ext import db
from google.appengine.tools import bulkloader
from google.appengine.api import datastore_types

class SearchRec(db.Model):
  WebSite = db.StringProperty()
  WebPage = db.StringProperty()
  DateStamp = db.DateTimeProperty(auto_now_add=True)
  IP = db.StringProperty()
  UserAgent = db.StringProperty()


class TrackerExporter(bulkloader.Exporter):
    def __init__(self):
        bulkloader.Exporter.__init__(self, 'SearchRec',
                                    [('WebSite', str, None),
                                    ('WebPage', str, None),
                                    ('DateStamp', lambda x: str(datetime.datetime.strptime(x, '%d/%m/%Y').date()), None),
                                    ('IP', str, None)
                                    ])

exporters = [TrackerExporter]

if __name__ == '__main__':
    bulkload.main(TrackerExporter)

Error:

File "tracker-export.py", line 89, in <lambda>
        ('DateStamp', lambda x: str(datetime.datetime.strptime(x, '%d/%m/%Y').date()
    ), None),

TypeError: strptime() argument 1 must be string, not datetime.datetime

Upvotes: 0

Views: 447

Answers (1)

Will McCutchen
Will McCutchen

Reputation: 13117

I'm not sure why exactly that would be happening. I'm not too familiar with the Bulk Exporting facilities of App Engine, but it sounds like the DateStamp field is being given to the bulk exporter as a string (which is what your converter expects) for the first 42200 records and then, for some reason, it is given as a real datetime.datetime object.

Anyway, here's a treatment for this particular symptom:

lambda x: str((x if isinstance(x, datetime.datetime) else datetime.datetime.strptime(x, '%d/%m/%Y')).date())

That should do the right thing if given either a string or a datetime.datetime object. This might be a case where you should give it a dedicated function instead of torturing a lambda like that.

Upvotes: 1

Related Questions