CODEWITHSUNDEEP
pythongoogle-app-enginegoogle-cloud-datastore
Shaun
Shaun

Reputation: 1601

Error when fetch Python NDB with repeated integer property

I have an app engine Python NDB Model that looks like this:

class Car(ndb.Model)
name=ndb.StringProperty()
tags=ndb.IntegerProperty(repeated=True)

when I go to fetch a Car by key I use:

car = ndb.Key('Car', long(6079586488025088)).get()

when I do this I am seeing:

 File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/key.py", line 532, in get
    return self.get_async(**ctx_options).get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 325, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 371, in _help_tasklet_along
    value = gen.send(val)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 689, in get
    pbs = entity._to_pb(set_key=False).SerializePartialToString()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3052, in _to_pb
    prop._serialize(self, pb, projection=self._projection)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1365, in _serialize
    values = self._get_base_value_unwrapped_as_list(entity)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1135, in _get_base_value_unwrapped_as_list
    wrapped = self._get_base_value(entity)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1123, in _get_base_value
    return self._apply_to_values(entity, self._opt_call_to_base_type)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1295, in _apply_to_values
    value[:] = map(function, value)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1177, in _opt_call_to_base_type
    value = _BaseValue(self._call_to_base_type(value))
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1198, in _call_to_base_type
    return call(value)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1274, in call
    newvalue = method(self, value)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1536, in _validate
    (value,))
BadValueError: Expected integer, got None

if I remove that property from the model definition it returns fine, so I know it's this property. In the datastore it is listed as having a null value for that field. Any idea why this is happening and how to deal with it? Thanks!

Upvotes: 0

Views: 669

Answers (1)

Patrick Costello
Patrick Costello

Reputation: 3626

This generally occurs when you first have a single, non-repeated property and then convert it to a repeated property. When you initially do a put(), if you have not yet set the property it will fill the value with None. However, if you then turn it into a repeated property, ndb will read this and think you want [None]. Because None is not a valid IntegerProperty, trying to serialize and put() the data will fail.

In your example it fails on a get() because after doing a get() from the datastore it tries to serialize the data and put it in memcache.

Depending on your situation, you have a couple of options:

  1. If you are only running in the devappserver, clear your datastore by running devappserver.py --clear_datastore

  2. Do a search for all objects with a None value and replace them with the empty list. This might look something like this:

    for c in Car.query(Car.tags=None):
  c.tags=[]
  c.put()

    Note that you have to be careful about a few things here. First, make sure that c.tags only is [None] and not [a, b, c, None], just in case. Second, if you have a lot of Cars with no tags, you won't be able to handle fixing them all in the same request. You'll either want to run on a backend, or pass the data on to Tasks for processing.

  3. This is pretty similar to #2, but if you have very little data you could use the Datastore viewer and simply resave the entities with tags = None.

Upvotes: 2

Related Questions