CODEWITHSUNDEEP
pythongoogle-app-engineapp-engine-ndb
TheJacobTaylor
TheJacobTaylor

Reputation: 4143

NDB query periodically not finding entity

I have an account key that I check when processing items. This key prevents me from spending time working on items that are no longer valid. On an inner loop in processing, I validate that I can still find the key that I am working on.

Entity definition:

class AuthorizedKey(ndb.Model):
    secret_key = ndb.StringProperty()
    ...additional fields...

Code I am using to check:

authorized_key = AuthorizedKey.query(AuthorizedKey.secret_key == first_key).get()

first_key is a string that matches an AuthorizedKey entity that has already been validated and is in the datastore. It has not been modified in weeks or months depending on the key.

This query periodically fails to retrieve a result. Roughly 1 in 10,000 times. Errors appear to be distributed among the keys (i.e. the problem is not with just one record).

Any suggestions on how to get rid of these errors? I need the information that is stored inside AuthorizedKey in order to accurately process the requests. If data is coming in that does not have a key, I want to stop processing it as quickly as possible.

Update --- The record that matches the query is typically created a few days before it is ever used. These records have been in place on average for months. They have worked millions of times, existed before, and continue to exist. Sometimes, the response to the get() call is None.

While multiple records have exhibited this problem, one particular record has been consistently accessed since about 2 days after its creation. I just added monitoring code on the inner loop a few days ago. There is no error message returned. 

"if not authorized_key:"

right after the code above returns False roughly once out of every 10,000 times. Accessing before and after works. An entity with a secret_key that matches my argument has existed for just shy of 8 months and has been successfully accessed millions of times. This is the reason for my question, to ask for help. I quietly fail in this case and automatically retry later. The retry works fine. Exactly the same data.

Aside from the odd nature of the issue, the main reasons that I worry about this are:

Upvotes: 0

Views: 307

Answers (2)

TheJacobTaylor
TheJacobTaylor

Reputation: 4143

It turns out that this internal processing code was called in two modes. The first mode pulls a batch and processes them. If there are too many items in the batch, it schedules a follow up task for later. When the follow up task is scheduled, it is in a namespace other than the default.

The second mode is as a follow up task, which carries forward the namespace from which it was queued. This means that I am now looking for the AuthorizedKey in the wrong namespace and not finding it. My batch size was rather high, but not astronomical. When one device is very active, or the system is heavily loaded, events can hit that mark, causing the error very sporadically.

Thank you all for your suggestions and debugging help!

Upvotes: 1

dragonx
dragonx

Reputation: 15143

You're using a non-ancestor query, so you may get eventually consistent results. If the entity is fairly new, then a query on it could fail to show it.

Upvotes: 2

Related Questions