CODEWITHSUNDEEP
google-app-engineblobstoreapp-engine-ndb
Dylan
Dylan

Reputation: 2684

Is NDB model _post_delete_hook called after transaction? OR Best way to clean a blob from the blobstore when its referencing entity is deleted

I am implementing a file storage system where multiple people can share a file uploaded to the blobstore, but once they have all deleted the file, it is removed from the blobstore.

I am doing this using a reference counting entity, which is deleted when its count is set to 0. I am also using the BlobInfo ndb model defined in google.appengine.ext.ndb.blobstore as I need to get a bunch of BlobInfos at a few points and wanted to be able to do that asynchronously.

When the reference counting entity is deleted, its _post_delete_hook method is called, which I have calling blobstore.delete_async with the BlobKey and it is also calling BlobInfo._key.delete_async() on the ndb BlobInfo instance. 

class FileCounter(ndb.Model):
    count = ndb.IntegerProperty(default=1)
    @classmethod
    def _post_delete_hook(cls, key, future):
        blob_key = cls.blob_key_for_key(key)
        blobinfo_extension.delete(blobstore.BlobInfo.get(blob_key))

and blobinfo_extension.delete is defined as:

@ndb.tasklet
def delete_async(blobinfo):
""" Delete both the blob info and the blob in the blobstore it refers to. """
    yield blobstore.delete_async(blobinfo.key()), blobinfo._key.delete_async()

def delete(blobinfo):
    delete_async(blobinfo).wait()

I am getting the count entity, decrementing its count and then checking if it is 0. If it is 0 then it is deleted. This is all done in a transaction. Does the _post_delete_hook method get called outside of the transaction? As you can see, it is accessing the BlobInfo entity and deleting that which would require a cross group transaction.

I am going to try this and see if it works, but I posted it here due to not being able to find any answers, so it may help someone else in the future.

This this a good solution for this problem, or should I just clean up the blobstore/blobinfo 'manually' when the count is decremented? I think perhaps I am abusing the post delete hook!

Upvotes: 1

Views: 612

Answers (1)

Guido van Rossum
Guido van Rossum

Reputation: 16890

Maybe you can use ndb.get_context().call_on_commit(<callback>) to do some work when the transaction commits? You might combine this with a regular _post_delete_hook that just adds the entity for the post-commit cleanup step.

Upvotes: 2

Related Questions