CODEWITHSUNDEEP
azure-cosmosdb
Allen Zhang
Allen Zhang

Reputation: 2552

Cosmos db document automatically archive with TTL

Is there a way to automatically move expired documents to blob storage via change feed? I Google but found no solution to automatically move expired documents to blob storage via the change feed option. Is it possible?

Upvotes: 0

Views: 1410

Answers (3)

Joe Moon
Joe Moon

Reputation: 134

To come back to this question, one way I've found that works is to make use of the in-built TTL (Let CosmosDB expire documents) and to have a backup script that queries for documents that are near the TTL, but with a safe window in case of latency - e.g. I have the window up to 24 hours.

The main reasoning for this is that issuing deletes as a query not only uses RUs, but quite a lot of them. Even when you slim your indexes down you can still have massive RU usage, whereas letting Cosmos TTL the documents itself induces no extra RU use.

A pattern that I came up with to help is to have my backup script enable the container-level TTL when it starts (doing an initial backup first to ensure no data loss occurs instantly) and to have a try-except-finally, with the finally removing/turning off the TTL to stop it potentially removing data in case my script is down for longer than the window. I'm not yet sure of the performance hit that might occur on large containers when it has to update indexes, but in a logical sense this approach seems to be effective.

Upvotes: 0

Jay Gong
Jay Gong

Reputation: 23792

As statements in the Cosmos db TTL document: When you configure TTL, the system will automatically delete the expired items based on the TTL value, unlike a delete operation that is explicitly issued by the client application.

So,it is controlled by the cosmos db system,not client side.You could follow and vote up this feedback to push the progress of cosmos db.

Upvotes: 0

Nick Chapsas
Nick Chapsas

Reputation: 7200

There is not built in functionality for something like that and the change feed would be of no use in this case.

The change feed processor (which is what the Azure Function trigger is using too) won't notify you for deleted documents so you can't listen for them.

Your best bet is to write some custom application that does scheduling archiving and deleted the archived document.

Upvotes: 1

Related Questions