abdul kalam
abdul kalam

Reputation: 11

Upsert Cosmos item TTL using Azure Data Factory Copy Activity

I have a requirement to upsert data from REST API to Cosmos DB and also maintain the item level TTL for particular time interval.

I have used ADF Copy activity to copy the data but for TTL, used additional custom column at source side with hardcoded value 30.

enter image description here

Noticed that time interval (seconds) updating as string instead of integer. Hence failing with the below error.

Details Failure happened on 'Sink' side. ErrorCode=UserErrorDocumentDBWriteError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Documents failed to import due to invalid documents which violate some of Cosmos DB constraints: 1) Document size shouldn't exceeds 2MB; 2) Document's 'id' property must be string if any, and must not include the following charaters: '/', '', '?', '#'; 3) Document's 'ttl' property must not be non-digital type if any.,Source=Microsoft.DataTransfer.DocumentDbManagement,'

ttl Mapping between Custom column to cosmos DB

enter image description here

When i use ttl1 instead of ttl, it is getting success and value stored as string.

enter image description here

Any suggestion please?

Upvotes: 0

Views: 600

Answers (1)

Utkarsh Pal
Utkarsh Pal

Reputation: 4544

Yes, that's the issue with additional columns in Copy activity. Even of you set it to int, it will change to string at the source.

The possible workaround is to create a Cosmos DB trigger in Azure function and add 'TTL' there.

Upvotes: 0

Related Questions