CODEWITHSUNDEEP
sql-serverazuret-sqlazure-eventhubazure-stream-analytics
saranraj kumar
saranraj kumar

Reputation: 217

How insert only distinct rows based on data in output/blob storage using azure stream analytics?

I'm trying to store only distinct rows in my output- Blob storage based on old data in that output-blob storage using azure stream analytics query language. Eg: On the first insert, I get some values to the streaming analytics input as below

Name, Score
saran, 100
raj,   90

and the above data is stored in blob as output.

On the second insert, I get the following table from stream analytics input

Name, Score
saran, 90
jeeva, 80

Now before storing this data to the blob, I have to compare the old data from a blob with new data and to insert only the distinct data to the blob and also I wanna update the value for already existing data. Thanks in Advance

Upvotes: 0

Views: 454

Answers (1)

Jay Gong
Jay Gong

Reputation: 23792

Your main requirement is implementing Upsert feature in the ASA blob storage output,and as i know , you can't get that feature in blob storage output.

However,you could consider loading the data into below output before blob storage(final destination) as workarounds:

1.Azure Cosmos DB SQL API:

Stream Analytics integration with Azure Cosmos DB allows you to insert or update records in your container based on a given Document ID column.More details,please refer to this official document.

2.SQL Database:

You could use stored procedure in the sql database to implement upsert,please refer to this thread:How to implement a conditional Upsert stored procedure?

After above process, you could load data into blob storage.For example,using copy activity in ADF.

Upvotes: 1

Related Questions