CODEWITHSUNDEEP
azureazure-stream-analytics
Tikke
Tikke

Reputation: 139

Concatenating IoT messages' data by Azure Stream Analytics

An IoT sensor takes 1000 measurements @ 1kHZ in every 10min and sends the values in ten separate messages into Azure IoT Hub. I am supposed to concatenate the ten separate messages back to one for further processing e.g. calculating RMS and FFT.

The messages have the following structure:

{
    "SampleID" : 12344,
    "PartionIdx" : 2,
    "NbrPartitions": 10,
    "Values" : [12,13,14,13,12,11,10,9]
}

So, the values of all messages having same SampleID should be concatenated together by PartitionIdx order, after all ten have been received. I tried to use Stream Analytics but failed.

Is this too complex task for Stream Analytics? If yes, is there any other options than just coding a Web Job doing the concatenation.

Upvotes: 1

Views: 103

Answers (1)

Vignesh Chandramohan
Vignesh Chandramohan

Reputation: 1306

There are two aspects to the question

  1. Knowing when all the values have arrived.
  2. Concatenating them together.

For 1, If the events for a particular device Id arrives in order to eventhub and to the same eventhub partition always or if you have an idea about how out of order the data can be, you can use TIMESTAMP BY OVER() clause to create a different timeline for every device. This will block output for a deviceId until enough data from that partition has been received.

For 2, you can either use Collect() like @js-azure mentioned. If you want the data to be formatted in a specific instead of array of records, you can use javascript aggregates.

Upvotes: 1

Related Questions