Is it possible to send large JSON data to IoT Hub?

Question

I'm asking this question behalf of a hardware developer (i don't have much knowledge in device side).

Right now IoT hub receiving data from the device every 2 minutes, due to some reasons planning to store the data locally with the device and send that in every 30 minutes. But the problem is, data is not getting saved to storage. 2 minute's data is getting saved to storage. But the 30 minute's data is not getting properly saved to storage. Only the first object in the array is getting saved.

I'm using Stream analytics to process the data from IoT hub and storing the data to table storage.

Current message Json data (every 2 minute) is :

{
    "deviceid": "testdevice12",
    "BatteryVoltage": "31.31",
    "PanelVoltage": "0.00",
    "PanelCurrent": "0.00",
    "OutputVoltage": "31.29",
    "OutputCurrent": "4.94",
    "RSSI": "13",
    "FreeHeapSize": "86.884",
    "timestamp": 1540470493
}

For 30 minutes it will be like the following:

[{
    "deviceid": "testdevice12",
    "BatteryVoltage": "31.31",
    "PanelVoltage": "0.00",
    "PanelCurrent": "0.00",
    "OutputVoltage": "31.29",
    "OutputCurrent": "4.94",
    "RSSI": "13",
    "FreeHeapSize": "86.884",
    "timestamp": 1540470493
}, {
    "deviceid": "testdevice12",
    "BatteryVoltage": "31.29",
    "PanelVoltage": "0.00",
    "PanelCurrent": "0.00",
    "OutputVoltage": "31.28",
    "OutputCurrent": "4.93",
    "RSSI": "0",
    "FreeHeapSize": "86.884",
    "timestamp": 1540470558
}, {
    "deviceid": "testdevice12",
    "BatteryVoltage": "31.30",
    "PanelVoltage": "0.00",
    "PanelCurrent": "0.00",
    "OutputVoltage": "31.29",
    "OutputCurrent": "4.94",
    "RSSI": "5",
    "FreeHeapSize": "86.884",
    "timestamp": 1540470621
}, {
    "deviceid": "testdevice12",
    "BatteryVoltage": "31.31",
    "PanelVoltage": "0.00",
    "PanelCurrent": "0.00",
    "OutputVoltage": "31.30",
    "OutputCurrent": "4.94",
    "RSSI": "5",
    "FreeHeapSize": "86.884",
    "timestamp": 1540470684
}, {
    "deviceid": "testdevice12",
    "BatteryVoltage": "31.30",
    "PanelVoltage": "0.00",
    "PanelCurrent": "0.00",
    "OutputVoltage": "31.28",
    "OutputCurrent": "4.94",
    "RSSI": "7",
    "FreeHeapSize": "86.884",
    "timestamp": 1540470760
}]

Here is the ASA query i'm Using:

WITH [StreamData] AS ( SELECT * FROM [IoTHubStream] WHERE [ObjectType] IS NULL -- Filter out device info and command responses ) 
SELECT IoTHub.ConnectionDeviceId AS DeviceId, PanelVoltage, PanelCurrent, BatteryVoltage, BatteryCurrent, OutputVoltage, OutputCurrent, Temperature, Humidity, EventProcessedUtcTime, PartitionId, EventEnqueuedUtcTime, * INTO [DeviceDataHistory] FROM [StreamData]

Answer 1

Got it -finally. There are few points: 1. JSON payload with cached data structure should have struture like below:

{
 "topic": "cachedData",
  "deviceId": "testdevice12",
  "data":
[{
"BatteryVoltage": "31.31",
"PanelVoltage": "0.00",
"PanelCurrent": "0.00",
"OutputVoltage": "31.29",
"OutputCurrent": "4.94",
"RSSI": "13",
"FreeHeapSize": "86.884",
"timestamp": 1540470493
}, {
"BatteryVoltage": "31.29",
"PanelVoltage": "0.00",
"PanelCurrent": "0.00",
"OutputVoltage": "31.28",
"OutputCurrent": "4.93",
"RSSI": "0",
"FreeHeapSize": "86.884",
"timestamp": 1540470558
}, {
"BatteryVoltage": "31.30",
"PanelVoltage": "0.00",
"PanelCurrent": "0.00",
"OutputVoltage": "31.29",
"OutputCurrent": "4.94",
"RSSI": "5",
"FreeHeapSize": "86.884",
"timestamp": 1540470621
}, {
"BatteryVoltage": "31.31",
"PanelVoltage": "0.00",
"PanelCurrent": "0.00",
"OutputVoltage": "31.30",
"OutputCurrent": "4.94",
"RSSI": "5",
"FreeHeapSize": "86.884",
"timestamp": 1540470684
}, {
"BatteryVoltage": "31.30",
"PanelVoltage": "0.00",
"PanelCurrent": "0.00",
"OutputVoltage": "31.28",
"OutputCurrent": "4.94",
"RSSI": "7",
"FreeHeapSize": "86.884",
"timestamp": 1540470760
}]
}

2. Azure Stream Analytics should look like below:

    SELECT   
     iothubAlias.deviceId,
     arrayElement.ArrayValue.BatteryVoltage,
     arrayElement.ArrayValue.PanelVoltage,
     arrayElement.ArrayValue.PanelCurrent,
     arrayElement.ArrayValue.OutputVoltage,
     arrayElement.ArrayValue.OutputCurrent,
     arrayElement.ArrayValue.RSSI,
     arrayElement.ArrayValue.FreeHeapSize,
     arrayElement.ArrayValue.timestamp
   INTO [DeviceDataHistory]
   FROM [IoTHubStream] as iothubAlias  
   CROSS APPLY GetArrayElements(iothubAlias.data) AS arrayElement
   

To retrieve multiple rows from the input you have to use "GetArrayElements" - you can read more here:

https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-parsing-json

https://msdn.microsoft.com/en-US/azure/stream-analytics/reference/getarrayelements-azure-stream-analytics

3. Remember to set PartitionKey and RowKey for the Azure Table set as output for the Azure Stream Analytics:

And final result look like below:

Try and let me know if it helped.