CODEWITHSUNDEEP
azureazure-storageazure-blob-storage
mickkk
mickkk

Reputation: 1192

Why IoT Hub message is not decoded corretly in storage JSON blob?

I am sending a string with an Azure Sphere dev kit using the provided function:

AzureIoT_SendMessage("Hello from sample App")

The message is sent to an IoT Hub and then routed to a storage blob with JSON encoding. If I look at the blob storage I get the following:

{"EnqueuedTimeUtc":"2019-05-22T12:33:42.2320000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"fbea*****************6d**********************9c0","connectionAuthMethod":"{\"scope\":\"device\",\"type\":\"x509Certificate\",\"issuer\":\"external\",\"acceptingIpFilterRule\":null}","connectionDeviceGenerationId":"63************22","enqueuedTime":"2019-05-22T12:33:42.2320000Z"},"Body":"SGVsbG8gZnJvbSBzYW1wbGUgQXBw"}

The field "body" does not show at all the string sent ("Hello from sample App") but it shows "SGVsbG8gZnJvbSBzYW1wbGUgQXBw". Why is this happening? And how can I fix it?

I found that if I format the storage as AVRO (instead of JSON) the string is rendered correctly however the message becomes (literally) a blob and it cannot be used in streaming service such as powerBI (for example). However the message can be found, with some other mess stuff, in the blob (see picture below with the default string message)

enter image description here

Upvotes: 1

Views: 2407

Answers (3)

BurkhardSchaefer
BurkhardSchaefer

Reputation: 17

See also pending feature request for IoT-Hub: feature request

Upvotes: -1

connor-ww
connor-ww

Reputation: 106

See Microsoft's IoT Hub message routing documentation - specifically the Azure Storage section. It says "When using JSON encoding, you must set the contentType to application/json and contentEncoding to UTF-8 in the message system properties. Both of these values are case-insensitive. If the content encoding is not set, then IoT Hub will write the messages in base 64 encoded format."

This blog post further expands on the topic explaining that content type and encoding need to be set as specific headers.

On setting the headers:

If you are using the Azure IoT Device SDKs, it is pretty straightforward to set the message headers to the required properties. If you are using a third-party protocol library, you can use this table to see how the headers manifest in each of the protocols that IoT Hub supports​:

Content headers table

Upvotes: 2

ExoV1
ExoV1

Reputation: 109

FWiW, using PowerQuery (PQ) within PowerBi, I was able to decode the {body:xxxencoded_base64...} portion of the JSON file sent to blob by Azure IoT Hub.

My steps in PowerBI: - Connect to blob account, container collecting the JSON files. - Within PQ, click the double-arrow expand on the initial binary column - For me, Column10 was the {body:xxx} column. Using PQ's Replace Values function, I removed the prefix "{body:" and the final "} - leaving just the encoded string. - Create a new column in PQ, using this M-code: =Binary.FromText([Column10],BinaryEncoding.Base64) - It's now a new Binary column, click the double-arrow and expand the binary. It'll reveal the decoded JSON table, complete w/ all your IoT telemetry.
HTHs

Upvotes: 1

Related Questions