CODEWITHSUNDEEP
azureazure-eventhub
Chris Gillum
Chris Gillum

Reputation: 15042

How are Azure Event Hub event properties serialized?

I want to produce and send EventData objects that have various application-specific properties set in the EventData.Properties dictionary. The Properties dictionary itself is defined as IDictionary<string, object> which means I can pass any data type as the value.

var eventData = new EventData(bytes);
eventData.Properties["Prop1"] = // string?
eventData.Properties["Prop2"] = // int?
eventData.Properties["Prop3"] = // DateTime?
eventData.Properties["Prop4"] = // Custom?

What data types am I actually allowed to pass into the Properties dictionary? Obviously this data needs to be serialized somehow, but the documentation does not mention anything about this.

Upvotes: 1

Views: 2032

Answers (2)

Arthur Erlendsson
Arthur Erlendsson

Reputation: 176

Following up on Chris's answer - AMQP headers are serialized according to the AMQP type spec - http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-types-v1.0-os.html.

For any Java developers viewing this answer - you can use the Apache qpid library to perform encode/decode operations. The decoder class can be found here - https://qpid.apache.org/releases/qpid-proton-j-0.33.1/api/index.html.

EH service is encoding-agnostic, so a consumer reading via Kafka protocol will need to manually decode AMQP headers. Native EH clients will decode the headers for you. See - https://github.com/Azure/azure-event-hubs-for-kafka/issues/56.

Upvotes: 1

Chris Gillum
Chris Gillum

Reputation: 15042

I found more information by trying something that didn't work, and then finding the source code by looking at the exception call-stack.

In my case, the supported property types seems to be derived from what the AMQP protocol supports (which is documented here). There is actually a big switch statement in the internal AmqpMessageConverter code which basically gives me my answer:

AmqpMessageConverter.TryGetAmqpObjectFromNetObject

To summarize here:

  • All .NET primitive types (int, string, double, etc.)
  • Guid
  • DateTime and DateTimeOffset
  • Stream
  • Uri
  • TimeSpan
  • byte[]
  • IList
  • IDictionary

My own custom type (a POCO - "Plain old CLR object") did was NOT accepted, and resulted in the following exception:

System.Runtime.Serialization.SerializationException: Serialization operation failed due to unsupported type EventHubsTesting.Program+Poco.
   at Microsoft.Azure.EventHubs.Amqp.AmqpMessageConverter.TryGetAmqpObjectFromNetObject(Object netObject, MappingType mappingType, Object& amqpObject)
   at Microsoft.Azure.EventHubs.Amqp.AmqpMessageConverter.UpdateAmqpMessageHeadersAndProperties(AmqpMessage message, String publisher, EventData eventData, Boolean copyUserProperties)
   at Microsoft.Azure.EventHubs.Amqp.AmqpMessageConverter.EventDataToAmqpMessage(EventData eventData)
   at Microsoft.Azure.EventHubs.Amqp.AmqpMessageConverter.EventDatasToAmqpMessage(IEnumerable`1 eventDatas, String partitionKey)
   at Microsoft.Azure.EventHubs.Amqp.AmqpEventDataSender.OnSendAsync(IEnumerable`1 eventDatas, String partitionKey)
   at Microsoft.Azure.EventHubs.EventDataSender.SendAsync(IEnumerable`1 eventDatas, String partitionKey)
   at Microsoft.Azure.EventHubs.EventHubClient.SendAsync(IEnumerable`1 eventDatas, String partitionKey)
   at EventHubsTesting.Program.Sender(CancellationTokenSource shutdownSource) in C:\EventHubsTesting\Program.cs:line 99

I assume that if I want to use custom types, then I'd need to serialize them myself before assigning them as application properties.

Upvotes: 0

Related Questions