CODEWITHSUNDEEP
azureazure-eventhubazure-monitoringazure-diagnostics
almisson
almisson

Reputation: 165

What is the partition key used by Azure Diagnostic Settings when sending data to Event Hub?

When configuring a diagnostic settings for a resource in Azure, and configuring that diagnostic setting to an event hub, what is the partition key used when messages are published to the event hub? Or is no partition key used (resulting in round-robin distribution across partitions)? I've searched through Microsoft documentation and haven't been able to find an answer to this question.

The specific use case we have is an application insights resource which has a diagnostic setting configured to send data to an event hub. We then have an Azure stream analytics job consuming from this event hub. We're trying to understand how data will be partitioned across that event hub so we can properly configure the partition key for the stream analytics job input.

Upvotes: 0

Views: 308

Answers (2)

vijaya
vijaya

Reputation: 1731

I do agree with @Jdresc you should do this from Stream Analytics side.

Refer this SO link to get more information about partition key.

  • When configuring a diagnostic setting for a resource in Azure, and configuring that diagnostic setting to an event hub, the partition key used when messages are published to the event hub is derived from the diagnostic setting's resource ID.

  • Refer this Github document. According this document,

The Event Data class has a Partition Key property that enables the sender to specify a value that is hashed to produce a partition assignment. Using a partition key ensures that all the events with the same key are sent to the same partition in the Event Hub. Common partition keys include user session IDs and unique sender IDs.

  • If you do not provide a value for Partition Key, sent events are distributed to partitions using a round-robin model. enter image description here
  • Thanks to @Jdresc for leading into right direction. As he said do this from stream analytics side as No partition key will be set by diagnostic settings for the events. Therefore, you should configure the partition key for the Stream Analytics job.

Upvotes: 0

Jdresc
Jdresc

Reputation: 688

Diagnostic Settings (Azure Monitor) will not set any Partition key for the events, hence the events will be load balanced in a round robin fashion across all the Event Hub partitions. You should do the same from the Stream Analytics side, and pull from all partitions.

Upvotes: 1

Related Questions