CODEWITHSUNDEEP
azure-eventhub
Motoko
Motoko

Reputation: 1132

How to know if IEventProcessor is able to handle the event volume in Azure EventHub

I have multiple instances of IEventProcessor running but how do I tell whether to scale up or scale down to handle the event volume in EventHub? Which API should I call?

Upvotes: 0

Views: 75

Answers (1)

Jesse Squire
Jesse Squire

Reputation: 7745

One of the common techniques that is used is "tracking the event backlog", where you'll periodically read the properties of an Event Hub partition and compare the delta between the sequence number of the event that you're currently processing with that of the event last published to the partition. While not a bulletproof and perfect approach, this provides a reasonable rough approach to understand if you're seeing events published at a greater rate than their being read when you track this across a few cycles.

While this is available with the EventProcessorClient in the newer Azure.Messaging.EventHubs.Processor library, unfortunately, it doesn't appear as if the legacy Microsoft.Azure.EventHubs processor exposes it.

If you're controlling your scaling through an orchestrator or other means not directly within your processor, then you may want to consider using the REST APIs. The Get Partition Metadata endpoint returns information about the size of the partition, the incoming/outgoing rates, and also the beginning/ending sequence numbers for comparison.

Upvotes: 1

Related Questions