CODEWITHSUNDEEP
azureservicebusazure-eventhub
Val
Val

Reputation: 61

How to get events count from Microsoft Azure EventHub?

I want to get events count from Microsoft Azure EventHub. I can use EventHubReceiver.Receive(maxcount) but it is slow on big number of big events.

There is NamespaceManager.GetEventHubPartition(..).EndSequenceNumber property that seems to be doing the trick but I am not sure if it is correct approach.

Upvotes: 6

Views: 9336

Answers (2)

Sreeram Garlapati
Sreeram Garlapati

Reputation: 4993

EventHub doesn't have a notion of Message count, as EventHub is a high-Throughput, low-latency durable stream of events on cloud - getting the CORRECT current count at a given point of time, could be wrong the very next milli-second!! and hence, it wasn't provided :)

Hmm, we should have named EventHubs something like a StreamHub - which would make this obvious!!

If what you are looking for is - how much is the Receiver lagging behind - then EventHubClient.GetPartitionRuntimeInformation().LastEnqueuedSequenceNumber is your Best bet.

As long as no messages are sent to the partition this value remains constant :)

On the Receiver side - when a message is received - receivedEventData.SequenceNumber will indicate the Current sequence number you are processing and the diff. between EventHubClient.GetPartitionRuntimeInformation().LastEnqueuedSequenceNumber and EventData.SequenceNumber can indicate how much the Receiver of a Partition is lagging behind - based on which, the receiver process can Scale up or down the no. of Workers (work distribution logic).

more on Event Hubs...

Upvotes: 14

Stefano d'Antonio
Stefano d'Antonio

Reputation: 6142

You can use Stream Analytics, with a simple query:

SELECT
    COUNT(*)
FROM
    YourEventHub
GROUP BY
    TUMBLINGWINDOW(DURATION(hh, <Number of hours in which the events happened>))

Of course you will need to specify a time window, but you can potentially run it from when you started collecting data to now.

You will be able to output to SQL/Blob/Service Bus et cetera.

Then you can get the message out of the output from code and process it. It is quite complicated for a one off count, but if you need it frequently and you have to write some code around it, it could be the solution for you.

Upvotes: 0

Related Questions