CODEWITHSUNDEEP
c#azureasynchronousazure-eventhub
Veronica_Zotali
Veronica_Zotali

Reputation: 252

Azure EventHub - How to remove the event?

Like allways, I would appreciate your help, as I am currently stuck!!!

We have a new project and we will be using the Azure EventHub. I have created demo app, where we can add events to the Event Hub and also where we can consume them using IEventProcessor(Receiver project). The questions is that every time, I execute the receiver project, I see the same events. Shouldn't we expect that those events will be deleted-removed after we consume them?

Example in the Receiver project:

foreach (EventData eventData in messages)
{
        string data = Encoding.UTF8.GetString(eventData.GetBytes());

        Console.WriteLine(string.Format("Message received.  Partition: '{0}', Data: '{1}'",
                context.Lease.PartitionId, data));
}

Is there a way to delete/remove the event after the Console.WriteLine or will the message be retained for a day? With the Queues , you can signal the completion , but with the EventHub is don't see any command, I can use to delete/remove it.

Any reply would be greatly appreciated. We have been instructed to use EventHub but a-b reasons, its not a matter of choice.

Upvotes: 1

Views: 6959

Answers (5)

Javier Solis
Javier Solis

Reputation: 131

The best option to consume an event hub is the EventProcessorHost framework. It gives you the possibility to checkpoint any message you had already read. To do that it stores (in a blob storage) the index of the last checkpointed message in order to resume the processing in case of a shutdown.

https://blogs.msdn.microsoft.com/servicebus/2015/01/16/event-processor-host-best-practices-part-1/

Probably you will also need a storage emulator for development purposes, but if you have an Azure account you could use a remote blob storage.

https://learn.microsoft.com/en-us/azure/storage/common/storage-use-emulator

Upvotes: -1

unconnected
unconnected

Reputation: 1011

In addition to all other answers, there's one more confusing point: EventHub may not delete messages older than retention period up to 30 days. It depends on load put over the hub.

E.g. retention period is 1 day, but if there're few messages they can be kept for a longer period.

Luckily they are not billed.

Upvotes: 0

JamesNW
JamesNW

Reputation: 125

The events in Azure Event Hub will be removed after they are buffered for the defined retention period. You can set the retention days from 1-7 days. There is no need for removing them manually.

Upvotes: 0

djn
djn

Reputation: 113

you should use the checkpoint and save the offset for the partition. AFAIK there is no way remove the events from the eventhub. It will be automatically erased from the eventhub after the events retention days. But I have also seen that we will get messages that have completed their retention days.

https://social.msdn.microsoft.com/Forums/windows/en-US/93b1bf18-2229-4da8-994f-ddc7c675f62f/message-retention-day-not-working?forum=servbus

So I believe they will get removed automatically when you hit the space quota, or it will be like a scheduled task, not sure though.

Upvotes: 1

fsimonazzi
fsimonazzi

Reputation: 3013

Make sure you call context.CheckpointAsync before exiting ProcessEventsAsync. That will store the client offset for the partition, and the next processor instance which gets assigned that partition will resume from the last stored offset.

See http://msdn.microsoft.com/en-us/library/dn751578.aspx for documentation (not a lot of information, though).

Upvotes: 5

Related Questions