CODEWITHSUNDEEP
apache-zookeeperdistributedapache-curator
Muhammad Gelbana
Muhammad Gelbana

Reputation: 4000

How to handle out of order Zookeeper notifications?

I have multiple processes operating on an in-memory queue. That queue is a manifestation of sequential znodes created/deleted at Zookeeper.

When a znode is added, an equivalent item is added to the queue at all the involved processes. And also when a znode is removed, the equivalent item is removed from the queue at every involved process.

The addition and removal signals are expected to be balanced because every added item should eventually be removed.

I faced a situation when a znode was added and removed very quickly and the removal notification was received at one of the processes before the addition notificaiton. So an attempt to remove that item occurred but failed because it wasn't actually there, and then the addition signal was received which added the item but then it was never removed.

A simple solution would be to assert the existence of the equivalent znode after adding the item to the queue and that's good enough for me now but it doesn't seem as efficient as it can get.

My question is if there is a way to handle this scenario in a more efficient or "zookeeper way"?

Upvotes: 0

Views: 205

Answers (1)

Andor Molnar
Andor Molnar

Reputation: 66

You're trying to use ZooKeeper as a message queue which is not designed for. There's no ordering neither delivery guarantee in ZooKeeper for watcher notifications. Instead you should use some messaging system like Kafka or RabbitMQ for this use case.

Upvotes: 1

Related Questions