CODEWITHSUNDEEP
apache-pulsar
dashenswen
dashenswen

Reputation: 600

does pulsar support multiple bookkeeper replicas in different clusters

I have a use case where requires data backup across multiple data centers and needs strong consistency. The ideal view is each segment is replicated to three clusters located at three different data centers. pulsar supports using multiple clusters as a large bookie pool but I didn't find how to configure the replicas in different clusters. Anyone has similar use case before? i think it should be not hard to do considering pulsar separate broker and storage + replicas in different clusters

Upvotes: 0

Views: 170

Answers (1)

Christophe Bornet
Christophe Bornet

Reputation: 1127

It's possible to enable a region aware placement policy of bookies (parameter bookkeeperClientRegionawarePolicyEnabled). You'll also need to configure the bookie region with the admin command set-bookie-rack This is not much documented in Pulsar/BookKeeper docs. See this blog post for more details : https://techblog.cdiscount.com/ensure-cross-datacenter-guaranteed-message-delivery-and-resilience-with-apache-pulsar/

Beware that due to the cross-region latency between the brokers and the bookies, the throughput will drop but that can't really be helped if you need strong consistency even in the case of a region failure.

Upvotes: 1

Related Questions