CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-pubsub
Christophe Bouhier
Christophe Bouhier

Reputation: 224

Loadbalancing subscribers - How is it done?

GCP pubsub docs mention load balancing for pull mode, it's not clear how to use it.

The Subsciption nor The Subscriber builder api, doesn't seem to have a method to turn this on.

Question: How to configure load balancing accross multiple pubsub subscribers?

Background:

We use multiple subscribers for the same topic, to achieve resilience. (Multiple endpoints can be queried for data from the same data store).

The subscriptions persist the messages, but with out distribution, all subscriptions get all messages, leading to data duplication in our data store. Perhaps this background, will give ideas for another way to achieve resilience.

Things we have thought of ourselves:

Technologies:

Upvotes: 2

Views: 3938

Answers (1)

Kamal Aboul-Hosn
Kamal Aboul-Hosn

Reputation: 17161

If all subscribers are receiving all messages, then it is likely that you are using different subscriptions for each subscriber. Load balancing happens when you have different subscribers all pulling from the same subscription. From the subscriber guide description of load balancing: "Multiple subscribers can make pull calls to the same "shared" subscription. Each subscriber will receive a subset of the messages" (Emphasis mine). When you use different subscriptions, you get fanout, where all subscribers receive all messages.

Upvotes: 3

Related Questions