CODEWITHSUNDEEP
load-balancinggoogle-cloud-pubsub
Danielle Laforte
Danielle Laforte

Reputation: 173

Google Pub/Sub - Multiple subscriptions to same topic to reduce load via REST API?

There will be one publisher to one topic. Let's say 1000s of messages/sec.

Can I load balance these by having multiple endpoints to receive the messages?

This is what I want to do: (Does the REST API for Google Pub/Sub allow this?)

JMS 2.0 provides a solution. You can create a "shared" nondurable subscription using a new method: createSharedConsumer. This method is available both on Session (for applications using the classic API) and on JMSContext (for applications using the simplified API). Since the two JVMs need to be able to identify the subscription that they need to share, they need to supply a name to identify the shared subscription, as shown in Listing 2.

private void createSharedConsumer(ConnectionFactory connectionFactory, Topic topic) throws JMSException {
   Connection connection = connectionFactory.createConnection();
   Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
   MessageConsumer messageConsumer = session.createSharedConsumer(topic,"mySubscription");
   connection.start();
   Message message = messageConsumer.receive(10000);
   while (message != null) {
      System.out.println("Message received: " + ((TextMessage) message).getText());
      message = messageConsumer.receive(10000);
   }
   connection.close();
}

Listing 2

If you run the code in Listing 2 in two separate JVMs, each message sent to the topic will be delivered to one or the other of the two consumers. This allows them to share the work of processing messages from the subscription.

http://www.oracle.com/technetwork/articles/java/jms2messaging-1954190.html

JMS 2.0 specification describes the concept of Shared Subscription where more than one subscriber/consumer share (a.k.a load balance) messages published on a topic. All consumers use same subscription id.

Upvotes: 5

Views: 13262

Answers (1)

Danielle Laforte
Danielle Laforte

Reputation: 173

I just found this:

Load balancing Multiple subscribers can make pull calls to the same "shared" subscription. Each subscriber will receive a subset of the messages. The push endpoint can be a load balancer.

from https://cloud.google.com/pubsub/docs/subscriber

Upvotes: 7

Related Questions