CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-functionsgoogle-cloud-storagegoogle-cloud-pubsub
OneWorld
OneWorld

Reputation: 992

GCP: Splitting traffic in ratio across regions (Active-Active)

I am designing an application that runs on multiple regions. say R1, R2.

Files are submitted to a multi-region cloud storage bucket. PUT event in the bucket will publish a notification to either directly trigger the cloud function or to an pub/sub topic.

I want 80% of processing to be done by R1, and 20% by R2.

Approach1: Have 2 Cloud functions: CF-R1, CF-R2. How do I ensure that 80% of storage bucket notifications trigger CF-R1 & 20% trigger CF-R2?

Approach 2: Have pub/sub topic which captures notification from the storage bucket. Is it possible to configure CF-R1 & CF-R2 on the topic so that I can split traffic?

Or any other approach to handle this scenario.

Upvotes: 0

Views: 241

Answers (1)

Iñigo González
Iñigo González

Reputation: 3955

Approach 1: Use a Load balancer with URL maps

You coudl use a Cloud function or Cloud Run and use a load balancer with a URL map (announced in June in this blog post - see documentation).

If you use the load balancer you can trigger the notification to the balancer directly or via pubsub with a PUSH subscription.

Note that the load balancer is a separate product and you must take a close look at usage and price.

Approach 2: Several pubsub subscriptions with a filter

I think the second option could be viable. Crazy to do for your case, but it will work.

Google has now in beta the option to apply a filter to a pubsub topic when you create a subscription.

Then, you can have a cloud function (or a cloud run) reacting to the pubsub notifications they recieve on their own subscription.

With this beta feature, you can filter by message values (equals ==, not equals !=, and hasPrefix).

The trick here is to have enough information to distribute the messages between the functions evenly because you cannot change the filter after you create the subscription.

If you can pass that information in your app, or as part of the filename, you can do it this way in an easy way.

If not, I guess the crc32 might have enough information for the filter you need. But this filter has a 128 character limit that you hit with this:

hasPrefix(attributes.crc32,"A") OR hasPrefix(attributes.crc32,"B") OR hasPrefix(attributes.crc32,"C") OR hasPrefix(attributes.crc32,"D") OR hasPrefix(attributes.crc32,"E")

With the filter above you have almost 10% of the CRC32 possible cases. Not bad for some simple cases, but not good for you since you would have to configure a lot of subscriptions.

Upvotes: 2

Related Questions