CODEWITHSUNDEEP
google-cloud-platform
mobileideafactory
mobileideafactory

Reputation: 1670

Propagating Error messages in Google Cloud Platform (GCP)

I am building a near real time service. The input is a cloud storage bucket and blob path to a photo image. This horizontally-scalable service is made up of multiple components including ML models running on k8s and Google Cloud Functions, each of which has a chance of failing for a variety of reasons. The ML models are independent and run in parallel. Each component is triggered by a PubSub push message topic unique to the component. Running the entire flow for one photo may take 15 seconds.

I want to return a meaning error message back to the service requester telling which component failed if there is a failure. Essentially, I want to report which image failed and where it failed.

What is the recommended practice for returning an error back to the requester?

Upvotes: 0

Views: 34

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75715

There is no built in service for this. But, because you already use PubSub for asynchronous call, I propose to use it also to push back the error.

You can do this in 2 flavors

First, create a PubSub topic for the errors, let's say 'error_topic'

1. Without message customization

  • In the PubSub message, the requester put which it is in the attribute (let's say 'requester' attribute name)
  • In the consumer service, if an error occurs, return an error code (500 for example) for push subscription or a NACK in pull subscription.
  • Configure the PubSub subscription to manage retry and dead letter topic (the dead letter topic is 'error_topic')
  • Then, create one subscription per requester on the 'error_topic' (use the filter capability for this) and consume the message in the requester services

2. With message customization

  • In the PubSub message, the requester put which it is in the attribute (let's say 'requester' attribute name)
  • The consumer service that raises the error create a new message with custom information and copies the 'requester' attribute value and then puts it in attribute of the message in the 'error_topic' (let's say 'original_requester' attribute name).
  • Then, create one subscription per requester on the 'error_topic' (use the filter capability for this) and consume the message in the requester services

Upvotes: 2

Related Questions