CODEWITHSUNDEEP
google-bigquery
Bryon
Bryon

Reputation: 23

How to receive notification of BigQuery Job errors

When A BigQuery Job has an error, I can view the error in the Job history and StackDriver Logs. I would like to be notified (by email) when any job fails.

In user preferences, I have enabled "StackDriver Error Reporting" for new errors. I have also looked at the StackDriver BigQuery Resource to see if I could create an Alerting Policy, but I don't understand if any of the conditions apply to BQ job errors.

Is it possible and if so how can I receive notification of errors in BigQuery Jobs?

Upvotes: 2

Views: 3448

Answers (1)

totooooo
totooooo

Reputation: 1415

If you want an email for every BQ job error in your project, you could define a log-based custom metric in Stackdriver. You would define it, thanks to filters on the content of the log, so that it counts the number of logs indicating an error on a BQ job. Then in Stackdriver Monitoring, you could define an alerting policy on this custom metric.

If you want to display the errors your jobs had in Error Reporting, you should add code to your Cloud Function so it reports the errors to Stackdriver when they happen, using the Error Reporting API. See this link for more details on how. This means your Cloud Function would analyze the answer from the BQ API, and if it sees the job had an error, it reports it to Stackdriver.

Stackdriver Error Reporting apparently supports email notification, but only for the first occurence of an error. (And this feature is in beta it seems)

So if you want to receive emails for each failed BQ job triggered by the Cloud Function, I believe you should implement the logic of sending the email inside the Cloud Function.

Upvotes: 3

Related Questions