CODEWITHSUNDEEP
google-bigqueryalertmonitoring
G Play
G Play

Reputation: 21

How to have "Alert" with BigQuery

I am facing a problem with the BigQuery project.

So, I want to have alerting in BigQuery based on the data and the threshold that I want to set. Since the data is constantly adding up every 15 minutes, I want BigQuery to first analyze the dataset automatically with the saved queries, and to notify me if there is something against the threshold.

I have tried the monitoring feature but the metrics are like added-rows, bytes etc. and they are not relevant because my main concern is the datum itself.

For example, with the "ID" and "result" column, I set the threshold on the column of "result" to a datum "5". If the data gets more than "5", I want to be notified that which "ID" went up.

Upvotes: 2

Views: 7393

Answers (2)

Michael Entin
Michael Entin

Reputation: 7744

What I did once was

  • setup a scheduled BigQuery query,
  • use ERROR function to run if the condition is not satisfied, and
  • when scheduling the query, check the box to notify if the query fails.

That is very simple but worked great for my simple case.

Upvotes: 10

Vibhor Gupta
Vibhor Gupta

Reputation: 699

You can try writing a simple python script as given below to notify as per your requirements.

from dateutil.parser import parse
import datetime
from google.cloud import bigquery


stream_query = """SELECT * FROM `PROJ.test123.TABLE4` WHERE 1=1"""

stream_client = bigquery.Client()

stream_Q = stream_client.query(stream_query)
stream_data_df = stream_Q.to_dataframe()
.......
.....
... As per your requirement it can send notification to MS teams channel, email, Pub/sub etc.

Upvotes: 0

Related Questions