CODEWITHSUNDEEP
google-bigquerymonitoringmonitorobservability
Dinesh
Dinesh

Reputation: 327

I am looking to create a custom metric based on BigQuery SQL

The requirement is to monitor the table customer which has a column ingest_timestamp. I want to send an alert to the user if no data has arrived within last 24 hours. chatGPT has provided me this solution

You can set up an alert in Cloud Monitoring to monitor the ingest_timestamp column in the customer table and trigger an alert if no data has been processed for the previous day.

Here are the steps to set up the alert in Cloud Monitoring:

Create a metric that captures the maximum value of the ingest_timestamp column in the customer table. You can use the following query to create the metric:


SELECT MAX(ingest_timestamp) FROM `my_project.my_dataset.customer`

Set up an alert policy in Cloud Monitoring that uses the metric created in step 1. You can use the following configuration to set up the alert policy:

Condition: Single time series
Aggregator: Max
Time range: Last 24 hours
Threshold: No data received
Notification: Choose the notification channel to receive the alert
Save the alert policy.

However I do not seems to find any create metric button on Metric Explorer screen. What am I missing here or chatGPT is just making things up ?

Upvotes: -1

Views: 1139

Answers (1)

MrRobot
MrRobot

Reputation: 501

As far as i know, you can't do this on GCP out of the box as GPT stated.

What you can do is create a custom metric (again as GPT stated), populate it with data, then place an alerting policy on top of it, and that should achieve your goal. It's a very similar option to what GPT proposed.

To create the custom metric, you'll have to create a custom metric descriptor, as documented here. Only then you can populate it with data. Make sure the data points are relevant to the alerting capabilities, so for example instead of the latest/max timestamp value as per the SQL query above, maybe fill it with the number of hours of delta between the function runtime and the max timestamp, this will make it easier to set up alerts later.

You can push the datapoints to the metric (as shown in the docs above) via a scheduled CloudRun Function for example, or even better, via a continuous query that pushes info to Pub/Sub, which triggers the function, that way you're alerted as soon as the data is pushed to your table.

Hope it helps, and one more important point, think of billing, these services come at a cost.

Upvotes: -1

Related Questions