How to apply Path Patterns in GCP Eventarc for BigQuery service's jobCompleted method?

Question

I am developing a solution where a cloud function calls BigQuery procedure and upon successful completion of this stored procedure trigger another cloud function. For this I am using Audit Logs "jobservice.jobcompleted" method. Problem with this approach is it will trigger cloud function on every job that are completed in BigQuery irrespective of dataset and procedure.

Is there any way to add path pattern to the filter so that it triggers only for specific query completion and not for all?

My query starts something like:

CALL storedProcedure() ...

Also, as I tried to create a 2nd Gen function from console, I tried Eventarc trigger. But to my surprise BigQuery Event provider doesn't have Event for "jobCompleted":

Now I'm wondering if it's possible to trigger based on job complete event.

Update: I changed my logic now to use google.cloud.bigquery.v2.TableService.InsertTable method to make sure after inserting a record to a table it will add AuditLog message so that I can trigger the next service. This insert statement is present as the last statement in BigQuery procedure.

After running the procedure, the insert statement is inserting the data but resource name is coming as projects/<project_name>/jobs.

I was expecting something like projects/<project_name>/tables/<table_name> so that I can apply path pattern on resource name.

Do I need to use different protoPayload.method?

Answer 1

Try to create a Log Sink for job completed with unique principal-email sv account and use pubsub with the sink. Get pubsub published event to run destination service.