Reputation: 7004
I would like to trigger a Cloud Function when new data has been imported into a BigQuery table. Ideally, I would like to extract all the rows (one column is ISIN) that have been inserted.
Would this be possible and how?
Upvotes: 16
Views: 16928
Reputation: 594
Update June 2022: Cloud Functions 2nd Generation supports Eventarc
Using Eventarc it's possible to listen to a long list of cloud events and then trigger a set of consumers including Cloud Functions.
The combination of serviceName = bigquery.googleapis.com
and methodName = jobservice.jobcompleted
with additional custom filters can be used to create the Eventrac sink, from which Cloud Functions can be triggered. You can read more here.
This is still not a native BigQuery <> Cloud Function
integration, because it uses a 3rd cloud service. Therefore it's not much different than the solution suggested before June 2022 of creating a sink with Cloud Logging. Yet, this is the better practice these days of implementing cross-service triggers on GCP.
Old answer: Unfortunately BigQuery has no triggers as a native feature as of May 2022 :/ However there is a nice workaround you can implement! :)
As you probably know, almost every operation in GCP is logged to Cloud Logging (previously StackDriver Logging). In this article they demonstrate how to trigger a Cloud Function whenever a BigQuery load job finishes. The idea is simple:
Google even published this concept as an article.
Upvotes: 17