CODEWITHSUNDEEP
google-cloud-platformterraformgoogle-cloud-storagegoogle-cloud-buildterraform-provider-gcp
Arindam
Arindam

Reputation: 342

Triggering a google_cloudbuild_trigger from terraform to create a google_storage_bucket_object

I have the following setup:

  1. A google_cloudbuild_trigger that runs on the latest github code and builds and uploads the build to a dataflow flex artifact location (on google storage)
  2. A dataflex template job that depends on the artifact being present.

I want to configure terraform so that if the artifact is not present, then automatically trigger the google_cloudbuild_trigger and wait for it to complete. If the artifact is present, then just continue using it.

Is this even possible in terraform ?

Snippets of my terraform script:

The following is the cloudbuild trigger:

resource "google_cloudbuild_trigger" "build_pipeline" {
  name = "build_pipeline"
  github {
    owner = "my-org"
    name  = "my-project"
    push {
      branch = "^my-branch$"
    }
  }

  filename = "path/cloudbuild.yaml"
  substitutions = {
    _PROJECT_ID = var.google_project_id
  }
}

The following is the dataflow flex template job:

resource "google_dataflow_flex_template_job" "dataflow_job" {
  provider                = google-beta
  name                    = "dataflow_job"
  container_spec_gcs_path = "${google_storage_bucket.project_store.url}/path/to/flex/template.json"
  project                 = var.google_project_id
  depends_on              = [google_bigquery_table.tables]
  parameters = { ...  }
}

I have tried creating a simple "data" resource like:

data "google_storage_bucket_object" "picture" {
  name   = "path/to/flex/template.json"
  bucket = google_storage_bucket.project_store.name
}

But I cannot figure out how to change this into something that triggers the google_cloudbuild_trigger.build_pipeline if the data resource doesn't exist.

Something like:

data "google_storage_bucket_object" "picture" {
  name   = "path/to/flex/template.json"
  bucket = google_storage_bucket.project_store.name
  if_does_not_exist_trigger = google_cloudbuild_trigger.build_pipeline
}

Upvotes: 1

Views: 1116

Answers (1)

Milind Joshi
Milind Joshi

Reputation: 1

There is no way to run gcp trigger via terraform. It gets invoked on certain event(code push).

Instead of creating google_dataflow_flex_template_job using terraform, use Dockerfile provided by gcp team. Build the image and push it using cloudbuild.yaml. In the same yaml, try to create dataflow jobs using gcloud dataflow flex-template build/run command.

The entire flow is, create the google_cloudbuild_trigger using terraform and providing the filename as cloudbuild.yaml along with necessary substitutions parameters. Second step is create and push docker image in cloudbuild.yaml. In the same cloudbuild.yaml, create dataflow jobs using gcloud commands.

Upvotes: 0

Related Questions