CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquerycronterraform
Pierre-Alexandre
Pierre-Alexandre

Reputation: 755

Cron job using App Engine Cron Service: every day at a given time

I am trying to deploy a Scheduling Query (BigQuery) using Terraform. I would like to have my Scheduling Query to run every day at 3 am. Scheduling Queries is using the App Engine Cron Service.

The example provided by Terraform has a schedule job included in the example. This works for me.

 schedule               = "first sunday of quarter 00:00"

However, I am trying to run my Scheduling query every day at 3:00 am. What I tried is: "every 24h at 03:00" and then "every 24h from 03:00" but none of those worked

Based on this documentation, Data transfer schedule...Examples of valid format: 1st,3rd monday of month 15:30, every wed,fri of jan,jun 13:15, and first sunday of quarter 00:00. See more explanation about the format here: https://cloud.google.com/appengine/docs/flexible/python/scheduling-jobs-with-cron-yaml#the_schedule_format NOTE: the granularity should be at least 8 hours, or less frequent.

my code: main.tf

resource "google_bigquery_data_transfer_config" "query_config" {

  for_each               = local.jobs

  display_name           = each.value.display_name
  data_source_id         = each.value.data_source_id
  schedule               = "every 24h at 03:00"
  destination_dataset_id = each.value.destination_dataset_id
  params = {
    destination_table_name_template = each.value.destination_table_name_template
    write_disposition               = each.value.write_disposition
    query                           = file("scheduling_queries/sql_repo/${each.value.display_name}.sql")
  }
}

Upvotes: 1

Views: 942

Answers (1)

Pierre-Alexandre
Pierre-Alexandre

Reputation: 755

Got it after a couple of failures: every day 03:15

Upvotes: 2

Related Questions