CODEWITHSUNDEEP
gitgoogle-bigquery
Daryl Wenman-Bateson
Daryl Wenman-Bateson

Reputation: 3939

Source control BigQuery Dataset with GitHub

Is there a graceful way to source control a datasets' table / view definitions between BigQuery and GitHub (maybe something similar to a Visual Studios -> Database Project) to ensure consistency and traceability across multiple developers?

I can see how we can manually craft view scripts and add them to a repo, however I'd ideally like to avoid the overhead of manually maintaining these files with the inevitable disconnect from BigQuery.

Does BigQuery natively provide any features to support;

  1. generating script from an existing BigQuery environment to extract tables / view definitions
  2. storing these scripts in a database project that could be kept in sync with BigQuery
  3. automate the release of dataset / table / view definition script to create a new environment
  4. comparison of a BigQuerys dataset definition to the source controlled (GitHub) model to identify metadata discrepancies

Upvotes: 0

Views: 709

Answers (1)

Pentium10
Pentium10

Reputation: 207893

BigQuery since it's a managed service it provides endpoints where it allows you the owner of the project to build your own Ops.

It's not a server, it's a warehouse service, and it doesn't provide natively all those automations.

You should constantly keep an active repo with all the information backed up, so in an eventually disconnect or needed rebuild you should have all the automation to create your datasat and replay any events. This is crucial for any managed service.

Upvotes: 1

Related Questions